需要Elasticsearch Logstash过滤器的帮助

ercv8c1e  于 2023-04-27  发布在  Logstash
关注(0)|答案(1)|浏览(301)

我想从另一个索引中丰富主机名,因为在某些情况下,主索引中缺少该值。因此,我使用Elasticsearch Logstash filter来查询主机名{如附件所示}。
然而,当我使用命令{manually}测试管道时,我得到了如下所述的错误-

[ERROR] 2023-04-24 10:02:58.784 [[main]-pipeline-manager] javapipeline - Pipeline error {:pipeline_id=>"main", :exception=>#<Elasticsearch::Transport::Transport::Errors::Unauthorized: [401] >, :backtrace=>["/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/elasticsearch-transport-7.17.1/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/elasticsearch-transport-7.17.1/lib/elasticsearch/transport/transport/base.rb:341:in `perform_request'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/elasticsearch-transport-7.17.1/lib/elasticsearch/transport/transport/http/manticore.rb:91:in `perform_request'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/elasticsearch-transport-7.17.1/lib/elasticsearch/transport/client.rb:197:in `perform_request'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/elasticsearch-7.17.1/lib/elasticsearch.rb:41:in `method_missing'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/elasticsearch-api-7.17.1/lib/elasticsearch/api/actions/ping.rb:38:in `ping'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/logstash-filter-elasticsearch-3.12.0/lib/logstash/filters/elasticsearch.rb:330:in `test_connection!'", "/appserver/logstash-8.4.3/vendor/bundle/jruby/2.6.0/gems/logstash-filter-elasticsearch-3.12.0/lib/logstash/filters/elasticsearch.rb:118:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:233:in `block in register_plugins'", "org/jruby/RubyArray.java:1865:in `each'", "/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:232:in `register_plugins'", "/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:599:in `maybe_setup_out_plugins'", "/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:245:in `start_workers'", "/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:190:in `run'", "/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:142:in `block in start'"], "pipeline.sources"=>["/data/logstash/pipelines/sendEmailAlerts_updated.conf"], :thread=>"#<Thread:0x1d8d4a8@/appserver/logstash-8.4.3/logstash-core/lib/logstash/java_pipeline.rb:130 run>"}
[INFO ] 2023-04-24 10:02:58.785 [[main]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"main"}
[ERROR] 2023-04-24 10:02:58.793 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
  • 注:其他管路工作正常,无任何误差;因此,Logstash配置文件看起来没问题。*

logstash conf文件

input {
        elasticsearch
        {
        hosts => "localhost:9200"
        user => "reader"
        password => "*******************"
        index => "*-testalert"
        query => '{ "query": {
                             "bool": {
                                      "must": [{"terms": { "kibana.alert.severity": [ "high", "critical"] }} ],
                                      "filter": [ {"range": {"@timestamp": { "gte": "now-2d"}}}]
                                     }
                             }
                  }'
        schedule => "/5 * * * *"
        size => 500
        scroll => "5m"
        docinfo => true
        docinfo_target => "[@metadata][doc]"
        codec => "json"
        }
    }


filter {
if [host][hostname] != ""
{
mutate {
        add_field => {
            "alertHostName" => "%{[host][hostname]}"
            "alertReason" => "%{kibana.alert.reason}"
            "alertSeverity" => "%{kibana.alert.severity}"
            "alertTime" => "%{kibana.alert.original_time}"

        }
    }
}
if [host][hostname] == ""
{
elasticsearch {
              hosts => "localhost:9200"
              index => ".fleet-agents"
              query => "{[local_metadata][host][id]}:%{[host][id]}"
              fields => {
                       "[local_metadata][host][id]" => "host_name"
                      }
              }

mutate {
        add_field => {
            "alertHostName" => "%{[host_name]}"
            "alertReason" => "%{kibana.alert.reason}"
            "alertSeverity" => "%{kibana.alert.severity}"
            "alertTime" => "%{kibana.alert.original_time}"

        }
    }
}
}

output {
stdout {
 codec => "json"
}
}

fjaof16o

fjaof16o1#

在第一个日志行中,您可以看到以下错误:
未经授权:[第401页]
所以看起来你只是在查询.fleet-agents索引的elasticsearch过滤器中缺少了一些身份验证。
您可能需要添加user => "reader"和相应的password,与输入中的相同。

相关问题