elasticsearch 用于单个文件节拍输入的两条管道|ELK版本(6.5.4)

w8ntj3qf  于 2022-12-11  发布在  ElasticSearch
关注(0)|答案(1)|浏览(174)

我尝试在logstash中创建多个管道第一管道和第二管道,侦听来自同一端口的beats事件,但收到一个错误,指出第二管道的地址正在使用中,我使用两个管道的原因是我需要 pipeline. worker:1 仅适用于日志顺序重要的选定索引。错误:

Pipeline_id:second-pipeline

Plugin: <LogStash::Inputs::Beats host=>"127.0.0.1", port=>5044, 
id=>"7c07a66c7959c1734f6aead8ca456bc7c3b086aafb7b5bd4882ee45e0f3c9fc5", 
enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_4d22b75f-e478-4fbc- 
b5fe-27ae02ac486b", enable_metric=>true, charset=>"UTF-8">, ssl=>false, 
add_hostname=>true, ssl_verify_mode=>"none", ssl_peer_metadata=>false, 
include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, 
tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", 
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", 
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", 
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", 
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], client_inactivity_timeout=>60, 
executor_threads=>8>
Error: Address already in use
Exception: Java::JavaNet::BindException
    Stack: sun.nio.ch.Net.bind0(Native Method)
 sun.nio.ch.Net.bind(sun/nio/ch/Net.java:438)
 sun.nio.ch.Net.bind(sun/nio/ch/Net.java:430)
 sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:225)

管道.yml

- pipeline.id: first-pipeline
      path.config: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/pipelines/api-address.config"
      pipeline.batch.size: 1

- pipeline.id: second-pipeline
      path.config: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/pipelines/my-config.config"
      pipeline.workers: 1
      pipeline.batch.size: 1
      queue.type: persisted
      path.queue: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/queue"

我得配置.配置

input {
    beats {
        host => "127.0.0.1"
        port => 5044
    }
}
filter {

}
output {

}

api地址.配置

input {
    beats {
        host => "127.0.0.1"
        port => 5044
    }
}
filter {

}
output {

}
jk9hmnmh

jk9hmnmh1#

您不能有两个具有相同端口的输入,但是您可以使用分发服务器模式在一个输入中接收所有内容,然后使用所需的配置将其发送到不同的管道。
因此您可以:
主流水线

input {
    beats {
        host => "127.0.0.1"
        port => 5044
    }
}
filter {
}
output {
    if([someField] === "some_value") {
        pipeline {
            send_to => first_pipeline
        }
    }
    else {
        pipeline {
            send_to => second_pipeline
        }
    }

}

则各个管线应如下所示:

input {
    pipeline {
        address => first_pipeline
    }
}

...

和管道文件:

- pipeline.id: master-pipeline
      path.config: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/pipelines/master-pipeline.config"
      pipeline.batch.size: 1
- pipeline.id: first-pipeline
      path.config: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/pipelines/api-address.config"
      pipeline.batch.size: 1
- pipeline.id: second-pipeline
      path.config: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/pipelines/my-config.config"
      pipeline.workers: 1
      pipeline.batch.size: 1
      queue.type: persisted
      path.queue: "/Users/gyrao/Documents/ELK/logstash-6.5.4/config/queue"

点击此处阅读更多信息:https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html#distributor-pattern

相关问题