logstash:提供整个文件,并通过使用换行符拆分来创建一个新事件

pzfprimi  于 2021-06-08  发布在  Kafka
关注(0)|答案(1)|浏览(573)

我有以下用于从kafka读取类似syslog的消息的logstash配置:

input {
    kafka {
        bootstrap_servers => "172.24.0.3:9092"
        topics => ["test"]
    }
}
filter {
    grok {
        match => { "message" => "%{SYSLOGTIMESTAMP}" }
    }
}
output {
    stdout { codec => rubydebug }
}

因此,当在logstash input发送syslog行时,在stdout生成以下消息:
来自Kafka

r = p1.send('test', b'Jul 16 09:07:47 ubuntu user: test500')

标准

{
       "message" => "Jul 16 09:07:47 ubuntu user: test500",
      "@version" => "1",
    "@timestamp" => 2018-07-16T12:29:57.854Z,
          "host" => "6d87dde4c74e"
}

现在,我想添加多行 \n 每行末尾的字符,logstash将输入作为两条分开的消息进行处理,以便logstash stdout类似于以下示例:
Kafka在同一封信里写了多行字

r = p1.send('test', b'Jul 16 09:07:47 ubuntu user: test501\nJul 16 09:07:47 ubuntu user: test502')

所需标准件

{
       "message" => "Jul 16 09:07:47 ubuntu user: test501",
      "@version" => "1",
    "@timestamp" => 2018-07-16T12:29:57.854Z,
          "host" => "6d87dde4c74e"
}
{
       "message" => "Jul 16 09:07:47 ubuntu user: test502",
      "@version" => "1",
    "@timestamp" => 2018-07-16T12:29:57.854Z,
          "host" => "6d87dde4c74e"
}

你知道如何在logstash上实现这种行为吗?

bvk5enib

bvk5enib1#

我通过使用line codec实现了上述行为:

input {
    kafka {
        bootstrap_servers => "172.24.0.3:9092"
        topics => ["test"]
        ## ## ## ## ## 
        codec => line
        ## ## ## ## ##
    }
    stdin {}
}

相关问题