Filebeat/Logstash从输出中删除不需要的字段和值

8yparm6h 于 2023-02-27 发布在 Logstash

关注(0)|答案(3)|浏览(678)

我的Filebeat配置非常简单-

- input_type: log
  paths:
    - C:\log\FilebeatInputTest.txt

output.logstash:
  hosts: ["http://X.X.X.X:XXXX"]

如果我在ilebeatInputTest.txt中写一些东西，比如-This is from Filebeat
我在ElasticSearch中得到类似于-....... "index": "logstash-" "source" : { "@timestamp": "2017-05-19T06:41:02.663Z", "beat": { "hostname": "CHITTARS02", "name": "CHITTARS02", "version": "5.4.0" }, "input_type": "log", "message": "This is from Filebeat", "offset": 23, "source": "C:\\log\\FilebeatInputTest.txt", "type": "log" } .....的输出
我的渠道是Filebeat(monitoring FilebeatInputTest.txt) > Logstash > Elasticsearch
logstash.cnf，具体如下-

input {

    beats {
        port => 25000
    }
}
output {

    elasticsearch {
        hosts => ["http://xx.xx.xx.xx:XX"]
        user => "elastic"
        password => "changeme"
    }
}

问题：我可以从输出中删除所有不需要的键和值吗？也就是说，我希望我的输出应该是这样的-
....... "index": "logstash-" "source" : { "message": "This is from Filebeat", } ......
我想删除"@timestamp", "beat","input_type""offset","source","type"
我试着跟着-

filter{
    prune {
        blacklist_names => ["@timestamp", "beat","input_type""offset","source","type"]
    }

}

还有

filter{
    mutate {
        remove_field => ["@timestamp", "beat","input_type""offset","source","type"]
    }
}

但是没有帮助，结果是一样的

logstash

来源：https://stackoverflow.com/questions/44067767/filebeat-logstash-remove-unwanted-fields-values-from-output

3条答案

按热度按时间

iqjalb3h1#

您使用的方法是正确的，但remove_field列表中有一处输入错误。您漏掉了一个逗号。它应该是：

filter{
    mutate {
        remove_field => [ "@timestamp", "beat", "input_type", "offset", "source", "type" ]
    }
}

赞(0）回复(0）举报 2023-02-27

bweufnob2#

另一个解决方案是使用**filebeat**删除这些字段。

processors:
  - add_host_metadata: ~
  - drop_fields:
    fields: ["type", "@version", "offset", "tags"]

赞(0）回复(0）举报 2023-02-27

wljmcqd83#

可能猜测是您忘记将端口放在引号中;即使用"25000"而不是25000。请尝试以下操作

input {

    beats {
        port => "25000"
    }
}

filter{
    mutate {
        remove_field => ["@timestamp", "beat","input_type","offset","source","type","@version","host","tags"]
    }
}

output {

    elasticsearch {
        hosts => ["http://xx.xx.xx.xx:XX"]
        user => "elastic"
        password => "changeme"
    }
}

输入

This is from Filebeat

产出

{
    "_index" : "logstash-",
    "_type" : "logs",
    "_id" : "AVwglLbLfqaeaIoZluvE",
    "_score" : 1.0,
    "_source" : {
      "message" : "This is from Filebeat"
    }
}

我还删除了字段"@version","host"和"tags"。
希望这个有用。

赞(0）回复(0）举报 2023-02-27

我来回答

Filebeat/Logstash从输出中删除不需要的字段和值

3条答案

相关问题

热门标签

最新问答