使用filebeat 7.9.3正确解析k8s docker容器json日志

bttbmeg0  于 2021-06-10  发布在  ElasticSearch
关注(0)|答案(0)|浏览(804)

我正在使用FileBeat7.9.3作为k8s上的守护程序。我无法解析springboot应用程序的docker容器日志,该应用程序用json将日志写入stdout。事实上,springboot应用程序日志的每一行都是这样写的:

{ "@timestamp": "2020-11-16T13:39:57.760Z", "log.level": "INFO", "message": "Checking comment 'se' done = true", "service.name": "conduit-be-moderator", "event.dataset": "conduit-be-moderator.log", "process.thread.name": "http-nio-8081-exec-2", "log.logger": "it.koopa.app.ModeratorController", "transaction.id": "1ed5c62964ff0cc2", "trace.id": "20b4b28a3817c9494a91de8720522972"}

但是/var/log/containers/writes log下对应的docker日志文件是这样写的:

{
"log": "{\"@timestamp\":\"2020-11-16T11:27:32.273Z\", \"log.level\": \"INFO\", \"message\":\"Checking comment 'a'\", \"service.name\":\"conduit-be-moderator\",\"event.dataset\":\"conduit-be-moderator.log\",\"process.thread.name\":\"http-nio-8081-exec-4\",\"log.logger\":\"it.koopa.app.ModeratorController\",\"transaction.id\":\"9d3ad972dba65117\",\"trace.id\":\"8373edba92808d5e838e07c7f34af6c7\"}\n",
"stream": "stdout",
"time": "2020-11-16T11:27:32.274816903Z" 
}

我总是在filebeat日志上收到这个

Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}

这是我的filebeat配置,它试图解析docker日志中的json日志消息,我在docker日志中使用decode\u json\u字段来捕获elasticsearch标准字段(我使用co.elastic.logging.logback.ecsencoder)

filebeat.yml: |-
filebeat.inputs:
- type: container

  #json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
  json.message_key: log

  paths:
    - /var/log/containers/*.log

  include_lines: "conduit-be-moderator"

  processors:
    - decode_json_fields:
        fields: ["log"]
        overwrite_keys: true
    - add_kubernetes_metadata:
        host: ${NODE_NAME}
        in_cluster: true
        matchers:
          - logs_path:
              logs_path: "/var/log/containers/"

processors:
  - add_cloud_metadata:
  - add_host_metadata:

我该怎么做???

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题