td-agent.config文件
<match test>
type webhdfs
host localhost
port 50070
path /test/%Y%m%d_%H
username hdfs
output_include_tag false
remove_prefix test
time_format %Y-%m-%d %H:%M:%S
output_include_time true
format json
localtime
buffer_type file
buffer_path /test/test
buffer_chunk_limit 4m
buffer_queue_limit 50
flush_interval 3s
</match>
在hdfs日志文件中显示如下:
2016-02-22 16:04:15 {"login_id":123,"email":"abcd@gmail.com"}
在存储到文件之前,是否有任何方法将fluentd时间字段而不是客户机时间嵌入到json数据中,例如:
{"time_key":"2016-02-22 16:04:15","login_id":123,"email":"abcd@gmail.com"}
1条答案
按热度按时间y3bcpkx11#
我有办法:
使用插件https://github.com/repeatedly/fluent-plugin-record-modifier
添加字段时间,然后按入hdfs
:)