debugging td-agent.log没有错误日志，但日志不再出现在http端点中

我正在使用fluentd将日志发送到我的coralogix帐户。
我配置好了所有的东西，使我的td-agent.service运行正常，没有错误，如td-agent.log所示。但是，我仍然找不到我帐户上的日志。
以下是我的td-agent.log的日志：

2023-02-04 20:09:08 +0800 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-02-04 20:09:08 +0800 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil
2023-02-04 20:09:08 +0800 [info]: adding match pattern="application.log" type="http"
2023-02-04 20:09:08 +0800 [warn]: #0 Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::HTTPOutput" secondary="Fluent::Plugin::StdoutOutput"
2023-02-04 20:09:08 +0800 [info]: adding source type="tail"
2023-02-04 20:09:08 +0800 [info]: #0 starting fluentd worker pid=5624 ppid=5621 worker=0
2023-02-04 20:09:08 +0800 [info]: #0 following tail of /var/log/Log.log
2023-02-04 20:09:08 +0800 [info]: #0 fluentd worker is now running worker=0

---〉请看我的td-agent.conf文件：

<source>
    @type tail
    @id tail_var_logs
    @label @CORALOGIX
    read_from_head true
    tag application.log
    path /var/log/Log.log
    pos_file /var/log/td-agent/tmp/coralog.pos
        path_key path
    <parse>
      @type none
    </parse>
  </source>

<label @CORALOGIX>
    <filter application.log>
    @type record_transformer
    @log_level warn
    enable_ruby true
    auto_typecast true
    renew_record true
    <record>
      applicationName "Example_App"
      subsystemName "Example_Subsystem"
      #text ${record.to_json}
    </record>
    </filter>

<match application.log>
    @type http
    endpoint https://api.coralogixsg.com/logs/rest/singles
    headers {"private_key":"<my private key>"}
    retryable_response_codes 503
    error_response_as_unrecoverable false
    <buffer>
      @type memory
      chunk_limit_size 10MB
      compress gzip
      flush_interval 1s
      retry_max_times 5
      retry_type periodic
      retry_wait 2
    </buffer>
    <secondary>
      #If any messages fail to send they will be send to STDOUT for debug.
      @type stdout
    </secondary>
</match>
</label>

---〉请使用td-agent -vv查看详细日志：

2023-02-05 08:48:49 +0800 [trace]: #0 fluent/log.rb:287:trace: enqueueing all chunks in buffer instance=2000
2023-02-05 08:48:54 +0800 [trace]: #0 fluent/log.rb:287:trace: enqueueing all chunks in buffer instance=2000
2023-02-05 08:49:00 +0800 [trace]: #0 fluent/log.rb:287:trace: enqueueing all chunks in buffer instance=2000
2023-02-05 08:49:05 +0800 [trace]: #0 fluent/log.rb:287:trace: enqueueing all chunks in buffer instance=2000
2023-02-05 08:49:10 +0800 [trace]: #0 fluent/log.rb:287:trace: enqueueing all chunks in buffer instance=2000
2023-02-05 08:49:16 +0800 [debug]: #0 fluent/log.rb:309:debug: tailing paths: target = /var/log/Log.log | existing = /var/log/Log.log

@azeem您是如何解决与http端点的连接问题的？遇到了与您类似的问题-缓冲区目录中没有显示任何内容，但服务器显然从syslog转发器接收到许多日志作为syslog传入。因此，没有任何内容进入我们的目的S3存储桶。

<source>
  @type syslog
    protocol_type tcp
    port 514
    bind 0.0.0.0
    <parse>19   @type regexp
      expression /^(?<message>.*)/
    </parse>
    tag firewall
</source>

<filter Company.name.** >
   @type record_transformer
   <record>
     tag ${tag}
     time ${time}
   </record>
</filter>

<match Company.name.**>
  @type s3
    aws_key_id redacted
    aws_sec_key redacted
    s3_bucket test-bucket
    s3_region us-east-1
    s3_object_key_format %{path}/%{time_slice}_%{index}.%{file_extension}
    include_time_key true
    time_slice_format %Y%m%d
    buffer_type file
    flush_interval 30s
    timekey 5
    timekey_use_utc true
    buffer_path /var/log/td-agent/buffer/s3
    buffer_chunk_limit 5MB
  <store>
    path ${tag[0]}/${tag[2]}/%Y/%m/%d/
  </store>
</match>

debugging td-agent.log没有错误日志，但日志不再出现在http端点中

1条答案

相关问题

热门标签

最新问答