FluentBit不适用于Kubernetes滤波器的多个尾部选项

h7appiyu  于 2022-11-28  发布在  Kubernetes
关注(0)|答案(1)|浏览(120)

这是一个在使用单尾输入时有效的方法

inputs: |
    [INPUT]
        Name    tail
        Tag     kube.*
        Path    /var/log/containers/*.log
        Parser  docker

  filters: |
    [FILTER]
        Name             kubernetes
        Match            *
        Kube_URL         https://kubernetes.default.svc:443
        Kube_CA_File     /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File  /var/run/secrets/kubernetes.io/serviceaccount/token
        Kube_Tag_Prefix  kube.var.log.containers.
        Merge_Log        On
        Merge_Log_Key    log_processed
    [FILTER]
        Name    Lua
        Match   kube.*
        code    function dummy_filter(a,b,c)local n=c;n["dummy"]="dummy";return 2,b,n end
        call    dummy_filter
    [FILTER]
        Name             parser
        Match            kube.*
        Key_Name         log
        Parser           tomcat_parser
        Preserve_Key     On
        Reserve_Data     On
    [FILTER]
        Name    Lua
        Match   kube.*
        code    function dummy_filter1(a,b,c)local n=c;n["dummy1"]="dummy1";return 2,b,n end
        call    dummy_filter1

  customParsers: |
    [PARSER]
       Format regex
       Name   tomcat_parser
       Regex  ^(?<apptime>[0-9-a-zA-Z]+\s[0-9:\.]+)\s+(?<level>[a-zA-Z]+)\s+\[(?<thread>[a-zA-Z]+)\]\s+(?<applog>.*$)

  outputs: |
    [OUTPUT]
        Name              cloudwatch_logs
        Match             kube.*
        Region            ${region}
        Log_Group_Name    /myapps/logs
        Log_Stream_Prefix my
        Auto_Create_Group On
        net.keepalive     Off

这是不起作用的。/myapps/tomcatlogs中的最终输出包含来自除kubernetes之外的所有3个剩余过滤器的数据。

inputs: |
    [INPUT]
        Name    tail
        Tag     kube.*
        Path    /var/log/containers/*.log
        Parser  docker
    [INPUT]
        Name    tail
        Tag     tomcat.*
        Path    /var/log/containers/tomcat*.log. (checked even *.log doesn't work)
        Parser  docker

  filters: |
    [FILTER]
        Name             kubernetes
        Match            *
        Kube_URL         https://kubernetes.default.svc:443
        Kube_CA_File     /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File  /var/run/secrets/kubernetes.io/serviceaccount/token
        Kube_Tag_Prefix  kube.var.log.containers.
        Merge_Log        On
        Merge_Log_Key    log_processed
    [FILTER]
        Name    Lua
        Match   tomcat.*
        code    function dummy_filter(a,b,c)local n=c;n["dummy"]="dummy";return 2,b,n end
        call    dummy_filter
    [FILTER]
        Name             parser
        Match            tomcat.*
        Key_Name         log
        Parser           tomcat_parser
        Preserve_Key     On
        Reserve_Data     On
    [FILTER]
        Name    Lua
        Match   tomcat.*
        code    function dummy_filter1(a,b,c)local n=c;n["dummy1"]="dummy1";return 2,b,n end
        call    dummy_filter1

  customParsers: |
    [PARSER]
       Format regex
       Name   tomcat_parser
       Regex  ^(?<apptime>[0-9-a-zA-Z]+\s[0-9:\.]+)\s+(?<level>[a-zA-Z]+)\s+\[(?<thread>[a-zA-Z]+)\]\s+(?<applog>.*$)

  outputs: |
    [OUTPUT]
        Name              cloudwatch_logs
        Match             kube.*
        Region            ${region}
        Log_Group_Name    /myapps/logs
        Log_Stream_Prefix my
        Auto_Create_Group On
        net.keepalive     Off
    [OUTPUT]
        Name              cloudwatch_logs
        Match             tomcat.*
        Region            ${region}
        Log_Group_Name    /myapps/tomcatlogs
        Log_Stream_Prefix my
        Auto_Create_Group On
        net.keepalive     Off

我不喜欢现有的sol,因为非tomcat日志也会在tomcat过滤器中被评估。任何指导都将受到赞赏。

erhoui1w

erhoui1w1#

您的tomcat Input将记录标记为tomcat. *,这意味着它们将被管理为:

tomcat.var.log.containers.tomcat*.log

你的Kubernetes过滤器

Kube_Tag_Prefix   kube.var.log.containers.

所以tomcat标记的记录不适合kube标记前缀,所以它不能正确解析日志文件名。你可以在[SERVICE]中将Log_level设置为debug for fluent-bit。这将为你提供关于正在发生的事情的更详细的信息。
希望这对你有帮助!

相关问题