logstash 带有grok模式的Nginx访问日志

af7jpaap  于 2023-03-16  发布在  Logstash
关注(0)|答案(1)|浏览(181)

我的nginx日志格式如下,有某些日志文件将有空值(如没有 request_time/hrrp_x_forwarded_forhttp_host),在这种情况下,grok模式不工作,它不会单独过滤消息正文,但作为一个完整的消息出现,有无论如何跳过空值?
例如,如果 $request_time 为空,则跳过此字段并继续其他字段。

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$request_time"  "$http_x_forwarded_for" $http_host ';```


grok {
        match => { "message" => "%{IPORHOST:remote_ip} - %{DATA:user_name} \[%{HTTPDATE:time}\] \"%{WORD:method}%{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent:bytes} (?:\"(?:%{DATA:referrer}|-))\" \"%{DATA:agent}\" (?:\"(?:%{NUMBER:request_time}|-))\"  (?:\"(?:%{DATA:http_x_forwarded_for}|-))\" (?:%{IPORHOST:http_host}) " }
        remove_field => "message"
      
    }
ftf50wuq

ftf50wuq1#

您的过滤器需要“-”,但request_time/hrrp_x_forwarded_for和http_host根本不存在请尝试以下操作:

grok {
        match => { "message" => "%{IPORHOST:remote_ip} - %{DATA:user_name} \[%{HTTPDATE:access_time}\] \"%{WORD:http_method} %{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent_bytes} \"%{DATA:referrer}\" \"%{DATA:agent}\"(?:(?: \"%{NUMBER:request_time}\"|-) (?:\"%{DATA:http_x_forwarded_for}\"|-) (?:\"%{IPORHOST:http_host}\"|-)|)" }
        remove_field => "message"
      
    }

相关问题