如何检查日志是否从suricata接收/解析数据到elasticsearch？

sxissh06 于 2021-06-15 发布在 ElasticSearch

关注(0)|答案(2)|浏览(694)

尝试在mac os x 10.10.3 yosemite上使用elasticsearch（v1.5.2）-logstash（v1.4.2）-kibana（v4.0.2）配置suricata v2.0.8。
苏里塔亚马尔：


# Extensible Event Format (nicknamed EVE) event log in JSON format

  - eve-log:
      enabled: yes
      type: file #file|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert
        - http:
            extended: yes     # enable this for extended logging information
            # custom allows additional http fields to be included in eve-log
            # the example below adds three additional fields when uncommented
            #custom: [Accept-Encoding, Accept-Language, Authorization]
        - dns
        - tls:
            extended: yes     # enable this for extended logging information
        - files:
            force-magic: yes   # force logging magic on all logged files
            force-md5: yes     # force logging of md5 checksums
        #- drop
        - ssh
        #- smtp
        #- flow

logstash.conf文件：

input {
  file {
    path => ["/var/log/suricata/eve.json"]
    sincedb_path => ["/var/lib/logstash/"]
    codec =>   json
    type => "SuricataIDPS"
    start_position => "beginning"
  }

}

filter {
  if [type] == "SuricataIDPS" {
    date {
      match => [ "timestamp", "ISO8601" ]
    }
    ruby {
      code => "if event['event_type'] == 'fileinfo'; event['fileinfo']['type']=event['fileinfo']['magic'].to_s.split(',')[0]; end;"
    }
  }

  if [src_ip]  {
    geoip {
      source => "src_ip"
      target => "geoip"
      #database => "/usr/local/opt/logstash/libexec/vendor/geoip/GeoLiteCity.dat"
      add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
      add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
    }
    mutate {
      convert => [ "[geoip][coordinates]", "float" ]
    }
    if ![geoip.ip] {
      if [dest_ip]  {
        geoip {
          source => "dest_ip"
          target => "geoip"
          #database => "/usr/local/opt/logstash/libexec/vendor/geoip/GeoLiteCity.dat"
          add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
          add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
        }
        mutate {
          convert => [ "[geoip][coordinates]", "float" ]
        }
      }
    }
  }
}

output {
  elasticsearch {
    host => localhost
    #protocol => http
  }
}

suricata成功地将所有事件记录到eve.json中。当我在浏览器中打开kibana时，我看不到任何 Jmeter 板或suricata的任何信息。。。所以我假设要么logstash不从eve.json读取数据，要么不将数据解析到elasticsearch（或者两者都是）。。。有什么方法可以检查发生了什么吗？

elasticsearch logstash kibana kibana-4 logstash-configuration

来源：https://stackoverflow.com/questions/30409035/how-to-check-if-logstash-receiving-parsing-data-from-suricata-to-elasticsearch

2条答案

按热度按时间

nnt7mjpx1#

我将nginx日志改编为suricata日志。我可以在suricata日志中找到geoip信息。我通过swatch进行自适应，并发送到filebeat中配置的日志文件。
例如：nginx.access.referer:et info nat（stun绑定请求）的会话遍历实用程序[**
nginx.access.geoip.location:{“lon”：-119.688，“lat”：45.8696}
使用样例读取suricata日志，并将它们发送到shell脚本，该脚本将执行改编。
例如：echo“$ip---[$nd4]\”get$ip2:$port2---$type http/1.1\“777 0\”$cve\“\”mozilla/5.0（none）（none）none\”>/var/log/suricata\u mod.log
然后配置filebeat.yml：
文档类型：nginx access
路径：
/var/log/surita\u mod.log
重新启动filebeat。
最后配置日志存储：

filter {
         if [type] == "nginx-access" {
       grok {
  match => { "message" => ["%{IPORHOST:[nginx][access][remote_ip]} - %{DATA:[nginx][access][user_name]} \[%{HTTPDATE:[nginx][access][time]}\] \"%{WORD:[nginx][access][method]} %{DATA:[nginx][access][url]} HTTP/%{NUMBER:[$
  remove_field => "message"} 

    mutate {      
    add_field => { "read_timestamp" => "%{@timestamp}" }} 

    date {
    match => [ "[nginx][access][time]", "dd/MMM/YYYY:H:m:s Z" ]
    remove_field => "[nginx][access][time]"}

    useragent {
       source => "[nginx][access][agent]"
       target => "[nginx][access][user_agent]"
       remove_field => "[nginx][access][agent]"} 

    geoip {
       source => "[nginx][access][remote_ip]"
       target => "[nginx][access][geoip]"
       database => "/opt/GeoLite2-City.mmdb"}} }     output {
      elasticsearch {
          hosts => [ "xxx.xxx.xxx.xxx:9200" ]
          manage_template => false
          document_type => "%{[@metadata][type]}"
          index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"}}

重新启动logstash。在kibana中创建一个filebeat-*索引。准备好了。

赞(0）回复(0）举报 2021-06-16

dgiusagp2#

在logstash中打开调试输出：

output {
   stdout {
      codec = rubydebug
   }
}

另外，尝试直接对elasticsearch（curl）而不是kibana运行查询。

赞(0）回复(0）举报 2021-06-15

我来回答

如何检查日志是否从suricata接收/解析数据到elasticsearch？

2条答案

相关问题

热门标签

最新问答