无特定固定日志格式的grok模式

vvppvyoh  于 2021-06-09  发布在  ElasticSearch
关注(0)|答案(0)|浏览(205)

我有一个如下格式的日志文件。日志格式可能会根据接收请求和响应的时间而改变。总之,没有具体的固定顺序。

2020-11-21 23:25:44 INFO  IMAClientHandler:691 - 10.112.2.66#3788 (XXSPID: 0004990218) Request: CBS:037.9295300;040.1775033     OKM:20201111;144358     PUT ENDEKS 0004990218b0000000001039.476000000526.586000000681.59900
2020-11-21 23:25:44 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0004990218) Put Endeks - Start: 021 Nov 2020 23:25:44:354
2020-11-21 23:25:44 INFO  PutIndexService:48 - 10.112.2.66 (XXSPID: 0004990218) Put Index Request: <?xml version="1.0" encoding="UTF-8" standalone="no"?><SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:cm="http://ouaf.oracle.com/webservices/cm/CM-IMAService"><SOAP-ENV:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" SOAP-ENV:mustUnderstand="1"><wsu:Timestamp wsu:Id="TS-23186FF29E3959712215507979422034"><wsu:Created>2020-11-21T20:25:44.354Z</wsu:Created><wsu:Expires>2020-11-21T20:27:44.354Z</wsu:Expires></wsu:Timestamp></wsse:Security></SOAP-ENV:Header><SOAP-ENV:Body><cm:CM-IMAService><cm:request><cm:index><cm:opticalPortRead>false</cm:opticalPortRead><cm:readDateTime>2020-11-11T14:43:58</cm:readDateTime><cm:servicePoint>0004990218</cm:servicePoint><cm:signalLevel>b</cm:signalLevel><cm:counters><cm:lastIndex>0001039.476</cm:lastIndex><cm:externalUOM>1.8.1</cm:externalUOM><cm:meterStatusCode>00</cm:meterStatusCode></cm:counters><cm:counters><cm:lastIndex>0000526.586</cm:lastIndex><cm:externalUOM>1.8.2</cm:externalUOM><cm:meterStatusCode>00</cm:meterStatusCode></cm:counters><cm:counters><cm:lastIndex>0000681.599</cm:lastIndex><cm:externalUOM>1.8.3</cm:externalUOM><cm:meterStatusCode>00</cm:meterStatusCode></cm:counters><cm:counters><cm:lastIndex>2247.661</cm:lastIndex><cm:externalUOM>1.8.0</cm:externalUOM></cm:counters></cm:index><cm:latitude>037.929530</cm:latitude><cm:longitude>040.177503</cm:longitude><cm:readerId>2940</cm:readerId><cm:dataSource>CMIM</cm:dataSource></cm:request></cm:CM-IMAService></SOAP-ENV:Body></SOAP-ENV:Envelope>
2020-11-21 23:25:44 INFO  PutIndexService:52 - 10.112.2.66 (XXSPID: 0013051187) Put Index Response: <?xml version='1.0' encoding='UTF-8'?><env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" env:mustUnderstand="1"><wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsu:Created>2020-11-21T20:25:44Z</wsu:Created><wsu:Expires>2020-11-21T20:26:44Z</wsu:Expires></wsu:Timestamp></wsse:Security></env:Header><env:Body><ouaf:CM-IMAService xmlns:ouaf="http://ouaf.oracle.com/webservices/cm/CM-IMAService"><ouaf:request><ouaf:latitude>037.929530</ouaf:latitude><ouaf:longitude>040.177503</ouaf:longitude><ouaf:dataSource>CMIM</ouaf:dataSource><ouaf:readerId>2926</ouaf:readerId><ouaf:index><ouaf:servicePoint>0013051187</ouaf:servicePoint><ouaf:readDateTime>2020-11-11T14:43:58+03:00</ouaf:readDateTime><ouaf:signalLevel>b</ouaf:signalLevel><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0006626.195</ouaf:lastIndex><ouaf:externalUOM>1.8.1</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0003605.748</ouaf:lastIndex><ouaf:externalUOM>1.8.2</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0004468.820</ouaf:lastIndex><ouaf:externalUOM>1.8.3</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:lastIndex>14700.763</ouaf:lastIndex><ouaf:externalUOM>1.8.0</ouaf:externalUOM></ouaf:counters><ouaf:opticalPortRead>false</ouaf:opticalPortRead></ouaf:index></ouaf:request><ouaf:response><ouaf:message1>SUCCESS</ouaf:message1></ouaf:response></ouaf:CM-IMAService></env:Body></env:Envelope>
2020-11-21 23:25:44 INFO  PutIndexService:75 - 10.112.2.66 (XXSPID: 0013051187) Put Index Response: SUCCESS
2020-11-21 23:25:44 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0013051187) Put Endeks - End: 021 Nov 2020 23:25:44:744
2020-11-21 23:25:44 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0013051187) Bill Service - Start: 021 Nov 2020 23:25:44:744
2020-11-21 23:25:44 INFO  BillService:63 - 10.112.2.66 Billing request: <?xml version="1.0" encoding="UTF-8" standalone="no"?><SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:cm="https://c2m-pp.depsas.com.tr/ouaf/webservices/CM-OnSpotBill?WSDL"><SOAP-ENV:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" SOAP-ENV:mustUnderstand="1"><wsu:Timestamp wsu:Id="TS-23186FF29E3959712215507979422034"><wsu:Created>2020-11-21T20:25:44.745Z</wsu:Created><wsu:Expires>2020-11-21T20:27:44.745Z</wsu:Expires></wsu:Timestamp></wsse:Security></SOAP-ENV:Header><SOAP-ENV:Body><cm:CM-OnSpotBill><cm:request><cm:spId>0013051187</cm:spId><cm:billSegmentEndDate>2020-11-11</cm:billSegmentEndDate></cm:request></cm:CM-OnSpotBill></SOAP-ENV:Body></SOAP-ENV:Envelope>
2020-11-21 23:25:46 INFO  PutIndexService:52 - 10.112.2.66 (XXSPID: 0001094950) Put Index Response: <?xml version='1.0' encoding='UTF-8'?><env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" env:mustUnderstand="1"><wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsu:Created>2020-11-21T20:25:46Z</wsu:Created><wsu:Expires>2020-11-21T20:26:46Z</wsu:Expires></wsu:Timestamp></wsse:Security></env:Header><env:Body><ouaf:CM-IMAService xmlns:ouaf="http://ouaf.oracle.com/webservices/cm/CM-IMAService"><ouaf:request><ouaf:latitude>037.929530</ouaf:latitude><ouaf:longitude>040.177503</ouaf:longitude><ouaf:dataSource>CMIM</ouaf:dataSource><ouaf:readerId>2925</ouaf:readerId><ouaf:index><ouaf:servicePoint>0001094950</ouaf:servicePoint><ouaf:readDateTime>2020-11-11T14:43:58+03:00</ouaf:readDateTime><ouaf:signalLevel>b</ouaf:signalLevel><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0000916.771</ouaf:lastIndex><ouaf:externalUOM>1.8.1</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0000475.189</ouaf:lastIndex><ouaf:externalUOM>1.8.2</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0000484.322</ouaf:lastIndex><ouaf:externalUOM>1.8.3</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:lastIndex>1876.282</ouaf:lastIndex><ouaf:externalUOM>1.8.0</ouaf:externalUOM></ouaf:counters><ouaf:opticalPortRead>false</ouaf:opticalPortRead></ouaf:index></ouaf:request><ouaf:response><ouaf:message1>SUCCESS</ouaf:message1></ouaf:response></ouaf:CM-IMAService></env:Body></env:Envelope>
2020-11-21 23:25:46 INFO  PutIndexService:75 - 10.112.2.66 (XXSPID: 0001094950) Put Index Response: SUCCESS
2020-11-21 23:25:46 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0001094950) Put Endeks - End: 021 Nov 2020 23:25:46:841
2020-11-21 23:25:46 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0001094950) Bill Service - Start: 021 Nov 2020 23:25:46:841
2020-11-21 23:25:46 INFO  BillService:63 - 10.112.2.66 Billing request: <?xml version="1.0" encoding="UTF-8" standalone="no"?><SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:cm="https://c2m-pp.depsas.com.tr/ouaf/webservices/CM-OnSpotBill?WSDL"><SOAP-ENV:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" SOAP-ENV:mustUnderstand="1"><wsu:Timestamp wsu:Id="TS-23186FF29E3959712215507979422034"><wsu:Created>2020-11-21T20:25:46.842Z</wsu:Created><wsu:Expires>2020-11-21T20:27:46.842Z</wsu:Expires></wsu:Timestamp></wsse:Security></SOAP-ENV:Header><SOAP-ENV:Body><cm:CM-OnSpotBill><cm:request><cm:spId>0001094950</cm:spId><cm:billSegmentEndDate>2020-11-11</cm:billSegmentEndDate></cm:request></cm:CM-OnSpotBill></SOAP-ENV:Body></SOAP-ENV:Envelope>
2020-11-21 23:25:47 INFO  PutIndexService:52 - 10.112.2.66 (XXSPID: 0004990218) Put Index Response: <?xml version='1.0' encoding='UTF-8'?><env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" env:mustUnderstand="1"><wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsu:Created>2020-11-21T20:25:47Z</wsu:Created><wsu:Expires>2020-11-21T20:26:47Z</wsu:Expires></wsu:Timestamp></wsse:Security></env:Header><env:Body><ouaf:CM-IMAService xmlns:ouaf="http://ouaf.oracle.com/webservices/cm/CM-IMAService"><ouaf:request><ouaf:latitude>037.929530</ouaf:latitude><ouaf:longitude>040.177503</ouaf:longitude><ouaf:dataSource>CMIM</ouaf:dataSource><ouaf:readerId>2940</ouaf:readerId><ouaf:index><ouaf:servicePoint>0004990218</ouaf:servicePoint><ouaf:readDateTime>2020-11-11T14:43:58+03:00</ouaf:readDateTime><ouaf:signalLevel>b</ouaf:signalLevel><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0001039.476</ouaf:lastIndex><ouaf:externalUOM>1.8.1</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0000526.586</ouaf:lastIndex><ouaf:externalUOM>1.8.2</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:meterStatusCode>00</ouaf:meterStatusCode><ouaf:lastIndex>0000681.599</ouaf:lastIndex><ouaf:externalUOM>1.8.3</ouaf:externalUOM></ouaf:counters><ouaf:counters><ouaf:lastIndex>2247.661</ouaf:lastIndex><ouaf:externalUOM>1.8.0</ouaf:externalUOM></ouaf:counters><ouaf:opticalPortRead>false</ouaf:opticalPortRead></ouaf:index></ouaf:request><ouaf:response><ouaf:message1>SUCCESS</ouaf:message1></ouaf:response></ouaf:CM-IMAService></env:Body></env:Envelope>
2020-11-21 23:25:47 INFO  PutIndexService:75 - 10.112.2.66 (XXSPID: 0004990218) Put Index Response: SUCCESS
2020-11-21 23:25:47 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0004990218) Put Endeks - End: 021 Nov 2020 23:25:47:879
2020-11-21 23:25:47 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0004990218) Bill Service - Start: 021 Nov 2020 23:25:47:879
2020-11-21 23:25:47 INFO  BillService:63 - 10.112.2.66 Billing request: <?xml version="1.0" encoding="UTF-8" standalone="no"?><SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:cm="https://c2m-pp.depsas.com.tr/ouaf/webservices/CM-OnSpotBill?WSDL"><SOAP-ENV:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" SOAP-ENV:mustUnderstand="1"><wsu:Timestamp wsu:Id="TS-23186FF29E3959712215507979422034"><wsu:Created>2020-11-21T20:25:47.879Z</wsu:Created><wsu:Expires>2020-11-21T20:27:47.879Z</wsu:Expires></wsu:Timestamp></wsse:Security></SOAP-ENV:Header><SOAP-ENV:Body><cm:CM-OnSpotBill><cm:request><cm:spId>0004990218</cm:spId><cm:billSegmentEndDate>2020-11-11</cm:billSegmentEndDate></cm:request></cm:CM-OnSpotBill></SOAP-ENV:Body></SOAP-ENV:Envelope>
2020-11-21 23:25:48 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0013051187) Bill Service Response: maliye_muhur   9093
m_fatura_seri   9093
m_fatura_no 668281706777
banka_talimat
message_1 SUCCESS
message_2
fatura_dipnot
dagitim_birim_fiyat 0.28965517

2020-11-21 23:25:48 INFO  IMAClientHandler:691 - 10.112.2.66 (XXSPID: 0013051187) Bill Service - End: 021 Nov 2020 23:25:48:679

我可以做什么来过滤这些日志如下?你能为它写一个奇怪的图案吗?或者别的什么?

{
  "event_start_time": [
    [
      "2020-11-21 23:25:44"
    ]
  ],
  "log_level": [
     [
      "INFO"
     ]
    ],
  "IP": [
    [
      "10.112.2.66"
    ]
  ],
 "XXSPID": [
    [
      "0004990218"
    ]
  ],
  "Put_Endeks_Start_Time": [
    [
      "21 Nov 2020 23:25:44:354"
    ]
  ],
  "Put_Endeks_end_Time": [
    [
      "21 Nov 2020 23:25:44:744"
    ]
  ],
  "bill_service_start_time": [
    [
      "21 Nov 2020 23:25:44:744"
    ]
  ],
  "message_1": [
    [
      "SUCCESS"
    ]
  ],
  "bill_service_end_time": [
    [
      "21 Nov 2020 23:25:48:679"
    ]
  ]
}

我在等你的帮助。谢谢你

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题