regex PCRE -仅当整行匹配特定字符串时才匹配变量字符串

ngynwnxp  于 2023-06-25  发布在  其他
关注(0)|答案(2)|浏览(131)

这是针对日志事件的，我想匹配可能存在于数据中的某些字段，但只针对包含某个字符串的事件，在本例中该字符串为type=“traffic”
显然我应该更新这个问题，而不是问另一个问题，对不起，我不能接受这个答案，有人在点上给了我。
谢谢！
下面是示例事件，我想捕获任何包含type=“traffic”但不包含type=“utm”的行中的特定字段

Apr 29 11:08:16 10.150.148.52 date=2023-04-29 time=11:08:17 devname="ABCZWPFWFTG-B" devid="ABCVM8V0000158159" eventtime=1682791697444922720 tz="-0700" logid="1059028704" type="utm" subtype="app-ctrl" eventtype="signature" level="information" vd="root" appid=40568 srcip=10.150.150.10 dstip=20.62.63.153 srcport=55544 dstport=443 srcintf="port2" srcintfrole="lan" dstintf="port1" dstintfrole="wan" proto=6 service="SSL" direction="incoming" policyid=1 sessionid=4976047 applist="PROD-APPCTRL-AZURE" action="pass" appcat="Web.Client" app="HTTPS.BROWSER" hostname="124537f1-b52d-4e77-a6bd-e73c9904ea48.agentsvc.azure-automation.net" incidentserialno=205723388 url="/" msg="Web.Client: HTTPS.BROWSER," apprisk="medium" scertcname="*.azure-automation.net" scertissuer="Microsoft RSA TLS CA 01"
Apr 29 11:08:16 10.150.148.52 date=2023-04-29 time=11:08:17 devname="ABCZWPFWFTG-B" devid="ABCVM8V0000158159" eventtime=1682791697444901220 tz="-0700" logid="1059028704" type="utm" subtype="app-ctrl" eventtype="signature" level="information" vd="root" appid=15895 srcip=10.150.150.10 dstip=20.62.63.153 srcport=55544 dstport=443 srcintf="port2" srcintfrole="lan" dstintf="port1" dstintfrole="wan" proto=6 service="SSL" direction="outgoing" policyid=1 sessionid=4976047 applist="PROD-APPCTRL-AZURE" action="pass" appcat="Network.Service" app="SSL" hostname="124537f1-b52d-4e77-a6bd-e73c9904ea48.agentsvc.azure-automation.net" incidentserialno=205723383 url="/" msg="Network.Service: SSL," apprisk="elevated" scertcname="*.azure-automation.net" scertissuer="Microsoft RSA TLS CA 01"
Apr 29 11:08:16 10.150.148.52 date=2023-04-29 time=11:08:17 devname="ABCZWPFWFTG-B" devid="ABCVM8V0000158159" eventtime=1682791697444603820 tz="-0700" logid="1059028704" type="utm" subtype="app-ctrl" eventtype="signature" level="information" vd="root" appid=40568 srcip=45.42.34.136 dstip=10.150.148.104 srcport=60638 dstport=443 srcintf="port1" srcintfrole="wan" dstintf="port5" dstintfrole="dmz" proto=6 service="SSL" direction="incoming" policyid=10 sessionid=4976049 applist="PROD-APPCTRL-AZURE" action="pass" appcat="Web.Client" app="HTTPS.BROWSER" hostname="www.testdata.com" incidentserialno=205723390 url="/" msg="Web.Client: HTTPS.BROWSER," apprisk="medium" scertcname="www.testdata.com"
May 19 16:32:23 10.150.160.13 date=2023-05-19 time=16:32:25 devname="fw1-test-lv-external" devid="ABC1K5DT918800482" eventtime=1684539145135795404 tz="-0700" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="PRODUCTION" srcip=10.150.161.11 srcport=64507 srcintf="ABCTCORPO3.3052" srcintfrole="lan" dstip=208.11.121.76 dstport=53 dstintf="ABCTINTPO1.3053" dstintfrole="wan" srccountry="Reserved" dstcountry="United States" sessionid=547154413 proto=17 action="accept" policyid=247 policytype="policy" poluuid="07588088-f351-51ec-153c-4a07e49c5818" policyname="Microsoft DNS to Umbrella" service="DNS" trandisp="snat" transip=38.70.139.3 transport=64507 duration=249 sentbyte=169 rcvdbyte=231 sentpkt=2 rcvdpkt=2 appcat="unscanned" srchwvendor="Cisco" devtype="Network" srcfamily="AP" osname="Cisco IOS" mastersrcmac="12:12:12:12:dc:27" srcmac="12:12:12:12:dc:27" srcserver=0
May 19 16:23:57 10.150.160.13 date=2023-05-19 time=16:23:58 devname="fw1-test-lv-external" devid="ABC1K5DT918800482" eventtime=1684538639125610717 tz="-0700" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="PRODUCTION" srcip=10.150.161.11 srcport=63392 srcintf="ABCTCORPO3.3052" srcintfrole="lan" dstip=208.11.121.76 dstport=53 dstintf="ABCTINTPO1.3053" dstintfrole="wan" srccountry="Reserved" dstcountry="United States" sessionid=547134202 proto=17 action="accept" policyid=247 policytype="policy" poluuid="07588088-f351-51ec-153c-4a07e49c5818" policyname="Microsoft DNS to Umbrella" service="DNS" trandisp="snat" transip=38.70.139.3 transport=63392 duration=145 sentbyte=230 rcvdbyte=382 sentpkt=3 rcvdpkt=3 appcat="unscanned" sentdelta=230 rcvddelta=382 srchwvendor="Cisco" devtype="Network" srcfamily="AP" osname="Cisco IOS" mastersrcmac="12:12:12:12:dc:27" srcmac="12:12:12:12:dc:27" srcserver=0
May 19 16:25:35 10.132.119.14 date=2023-05-19 time=16:25:36 devname="FW1-testMAIN-ABCT01" devid="ABCT3KD3Z17800372" eventtime=1684538737153322514 tz="-0700" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="PRODUCTION" srcip=10.151.143.4 srcport=50423 srcintf="port4" srcintfrole="lan" dstip=52.111.145.1 dstport=443 dstintf="port3" dstintfrole="undefined" srccountry="Reserved" dstinetsvc="Microsoft-Office365" dstcountry="United States" dstregion="California" dstcity="San Jose" dstreputation=5 sessionid=3673596551 proto=6 action="accept" policyid=10045 policytype="policy" poluuid="96f15028-15d6-51e9-6b81-d98bf1466b99" user="JULLOPEZ" authserver="FSSO_PSR" service="Microsoft-Office365" trandisp="snat" transip=199.68.152.135 transport=50423 appid=41468 app="Microsoft.Office.365.Portal" appcat="Collaboration" apprisk="elevated" applist="Edge-Prod-Block-Mode-P2P_PROXY" duration=30553 sentbyte=94401 rcvdbyte=112203 sentpkt=1052 rcvdpkt=1539 sentdelta=254 rcvddelta=230
May 19 16:26:00 10.150.160.13 date=2023-05-19 time=16:26:01 devname="fw1-test-lv-external" devid="ABC1K5DT918800482" eventtime=1684538762118706615 tz="-0700" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="PRODUCTION" srcip=10.150.106.11 srcport=54254 srcintf="ABCTCORPO3.3052" srcintfrole="lan" dstip=17.188.143.10 dstport=443 dstintf="ABCTINTPO1.3053" dstintfrole="wan" srccountry="Reserved" dstcountry="United States" sessionid=513091673 proto=6 action="accept" policyid=83 policytype="policy" poluuid="2k2k2-b4c8-51e9-512e-62cf5b7e3bcd" policyname="Internal Server Nets Outbound" service="HTTPS" trandisp="snat" transip=38.70.139.3 transport=54254 appid=42662 app="Apple.Services" appcat="General.Interest" apprisk="elevated" applist="PROD-APPCTRL_LV-EXT" appact="detected" duration=686309 sentbyte=22070530 rcvdbyte=14199406 sentpkt=279649 rcvdpkt=148924 sentdelta=3600 rcvddelta=2352 srchwvendor="Cisco" devtype="Network" srcfamily="AP" osname="Cisco IOS" mastersrcmac="12:12:12:12:dc:27" srcmac="12:12:12:12:dc:27" srcserver=0
May 19 16:26:59 10.151.129.106 date=2023-05-19 time=16:27:00 devname="FW1-testPSR-DC" devid="ABCT3KD3Z17800305" eventtime=1684538820421783095 tz="-0700" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="DataCenter" srcip=10.151.110.100 srcname="PSRPSOLAPP01.test.NET" identifier=2875 srcintf="Enterprise_ACI" srcintfrole="wan" dstip=10.132.116.4 dstname="10.132.116.4" dstintf="Enterprise" dstintfrole="lan" srccountry="Reserved" dstcountry="Reserved" sessionid=3796675657 proto=1 action="accept" policyid=10654 policytype="policy" poluuid="6ddcbe16-9058-51ec-2052-64e35cf6fddc" policyname="Solarwinds Catch-ALL" user="SVC-SOLARWINDS-IPAM" authserver="FSSO_PSR" service="PING" trandisp="noop" duration=60 sentbyte=59 rcvdbyte=59 sentpkt=1 rcvdpkt=1 appcat="unscanned"
May 19 16:33:13 10.150.148.52 date=2023-05-19 time=16:33:14 devname="ABCZWPFWFTG-B" devid="ABCVM8V0000158159" eventtime=1684539194871377700 tz="-0700" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="root" srcip=10.151.100.36 identifier=18877 srcintf="TUNNEL_SCH" srcintfrole="undefined" dstip=10.150.148.52 dstintf="port2" dstintfrole="lan" srccountry="Reserved" dstcountry="Reserved" sessionid=105443335 proto=1 action="accept" policyid=8 policytype="policy" poluuid="06d7ce0e-e8ae-51ed-b77f-59e907ddba86" policyname="test TO AZURE LAN" service="icmp/8/0" trandisp="noop" appid=24466 app="Ping" appcat="Network.Service" apprisk="elevated" applist="PROD-APPCTRL-AZURE" duration=60 sentbyte=84 rcvdbyte=84 sentpkt=1 rcvdpkt=1 vpn="TUNNEL_SCH" vpntype="ipsec-static" utmaction="allow" countapp=1 masterdstmac="12:12:12:12:9a:bc" dstmac="12:12:12:12:9a:bc" dstserver=1

这个正则表达式是基于其中一个答案的，所以我根据实际数据重写了它，但它不工作

.*type="(anomaly|log|event|utm)".*(*SKIP)(*FAIL)|(^.{15})\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).+?(devname=\S+)\s(devid=\S+).+?(?:vd=\H+)|(srcip=\H+)|(srcport=\H+)|(srcintf=\H+)|(dstip=\H+)|(dstport=\H+)|(dstintf=\H+)|(proto=\H+)|(action=\H+)|(policyid=\H+)|(user=\H+)|(service=\H+)|(transport=\H+)|(app=\H+)|(applist=\H+)|(vpn=\H+)|(vpntype="?\S+"?)|(?:\s+)|(?:\S+=".+?")|(?:\S+=\S+)

以下是到目前为止最好的解决方案的regex示例链接：
https://regex101.com/r/BDkbMb/1

regex

2条答案

按热度按时间
ccrfmcuu

ccrfmcuu1#

PCRE/PCRE 2（使用\K和\G）：

(?(DEFINE)                 # Subpattern declaration:
  (?<field>                # Match a field
    \b                     # with a key that is either
    (?:field1|field2)      # 'field1' or 'field2' (insert other field names here)
    =\S+                   # then a '=' and 1+ non-whitespace chars.
    \b                     #
  )                        #
)                          #
                           # Main pattern:
\g<field>(?=.*\btype=a\b)  # A field followed by anything then 'type=a'
|                          # or
(?:\btype=a\b|\G(?!^)).*?  # 'type=a' or the end of the last match, followed by anything,
\K                         # all of which we forfeit (not included in the match),
\g<field>                  # then a field.

试试on regex101.com。
ECMAScript/.NET（带lookbehind）：

\b(?:field[1234])=\S+\b  # Match a field
(?=.*\btype=a\b)         # followed by 'type=a'
|                        # or
(?<=\btype=a\b.*)        #         preceded by 'type=a'.
\b(?:field[1234])=\S+\b  # a field

试试on regex101.com。

赞(0）分享  回复(0）举报  2023-06-25
r8uurelv

r8uurelv2#

您可以通过匹配日志的开始日期、时间和IP开始。
然后跳过匹配所有你不想要的键=值对，匹配那些匹配其中一个替代项的键=值对：

(?:^[A-Z][a-z]+\h+\d{1,2}\h+\d{2}:\d{2}:\d{2}\h+\d{1,3}(?:\.\d{1,3}){3}\b(?=.*?\btype="traffic")(?!.*\btype="utm")|\G(?!^))(?:(?!\h+(?:dev(?:name|id)|vd|src(?:ip|port|intf)|dst(?:ip|port|intf)|proto|action|policyid|user|service|transport|app(?:list)?|vpn(?:type)?)=)\h+\S+)*+\h*\K[^\s=]+=\S+

说明

  • (?:备选方案的非捕获组
  • ^字符串开始
  • [A-Z][a-z]+\h+\d{1,2}\h+\d{2}:\d{2}:\d{2}\h+\d{1,3}(?:\.\d{1,3}){3}\b匹配前导数据、时间和类似ip的格式
  • (?=.*?\btype="traffic")(?!.*\btype="utm")Assert右边是type="traffic"而不是type="utm"
  • |或
  • \G(?!^)在上一个匹配的末尾而不是开始处声明当前位置
  • )关闭非捕获组
  • (?:非捕获组
  • (?!负向前看，Assert从当前位置向右不是
  • \h+匹配1+水平空格字符
  • (?:dev(?:name|id)|vd|src(?:ip|port|intf)|dst(?:ip|port|intf)|proto|action|policyid|user|service|transport|app(?:list)?|vpn(?:type)?)=)匹配其中一个选项
  • \h+\S+匹配1+个水平空格字符和1+个非空格字符
  • )*+关闭非捕获组，并选择使用所有格量词重复
  • \h*\K匹配可选的水平空格字符，并忘记匹配的字符
  • [^\s=]+=\S+匹配键值对

Regex demo

赞(0）分享  回复(0）举报  2023-06-25
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com