Selenium:当超过32个会话同时处于活动状态时,Chrome开始崩溃,无法创建会话

sd2nnvve  于 2023-06-27  发布在  Go
关注(0)|答案(2)|浏览(198)

我们正在尝试通过Selenium同时处理多个页面。我们最多可以得到32个会话,但是一旦我们试图进一步增加会话计数,所有会话都开始变得不稳定,它们容易出错并崩溃。此外,我们得到了许多错误,sesssion无法启动(错误日志在底部)
我已经尝试了超过5个版本的selenium(在102和最新的114之间),使用适当的chrome驱动程序和chrome版本,但迟早整个事情都会出现上面的错误。Chrome总是崩溃。我还尝试过在中心节点配置或独立配置中运行.
为什么Chrome会崩溃?我确信内存不是问题,因为我也尝试过在上面的命令行中添加“-Xms 64 g-Xmx 80 g”来增加jvm堆大小,但是没有;根据selenium的要求,我已经安装并使用了oracle jdk(不是jre),但仍然没有任何帮助。如何运行多个会话(即超过32个)?

**操作系统:**Ubuntu 22.04.2 LTS,运行在32核Ryzen™ 9和128 GB RAM上。
**java:**java版本“20.0.1”2023-04-18 Java(TM)SE Runtime Environment(build 20.0.1+9-29)Java HotSpot(TM)64位服务器VM(build 20.0.1+9-29,混合模式,共享)
谷歌浏览器114.0.5735.133
启动selenium的命令:

xvfb-run -a java -Dwebdriver.chrome.driver="/home/forge/domain/storage/selenium/chromedriver-114.0.5735.16" -jar "/home/forge/domain/storage/selenium/selenium-server-4.10.0.jar" standalone --override-max-sessions true --max-sessions 60 --port 4646 --allow-cors true

错误日志:

10:39:23.636 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: 
Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) 
Host info: host: 'domain', ip: '127.0.0.1'
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
10:39:23.636 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. 
Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) 
Host info: host: 'domain', ip: '127.0.0.1'
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:210)
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:69)
    at org.openqa.selenium.grid.node.local.SessionSlot.apply(SessionSlot.java:147)
    at org.openqa.selenium.grid.node.local.LocalNode.newSession(LocalNode.java:467)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:643)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:560)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:814)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:773)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1623)

mem stats内存:131576260 15530844 76271352 82296 39774064 114669600交换:5237752 0 5237752

我正在创建会话,如RemoteWebDriver::create($host,$capabilities,3600000,3600000);

'--remote-allow-origins=*',
            '--window-size=1920,1080',
            '--enable-experimental-extension-apis',
            '--disable-infobars',
            '--disable-notifications',
            '--disable-device-discovery-notifications',
            '--disable-gpu',
            '--allow-outdated-plugins',
            '--disable-web-security',
            '--allow-running-insecure-content',
            '--allow-file-access-from-log_files',
            '--audio-output-channels=0',
            '--ignore-certificate-error',
            '--disable-dev-shm-usage',
            '--disable-browser-side-navigation',
            '--dns-prefetch-disable',
            '--disable-geolocation',
            '--disable-search-geolocation-disclosure',
            '--enable-strict-powerful-feature-restrictions',
            '--disable-translate',
            '--disable-sync',
            '--mute-audio',
            '--lang=en-US',
            '--use-fake-ui-for-media-stream',
            '--use-fake-device-for-media-stream',
            '--allow-file-access-from-files',
            '--reduce-security-for-testing',
            '--allow-insecure-localhost',
            '--allow-legacy-extension-manifests',
            '--allow-loopback-in-peer-connection',
            '--allow-outdated-plugins',
            '--allow-unsecure-dlls',
            '--alsa-mute-device-name',
            '--also-emit-success-logs',
            '--alsa-mute-element-name',
            '--no-first-run',
            '--disable-default-apps',
            '--disable-extensions-http-throttling',
            '--disable-blink-features=AutomationControlled',
wkyowqbh

wkyowqbh1#

看你的日志,你似乎在使用SeleniumGrid功能,你读过文档了吗?您是否在设计中考虑了他们的建议?因为这可能是CPU限制,而不是内存限制。
默认情况下,节点支持的最大并发会话量受可用CPU数量的限制。例如,如果Node机器有8个CPU,它最多可以运行8个并发浏览器会话(Safari除外,它总是一个)。此外,预计每个浏览器会话应使用大约1GB RAM。
请注意,默认值(每个浏览器1CPU/1GB RAM)是一个建议,它们不能应用于您的上下文。建议将它们用作参考,但持续测量性能将有助于确定环境的理想值。
来源
当您声明时,请考虑上述文档:
我们最多可以得到32个会话,但是一旦我们试图进一步增加会话计数,所有会话都开始变得不稳定
并将提供的CPU信息:
运行在32核Ryzen™ 9上
事情似乎正在好转。

0h4hbjxa

0h4hbjxa2#

@BernardV手册指出了如何通过使用这两个参数来覆盖会话数的方法,并指出“不推荐”而不是禁止。

"--override-max-sessions
      The # of available processors is the recommended max sessions value (1 
      browser session per processor). **Setting this flag to true allows the 
      recommended max value to be overwritten**. Session stability and 
      reliability might suffer as the host could run out of resources.
      Default: false
--max-sessions
      Maximum number of concurrent sessions. Default value is the number of 
      available processors."

我试过这个,因为在其他系统上它曾经运行得很好。你的答案不; t带来任何光对chrome崩溃btw,这也发生,即使运行(cpu计数)会话。谢谢你的时间,但真的,我会很感激一些回答的用户有过这种情况。

相关问题