我的Jenkins示例位于私有VPC网络中的EC2计算机(t3.medium)上,由Nginx**提供服务,在长时间构建期间丢失连接。**工作线程是相同类型的-相同区域/子网中的EC2计算机,并且具有相同的JAVA版本。
Jenkins version: Jenkins 2.319.3
Java: openjdk version "1.8.0_312"
OS: Ubuntu 20.02
通过SSH连接实现连接。
我试图解决这个问题:
1.我更改了EC2类型。由于内存不足,我更改了类型,问题仍然存在。
1.更新JAVA版本-我把JAVA升级到了Java11。没有任何效果。
1.我更改了代理/工作SSHD配置:(添加了客户端活动间隔80)
1.我增加了worker配置中的连接超时秒数(60-〉6000)
1.我使用选项通过命令将工作器连接到Jenkins Master。连接仍在丢失。
1.我配置了更积极的TCPKeepAlive参数:
sysctl -w net.ipv4.tcp_keepalive_time=120
sysctl -w net.ipv4.tcp_keepalive_intvl=30
sysctl -w net.ipv4.tcp_keepalive_probes=8
sysctl -w net.ipv4.tcp_fin_timeout=30
1.我在JAVA选项中添加了hudson. slaves. ChannelPinger. pingIntervalSeconds = -1
你知道这里会出什么问题吗?
错误:
04:01:35 FATAL: command execution failed
04:01:36 java.io.EOFException
04:01:36 at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
04:01:36 at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
04:01:36 at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
04:01:36 at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
04:01:36 at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
04:01:36 at hudson.remoting.Command.readFrom(Command.java:142)
04:01:36 at hudson.remoting.Command.readFrom(Command.java:128)
04:01:36 at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
04:01:36 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
04:01:36 Caused: java.io.IOException: Unexpected termination of the channel
04:01:36 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
参考文献:
- https://github.com/jenkinsci/ssh-slaves-plugin/blob/main/doc/TROUBLESHOOTING.md#threads-stuck-at-credentialsprovidertrackall
- https://issues.jenkins.io/browse/JENKINS-44001?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&showAll=true
- https://groups.google.com/g/jenkinsci-issues/c/t2p0Sk1LqXI/m/6lwnqQxth6QJ
Nginx配置:
upstream jenkins {
server 127.0.0.1:8080;
}
server {
listen 443 ssl;
server_name XXX.CCC.net;
ssl_certificate /etc/nginx/valid_cert/XXX.pem;
ssl_certificate_key /etc/nginx/valid_cert/XXX.CCC.net.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/jenkins.access.log;
ssl_session_cache shared:SSL:10m;
ssl_stapling on;
ssl_stapling_verify on;
location / {
try_files $uri @app;
}
location @app {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_next_upstream error;
proxy_pass http://jenkins;
proxy_redirect http:// https://;
proxy_read_timeout 150;
}
}
1条答案
按热度按时间nbnkbykc1#
1.我更改了EC2类型。由于内存不足,我更改了类型,问题仍然存在。
1.更新JAVA版本-我把JAVA升级到了Java11。没有任何效果。
1.我更改了代理/工作SSHD配置:(添加了客户端活动间隔80)
1.我增加了worker配置中的连接超时秒数(60-〉6000)
1.我使用选项通过命令将工作器连接到Jenkins Master。连接仍在丢失。
1.我配置了更积极的TCPKeepAlive参数:
1.我在JAVA选项中添加了hudson. slaves. ChannelPinger. pingIntervalSeconds = -1