在将Jenkins升级到2.375.4
版本,将Kubernetes AWS EKS集群升级到v1.23
沿着将容器运行时从docker
更改为containerd
之后,我有时会在通过Jenkins代理(Jenkins slave)在Kubernetes AWS EKS集群上运行的Jenkins作业上遇到以下错误。
下面是我得到的错误:
03:39:51 java.nio.channels.ClosedChannelException
03:39:51 Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip-10-20-53-103.eu-west-1.compute.internal/10.20.53.103:38004
03:39:51 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
03:39:51 at hudson.remoting.Request.call(Request.java:199)
03:39:51 at hudson.remoting.Channel.call(Channel.java:999)
03:39:51 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:153)
03:39:51 at jdk.internal.reflect.GeneratedMethodAccessor1121.invoke(Unknown Source)
03:39:51 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
03:39:51 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
03:39:51 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:138)
03:39:51 at com.sun.proxy.$Proxy262.execute(Unknown Source)
03:39:51 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1359)
03:39:51 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
03:39:51 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
03:39:51 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
03:39:51 at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
03:39:51 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
03:39:51 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
03:39:51 Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: df677487-c98d-4870-aa71-74faab41e552
03:39:51 Also: org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for current-frontend-e2e-native-deps-9106-4ptkj-3hltr-q0stv; current-frontend-e2e-native-deps-9106-4ptkj-3hltr-q0stv was marked offline: Connection was broken
03:39:51 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:182)
03:39:51 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:154)
03:39:51 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$Translator.get(ExecutorStepDynamicContext.java:147)
03:39:51 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:164)
03:39:51 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:154)
03:39:51 at org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:95)
03:39:51 at org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:139)
03:39:51 at org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:137)
03:39:51 at org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297)
03:39:51 at org.jenkinsci.plugins.workflow.cps.CpsBodySubContext.doGet(CpsBodySubContext.java:88)
03:39:51 at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:75)
03:39:51 at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Callback.finished(CoreWrapperStep.java:187)
03:39:51 at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Execution2$Callback2.finished(CoreWrapperStep.java:150)
03:39:51 at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution$TailCall.lambda$onFailure$1(GeneralNonBlockingStepExecution.java:156)
03:39:51 at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution.lambda$run$0(GeneralNonBlockingStepExecution.java:77)
03:39:51 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
03:39:51 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
03:39:51 Caused: hudson.remoting.RequestAbortedException
03:39:51 at hudson.remoting.Request.abort(Request.java:346)
03:39:51 at hudson.remoting.Channel.terminate(Channel.java:1080)
03:39:51 at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:241)
03:39:51 at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:221)
03:39:51 at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
03:39:51 at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:289)
03:39:51 at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:168)
03:39:51 at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
03:39:51 at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:155)
03:39:51 at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:143)
03:39:51 at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)
03:39:51 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:30)
03:39:51 at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:70)
03:39:51 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
03:39:51 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
03:39:51 at java.base/java.lang.Thread.run(Thread.java:829)
这是什么原因呢?怎么修?
1条答案
按热度按时间t5fffqht1#
可能的解决方案:
1.请确保您的
kubernetes-plugin
是最新版本,而不是过时的。1.确保您的Jenkins master的
java
版本与Jenkins slave的java
版本匹配,以避免任何不兼容。1.确保pod没有节流,并且有足够的CPU和/或内存。如果没有,请增加其中一个或两个以解决此问题。
**我是如何找到答案的?三个?
在Grafana上查看该作业pod的容器的指标,我意识到CPU使用率达到100%,这导致了jnlp容器的CPU节流。增加其CPU请求和限制修复了该问题。
旧配置:
新配置: