groovy 客户端异常:70000毫秒后未准备就绪

gywdnpxw  于 2022-11-01  发布在  其他
关注(0)|答案(2)|浏览(287)

从AKSJenkins代理pod得到这个错误。任何想法是什么原因会导致这个错误?故障排除步骤我做了。
将Jenkins恢复到旧版本=〉导致相同的错误将Jenkins升级到所有新版本,包括正在使用的插件=〉导致相同的错误。根据github中的一些建议将Jenkins K8s和K8s API插件降级到稳定版本。=〉相同的错误创建全新的集群并安装Jenkins和作业pod,开始出现相同的错误。=〉相同的错误
如何解决这个问题?

18:23:33  [Pipeline] // podTemplate
18:23:33  [Pipeline] End of Pipeline
18:23:33  io.fabric8.kubernetes.client.KubernetesClientException: not ready after 5000 MILLISECONDS
18:23:33    at io.fabric8.kubernetes.client.utils.Utils.waitUntilReadyOrFail(Utils.java:176)
18:23:33    at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:322)
18:23:33    at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:84)
18:23:33    at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:413)
18:23:33    at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:330)
18:23:33    at hudson.Launcher$ProcStarter.start(Launcher.java:507)
18:23:33    at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:176)
18:23:33    at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:132)
18:23:33    at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:324)
18:23:33    at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:319)
18:23:33    at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:193)
18:23:33    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
18:23:33    at jdk.internal.reflect.GeneratedMethodAccessor6588.invoke(Unknown Source)
18:23:33    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
18:23:33    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
18:23:33    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
18:23:33    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
18:23:33    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
18:23:33    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
18:23:33    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
18:23:33    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
18:23:33    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
18:23:33    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
18:23:33    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:158)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
18:23:33    at WorkflowScript.run(WorkflowScript:114)
18:23:33    at ___cps.transform___(Native Method)
18:23:33    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
18:23:33    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
18:23:33    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
18:23:33    at jdk.internal.reflect.GeneratedMethodAccessor210.invoke(Unknown Source)
18:23:33    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
18:23:33    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
18:23:33    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
18:23:33    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
18:23:33    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
18:23:33    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
18:23:33    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
18:23:33    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
18:23:33    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
18:23:33    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
18:23:33    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
18:23:33    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
18:23:33    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
a0zr77ik

a0zr77ik1#

**EDIT:**看起来没有什么区别。仍然得到5000毫秒的超时,所以不确定这个方法是否有效(至少使用环境变量)。如果你真的能够改变超时,它可能会有效,但我还没有弄清楚。

开始看到同样的问题后,更新Jenkins(和插件)只-而不是K8S集群。
只不过我得到的是5000毫秒而不是7000毫秒。

io.fabric8.kubernetes.client.KubernetesClientException: not ready after 5000 MILLISECONDS

深入研究Github上的堆栈跟踪和源代码会导致这个默认超时(6年来一直没有改变),所以不知何故,你似乎有一个非默认的超时。

public static final Long DEFAULT_WEBSOCKET_TIMEOUT = 5 * 1000L;

它似乎可以被pod上的环境变量KUBERNETES_WebSocket_TIMEOUT_SYSTEM_PROPERTY覆盖。我只是尝试将我的设置提高到10秒,看看是否会有不同。
可能值得一试。如果是这样,这可能表明集群API服务器的响应速度比预期的要慢。我不知道在我的情况下,在升级的时候有什么会影响集群性能,而且由于默认的超时时间多年没有改变,这似乎很奇怪。也许一些代码被重构了,不再忽略超时/重试等-只是猜测。
编辑:我正在裸机集群上运行

ubof19bj

ubof19bj2#

我的解决方案是在控制器(Jenkins)服务行命令中添加-Dkubernetes.websocket.timeout=60000 -Dorg.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.websocketConnectionTimeout=60000这也适用于插件的最后版本

相关问题