Kubernetes云配置作为Jenkins中的代理：问题-连续Pod创建和未知客户端名称错误

thtygnil  于 2023-10-17  发布在  Jenkins
关注(0)|答案(1)|浏览(147)

我正在配置Kubernetes云环境，以管理QA Jenkins服务器的代理配置。然而，我们遇到了两个主要问题：
1.持续Pod创建：在启动Jenkins作业后，我们观察到一个看似相似的名称（例如，sample-job-2-9-x79k3-vlvp9-v6lkj）的pod的连续创建。这导致大量pod被不必要地产生，消耗资源。

  1. Pod日志中出现“未知客户端名称”错误：在创建的pod中，我们看到错误消息“Unknown client name：sample-job-2-9-x79k3-vlvp9-v6lkj”。此错误导致中断并阻止成功的代理连接。
    我已经遵循了几个Jenkins文档，并验证了设置。然而，这些问题仍然存在，影响了Jenkins管道的执行。
    包含在Jenkins和代理级别了解环境和错误日志所需的大部分信息。
Jenkins Job log:
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes-test jenkinsns/sample-job-2-9-x79k3-vlvp9-g134p
**ERROR: Failed to launch sample-job-2-9-x79k3-vlvp9-g134p**
Also:   java.lang.Throwable: waiting here
    at io.fabric8.kubernetes.client.utils.Utils.waitUntilReady(Utils.java:173)
    at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.waitUntilCondition(BaseOperation.java:890)
    at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.waitUntilReady(BaseOperation.java:878)
    at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.waitUntilReady(BaseOperation.java:93)
    at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:169)
    at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:298)
    at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
io.fabric8.kubernetes.client.KubernetesClientException: **Failure executing: GET at: https://10.65.184.52:6443/api/v1/namespaces/jenkinsns/pods?fieldSelector=metadata.name%3Dsample-job-2-9-x79k3-vlvp9-g134p&resourceVersion=759022&allowWatchBookmarks=true&watch=true. Message: Forbidden.**
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:728)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:708)
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.lambda$start$3(WatchConnectionManager.java:128)
#############################################################################
Kubernetes cloud configuration in Jenkins->Manage Jenkins->Cloud section

Name : kubernetes-test
Kubernetes URL : https://10.65.184.52:6443
Disable https certificate check : Checked
Kubernetes Namespace : jenkinsns
Credentials : Added (secret text id)
Jenkins URL: http://asvdqjenkins01:8080/
Jenkins tunnel: asvdqjenkins01:5555
Pod Labels by default: Key -> jenkins, Value -> salve

Pod Templates -> Not added
#############################################################################
Jenkins job name: sample_job_2
Jenkins job pipeline script:

pipeline {
    agent {
        // Reference the Kubernetes cloud configuration by its name
        kubernetes {
            cloud 'kubernetes-test'
            defaultContainer 'jnlp'
        }
    }
    
    stages {
        stage('Test') {
            steps {
                // Add your testing steps here
                sh 'echo "Testing in Kubernetes"'
            }
        }
    }
}
#############################################################################

Job is running continously and pods are getting created one after another. Here is the screenshot.

root@invk8-dqjenkins1-m1:~# kubectl get pods -n jenkinsns
NAME                               READY   STATUS    RESTARTS   AGE
sample-job-2-7-twtpt-510vt-mxcf5   1/1     Running   0          27m
sample-job-2-7-twtpt-510vt-xvn79   1/1     Running   0          27m
sample-job-2-9-x79k3-vlvp9-g134p   1/1     Running   0          13m
sample-job-2-9-x79k3-vlvp9-v6lkj   1/1     Running   0          13m
Here is the Pod log from one such pods sample-job-2-9-x79k3-vlvp9-v6lkj:

Aug 23, 2023 5:46:21 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: sample-job-2-9-x79k3-vlvp9-v6lkj
Aug 23, 2023 5:46:21 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3142.vcfca_0cd92128
Aug 23, 2023 5:46:21 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using /home/jenkins/agent/remoting as a remoting work directory
Aug 23, 2023 5:46:21 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
INFO: Both error and output logs will be printed to /home/jenkins/agent/remoting
Aug 23, 2023 5:46:21 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://asvdqjenkins01:8080/]
Aug 23, 2023 5:46:22 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Aug 23, 2023 5:46:22 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
Aug 23, 2023 5:46:22 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
  Agent address: asvdqjenkins01
  Agent port:    5555
  Identity:      88:f3:10:6d:77:7c:c4:68:4a:eb:96:9a:b2:a3:01:e2
Aug 23, 2023 5:46:22 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Aug 23, 2023 5:46:22 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to asvdqjenkins01:5555
Aug 23, 2023 5:46:22 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Aug 23, 2023 5:46:22 AM org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader run
INFO: Waiting for ProtocolStack to start.
Aug 23, 2023 5:46:23 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 88:f3:10:6d:77:7c:c4:68:4a:eb:96:9a:b2:a3:01:e2
Aug 23, 2023 5:46:23 AM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to asvdqjenkins01/10.23.185.62:5555] Local headers refused by remote: Unknown client name: sample-job-2-9-x79k3-vlvp9-v6lkj
Aug 23, 2023 5:46:23 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: **Unknown client name: sample-job-2-9-x79k3-vlvp9-v6lkj
        at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
        at hudson.remoting.Engine.innerRun(Engine.java:814)
        at hudson.remoting.Engine.run(Engine.java:543)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: sample-job-2-9-x79k3-vlvp9-v6lkj**
        at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:380)
        at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:435)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
        at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:289)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:168)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
        at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:155)
        at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$700(BIONetworkLayer.java:51)
        at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:257)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:125)
        at java.base/java.lang.Thread.run(Unknown Source)
        Suppressed: java.nio.channels.ClosedChannelException
                ... 7 more
Aug 23, 2023 5:46:23 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: reconnect rejected, sleeping 10s:
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
        at hudson.remoting.Engine.onConnectionRejected(Engine.java:893)
        at hudson.remoting.Engine.innerRun(Engine.java:840)
        at hudson.remoting.Engine.run(Engine.java:543)
#############################################################################

Here is the result output of pod details: 
`kubectl describe pod sample-job-2-9-x79k3-vlvp9-v6lkj -n jenkinsns`

Name:         sample-job-2-9-x79k3-vlvp9-v6lkj
Namespace:    jenkinsns
Priority:     0
Node:         invk8-dqjenkins1-w2/10.65.186.128
Start Time:   Wed, 23 Aug 2023 11:16:20 +0530
Labels:       jenkins=slave
              jenkins/label=sample_job_2_9-x79k3
              jenkins/label-digest=5948c9090bdf7c052caef5b9ef866f7bb0b8bf64
Annotations:  buildUrl: http://asvdqjenkins01:8080/job/sample_job_2/9/
              cni.projectcalico.org/containerID: 567412e70106ba3acc4dd821f23851050582dedf21572d64efea5c08ae5faba9
              cni.projectcalico.org/podIP: 192.168.39.46/32
              cni.projectcalico.org/podIPs: 192.168.39.46/32
              runUrl: job/sample_job_2/9/
Status:       Running
IP:           192.168.39.46
IPs:
  IP:  192.168.39.46
Containers:
  jnlp:
    Container ID:   docker://4999eebd72f259382de1ba00d63e40b6d5ac90cc5e1dee7b35cc4aee2f6ec844
    Image:          jenkins/inbound-agent:3142.vcfca_0cd92128-1
    Image ID:       docker-pullable://jenkins/inbound-agent@sha256:704a4b18ac78355701e89584999f2a4ba54e10cc674421a65a61763b0032eafe
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Wed, 23 Aug 2023 11:16:21 +0530
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:     100m
      memory:  256Mi
    Environment:
      JENKINS_SECRET:         3efe2e05d59f988076ab1ed9836b9236c2d439cd2b99640ecd7123e94d76cbbe
      JENKINS_TUNNEL:         asvdqjenkins01:5555
      JENKINS_AGENT_NAME:     sample-job-2-9-x79k3-vlvp9-v6lkj
      JENKINS_NAME:           sample-job-2-9-x79k3-vlvp9-v6lkj
      JENKINS_AGENT_WORKDIR:  /home/jenkins/agent
      JENKINS_URL:            http://asvdqjenkins01:8080/
    Mounts:
      /home/jenkins/agent from workspace-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lqppt (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  workspace-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kube-api-access-lqppt:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  24m   default-scheduler  Successfully assigned jenkinsns/sample-job-2-9-x79k3-vlvp9-v6lkj to invk8-dqjenkins1-w2
  Normal  Pulled     24m   kubelet            Container image "jenkins/inbound-agent:3142.vcfca_0cd92128-1" already present on machine
  Normal  Created    24m   kubelet            Created container jnlp
  Normal  Started    24m   kubelet            Started container jnlp
#############################################################################

Jenkins setup:

Jenkins: 2.401.3
OS: Linux - 5.15.0-79-generic
Java: 11.0.20 - Ubuntu (OpenJDK 64-Bit Server VM)
---
analysis-model-api:11.6.0
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
authentication-tokens:1.53.v1c90fd9191a_b_
bootstrap5-api:5.3.0-1
bouncycastle-api:2.29
branch-api:2.1122.v09cb_8ea_8a_724
build-name-setter:2.3.0
build-timeout:1.31
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.0
cloudbees-folder:6.848.ve3b_fd7839a_81
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.10.0-68.v0d0b_c439292b_
conditional-buildstep:1.4.3
config-file-provider:953.v0432a_802e4d2
credentials:1271.v54b_1c2c6388a_
credentials-binding:631.v861c06d062b_4
dashboard-view:2.495.v07e81500c3f2
data-tables-api:1.13.5-1
display-url-api:2.3.8
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:513.vc48a_a_075a_d93
echarts-api:5.4.0-5
email-ext:2.100
embeddable-build-status:412.v09da_db_1dee68
font-awesome-api:6.4.0-2
forensics-api:2.3.0
git-client:4.4.0
gradle:2.8.2
htmlpublisher:1.32
instance-identity:173.va_37c494ec4e5
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.15.2-350.v0c2f3f8fc595
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:73.vddf737284550
jquery3-api:3.7.0-1
junit:1217.v4297208a_a_b_ce
kubernetes:3995.v227c16b_675ee
kubernetes-client-api:6.4.1-215.v2ed17097a_8e9
kubernetes-credentials:0.10.0
kubernetes-pipeline-devops-steps:1.6
mailer:463.vedf8358e006b_
matrix-auth:3.1.10
matrix-project:808.v5a_b_5f56d6966
metrics:4.2.18-442.v02e107157925
mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_
mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
p4:1.14.2
pam-auth:1.10
parameterized-trigger:2.46
pipeline-build-step:505.v5f0844d8d126
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:671.v07c339c842e8
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2144.v077a_d1928a_40
pipeline-model-definition:2.2144.v077a_d1928a_40
pipeline-model-extensions:2.2144.v077a_d1928a_40
pipeline-rest-api:2.33
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40
pipeline-stage-view:2.33
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.3.0
prism-api:1.29.0-7
resource-disposer:0.23
role-strategy:680.v3a_6a_1698b_864
run-condition:1.6
saml:4.429.v9a_781a_61f1da_
scm-api:676.v886669a_199a_a_
script-security:1269.v639888f5e366
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
ssh-agent:333.v878b_53c89511
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.916.vd17b_43357ce4
sshd:3.312.v1c601b_c83b_0e
structs:324.va_f5d6774f3a_d
thinBackup:1.18
throttle-concurrents:2.14
timestamper:1.26
token-macro:384.vf35b_f26814ec
trilead-api:2.84.v72119de229b_7
variant:59.vf075fe829ccb
warnings-ng:10.4.0
workflow-aggregator:596.v8c21c963d92d
workflow-api:1259.vb_47f14fffc8a_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3744.v6f2c0fe0e54d
workflow-durable-task-step:1284.v4fcd365b_75b_e
workflow-job:1326.ve643e00e9220
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:848.v5a_383b_d14921
ws-cleanup:0.45
Jenkins

1条答案

按热度按时间
q35jwt9p

q35jwt9p1#

您能否分享有关如何安装Jenkins主示例的更多详细信息？从你的日志看起来像豆荚失败的主要原因是这个错误消息

io.fabric8.kubernetes.client.KubernetesClientException: **Failure executing: GET at: https://10.65.184.52:6443/api/v1/namespaces/jenkinsns/pods?fieldSelector=metadata.name%3Dsample-job-2-9-x79k3-vlvp9-g134p&resourceVersion=759022&allowWatchBookmarks=true&watch=true. Message: Forbidden.**

在我看来，这与你的Jenkins主程序拥有的RBAC权限有关。您可以使用此角色和以下角色绑定作为参考来正确设置它。
目前看来，您错过了podsGET权限。

赞(0）分享  回复(0）举报  2023-10-17
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com