Ansible kubernetes模块无法使用kubeconfig

jm81lzqq 于 2023-01-20 发布在 Kubernetes

关注(0)|答案(1)|浏览(143)

我正在编写一个Ansible剧本，它使用kubernetes模块来修改集群上的ConfigMap条目。需要注意的一个重要警告是，我正在运行一个包含Ansible安装的docker映像来完成这项工作。我运行docker映像，并将必要的输入交给它来完成它的工作。下面是run命令的一个示例：

$ docker run --rm -it -e ANSIBLE_CONFIG=/play-config/ansible.cfg -e K8S_AUTH_KUBECONFIG=/play-config/gagnon.config -e K8S_AUTH_CONTEXT=kubernetes-admin@kubernetes -v "C:\Users\jgagnon\gagnon-test\local-kube-prometheus-stack\ansible":/play-config cytopia/ansible:latest-tools

然后，在运行容器中：

$ ansible-playbook /play-config/playbook-arc-control-plane.yaml -u jgagnon

经过最初的一些障碍后，我发现需要在目标群集节点上安装一些缺少的依赖项，我相信我已经满足了依赖项要求。现在我遇到了一个问题，即当剧本尝试使用kubernetes.core.k8s_json_patch更改ConfigMap时失败。我尝试了许多方法来查看是否可以纠正该问题，但都无济于事。我一直收到错误：

"msg": "Failed to load kubeconfig due to Invalid kube-config file. No configuration found."

以下是剧本（来自剧本-弧-控制-平面. yaml）：

- name: "Make kube-proxy visible to Prometheus"
  hosts: control_planes

  become_user: root
  become: true

  tasks:
    - name: "Install pip"
      shell:
        cmd: "apt-get install -y python3-pip"

    - name: "Install jsonpatch"
      shell:
        cmd: "apt-get install -y python3-jsonpatch"

    - name: "Install kubernetes Ansible module"
      pip:
        name:
          kubernetes

    - debug:
        var: lookup('env', 'K8S_AUTH_KUBECONFIG')
    - debug:
        var: lookup('env', 'K8S_AUTH_CONTEXT')

    - name: "Patch kube-proxy ConfigMap metricsBindAddress"
      kubernetes.core.k8s_json_patch:
        kind: ConfigMap
        name: kube-proxy
        namespace: kube-system
        context: "{{ lookup('env', 'K8S_AUTH_CONTEXT') }}"
        kubeconfig: "{{ lookup('env', 'K8S_AUTH_KUBECONFIG') }}"
        patch:
          - op: replace
            path: /data/config.conf/metricsBindAddress
            value: 0.0.0.0

以下是行动手册控制台输出的一部分（指定了-vvv标记）：

TASK [debug] ***********************************************************************************************************************************
task path: /play-config/playbook-arc-control-plane.yaml:180
ok: [gagnon-m1] => {
    "lookup('env', 'K8S_AUTH_KUBECONFIG')": "/play-config/gagnon.config"
}

TASK [debug] ***********************************************************************************************************************************
task path: /play-config/playbook-arc-control-plane.yaml:182
ok: [gagnon-m1] => {
    "lookup('env', 'K8S_AUTH_CONTEXT')": "kubernetes-admin@kubernetes"
}

TASK [Patch kube-proxy ConfigMap metricsBindAddress] *******************************************************************************************
task path: /play-config/playbook-arc-control-plane.yaml:185
...
The full traceback is:
  File "/tmp/ansible_kubernetes.core.k8s_json_patch_payload_aqz5jjfp/ansible_kubernetes.core.k8s_json_patch_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 256, in get_api_client
    _load_config()
  File "/tmp/ansible_kubernetes.core.k8s_json_patch_payload_aqz5jjfp/ansible_kubernetes.core.k8s_json_patch_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 218, in _load_config
    kubernetes.config.load_kube_config(
  File "/usr/local/lib/python3.8/dist-packages/kubernetes/config/kube_config.py", line 813, in load_kube_config
    loader = _get_kube_config_loader(
  File "/usr/local/lib/python3.8/dist-packages/kubernetes/config/kube_config.py", line 770, in _get_kube_config_loader
    raise ConfigException(
fatal: [gagnon-m1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": "kubernetes-admin@kubernetes",
            "host": null,
            "impersonate_groups": null,
            "impersonate_user": null,
            "kind": "ConfigMap",
            "kubeconfig": "/play-config/gagnon.config",
            "name": "kube-proxy",
            "namespace": "kube-system",
            "no_proxy": null,
            "password": null,
            "patch": [
                {
                    "op": "replace",
                    "path": "/data/config.conf/metricsBindAddress",
                    "value": "0.0.0.0"
                }
            ],
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "username": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to load kubeconfig due to Invalid kube-config file. No configuration found."
}

我已经验证了引用的kubeconfig文件（/play-config/gagnon.config）存在于容器中，而且，我已经使用这个配置文件几个月了，没有任何问题，所以我非常确定它是有效的。
有人知道问题出在哪里吗？我假设我已经满足了所有的依赖项，否则我根本不会期望这个任务运行（或者至少因为不同的原因而失败）。

- 更新**：

我怀疑（但无法验证）问题源于kubernetes.core.k8s_json_patch命令中指定的路径不正确。
如果将ConfigMap转储为JSON，则数据不会表示为JSON，而只是一个字符串。
例如：

{
    "apiVersion": "v1",
    "data": {
        "config.conf": "apiVersion: kubeproxy.config.k8s.io/v1alpha1\nbindAddress: 0.0.0.0\nbindAddressHardFail: false\nclientConnection:\n  acceptContentTypes: \"\"\n  burst: 0\n  contentType: \"\"\n  kubeconfig: /var/lib/kube-proxy/kubeconfig.conf\n  qps: 0\nclusterCIDR: \"\"\nconfigSyncPeriod: 0s\nconntrack:\n  maxPerCore: null\n  min: null\n  tcpCloseWaitTimeout: null\n  tcpEstablishedTimeout: null\ndetectLocal:\n  bridgeInterface: \"\"\n  interfaceNamePrefix: \"\"\ndetectLocalMode: \"\"\nenableProfiling: false\nhealthzBindAddress: \"\"\nhostnameOverride: \"\"\niptables:\n  masqueradeAll: false\n  masqueradeBit: null\n  minSyncPeriod: 0s\n  syncPeriod: 0s\nipvs:\n  excludeCIDRs: null\n  minSyncPeriod: 0s\n  scheduler: \"\"\n  strictARP: false\n  syncPeriod: 0s\n  tcpFinTimeout: 0s\n  tcpTimeout: 0s\n  udpTimeout: 0s\nkind: KubeProxyConfiguration\nmetricsBindAddress: 0.0.0.0\nmode: \"\"\nnodePortAddresses: null\noomScoreAdj: null\nportRange: \"\"\nshowHiddenMetricsForVersion: \"\"\nudpIdleTimeout: 0s\nwinkernel:\n  enableDSR: false\n  forwardHealthCheckVip: false\n  networkName: \"\"\n  rootHnsEndpointName: \"\"\n  sourceVip: \"\"",
        "kubeconfig.conf": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n    certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt\n    server: https://172.20.8.68:6443\n  name: default\ncontexts:\n- context:\n    cluster: default\n    namespace: default\n    user: default\n  name: default\ncurrent-context: default\nusers:\n- name: default\n  user:\n    tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token"
    },
    "kind": "ConfigMap",
    "metadata": {
        "annotations": {
            "kubeadm.kubernetes.io/component-config.hash": "sha256:aa87680dfe2321f98df103555d18d439916b19e0bf23bd0f98bb3e27c5adfc08"
        },
        "creationTimestamp": "2022-08-22T12:08:21Z",
        "labels": {
            "app": "kube-proxy"
        },
        "name": "kube-proxy",
        "namespace": "kube-system",
        "resourceVersion": "21706920",
        "uid": "97594de0-5aaa-4ea0-bd8c-a2f5fb357be7"
    }
}

我正在尝试修改ConfigMap数据中config.conf项中包含的metricsBindAddress字段的值。上面提供的示例将路径指定为/data/config.conf/metricsBindAddress。我认为这就是发生故障的原因。
为了验证我的理论，我将路径改为/data[config.conf]metricsBindAddress。我不知道会发生什么，但令我惊讶的是，它没有抛出错误。然而，它也没有改变感兴趣的领域。尽管如此，还是取得了进展。
我不知道在Ansible kubernetes模块的上下文中指定路径以获取所需内容的正确方法。

kubernetes

来源：https://stackoverflow.com/questions/75110843/ansible-kubernetes-module-fails-to-use-kubeconfig

1条答案

按热度按时间

vojdkbi01#

借助Carlos Monroy Nieblas的有益见解，我能够弄清楚需要采取哪些不同的做法。docker run命令中指定的K8S_AUTH_KUBECONFIG环境变量需要指向位于kubernetes节点上的kubeconfig文件，而不是Ansible节点。
我还通过另一个来源发现我需要修改我获取和修改ConfigMap条目的方法。ConfigMap条目中的“数据”是一个字符串，需要作为一个整体拉取、修改并重新插入到Map中。
Docker运行命令现在为：

docker run --rm -it -e ANSIBLE_CONFIG=/play-config/ansible.cfg -e K8S_AUTH_KUBECONFIG=/home/jgagnon/.kube/config -v "C:\Users\jgagnon\gagnon-test\local-kube-prometheus-stack\ansible":/play-config cytopia/ansible:latest-tools ansible-playbook /play-config/playbook-arc-control-plane.yaml -u jgagnon

现在的剧本是：

- name: "Make kube-proxy visible to Prometheus"
  hosts: control_planes
  # hosts: arc-master-1

  become_user: root
  become: true

  tasks:
    - name: "Install pip"
      shell:
        cmd: "apt-get install -y python3-pip"

    - name: "Install jsonpatch"
      shell:
        cmd: "apt-get install -y python3-jsonpatch"

    - name: "Install kubernetes Ansible module"
      pip:
        name:
          kubernetes

    - name: "Get kube-proxy ConfigMap"
      kubernetes.core.k8s_info:
        kind: ConfigMap
        name: kube-proxy
        namespace: kube-system
        kubeconfig: "{{ lookup('env', 'K8S_AUTH_KUBECONFIG') }}"
      register: kube_proxy_data

    - name: "Patch kube-proxy ConfigMap metricsBindAddress"
      ansible.builtin.set_fact:
        new_data: "{{ kube_proxy_data.resources[0].data['config.conf'] |
                      from_yaml | combine(patch) }}"
      vars:
        patch:
          "metricsBindAddress": "0.0.0.0"

    - name: "Replace kube-proxy config.conf data"
      kubernetes.core.k8s_json_patch:
        kind: ConfigMap
        name: kube-proxy
        namespace: kube-system
        kubeconfig: "{{ lookup('env', 'K8S_AUTH_KUBECONFIG') }}"
        patch:
          - op: replace
            path: /data/config.conf
            value: "{{ new_data | to_nice_yaml(indent=2, width=120) }}"

赞(0）回复(0）举报 2023-01-20

我来回答

Ansible kubernetes模块无法使用kubeconfig

1条答案

相关问题

热门标签

最新问答