centos kubernetes入口502网关错误

idv4meu8  于 2022-11-07  发布在  Kubernetes
关注(0)|答案(2)|浏览(164)

我在裸机上安装了Kubernetes Cluster(使用VMware虚拟机),其中包含以下节点

master-01   Ready    control-plane,master   5d3h   v1.21.3
master-02   Ready    control-plane,master   5d3h   v1.21.3
master-03   Ready    control-plane,master   5d3h   v1.21.3
worker-01   Ready    <none>                 5d2h   v1.21.3
worker-02   Ready    <none>                 5d2h   v1.21.3
worker-03   Ready    <none>                 5d2h   v1.21.3

Metallb安装为群集的负载平衡器,Calico安装为CNI
我还安装了nginx入口控制器与helm

$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update
$ helm install ingress-controller nginx-stable/nginx-ingress

我部署了一个简单的nginx服务器进行测试

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-app
  template:
    metadata:
      labels:
        app: nginx-app
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx-app
  #type: LoadBalancer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-myapp
  annotations:
    # use the shared ingress-nginx
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: myapp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80

我的负载平衡器类型的部署从metallb获得其IP,并且工作正常,但是当我添加入口时,尽管分配了IP,我还是收到错误502 bad gateway,如下所示:
防火墙已启用,但所需端口已打开

6443/tcp 2379-2380/tcp 10250-10252/tcp 179/tcp 7946/tcp 7946/udp 8443/tcp on master nodes
10250/tcp 30000-32767/tcp 7946/tcp 7946/udp 8443/tcp 179/tcp on worker nodes

我的服务和pod工作正常

kubectl get svc -o wide
NAME                               TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE   SELECTOR
ingress-controller-nginx-ingress   LoadBalancer   10.101.17.180   10.1.210.100   80:31509/TCP,443:30004/TCP   33m   app=ingress-controller-nginx-ingress
kubernetes                         ClusterIP      10.96.0.1       <none>         443/TCP                      6d    <none>
nginx-service                      ClusterIP      10.101.48.198   <none>         80/TCP                       31m   app=nginx-app

我的入口日志显示错误,没有到内部IP的路由

2021/07/29 07:46:24 [error] 42#42: *8 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:07:46:24 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
W0729 07:50:16.416830       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET / HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://10.1.210.100/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
W0729 07:56:43.420282       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:05:28.422594       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:10:45.425329       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
2021/07/29 08:13:59 [error] 42#42: *12 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:13:59 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
2021/07/29 08:14:09 [error] 42#42: *14 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:14:09 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"

有什么想法吗?
编辑:如这里所问,服务和pod的描述

$ kubectl describe pod nginx-deployment-6f7d8d4d55-sncdr
Name:         nginx-deployment-6f7d8d4d55-sncdr
Namespace:    default
Priority:     0
Node:         worker-01/10.1.210.63
Start Time:   Thu, 29 Jul 2021 08:43:59 +0100
Labels:       app=nginx-app
              pod-template-hash=6f7d8d4d55
Annotations:  cni.projectcalico.org/podIP: 192.168.171.17/32
              cni.projectcalico.org/podIPs: 192.168.171.17/32
Status:       Running
IP:           192.168.171.17
IPs:
  IP:           192.168.171.17
Controlled By:  ReplicaSet/nginx-deployment-6f7d8d4d55
Containers:
  nginx:
    Container ID:   docker://fc61b73f8a833ad13b8956d8ce151b221b75a58a9a2fbae928464f3b0a77cca2
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:8f335768880da6baf72b70c701002b45f4932acae8d574dedfddaf967fc3ac90
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 29 Jul 2021 08:44:01 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wkc48 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-wkc48:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  16m   default-scheduler  Successfully assigned default/nginx-deployment-6f7d8d4d55-sncdr to worker-01
  Normal  Pulling    16m   kubelet            Pulling image "nginx"
  Normal  Pulled     16m   kubelet            Successfully pulled image "nginx" in 1.51808376s
  Normal  Created    16m   kubelet            Created container nginx
  Normal  Started    16m   kubelet            Started container nginx

$ kubectl describe svc ingress-controller-nginx-ingress
Name:                     ingress-controller-nginx-ingress
Namespace:                default
Labels:                   app.kubernetes.io/instance=ingress-controller
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=ingress-controller-nginx-ingress
                          helm.sh/chart=nginx-ingress-0.10.0
Annotations:              meta.helm.sh/release-name: ingress-controller
                          meta.helm.sh/release-namespace: default
Selector:                 app=ingress-controller-nginx-ingress
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.101.17.180
IPs:                      10.101.17.180
LoadBalancer Ingress:     10.1.210.100
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  31509/TCP
Endpoints:                192.168.37.202:80
Port:                     https  443/TCP
TargetPort:               443/TCP
NodePort:                 https  30004/TCP
Endpoints:                192.168.37.202:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     31108
Events:
  Type    Reason        Age                    From                Message
  ----    ------        ----                   ----                -------
  Normal  IPAllocated   18m                    metallb-controller  Assigned IP "10.1.210.100"
  Normal  nodeAssigned  3m21s (x182 over 18m)  metallb-speaker     announcing from node "worker-02"

$ kubectl describe svc nginx-service
Name:              nginx-service
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=nginx-app
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.101.48.198
IPs:               10.101.48.198
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         192.168.171.17:80
Session Affinity:  None
Events:            <none>

$ kubectl exec -it ingress-controller-nginx-ingress-dd5db86dc-gqdpm -- /bin/bash
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17:80
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 10.101.48.198
curl: (7) Failed to connect to 10.101.48.198 port 80: Connection timed out
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl nginx-deployment-6f7d8d4d55-sncdr
curl: (6) Could not resolve host: nginx-deployment-6f7d8d4d55-sncdr
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$

老实说,我不明白为什么curl svcip不再起作用了;昨天还管用。

jaxagkaj

jaxagkaj1#

问题是一个防火墙问题我禁用了firewalld,它现在工作了,我以为必须打开端口8443,但它似乎是另一个端口,如果有人能告诉我是哪一个
谢谢你

h4cxqtbf

h4cxqtbf2#

我在k3s中的traefik入口也遇到了类似的问题。我在firewalldfirewall-cmd --permanent --add-masquerade && firewall-cmd --reload中启用了masquerade
这个想法归功于这篇文章:https://github.com/k3s-io/k3s/issues/1646#issuecomment-881191877

相关问题