kubernetes LoadBalancer Controller在AWS EKS中无法正常工作

vx6bjr1n  于 12个月前  发布在  Kubernetes
关注(0)|答案(1)|浏览(141)

我用eksctl创建了eks集群，并部署了应用程序，服务和入口，但没有创建负载均衡器：（
kubectl版本1.23.6
Eksctl版本：0.165.0
对于负载均衡器控制器安装，我遵循本指南https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/deploy/installation/
描述pod

kubectl describe pod  techcare-apigw-65f496dd85-f8s28 -n techcare                                                                                                        ✔  18.10.0 Node  18.10.0 Node  [email protected] ○  10:36:20  
Name:         techcare-apigw-65f496dd85-f8s28
Namespace:    techcare
Priority:     0
Node:         ip-192-168-17-235.eu-central-1.compute.internal/192.168.17.235
Start Time:   Fri, 08 Dec 2023 22:09:36 +0100
Labels:       app=techcare-apigw
              pod-template-hash=65f496dd85
Annotations:  <none>
Status:       Running
IP:           192.168.3.4
IPs:
  IP:           192.168.3.4
Controlled By:  ReplicaSet/techcare-apigw-65f496dd85
Containers:
  techcare-apigw:
    Container ID:   containerd://cbf1050a4aca263fbc86c97c759dda48694dc637fe1da40dbf0430319d5da5ef
    Image:          435109652403.dkr.ecr.eu-central-1.amazonaws.com/api-gateway:1.0.57
    Image ID:       435109652403.dkr.ecr.eu-central-1.amazonaws.com/api-gateway@sha256:e362e6e1bf42002b510dff2a621e85e7f1fc304a1fef5c02095919129b0a0f45
    Port:           3000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 08 Dec 2023 22:10:12 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-q92s4 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      Tre 
Volumes:
  kube-api-access-q92s4:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  27m   default-scheduler  Successfully assigned techcare/techcare-apigw-65f496dd85-f8s28 to ip-192-168-17-235.eu-central-1.compute.internal
  Normal  Pulling    27m   kubelet            Pulling image "435109652403.dkr.ecr.eu-central-1.amazonaws.com/api-gateway:1.0.57"
  Normal  Pulled     26m   kubelet            Successfully pulled image "435109652403.dkr.ecr.eu-central-1.amazonaws.com/api-gateway:1.0.57" in 34.870208122s (34.870944373s including waiting)
  Normal  Created    26m   kubelet            Created container techcare-apigw
  Normal  Started    26m   kubelet            Started container techcare-apigw

字符串
描述入口

kubectl describe ing example-ingress -n techcare                                                                                                                         ✔  18.10.0 Node  18.10.0 Node  [email protected] ○  10:36:37  
Name:             example-ingress
Labels:           <none>
Namespace:        techcare
Address:          
Default backend:  techcare-apigw-s:80 (192.168.3.4:3000)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           
              /   techcare-apigw-s:80 (192.168.3.4:3000)
Annotations:  alb.ingress.kubernetes.io/scheme: internet-facing
              alb.ingress.kubernetes.io/target-type: alb
Events:       <none>

型
描述服务：

kubectl describe service techcare-apigw-s  -n techcare                                                                                                                   ✔  18.10.0 Node  18.10.0 Node  [email protected] ○  10:35:56  
Name:              techcare-apigw-s
Namespace:         techcare
Labels:            <none>
Annotations:       <none>
Selector:          app=techcare-apigw
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.100.50.10
IPs:               10.100.50.10
Port:              http  80/TCP
TargetPort:        3000/TCP
Endpoints:         192.168.3.4:3000
Port:              https  443/TCP
TargetPort:        3000/TCP
Endpoints:         192.168.3.4:3000
Session Affinity:  None
Events:            <none>

型
清单部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: techcare-apigw
  namespace: techcare
spec:
  replicas: 1
  selector:
    matchLabels:
      app: techcare-apigw
  template:
    metadata:
      labels:
        app: techcare-apigw
    spec:
      containers:
        - name: techcare-apigw
          image: 435109652403.dkr.ecr.eu-central-1.amazonaws.com/api-gateway:1.0.57
          ports:
            - containerPort: 3000
              name: techcare-apigw

型
售后服务：

apiVersion: v1
kind: Service
metadata:
  name: techcare-apigw-s
  namespace: techcare
spec:
  selector:
    app: techcare-apigw
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000
      name: http
    - protocol: TCP
      port: 443
      targetPort: 3000
      name: https
  type: ClusterIP

型
入口：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: techcare
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: alb
spec:
  defaultBackend:
    service:
      name: techcare-apigw-s
      port:
        number: 80
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: techcare-apigw-s
                port:
                  number: 80

型
pod：

kubectl get po -A                                                                                                                                                    127 ✘  18.10.0 Node  18.10.0 Node  [email protected] ○  10:46:37  
NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE
kube-system   aws-load-balancer-controller-75776fdbcf-wrl5l   1/1     Running   0          41m
kube-system   aws-load-balancer-controller-75776fdbcf-xms7t   1/1     Running   0          41m
kube-system   aws-node-6n9ph                                  1/1     Running   0          63m
kube-system   aws-node-g4fnw                                  1/1     Running   0          63m
kube-system   aws-node-h8j7n                                  1/1     Running   0          63m
kube-system   aws-node-vfchz                                  1/1     Running   0          63m
kube-system   coredns-7bc655f56f-8mr27                        1/1     Running   0          73m
kube-system   coredns-7bc655f56f-jqtlx                        1/1     Running   0          73m
kube-system   kube-proxy-5bw4p                                1/1     Running   0          63m
kube-system   kube-proxy-925qt                                1/1     Running   0          63m
kube-system   kube-proxy-h96kj                                1/1     Running   0          63m
kube-system   kube-proxy-n4x9f                                1/1     Running   0          63m
techcare      techcare-apigw-65f496dd85-f8s28                 1/1     Running   0          37m

型
负载平衡器控制器日志：

kubectl logs aws-load-balancer-controller-75776fdbcf-wrl5l -n kube-system                                                                                                ✔  18.10.0 Node  18.10.0 Node  [email protected] ○  10:46:48  

{"level":"info","ts":"2023-12-08T21:05:24Z","msg":"version","GitVersion":"v2.6.2","GitCommit":"5e07dced064969588164c60cd493ce7921e913da","BuildDate":"2023-10-27T19:20:16+0000"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.metrics","msg":"Metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"setup","msg":"adding health check for controller"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/mutate-v1-pod"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/mutate-v1-service"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-elbv2-k8s-aws-v1beta1-ingressclassparams"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/mutate-elbv2-k8s-aws-v1beta1-targetgroupbinding"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-elbv2-k8s-aws-v1beta1-targetgroupbinding"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-networking-v1-ingress"}
{"level":"info","ts":"2023-12-08T21:05:24Z","logger":"setup","msg":"starting podInfo repo"}
{"level":"info","ts":"2023-12-08T21:05:26Z","logger":"controller-runtime.webhook.webhooks","msg":"Starting webhook server"}
{"level":"info","ts":"2023-12-08T21:05:26Z","logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"}
{"level":"info","ts":"2023-12-08T21:05:26Z","logger":"controller-runtime.webhook","msg":"Serving webhook server","host":"","port":9443}
{"level":"info","ts":"2023-12-08T21:05:26Z","msg":"Starting server","path":"/metrics","kind":"metrics","addr":"[::]:8080"}
{"level":"info","ts":"2023-12-08T21:05:26Z","msg":"Starting server","kind":"health probe","addr":"[::]:61779"}
{"level":"info","ts":"2023-12-08T21:05:26Z","logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"}
I1208 21:05:26.950628       1 leaderelection.go:248] attempting to acquire leader lease kube-system/aws-load-balancer-controller-leader...
kubectl get ingress --all-namespaces                                                                                                                                     ✔  18.10.0 Node  18.10.0 Node  [email protected] ○  11:06:12  

NAMESPACE   NAME              CLASS    HOSTS   ADDRESS   PORTS   AGE
techcare    example-ingress   <none>   *                 80      54m


的数据

kubernetes

1条答案

按热度按时间
whhtz7ly

whhtz7ly1#

我建议你先把AWS EKS升级到1.27或1.28，记住控制器不能开箱即用，你需要给予权限，如果你使用AWS CDK，你可以在这里看到我的代码片段。
我强烈建议您使用AWS CDK将核心功能应用于您的AWS EKS，以使您的地面系统对您和您的团队更加透明。

应用LoadBalancer with helm：

def __install_load_balancer_controller(self, cluster: eks.Cluster):
    service_account = "aws-loadbalancer-controller-sa"

    aws_alb_sa = eks.ServiceAccount(
        self,
        "AWSLoadBalancerController",
        cluster=cluster,
        namespace="kube-system",
    )
    with open(
        os.path.join(
            "infrastructure/k8sStack/iam-policy/loadbalancer-controller/iam-policy.json"
        ),
        "r",
    ) as f:
        policy_document = json.load(f)
        for s in policy_document.get("Statement"):
            policy_statement = iam.PolicyStatement.from_json(s)
            aws_alb_sa.add_to_principal_policy(policy_statement)

    eks.HelmChart(
        self,
        "eks-aws-load-balancer-controller",
        cluster=cluster,
        chart="aws-load-balancer-controller",
        repository="https://aws.github.io/eks-charts",
        release="aws-load-balancer-controller",
        namespace="kube-system",
        version="1.6.1",
        values={
            "clusterName": cluster.cluster_name,
            "serviceAccount": {
                "create": False,
                "name": aws_alb_sa.service_account_name,
            },
        },
        wait=True,
    )

字符串

iam-policy.json：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeAddresses",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeVpcs",
                "ec2:DescribeVpcPeeringConnections",
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeInstances",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribeTags",
                "ec2:GetCoipPoolUsage",
                "ec2:DescribeCoipPools",
                "elasticloadbalancing:DescribeLoadBalancers",
                "elasticloadbalancing:DescribeLoadBalancerAttributes",
                "elasticloadbalancing:DescribeListeners",
                "elasticloadbalancing:DescribeListenerCertificates",
                "elasticloadbalancing:DescribeSSLPolicies",
                "elasticloadbalancing:DescribeRules",
                "elasticloadbalancing:DescribeTargetGroups",
                "elasticloadbalancing:DescribeTargetGroupAttributes",
                "elasticloadbalancing:DescribeTargetHealth",
                "elasticloadbalancing:DescribeTags"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cognito-idp:DescribeUserPoolClient",
                "acm:ListCertificates",
                "acm:DescribeCertificate",
                "iam:ListServerCertificates",
                "iam:GetServerCertificate",
                "waf-regional:GetWebACL",
                "waf-regional:GetWebACLForResource",
                "waf-regional:AssociateWebACL",
                "waf-regional:DisassociateWebACL",
                "wafv2:GetWebACL",
                "wafv2:GetWebACLForResource",
                "wafv2:AssociateWebACL",
                "wafv2:DisassociateWebACL",
                "shield:GetSubscriptionState",
                "shield:DescribeProtection",
                "shield:CreateProtection",
                "shield:DeleteProtection"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupIngress"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSecurityGroup"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": "arn:aws:ec2:*:*:security-group/*",
            "Condition": {
                "StringEquals": {
                    "ec2:CreateAction": "CreateSecurityGroup"
                },
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags",
                "ec2:DeleteTags"
            ],
            "Resource": "arn:aws:ec2:*:*:security-group/*",
            "Condition": {
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "true",
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:DeleteSecurityGroup"
            ],
            "Resource": "*",
            "Condition": {
                "Null": {
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:CreateLoadBalancer",
                "elasticloadbalancing:CreateTargetGroup"
            ],
            "Resource": "*",
            "Condition": {
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:CreateListener",
                "elasticloadbalancing:DeleteListener",
                "elasticloadbalancing:CreateRule",
                "elasticloadbalancing:DeleteRule"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:AddTags",
                "elasticloadbalancing:RemoveTags"
            ],
            "Resource": [
                "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
                "arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
                "arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*"
            ],
            "Condition": {
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "true",
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:AddTags",
                "elasticloadbalancing:RemoveTags"
            ],
            "Resource": [
                "arn:aws:elasticloadbalancing:*:*:listener/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:*:listener/app/*/*/*",
                "arn:aws:elasticloadbalancing:*:*:listener-rule/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:*:listener-rule/app/*/*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:ModifyLoadBalancerAttributes",
                "elasticloadbalancing:SetIpAddressType",
                "elasticloadbalancing:SetSecurityGroups",
                "elasticloadbalancing:SetSubnets",
                "elasticloadbalancing:DeleteLoadBalancer",
                "elasticloadbalancing:ModifyTargetGroup",
                "elasticloadbalancing:ModifyTargetGroupAttributes",
                "elasticloadbalancing:DeleteTargetGroup"
            ],
            "Resource": "*",
            "Condition": {
                "Null": {
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:AddTags"
            ],
            "Resource": [
                "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
                "arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
                "arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "elasticloadbalancing:CreateAction": [
                        "CreateTargetGroup",
                        "CreateLoadBalancer"
                    ]
                },
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:RegisterTargets",
                "elasticloadbalancing:DeregisterTargets"
            ],
            "Resource": "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:SetWebAcl",
                "elasticloadbalancing:ModifyListener",
                "elasticloadbalancing:AddListenerCertificates",
                "elasticloadbalancing:RemoveListenerCertificates",
                "elasticloadbalancing:ModifyRule"
            ],
            "Resource": "*"
        }
    ]
}

型

赞(0）分享  回复(0）举报  12个月前
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com