如何减少Azure AKS IP地址占用？

升级到AKS v. 1.24（网络类型：Azure CNI），发现kube-system pod占用了大部分IP空间，留给应用部署的IP非常少。
例如，下面是在其中一个虚拟机上运行的kube-system pod：

NAMESPACE                NAME                                                   READY   STATUS    RESTARTS   AGE   IP             NODE                                NOMINATED NODE   READINESS GATES
kube-system              ama-logs-m5rrq                                         2/2     Running   0          25d   10.65.197.16   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              ama-logs-rs-c4f77bf75-cpvgl                            1/1     Running   0          25d   10.65.197.24   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              azure-ip-masq-agent-rcc7m                              1/1     Running   0          25d   10.65.197.15   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              azure-npm-lg7f6                                        1/1     Running   0          25d   10.65.197.15   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              cloud-node-manager-h7w25                               1/1     Running   0          25d   10.65.197.15   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              coredns-59b6bf8b4f-bp9nb                               1/1     Running   0          25d   10.65.197.17   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              csi-azurefile-node-bdpwc                               3/3     Running   0          8d    10.65.197.15   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              konnectivity-agent-85c86bd7ff-wsj5k                    1/1     Running   0          25d   10.65.197.23   aks-agentpool-45621317-vmss00000b   <none>           <none>
kube-system              kube-proxy-hzxhl                                       1/1     Running   0          8h    10.65.197.15   aks-agentpool-45621317-vmss00000b   <none>           <none>

由于我们部署到专用网络，这正在成为一个挑战，不断提供更大的网络，并不断增加每个IP的最大pod。
你知道怎么才能腾出一些空间吗？

如果使用Azure CNI插件创建AKS群集，则Kubernetes群集中的所有Pod都使用我们为AKS群集创建的虚拟网络的IP部分。

所有群集节点和pod都将成为群集子网（10.224.0.0/16）的一部分。因此，如果我们在AKS群集上托管1000个pod，它将从VNET的子网中选择1000个IP。请检查以下pod IP详细信息：

NAMESPACE  NAME  READY  STATUS  RESTARTS  AGE  IP  NODE  NOMINATED NODE  READINESS GATES
default  nginxd-67d79c7976-7zv7v  1/1  Running  0  4h19m  10.224.0.17  aks-agentpool-20514180-vmss000000  <none>  <none>
default  nginxd-67d79c7976-kfwwj  1/1  Running  0  4h19m  10.224.0.86  aks-agentpool-20514180-vmss000000  <none>  <none>
default  nginxd-67d79c7976-rt5j9  1/1  Running  0  4h19m  10.224.0.55  aks-agentpool-20514180-vmss000000  <none>  <none>
default  nginxd-67d79c7976-s8pms  1/1  Running  0  4h19m  10.224.0.108  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  azure-ip-masq-agent-sfkzv  1/1  Running  0  5h  10.224.0.4  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  cloud-node-manager-5td8p  1/1  Running  0  5h  10.224.0.4  aks-agentpool-20514180-vmss000000 <none>  <none>
kube-system  coredns-59b6bf8b4f-j7fwq  1/1  Running  0  5h1m  10.224.0.112  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  coredns-59b6bf8b4f-tcsc7  1/1  Running  0  5h  10.224.0.96  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  coredns-autoscaler-5655d66f64-8tddq  1/1  Running  0  5h1m  10.224.0.56  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  csi-azuredisk-node-p92rv  3/3  Running  0  5h  10.224.0.4  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  csi-azurefile-node-fn8pw  3/3  Running  0  5h  10.224.0.4  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  konnectivity-agent-66c54fd7cd-9trm7  1/1  Running  0  4h48m  10.224.0.102  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  konnectivity-agent-66c54fd7cd-c96sp  1/1  Running  0  4h48m  10.224.0.88  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  kube-proxy-dkxpp  1/1  Running  0  5h  10.224.0.4  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  metrics-server-7dd74d8758-j2lph  2/2  Running  0  5h  10.224.0.46  aks-agentpool-20514180-vmss000000  <none>  <none>
kube-system  metrics-server-7dd74d8758-rh266  2/2  Running  0  5h  10.224.0.104  aks-agentpool-20514180-vmss000000  <none>  <none>

所有服务都将从服务CDIR获取IP。请检查下面的服务IP：

NAMESPACE  NAME  TYPE  CLUSTER-IP  EXTERNAL-IP  PORT(S)  AGE
default  kubernetes  ClusterIP  10.0.0.1  <none>  443/TCP  5h6m
default  nginx-service  LoadBalancer  10.0.174.204  20.246.132.249  80:32229/TCP  4h20m
kube-system  kube-dns  ClusterIP  10.0.0.10  <none>  53/UDP,53/TCP  5h5m
kube-system  metrics-server  ClusterIP  10.0.5.36  <none>  443/TCP  5h5m

配置后，交叉检查AKS联网中的联网详细信息。
AKS集群〉网络：

如果您选择Azure-CNI，则必须相应地规划IP空间。
要防止pod使用真实的的IP地址，请使用kubenet插件或azure-cni-overlay网络插件。
azure-cni-overlay-plugin处于预览状态，并且在美国中西部和美国东部地区可用。
如果您选择选择kubenet插件，它将为我们创建一个默认值的网络，并将创建一个逻辑IP空间，以利用集群中的Pod。
这些IP可跨多个群集重用。

AKS成功配置后，用户可以在网络选项卡中查看网络详细信息。

Kubenet网络插件的AKS群集的默认值如下：

这里，当pod在集群中创建时，它会从Pod CIDR中选择IP。这是Azure-CNI和Kubenet插件之间的主要区别。
Pod CIDR是指IP的逻辑分组，它不属于我们在AKS群集期间创建的VNet。
检查下面分配给Pod的IP。它将与Pod CIDR范围匹配：

NAMESPACE  NAME  READY  STATUS  RESTARTS  AGE  IP  NODE  NOMINATED NODE  READINESS GATES
default  nginxd01-784cf56f68-db6wz  1/1  Running  0  4h26m  10.244.0.13  aks-agentpool-97773100-vmss000000  <none>  <none>
default  nginxd01-784cf56f68-fqtjn  1/1  Running  0  4h26m  10.244.0.12  aks-agentpool-97773100-vmss000000  <none>  <none>
default  nginxd01-784cf56f68-mj4w9  1/1  Running  0  4h26m  10.244.0.14  aks-agentpool-97773100-vmss000000  <none>  <none>
default  nginxd01-784cf56f68-vwx84  1/1  Running  0  4h26m  10.244.0.11  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  azure-ip-masq-agent-mgdbr  1/1  Running  0  4h51m  10.224.0.4  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  cloud-node-manager-7zg2b  1/1  Running  0  4h51m  10.224.0.4  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  coredns-59b6bf8b4f-hh8l5  1/1  Running  0  4h50m  10.244.0.8  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  coredns-59b6bf8b4f-rsmmb  1/1  Running  0  4h52m  10.244.0.5  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  coredns-autoscaler-5655d66f64-nb2kk  1/1  Running  0  4h52m  10.244.0.6  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  csi-azuredisk-node-lm4mf  3/3  Running  0  4h51m  10.224.0.4  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  csi-azurefile-node-p5mws  3/3  Running  0  4h51m  10.224.0.4  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  konnectivity-agent-5f4cf5c7dd-r85ht  1/1  Running  0  4h10m  10.244.0.15  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  konnectivity-agent-5f4cf5c7dd-vg6xn  1/1  Running  0  4h10m  10.244.0.16  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  kube-proxy-m8n8x  1/1  Running  0  4h51m  10.224.0.4  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  metrics-server-7dd74d8758-6lqgs  2/2  Running  0  4h50m  10.244.0.10  aks-agentpool-97773100-vmss000000  <none>  <none>
kube-system  metrics-server-7dd74d8758-s4sl9  2/2  Running  0  4h50m  10.244.0.9  aks-agentpool-97773100-vmss000000  <none>  <none>

确保Pod CIDR范围不能与网络中的任何其他IP重叠。Kubenet网络插件也使用路由表。
如果用户需要更灵活地控制IP地址空间，请选择Azure-cli-overlay。但它处于预览状态，不建议用于生产工作负载。
蓝色-CNI-覆盖图参考：azure-CNI-overlay network

如何减少Azure AKS IP地址占用？

1条答案

相关问题

热门标签

最新问答