发生了什么?
我有两个控制器:
- 一个对资源进行版本为 "v1" 的 SSA(Static Set Action)操作
- 一个对资源
/status
进行版本为 v1beta1 的 PUT 操作
每次 SSA 操作执行全量写入时,都会递增 resourceVersion,导致无限循环,因为它们不断应用。
如果两者使用相同的版本,就不会发生这种情况。
你期望会发生什么?
SSA 调用(不修改任何内容)是一个 NOP(无操作)。
我们如何尽可能最小化地重现它?
抱歉,这不是一个 最小 的复现器,但应该很容易做到:spec.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: hyper
namespace: default
annotations:
networking.istio.io/service-type: "ClusterIP"
ownerReferences:
- apiVersion: networking.example.io/v1
kind: SuperService
name: hyper
uid: "c5725128-104a-4c64-968b-4565a090d520"
spec:
gatewayClassName: istio
infrastructure:
annotations:
proxy.istio.io/config: |-
proxyMetadata:
ENABLE_HBONE: "false"
listeners:
- name: "http"
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: Same
- name: "hyper-https"
protocol: HTTPS
allowedRoutes:
namespaces:
from: Same
port: 443
tls:
mode: Terminate
certificateRefs:
- name: hyper-tls
group: ""
kind: Secret
status.yaml
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
annotations:
gateway.istio.io/controller-version: '5'
networking.istio.io/service-type: ClusterIP
creationTimestamp: null
name: hyper
namespace: default
ownerReferences:
- apiVersion: networking.example.io/v1
kind: SuperService
name: hyper
uid: c5725128-104a-4c64-968b-4565a090d520
resourceVersion: '384735'
uid: 5dc737e6-2a53-4c48-9c66-555f8c534dd9
spec:
gatewayClassName: ''
listeners: null
status:
addresses:
- type: Hostname
value: hyper-istio.default.svc.cluster.local
- type: IPAddress
value: 10.96.212.50
conditions:
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: Resource accepted
observedGeneration: 1
reason: Accepted
status: 'True'
type: Accepted
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: Resource programmed, assigned to service(s) hyper-istio.default.svc.cluster.local:443
and hyper-istio.default.svc.cluster.local:80
observedGeneration: 1
reason: Programmed
status: 'True'
type: Programmed
listeners:
- attachedRoutes: 1
conditions:
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: No errors found
observedGeneration: 1
reason: Accepted
status: 'True'
type: Accepted
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: No errors found
observedGeneration: 1
reason: NoConflicts
status: 'False'
type: Conflicted
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: No errors found
observedGeneration: 1
reason: Programmed
status: 'True'
type: Programmed
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: No errors found
observedGeneration: 1
reason: ResolvedRefs
status: 'True'
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 1
conditions:
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: No errors found
observedGeneration: 1
reason: Accepted
status: 'True'
type: Accepted
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: No errors found
observedGeneration: 1
reason: NoConflicts
status: 'False'
type: Conflicted
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: Bad TLS configuration
observedGeneration: 1
reason: Invalid
status: 'False'
type: Programmed
- lastTransitionTime: '2023-11-30T21:18:10Z'
message: invalid certificate reference /Secret/hyper-tls., secret default/hyper-tls
not found
observedGeneration: 1
reason: InvalidCertificateRef
status: 'False'
type: ResolvedRefs
name: hyper-https
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
然后,在之间交替执行以下操作:
kubectl apply --server-side spec.yaml
curl /apis/gateway.networking.k8s.io/v1beta1/namespaces/default/gateways/hyper/status -X PUT -d @status.yaml -H "Content-type: application/yaml"
(在 status.yaml 中更新 resourceVersion)。
每次你都会看到对象发生变化。下面是在应用前后的对比:
应用后
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
annotations:
gateway.istio.io/controller-version: "5"
networking.istio.io/service-type: ClusterIP
creationTimestamp: "2023-11-30T21:18:10Z"
generation: 1
managedFields:
- apiVersion: gateway.networking.k8s.io/v1beta1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:gateway.istio.io/controller-version: {}
manager: istio.io/gateway-controller
operation: Apply
time: "2023-11-30T21:18:10Z"
- apiVersion: gateway.networking.k8s.io/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:networking.istio.io/service-type: {}
f:ownerReferences:
k:{"uid":"c5725128-104a-4c64-968b-4565a090d520"}: {}
f:spec:
f:gatewayClassName: {}
f:infrastructure:
f:annotations:
f:proxy.istio.io/config: {}
f:listeners:
k:{"name":"http"}:
.: {}
f:allowedRoutes:
f:namespaces:
f:from: {}
f:name: {}
f:port: {}
f:protocol: {}
k:{"name":"hyper-https"}:
.: {}
f:allowedRoutes:
f:namespaces:
f:from: {}
f:name: {}
f:port: {}
f:protocol: {}
f:tls:
f:certificateRefs: {}
f:mode: {}
manager: kubectl
operation: Apply
time: "2023-11-30T21:18:10Z"
- apiVersion: gateway.networking.k8s.io/v1beta1
fieldsType: FieldsV1
fieldsV1:
f:status:
f:addresses: {}
f:conditions:
k:{"type":"Accepted"}:
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
k:{"type":"Programmed"}:
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:listeners:
.: {}
k:{"name":"http"}:
.: {}
f:attachedRoutes: {}
f:conditions:
.: {}
k:{"type":"Accepted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Conflicted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Programmed"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"ResolvedRefs"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
f:name: {}
f:supportedKinds: {}
k:{"name":"hyper-https"}:
.: {}
f:attachedRoutes: {}
f:conditions:
.: {}
k:{"type":"Accepted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Conflicted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Programmed"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"ResolvedRefs"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
f:name: {}
f:supportedKinds: {}
manager: pilot-discovery
operation: Update
subresource: status
time: "2023-11-30T21:18:10Z"
name: hyper
namespace: default
ownerReferences:
- apiVersion: networking.example.io/v1
kind: SuperService
name: hyper
uid: c5725128-104a-4c64-968b-4565a090d520
resourceVersion: "384735"
uid: 5dc737e6-2a53-4c48-9c66-555f8c534dd9
spec:
gatewayClassName: istio
infrastructure:
annotations:
proxy.istio.io/config: |-
proxyMetadata:
ENABLE_HBONE: "false"
listeners:
- allowedRoutes:
namespaces:
from: Same
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
name: hyper-https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: hyper-tls
mode: Terminate
status:
addresses:
- type: Hostname
value: hyper-istio.default.svc.cluster.local
- type: IPAddress
value: 10.96.212.50
conditions:
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: Resource accepted
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: Resource programmed, assigned to service(s) hyper-istio.default.svc.cluster.local:443
and hyper-istio.default.svc.cluster.local:80
observedGeneration: 1
reason: Programmed
status: "True"
type: Programmed
listeners:
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: NoConflicts
status: "False"
type: Conflicted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: NoConflicts
status: "False"
type: Conflicted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: Bad TLS configuration
observedGeneration: 1
reason: Invalid
status: "False"
type: Programmed
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: invalid certificate reference /Secret/hyper-tls., secret default/hyper-tls
not found
observedGeneration: 1
reason: InvalidCertificateRef
status: "False"
type: ResolvedRefs
name: hyper-https
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
状态更新后
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
annotations:
gateway.istio.io/controller-version: "5"
networking.istio.io/service-type: ClusterIP
creationTimestamp: "2023-11-30T21:18:10Z"
generation: 1
managedFields:
- apiVersion: gateway.networking.k8s.io/v1beta1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:gateway.istio.io/controller-version: {}
manager: istio.io/gateway-controller
operation: Apply
time: "2023-11-30T21:18:10Z"
- apiVersion: gateway.networking.k8s.io/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:networking.istio.io/service-type: {}
f:ownerReferences:
k:{"uid":"c5725128-104a-4c64-968b-4565a090d520"}: {}
manager: kubectl
operation: Apply
time: "2023-11-30T21:18:10Z"
- apiVersion: gateway.networking.k8s.io/v1beta1
fieldsType: FieldsV1
fieldsV1:
f:status:
f:addresses: {}
f:conditions:
k:{"type":"Accepted"}:
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
k:{"type":"Programmed"}:
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:listeners:
.: {}
k:{"name":"http"}:
.: {}
f:attachedRoutes: {}
f:conditions:
.: {}
k:{"type":"Accepted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Conflicted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Programmed"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"ResolvedRefs"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
f:name: {}
f:supportedKinds: {}
k:{"name":"hyper-https"}:
.: {}
f:attachedRoutes: {}
f:conditions:
.: {}
k:{"type":"Accepted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Conflicted"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Programmed"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"ResolvedRefs"}:
.: {}
f:lastTransitionTime: {}
f:message: {}
f:observedGeneration: {}
f:reason: {}
f:status: {}
f:type: {}
f:name: {}
f:supportedKinds: {}
manager: pilot-discovery
operation: Update
subresource: status
time: "2023-11-30T21:18:10Z"
name: hyper
namespace: default
ownerReferences:
- apiVersion: networking.example.io/v1
kind: SuperService
name: hyper
uid: c5725128-104a-4c64-968b-4565a090d520
resourceVersion: "384769"
uid: 5dc737e6-2a53-4c48-9c66-555f8c534dd9
spec:
gatewayClassName: istio
infrastructure:
annotations:
proxy.istio.io/config: |-
proxyMetadata:
ENABLE_HBONE: "false"
listeners:
- allowedRoutes:
namespaces:
from: Same
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
name: hyper-https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: hyper-tls
mode: Terminate
status:
addresses:
- type: Hostname
value: hyper-istio.default.svc.cluster.local
- type: IPAddress
value: 10.96.212.50
conditions:
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: Resource accepted
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: Resource programmed, assigned to service(s) hyper-istio.default.svc.cluster.local:443
and hyper-istio.default.svc.cluster.local:80
observedGeneration: 1
reason: Programmed
status: "True"
type: Programmed
listeners:
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: NoConflicts
status: "False"
type: Conflicted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: No errors found
observedGeneration: 1
reason: NoConflicts
status: "False"
type: Conflicted
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: Bad TLS configuration
observedGeneration: 1
reason: Invalid
status: "False"
type: Programmed
- lastTransitionTime: "2023-11-30T21:18:10Z"
message: invalid certificate reference /Secret/hyper-tls., secret default/hyper-tls
not found
observedGeneration: 1
reason: InvalidCertificateRef
status: "False"
type: ResolvedRefs
name: hyper-https
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
差异:
27,51d26
< f:spec:
< f:gatewayClassName: {}
< f:infrastructure:
< f:annotations:
< f:proxy.istio.io/config: {}
< f:listeners:
< k:{"name":"http"}:
< .: {}
< f:allowedRoutes:
< f:namespaces:
< f:from: {}
< f:name: {}
< f:port: {}
< f:protocol: {}
< k:{"name":"hyper-https"}:
< .: {}
< f:allowedRoutes:
< f:namespaces:
< f:from: {}
< f:name: {}
< f:port: {}
< f:protocol: {}
< f:tls:
< f:certificateRefs: {}
< f:mode: {}
164c139
< resourceVersion: "384735"
---
> resourceVersion: "384769"
基本上,当我们进行状态更新时,spec
被移除。
我们需要了解其他信息吗?
- 无响应*
Kubernetes 版本:
$ kubectl version
v1.28.0
云提供商:kind
操作系统版本:
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here
安装工具:
容器运行时(CRI)和版本(如适用):
相关插件(CNI、CSI 等)和版本(如适用):
6条答案
按热度按时间w9apscun1#
/sig api-machinery
cc @apelisse
kq0g1dla2#
你能复现一个尽可能简单的示例吗?这样我就能看懂发生了什么。
xqnpmsa83#
/triage accepted
gzszwxb44#
你能重现一个尽可能简单的示例吗?这样我可以看到发生了什么?
将它放入我的队列,希望几天内能处理完。
ee7vknir5#
编辑:我认为这是错误的...请稍等,我会进一步探索
好的,这是一个更小的复制品。首先
kubectl apply -k "https://github.com/kubernetes-sigs/gateway-api/config/crd?ref=v1.0.0"
然后设置3个文件
spec.yaml
status.json
现在运行
然后观察
vbopmzt16#
好的,我不确定我在上面说了什么,我甚至没有在我的复现器中使用服务器端应用。忽略我之前的评论 - 现在我认为我更好地理解了这个问题。
spec.yaml
status.json
复现器:
使用 force=true 以获得原始行为("资源版本不断更新")。force=false 使问题更加明显,尽管如此:
所以即使我们正在写入
/status
子资源,我们最终还是弄乱了spec
的东西。这似乎是可能的意外情况?这种情况只发生在跨版本调用时。如果你将 curl 调用和 status.json 更改为
v1
,一切都可以正常工作。