Waiting for http-01 challenge propagation: failed to perform self check GET request
,它类似于这个bug https://github.com/jetstack/cert-manager/issues/656,但GitHub ticket评论中的所有解决方案都没有帮助。
我正在尝试在DigitalOcean上设置CertManager
,如本教程所述:https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes我没有收到任何错误,但来自CertManager
的请求在挂起状态等待超过40小时。
我已经成功地用Nginx配置了Ingress,然后我创建了一个命名空间并创建了CertManager
CRD:
$ kubectl create namespace cert-manager
$ kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.12.0/cert-manager.yaml
我可以看到所有CertManager
Pod,如预期:
$ kubectl get pods --namespace cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-5c47f46f57-gxhwv 1/1 Running 0 42h
cert-manager-cainjector-6659d6844d-xp75s 1/1 Running 0 42h
cert-manager-webhook-547567b88f-k4dv2 1/1 Running 0 42h
然后我创建了staging issuer:
---
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: cert-manager
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: some@email.here
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
并更新了Ingress配置:
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: echo-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
# cert-manager.io/cluster-issuer: "letsencrypt-prod"
cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
tls:
- hosts:
- echo.some.domain
secretName: ingress-tls
rules:
- host: echo.some.domain
http:
paths:
- backend:
serviceName: echo1
servicePort: 80
但之后,CertManager
没有更新证书,并在InProgress
状态下等待:
$ date
Wed 18 Dec 2019 01:58:08 PM MSK
$ kubectl describe cert
...
Status:
Conditions:
Last Transition Time: 2019-12-16T17:23:56Z
Message: Waiting for CertificateRequest "ingress-tls-1089568541" to complete
Reason: InProgress
Status: False
Type: Ready
Events: <none>
它没有将Fake LE Intermediate X1
用作CN
,而是返回CN=Kubernetes Ingress Controller Fake Certificate,O=Acme Co
$ kubectl describe CertificateRequest
Status:
Conditions:
Last Transition Time: 2019-12-16T17:50:05Z
Message: Waiting on certificate issuance from order default/ingress-tls-1089568541-1576201144: "pending"
Reason: Pending
Status: False
Type: Ready
Events: <none>
CertManager
有什么问题,如何解决?
更新:
Ingress日志包含以下错误:
$ kubectl -n ingress-nginx logs nginx-ingress-controller-7754db565c-g557h
I1218 17:24:30.331127 6 status.go:295] updating Ingress default/cm-acme-http-solver-4dkdn status from [] to [{xxx.xxx.xxx.xxx }]
I1218 17:24:30.333250 6 status.go:295] updating Ingress default/cm-acme-http-solver-9dpqc status from [] to [{xxx.xxx.xxx.xxx }]
I1218 17:24:30.341292 6 event.go:209] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"cm-acme-http-solver-4dkdn", UID:"2e523b74-8bbb-41c7-be8a-44d8db8abd6e", APIVersion:"extensions/v1beta1", ResourceVersion:"722472", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/cm-acme-http-solver-4dkdn
I1218 17:24:30.344340 6 event.go:209] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"cm-acme-http-solver-9dpqc", UID:"b574a3b6-6c5b-4266-a4e2-6ff2de2d78e0", APIVersion:"extensions/v1beta1", ResourceVersion:"722473", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/cm-acme-http-solver-9dpqc
W1218 17:24:30.442276 6 controller.go:1042] Error getting SSL certificate "default/ingress-tls": local SSL certificate default/ingress-tls was not found. Using default certificate
W1218 17:24:30.442950 6 controller.go:1042] Error getting SSL certificate "default/ingress-tls": local SSL certificate default/ingress-tls was not found. Using default certificate
W1218 17:24:33.775476 6 controller.go:1042] Error getting SSL certificate "default/ingress-tls": local SSL certificate default/ingress-tls was not found. Using default certificate
W1218 17:24:33.775956 6 controller.go:1042] Error getting SSL certificate "default/ingress-tls": local SSL certificate default/ingress-tls was not found. Using default certificate
更新2:ingress-tls
的Secret按预期可用:
$ kubectl get secret ingress-tls -o yaml
apiVersion: v1
data:
ca.crt: ""
tls.crt: ""
tls.key: <secret-key-data-base64-encoded>
kind: Secret
metadata:
annotations:
cert-manager.io/certificate-name: ingress-tls
cert-manager.io/issuer-kind: ClusterIssuer
cert-manager.io/issuer-name: letsencrypt-staging
creationTimestamp: "2019-12-16T17:23:56Z"
name: ingress-tls
namespace: default
resourceVersion: "328801"
selfLink: /api/v1/namespaces/default/secrets/ingress-tls
uid: 5d640b66-1572-44a1-94e4-6d85a73bf21c
type: kubernetes.io/tls
更新3:
我发现cert-manager
pod出现故障,日志如下:
E1219 11:06:08.294011 1 sync.go:184] cert-manager/controller/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://<some.domain>/.well-known/acme-challenge/<some-path>': Get http://<some.domain>/.well-known/acme-challenge/<some-path>: dial tcp xxx.xxx.xxx.xxx:80: connect: connection timed out" "dnsName"="<some.domain>" "resource_kind"="Challenge" "resource_name"="ingress-tls-1089568541-1576201144-1086699008" "resource_namespace"="default" "type"="http-01"
攻毒状态:
$ kubectl describe challenge ingress-tls-1089568541-1576201144-471532423
Name: ingress-tls-1089568541-1576201144-471532423
Namespace: default
Labels: <none>
Annotations: <none>
API Version: acme.cert-manager.io/v1alpha2
Kind: Challenge
Metadata:
Creation Timestamp: 2019-12-19T11:32:19Z
Finalizers:
finalizer.acme.cert-manager.io
Generation: 1
Owner References:
API Version: acme.cert-manager.io/v1alpha2
Block Owner Deletion: true
Controller: true
Kind: Order
Name: ingress-tls-1089568541-1576201144
UID: 7d19d86f-0b56-4756-aa20-bb85caf80b9e
Resource Version: 872062
Self Link: /apis/acme.cert-manager.io/v1alpha2/namespaces/default/challenges/ingress-tls-1089568541-1576201144-471532423
UID: 503a8b4e-dc60-4080-91d9-2847815af1cc
Spec:
Authz URL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/123456
Dns Name: <domain>
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-staging
Key: <key>
Solver:
http01:
Ingress:
Class: nginx
Token: <token>
Type: http-01
URL: https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/12345/abc
Wildcard: false
Status:
Presented: true
Processing: true
Reason: Waiting for http-01 challenge propagation: failed to perform self check GET request 'http://<domain>/.well-known/acme-challenge/<token>': Get http://<domain>/.well-known/acme-challenge/<token>: dial tcp xxx.xxx.xxx.xxx:80: connect: connection timed out
State: pending
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Started 4m28s cert-manager Challenge scheduled for processing
Normal Presented 4m28s cert-manager Presented challenge using http-01 challenge mechanism
我试图删除挑战以重新触发它,但它失败,并在1 - 2分钟后相同的错误。我检查了我可以从集群节点访问挑战URL(使用new pod内部的kubectl run -it ...
和wget http://<domain>/.well-known/acme-challenge/<token>
)。
6条答案
按热度按时间hgc7kmma1#
这可能值得一看。我在
Connection Timeout
上遇到了类似的问题在
ingress-nginx
服务中更改LoadBalancer
。添加/更改
externalTrafficPolicy: Cluster
。原因是,具有证书颁发者的pod与负载均衡器位于不同的节点上,因此它不能通过入口与自己对话。
下面是取自https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.26.1/deploy/static/provider/cloud-generic.yaml的完整块
kgsdhlau2#
在我的例子中,
cert-manager
希望通过内部ip地址请求质询。无法执行自检GET请求'http:///. well-known/acme-challenge/':获取http:///. well-known/acme-challenge/:拨打tcp 10.67.0.8:80:connect:连接超时
即DNS解析被破坏。我修改了
cert-manager
的部署,使其只接受外部DNS服务器,如下所示这就是你要做的还创建了一个Issue,因此我们可以在安装helm时更改它
mwecs4sa3#
我有完全相同的问题,它似乎与数字海洋负载平衡器如何工作的错误有关。此线程lets-encrypt-certificate-issuance建议将注解
service.beta.kubernetes.io/do-loadbalancer-hostname: "kube.mydomain.com"
添加到负载均衡器。在我的例子中,我没有负载均衡器的yaml配置文件,我只是从nginx-ingress install script复制了负载均衡器声明,并将新的配置应用到kubernetes集群。下面是负载均衡器的最终配置。pobjuy324#
我没有找到这个问题的原因,所以我会张贴我是如何解决它的答案。看起来和这个bug的问题一样。我通过卸载
cert-manager
fully并重新安装它来修复它,而不更改任何配置或设置。g9icjywg5#
我的一个CertManager Pod被冻结,所以我将它们全部删除,然后它们重新启动。证书立即更新。
kubectl get pods -n cert-manager
(或您的pod所在的任何名称空间)那就统统删掉。
kubectl delete pod -n cert-manager cert-manager-xxxx cert-manager-cainjector-xxxx cert-manager-webhook-xxxx
lf5gs5x26#
数字海洋k8s部署。
generic
nginx-ingress
不能工作。从Marketplace安装他们的nginx-ingress Helm Chart。在注解中添加service.beta.kubernetes.io/do-loadbalancer-hostname: "kube.mydomain.com"
,就可以开始了。不幸的是,这些云提供商都有自己的定制需求,这些基本应用程序在一个上起作用,在另一个上不起作用。最好的,云不可知的方式似乎是拥有我们的VM/示例,并使用k8s提供器(如Rancher或RKE)部署k8s,这样我们就不会一直在云上定制我们的部署。