azurelets-encryptnginx-ingresscert-manager

Waiting for HTTP-01 challenge propagation: failed to perform self check GET request


I'm trying to secure my nginx-ingress connection with let's encrypt, following this tutorial (https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/blob/main/03-setup-ingress-controller/nginx.md).

I installed cert-manager (v1.8.0) using helm.

Applied my ClusterIssuer kubectl apply -f issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-nginx
spec:
  # ACME issuer configuration
  # `email` - the email address to be associated with the ACME account (make sure it's a valid one)
  # `server` - the URL used to access the ACME server’s directory endpoint
  # `privateKeySecretRef` - Kubernetes Secret to store the automatically generated ACME account private key
  acme:
    email: 'myemail'
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-nginx-private-key
    solvers:
      # Use the HTTP-01 challenge provider
      - http01:
          ingress:
            class: nginx

Then applied my ingress kubectl apply -f ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-echo
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-nginx
spec:
  tls:
  - hosts:
    - www.exmple.com
    secretName: letsencrypt-nginx-echo
  rules:
    - host: www.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: backend
                port:
                  number: 80
  ingressClassName: nginx

For debugging I ran

$ kubectl get certificate
NAME                     READY   SECRET                   AGE
letsencrypt-nginx-echo   False   letsencrypt-nginx-echo   39s


$ kubectl describe certificate
[...]
Status:
  Conditions:
    Last Transition Time:        2022-05-12T17:24:32Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      True
    Type:                        Issuing
    Last Transition Time:        2022-05-12T17:24:32Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      False
    Type:                        Ready
  Next Private Key Secret Name:  letsencrypt-nginx-echo-nxzw6
Events:
  Type    Reason     Age    From                                       Message
  ----    ------     ----   ----                                       -------
  Normal  Issuing    3m23s  cert-manager-certificates-trigger          Issuing certificate as Secret does not exist
  Normal  Generated  3m23s  cert-manager-certificates-key-manager      Stored new private key in temporary Secret resource "letsencrypt-nginx-echo-nxzw6"
  Normal  Requested  3m23s  cert-manager-certificates-request-manager  Created new CertificateRequest resource "letsencrypt-nginx-echo-x2flf"


$ kubectl describe certificaterequest
Status:
  Conditions:
    Last Transition Time:  2022-05-12T17:24:32Z
    Message:               Certificate request has been approved by cert-manager.io
    Reason:                cert-manager.io
    Status:                True
    Type:                  Approved
    Last Transition Time:  2022-05-12T17:24:33Z
    Message:               Waiting on certificate issuance from order default/letsencrypt-nginx-echo-x2flf-1264636722: "pending"
    Reason:                Pending
    Status:                False
    Type:                  Ready
Events:
  Type    Reason           Age   From                                          Message
  ----    ------           ----  ----                                          -------
  Normal  cert-manager.io  5m2s  cert-manager-certificaterequests-approver     Certificate request has been approved by cert-manager.io
  Normal  OrderCreated     5m1s  cert-manager-certificaterequests-issuer-acme  Created Order resource default/letsencrypt-nginx-echo-x2flf-1264636722

$ kubectl describe order
Status:
  Authorizations:
    Challenges:
      Token:        bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU
      Type:         http-01
      URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/107853386656/VmvKxA
      Token:        bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU
      Type:         dns-01
      URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/107853386656/LgcZ5Q
      Token:        bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU
      Type:         tls-alpn-01
      URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/107853386656/Ut9rIQ
    Identifier:     www.example.com
    Initial State:  pending
    URL:            https://acme-v02.api.letsencrypt.org/acme/authz-v3/107853386656
    Wildcard:       false
  Finalize URL:     https://acme-v02.api.letsencrypt.org/acme/finalize/540497076/88058915876
  State:            pending
  URL:              https://acme-v02.api.letsencrypt.org/acme/order/540497076/88058915876
Events:
  Type    Reason   Age    From                 Message
  ----    ------   ----   ----                 -------
  Normal  Created  6m16s  cert-manager-orders  Created Challenge resource "letsencrypt-nginx-echo-x2flf-1264636722-1300283520" for domain "www.example.com"

$ kubectl describe challenge
Spec:
  Authorization URL:  https://acme-v02.api.letsencrypt.org/acme/authz-v3/107853386656
  Dns Name:           www.example.com
  Issuer Ref:
    Group:  cert-manager.io
    Kind:   ClusterIssuer
    Name:   letsencrypt-nginx
  Key:      bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU.NSQqkslrJ8YD-aL7n_dLekPhCAy4DkdFIOF0DCAHGzo
  Solver:
    http01:
      Ingress:
        Class:  nginx
  Token:        bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU
  Type:         HTTP-01
  URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/107853386656/VmvKxA
  Wildcard:     false
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for HTTP-01 challenge propagation: failed to perform self check GET request 'http://www.example.com/.well-known/acme-challenge/bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU': Get "https://www.example.com:443/.well-known/acme-challenge/bArXItH3_w1FLvjPfFprj2ksjFHPwZ0K6Vb25MlybRU": remote error: tls: unrecognized name
  State:       pending
Events:
  Type    Reason     Age    From                     Message
  ----    ------     ----   ----                     -------
  Normal  Started    8m45s  cert-manager-challenges  Challenge scheduled for processing
  Normal  Presented  8m45s  cert-manager-challenges  Presented challenge using HTTP-01 challenge mechanism

If I describe the ingress I get

TLS:
  letsencrypt-nginx-echo terminates www.example.com
Rules:
  Host               Path  Backends
  ----               ----  --------
  www.example.com
                     /   backend:80 ('//myip')
Annotations:         cert-manager.io/cluster-issuer: letsencrypt-nginx
Events:
  Type     Reason                     Age   From                       Message
  ----     ------                     ----  ----                       -------
  Warning  AddedOrUpdatedWithWarning  12m   nginx-ingress-controller   Configuration for default/ingress-echo was added or updated ; with warning(s): TLS secret letsencrypt-nginx-echo is invalid: secret doesn't exist or of an unsupported type
  Normal   CreateCertificate          12m   cert-manager-ingress-shim  Successfully created Certificate "letsencrypt-nginx-echo"

Solution

  • I finally managed to fix the problem. Cert manager was creating an ingresss acme-http-solver which pointed to no address. After adding acme.cert-manager.io/http01-edit-in-place: "true" to my ingress file, everything seems to work.

    It may not be sufficient to just update the resource, but actually do delete and re-create it. See Issue 6065