1. Delegate a small zone to the machine that will run acme-dns (records in your real DNS):

   1 2	auth.example.com. NS acme.example.com. acme.example.com. A 203.0.113.10

2. acme-dns on that machine (/etc/acme-dns/config.cfg), DNS on 53 and the API on 443:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

   [general] listen = "0.0.0.0:53" protocol = "both" domain = "auth.example.com" nsname = "acme.example.com" nsadmin = "hostmaster.example.com" records = [ "auth.example.com. A 203.0.113.10", "auth.example.com. NS acme.example.com.", ] [api] ip = "0.0.0.0" port = "443" tls = "letsencrypt" disable_registration = false

3. Register a client and create the CNAME:

   1 2 3

   curl -s -X POST https://acme.example.com/register -H 'Content-Type: application/json' \ -d '{"allowfrom": ["203.0.113.0/24"]}' # {"username":"eabcdb41-…","password":"pbAX…","fulldomain":"d420c923-….auth.example.com","subdomain":"d420c923-…","allowfrom":["203.0.113.0/24"]}

   1	_acme-challenge.example.com. CNAME d420c923-….auth.example.com.

4. certbot with the hook:

   1 2 3 4 5

   curl -fsSL -o /etc/letsencrypt/acme-dns-auth.py https://raw.githubusercontent.com/joohoi/acme-dns-certbot-joohoi/master/acme-dns-auth.py chmod +x /etc/letsencrypt/acme-dns-auth.py sed -i 's|^#!/usr/bin/env python$|#!/usr/bin/env python3|; s|^ACMEDNS_URL = .*|ACMEDNS_URL = "https://acme.example.com"|' /etc/letsencrypt/acme-dns-auth.py certbot certonly --manual --manual-auth-hook /etc/letsencrypt/acme-dns-auth.py \ --preferred-challenges dns --debug-challenges -d '*.example.com' -d example.com

   The first time, the hook registers itself, stores the credentials in /etc/letsencrypt/acmedns.json and pauses so you can create the CNAME it prints; Enter, and it validates. Renewals (certbot renew) never ask again.

Why

DNS-01 is the only challenge that gives wildcards and the only one that works for machines with no public port 80. The usual price is an API token from your DNS provider on every server that requests certificates; that token can typically change any record in the zone, and it sits in plain text under /etc/letsencrypt.

acme-dns turns that around: it is an authoritative DNS server for a zone of its own (auth.example.com) that only serves TXT records under random subdomains, with an API to update them. Since _acme-challenge.example.com is a CNAME to one of those subdomains, Let’s Encrypt follows the chain and validates the TXT there. Each client’s credentials only work for its subdomain; if stolen, the worst case is someone requesting a certificate for that name, not rewriting your zone.

cert-manager

The same scheme, with the credentials in a Secret:

1

kubectl -n cert-manager create secret generic acme-dns --from-file=acmedns.json=./acmedns.json

1
2
3
4
5
6
7
8
9

{
  "example.com": {
    "username": "eabcdb41-d89f-4580-826f-3e62e9755ef2",
    "password": "<password from the registration>",
    "fulldomain": "d420c923-bbd7-4056-ab64-c3ca54c9b3cf.auth.example.com",
    "subdomain": "d420c923-bbd7-4056-ab64-c3ca54c9b3cf",
    "allowfrom": ["203.0.113.0/24"]
  }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-dns-account-key
    solvers:
    - dns01:
        acmeDNS:
          host: https://acme.example.com
          accountSecretRef:
            name: acme-dns
            key: acmedns.json

The JSON key is the certificate’s domain; *.example.com and example.com share the same entry (the _acme-challenge.example.com CNAME covers both). Every distinct domain needs its own registration and CNAME.

Traps

• allowfrom with the IP acme-dns sees: the public egress IP of the server or the cluster, not the pod’s. If /update fails with 401, it is this or the password.
• CNAME TTL: keep it low (60–300 s) the first time; with a one-hour TTL, a mistake in the CNAME takes an hour to correct in Let’s Encrypt’s eyes. The TXT is served by acme-dns with TTL 1.
• A real port 53: acme-dns must be reachable over UDP and TCP from the Internet, and be the NS for auth.example.com in your parent zone. Without the NS record there is no delegation and validation returns NXDOMAIN.
• One registration per domain, not per server: if three servers request certificates for example.com, they share the same registration and the same acmedns.json, or each registers its own and the CNAME can only point to one. Share the credentials.
• --manual renews fine because the hook is stored in /etc/letsencrypt/renewal/<domain>.conf; what you cannot do is change the hook without running certonly again.
• Version: acme-dns 2.0.2 is from February 2025 and the project is still active; the certbot hook has not changed in years and works with Python 3 after fixing the shebang.

References: acme-dns, acme-dns-certbot-joohoi and ACMEDNS in cert-manager.

See also

• cert-manager: the HTTP-01 challenge returns 308 — when HTTP-01 is enough, and when it is not.
• HAProxy on pfSense with ACME and Cloudflare — DNS-01 with the provider’s token, the case acme-dns avoids.
• 2048-bit DKIM in a split TXT — another DNS record that breaks over details.

1

Waiting for HTTP-01 challenge propagation: wrong status code '308', expected '200'

cert-manager creates a temporary Ingress (cm-acme-http-solver-…) to answer the challenge, but without a class your ingress controller does not adopt it; the request to /.well-known/acme-challenge/… gets answered by the application’s Ingress, which redirects to HTTPS with 308. Put the class in the solver:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
    - http01:
        ingress:
          ingressClassName: nginx        # cert-manager >= 1.12; before: class: nginx

Apply and delete the stuck challenge so it retries:

1

kubectl delete challenge -n <ns> --all

Diagnosis

1
2
3
4

kubectl get certificate,certificaterequest,order,challenge -A
kubectl describe challenge -n <ns> <name>              # Status → Reason is the line you want
kubectl get ingress -n <ns>                            # does cm-acme-http-solver-xxxx show CLASS <none>?
curl -i http://app.example.com/.well-known/acme-challenge/test   # from outside and from a pod

Before asking Let’s Encrypt to validate, cert-manager runs a self-check: it tries the challenge URL itself. The Challenge’s Reason is the result of that self-check and almost always points at the cause.

Why ingressClassName

Since Kubernetes 1.18 an Ingress’s class goes in spec.ingressClassName; the kubernetes.io/ingress.class annotation is deprecated and modern controllers (ingress-nginx 1.x, Traefik) may ignore it. In cert-manager, the solver’s class field still writes the annotation; ingressClassName (since 1.12) writes the field. Use one of the two, and in 2026 the right one is ingressClassName.

Without a class, the temporary Ingress exists but nobody serves it. The controller routes the request by host, finds the app’s Ingress, and applies its rules: ssl-redirect → 308. That is why the error looks like the app’s and not cert-manager’s.

Alternative: DNS-01 with Cloudflare

When the cluster is not reachable from the Internet on port 80, or you want a wildcard, the challenge goes through DNS. Token in Cloudflare with Zone → DNS → Edit and Zone → Zone → Read on the zone:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
type: Opaque
stringData:
  api-token: <token>
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-dns-account-key
    solvers:
    - dns01:
        cloudflare:
          apiTokenSecretRef:
            name: cloudflare-api-token
            key: api-token

With DNS-01 there is no temporary Ingress, no HTTP self-check, no hairpin; the self-check queries the TXT record, and if the cluster’s DNS resolves against an internal server with the zone cached, the Reason will be DNS record for … not yet propagated: patience, or --dns01-recursive-nameservers=1.1.1.1:53 on cert-manager.

Traps

• Staging first. https://acme-staging-v02.api.letsencrypt.org/directory has no issuance limits; production blocks 5 issuances per name per week, and every failed Challenge that reaches Let’s Encrypt counts as an attempt.
• Delete the Challenge, not the Certificate. Deleting the Order or the Challenge forces a retry with the new configuration; deleting the Certificate also deletes the secret and leaves the app without TLS meanwhile.
• pathType: Exact on the solver Ingress (recent cert-manager) clashes with some old controllers; if you see 503 on OpenShift, that is it.
• Issuer per namespace vs ClusterIssuer: the Ingress must annotate the one that exists (cert-manager.io/cluster-issuer or cert-manager.io/issuer); annotating the one that does not leaves the Certificate in Issuing forever with no Challenge to describe.

References: HTTP01 solver, Cloudflare DNS01, and Troubleshooting ACME in the cert-manager documentation.

1. Cloudflare → My Profile → API Tokens → Create Token → Edit zone DNS template: permissions Zone → Zone → Read and Zone → DNS → Edit, Zone Resources limited to your zone. Copy the token; the Account ID and Zone ID are on the zone’s Overview page.
2. pfSense → System → Package Manager → install acme and haproxy.
3. Services → ACME → Account keys → Add: ACME Server Let’s Encrypt Production ACME v2 (or Staging to test) → Create new account key → Register → Save.
4. Services → ACME → Certificates → Add: name, the account, Domain SAN list: app.example.com (and *.example.com if you want a wildcard) with method DNS-Cloudflare, and in its fields only Token, Account ID and Zone ID. Actions list: Restart Local Service → haproxy. Save → Issue.
5. Services → HAProxy → Frontend → edit → SSL Offloading ✔ on the listen address → Certificate: the one ACME just created → Save → Apply Changes.

Automatic renewal (the package’s daily cron) and HAProxy restarted on every renewal.

Cloudflare: the token

The Global API Key old guides ask for gives access to the whole account; an API token with minimal permissions is the right thing and is what the package’s DNS-Cloudflare method uses when you fill in Token:

With Token one of the two IDs is enough; setting both does no harm. Token permissions: Zone:Read and DNS:Edit on the specific zone (Include → Specific zone).

If you enable Client IP Address Filtering on the token, it must include the firewall’s public IPv4 and also IPv6: pfSense goes out through whichever it has, and a token limited to v4 fails as soon as the API call goes over v6.

ACME: account and certificate

Account. Create two: one against Staging to test and one against Production. Staging has no issuance limits but its certificates are not trusted; once everything works, switch the certificate’s account to production and issue again.

Certificate. Under Services → ACME → Certificates → Add:

• Domain SAN list: one entry per name. For a *.example.com wildcard the validation has to be DNS-01 (HTTP-01 does not allow it). A wildcard does not cover the apex: add example.com as a second entry.
• Method: DNS-Cloudflare on each entry, with the three fields.
• DNS Sleep: Cloudflare propagates in seconds; the default is plenty. If issuance fails with TXT not found, raise it to 120.
• Key: RSA 2048 or ECDSA 256; both work fine with HAProxy.
• Actions list: Restart Local Service → haproxy. Without it, HAProxy keeps serving the old certificate until someone restarts it, even though ACME has already renewed.

Issue from the certificate list. The result shows up under System → Certificates with the name you gave it; that is where HAProxy will see it.

HAProxy: using the certificate

Services → HAProxy:

1. Settings: enable HAProxy, and leave Max SSL Diffie-Hellman size at 2048 or more.
2. Frontend → your HTTPS frontend → External address: port 443, SSL Offloading ✔ → SSL Offloading section → Certificate: the ACME one. For several domains on the same frontend, Additional certificates with the rest; HAProxy picks by SNI.
3. Save → Apply Changes.

Test from outside:

1
2

openssl s_client -connect app.example.com:443 -servername app.example.com </dev/null 2>/dev/null \
  | openssl x509 -noout -issuer -dates

issuer=C = US, O = Let's Encrypt and notAfter 90 days out.

As a bonus, the pfSense GUI with the same certificate: System → Advanced → Admin Access → SSL/TLS Certificate, and a second action on the ACME certificate: Shell Command /etc/rc.restart_webgui.

Traps

• Cloudflare in proxy mode (orange cloud) does not affect DNS-01, but it does affect how traffic arrives: if the A record is proxied, HAProxy sees Cloudflare’s IP as the source. For an internal service behind pfSense, grey cloud.
• Issue fails with Invalid request headers or Authentication error: token without permission on that zone, token IP filter, or the Zone ID of another zone.
• Production rate limits: five certificates per set of names per week. Test on staging.
• The certificate renews; HAProxy does not reload it by itself. The restart action is mandatory. Check under Services → ACME that the last renewal has a date and in openssl s_client that notAfter moved.
• ACME certificates renew at 2/3 of their lifetime (about 60 days). If you see the cron running and nothing changes before then, that is normal.

Reference: ACME package in the Netgate documentation and acme.sh’s DNS provider list (dns_cf section).