openova/platform/cert-manager-powerdns-webhook
e3mrah 5a403e66b1
fix(tls): DNS-01 wildcard TLS chain — solverName pdns, NodePort 30053, dynadot test fix (#582)
* fix(bp-harbor): CNPG database must be 'registry' not 'harbor' — matches coreDatabase

Harbor upstream always connects to a database named 'registry'
(harbor.database.external.coreDatabase default). The CNPG Cluster was
initialised with database='harbor', causing:

  FATAL: database "registry" does not exist (SQLSTATE 3D000)

Fix: change postgres.cluster.database default from 'harbor' → 'registry'
in values.yaml and cnpg-cluster.yaml template. Both the CNPG bootstrap
and Harbor's coreDatabase now use 'registry'.

Runtime fix on otech22: CREATE DATABASE registry OWNER harbor was run
against harbor-pg-1. harbor-core is now 1/1 Running.

Bump bp-harbor 1.2.1 → 1.2.2. Bootstrap-kit refs updated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tls): DNS-01 wildcard TLS chain — solverName, NodePort 30053, dynadot test fix

Five independent fixes that together complete the DNS-01 wildcard TLS chain
for per-Sovereign certificate autonomy:

1. cert-manager-powerdns-webhook solverName mismatch (root cause of #550 echo):
   - values.yaml: `webhook.solverName: powerdns` → `pdns`
   - The zachomedia binary's Name() returns "pdns" (hardcoded). cert-manager
     calls POST /apis/<groupName>/v1alpha1/<solverName>; when solverName is
     "powerdns" cert-manager gets 404 → "server could not find the resource".

2. cert-manager-dynadot-webhook solver_test.go mock format:
   - writeOK() and error injection used old ResponseHeader-wrapped format
   - Real api3.json returns ResponseCode/Status directly in SetDnsResponse
   - This caused the image build to fail at ccc38987 so the dynadot fix
     never shipped; solver tests now pass cleanly (go test ./... OK)

3. PowerDNS NodePort 30053 anycast overlay (bootstrap-kit and template):
   - _template/bootstrap-kit/11-powerdns.yaml: adds anycast NodePort values
   - omantel + otech bootstrap-kit: same NodePort 30053 overlay applied
   - anycast-endpoint.yaml: optional nodePort field rendered in port list

4. Hetzner LB + firewall for DNS port 53 (infra/hetzner/main.tf):
   - hcloud_load_balancer_service.dns: TCP:53 → NodePort 30053
   - Firewall: TCP+UDP :53 from 0.0.0.0/0,::/0

5. dynadot-client JSON parsing fix (core/pkg/dynadot-client):
   - AddRecord + SetFullDNS: struct no longer wraps respHeader in ResponseHeader
   - client_test.go: mock responses updated to real api3.json format

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: alierenbaysal <alierenbaysal@openova.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 13:49:58 +04:00
..
chart fix(tls): DNS-01 wildcard TLS chain — solverName pdns, NodePort 30053, dynadot test fix (#582) 2026-05-02 13:49:58 +04:00
blueprint.yaml feat(cert-manager): bp-cert-manager-powerdns-webhook (#373) (#410) 2026-05-01 16:44:27 +04:00
README.md feat(cert-manager): bp-cert-manager-powerdns-webhook (#373) (#410) 2026-05-01 16:44:27 +04:00

bp-cert-manager-powerdns-webhook

Catalyst Blueprint for the cert-manager DNS-01 external webhook for PowerDNS. Closes openova#373.

What it is

A wrapper around the upstream zachomedia/cert-manager-webhook-pdns binary that satisfies cert-manager's external webhook contract (webhook.acme.cert-manager.io/v1alpha1Present / CleanUp on a ChallengeRequest) and writes ACME challenge TXT records to the per-Sovereign PowerDNS instance via PowerDNS's REST API.

The chart structure mirrors bp-cert-manager-dynadot-webhook (its sibling for the parent-zone DNS-01 path); the only differences are the upstream image (zachomedia/cert-manager-webhook-pdns vs the Catalyst- authored Dynadot binary) and the per-issuer config block (PowerDNS host

  • API key vs Dynadot api-key + api-secret + managed-domains).

Why two webhooks (dynadot + powerdns) on the same fleet

The two webhooks solve DNS-01 challenges against different DNS authorities at different lifecycle stages of a Sovereign:

Stage Authority Solver Blueprint
Onboarding (parent-zone cert) Dynadot (omani.works) dynadot bp-cert-manager-dynadot-webhook
Post-handover (Sovereign-zone cert) Sovereign's own PowerDNS (omantel.omani.works) powerdns bp-cert-manager-powerdns-webhook

Per ADR-0001 §9.4 the goal is zero contabo dependency post-handover: once a Sovereign's NS delegation flips, its TLS renewals must run entirely against its own PowerDNS — no reachback to openova-controlled Dynadot credentials. This blueprint is the post-handover companion.

What this chart deploys

Resource Purpose
Deployment Runs the upstream zachomedia/cert-manager-webhook-pdns image as a non-root pod.
Service ClusterIP fronting the Deployment on port 443.
APIService Registers v1alpha1.acme.powerdns.openova.io so the kube-apiserver routes ChallengeRequest calls to the Service.
Issuer (selfsigned) Bootstraps the CA chain that issues the webhook's serving cert.
Issuer (CA) Signs the leaf serving cert from the CA Secret.
Certificate (CA) Root CA cert used by the APIService's cert-manager.io/inject-ca-from annotation.
Certificate (serving) Leaf cert mounted into the Deployment at /tls.
ServiceAccount Identity for the Deployment.
ClusterRoleBinding (auth-delegator) Lets the aggregated apiserver delegate auth back to kube-apiserver.
RoleBinding (auth-reader) Reads extension-apiserver-authentication ConfigMap from kube-system.
ClusterRole + ClusterRoleBinding (secret-reader) Grants the SA get on Secrets cluster-wide so it can read the operator-supplied PowerDNS API-key Secret on demand.
ClusterRole + ClusterRoleBinding (domain-solver) Lets cert-manager create ChallengeRequest CRs in the webhook's API group.
ClusterIssuer (letsencrypt-dns01-prod-powerdns) Paired DNS-01 issuer. Only renders when clusterIssuer.enabled=true AND powerdns.host is set (skip-render pattern).

Pairing with bp-cert-manager and bp-powerdns

The blueprint declares both as depends: in blueprint.yaml:

  • bp-cert-manager provides the cert-manager controllers + CRDs.
  • bp-powerdns provides the per-Sovereign DNS authority + REST API.

Flux dependsOn enforces ordering at the HelmRelease level (see clusters/_template/bootstrap-kit/<NN>-bp-cert-manager-powerdns-webhook.yaml).

Configuration (per-Sovereign overlay)

The chart's defaults render a runnable webhook + skip the ClusterIssuer. Cluster overlays MUST set the following for the issuer to materialise:

clusterIssuer:
  enabled: true
  email: ops@<sovereign-fqdn>
  acmeServer: https://acme-v02.api.letsencrypt.org/directory   # or staging during bring-up

powerdns:
  # In-cluster routing (preferred — no external network hop, no public TLS):
  host: "http://powerdns.powerdns:8081"
  # ...or external routing when the API is fronted by an ingress:
  # host: "https://pdns.<sovereign-fqdn>"
  apiKeySecretRef:
    name: powerdns-api-credentials   # canonical name from bp-powerdns
    key: api-key
    namespace: openova-system        # canonical namespace; webhook reads cross-NS via clientset

Per docs/INVIOLABLE-PRINCIPLES.md #4 every URL/zone is operator- overridable. No hardcoded omantel.omani.works lives in this chart.

Smoke test

Once both charts (bp-cert-manager + bp-cert-manager-powerdns-webhook) are reconciled on a Sovereign:

# Verify the webhook is running and the APIService is healthy
kubectl get -n cert-manager deploy/release-name-bp-cert-manager-powerdns-webhook
kubectl get apiservices.apiregistration.k8s.io v1alpha1.acme.powerdns.openova.io

# Issue a wildcard cert against the Sovereign apex
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-omantel-omani-works
  namespace: kube-system
spec:
  secretName: wildcard-omantel-omani-works-tls
  issuerRef:
    name: letsencrypt-dns01-prod-powerdns
    kind: ClusterIssuer
  dnsNames:
    - "*.omantel.omani.works"
EOF

# Watch the Order + Challenge progress
kubectl get certificate,order,challenge -A -w

See also