planetlabs / hal5d Goto Github PK

https://github.com/negz/hal5d/blob/937aa88/internal/cert/manager.go#L449

Currently hal5d validates new certificates by:

1. Writing them out as a temporary file in the real, production haproxy certificate dir
2. Running haproxy -c against the real, production haproxy config
3. Either 'committing' (by renaming the cert and reloading haproxy) or 'rolling back' (by deleting the cert) the new cert.

If hal5d crashes or is shutdown anywhere between steps 1 and 3 it will forget all about the temporary certificate file it wrote out. If the certificate was bad then all future certificates will fail validation and not be loaded, because haproxy -c validates all certificates in the directory as a whole, not just the new incoming ones. Further more, if haproxy were to restart for some reason it would be unable to start due to the invalid certificate.

hal5d should write temporary certificate files to an alternative 'staging' directory and validate against that rather than the production directory. The staging directory should either be cleaned before each validation run, or unique (i.e. a temporary directory) to each run.

https://github.com/negz/hal5d/blob/937aa88/internal/cert/manager.go#L409

When Kubernetes notifies hal5d about an ingress certificate hal5d should avoid writing the new certificate (and reloading) if the certificate matches one already on the disk. In practice this does not seem to be working; our stats show the same certificates being rewritten and haproxy reloaded every 30 minutes (i.e. the Kubernetes watch refresh interval) despite these certificates being unchanged.