Add support for alternative DNS resolvers in lego about fritz-tls

[tisba]

Interesting 🤔

I think I first have to update and fix some dependencies, but then I'll look more closely into this matter. AFAIK there should be no reason not to support this. Quite the contrary, that the authoritative name server is not used sounds more like a bug.

[lozzd]

Yes I'm a bit confused and was trying to dig through the lego source to see why it behaves differently on these two systems. On one, it defaulted to going to the authoratative nameservers... and on the other, it used the system resolver. I can't figure out why the difference in behaviour.

It's not the end of the world, it just means waiting 1.5 hours for full propogation to have completed instead of ~10-15 mins for the authoratative nameserver to show the new record being there.

[tisba]

Hey @lozzd 👋

Can you check out if v0.10.0 solves this issue for you? I just published a new version.

In afcc8d3 I added the option to use a custom DNS resolver. In my testing it got picked up, so I hope this solves your issue.

PS: Be aware that I dropped --auto-cert and made it the default with this release as well. I got tiered typing it over and over again 🙈

[lozzd]

Thanks, I'll give it a shot!
It's still using my system resolvers by default (tried that firt) versus on that other system where it immediately fell back to using the authoratative nameservers, I really don't understand why. It'd make way more sense if it always went straight to the authoratative to skip any caching issues. 🤔

Hopefully it'll work, because now the solving of the DNS takes so long that the Fritzbox session has expired by the time the DNS caches have cleared, so the upload fails! 🙄 I guess if that continues it raises the question - chicken or egg - login to the Fritzbox to see if the auth works, or skip that, try the cert first, and then hope the login works - or just have code to renew the session I guess.

Thanks again!

[lozzd]

Hmmm - well. This was a vast improvement, it took just 22 minutes to verify the DNS entries - amazing!
But upload still failed because I got kicked back to the login page on the FritzBox. That's really bizare - this worked once! With a much longer propogation time. I wonder what the session expiry is on the Fritzbox?

[tisba]

I think I have to read more of lego code to properly understand this. The resolver option I've added is only used for the client to check if the entry has been propagated and then triggers the check by Let's Encrypt. Let's Encrypt is of corse not using the resolver you specify, because that would defeat the whole verification purpose 😉

I'm not sure about the timeout duration. My guess is something around 10 minutes. I found this document where it states it is 20 minutes. The problem with the timed out session is maybe a slightly different one. My initial idea was that I want to check that the login works first before continuing. But it should be no problem to re-authenticate if the session is lost 🤔

[tisba]

Could you try --dns-resolver with the authoritative name server for the domain you're trying to obtain a cert for? That should be much quicker if this is a DNS propagation issue.

[tisba]

I've dug a bit deeper into the lego code and this is what I think is going on:

1. lego is doing a "pre check" to see if the DNS entry has been propagated. This is being handled by dns01.Solve in https://github.com/go-acme/lego/blob/v4.5.3/challenge/dns01/dns_challenge.go#L131-L137
2. First the system resolver (or the resolver specified by --dns-resolver in fritz-tls) is being used to check for the updated DNS entry, see checkDNSPropagation https://github.com/go-acme/lego/blob/v4.5.3/challenge/dns01/precheck.go#L55-L76.
3. if this check is successful, only then are the authoritative name servers asked directly in checkAuthoritativeNss https://github.com/go-acme/lego/blob/v4.5.3/challenge/dns01/precheck.go#L79-L110

As far as I can tell, there is no way to change this behaviour. I my guess would be, that they don't want to hit authoritative name servers unnecessarily. I'm collecting a bit more information and will open an issue with lego for clarification.

In the meantime, I just pushed v0.11.0 which should work as a workaround for you: The FRITZ!Box session is checked before the certificate upload and another login is performed if necessary. It won't be a speed up in your case, but it might help to succeed.

[lozzd]

Hey,
Confirmed both the --dns-resolver and session renewal are now working perfectly:

2021/10/20 09:04:07 Session expired, re-authenticating...
2021/10/20 09:04:14 TLS certificate upload successful!
2021/10/20 09:04:17 TLS certificate installation verified!

It was roughly ~30 minutes start to finish for this to work, which is totaly fine for my purposes.

[tisba]

Nice, glad to hear!

It's still quite strange that it takes so long…