Challenge hosts and ports should be configurable about pebble

[shred]

Thank you, @ldez. Yes, this was the jigsaw piece I was missing. 😄

[shred]

I have found out that on MacOS, the host network mode that I was using, does not allow the Pebble container to connect to the mock servers. So I did what @jsha suggested. I run the mock servers in a second container, so I can use the privileged ports.

It's still not that easy, though. Docker only accepts IP addresses for the --dns option, so I cannot start the Pebble container with something like --dns my-mock-container. I'd have to start my mock server container, retrieve its IP address, then start the pebble container with that IP address in the --dns option. This is not possible with the maven docker plugin. I am now changing the /etc/resolv.conf of the pebble container after starting it up, which is a very bad workaround.

The good news is that all my integration tests are working now, even the DNS-01 challenge. So acme4j is probably the first feature complete ACME v2 library. 😏 It still needs some polishing, though.

The bad news is that it would still be the most elegant way if Pebble had an option for the host name (not just the IP) of the DNS server to query.

[cpu]

Hi @shred - This is great feedback, thank you!

The host should be configurable, too, so all http and tls-sni verifications are always requested from the configured host, irregarding of the domain to be validated.

This sounds like it would overlap with the mock DNS request. Would it be sufficient to provide the ability to specify a mock DNS server and then allow that DNS server to map the validation domain to whatever test host you want to be used for responding to the validation? Maybe I'm misunderstanding this part of your request.

[shred]

If you ask me for my oppinion, then no, it doesn't overlap... 😄 For my integration tests, I wrote a very simple mock DNS server that only returns the requested TXT record. I also prefer to keep the http, tls-sni, and dns challenge integration tests separated (this is, for the http challenge I only start a test http server).

But you are right... It would be sufficient if the DNS host and port is configurable.

[jsha]

You might be interested in go get github.com/letsencrypt/boulder/test/dns-test-srv. It handle both the TXT record challenges and the faking out returning localhost for HTTP and TLS-SNI challenges.

[shred]

Thank you, @jsha. The source code of dns-test-srv is actually very helpful for extending my mock DNS server. So all I need is a way to configure Pebble's DNS settings.

[cpu]

So all I need is a way to configure Pebble's DNS settings.

I'll try to submit a PR for that in the next couple of days. Assigning this issue to myself as a reminder. Thanks!

[cpu]

@shred Having looked into this a little more and talking with @jsha I think the best path forward is to set the system resolver to a fake DNS server. Unfortunately Go's net package doesn't provide any method for specifying a resolver to use. Boulder works around this by carrying a large dependency and doing all of its own DNS lookups, wrapping the net interfaces as required. We're not super keen on having to do the same for Pebble since it seems reasonable to expect that a test server can be run in an environment with a test system DNS resolver.

How much of an inconvenience is that for your setup? There's a chance we'll bite the bullet eventually and incorporate miekg/dns but it likely won't be a short-term change.

[shred]

Thank you for looking into it. Maybe I should tell you how my integration test works. acme4j is written in Java, and uses maven for building. Via a maven docker plugin, maven first builds and starts a docker image of Pebble. After that, it runs the integration tests against the container (the integration tests start mock servers for the challenges), and finally maven shuts down the Pebble docker container. So basically, all I need for running the tests is a fresh system with Java, Maven and Docker installed. All I need to do is invoke mvn -P ci verify for a full integration test of acme4j. The tests run in userland, this is why I cannot use privileged ports for the mock servers.

What I have tried is to change the Docker container's DNS settings, but as far as I know, it is not possible to use a DNS server on a different port. At the moment, I configured the container to run in host network mode, and added a hosts record so example.com always resolves to 127.0.0.1. Now I can perform http and tls-sni challenges for example.com. What I cannot do yet is testing dns challenges.

Since acme4j is a project that I do for fun, I can live with that. It is just bugging me that there is a gap in my integration tests that I know of. There may be other solutions, like an iptable forward to a different DNS port, or running a DNS server in a second docker container. Configuring the DNS host and port on Pebble would be the simplest solution though. At least for me... 😉

However, I understand that Pebble is meant to be a test server. Tests are somewhat limited if you have to use real world domains, so I think you probably need to address that problem sooner or later?

[cpu]

@shred Thanks for the detailed description of your process. I appreciate it!

It's definitely frustrating that Docker won't let you specify a containers DNS server's port in addition to the address. That would probably resolve this particular instance of the overall problem.

I'll continue to noodle this a little bit but I'm coming around to the idea that its onerous to place this burden on the end user. You won't be the only one using Pebble in a Docker environment.

[jsha]

Is it possible to run a Docker container with dnsmasq that publishes port 53, and point DNS at that? In other words, us the Docker daemon's root privileges to bind and forward port 53?

[cpu]

@shred Thanks to @ldez and 03c9ed8 there is now the ability to specify a custom DNS resolver for Pebble to use 🎉

Do you think there is more work to do under this issue or can it be closed?