Comments (12)
Thank you, @ldez. Yes, this was the jigsaw piece I was missing. 😄
from pebble.
I have found out that on MacOS, the host network mode that I was using, does not allow the Pebble container to connect to the mock servers. So I did what @jsha suggested. I run the mock servers in a second container, so I can use the privileged ports.
It's still not that easy, though. Docker only accepts IP addresses for the --dns
option, so I cannot start the Pebble container with something like --dns my-mock-container
. I'd have to start my mock server container, retrieve its IP address, then start the pebble container with that IP address in the --dns
option. This is not possible with the maven docker plugin. I am now changing the /etc/resolv.conf
of the pebble container after starting it up, which is a very bad workaround.
The good news is that all my integration tests are working now, even the DNS-01 challenge. So acme4j is probably the first feature complete ACME v2 library. 😏 It still needs some polishing, though.
The bad news is that it would still be the most elegant way if Pebble had an option for the host name (not just the IP) of the DNS server to query.
from pebble.
Hi @shred - This is great feedback, thank you!
The host should be configurable, too, so all http and tls-sni verifications are always requested from the configured host, irregarding of the domain to be validated.
This sounds like it would overlap with the mock DNS request. Would it be sufficient to provide the ability to specify a mock DNS server and then allow that DNS server to map the validation domain to whatever test host you want to be used for responding to the validation? Maybe I'm misunderstanding this part of your request.
from pebble.
If you ask me for my oppinion, then no, it doesn't overlap... 😄 For my integration tests, I wrote a very simple mock DNS server that only returns the requested TXT record. I also prefer to keep the http, tls-sni, and dns challenge integration tests separated (this is, for the http challenge I only start a test http server).
But you are right... It would be sufficient if the DNS host and port is configurable.
from pebble.
You might be interested in go get github.com/letsencrypt/boulder/test/dns-test-srv
. It handle both the TXT record challenges and the faking out returning localhost for HTTP and TLS-SNI challenges.
from pebble.
Thank you, @jsha. The source code of dns-test-srv
is actually very helpful for extending my mock DNS server. So all I need is a way to configure Pebble's DNS settings.
from pebble.
So all I need is a way to configure Pebble's DNS settings.
I'll try to submit a PR for that in the next couple of days. Assigning this issue to myself as a reminder. Thanks!
from pebble.
@shred Having looked into this a little more and talking with @jsha I think the best path forward is to set the system resolver to a fake DNS server. Unfortunately Go's net
package doesn't provide any method for specifying a resolver to use. Boulder works around this by carrying a large dependency and doing all of its own DNS lookups, wrapping the net
interfaces as required. We're not super keen on having to do the same for Pebble since it seems reasonable to expect that a test server can be run in an environment with a test system DNS resolver.
How much of an inconvenience is that for your setup? There's a chance we'll bite the bullet eventually and incorporate miekg/dns
but it likely won't be a short-term change.
from pebble.
Thank you for looking into it. Maybe I should tell you how my integration test works. acme4j is written in Java, and uses maven for building. Via a maven docker plugin, maven first builds and starts a docker image of Pebble. After that, it runs the integration tests against the container (the integration tests start mock servers for the challenges), and finally maven shuts down the Pebble docker container. So basically, all I need for running the tests is a fresh system with Java, Maven and Docker installed. All I need to do is invoke mvn -P ci verify
for a full integration test of acme4j. The tests run in userland, this is why I cannot use privileged ports for the mock servers.
What I have tried is to change the Docker container's DNS settings, but as far as I know, it is not possible to use a DNS server on a different port. At the moment, I configured the container to run in host network mode, and added a hosts record so example.com
always resolves to 127.0.0.1
. Now I can perform http and tls-sni challenges for example.com
. What I cannot do yet is testing dns challenges.
Since acme4j is a project that I do for fun, I can live with that. It is just bugging me that there is a gap in my integration tests that I know of. There may be other solutions, like an iptable
forward to a different DNS port, or running a DNS server in a second docker container. Configuring the DNS host and port on Pebble would be the simplest solution though. At least for me... 😉
However, I understand that Pebble is meant to be a test server. Tests are somewhat limited if you have to use real world domains, so I think you probably need to address that problem sooner or later?
from pebble.
@shred Thanks for the detailed description of your process. I appreciate it!
It's definitely frustrating that Docker won't let you specify a containers DNS server's port in addition to the address. That would probably resolve this particular instance of the overall problem.
I'll continue to noodle this a little bit but I'm coming around to the idea that its onerous to place this burden on the end user. You won't be the only one using Pebble in a Docker environment.
from pebble.
Is it possible to run a Docker container with dnsmasq that publishes port 53, and point DNS at that? In other words, us the Docker daemon's root privileges to bind and forward port 53?
from pebble.
@shred Thanks to @ldez and 03c9ed8 there is now the ability to specify a custom DNS resolver for Pebble to use 🎉
Do you think there is more work to do under this issue or can it be closed?
from pebble.
Related Issues (20)
- Golang, apk and zlib versions are outdated HOT 2
- Allow to force auth challenge HOT 1
- Implement the "dns-account-01" Challenge in Pebble HOT 9
- Full http logging HOT 1
- fix appveyor CI
- Support must-staple extension HOT 1
- Fix `golangci-lint` HOT 3
- Regression time limit exceeded / TimeoutError HOT 5
- Request for a new release HOT 6
- v2.5.0 docker push failed HOT 9
- ci: AppVeyor is broken HOT 1
- Remove DockerHub images of pebble and pebble-challtestsrv HOT 4
- Cannot set DNS server in Docker image HOT 10
- Docker: Use hostname instead of IP addresses HOT 7
- New Certificates aren't getting Ready HOT 2
- EAB with pebble 2.5.x HOT 12
- Pebble fails to start with externalAccountBinding test config
- The request specified an account that does not exist, [certbot and pebble] HOT 2
- The key authorization file from the server did not match this challenge HOT 1
- Pebble seems to reuse challenges object for different orders HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pebble.