Concurrency error when trying to get a certificate. about pebble

[cpu]

Thanks for the bug report @BenjaminSchubert 🍰

Can you share the Certbot command line you ran when you produced this Pebble output?

[BenjaminSchubert]

Ensuring we don't have anything in /etc/letsencrypt; I run:

certbot certonly --standalone -m [email protected] -d [email protected] --agree-tos

On a fedora:27 image up to date.

[BenjaminSchubert]

@cpu here is a vagrant file that contains my environment and the steps I perform.

To reproduce you can do:

# create Vagrantfile with the content below
vagrant up
vagrant ssh
sudo su
certbot certonly --standalone -m [email protected] -d [email protected] --agree-tos  # will fail
tmux attach  # will allow you to see the error

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
  config.vm.box = "fedora/27-cloud-base"

  config.vm.provider :libvirt do |libvirt|
    libvirt.memory = 4096
  end

  config.vm.define "letsencrypt" do |vm0|
    vm0.vm.hostname = "acme-v01.api.letsencrypt.org"

    vm0.vm.provision "shell", inline: <<-SHELL
      dnf update --assumeyes
      dnf install --assumeyes git golang tmux

      echo 10.0.21.244 acme-v01.api.letsencrypt.org >> /etc/hosts
    
      go get -u github.com/letsencrypt/pebble/...
      go get github.com/jsha/minica

      cd /root/go/src/github.com/letsencrypt/pebble

      # update certificates
      cp test/certs/pebble.minica.pem /etc/pki/ca-trust/source/anchors/test.crt
      update-ca-trust

      # use the default directory
      sed -i s:/dir:/directory:g wfe/wfe.go
      go install ./...

      echo 'export PATH=$PATH:/root/go/bin' > /root/.bashrc
      source /root/.bashrc

      cd test/certs
      minica -ca-cert pebble.minica.pem -ca-key pebble.minica.key.pem -domains acme-v01.api.letsencrypt.org
      cd ../..

      sed -i s/14000/443/g test/config/pebble-config.json
      sed -i s/localhost/acme-v01.api.letsencrypt.org/g test/config/pebble-config.json

      tmux new-session -d -s pebble
      tmux send-keys pebble C-m
    SHELL
  end
end

[cpu]

@BenjaminSchubert Thanks for the detailed reproduction notes & the Vagrantfile! That's a great help. I'll try to take a look at this crash this afternoon.

[cpu]

Hi again @BenjaminSchubert 👋 I was able to reproduce this bug and put a fix out in #120 Thanks again for the bug report!!

certbot certonly --standalone -m [email protected] -d [email protected] --agree-tos

The issue here is that the -d [email protected] argument is invalid. -d is for a domain name. I think Pebble's input validation could use some work because it didn't reject this outright as an invalid domain name, and instead tried to do an HTTP-01 validation to "http://test%40test.com:5002/.well-known/acme-challenge/<token>" which failed. That tickled a bug that we didn't know about that caused Pebble to crash :-)

With the bug fixed applied you'll get a different result:

FailedChallenges: Failed authorization procedure. [email protected] (http-01): urn:ietf:params:acme:error:malformedRequest :: Invalid URL "http://test%40test.com:5002/.well-known/acme-challenge/h3sCmsNTEFS-0yOikQEHrFUPEGCcGVPF4ncS9NwmZio"

As a side-note: You might be able to simplify your Vagrantfile if you use the SERVER and REQUESTS_CA_BUNDLE environment variables with Certbot. That will let you point Certbot at the Pebble CA PEM file and the Pebble listen address without having to change system-wide trust stores or adding /etc/host entries to mock out acme-v01.api.letsencrypt.org.

I run Certbot against Pebble on my localhost like this: REQUESTS_CA_BUNDLE=~/go/src/github.com/letsencrypt/pebble/test/certs/pebble.minica.pem SERVER=https://localhost:14000/dir certbot ...

[cpu]

I created #121 to fix the input validation to reject nonsense domains earlier.

[cpu]

And with #120 now merged this should be fixed in master 👍