Toilal/docker-devbox

Docker Devbox is a set of tools build on top of Docker that automates environments setup for web applications, from development to production.

★ 0Forks 0ShellGitHub ↗Compare

README

docker-devbox

Docker Devbox is a set of tools build on top of Docker that automates environments setup for web applications, from development to production.

It relies on ddb, a command line tool that provides features to generate, activate and adjust configuration files based on a single overridable and extendable configuration, while enhancing the developer experience and reducing manual operations.

Design goals

  • Give the developer a clear and native experience, but use docker containers under the hood.
  • Isolate each project, but share common patterns and tools.
  • Keep control on how containers are built, by keeping Dockerfile and docker-compose.yml visible and editable.
  • Deploy to stage and production environment with no change to the project source code.
  • Workaround usual caveats of docker on development environments.

Features

  • Activate the project environment automatically when cd into the project folder (SmartCD).
  • Access application through .test development domain name (Traefik).
  • Generate trusted SSL certificate automatically through a development certificate authority (Cloudflare CFSSL or mkcert)
  • Install CA certificates automatically to docker images, to support containers SSL inter-communication and SSL Corporate proxies like Palo Alto SSL Inbound Inspection .
  • Brings project containers commands to shell PATH and bind current working directory, commands behave as if there were installed right on the host (For example, composer install and npm install will just work as usual, psql and mysql can connect to the database).
  • Fix usual permission issues by automating local volume directory creation and fixuid integration.
  • Configure each target environment (dev, stage, prod) with environment variables only.
  • Introduce environment variables into configuration files with a template engine (Mo - Mustache Templates in Bash).
  • Enable configuration files matching the active environment with simple symlinks creation automation (mo pure bash templating engine).
  • Switch to a real public domain name with no pain (Traefik and Let's Encrypt).
  • Access application from a private network remotely through an automated SSH tunnel (ngrok , Serveo or ssi.sh)

Requirements

Docker Devbox runs natively on any Linux only, but Windows and MacOS users may use docker-devbox-vagrant to run it inside a Vagrant managed VirtualBox VM based on Ubuntu Server.

  • Docker >= 18.09.6
  • Docker compose plugin >= 2
  • GNU Bash >= 4.0
  • curl

Install or Update

curl -L https://github.com/Toilal/docker-devbox/raw/master/installer | bash

This will install everything required for Docker Devbox, but docker, docker compose and bash should be installed manually before.

Docker Devbox will install Traefik in a docker container and binds tcp/80,tcp/443 to host, so those ports should be available.

Port tcp/7780 should also be available for CFSSL container (local certificate authority service).

Installation script may ask for sudo password to install some dependencies, like curl, git and make.

Traefik, CFSSL and Portainer configurations live in the services directory of this repository. The installer copies them into ~/.docker-devbox/<service>, retrieving them from the local checkout when run from one, or cloning this repository into ~/.docker-devbox/.src otherwise.

Reinstalling replaces those service directories, but the local configuration and state they hold are set aside first and put back afterwards:

Path Contents
ddb.local.yml, ddb.local.yaml your own overrides of the service configuration
traefik/acme.json certificates issued by Let's Encrypt
traefik/config per domain TLS configuration written by ddb configure
.certs certificates of the service own domain

The last two are dropped when DOCKER_DEVBOX_RESET is set, as a new certificate authority makes them useless. Docker volumes, holding the certificate authority and the portainer data, are kept as well. Anything else in a service directory is replaced by the repository content, so local edits of the versioned files do not survive.

Development domain name configuration (.test)

To access application through .test development domain name, you have to setup your system for those domains to be resolved as docker host IP.

On Linux, dnsmasq can be used for this purpose.

On Windows, Acrylic DNS proxy can be used for this purpose.

Linux (dnsmasq)

  • Ubuntu Server (without NetworkManager)
sudo apt-get install -y dnsmasq

DOCKER_HOST_IP=$(ip -4 addr show docker0 | grep -Po 'inet \K[\d.]+')
sudo sh -c "echo address=/.test/$DOCKER_HOST_IP>/etc/dnsmasq.d/test-domain-to-docker-host-ip"

sudo service dnsmasq restart
  • Ubuntu Desktop (with NetworkManager)

NetworkManager from desktop brings it's own dnsmasq daemon.

sudo mv /etc/resolv.conf /etc/resolve.conf.bak
sudo ln -s /var/run/NetworkManager/resolv.conf /etc/resolv.conf

sudo sh -c 'cat << EOF > /etc/NetworkManager/conf.d/use-dnsmasq.conf
[main]
dns=dnsmasq
EOF'

sudo sh -c 'cat << EOF > /etc/NetworkManager/dnsmasq.d/test-domain-to-docker-host-ip
address=/.test/$(ip -4 addr show docker0 | grep -Po "inet \K[\d.]+")
EOF'

sudo service NetworkManager restart

Windows (Acrylic DNS proxy)

Download Acrylic DNS proxy for Windows, and perform installation.

Then open Acrylic UI and configure the Host configuration with such entry

192.168.1.100 *.test

The IP address should match the IP of the docker engine.

Configure local CA certificate

Docker Devbox automatically generates development certificate for HTTPS support, but you need to register the local CA certificate using mkcert.

Linux

Run the following commands from docker devbox shell.

# This dependency is required to support Chrome and Firefox.
sudo apt-get install libnss3-tools

# Uninstall any previous CA cert
mkcert -uninstall

# Move to cfssl container directory
cd ~/.docker-devbox/cfssl

# Replace default mkcert key/pair with CFSSL public key.
rm -Rf $(mkcert -CAROOT) && mkdir -p $(mkcert -CAROOT)
docker compose cp intermediate:/etc/cfssl/ca.pem $(mkcert -CAROOT)/rootCA.pem

# Install CFSSL CA Certificate with mkcert.
mkcert -install 

Windows

On Windows, you should install the CA certificate inside the VM where docker-devbox is installed with the previous linux procedure, but you should also install the CA certificate on your host, for browser to aknowlegdge the development certificates.

  • Download mkcert for Windows, and set CAROOT environment variable to some directory, like C:\mkcert-ca.

  • Extract the CFSSL ca certificate from docker with the following command

# Inside docker-devbox shell
cd ~/.docker-devbox/cfssl
docker compose cp intermediate:/etc/cfssl/ca.pem ../certs/mkcert-ca/rootCA.pem
  • Copy ~/.docker-devbox/certs/mkcert-ca/rootCA.pem to the host, inside CAROOT directory.

  • Close all cmd.exe, and open a new one to check that CAROOT environment variable is defined.

# This should output CAROOT environment variable
mkcert -CAROOT
  • Install CA certificate
mkcert -install

Installation environment variables

Environment variables available for installer script:

  • Partial installs:
    • DOCKER_DEVBOX_DISABLE_SMARTCD: Disable SmartCD.
    • DOCKER_DEVBOX_DISABLE_CFSSL: Disable CFSSL.
    • DOCKER_DEVBOX_DISABLE_PORTAINER: Disable portainer.
    • DOCKER_DEVBOX_DISABLE_REVERSE_PROXY: Disable reverse-proxy feature (traefik).
    • DOCKER_DEVBOX_DISABLE_OPTIONAL_DEPENDENCIES: Disable the installation of mkcert.
    • DOCKER_DEVBOX_DISABLE_DDB: Keep the ddb binary already available instead of downloading one. The installer fails if no ddb is found in ~/.docker-devbox/bin nor in PATH. Useful to run the installer against a locally built ddb. Only the download is skipped: core.github_repository is still kept aligned as described below.
    • DOCKER_DEVBOX_MINIMAL: Creates the required folder, download the ddb binary and create reverse-proxy network only. Does not install other tools like smartcd, cfssl, portainer, etc.
    • DOCKER_DEVBOX_CI: Equivalent to DOCKER_DEVBOX_MINIMAL and DOCKER_DEVBOX_DISABLE_OPTIONAL_DEPENDENCIES, recommanded for CI.
  • Specific version installs:
    • DOCKER_DEVBOX_DDB_VERSION: Install a specific version of ddb (ex: v2.0.1). When unset, gets the latest version
    • DOCKER_DEVBOX_DDB_URL: Download the ddb binary from this url instead of a github release, which takes precedence over DOCKER_DEVBOX_DDB_VERSION and DOCKER_DEVBOX_DDB_REPOSITORY. Any protocol supported by curl works, including file:///path/to/ddb to install a local build.
    • DOCKER_DEVBOX_DDB_REPOSITORY: Github repository (owner/name) to download the ddb release from. Default is inetum-orleans/docker-devbox-ddb. The installer keeps core.github_repository in ~/.docker-devbox/ddb.yml aligned with this value so ddb self-update uses the same repository, creating or updating that file as needed. A core section written in flow style (core: {...}) can't be updated automatically and has to be edited manually.
    • DOCKER_DEVBOX_SMARTCD_BRANCH: Use a specific smartcd (inetum fork) branch.
    • DOCKER_DEVBOX_REPOSITORY: Repository to retrieve services (traefik, cfssl, portainer) from. Default is https://github.com/Toilal/docker-devbox.git.
    • DOCKER_DEVBOX_BRANCH: Branch of DOCKER_DEVBOX_REPOSITORY to retrieve services from. Default is master. Ignored when the installer runs from a local checkout, as services are then taken from the checkout itself.
    • DOCKER_DEVBOX_DDB_ASSET_NAME: Custom ddb release asset name to install ddb. It was set to "ddb-linux-older-glibc" to install ddb on older linux distributions, like Ubuntu 16.04. This asset is not compiled anymore, but the option sticked. You should also add this value to core.release_asset_name in ddb configuration to make self-update command download this asset.
  • Misc:
    • DOCKER_DEVBOX_CURL_OPTS_GITHUB_API: Additional curl options to pass when accessing github api. You can set this variable to -u <username:token> using a Github Personnal Access Token if you encounter 403 errors due to rate limiting.
    • DOCKER_DEVBOX_RESET: Reinstall the services from scratch instead of preserving their state: their docker volumes, ~/.docker-devbox/certs and the certificates of the service directories are removed. This issues a brand new certificate authority, which then has to be trusted again on the clients, and makes every already generated project certificate useless. Local configuration files are still preserved.
    • DOCKER_DEVBOX_SKIP_DOCKER_CHECKS: Force installation even if docker binary is unavailable.
    • DOCKER_DEVBOX_REVERSE_PROXY_NETWORK: Name of the reverse proxy network. Default is reverse-proxy.
    • DOCKER_DEVBOX_ALLOW_ROOT: Allow the script to be run as root. This is not recommended.

Environment variables can be set right before bash invocation in the installer one-liner.

curl -L https://github.com/Toilal/docker-devbox/raw/master/installer | \
DOCKER_DEVBOX_CI=1 \
bash