tonyskapunk / dci-dev-env

Dci development environment

Distributed CI environment for development

This repository is used to store the configuration of docker compose in order to provide an environment for development.

Installation

To run DCI in docker-compose, first clone this repository:

git clone [email protected]:redhat-cip/dci-dev-env.git

Then, bootstrap dci-dev-env:

./utils/bootstrap.sh

Usage

Once install, you can launch the environment:

docker-compose up -d

Then, you can attach containers in order to run parts of the applications:

docker-compose exec <container-name> bash

For instance docker-compose exec api bash

Usage with podman (Experimental)

You can launch the API containers:

make build
make pull
make up

Then, you can attach containers using

podman exec -it api bash

If you want to launch the whole stack, you can:

make build-all
make pull
make all

See Makefile for extra containers and documentation

Containers

Here is the list of containers for running the application:

• ansible: contains the dci-ansible code ;
• api: contains the API of the application and serves it on localhost port 5000, must be started manually ;
• client: contains the dciclient (Python binding) and dcictl (CLI) clients to DCI Control Server ;
• db: contains the postgresql database and serves it on localhost port 5432, started by default ;
• doc: helper to build the project's documentation ;
• keycloak: keycloak server for SSO. ⚠️ Use localhost as keycloak's domain as it's set to this value and not 127.0.0.1 ;
• ui: contains the web app of DCI and serves it on localhost port 8000.
• swift: storage backend for the API using a Swift server.

API Container

You can initialize or re-initialize the database from the API container by running:

docker-compose exec api ./bin/dci-dbprovisioning

Client Container

This container allows one to run the python-dciclient within it.

This container is special in several ways compares to the others:

• It runs systemd ;
• It runs an sshd daemon that belong to root:root.

Client Usage

To initialize this container you need to install the dciclient library, as well as the agents and feeders:

cd /opt/python-dciclient && pip install --editable ./
cd /opt/python-dciclient/agents && pip install --editable ./
cd /opt/python-dciclient/feeders && pip install --editable ./

Then, Create a local.sh file with the following credentials and source it:

API_CONTAINER_IP="$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <CONTAINER_ID>)"
export DCI_LOGIN=admin
export DCI_PASSWORD=admin
export DCI_CS_URL=http://$API_CONTAINER_IP:5000

Keycloak Container

This container allows to use SSO based authentication.

Provisioning

It's provisioned by default by the following:

• The user admin: username=admin, password=admin ;
• A realm dci-test ;
• A client dci-cs ;
• A lambda user within the dci-test realm: username=dci, password=dci

The client dci-cs is configured to allows OIDC Implicit flow and Direct Access protocols.

Use OIDC Implicit flow

Generate a random value using an UUID, then we will use it for the nonce field in the following link:

http://localhost:8180/auth/realms/dci-test/protocol/openid-connect/auth?\
client_id=dci-cs&\
response_type=id_token&\
scope=openid&\
nonce=YOU_MUST_REPLACE_ME&\
redirect_uri=http://localhost:8000/login

This link will redirect the browser to Keycloak SSO login page, once authenticated it will redirect to the redirect_uri value, e.g.:

http://localhost:8000/login?id_token=eyJhs...EPscxhRMuEdA

The id_token parameter correspond to the JSON Web Token (jwt) generated by Keycloak and will be used to authenticated the client on the server.

Use Direct Access protocol

From the CLI:

$ curl \
  --data "client_id=dci-cs" \
  --data "username=dci" \
  --data "password=dci" \
  --data "grant_type=password" \
  http://localhost:8180/auth/realms/dci-test/protocol/openid-connect/token

This will get a JWT and will be used to authenticated the client on the server .

Ansible Container

You can use the Ansible container to run tox:

docker-compose exec ansible tox

and the functional tests:

docker-compose exec ansible bash -c 'cd tests; ./run_tests.sh'

Swift Container

If you want to enable the swift storage for the API:

ln -s dci-swift.yml docker-compose.override.yml
docker-compose build
docker-compose up -d

Swift is exposed on the non-standard port 5001. If you want to interact with it, you can use your local swift client.

source dci-swift/openrc_dci.sh
swift upload fstab /etc/fstab
swift list

Documentation Container

This container generates DCI documentation.

Use the following command to generate the doc:

docker-compose -f dci-extra.yml run doc

Contribute

You will need to install docker-compose and git-review:

pip install -U -r requirements.txt