shalberd / s2i-thoth

Thoth's addition to OpenShift's s2i Python builds to benefit from Thoth's recommendations in your application

s2i-thoth

Experimental Thoth container images:

• quay.io/thoth-station/s2i-thoth-ubi8-py39
• quay.io/thoth-station/s2i-thoth-ubi8-py38
• quay.io/thoth-station/s2i-thoth-ubi8-py36
• quay.io/thoth-station/s2i-thoth-f34-py39
• quay.io/thoth-station/s2i-thoth-f35-py310

Artifacts needed to build s2i-thoth-* container images.

These container images are complaint with OpenShift's s2i build process to build Python applications. They extend the functionality of base s2i Python container images so that applications using these container images benefit from Thoth's recommendations.

See thoth-station/s2i-example repository with OpenShift s2i examples.

Note: When using Thoth S2I build process, it is required to use .thoth.yaml file even if all the Thoth features are turned off. A simple .thoth.yaml file as generated by thamos config should work in most cases (see thamos repository for more info).

Configuration options

Configuration options for Thoth's s2i image:

• THOTH_ADVISE - always use the recommended stack by Thoth (even if the lock file is present in the repo)
• THOTH_OFF - disable all the Thoth features, only THOTH_FORCE_GENERATE_CONFIG can be supplied if .thoth.yaml is not available
• THOTH_CONFIG_CHECK - verify values stated in the configuration file match the build environment
• THOTH_FORCE_GENERATE_CONFIG - generate configuration file during the build process, overwrites already existing configuration file if present
• THOTH_PROVENANCE_CHECK - verify stack provenance - the provenance check is triggered only if the lock file is not comming from Thoth's recommendation engine (otherwise the stack has already verified provenance)
• THOTH_ASSEMBLE_DEBUG - run s2i's assemble script in verbose mode
• THOTH_DRY_RUN - submit stack to Thoth's recommendation engine but do NOT use the recommended lock file, use the lock file present in the repo instead
• THOTH_FROM_MASTER - Use Thamos from git instead of a PyPI release - handy if the released Thamos has a bug which was fixed in the master branch
• THOTH_HOST - Thoth's host to reach out to for recommendations (defaults to prod deployment at khemenu.thoth-station.ninja)
• THOTH_ERROR_FALLBACK - fallback to the lock file present in the repository if the submitted Thoth analysis fails

See also configuration options for Thoth's client present in Thamos repository.

Building container images

Container images are automatically built in quay.io when pushed to master branch.

Building container images locally

If you wish to build container images locally, you can do so (an example for ubi8-py39):

podman build -f ubi8-py39/Dockerfile

Updating the dependencies

This script can be used for updating the requirements for each of the environments.

Execute command: ./update-requirements.sh <sub-directories>

Example: ./update-requirements.sh ubi8-py39 f34-py39

Prerequisite: Pipenv, micropipenv and different Python interpreter versions

Updating and releasing new container images

This repository is managed by Kebechet so updates of all the Python packages it uses are performed automatically. If you wish to release a new version of Thoth's s2i, you can do so by performing:

find -iname Dockerfile -exec sed -i 's/THOTH_S2I_VERSION=0.23.0/THOTH_S2I_VERSION=0.24.0/g' {} \; && git commit -m "Version 0.24.0" . && git tag v0.24.0

A subsequent pull request to this repository is needed.

S2I Assemble Patches

Most of the work when releasing a new container image comes from updating the s2i_assemble.patch file. This file dictates how to change the base image's s2i assemble script, usually available at /usr/libexec/s2i/assemble of a container running the base image, to provide the new image with everything it may need. As with any patch file, the s2i_assemble.patch can be generated manaually, however this is not recommend. The recommended methodology to create one is as follows:

1. create directories a and b in the root of the s2i-thoth repo.
2. find the assemble script of your base image.

• This starts by finding the base image. For instance, fedora base images are available at registry.fedoraproject.org/

• Either:

  1. locate the s2i/assemble script in source from wherever the image is built. This varies depending on what type of image it is. Copy this file to a/assemble.
  2. create a temporary Dockerfile using the base image that will print its s2i assemble script. Then build a new image (podman build .) using that base image, copy that output and save it to a/assemble. It is perfered that you can find it in source because any added whitespacing will not present an issue when you go to build the final image.

  • Dockerfile Example for f34:

  FROM registry.fedoraproject.org/f34/python3:0-31.container
  RUN cat /usr/libexec/s2i/assemble

3. Construct a new s2i assemble script in b/assemble. This should reflect what you want the final patched s2i assemble script to be.

• For reference on what your final assemble script should look like refer to any image overlay in this repository. Find its base image s2i assemble script as described above, and patch it with the s2i_assemble.patch in the corresponding directory, ex: patch a/assemble -i <OVERLAY_NAME>/s2i_assemble.patch. If this succeeds it will modify the base assemble script with your patch, and can serve as an example of what your final s2i assemble script should look like.
• NOTE: this may vary a lot by image, so thinnk about what the image chooses to include and remove rather than attempting to replicate it exactly for another overlay.

4. Run a diff between a/assemble and b/assemble, and the output should become your s2i_assemble.patch. Make sure to place it in the correct overlay. Example: diff a/assemble b/assemble > <OVERLAY_NAME>/s2i_assemble.patch.
5. Verify that your image builds correctly. Navigate to your overlay directory with the new Dockerfile, requirements.in, requirements.txt, and s2i_assemble.patch that you just generated. Run podman build . in the new overlay to verify that the image is buildable.

Importing image into OpenShift's registry

oc import-image quay.io/thoth-station/s2i-thoth-ubi8-py39 -n <your-namespace>
oc import-image quay.io/thoth-station/s2i-thoth-f34-py39 -n <your-namespace>