wtsi-npg / containers Goto Github PK

The helper script uses singularity from PATH which may not be compatible with the singularity used when the wrappers were deployed. Maybe consider keeping in the wrapper the resolved path to singularity used for deploying.

The new images should also be tagged "latest".

GLIBC error from node, as seen in workflow.
v3 will be deprecated in November

Consider singularity build --fakeroot in the future?

Originally posted by @dkj in #12 (comment)

See https://github.com/wtsi-npg/containers/actions/runs/963651795

All wrapper scripts rely on the colocated singularity-service-docker. Which means we cannot create links to the wrappers of multiple containers in one path. So if we have a container for samtools and one for irods clients, the structure would look like:

samtools/
└── bin
    ├── samtools
    ...
    └── singularity-service-docker

irods-clients/
└── bin
    ├── baton-chmod
    ...
    └── singularity-service-docker

So it would not be possible to link samtools/bin/* and irods-clients/bin/* in a single path. As the different singularity-service-docker, one from samtools and one from irods-clients, may not be compatible.

This is a maintenance issue to remind us to review regularly our Conda package build environment, which provides glibc. Currently CentOS 7 is used.

These conda-forge issues provide some helpful context:

conda-forge/conda-forge.github.io#1436
conda-forge/conda-forge.github.io#1432

Currently the singularity-*-docker wrapper scripts produced when installing, will pull the image from docker or rely on the singluarity cache. But it should be possible to use a .sif file saved locally to execute.

So instead of using
singularity exec/run "docker://$DOCKER_REGISTRY/$DOCKER_USER/$DOCKER_IMAGE:$DOCKER_TAG" "$@"

We would like to be able to do ask the wrapper to do
singularity exec/run "${CONTAINER}" "$@"
where $CONTAINER is either a docker://... or a <path to local .sif file> and the behaviour is controlled by the calling wrapper executable. Essentially define where to find the image at run time when executing write_wrapper
.

Described here:

https://wiki.debian.org/DebianRepository/UseThirdParty

This particularly refers to the iRODS apt repository, which is used in several builds.

See wtsi-npg/perl-irods-wrap#192

This was a dependency of the Python scripts in ./tools/bin/, but not any longer.

The Action contains a hard-coded list of containers to push, while a list is already generated by the Makefile. The Action should run the Makefile, rather than have its own list.

https://github.com/wtsi-npg/containers/actions/runs/961330211

Third party actions are a potential attack surface and may also be modified (re-tagged with the same version), so are disabled. We currently trust GitHub's Actions (as we trust GitHub) and our own respositories. Solutions

• Whitelist the Action
• Use an alternative Action
• Fork the Action

We use git describe to generate Docker image tags when building from a tagged release. Even when correctly tagged, git describe is not finding the annotated tag on the release commit, when checked out by https://github.com/actions/checkout/ v2

Add a changelog and backdate its contents to include all existing releases.