wtsi-npg / containers Goto Github PK
View Code? Open in Web Editor NEWSingularity and Docker material
Singularity and Docker material
The helper script uses singularity from PATH which may not be compatible with the singularity used when the wrappers were deployed. Maybe consider keeping in the wrapper the resolved path to singularity used for deploying.
containers/.github/workflows/pushing_container_image.yml
Lines 44 to 48 in 58f1773
The new images should also be tagged "latest".
GLIBC error from node, as seen in workflow.
v3 will be deprecated in November
Consider singularity build --fakeroot
in the future?
Originally posted by @dkj in #12 (comment)
All wrapper scripts rely on the colocated singularity-service-docker
. Which means we cannot create links to the wrappers of multiple containers in one path. So if we have a container for samtools and one for irods clients, the structure would look like:
samtools/
└── bin
├── samtools
...
└── singularity-service-docker
irods-clients/
└── bin
├── baton-chmod
...
└── singularity-service-docker
So it would not be possible to link samtools/bin/*
and irods-clients/bin/*
in a single path. As the different singularity-service-docker
, one from samtools and one from irods-clients, may not be compatible.
This is a maintenance issue to remind us to review regularly our Conda package build environment, which provides glibc. Currently CentOS 7 is used.
These conda-forge issues provide some helpful context:
conda-forge/conda-forge.github.io#1436
conda-forge/conda-forge.github.io#1432
Currently the singularity-*-docker
wrapper scripts produced when installing, will pull the image from docker or rely on the singluarity cache. But it should be possible to use a .sif
file saved locally to execute.
So instead of using
singularity exec/run "docker://$DOCKER_REGISTRY/$DOCKER_USER/$DOCKER_IMAGE:$DOCKER_TAG" "$@"
We would like to be able to do ask the wrapper to do
singularity exec/run "${CONTAINER}" "$@"
where $CONTAINER is either a docker://...
or a <path to local .sif file>
and the behaviour is controlled by the calling wrapper executable. Essentially define where to find the image at run time when executing write_wrapper
.
Described here:
https://wiki.debian.org/DebianRepository/UseThirdParty
This particularly refers to the iRODS apt repository, which is used in several builds.
This was a dependency of the Python scripts in ./tools/bin/
, but not any longer.
The Action contains a hard-coded list of containers to push, while a list is already generated by the Makefile. The Action should run the Makefile, rather than have its own list.
https://github.com/wtsi-npg/containers/actions/runs/961330211
Third party actions are a potential attack surface and may also be modified (re-tagged with the same version), so are disabled. We currently trust GitHub's Actions (as we trust GitHub) and our own respositories. Solutions
We use git describe
to generate Docker image tags when building from a tagged release. Even when correctly tagged, git describe
is not finding the annotated tag on the release commit, when checked out by https://github.com/actions/checkout/ v2
Add a changelog and backdate its contents to include all existing releases.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.