Comments (14)
@gaiborjosue - take a look at this section: https://apptainer.org/docs/user/latest/docker_and_oci.html#cmd-entrypoint-behaviour
i'll try to create a simple miniconda/mamba container with multiple environments to see whether i can support this.
yes, to build singularity you have to be root, or you can use a docker apptainer container in privileged mode to build the recipe
from trained-models.
I will take a look at that right now. Thank you for the resource.
from trained-models.
Hello @satra, I was just wondering if your container testing supports the multi environment activation. I tested with cmd and entrypoint but it still does not work for me. Thank you!
from trained-models.
just like this example from the link above:
# CMD="date"
# Runs 'date'
$ apptainer run mycontainer.sif
Wed 06 Oct 2021 02:45:39 PM CDT
# Runs 'echo hello'
$ apptainer run mycontainer.sif echo hello
hello
can't you simply do:
$ apptainer run mycontainer.sif /path/to/conda/env/bin/python ...
from trained-models.
requires knowing the environment name ahead of time..meaning we either have to parse the file for the name or activate an environment that is not base. I still think..standardizing it by installing in base environment is easy and maintainable.
from trained-models.
I guess I am addressing a different issue 😄 . I will let @gaiborjosue respond first.
from trained-models.
Hello, I agree with Harsha. That would require us having to parse the conda env name somehow.
from trained-models.
supports the multi environment activation
i thought your question was about multiple environments, not a single environment. a single environment can be made default with CMD
or ENTRYPOINT
as demonstrated in the examples on apptainer.
another way of controlling environments is to modify the default environment in which a script is executed through controlling shebang #!
. this can be done through a fit.sh
or predict.sh
script that wraps the python or other script the model provides. if the script wants different environments that's part of the ingest of script process. if it's simply a default environment then that can always be included in the CMD/ENTRYPOINT
.
from trained-models.
@satra Thanks for the response. Yes, my question derivated from the suggestion of allowing users to have multiple environments for their model's dockerfile. Just in case they have different use-cases for the same model. However, in the current structure, cli does not activate a specific environment (other than base) automatically. Therefore, the problem was that we don't have any way of knowing which environment the user wants to activate for that specific use case scenario.
fit.sh
orpredict.sh
Also, per my understanding, cli when building the image, can't/does not run any .sh file. Therefore, the ideal thing would be to do it everything inside the Dockerfile. Is this possible?
from trained-models.
cli when building the image
the cli doesn't build an image right, it uses the image built from the dockerfile. so if the dockerfile has a few commands added that creates a startup script that can use appropriate environments in the image that should be fine.
- a docker image is essentially an OS, it can have many environments in it, including different conda environments.
- when calling a script inside the instance of that image, one can both initialize and refer to any of those environments.
it may be useful to write down the exact sets of commands of what it is you are trying to do with a specific image and we can work through that together. all of this should simply be a manipulation of scripts and entrypoints.
from trained-models.
@gaiborjosue I guess this is resolved as well. Isn't it?
from trained-models.
@hvgazula Right now, we are using base env at startup. Will work on this again, sorry if I have put this issue on hold.
from trained-models.
See here for why singularity image cannot activate the conda environment on entry.
from trained-models.
See here for why singularity image cannot activate the conda environment on entry.
most of that response is about layer reduction, which is irrelevant to the notion of conda environment activation. the only two instruction types that are relevant are CMD
and ENTRYPOINT
.
please see my comments here on how to achieve this: #75 (comment) (or discuss why that cannot be implemented).
from trained-models.
Related Issues (20)
- Unable to run issues-helper@v3 in the workflow HOT 1
- Disable running action everytime an issue is commented/replied HOT 6
- Include a license file for every model that's included
- Push docker image to hub with tag HOT 2
- Path to data in test command should be absolute and relative
- cleanup singularity test command in new_model.yml and update_model.yml HOT 2
- Inconsistent actions/checkout version in new_model.yml
- the steps failed and success are duplicated except for the value and message in new_model.yml HOT 1
- fix(Singularity&Datalad): Fixed singularity command and datalad installation on EC2
- removing files from datalad/annex does not remove them from osf HOT 5
- datalad push fails to find storage targets in the add/update model workflow HOT 1
- New Model: PialNN/pialnn/1.0.0 HOT 4
- improve workflow to perform all operations on self-hosted runner HOT 7
- spec.yaml is not updated correctly HOT 5
- Fix(modelData): Fix the get model data workflow HOT 2
- Delete issue branch if workflow fails HOT 7
- Add model_card.yaml to current models
- Replace git checkout with datalad checkout in action workflows wherever appropriate HOT 6
- storing models in multiple remotes
- Add GRACE to model zoo
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trained-models.