eic / eic-shell

EIC software environment container utilities

EIC software environment container utilities

Running in browser

Local Installation

The environment has been tested on linux (requires singularity v3+ or apptainer v1+) and MacOS (requires docker)

Please follow the steps below to setup and run the container in your environment.

1. Create a local directory that you want to work in, e.g $HOME/eic, and go into this directory.

mkdir $HOME/eic
cd $HOME/eic

2. Execute the following line in your terminal to setup your environment in this directory to install the latest stable container

curl -L https://github.com/eic/eic-shell/raw/main/install.sh | bash

3. You can now load your development environment by executing the eic-shell script that is in your top-level working directory.

./eic-shell

4. Within your development environment (eic-shell), you can install software to the internal prefix $EIC_SHELL_PREFIX

Installation for Development Usage

Note: this container download script is meant for expert usage. If it is unclear to you why you would want to do this, you are probably looking for the simple installation above.

You can use the same install scripts to setup other container setups, including jug_dev (the main development container). Note that for jug_dev there is no nighlty release, and the appropriate version (tag) would be testing. To setup the jug_dev:testing environment, do

curl -L https://github.com/eic/eic-shell/raw/main/install.sh | bash -s -- -c jug_dev -v testing

Included software:

• Included software (for the exact versions, use the command eic-info inside the container):
  • gcc
  • madx
  • cmake
  • fmt cxxstd=17
  • spdlog
  • nlohmann-json
  • heppdt
  • clhep cxxstd=17
  • eigen
  • python
  • py-numpy
  • py-pip
  • pkg-config
  • xrootd cxxstd=17 +python
  • root cxxstd=17 +fftw +fortran +gdml +http +mlp +pythia8 +root7 +tmva +vc +xrootd +ssl ^mesa swr=none +opengl -llvm -osmesa
  • pythia8 +fastjet
  • fastjet
  • hepmc3 +python +rootio
  • stow
  • cairo +fc+ft+X+pdf+gobject
  • podio
  • geant4 cxxstd=17 +opengl +vecgeom +x11 +qt +threads ^qt +opengl
  • dd4hep +geant4 +assimp +hepmc3 +ipo +lcio
  • acts +dd4hep +digitization +identification +json +tgeo +ipo +examples +fatras +geant4
  • genfit
  • gaudi
  • dawn
  • dawncut
  • opencascade
  • emacs toolkit=athena
  • imagemagick
  • igprof
• The singularity build exports the following applications:
  • eic-shell: a development shell in the image

Using the docker container for your CI purposes

These instructions are old and need updating. In general we recommend using eicweb/juggler:latest for most CI usages. This image is functionally identical to jug_xl:nightly

The docker containers are publicly accessible from Dockerhub. You probably want to use the default jug_xl container. Relevant versions are:

• eicweb/jug_xl:nightly: nightly release, with latest detector and reconstruction version. This is probably what you want to use unless you are dispatching a large simulation/reconstruciton job
• eicweb/jug_xl:3.0-stable: latest stable release, what you want to use for large simulation jobs (for reproducibility). Please coordinate with the software group to ensure all desired software changes are present in this container.

1. To load the container environment in your run scripts, you have to do nothing special.
   The environment is already setup with good defaults, so you can use all the programs in the container as usual and assume everything needed to run the included software is already setup.

2. If using this container as a basis for a new container, you can direction access the full container environment from a docker RUN shell command with no further action needed. For the most optimal experience, you can install your software to /usr/local to fully integrate with the existing environment.

eic-shell's Issues

An apparent bug in the eic-shell installation script

After running the eic-shell installation script (which is currently possible only on systems with Singularity -- subject to another issue report), the /usr/local contains '${name}.jar'. This certainly looks like a bug.

Running local version option

Hi @wdconinc,

I would like to know if the shell allows to create an image of a local version of the container with a modified spack yaml.

Support multiple containers in eic-shell

Right now we can only support a single container at a time in eic-shell. As we develop additional environments (focused on AI/ML, production, etc), we should have a way to run those seamlessly, e.g. eic-shell --container ml --version 23.04.0. This, in turn, means that the installation and validation functionality needs to go into eic-shell, not install.sh. In fact, it means that install.sh would ONLY install eic-shell, branch to docker vs singularity,, and then use eic-shell for the actual container pulling with argument passthrough.

Steps:

• move from $1 and shift to using getopt so we don't eat all the arguments
• eat those arguments that are not supported by eic-shell (prefix etc), but keep the others
• pass $@ to install_singularity and install_docker
• move the image installation into the bundled eic-shell script
• remove support for singularity version 2
• maintain the eic-shell for singularity and for docker as separate scripts in this repo (allowing CI)
• merge scripts in CI publication to the eic.github.io/eic-shell location

Ensure Docker usage is possible while using eic-shell (it's not currently)

In the tutorial, Docker and Singularity are mentioned on same footing, i.e. a reasonable person would assume that either would work with the image(s) provided. However, if Singularity is not present on the system, the installation will fail. Proposal: make the choice of Docker vs Singularity explicit at install time.

Reasoning -- making Singularity a hard requirement limits the scope of usage of eic-shell. Many users are heavily invested into Docker on their workstations and laptops and won't want to install yet another container system, with all sorts of related bloat.

Add warning when installing custom container

Is your feature request related to a problem? Please describe.
On linux, singularity pull creates a multi-GB cache in ~/.singularity

Describe the solution you'd like
That's easy to avoid, just set SINGULARITY_CACHEDIR. The user should know this.
I would just add "This can add a large cache file in your $HOME, to avoid export (or setenv) SINGULARITY_CACHEDIR to the desired one" to the -n, -c, and -v options. I'd be happy to do that, but it's an. unwieldy sentence, it maybe best to group these three and have the sentence just once before them ("The following options...").
Let me know which way, I'll make the PR

Describe alternatives you've considered
I mean, one could add another option and set the variable inside the script, but that seems like overkill.

Additional context
Add any other context or screenshots about the feature request here.

Adding EOS folder to eic-shell

Is your feature request related to a problem? Please describe.
When setting up eic-shell on lxplus.cern.ch, the mounted folder /eos is removed.

Describe the solution you'd like
Add /eos to install.sh (L186)