conda-incubator / conda-env-builder Goto Github PK

We already mention it in the scala docs here:

And it's part of the conda doc recommendations here: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment

We'd want to support opting out if we make it the default, which could be inelegant.

Thoughts?

This is not an issue. I am just curious about the motivation of this project. I also suffered issues when installing bio tools inside the same namespace, for example, if I install GATK4/Picard requiring openjdk8 first, then install Trinity requiring openjdk7, the default higher version of JDK is overwritten by the older version, then Picard is corrupted. Basically, this can be solved via installing Trinity first then Picard, at last, the openjdk8 will be kept and there will be no issues.

Did you know the following project:
https://github.com/conda-tools/conda-execute

I'd like to help if you can share with me more about the issues encountered.
Thanks!

I would expect that the error would be "inherited environment samtools does not exist". Instead we get "Found a cyclical dependency in the environment inheritance."

Input YAML

name: example
environments:
  defaults:
    steps:
      - conda:
          channels:
            - conda-forge
            - bioconda
          requirements:
            - bwa=0.7.17
            - python=3.6.10
  bwa:
    group: alignment
    inherits:
      - samtools
    steps:
      - conda:
          requirements:
            - samtools
            - bwa

conda-env-builder Compile -c config.yaml 
...
[2020/12/15 21:31:19 | CondaEnvironmentBuilderMain | Info] Executing Compile from conda-env-builder version 010c473 as [email protected] on JRE 1.8.0_192-b01
[2020/12/15 21:31:19 | CondaEnvironmentBuilderMain | Info] Compile failed. Elapsed time: 0.02 minutes.
Exception in thread "main" java.lang.IllegalArgumentException: Found a cyclical dependency in the environment inheritance.
	at com.github.nh13.condaenvbuilder.api.Spec$.com$github$nh13$condaenvbuilder$api$Spec$$compile(Spec.scala:160)
	at com.github.nh13.condaenvbuilder.api.Spec.compiled(Spec.scala:22)
	at com.github.nh13.condaenvbuilder.tools.Compile.execute(Compile.scala:30)
	at com.github.nh13.condaenvbuilder.cmdline.CondaEnvironmentBuilderMain.makeItSo(CondaEnvironmentBuilderMain.scala:62)
	at com.github.nh13.condaenvbuilder.cmdline.CondaEnvironmentBuilderMain.makeItSoAndExit(CondaEnvironmentBuilderMain.scala:47)
	at com.github.nh13.condaenvbuilder.cmdline.CondaEnvironmentBuilderMain$.main(CondaEnvironmentBuilderMain.scala:20)
	at com.github.nh13.condaenvbuilder.cmdline.CondaEnvironmentBuilderMain.main(CondaEnvironmentBuilderMain.scala)

conda clean -y --tarballs --packages

I have found platform specifications to be a useful feature in conda-lock, as it provides an option for locking dependencies per environment specified. It would be great if conda-env-builder could also include this option.

Currently, conda-env-builder does not support platform specifications in the environment.yaml file. This requires developers to manually add platform specifications, which can be a time-consuming process and could potentially lead to errors.

I understand that adding new features can be a complex process, but I wanted to request that this feature be considered for future development. Thank you for your work on this tool.

See: https://github.com/conda-incubator/conda-lock. We could then support multi-arch lock files.

Possible Implementation

The Solve command would have an option --conda-lock=<arch> that would run:

conda-lock lock -f <env-name>.yaml -p <arch> --lockfile <env-name>-conda-lock.yml
conda-lock render --kind explicit --filename-template <env-name>-<arch>.lock <env-name>-conda-lock.yml

The <env-name>-<arch>.lock would then be parsed so that the URL/hash given for conda packages is stored in the version string in the output YAML, while for pip packages, everything after the # pip <package-name> @.

Then Assemble would generate a <env>-<arch>.lock file instead of a <env>.yaml for <env-name>.build-conda.sh to use. The latter would use conda-lock install --name <env> <env>-<arch>.lock. Does Assemble know it's a conda-lock version if the "version" is a URL, or should we have an explicit option --conda-lock=<arch>? If the latter, must the architecture be encoded in the filename?

Note: both the conda-lock render and install commands use the --mamba option in conda-lock lock, so set those if --mamba given to conda-env-bulider