lib/R/etc/ldpaths: No such file or directory about r-base-feedstock

[lecorguille]

Hi,
We're facing the same issue with our Galaxy instance.

/some_path/_conda/envs/mulled-v1-10760b69799f3df14ac223943c74682453f4c44945480a6c4ee3ccaff1510f7f/lib/R/etc/ldpaths: No such file or directory

I have to reinstall r-base within the env:

conda install r-base==3.5.1 --force-reinstall

But I don't know how many time it will be up.

So as suggested by @pbordron (we know each other), I edited the bash script

vi /some_path/_conda/envs/mulled-v1-10760b69799f3df14ac223943c74682453f4c44945480a6c4ee3ccaff1510f7f/etc/conda/activate.d/activate-r-base.sh

#!/usr/bin/env sh
#R CMD javareconf > /dev/null 2>&1 || true

It should be harmless in the context of Galaxy since we never stack env. 🤞

(FYI: @fgiacomoni)

[Fedorov113]

I ran into this issue with Snakemake. I use it with --use-conda flag, and when running parallel jobs with R, at some point ldpaths file just disappears. Nothing helped me to overcome this yet, really waiting for the solution.

[nehaljwani]

I hit this problem when hundreds of processes tried to activated a single read only conda environment and spawned hundreds of Java processes, which ended up consuming system resources.

I have a suggestion for this situation which follows up on @mbargull 's second point:

Let the user take care of running `javareconf.

And it is least disruptive to the current setup, because it doesn't change the the status quo.

In the activation script, introduce an env var like CONDA_R_IGNORE_JAVARECONF. If this is set, then the activation script shouldn't run R CMD javareconf > /dev/null 2>&1 || true.
That way end users of clusters (where environments don't change that often or have nothing to do with Java) can simply export this env var and activation with be quicker and there will be no mayhem.

[mingwandroid]

IMHO, I think a way to solve this issue is to replace the file only when needed, not at each environment activation.

This is exactly when the file needs to be updated.

Can you not mount your envs on another file system? NFS is often problematic.

My only suggestion would be to try to minimize these time windows. You could copy the original ldpaths to e.g. ldpaths.$$, modify that, then replace ldpaths with ldpaths.$$. This could be coupled with a loop to make sure ldpaths exists at the point of consumption with a limit on the number of times it spins waiting for it to appear.

Please feel free to submit a PR. Personally I cannot reproduce your error since I do not use NFS.

[pbordron]

The point is that R CMD javareconf replaces ldpaths file each time it is called, even if there is no change (see the script here)

The best solution will be to have a ldpaths file that is proper to each conda env activation, but it is not so simple.

An easy but yet incomplete solution can be to modify the following patch such as the files are only replaced when the new file is different.

Here a proposition that is haven't tested:
https://github.com/pbordron/r-base-feedstock/blob/d6aef51add58ad802f5d04467504134bf31817ac/recipe/0017-Foribly-remove-then-forcibly-mv-in-javareconf.in.patch

[mingwandroid]

The best solution will be to have a ldpaths file that is proper to each conda env activation, but it is not so simple.

It is not so simple at all, in fact it's impossible. There is no such thing as a 'file that is proper to each conda env' because we deliberately support a range of Javas both from conda packages, from the system package manager and 3rd party. I have gone to extensive lengths to make this work.

You've replaced a race condition around ldpaths with one around ldpaths.new, an improvement but my suggestion is still better. You can get the best of both worlds by replacing .new in your patch with .new.$$

Please work on a PR if you want this to work on nfs.

[johanneskoester]

@pbordron it is indeed important that conda packages work on NFS. If I can help you with the PR, please let me know. I won't have the time to work on it on my own, but I would be happy to help in any way. Thanks a lot for pointing this out!

[pbordron]

From now, I do not have the time nor the environment for looking for a fix.
The problem does not only append with NFS, but also with BeeGFS.
The fact that the ldpaths is replaced in two times is the main cause (i.e file ldpaths is removed and then ldpaths.new is moved to ldpaths).

[RodrigoGM]

Hello,
I've recently encountered this error in my miniconda3 as well. I have a brand new miniconda3 install due to a corrupt libevent file last week. In the new install, the error

~/opt/miniconda3/bin/R: line 238: ~/opt/miniconda3/lib/R/etc/ldpaths: No such file or directory

began once I created a new environment. I also use a cluster with CentOS_6 and CentOS_7 nodes, all under LSF. Is there a any other way to fix to this issue other than the patch mentioned above ? Any other ideas on how to fix the issue ?

Many thanks in advance,

base environment

(base) bash-4.1$ conda list

# packages in environment at $HOME/opt/miniconda3:
#
# Name                    Version                   Build  Channel
_r-mutex                  1.0.0               anacondar_1    defaults
asn1crypto                0.24.0                   py37_0    defaults
binutils_impl_linux-64    2.31.1               h6176602_1    conda-forge
binutils_linux-64         2.31.1               h6176602_3    conda-forge
bowtie                    1.0.0                         1    bioconda
bwa                       0.7.17               h84994c4_4    bioconda
bwidget                   1.9.11                        1    defaults
byobu                     5.98                 hb42da9c_2    bioconda
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2018.11.29           ha4d7672_0    conda-forge
cairo                     1.14.12              h8948797_3    defaults
certifi                   2018.11.29            py37_1000    conda-forge
cffi                      1.11.5           py37he75722e_1    defaults
chardet                   3.0.4                    py37_1    defaults
conda                     4.6.2                    py37_0    conda-forge
conda-env                 2.6.0                         1    defaults
cryptography              2.4.2            py37h1ba5d50_0    defaults
curl                      7.62.0               hbc83047_0    defaults
emacs                     26.1              he9320e1_1002    conda-forge
fontconfig                2.13.0               h9420a91_0    defaults
freetype                  2.9.1             h94bbf69_1005    conda-forge
fribidi                   1.0.5             h14c3975_1000    conda-forge
gcc_impl_linux-64         7.3.0                habb00fd_1    conda-forge
gcc_linux-64              7.3.0                h553295d_3    conda-forge
gettext                   0.19.8.1          h9745a5d_1001    conda-forge
gfortran_impl_linux-64    7.3.0                hdf63c60_1    conda-forge
gfortran_linux-64         7.3.0                h553295d_3    conda-forge
giflib                    5.1.4             h14c3975_1001    conda-forge
glib                      2.56.2            had28632_1001    conda-forge
gnutls                    3.6.5             hd3a4fd2_1001    conda-forge
graphite2                 1.3.13            hf484d3e_1000    conda-forge
gsl                       2.4                  h14c3975_4    defaults
gxx_impl_linux-64         7.3.0                hdf63c60_1    conda-forge
gxx_linux-64              7.3.0                h553295d_3    conda-forge
harfbuzz                  1.9.0             he243708_1001    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
idna                      2.8                      py37_0    defaults
jpeg                      9c                h14c3975_1001    conda-forge
krb5                      1.16.1               h173b8e3_7    defaults
ldc                       1.11.0               hb2c9046_0    conda-forge
libcurl                   7.62.0               h20c2e04_0    defaults
libdeflate                1.0                  h14c3975_1    bioconda
libedit                   3.1.20170329         h6b74fdf_2    defaults
libevent                  2.0.22                        0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 8.2.0                hdf63c60_1    defaults
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libiconv                  1.15              h14c3975_1004    conda-forge
libpng                    1.6.36            h84994c4_1000    conda-forge
libssh2                   1.8.0                         1    conda-forge
libstdcxx-ng              8.2.0                hdf63c60_1    defaults
libtiff                   4.0.10            h648cc4a_1001    conda-forge
libuuid                   1.0.3                         1    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxml2                   2.9.8        
newt                      0.52.20         py37h14c3975_1000    conda-forge
oniguruma                 6.9.0             h14c3975_1001    conda-forge
openssl                   1.1.1a            h14c3975_1000    conda-forge
pango                     1.42.4               h049681c_0    defaults
pcre                      8.42                 h439df22_0    defaults
perl                      5.26.2            h14c3975_1001    conda-forge
pip                       18.1                     py37_0    defaults
pixman                    0.34.0            h14c3975_1003    conda-forge
popt                      1.16                          1    bioconda
pthread-stubs             0.4               h14c3975_1001    conda-forge
pycosat                   0.6.3            py37h14c3975_0    defaults
pycparser                 2.19                     py37_0    defaults
pyopenssl                 18.0.0                   py37_0    defaults
pysocks                   1.6.8                    py37_0    defaults
python                    3.7.1                h0371630_7    defaults
r-assertthat              0.2.0            r351h6115d3f_0    r
r-backports               1.1.2            r351h96ca727_0    r
r-base                    3.5.1                h1e0a451_2    r
r-base64enc               0.1_3            r351h96ca727_4    r
r-bh                      1.66.0_1         r351h6115d3f_0    r
r-bindr                   0.1.1            r351h6115d3f_0    r
r-bindrcpp                0.2.2            r351h29659fb_0    r
r-broom                   0.5.0            r351h6115d3f_0    r
r-callr                   2.0.4            r351h6115d3f_0    r
r-cellranger              1.1.0            r351h6115d3f_0    r
r-cli                     1.0.0            r351h6115d3f_1    r
r-clipr                   0.4.1            r351h6115d3f_0    r
r-colorspace              1.3_2            r351h96ca727_0    r
r-crayon                  1.3.4            r351h6115d3f_0    r
r-curl                    3.2              r351hadc6856_1    r
r-dbi                     1.0.0            r351h6115d3f_0    r
r-dbplyr                  1.2.2            r351hf348343_0    r
r-dichromat               2.0_0            r351h6115d3f_4    r
r-digest                  0.6.15           r351h96ca727_0    r
r-dplyr                   0.7.6            r351h29659fb_0    r
r-evaluate                0.11             r351h6115d3f_0    r
r-fansi                   0.2.3            r351h96ca727_0    r
r-forcats                 0.3.0            r351h6115d3f_0    r
r-ggplot2                 3.0.0            r351h6115d3f_0    r
r-glue                    1.3.0            r351h96ca727_0    r
r-gtable                  0.2.0            r351h6115d3f_0    r
r-haven                   1.1.2            r351h29659fb_0    r
r-highr                   0.7              r351h6115d3f_0    r
r-hms                     0.4.2            r351h6115d3f_0    r
r-htmltools               0.3.6            r351h29659fb_0    r
r-httr                    1.3.1            r351h6115d3f_1    r
r-jsonlite                1.5              r351h96ca727_0    r
r-knitr                   1.20             r351h6115d3f_0    r
r-labeling                0.3              r351h6115d3f_4    r
r-lattice                 0.20_35          r351h96ca727_0    r
r-lazyeval                0.2.1            r351h96ca727_0    r
r-lubridate               1.7.4            r351h29659fb_0    r
r-magrittr                1.5              r351h6115d3f_4    r
r-markdown                0.8              r351h96ca727_0    r
r-mass                    7.3_50           r351h96ca727_0    r
r-matrix                  1.2_14           r351h96ca727_0    r
r-mgcv                    1.8_24           r351h96ca727_0    r
r-mime                    0.5              r351h96ca727_0    r
r-mo
r-openssl                 1.0.2            r351h96ca727_1    r
r-pillar                  1.3.0            r351h6115d3f_0    r
r-pkgconfig               2.0.1            r351h6115d3f_0    r
r-plogr                   0.2.0            r351h6115d3f_0    r
r-plyr                    1.8.4            r351h29659fb_0    r
r-praise                  1.0.0            r351h6115d3f_4    r
r-processx                3.1.0            r351h29659fb_0    r
r-purrr                   0.2.5            r351h96ca727_0    r
r-r6                      2.2.2            r351h6115d3f_0    r
r-rcolorbrewer            1.1_2            r351h6115d3f_0    r
r-rcpp                    0.12.18          r351h29659fb_0    r
r-readr                   1.1.1            r351h29659fb_0    r
r-readxl                  1.1.0            r351h29659fb_0    r
r-rematch                 1.0.1            r351h6115d3f_0    r
r-reprex                  0.2.0            r351h6115d3f_0    r
r-reshape2                1.4.3            r351h29659fb_0    r
r-rlang                   0.2.1            r351h96ca727_0    r
r-rmarkdown               1.10             r351h6115d3f_0    r
r-rprojroot               1.3_2            r351h6115d3f_0    r
r-rstudioapi              0.7              r351h6115d3f_0    r
r-rvest                   0.3.2            r351h6115d3f_0    r
r-scales                  0.5.0            r351h29659fb_0    r
r-selectr                 0.4_1            r351h6115d3f_0    r
r-stringi                 1.2.4            r351h29659fb_0    r
r-stringr                 1.3.1            r351h6115d3f_0    r
r-testthat                2.0.0            r351h29659fb_0    r
r-tibble                  1.4.2            r351h96ca727_0    r
r-tidyr                   0.8.1            r351h29659fb_0    r
r-tidyselect              0.2.4            r351h29659fb_0    r
r-tidyverse               1.2.1            r351h6115d3f_0    r
r-tinytex                 0.6              r351h6115d3f_0    r
r-utf8                    1.1.4            r351h96ca727_0    r
r-viridislite             0.3.0            r351h6115d3f_0    r
r-whisker                 0.3_2            r351hf348343_4    r
r-withr                   2.1.2            r351h6115d3f_0    r
r-xfun                    0.3              r351h6115d3f_0    r
r-xml2                    1.2.0            r351h29659fb_0    r
r-yaml                    2.2.0            r351h96ca727_0    r
readline                  7.0                  h7b6447c_5    defaults
requests                  2.21.0                   py37_0    defaults
ruamel_yaml               0.15.46          py37h14c3975_0    defaults
sambamba                  0.6.8                h682856c_1    bioconda
samtools                  1.9                  h91753b0_5    bioconda
setuptools                40.6.3                   py37_0    defaults
six                       1.12.0                   py37_0    defaults
slang                     2.3.2             hbb9399a_1000    conda-forge
sqlite                    3.26.0               h7b6447c_0    defaults
star                      2.6.1b                        0    bioconda
tk                        8.6.8                hbc83047_0    defaults
tktable                   2.10                 h14c3975_0    defaults
tmux                      2.7               h87f082e_1003    conda-forge
urllib3                   1.24.1                   py37_0    defaults
wheel                     0.32.3                   py37_0    defaults
xorg-fixesproto           5.0               h14c3975_1002    conda-forge
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.9             h14c3975_1004    conda-forge
xorg-libsm                1.2.2                h470a237_5    conda-forge
xorg-libx11               1.6.7             h14c3975_1000    conda-forge
xorg-libxau               1.0.8             h14c3975_1006    conda-forge
xorg-libxaw               1.0.13            h14c3975_1002    conda-forge
xorg-libxdmcp             1.1.2             h14c3975_1007    conda-forge
xorg-libxext              1.3.3             h14c3975_1004    conda-forge
xorg-libxfixes            5.0.3             h14c3975_1004    conda-forge
xorg-libxft               2.3.2                h7c24a56_3    conda-forge
xorg-libxmu               1.1.2             h14c3975_1002    conda-forge
xorg-libxpm               3.5.12            h14c3975_1002    conda-forge
xorg-libxrender           0.9.10            h14c3975_1002    conda-forge
xorg-libxt                1.1.5             h14c3975_1002    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h14c3975_1007    conda-forge
xz                        5.2.4                h14c3975_4    defaults
yaml                      0.1.7                had09818_2    defaults
zlib                      1.2.11               h7b6447c_3    defaults

second environment

(base) bash-4.1$ conda list -n cnv

(base) bash-4.1$ conda list -n cnv
# packages in environment at $HOME/opt/miniconda3/envs/cnv:
#
# Name                    Version                   Build  Channel
_r-mutex                  1.0.0               anacondar_1    defaults
binutils_impl_linux-64    2.31.1               h6176602_1    conda-forge
binutils_linux-64         2.31.1               h6176602_3    conda-forge
bioconductor-biobase      2.42.0           r351h14c3975_0    bioconda
bioconductor-biocgenerics 0.28.0                   r351_1    bioconda
bioconductor-biocparallel 1.16.2           r351h1c2f66e_1    bioconda
bioconductor-biostrings   2.50.1           r351h14c3975_0    bioconda
bioconductor-cghbase      1.42.0                   r351_0    bioconda
bioconductor-cghcall      2.44.0                   r351_0    bioconda
bioconductor-dnacopy      1.56.0           r351h9ac9557_0    bioconda
bioconductor-genomeinfodb 1.18.1                   r351_0    bioconda
bioconductor-genomeinfodbdata 1.2.0                    r351_0    bioconda
bioconductor-genomicranges 1.34.0           r351h14c3975_0    bioconda
bioconductor-impute       1.56.0           r351h9ac9557_0    bioconda
bioconductor-iranges      2.16.0           r351h14c3975_0    bioconda
bioconductor-limma        3.38.3           r351h14c3975_0    bioconda
bioconductor-marray       1.60.0                   r351_0    bioconda
bioconductor-qdnaseq      1.18.0                   r351_0    bioconda
bioconductor-rsamtools    1.34.0           r351hf484d3e_0    bioconda
bioconductor-s4vectors    0.20.1           r351h14c3975_0    bioconda
bioconductor-xvector      0.22.0           r351h14c3975_0    bioconda
bioconductor-zlibbioc     1.28.0           r351h14c3975_0    bioconda
bwidget                   1.9.11                        1    defaults
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2018.11.29           ha4d7672_0    conda-forge
cairo                     1.14.12           h80bd089_1005    conda-forge
curl                      7.63.0            h646f8bb_1000    conda-forge
fontconfig                2.13.1            h2176d3f_1000    conda-forge
freetype                  2.9.1             h94bbf69_1005    conda-forge
gcc_impl_linux-64         7.3.0                habb00fd_1    conda-forge
gcc_linux-64              7.3.0                h553295d_3    conda-forge
gettext                   0.19.8.1          h9745a5d_1001    conda-forge
gfortran_impl_linux-64    7.3.0                hdf63c60_1    conda-forge
gfortran_linux-64         7.3.0                h553295d_3    conda-forge
glib                      2.56.2            had28632_1001    conda-forge
graphite2                 1.3.13            hf484d3e_1000    conda-forge
gxx_impl_linux-64         7.3.0                hdf63c60_1    conda-forge
gxx_linux-64              7.3.0                h553295d_3    conda-forge
harfbuzz                  1.9.0             he243708_1001    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
krb5                      1.16.3            hc83ff2d_1000    conda-forge
libcurl                   7.63.0            h01ee5af_1000    conda-forge
libedit                   3.1.20170329      hf8c457e_1001    conda-forge
libffi                    3.2.1             hf484d3e_1005    conda-forge
libgcc-ng                 7.3.0                hdf63c60_0    conda-forge
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libiconv                  1.15              h14c3975_1004    conda-forge
libpng                    1.6.36            h84994c4_1000    conda-forge
libssh2                   1.8.0             h1ad7b7a_1003    conda-forge
libstdcxx-ng              7.3.0                hdf63c60_0    conda-forge
libtiff                   4.0.10            h648cc4a_1001    conda-forge
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxml2                   2.9.8             h143f9aa_1005    conda-forge
make                      4.2.1             h14c3975_2004    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
openssl                   1.0.2p            h14c3975_1002    conda-forge
pango                     1.40.14           hf0c64fd_1003    conda-forge
pcre                      8.41              hf484d3e_1003    conda-forge
pixman                    0.34.0            h14c3975_1003    conda-forge
pthread-stubs             0.4               h14c3975_1001    conda-forge
r-base                    3.5.1             he45234b_1005    conda-forge
r-bh                      1.66.0_1         r351h6115d3f_0    r
r-bitops                  1.0_6            r351h96ca727_4    r
r-formatr                 1.5              r351h6115d3f_0    r
r-futile.logger           1.4.3           r351h6115d3f_1001    conda-forge
r-futile.options          1.0.1           r351h6115d3f_1000    conda-forge
r-lambda.r                1.2.3           r351h6115d3f_1000    conda-forge
r-matrixstats             0.54.0          r351h96ca727_1000    conda-forge
r-r.methodss3             1.7.1            r351h6115d3f_0    r
r-r.oo                    1.22.0           r351h6115d3f_0    r
r-r.utils                 2.6.0            r351h6115d3f_0    r
r-rcurl                   1.95_4.11        r351h96ca727_0    r
r-snow                    0.4_3           r351h6115d3f_1000    conda-forge
r-snowfall                1.84_6.1        r351h6115d3f_1001    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
tk                        8.6.9             h84994c4_1000    conda-forge
tktable                   2.10                 h14c3975_0    defaults
wget                      1.19.5               h1ad7b7a_0    defaults
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.9             h14c3975_1004    conda-forge
xorg-libsm                1.2.3             h4937e3b_1000    conda-forge
xorg-libx11               1.6.7             h14c3975_1000    conda-forge
xorg-libxau               1.0.8             h14c3975_1006    conda-forge
xorg-libxdmcp             1.1.2             h14c3975_1007    conda-forge
xorg-libxext              1.3.3             h14c3975_1004    conda-forge
xorg-libxrender           0.9.10            h14c3975_1002    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h14c3975_1007    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
zlib                      1.2.11            h14c3975_1004    conda-forge

conda install info

(base) bash-4.1$ conda info

     active environment : base
    active env location : $HOME/opt/miniconda3
            shell level : 1
       user config file : $HOME/.condarc
 populated config files : $HOME/.condarc
          conda version : 4.6.2
    conda-build version : not installed
         python version : 3.7.1.final.0
       base environment : $HOME/opt/miniconda3  (writable)
           channel URLs : https://conda.anaconda.org/r/linux-64
                          https://conda.anaconda.org/r/noarch
                          https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://conda.anaconda.org/dranew/linux-64
                          https://conda.anaconda.org/dranew/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : $HOME/opt/miniconda3/pkgs
                          $HOME/.conda/pkgs
       envs directories : $HOME/opt/miniconda3/envs
                          $HOME/.conda/envs
               platform : linux-64
             user-agent : conda/4.6.2 requests/2.21.0 CPython/3.7.1 Linux/2.6.32-696.3.2.el6.x86_64 centos/6.9 glibc/2.12
                UID:GID : ...:...
             netrc file : $HOME/.netrc
           offline mode : False

[pbordron]

@RodrigoGM the patch is not tested and replace a race condition with another.

[NTLx]

I also had this issue, and solved this in a tricky way.

In my other conda env, there was another ldpaths file, the path like ~/miniconda3/envs/py2.7/lib/R/etc/ldpaths, I copy this ldpaths file to where it is needed, and change the path in this file to fit my need, then everything is OK.

I may face this problem again while change conda envs, but at least I could return work ASAP. (I already backup ldpaths file for emergency)

Looking forward to someone who can give the perfect solution.

[Finesim97]

I ran into this issue with Snakemake. I use it with --use-conda flag, and when running parallel jobs with R, at some point ldpaths file just disappears. Nothing helped me to overcome this yet, really waiting for the solution.

Hey I had the exact same problem (running snakelike in parallel) and lecorguille solution helped. Inside the currently used conda directory (should be in the error log or just look for the most recent folder) inside .snakemake/conda/<hashhere> and edit the file .snakemake/conda/<hashhere>/etc/conda/activate.d/activate-r-base.sh and comment out the line described. This may not work with your setup, but "worked for me".

From

R CMD javareconf > /dev/null 2>&1 || true

to

# R CMD javareconf > /dev/null 2>&1 || true

Best Regards

[Fedorov113]

@Finesim97 well, I've just commented lines from 416-432 in env/lib/R/bin/javareconf (the ones that are responsible for updating files) and everything works great. It is partially equal to your solution. Thank you!

But what's the point of updating ldpaths and Makeconf each activation anyway? I don't think users update their Java that often. Setting values for this files on first installation should be enough. And than maybe we can introduce possibility of manually invoking update? If the user reinstalled Java or something.

@mingwandroid said that activation is

exactly when the file needs to be updated.

But as it can be seen, users just comment out this step altogether.

[mingwandroid]

If you want to query why the R foundation made this choice you should ask them.

I don't think users update their Java that often

That you do not need this doesn't mean others do not. We need a solution that works correctly in all cases (and it's not difficult either). Reconfiguring Java for each env is not going to be undone for it is the correct thing to do. If you convince R upstream to remove javareconf form R altogether (say moving it into rJava itself) then that would also work, however I don't know enough though too say this is a good idea or even possible.

[valscherz]

I ran into this issue with Snakemake. I use it with --use-conda flag, and when running parallel jobs with R, at some point ldpaths file just disappears. Nothing helped me to overcome this yet, really waiting for the solution.

Hey I had the exact same problem (running snakelike in parallel) and lecorguille solution helped. Inside the currently used conda directory (should be in the error log or just look for the most recent folder) inside .snakemake/conda/<hashhere> and edit the file .snakemake/conda/<hashhere>/etc/conda/activate.d/activate-r-base.sh and comment out the line described. This may not work with your setup, but "worked for me".

From

R CMD javareconf > /dev/null 2>&1 || true

to

# R CMD javareconf > /dev/null 2>&1 || true

Best Regards

I am experimenting the same issue... I have not tested your solution, mine was to remove the environment to force it to be reinstalled. I will try yours when I have time. Maybe something for the snakemake devloppers to look at? @johanneskoester ?

[johanneskoester]

So @mingwandroid, it boils down to having something like an environment-wide lock that is acquired before the activate script, ensuring that there are no two activate scripts executed at the same time, right? I would think that this could simply be implemented inside conda activate. Am I getting this right?

[mingwandroid]

Yes. Making the lock as short lived as possible would be ideal

[johanneskoester]

Great. Is this something that somebody on your side wants to do, or would you rather like me to provide a PR? If it should be me, can you give me a pointer where that would be in the code base? Sorry for asking, I don't want to bother you more than necessary, but my time is very limited these days.

[mingwandroid]

Mine is also limited. Ping @msarahan

[johanneskoester]

@msarahan, what do you think?

[mbargull]

I took only a quick look at this. IIUC, the activation script is just to give some users the convenience of having R "automagically" (re-)configure itself according to the currently available (external) Java every time an environment is activated.
So, okay, this may be beneficial for some users (i.e., those that change their Java configuration after this package has been installed, but don't care to run javareconf themselves). But frankly, in my opinion, putting this in an activation script is a horrible hack.

---

My general advice on (de-)activation scripts:

1. avoid creating them whenever you can accomplish your goal in other, saner manners (e.g., try inferring invariants at build or install time instead of re-evaluating things during environment activation),
2. only alter volatile state (e.g., environment variables and such),
3. do proper cleanup in deactivation scripts (e.g., if possible, be non-disruptive: revert only the changes from your activation script -- no more, no less),
4. don't expect your deactivation script to be run in all cases (e.g., closing a shell session will not run them),
5. if you need to save temporary state, try to use somewhat "unique" environment variable names (e.g., specific to the package, possibly shell session etc. => think about how to accomplish point 3. above under the assumptions that other packages' (de-)activation scripts get run, plus "Conda environment stacking" might be a thing..),
6. if environment variables don't cut it for you, try temporary storage, e.g., mktemp and friends (can be tricky due to non-POSIX-y stuff, permission issues, etc. => might need fallbacks if mktemp fails or something),
7. try to adhere to point 2. (possibly by doing point 1.!),
8. if you really might need to change some global/persistent state:
   8.1. avoid unnecessary writes (e.g., if the file state's content already is running, you needn't echo 'running' > state)
   8.2. prepare for possible concurrent access (e.g., use locks or, if possible, atomic operations)
9. (there are possible more things I currently didn't think of) ... be fast, be safe (e.g., try writing POSIX-compatible scripts that work with sh -ue), ...

---

Getting back to the the case at hand:
IMO, true solutions to the problem are one of the following:

1. Change R (or whatever is responsible for the R <-> Java interaction) to not depend on persistent state,
2. Let the user take care of running `javareconf.

I have no idea how R works (and am not really interested in that anyway), hence I can't comment if 1. is possible / feasible.
To make 2. work, you can either
2.1. Outright remove the javareconf call in the activation script and tell the users to manage their R-Java stuff themselves,
2.2. Copy the javareconf script and replace all of its writes by simple checks for changed files. If a change is detected, output a message to the user to run javareconf themselves.

Those are disruptive changes, of course. I won't expect 1. to happen. However, 2. is a sane compromise in my book. But since I don't really care much, I won't argue or push for that change.

---

Adding locks to Conda environments during activation is not an option for me. Having conda lock environments during its own operation (i.e., install/update/remove actions) makes sense.
Environment activation on the other hand should not mess with persistent state if ever possible, so locking an environment doesn't make sense for this.
If there are some "black sheep" activation scripts like the one of this package, then those scripts should handle concurrent access themselves, not conda.

Imagine if the javareconf script didn't change a file inside the Conda environment itself but in $HOME/.config or the like. If we created two different environments with r-base and concurrently activate them, then locking either of the environments doesn't help at all because the writes happen at some outside location.
=> conda cannot know if and where activation scripts change global state => conda can't know what to lock => activation scripts should handle concurrent writes themselves.

---

I'll open a pull request to address "8.1. avoid unnecessary writes" from my recommendation list above.
NB: The patch at

r-base-feedstock/recipe/0017-Foribly-remove-then-forcibly-mv-in-javareconf.in.patch

Line 20 in 9934a07

+ rm -rf "${file}" >/dev/null 2>&1

increases the likely hood of this issue appearing.

[mingwandroid]

IMHO while we have no have Java ecosystem to speak of, allowing different ones per env is important, no necessary. We can do better around our locking here but I don't think we should throw the baby out with the bath water.

These are decisions for R upstream. I didn't write R CMD javareconf or come up with the idea, but I believe per env activation is right. Unless you can carefully articulated why not then we're at an impasse on that point. Please try our R with a range of Javas. I went to extreme lengths to maintain good compatibility for our users because we have nothing much for them ourselves.

I'm not so busy on R these days FWIW but I don't think this is the right track to take.

[mingwandroid]

I disagree with your point that activation scripts shouldn't write files beyond accepting they shouldn't exist when not necessary. Writing env local files is appropriate but we should lock things better.

At the end of the day R provides this facility and I thought it there to take advantage of, that this was most appropriate place (think people experimenting with different Javas with R) and we should be as dynamic and friendly to all the Javas as possible and I believe using it is entirely reasonable (if bugged currently .. be aware we have a few openjdks too, conda-forge's coming from Azul still I think and ADs coming from Jetbrains, and I want to support system and legacy Javas).

I'm just trying to broaden our appeal, I don't use Java at all and R little (ironically usually when trying to fix issues with rJava, yes, largely to do with trying to be broadly compatible (and not surprise or burden our users - which Java should be active? How do I do that?).
I don't know how much of R's Java goes through rJava. If all, then perhaps someone could propose moving that functionality into rJava itself, I'd be supportive of that if it's appropriate. That we must people could ignore this (and we can make it work right for other packages that much want to write env local lconfig files during activation).

[mbargull]

[...] allowing different ones per env [...]
[...] but I believe per env activation is right [...]

I don't disagree with the "per env" part, just the "activation" one. My stance is just that reconfiguration during every environment activation is excessive and can lead to concurrency issues during writes (i.e., this very issue). I'd rather have those changes made on demand, which means when r-base is installed/updated (which is already the case via the post-link script) or by the user.

I went to extreme lengths to maintain good compatibility for our users because we have nothing much for them ourselves.

Glad to hear, I'll just take your word on this ;).

Please try our R with a range of Javas.

I'll leave this to others since I'm with you with "I don't use Java at all and R little" -- for me it is rather "I don't use R at all and Java little" ;).

I disagree with your point that activation scripts shouldn't write files beyond accepting they shouldn't exist when not necessary.

The thing is just that I believe users likely won't expect environment activations to do much special things, esp. not things that might fail. Plus, authors of activation scripts have to be aware of possible concurrency issues and handle them accordingly.

Writing env local files is appropriate but we should lock things better.

IMO, only in exceptional circumstances for activation scripts. And due to the "exceptional" part, I'd rather not have conda itself deal with this (rather just have the activation script at least do something like lockfile "${CONDA_PREFIX}/path/to/file.lock" ; write-to-file "${CONDA_PREFIX}/path/to/file" ; rm -f "${CONDA_PREFIX}/path/to/file.lock").

---

Overall my opinion is just

2.2. Copy the javareconf script and replace all of its writes by simple checks for changed files. If a change is detected, output a message to the user to run javareconf themselves.

[...] is a sane compromise in my book. But since I don't really care much, I won't argue or push for that change.

and hence I think for now we should just try to reduce the concurrent write situation here and, if someone is willing to dig into it (i.e., not me), add some proper file locking in javareconf or the activation script.

[mingwandroid]

On gitter people are expecting to be able to operate on the same conda env from multiple processes at the same time and that's what is happening here. I'd rather we fixed it higher level. In no way should an env been mutable by two conda at once. I'd this bug helps us to shake out issues there then that's a win.

But yes, happy for wanting reasonable here too, just want to be flexible and am very concerned about UX and compat.

[epruesse]

Moving the conversation from gitter over here. My points were:

• Concurrently entering (activating) the same environment from multiple processes, possibly running on different computers with a shared filesystem, is a pretty common use case. A classic example is parallel execution of jobs on a cluster, e.g. using snakemake as presented above.
• An environment changing underneath running processes because it is activated from elsewhere is clearly unexpected behavior. It will break things. Not every job starts R right away, and this kind of thing is guaranteed to break at the most unfortunate time.
• Concurrently running conda install or other operations obviously altering an environment OTOH is expected to either break (current behavior), fail or block. It is also reasonably easy to avoid this. It would be awesome to have locking for package cache and env to allow safe concurrent calls to those, but there is no way that two things are adding packages to the same env at the same time.
• Locking across operating systems and shared filesystems is finicky. Even highly polished tools such as sqlite3 discourage accessing the same database from multiple hosts. So using locking needs to come with big warning signs.

I would therefore argue that conda activate should be free of side effects.

If things specific to the local host really really really need to be altered, the respective configuration would have to be placed in a temporary directory unique to the activation session. We'd also have to accept that it would not be reliably cleaned away in all cases.

Using locking to protect from things changing underneath running processes (which would lead to undefined results) would require acquiring the lock in activate and releasing it in deactivate. The latter doesn't necessarily happen explicitly though, so that's the first major problem. The other is that the env can then no longer be used in parallel, meaning a cluster job running 50 nodes would need 50 copies of the env. Possible, but a very ugly brute force solution.

In the case at hand, I don't see why host local JVMs need to be supported at all. IMO, the root package requiring the javareconf files should require a JVM installed into the environment and handle the ldpath setup in post-install.sh. That way, it's set up at the time where modifying the env is expected, and the env remains static during use.

In summary, my $0.02 based on experience parallelizing C/C++ and Python apps is that activate should be "const", i.e. idempotent and without side-effects. Everything else will either be slow (=one process per env) or turn out to be a nightmare with constant breakage and patch after patch as more and more corner cases turn up.

[mingwandroid]

If environment mutation is locked which I think it should be and am not sure if that's a bug or a missing feature, then that would include activation (because we've always allowed env mutation, and this problem can happen with anything, env vars for example).

If we fix all that then this is automatically fixed. Though @mbargull's PR will help in the meantime.

[msarahan]

I would really like if activation were assumed to be "const." I agree with Ray, though, that we have the existing expectations to contend with. Perhaps with conda/conda#8727 we can have a "safe" way of activation: if any arbitrary shell scripts are present, it is considered unsafe (mutation may happen), and must be locked. If no unsafe scripts are present, though, conda will not require locking for activation, and perhaps can go faster.

[mingwandroid]

Using locking to protect from things changing underneath running processes (which would lead to undefined results) would require acquiring the lock in activate and releasing it in deactivate

@epruesse, it absolutely would not.

[hdraisma]

I was wondering, would it be possible to give an update re (the resolution of) this issue?

Thank you very much in advance

[freekvh]

Dear all, I also run into this issue frequently when processing a lot of files in parallel using snakemake with--use-conda (on an NFS share). I see that there is a fix: kpalin@9eda35b. But since this issue is still open, does that mean it is not in any release yet?

My apologies for my ignorance, I'm not able to determine if that commit is in any release...

By the way, setting my conda envs as read only would be an option for me (actually I'd prefer it, and keep their management restricted to 1 or 2 accounts only).