Packages installed via `install.packages` in R conflict with packages installed via `conda`. about r-base-feedstock

[jdblischak]

@izahn Thanks for the detailed example. I was able to reproduce it (copied below). My initial thoughts are:

1. This is only a warning. The R and r-brio installed in the environment rtest2 appear to work fine. The warning comes because the file packages.html was modified when brio was installed with install.packages(). There isn't any actual security or functionality concern, but I understand why users may be concerned when they see those messages.

2. I'm open to the potential of using the site-library. Other than removing this warning, I'd have to think about the potential side effects and collisions with other install directories. Also I suspect you'll get similar push back as in my PR #65.

Output from reproducing example

mamba create --yes -n rtest1 r-base r-brio 
conda activate rtest1
Rscript -e '.libPaths()'
[1] "/home/jdblischak/miniconda3/envs/rtest1/lib/R/library"
Rscript -e 'find.package("brio")'
## [1] "/home/jdblischak/miniconda3/envs/rtest1/lib/R/library/brio"
Rscript -e 'library("brio")'
Rscript -e "install.packages('brio', repos='https://cran.rstudio.com')"
## ...
## Updating HTML index of packages in '.Library'
## Making 'packages.html' ... done
Rscript -e 'find.package("brio")'
## [1] "/home/jdblischak/miniconda3/envs/rtest1/lib/R/library/brio"
Rscript -e 'library("brio")'
conda deactivate

mamba create --yes -n rtest2 r-base r-brio
## ...
## WARNING Size incorrect, file modified in package cache "/home/jdblischak/miniconda3/pkgs/r-base-4.0.5-h8cab1ac_0/lib/R/doc/html/packages.html"
## ...
## SafetyError: The package for r-base located at /home/jdblischak/miniconda3/pkgs/r-base-4.0.5-h8cab1ac_0
## appears to be corrupted. The path 'lib/R/doc/html/packages.html'
## has an incorrect size.
##   reported size: 2946 bytes
##   actual size: 3102 bytes
## ...
conda activate rtest2
Rscript -e 'find.package("brio")'
## [1] "/home/jdblischak/miniconda3/envs/rtest2/lib/R/library/brio"
Rscript -e 'library("brio")'
conda deactivate

mamba env remove -n rtest1
mamba env remove -n rtest2
conda clean --yes --all

mamba create --yes -n rtest1 r-base r-brio
mkdir ~/.conda/envs/rtest1/lib/R/site-library
## mkdir: cannot create directory ‘/home/jdblischak/.conda/envs/rtest1/lib/R/site-library’: No such file or directory
echo $CONDA_PREFIX
## /home/jdblischak/miniconda3
mkdir $CONDA_PREFIX/envs/rtest1/lib/R/site-library

conda activate rtest1
Rscript -e "install.packages('brio', repos='https://cran.rstudio.com')"
## Installing package into ‘/home/jdblischak/miniconda3/envs/rtest1/lib/R/site-library’
## ...
conda deactivate
ls $CONDA_PREFIX/envs/rtest1/lib/R/site-library
## brio

mamba create --yes -n rtest2 r-base r-brio 

conda info
## 
##      active environment : base
##     active env location : /home/jdblischak/miniconda3
##             shell level : 1
##        user config file : /home/jdblischak/.condarc
##  populated config files : /home/jdblischak/.condarc
##           conda version : 4.10.1
##     conda-build version : 3.21.4
##          python version : 3.7.9.final.0
##        virtual packages : __linux=4.4.0=0
##                           __glibc=2.27=0
##                           __unix=0=0
##                           __archspec=1=x86_64
##        base environment : /home/jdblischak/miniconda3  (writable)
##       conda av data dir : /home/jdblischak/miniconda3/etc/conda
##   conda av metadata url : https://repo.anaconda.com/pkgs/main
##            channel URLs : https://conda.anaconda.org/conda-forge/linux-64
##                           https://conda.anaconda.org/conda-forge/noarch
##                           https://conda.anaconda.org/bioconda/linux-64
##                           https://conda.anaconda.org/bioconda/noarch
##                           https://repo.anaconda.com/pkgs/main/linux-64
##                           https://repo.anaconda.com/pkgs/main/noarch
##                           https://repo.anaconda.com/pkgs/r/linux-64
##                           https://repo.anaconda.com/pkgs/r/noarch
##           package cache : /home/jdblischak/miniconda3/pkgs
##                           /home/jdblischak/.conda/pkgs
##        envs directories : /home/jdblischak/miniconda3/envs
##                           /home/jdblischak/.conda/envs
##                platform : linux-64
##              user-agent : conda/4.10.1 requests/2.25.1 CPython/3.7.9 Linux/4.4.0-19041-Microsoft ubuntu/18.04.2 glibc/2.27
##                 UID:GID : 1000:1000
##              netrc file : None
##            offline mode : False

[izahn]

1. This is only a warning.

The actual message says SafetyError, which at least sounds worse than a warning!

The R and r-brio installed in the environment rtest2 appear to work fine. The warning comes because the file packages.html was modified when brio was installed with install.packages(). There isn't any actual security or functionality concern, but I understand why users may be concerned when they see those messages.

I think the fundamental issue is that conda and install.packages from R should not be trying to install things in the same place. Things might work after R overwrites a conda package, but they might not, and on the face of it this sounds like a bad idea. It definitely should not be the default behavior.

2. I'm open to the potential of using the site-library. Other than removing this warning, I'd have to think about the potential side effects and collisions with other install directories.

site-library is the documented and official place to install installation-wide packages. That is why just creating that directory is enough to get R to install packages to it by default. I don't see any reason why conda should not take advantage of this standard R mechanism.

Also I suspect you'll get similar push back as in my PR #65.

Maybe, but from my perspective this is clearly a bug, and there is an easy and non-hacky solution. We should fix it!

[jdblischak]

I think the fundamental issue is that conda and install.packages from R should not be trying to install things in the same place.

I thought of a potential issue. If site-library is created when r-base is installed, how will that affect conda R recipes built with conda build? Since build.sh simply calls R CMD INSTALL --build, I think it would install the package in site-library. I think we would need to modify build.sh to specify --library to point to the current install location. Otherwise conda and install.packages() will still be writing to the same directory.

$ R CMD INSTALL --help
Usage: R CMD INSTALL [options] pkgs

Install the add-on packages specified by pkgs.  The elements of pkgs can
be relative or absolute paths to directories with the package
sources, or to gzipped package 'tar' archives.  The library tree
to install to can be specified via '--library'.  By default, packages are
installed in the library tree rooted at the first directory in
.libPaths() for an R session run in the current environment.

Maybe, but from my perspective this is clearly a bug, and there is an easy and non-hacky solution. We should fix it!

I say go for it then! Worse case scenario: the issue is better known and better documented.

[izahn]

Definitely need to be careful not to affect the conda build process for R packages! My first thought is to create the site-library directory in recipe/post-link.sh, and only if CONDA_BUILD is not set.

[izahn]

I gave it a quick stab, seems harder then I expected :-(