What:
Fresh install on Ubuntu 20.04 with miniconda3, python 3.7 RDKit installs ok but Python3-saml has become incompatible.
First noted with 2020.09.04 but after experiments found first version where break occurred - 2020.03.6
Terse:
Demonstrated with
python3-saml
Some issues with rpy2 but:
$ conda install -c conda-forge rpy2
appears to install and work ok on Ubuntu 20.04 (not Mac OSX)
TERSE:
Up to, and including rdkit==2019.09.3 python3-saml==1.9.0 would install and work together
With a python 3.7 condo env:
$ conda install -c conda-forge rdkit==2020.03.6 -y
$ conda install -c conda-forge python3-saml
Python3-saml failed to install with multiple conflicts
- there are references to Python 3.8 - did we miss something and RDkit is Python3.8 now?
rdkit==2020.09.3 and rdkit==2020.09.4 also are incompatible
Good News - There is compatibility with (this is a section of our build script)
pip install mod_wsgi==4.7.1
pip install -i https://pypi.anaconda.org/OpenEye/simple OpenEye-toolkits
conda install -c conda-forge rpy2 -y
conda install -c conda-forge scikit-learn==0.21.3 -y
conda install -c conda-forge matplotlib==3.3.3 -y
For addition clarity:
pip install python3-saml
does not work - this is PEP517 related but it does reveal that libxml2 and libxmlsec might be the issue
[Spent about a day try to work around this one without success]
These are the results on Ubuntu 20, on Mac OSX Catalina things are worse (that will required another day of experimentation - Rdkit 2018 appears stable here with all other module combinations)
Verbose (-vvv)
These are the experiments I performed focussing on just RDkit and Python3-saml for reproducibility.
Experiment details:
All builds were:
- performed on Amazon Web Services (AWS) Ubuntu 20.04 instances (unless otherwise stated), using t2.large
- MINICONDA="Miniconda3-4.5.12-Linux-x86_64.sh"
- with Python 3.7 environments
- as root user
After each experiment the AWS instance was terminated and a new instance used each time.
Control - 1 - establish older version of RDkit installs with
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2019.03.1.0 -y
conda install -c conda-forge python3-saml
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.8.3
latest version: 4.9.2
Please update conda by running
$ conda update -n base -c defaults conda
Package Plan
environment location: /root/miniconda3/envs/test_rdkit
added / updated specs:
- python3-saml
The following packages will be downloaded:
package | build
---------------------------|-----------------
defusedxml-0.6.0 | py_0 22 KB conda-forge
isodate-0.6.0 | py_1 25 KB conda-forge
libgcrypt-1.8.7 | h36c2ea0_0 610 KB conda-forge
libgpg-error-1.41 | h2531618_0 235 KB
libtool-2.4.6 | h58526e2_1007 497 KB conda-forge
libxml2-2.9.10 | hb55368b_3 1.2 MB
libxmlsec1-1.2.29 | h8fe84ac_0 516 KB
libxslt-1.1.34 | hc22bd24_0 432 KB
lxml-4.6.2 | py37h9120a33_0 1.2 MB
python3-saml-1.9.0 | py_0 227 KB
xmlsec-1.3.3 | py37h0e8eec6_0 289 KB
------------------------------------------------------------
Total: 5.2 MB
The following NEW packages will be INSTALLED:
defusedxml conda-forge/noarch::defusedxml-0.6.0-py_0
isodate conda-forge/noarch::isodate-0.6.0-py_1
libgcrypt conda-forge/linux-64::libgcrypt-1.8.7-h36c2ea0_0
libgpg-error pkgs/main/linux-64::libgpg-error-1.41-h2531618_0
libtool conda-forge/linux-64::libtool-2.4.6-h58526e2_1007
libxmlsec1 pkgs/main/linux-64::libxmlsec1-1.2.29-h8fe84ac_0
libxslt pkgs/main/linux-64::libxslt-1.1.34-hc22bd24_0
lxml pkgs/main/linux-64::lxml-4.6.2-py37h9120a33_0
python3-saml pkgs/main/noarch::python3-saml-1.9.0-py_0
xmlsec pkgs/main/linux-64::xmlsec-1.3.3-py37h0e8eec6_0
The following packages will be UPDATED:
libxml2 conda-forge::libxml2-2.9.9-h13577e0_2 --> pkgs/main::libxml2-2.9.10-hb55368b_3
....lots
conda list
...
python-dateutil 2.8.1 py_0 conda-forge
python3-saml 1.9.0 py_0
python_abi 3.7 1_cp37m conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
rdkit 2019.03.1 py37h9c20d5c_0 conda-forge
readline 8.1 h27cfd23_0
...
SUCCESSFUL build***
Control 2 - newer version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2019.09.3 -y
conda install -c conda-forge python3-saml
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.8.3
latest version: 4.9.2
Please update conda by running
$ conda update -n base -c defaults conda
Package Plan
environment location: /root/miniconda3/envs/test_rdkit
added / updated specs:
- python3-saml
The following packages will be downloaded:
package | build
---------------------------|-----------------
boost-1.70.0 | py37h9de70de_1 337 KB conda-forge
boost-cpp-1.70.0 | ha2d47e9_1 21.1 MB conda-forge
cairo-1.16.0 | h18b612c_1001 1.5 MB conda-forge
defusedxml-0.6.0 | py_0 22 KB conda-forge
fontconfig-2.13.1 | he4413a7_1000 327 KB conda-forge
gettext-0.19.8.1 | h5e8e0c9_1 3.5 MB conda-forge
icu-58.2 | hf484d3e_1000 22.6 MB conda-forge
isodate-0.6.0 | py_1 25 KB conda-forge
libgcrypt-1.8.7 | h36c2ea0_0 610 KB conda-forge
libgpg-error-1.39 | he1b5a44_0 269 KB conda-forge
libtool-2.4.6 | h58526e2_1007 497 KB conda-forge
libxml2-2.9.9 | h13577e0_2 1.3 MB conda-forge
libxmlsec1-1.2.29 | h8fe84ac_0 516 KB
libxslt-1.1.33 | h7d1a2b0_0 426 KB
lxml-4.4.1 | py37h7ec2d77_0 1.5 MB conda-forge
python3-saml-1.9.0 | py_0 227 KB
rdkit-2019.09.3 | py37hb31dc5d_0 23.7 MB conda-forge
xmlsec-1.3.3 | py37h0e8eec6_0 289 KB
------------------------------------------------------------
Total: 78.8 MB
The following NEW packages will be INSTALLED:
defusedxml conda-forge/noarch::defusedxml-0.6.0-py_0
gettext conda-forge/linux-64::gettext-0.19.8.1-h5e8e0c9_1
isodate conda-forge/noarch::isodate-0.6.0-py_1
libgcrypt conda-forge/linux-64::libgcrypt-1.8.7-h36c2ea0_0
libgpg-error conda-forge/linux-64::libgpg-error-1.39-he1b5a44_0
libtool conda-forge/linux-64::libtool-2.4.6-h58526e2_1007
libxmlsec1 pkgs/main/linux-64::libxmlsec1-1.2.29-h8fe84ac_0
libxslt pkgs/main/linux-64::libxslt-1.1.33-h7d1a2b0_0
lxml conda-forge/linux-64::lxml-4.4.1-py37h7ec2d77_0
python3-saml pkgs/main/noarch::python3-saml-1.9.0-py_0
xmlsec pkgs/main/linux-64::xmlsec-1.3.3-py37h0e8eec6_0
The following packages will be DOWNGRADED:
boost 1.72.0-py37h48f8a5e_1 --> 1.70.0-py37h9de70de_1
boost-cpp 1.72.0-h9359b55_3 --> 1.70.0-ha2d47e9_1
cairo 1.16.0-h3fc0475_1005 --> 1.16.0-h18b612c_1001
fontconfig 2.13.1-h7e3eb15_1002 --> 2.13.1-he4413a7_1000
icu 67.1-he1b5a44_0 --> 58.2-hf484d3e_1000
libxml2 2.9.10-h68273f3_2 --> 2.9.9-h13577e0_2
rdkit 2019.09.3-py37he9dd633_1 --> 2019.09.3-py37hb31dc5d_0
conda list
...
python-dateutil 2.8.1 py_0 conda-forge
python3-saml 1.9.0 py_0
python_abi 3.7 1_cp37m conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
rdkit 2019.09.3 py37hb31dc5d_0 conda-forge
readline 8.1 h27cfd23_0
setuptools 52.0.0 py37h06a4308_0
...
SUCCESSFUL build***
Expt 1 - moving newer version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2020.03.6 -y
conda install -c conda-forge python3-saml
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
Conda detects conflicts between RDkit install and Python3-SAML
Expt 2 - newer version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
#Conda init bash
#
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2020.09.3 -y
conda install -c conda-forge python3-saml
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
--> Fails to install python3-saml and reports a huge amount of output - see included file.
Expt 3 - latest version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2020.09.4 -y
conda install -c conda-forge python3-saml
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
--> Fails to install python3-saml and reports a huge amount of output - see included file.
Expt 4 - Install Python3-saml first then RDkit - check if it works the other way
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge python3-saml -y
conda install -c conda-forge rdkit==2020.09.4 -y
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!