ansible role to install miniconda, Python-agnostic binary package manager
Home Page: https://galaxy.ansible.com/uchida/miniconda
License: Creative Commons Zero v1.0 Universal
I don't know if this is fixable (or if it should be fixed) in the role as it's technically a problem of conda env, but basically when there's a mistake in the pip section of miniconda_env the process returns exit status 0, despite the fact that the command failed.
For example, given this environment.yml:
dependencies:
- pip
- pip:
- flask-restful==0.3.5
- /var/www/recipestore/foodlab
- jwt==0.3.2
- pyasn1==0.1.9
- pycrypto==2.6.1
- requests-cache==0.4.12
- sqlalchemy-utils==0.32.1
name: eatingproject
The file has a mistake and prints to standard error
"/opt/miniconda/bin/conda" env update -f "/tmp/eatingproject-environment.yml" > hello
Invalid requirement: '/var/www/recipestore/foodlab'
It looks like a path. Does it exist ?
However, process has exit status 0:
echo $?
0
Is it possible to trigger a failure in this case?
btw I opened an issue here:
https://github.com/conda/conda-env/issues/255
The pip section doesn't get serialized as a bulleted list, but as a json list like in this example. I think this can be fixed by using to_nice_yaml instead.
That seems to be different to what is produced by conda env export.
dependencies:
- cloudpickle=0.1.1=py35_0
- flask=0.10.1=py35_1
- flask-cors=2.1.2=py35_0
- gunicorn=19.1.0=py35_0
- h5py=2.6.0=np110py35_1
- hdf5=1.8.16=0
- heapdict=1.0.0=py35_0
- joblib=0.9.4=py35_0
- jsonschema=2.4.0=py35_0
- nltk=3.2.1=py35_0
- numpy=1.11.0=py35_1
- pandas=0.17.0=np110py35_0
- psycopg2=2.6.1=py35_1
- py=1.4.31=py35_0
- pyyaml=3.11=py35_1
- scikit-learn=0.17.1=np111py35_0
- scipy=0.17.0=np111py35_2
- sqlalchemy=1.0.12=py35_0
- sqlite=3.13.0=0
- yaml=0.1.6=0
- pip: [flask-restful==0.3.5, /var/www/recipestore/foodlab, jwt==0.3.2, pyasn1==0.1.9,
pycrypto==2.6.1, requests-cache==0.4.12, sqlalchemy-utils==0.32.1]
This is is failing for me. Because the prefix path gets created by ansible (uchida.miniconda : directory /root/miniconda3 exists), the installer skips actually installing:
ok: [digitalocean2] => {"changed": false, "cmd": "/bin/bash \"/tmp/Miniconda-3.16.0-Linux-x86_64.sh\" -b -p \"/root/miniconda3/\"", "rc": 0, "stderr": false, "stdout": "skipped, since /root/miniconda3/ exists", "stdout_lines": ["skipped, since /root/miniconda3/ exists"]}
___________________________________________________
< TASK [uchida.miniconda : miniconda is up-to-date] >
---------------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
I'ld propose such a task (optional) added at the end of the role.
I'ld also add a re-usable ENV Var that contains the miniconda HOME (dir), for example named CONDA_HOME:
Both together I currently set as a post_task in my playbook using (but would prefer it added to the role):
- name: Export python_conda ENV variables
copy: content="export CONDA_HOME={{miniconda_prefix}}\nexport CONDA_PYTHON=$CONDA_HOME/bin/python\nexport PATH=$CONDA_HOME/bin:$PATH\n" dest="/etc/profile.d/python_conda_exports.sh" mode=0755
Any suggestions ?
as there've been 12 commits to master since the last release, would you mind making a new one?
since currently the role is still using the old style (which has various drawbacks and is not at all recommended anymore) .. snippet from the main tasks:
file:
path="{{ miniconda_prefix | dirname }}"
( Actually it's mostly replacing a number of = by : )
Other suggestions/wishes I have for this role:
main.yml tasks only contain import_tasksto install.yml and environment.yml tasks file (TODO move main logic out into a new install.yml tasks file)ps: I checked the existing forks (https://github.com/uchida/ansible-miniconda-role/network) but nobody has done that yet
This occurs using recent miniconda versions, e.g.:
- role: uchida.miniconda
tags: miniconda
vars:
miniconda_python: 3
miniconda_version: 4.6.14
miniconda_prefix: "{{ vm_codes_folder }}/miniconda"
miniconda_env:
name: addis
channels:
- conda-forge
dependencies:
- rdkit=2019.03.2.0
- ase=3.17.0
- scipy=1.3.0
- matplotlib=3.1.0
- jupyter=1.0.0The first time I run the role (and only the first time), I get the following error:
TASK [uchida.miniconda : conda environment addis is created] *******************
fatal: [default]: FAILED! => changed=true
cmd:
- ${HOME}/codes/miniconda/bin/conda
- env
- create
- -f
- /tmp/addis-environment.yml
- -q
delta: '0:00:00.487782'
end: '2019-06-02 22:41:45.465240'
msg: non-zero return code
rc: 1
start: '2019-06-02 22:41:44.977458'
stderr: |2-
NotWritableError: The current user does not have write permissions to a required path.
path: /home/max/codes/miniconda/envs/.conda_envs_dir_test
uid: 1001
gid: 100
If you feel that permissions on this path are set incorrectly, you can manually
change them by executing
$ sudo chown 1001:100 /home/max/codes/miniconda/envs/.conda_envs_dir_test
In general, it's not advisable to use 'sudo conda'.
stderr_lines:
- ''
- 'NotWritableError: The current user does not have write permissions to a required path.'
- ' path: /home/max/codes/miniconda/envs/.conda_envs_dir_test'
- ' uid: 1001'
- ' gid: 100'
- ''
- If you feel that permissions on this path are set incorrectly, you can manually
- change them by executing
- ''
- ' $ sudo chown 1001:100 /home/max/codes/miniconda/envs/.conda_envs_dir_test'
- ''
- In general, it's not advisable to use 'sudo conda'.
stdout: ''
stdout_lines: <omitted>
to retry, use: --limit @/Users/leopold/Personal/Postdoc-MARVEL/repos/quantum-mobile/playbook.retry
When I go to the directory, everything seems fine in terms of permissions.
When I run the role a second time, it completes without warnings.
When using the role with ansible-core 2.12:
[DEPRECATION WARNING]: "include" is deprecated, use include_tasks/import_tasks instead. This feature will be removed in version 2.16.
This link has guidance on whether to switch to include_tasks or import_tasks:
https://docs.ansible.com/ansible-core/2.12/user_guide/playbooks_reuse.html#re-using-files-and-roles
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.