Python compatibility issue about lsst-cvmfs

[airnandez]

I'm not sure I fully understand what you are trying to do. If your only interest is using EUPS, I would suggest installing it from source so that it uses the version of Python shipped with the GridPP VM. Although I don't know for sure, I would say that EUPS does not put strong requirements on Python. That being said, Python v2.6.6 may be a bit too old for EUPS, but it is worth trying.

So far I have not had reports on conflicting versions of Python using this distribution of the LSST software stack. Maybe people using it use recent versions of Linux distributions which include a more recent version of Python, such as Python 2.7.5. By the way, the LSST stack is supported on CentOS 7 but is known to work on other distributions such as Ubuntu.

A potential solution would be to use a Docker container, but I don't know if the GridPP VM allows you to do that.

I'm sorry I'm not able to be more helpful.

[djwhiteastro]

I'm trying to leverage EUPS on GridPP. I have a workflow we are trying to get working on GridPP that requires the installation of several pieces of software that aren't standard on GridPP, and EUPS allows us to do that easily. Since your version of EUPS is available from the LSST CVMFS, and this work is under LSST, it made sense to do this. We can simply package up our own EUPS-registered software and ship them off with our workflow to run on GridPP (with access to the LSST CVMFS, since this is work done under LSST VO registered accounts).

EUPS works fine with Python 2.6.6, and it also works fine with Python 2.7.8 when done manually, but the issue comes with the way that EUPS is set up on the LSST CVMFS. When we run "setup" on our Python 2.7 package, all of the path variables change to point to the libraries and modules for that Python 2.7 install. However, your install for EUPS doesn't use the version of Python that is in the path (in this case 2.7) but specifically calls the system-wide version of Python regardless. So, in my setup now, I have Python 2.6 looking for modules/libraries/paths created for Python 2.7. This causes major problems.

Is there a reason why this is so? I've looked at the versions of EUPS available on github, and they don't seem to do this. It seems to call Python in a general sense. Do you have an old version of EUPS? Is the only option to download, build, install and package the entirety of EUPS as a standalone package and ship that off with our packages?

[airnandez]

Thanks for your clarification, I think I can now see what you are trying to accomplish.

The LSST software stack available through CernVM FS is installed from source following the instructions here. As part of this installation process, a version of EUPS certified compatible with each version of the LSST software stack is also installed.

EUPS requires the path of the Python interpreter to be known at installation time. See here. For your particular case, it would be more convenient if the EUPS Python scripts used the shebang #!/usr/bin/env python instead of the absolute path of a Python interpreter. However, I don't know the reason why this is not done.

Since the machines where the LSST binary is built run CentOS 7, I use the Python which comes with the system, i.e. /usr/bin/python. This is the reason /usr/bin/python is the path that you find "hardcoded" in the EUPS Python scripts.

The EUPS experts may be able to tell you why the Python path is needed at installation time. See: https://github.com/RobertLuptonTheGood/eups.

[djwhiteastro]

Ah I see! I didn't realise it was set at installation. That makes sense now. It isn't an issue if the version of python that was setup was kept separate from the system version, but it seems to not be. That might be something I need to ask a Python expert about. At this stage I'm looking at a Chicken/Egg situation though if that's not easy to fix. I might have to use LSST EUPS to setup Python2.7, use that to install EUPS as an EUPS package that links to the correct Python version (2.7), and bundle it all off together. This could be fun... Thanks for your help!

…

On 22 March 2017 at 16:15, Fabio Hernandez ***@***.***> wrote: Thanks for your clarification, I think I can now see what you are trying to accomplish. The LSST software stack available through CernVM FS is installed from source following the instructions here <https://pipelines.lsst.io/install/newinstall.html>. As part of this installation process, a version of EUPS certified compatible with each version of the LSST software stack is also installed. EUPS requires the path of the Python interpreter to be known at installation time. See here <https://github.com/RobertLuptonTheGood/eups/blob/master/configure.ac#L25>. For your particular case, it would be more convenient if the EUPS Python scripts used the shebang #!/usr/bin/env python instead of the absolute path of a Python interpreter. However, I don't know the reason why this is not done. Since the machines where the LSST binary is built run CentOS 7, I use the Python which comes with the system, i.e. /usr/bin/python. This is the reason /usr/bin/python is the path that you find "hardcoded" in the EUPS Python scripts. The EUPS experts may be able to tell you why the Python path is needed at installation time. See: https://github.com/RobertLuptonTheGood/eups. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKeL573uFdNPmjaGGiqA7YpDcj0epPYsks5roUksgaJpZM4MlN2c> .