Library error with Python 3 and Anaconda about treecorr

[rmjarvis]

The error you are showing seems to be in flask, not TreeCorr. I don't know anything about flask, so there's not much I can do with this.

[rmjarvis]

It looks like this is a "feature" of Python3.4. http://stackoverflow.com/questions/25235013/cythondistutils-build-on-ubuntu-python-3-changes-the-module-lib-name-during-lin

I agree with the author of that SO question, "I feel a rant coming on." Why would they make this idiotic change? It serves no apparent purpose other than to break existing code that expects them to be sensible.

Anyway, when I get some time, I'll look into it. Maybe there is something in distutils that will return the actual library name that they build, which I can write somewhere so we can load the correct file.

[rmjarvis]

@gogrean I believe this is fixed now with TreeCorr 3.3.6.

Please try pip install -U treecorr and see if it works on your system.

[rmjarvis]

Assuming this is resolved.

[niteshb0804]

Same error but Python2.7

import treecorr
.
.
OSError: cannot load library '/home/nitesh/anaconda2/lib/python2.7/site-packages/treecorr/_treecorr.so': /home/nitesh/anaconda2/lib/python2.7/site-packages/treecorr/_treecorr.so: undefined symbol: GOMP_parallel. Additionally, ctypes.util.find_library() did not manage to locate a library called '/home/nitesh/anaconda2/lib/python2.7/site-packages/treecorr/_treecorr.so'

Any suggestions? I tried reinstalling, upgrade treecorr...

[rmjarvis]

This normally means you are loading the wrong libgomp library at runtime. Anaconda is tricky because they have their own libgomp file. And they can have (if you do conda install gcc) their own compiler. So you need to make sure you have your environment variables set correctly to load the right library.

Typically, this is done by setting your LD_LIBRARY_PATH to include the anaconda2/lib directory, which is probably where the correct libgomp is located. And also set your PATH to include anaconda2/bin to get their compiler.

You can check which libgomp library treecorr was compiled to use with

$ ldd /home/nitesh/anaconda2/lib/python2.7/site-packages/treecorr/_treecorr.so

On a machine I have using anaconda, this is what is output:

$ ldd ~/work/.conda/envs/py2.7/lib/python2.7/site-packages/TreeCorr-3.4-py2.7-linux-x86_64.egg/treecorr/_treecorr.so
	linux-vdso.so.1 =>  (0x00007ffdc44e0000)
	libpython2.7.so.1.0 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libpython2.7.so.1.0 (0x00007fe1fbefc000)
	libstdc++.so.6 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libstdc++.so.6 (0x00007fe1fbbc2000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fe1fb93e000)
	libgomp.so.1 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libgomp.so.1 (0x00007fe1fb71b000)
	libgcc_s.so.1 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libgcc_s.so.1 (0x00007fe1fb509000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe1fb2ec000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fe1faf58000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fe1fad54000)
	libutil.so.1 => /lib64/libutil.so.1 (0x00007fe1fab51000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe1fc5bc000)
	librt.so.1 => /lib64/librt.so.1 (0x00007fe1fa949000)

And the /astro/u/mjarvis/.conda/envs/py2.7/lib/ directory is in my LD_LIBRARY_PATH environment variable. If you have a different libgomp library in your library path, then you can sometimes get mismatches in what setup.py found at build time and what python finds at runtime.

Without more details about your particular setup, I can't really diagnose further, but hopefully this gives you enough to investigate on your own.

[niteshb0804]

First of all many thanks for such a quick reply. Following your instructions: ldd /home/nitesh/anaconda2/lib/python2.7/site-packages/treecorr/_treecorr.so linux-vdso.so.1 => (0x00007ffc54b03000) libpython2.7.so.1.0 => /home/nitesh/anaconda2/lib/libpython2.7.so.1.0 () libstdc++.so.6 => /home/nitesh/anaconda2/lib/libstdc++.so.6 (0x00007fb2) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb25d100000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb25cd20000) libgomp.so.1 => /home/nitesh/anaconda2/lib/libgomp.so.1 (0x00007fb25cb1) libgcc_s.so.1 => /home/nitesh/anaconda2/lib/libgcc_s.so.1 (0x00007fb25c) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb25c6) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb25c4d8000) libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fb25c2d4000) /lib64/ld-linux-x86-64.so.2 (0x00007fb25ddf8000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fb25c0cc000) seems alright?

…

On Wed, Mar 7, 2018 at 10:48 PM, Mike Jarvis ***@***.***> wrote: This normally means you are loading the wrong libgomp library at runtime. Anaconda is tricky because they have their own libgomp file. And they can have (if you do conda install gcc) their own compiler. So you need to make sure you have your environment variables set correctly to load the right library. Typically, this is done by setting your LD_LIBRARY_PATH to include the anaconda2/lib directory, which is probably where the correct libgomp is located. And also set your PATH to include anaconda2/bin to get their compiler. You can check which libgomp library treecorr was compiled to use with $ ldd /home/nitesh/anaconda2/lib/python2.7/site-packages/treecorr/_treecorr.so On a machine I have using anaconda, this is what is output: $ ldd ~/work/.conda/envs/py2.7/lib/python2.7/site-packages/TreeCorr-3.4-py2.7-linux-x86_64.egg/treecorr/_treecorr.so linux-vdso.so.1 => (0x00007ffdc44e0000) libpython2.7.so.1.0 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libpython2.7.so.1.0 (0x00007fe1fbefc000) libstdc++.so.6 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libstdc++.so.6 (0x00007fe1fbbc2000) libm.so.6 => /lib64/libm.so.6 (0x00007fe1fb93e000) libgomp.so.1 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libgomp.so.1 (0x00007fe1fb71b000) libgcc_s.so.1 => /astro/u/mjarvis/.conda/envs/py2.7/lib/libgcc_s.so.1 (0x00007fe1fb509000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe1fb2ec000) libc.so.6 => /lib64/libc.so.6 (0x00007fe1faf58000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fe1fad54000) libutil.so.1 => /lib64/libutil.so.1 (0x00007fe1fab51000) /lib64/ld-linux-x86-64.so.2 (0x00007fe1fc5bc000) librt.so.1 => /lib64/librt.so.1 (0x00007fe1fa949000) And the /astro/u/mjarvis/.conda/envs/py2.7/lib/ directory is in my LD_LIBRARY_PATH environment variable. If you have a different libgomp library in your library path, then you can sometimes get mismatches in what setup.py found at build time and what python finds at runtime. Without more details about your particular setup, I can't really diagnose further, but hopefully this gives you enough to investigate on your own. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AjbeAy6wDV7qWC7Ll9IDC79ePiVaKs6Tks5tcBZ0gaJpZM4Ks5W1> .

-- *Nitesh Bhardwaj* *J. H. Oort Building* *Niels Bohrweg 2* *NL 2333 CA Leiden* *Office: 470* *Phone: [phone]*

[rmjarvis]

Apparently not. It depends on what compiler gets used by setup.py and what your LD_LIBRARY_PATH is. The problem is some kind of mismatch between what gets used at build time and what is found at run time.

My best guess is that your compiler is not the Anaconda gcc, but rather the system gcc, which has its own libgomp that is not the one listed above. Hence a mismatch. You might have success with switching to the Anaconda compiler (via conda install gcc).

But I can't properly diagnose this from what you've posted. You'll need to investigate this on your own some I'm afraid.

[niteshb0804]

I understand. I followed the steps quite precisely as mentioned on your GitHub page for TreeCorr. starting from pip install My LD_LIBRARY_PATH is /home/nitesh/anaconda2/lib Additionally, I installed libffi part using apt-get and manually added the path info export C_INCLUDE_PATH=$C_INCLUDE_PATH:{prefix}/include export LIBRARY_PATH=$LIBRARY_PATH:{prefix}/lib export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:{prefix}/lib to .bashrc, where my prefix is /home/nitesh/anaconda2 Kindly do suggest alternates if things continue to not work out, as I have tried the documented means for the diagnosis.

…

On Thu, Mar 8, 2018 at 12:42 AM, Mike Jarvis ***@***.***> wrote: Apparently not. It depends on what compiler gets used by setup.py and what your LD_LIBRARY_PATH is. The problem is some kind of mismatch between what gets used at build time and what is found at run time. My best guess is that your compiler is not the Anaconda gcc, but rather the system gcc, which has its own libgomp that is not the one listed above. Hence a mismatch. You might have success with switching to the Anaconda compiler (via conda install gcc). But I can't properly diagnose this from what you've posted. You'll need to investigate this on your own some I'm afraid. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AjbeA9LYMBdQ2zLD-rFZltvN69a4zkHNks5tcDE0gaJpZM4Ks5W1> .

-- *Nitesh Bhardwaj* *J. H. Oort Building* *Niels Bohrweg 2* *NL 2333 CA Leiden* *Office: 470* *Phone: [phone]*

[Hasibus]

HI ,
I am facing also Anconda 3 Library error problem when I run of my python script

[Hasibus]

Thanls .

The error you are showing seems to be in flask, not TreeCorr. I don't know anything about flask, so there's not much I can do with this.

Thanks.
Someone please help me