pip install fails for Python 3.7 about compression

[fab-jul]

Well, maybe using conda is flawed anyway. Trying to import tensorflow_compression I get

tensorflow.python.framework.errors_impl.NotFoundError: [...]/libtensorflow_compression.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv

I have Python 3.6.9, TF 1.14.0, pip 19.1.1, all installed in a conda env.

[jonycgn]

Hi Fabian,

we have Python 2.7, 3.7 on Darwin, and Python 2.7, 3.3-3.6 on Linux. That's because the custom-op Docker image provided by the TensorFlow team we need to use to build the Linux wheel files doesn't support 3.7 yet.

Are you sure TF 1.14 actually gets used? The error message looks like it's trying to access TF 1.13.

[fab-jul]

I'm not sure which TF got used. I now created a fresh conda with only TF 1.14. Doing python -c "import tensorflow as tf; print(tf.__version__)" gives 1.14.0. pip install tensorflow-compression --no-cache-dir says it's installing tensorflow_compression-1.2b2-cp36-cp36m-manylinux1_x86_64.whl. Is there something else I can check?

Maybe it's also due to conda? I installed TensorFlow with conda, maybe I should use pip? OTOH TensorFlow is shown when I do pip list.

[fab-jul]

Ok, created yet another conda environment, with only python and pip. After pip install tensorflow; pip install tensorflow-compression, the tests run, so all good. Not sure if I can get this to work with GPU support, and why it doesn't work with conda install tensorflow-gpu.

[jonycgn]

When you do conda install, I think it pulls packages from Continuum's repository, not the official TensorFlow ones. It's possible that these were compiled in a different way, or contain other changes.

Regarding GPU support: Have you tried pip install tensorflow-gpu tensorflow-compression? If not, could you verify that that works? This would be good for us to know before we release 1.2.

[fab-jul]

That has different problems. If I do

conda create --name gpupip10 python=3.6 cudatoolkit=10.0  
conda activate gpupip10
pip install tensorflow-gpu tensorflow-compression
python -c "import tensorflow_compression"

Then that works, i.e., no errors. However, now TF is not installed with proper GPU support. The test fails with

Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory

AFAIK, conda install tensorflow-gpu is the way to get proper GPU support for TensorFlow with conda. But this does apparently not play nice with tensorflow-compression. I suppose the way to go is to manually install CUDA and cudnn, and then everything would work (which I did in the past, where python and pip is provided by conda, and I make sure libcudnn is in $LD_LIBRARY_PATH). It's just that conda is much more convenient for our setup.

[jonycgn]

cudnn is not part of the cuda toolkit. Could it be that you’re just missing a dependency in that environment? As in, can you do `conda install cudnn` or similar to fix the error?

…

On Aug 13, 2019, at 01:04, fab-jul ***@***.***> wrote: That has different problems. If I do conda create --name gpupip10 python=3.6 cudatoolkit=10.0 conda activate gpupip10 pip install tensorflow-gpu tensorflow-compression python -c "import tensorflow_compression" Then that works, i.e., no errors. However, now TF is not installed with proper GPU support. The test fails with Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory AFAIK, conda install tensorflow-gpu is the way to get proper GPU support for TensorFlow with conda. But this does apparently not play nice with tensorflow-compression. I suppose the way to go is to manually install CUDA and cudnn, and then everything would work (which I did in the past, where python and pip is provided by conda, and I make sure libcudnn is in $LD_LIBRARY_PATH). It's just that conda is much more convenient for our setup. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[fab-jul]

That's weird. I thought I tried that before posting and it gave me errors. Now it works, tests give me

Ran 99 tests in 149.250s

OK (skipped=11)

[fab-jul]

So, to summarize, it's possible to

conda create --name ENV_NAME python=3.6 cudatoolkit=10.0 cudnn
conda activate ENV_NAME
pip install tensorflow-gpu tensorflow-compression

[jonycgn]

Good! We'll take that into account in the installation instructions.

[xinziXu]

Hi, I used commad lines:

conda create --name ENV_NAME python=3.6 cudatoolkit=10.0 cudnn
conda activate ENV_NAME
pip install tensorflow-gpu tensorflow-compression

And I ran test code successfully. However, I met an error when running tfci.py. The error says

Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.

How can I fix it?

[jonycgn]

Could you post the full error message, please?

[jonycgn]

I responded in the other issue. Closing this again.

[ndenStanford]

Hello Johannes,

I also have a problem trying to install the library on Mac. I have tried the following command.

conda create --name ENV_NAME python=3.6
conda activate ENV_NAME
pip install tensorflow tensorflow-compression

However, I am still getting the error similar to Fabian

ERROR: Could not find a version that satisfies the requirement tensorflow-compression
ERROR: No matching distribution found for tensorflow-compression

Should the same command work at this current stage of the library development? Thanks!

[jonycgn]

Hi, I just fixed a problem in the beta release pip files. Please try pinning the version numbers, as described in the README:

With this, you should get the stable version:
pip install tensorflow==1.15 tensorflow-compression==1.3

This should give you the beta version:
pip install tensorflow==2.3 tensorflow-compression==2.0b1

The latter should work for Python 3.6 to 3.8, for both MacOS and Linux. For the former, it will depend on the Python version.

[hgcewad754]

HI！I have install tensoflow 2.3 and tensorflow-gpu 2.3 in anaconda, but use "pip install tensorflow==2.3 tensorflow-compression==2.0b1" cannot install tensorflow-compression==2.0b1,it shows as below

ERROR: Could not find a version that satisfies the requirement tensorflow-compression (from versions: none)
ERROR: No matching distribution found for tensorflow-compression

can i install tensorflow compression after download the zip file?

[hgcewad754]

in windows

[hgcewad754]

I have seen all version of tfc , only for mac and linux?T T

[jonycgn]

Yes, pip packages are only available for Mac and Linux. Unfortunately, we can't support Windows at this point in time.

You may want to try installing Docker or WSL, but I can't provide much guidance there, since I don't have a Windows system myself.