Comments (15)
We weren't able to reproduce this error on our machines. I'm not sure if this is an Red Hat thing or what. I will explore this issue further. In the mean time, you may want to try to install in an Ubuntu 14.04 VM.
from crosscat.
I just had success installing crosscat on CentOS-7.0-1406-x86 (I used the minimal install, which required installing git, gcc, and g++). I followed the procedure currently outlined in the readme (which changed since you raised the issue) but replaced the apt-get
s with their yum
counterparts.
$ sudo yum install -y Cython boost-devel python-setuptools numpy scipy
$ git clone https://github.com/mit-probabilistic-computing-project/crosscat.git
$ cd crosscat
$ python setup.py build
$ sudo python setup.py install
I'm going to go ahead and close this issue, but please let me know if this process doesn't work for you.
from crosscat.
Any process that relies on 'sudo' is not workable: I am running this on a cluster with other users, and I don't have these rights. I'm not sure why this installation process would be different anyway - I have all the requirements of crosscat (Cython, numpy, scipy, etc..) in my virtualenv so I don't think that's the problem. It always complains about the lack of *.c files:
building 'crosscat.cython_code.CyclicComponentModel' extension
gcc -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c crosscat/cython_code/CyclicComponentModel.c -o build/temp.linux-x86_64-2.7/crosscat/cython_code/CyclicComponentModel.o
gcc -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/utils.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/utils.o
gcc -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/numerics.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/numerics.o
gcc -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/RandomNumberGenerator.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/RandomNumberGenerator.o
gcc -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/ComponentModel.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/ComponentModel.o
gcc -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/CyclicComponentModel.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/CyclicComponentModel.o
gcc: error: crosscat/cython_code/CyclicComponentModel.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 1
which sounds like a problem with setup.py not calling Cython or Cython not building the extension correctly.
from crosscat.
From a virtualenv, doing python setup.py build
and then pip install .
worked for me without sudo
(doing pip install .
alone did not work).
from crosscat.
I tried again, I recloned the repository. I get the same results with python setup.py build
as pip install .
:
$ git clone https://github.com/mit-probabilistic-computing-project/crosscat.git
Initialized empty Git repository in /ahg/regevdata/users/yarden/software/crosscat/.git/
remote: Counting objects: 15298, done.
remote: Total 15298 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (15298/15298), 7.96 MiB | 3.00 MiB/s, done.
Resolving deltas: 100% (8750/8750), done.
$ cd crosscat/
$ ls
LICENSE Makefile NOTICE README.md README_STARCLUSTER.md cpp_code/ crosscat/ debian/ docs/ examples/ git_unittest_hook.sh* requirements.txt scripts/ setup.py starcluster.config website/ www/
$ python setup.py build
which: no ccache in (/ahg/regevdata/users/yarden/myenv/bin:/home/unix/yarden/bin:/broad/lsf9/9.1/linux2.6-glibc2.3-x86_64/bin:/ahg/regevdata/users/yarden/myenv/bin:/home/unix/yarden/bin:/ahg/regevdata/users/yarden/myenv/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/emacs_24.2/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/mysql_5.6.20/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/perl_5.14.2/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bioperl_1.6.923/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/sqlite_3.6.19/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/tophat_2.0.10/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bowtie2_2.0.5:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bowtie_0.12.9:/broad/software/free/Linux/redhat_6_x86_64/pkgs/samtools/samtools_0.1.19/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/zlib_1.2.6/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bedtools_version-2.16.1/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/scipy_0.13.0-python-2.7.1-sqlite3-rtrees/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/matplotlib_1.3.1-python-2.7.1-sqlite3-rtrees/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/db_4.7.25/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/tcltk8.5.9/bin:/broad/lsf9/9.1/linux2.6-glibc2.3-x86_64/etc:/broad/software/free/Linux/redhat_6_x86_64/pkgs/lsf_drmaa-1.1.1/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/r_3.1.0-bioconductor-2.14/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/gcc_4.9.0/bin:/home/unix/yarden/bin:/broad/tools/NoArch/pkgs/local:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/unix/yarden/regevdata/bedtools2/bin:/home/unix/yarden/regevdata/software/sratoolkit.2.4.2-ubuntu64/bin:/home/unix/yarden/regevdata/software/ncftp-3.2.5/bin:/home/unix/yarden/bin/:/home/unix/yarden/regevdata/bedtools2/bin:/home/unix/yarden/regevdata/software/sratoolkit.2.4.2-ubuntu64/bin:/home/unix/yarden/regevdata/software/ncftp-3.2.5/bin:/home/unix/yarden/bin/:/home/unix/yarden/regevdata/bedtools2/bin:/home/unix/yarden/regevdata/software/sratoolkit.2.4.2-ubuntu64/bin:/home/unix/yarden/regevdata/software/ncftp-3.2.5/bin:/home/unix/yarden/bin/)
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/crosscat
copying crosscat/EngineTemplate.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/starcluster_plugin.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/LocalEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/settings.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/JSONRPCEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/CrossCatClient.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/HadoopEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/IPClusterEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/__init__.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/MultiprocessingEngine.py -> build/lib.linux-x86_64-2.7/crosscat
creating build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/plot_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/validate_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/data_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/useCase_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/general_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/sample_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/file_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/api_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/hadoop_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/geweke_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/inference_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/mutual_information_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/diagnostic_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/enumerate_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/timing_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/xnet_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/convergence_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
creating build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/convergence_test.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/parse_convergence_results.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/generate_convergence_script.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/plot_convergence_results.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/automated_convergence_tests.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
creating build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/server_jsonrpc.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/test_resume.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/stub_client_jsonrpc.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/test_engine.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
creating build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_multinomial_impute.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/mixed_state_test.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_sample.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_missing_value.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/continuous_component_model_test.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_pred_prob_and_density.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/setup.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/state_test.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
creating build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/timing_analysis.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/cpp_long_tests.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/test_sampler_enumeration.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/cpp_unit_tests.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/test_log_likelihood.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/geweke_on_schemas.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/test_pred_prob_and_density.py -> build/lib.linux-x86_64-2.7/crosscat/tests
creating build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_predictive_confidence.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_kl_divergence_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_impute_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/synthetic_data_generator.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/quality_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_component_model_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_mixture_inference_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
creating build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/CyclicComponentModel.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/MultinomialComponentModel.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/ContinuousComponentModel.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
running build_ext
building 'crosscat.cython_code.CyclicComponentModel' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/crosscat
creating build/temp.linux-x86_64-2.7/crosscat/cython_code
creating build/temp.linux-x86_64-2.7/cpp_code
creating build/temp.linux-x86_64-2.7/cpp_code/src
gcc -pthread -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c crosscat/cython_code/CyclicComponentModel.c -o build/temp.linux-x86_64-2.7/crosscat/cython_code/CyclicComponentModel.o
gcc -pthread -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/utils.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/utils.o
gcc -pthread -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/numerics.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/numerics.o
gcc -pthread -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/RandomNumberGenerator.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/RandomNumberGenerator.o
gcc -pthread -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/ComponentModel.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/ComponentModel.o
gcc -pthread -fno-strict-aliasing -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/tcltk8.5.9/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/sqlite_3.7.5/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/db_4.7.25/include -DNDEBUG -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/CyclicComponentModel.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/CyclicComponentModel.o
gcc: error: crosscat/cython_code/CyclicComponentModel.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 1
Is there a recommended Cython version? My cython version is:
$ python -c 'import cython; print cython.__version__'
0.21.1
Also, I noticed the first line in the output is:
which: no ccache in ...
I'm not sure what that means, but maybe it's related to the issue?
from crosscat.
I get the no ccache message as well. It shouldn't affect crosscat's building--just the time it takes to build. I was originally using cython 0.21.2 but I downgraded to 0.21.1 and was still unable to recreate the issue.
I notice that you are not getting any compiler output after the gcc
commands. Your compiler flags are definitely different than mine. Could you please try running with
$ CFLAGS="-O2 -Wall" python setup.py build
and share the output? Also what version of gcc are you running?
from crosscat.
When run the command you suggested with CFLAGS
set, I get:
which: no ccache in (/broad/lsf9/9.1/linux2.6-glibc2.3-x86_64/bin:/ahg/regevdata/users/yarden/myenv/bin:/home/unix/yarden/bin:/ahg/regevdata/users/yarden/myenv/bin:/home/unix/yarden/bin:/ahg/regevdata/users/yarden/myenv/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/emacs_24.2/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/mysql_5.6.20/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/perl_5.14.2/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bioperl_1.6.923/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/sqlite_3.6.19/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/tophat_2.0.10/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bowtie2_2.0.5:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bowtie_0.12.9:/broad/software/free/Linux/redhat_6_x86_64/pkgs/samtools/samtools_0.1.19/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/zlib_1.2.6/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/bedtools_version-2.16.1/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/scipy_0.13.0-python-2.7.1-sqlite3-rtrees/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/matplotlib_1.3.1-python-2.7.1-sqlite3-rtrees/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/db_4.7.25/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/tcltk8.5.9/bin:/broad/lsf9/9.1/linux2.6-glibc2.3-x86_64/etc:/broad/software/free/Linux/redhat_6_x86_64/pkgs/lsf_drmaa-1.1.1/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/r_3.1.0-bioconductor-2.14/bin:/broad/software/free/Linux/redhat_6_x86_64/pkgs/gcc_4.9.0/bin:/home/unix/yarden/bin:/broad/tools/NoArch/pkgs/local:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/unix/yarden/regevdata/bedtools2/bin:/home/unix/yarden/regevdata/software/sratoolkit.2.4.2-ubuntu64/bin:/home/unix/yarden/regevdata/software/ncftp-3.2.5/bin:/home/unix/yarden/bin/:/home/unix/yarden/regevdata/bedtools2/bin:/home/unix/yarden/regevdata/software/sratoolkit.2.4.2-ubuntu64/bin:/home/unix/yarden/regevdata/software/ncftp-3.2.5/bin:/home/unix/yarden/bin/:/home/unix/yarden/regevdata/bedtools2/bin:/home/unix/yarden/regevdata/software/sratoolkit.2.4.2-ubuntu64/bin:/home/unix/yarden/regevdata/software/ncftp-3.2.5/bin:/home/unix/yarden/bin/)
running build
running build_py
copying crosscat/EngineTemplate.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/starcluster_plugin.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/LocalEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/settings.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/JSONRPCEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/CrossCatClient.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/HadoopEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/IPClusterEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/__init__.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/MultiprocessingEngine.py -> build/lib.linux-x86_64-2.7/crosscat
copying crosscat/utils/plot_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/validate_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/data_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/useCase_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/general_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/sample_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/file_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/api_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/hadoop_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/geweke_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/inference_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/mutual_information_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/diagnostic_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/enumerate_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/timing_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/xnet_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/utils/convergence_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/utils
copying crosscat/convergence_analysis/convergence_test.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/parse_convergence_results.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/generate_convergence_script.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/plot_convergence_results.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/automated_convergence_tests.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/convergence_analysis/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/convergence_analysis
copying crosscat/jsonrpc_http/server_jsonrpc.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/test_resume.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/stub_client_jsonrpc.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/jsonrpc_http/test_engine.py -> build/lib.linux-x86_64-2.7/crosscat/jsonrpc_http
copying crosscat/cython_code/test_multinomial_impute.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/mixed_state_test.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_sample.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_missing_value.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/continuous_component_model_test.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/test_pred_prob_and_density.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/setup.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/cython_code/state_test.py -> build/lib.linux-x86_64-2.7/crosscat/cython_code
copying crosscat/tests/timing_analysis.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/cpp_long_tests.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/test_sampler_enumeration.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/cpp_unit_tests.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/test_log_likelihood.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/geweke_on_schemas.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/test_pred_prob_and_density.py -> build/lib.linux-x86_64-2.7/crosscat/tests
copying crosscat/tests/quality_tests/test_predictive_confidence.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_kl_divergence_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_impute_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/synthetic_data_generator.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/quality_test_utils.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_component_model_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/quality_tests/test_mixture_inference_quality.py -> build/lib.linux-x86_64-2.7/crosscat/tests/quality_tests
copying crosscat/tests/component_model_extensions/CyclicComponentModel.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/MultinomialComponentModel.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/__init__.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
copying crosscat/tests/component_model_extensions/ContinuousComponentModel.py -> build/lib.linux-x86_64-2.7/crosscat/tests/component_model_extensions
running build_ext
building 'crosscat.cython_code.CyclicComponentModel' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/crosscat
creating build/temp.linux-x86_64-2.7/crosscat/cython_code
creating build/temp.linux-x86_64-2.7/cpp_code
creating build/temp.linux-x86_64-2.7/cpp_code/src
gcc -pthread -DNDEBUG -O2 -Wall -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c crosscat/cython_code/CyclicComponentModel.c -o build/temp.linux-x86_64-2.7/crosscat/cython_code/CyclicComponentModel.o
gcc -pthread -DNDEBUG -O2 -Wall -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/utils.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/utils.o
gcc -pthread -DNDEBUG -O2 -Wall -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/numerics.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/numerics.o
gcc -pthread -DNDEBUG -O2 -Wall -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/RandomNumberGenerator.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/RandomNumberGenerator.o
gcc -pthread -DNDEBUG -O2 -Wall -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/ComponentModel.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/ComponentModel.o
gcc -pthread -DNDEBUG -O2 -Wall -fPIC -Icpp_code/include/CrossCat -I/ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/numpy/core/include -I/broad/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/include/python2.7 -c cpp_code/src/CyclicComponentModel.cpp -o build/temp.linux-x86_64-2.7/cpp_code/src/CyclicComponentModel.o
gcc: error: crosscat/cython_code/CyclicComponentModel.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 1
cpp_code/src/CyclicComponentModel.cpp: In member function 'virtual double CyclicComponentModel::get_draw_constrained(int, std::vector<double>) const':
cpp_code/src/CyclicComponentModel.cpp:199:23: warning: variable 'b_a' set but not used [-Wunused-but-set-variable]
double kappa, a, b, b_a;
^
cpp_code/src/CyclicComponentModel.cpp:245:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
cpp_code/src/utils.cpp: In function 'void construct_cyclic_specific_hyper_grid(int, std::vector<double>, std::vector<double>&, std::vector<double>&)':
cpp_code/src/utils.cpp:428:10: warning: unused variable 'var' [-Wunused-variable]
double var = calc_sum_sq_deviation(col_data)/N;
^
cpp_code/src/numerics.cpp: In function 'double numerics::vonmises_rand(double, double, int)':
cpp_code/src/numerics.cpp:84:20: warning: unused variable 'vmr_a' [-Wunused-variable]
double vmr_a = vmr;
^
cpp_code/src/numerics.cpp:91:20: warning: unused variable 'vmr_a' [-Wunused-variable]
double vmr_a = vmr;
My gcc is:
$ gcc --version
gcc (GCC) 4.9.0
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
from crosscat.
I am going to spend some time today with our packaging. It's probably best to distribute the cython-generated .cpp files. I'll let you know when it's ready.
from crosscat.
The GCC version and CFLAGS are irrelevant, and pip is a red herring. What's happening (or, failing to happen) is that Cython is somehow not getting run to compile the .pyx files into .c/.cpp files. The problem is not that Cython is not available -- if that were the case, setup.py would fail altogether because it imports from Cython.Distutils.
For 'source distributions' generated from the real source in Git, i.e. the product of python setup.py sdist
, it may be appropriate to include the Cython-generated .c/.cpp files. But that doesn't resolve the question of why Cython isn't getting run for @yarden.
from crosscat.
I'm using the version of Cython packaged with Ubuntu 14.04, namely 0.20.1post0. Generally I don't trust pip at all and don't tolerate using it on any of my systems, but I'll see what happens if I make it install Cython through pip in a virtualenv.
from crosscat.
I removed my crosscat and cython system packages from apt, and ran
virtualenv /tmp/venv
. /tmp/venv/activate
pip install cython
pip install .
and the installation succeeded. This is from Ubuntu 14.04, and this installed Cython 0.21.2 in the virtualenv. (Without pip install cython
first, running pip install .
will fail because setup.py imports Cython.Distutils. I don't know enough about how pip works to make this DTRT.)
from crosscat.
It looks like maybe pip and Cython specifically have a problem (http://stackoverflow.com/questions/26069080/pip-does-not-acknowledge-cython). However, it does not explain the current issue. I tried doing:
$ pip install --upgrade cython
$ pip install .
and got the same errors:
...
gcc: error: crosscat/cython_code/CyclicComponentModel.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 1
Using sdist
to make a non-Cython dependent source distribution would be great.
Python packaging is a broken business, but using a package manager like pip+virtualenv or easy_install is pretty much the only option for doing a complete install. python setup.py
does not do a complete install of packages and obviously doesn't let users install the program and its dependencies from pypi or other repositories with a single command. As far as I know, pip+virtualenv is the only game in town for managing multiple sets of potentially conflicting Python dependencies, which inevitably happens.
Update: I upgraded setuptools and now it appears to work:
$ pip install setuptools
Requirement already satisfied (use --upgrade to upgrade): setuptools in /ahg/regevdata/users/yarden/myenv/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
$ pip install --upgrade setuptools
Collecting setuptools from https://pypi.python.org/packages/3.4/s/setuptools/setuptools-12.0.5-py2.py3-none-any.whl#md5=87647eb8380c5da10e63fdd7274d8b49
Downloading setuptools-12.0.5-py2.py3-none-any.whl (502kB)
100% |################################| 503kB 437kB/s
Installing collected packages: setuptools
Found existing installation: setuptools 0.6rc11
Uninstalling setuptools-0.6rc11:
Successfully uninstalled setuptools-0.6rc11
I'm able to run the dha_example.py
at least:
$ python examples/dha_example.py www/data/dha.csv --num_chains 2 --num_transitions 2
[6.7, 5.8, 15.9, 87.7, 92.0]
[6.71, 3.24, 15.05, 87.97, 86.27]
[9.5, 5.8, 21.4, 86.7, 89.2]
[9.03, 7.38, 19.78, 87.76, 86.24]
[9.2, 15.2, 20.2, 82.2, 85.3]
[11.19, 10.06, 22.88, 86.14, 84.61]
[9.4, 13.2, 25.1, 92.3, 92.2]
[11.3, 9.92, 23.03, 89.08, 87.63]
[11.3, 9.7, 23.6, 80.8, 86.3]
[11.22, 10.0, 22.79, 86.14, 84.43]
[7.8, 8.4, 19.9, 89.5, 89.4]
[9.08, 7.37, 19.87, 89.13, 87.68]
[8.4, 7.6, 19.5, 89.7, 92.4]
[11.37, 9.83, 23.04, 89.21, 87.71]
[6.5, 2.7, 15.1, 87.1, 85.5]
[7.28, 4.47, 17.08, 87.86, 85.97]
So it looks like old setuptools is the problem.
from crosscat.
Problem caused by prehistoric version of setuptools, not by anything in crosscat. We can open another issue for running cython during python setup.py sdist
and making python setup.py build
DTRT (if possible given how unbelievably brain-damaged the Python build nonsense is).
from crosscat.
A simple fix for the problem that I had is to add setuptools >= x
in the setup.py
requirements
section, to ensure old versions aren't used.
About Cython: It's possible to make setup.py sdist do the right thing with Cython - I have a template that does that here, FYI: https://github.com/yarden/MISO/blob/newmiso/setup.py
But that issue is not pressing for me now that Crosscat is working.
from crosscat.
FYI, we just pushed changes to make python setup.py sdist
run Cython so that python setup.py build
from the resulting 'source' distribution need not.
from crosscat.
Related Issues (20)
- test_multiple_col_ensure.py is stochastic
- pls help run on windows 10 machine with anaconda python 2.7 HOT 3
- dha_example_multiprocessing HOT 1
- missing dependencies / environment compatibilities HOT 5
- rename pypi package: CrossCat -> crosscat
- How to sample from the posterior HOT 12
- Is it possible to engage more than one core? HOT 5
- .
- How to avoid collapsing to one view?
- Prevent test suite from generating matplotlib figures using X11-based backend HOT 3
- cannot import crosscat.LocalEngine HOT 1
- Allow multistate queries in sample_utils to utilize multiprocessing
- Python 3 compatibility HOT 1
- Hadoop state non-functional HOT 9
- continuous component model can't handle constant column HOT 1
- Crosscat hyperprior grid on variance parameter is broader than it needs to be HOT 9
- Crosscat should use 256-bit seeds
- "pip install crosscat" fails HOT 3
- Crosscat build fails in anaconda environment: "cpp_code/src/weakprng.cpp:314:38: error: βUINT64_Cβ was not declared in this scope~ HOT 6
- Multistate conditional sampling is inaccurate HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crosscat.