kaggle / docker-python Goto Github PK
View Code? Open in Web Editor NEWKaggle Python docker image
License: Apache License 2.0
Kaggle Python docker image
License: Apache License 2.0
When I pull the container from dockerhub or build it with this script the plotly package isn't available. I have to manually install it by running and committing to the container. I can see where the pip install plotly is happening but I'm not sure why it isn't getting installed into the container on dockerhub.
Thanks!
Hi
is it possible for you to add the md5 package to be able to use this please ?
https://github.com/msegala/Ensemble-Tools/blob/master/blend_proba.py
Thanks
I pulled kaggle python image and seems to work fine, but I'm not sure what to do to use my GPU with theano or tensorflow.
Is it as simple as changing flags in .theanorc file? Can i edit it directly in the docker image?
New to docker here :)
Collecting vowpalwabbit
Downloading vowpalwabbit-8.4.0.tar.gz (10.2MB)
Building wheels for collected packages: vowpalwabbit
Running setup.py bdist_wheel for vowpalwabbit: started
Running setup.py bdist_wheel for vowpalwabbit: still running...
Running setup.py bdist_wheel for vowpalwabbit: finished with status 'error'
Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/tmp7jaam29ppip-wheel- --python-tag cp36:
Failed building wheel for vowpalwabbit
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/pyvw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/init.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/sklearn_vw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
running egg_info
writing vowpalwabbit.egg-info/PKG-INFO
writing dependency_links to vowpalwabbit.egg-info/dependency_links.txt
writing top-level names to vowpalwabbit.egg-info/top_level.txt
warning: manifest_maker: standard file '-c' not found
reading manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '.dll' under directory 'bin'
warning: no previously-included files matching '.o' found anywhere in distribution
warning: no previously-included files matching '.exe' found anywhere in distribution
warning: no previously-included files matching '.pyc' found anywhere in distribution
writing manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
running build_ext
cd vowpalwabbit; make -j 8 things
make[1]: Entering directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/vowpalwabbit'
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c main.cc -o main.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c hash.cc -o hash.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c global_data.cc -o global_data.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c io_buf.cc -o io_buf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_regressor.cc -o parse_regressor.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_primitives.cc -o parse_primitives.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c unique_sort.cc -o unique_sort.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cache.cc -o cache.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c rand48.cc -o rand48.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c simple_label.cc -o simple_label.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c multiclass.cc -o multiclass.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c oaa.cc -o oaa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c multilabel_oaa.cc -o multilabel_oaa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c boosting.cc -o boosting.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c ect.cc -o ect.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c marginal.cc -o marginal.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c autolink.cc -o autolink.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c binary.cc -o binary.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c lrq.cc -o lrq.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cost_sensitive.cc -o cost_sensitive.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c multilabel.cc -o multilabel.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c label_dictionary.cc -o label_dictionary.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c csoaa.cc -o csoaa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb.cc -o cb.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_adf.cc -o cb_adf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_algs.cc -o cb_algs.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search.cc -o search.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_meta.cc -o search_meta.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_sequencetask.cc -o search_sequencetask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_dep_parser.cc -o search_dep_parser.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_hooktask.cc -o search_hooktask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_multiclasstask.cc -o search_multiclasstask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_entityrelationtask.cc -o search_entityrelationtask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_graph.cc -o search_graph.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_example.cc -o parse_example.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c scorer.cc -o scorer.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c network.cc -o network.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_args.cc -o parse_args.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c accumulate.cc -o accumulate.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c gd.cc -o gd.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c learner.cc -o learner.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c mwt.cc -o mwt.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c lda_core.cc -o lda_core.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c gd_mf.cc -o gd_mf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c mf.cc -o mf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c bfgs.cc -o bfgs.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c noop.cc -o noop.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c print.cc -o print.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c example.cc -o example.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parser.cc -o parser.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c loss_functions.cc -o loss_functions.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c sender.cc -o sender.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c nn.cc -o nn.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c confidence.cc -o confidence.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c bs.cc -o bs.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cbify.cc -o cbify.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c explore_eval.cc -o explore_eval.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c topk.cc -o topk.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c stagewise_poly.cc -o stagewise_poly.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c log_multi.cc -o log_multi.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c recall_tree.cc -o recall_tree.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c active.cc -o active.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c active_cover.cc -o active_cover.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c kernel_svm.cc -o kernel_svm.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c best_constant.cc -o best_constant.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c ftrl.cc -o ftrl.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c svrg.cc -o svrg.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c lrqfa.cc -o lrqfa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c interact.cc -o interact.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c comp_io.cc -o comp_io.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c interactions.cc -o interactions.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c vw_exception.cc -o vw_exception.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c vw_validate.cc -o vw_validate.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c audit_regressor.cc -o audit_regressor.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c gen_cs_example.cc -o gen_cs_example.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_explore.cc -o cb_explore.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c action_score.cc -o action_score.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_explore_adf.cc -o cb_explore_adf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c OjaNewton.cc -o OjaNewton.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_example_json.cc -o parse_example_json.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c baseline.cc -o baseline.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c classweight.cc -o classweight.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c allreduce_sockets.cc -o allreduce_sockets.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c allreduce_threads.cc -o allreduce_threads.o
ar rcs liballreduce.a allreduce_sockets.o allreduce_threads.o vw_exception.o
ar rcs libvw.a hash.o global_data.o io_buf.o parse_regressor.o parse_primitives.o unique_sort.o cache.o rand48.o simple_label.o multiclass.o oaa.o multilabel_oaa.o boosting.o ect.o marginal.o autolink.o binary.o lrq.o cost_sensitive.o multilabel.o label_dictionary.o csoaa.o cb.o cb_adf.o cb_algs.o search.o search_meta.o search_sequencetask.o search_dep_parser.o search_hooktask.o search_multiclasstask.o search_entityrelationtask.o search_graph.o parse_example.o scorer.o network.o parse_args.o accumulate.o gd.o learner.o mwt.o lda_core.o gd_mf.o mf.o bfgs.o noop.o print.o example.o parser.o loss_functions.o sender.o nn.o confidence.o bs.o cbify.o explore_eval.o topk.o stagewise_poly.o log_multi.o recall_tree.o active.o active_cover.o kernel_svm.o best_constant.o ftrl.o svrg.o lrqfa.o interact.o comp_io.o interactions.o vw_exception.o vw_validate.o audit_regressor.o gen_cs_example.o cb_explore.o action_score.o cb_explore_adf.o OjaNewton.o parse_example_json.o baseline.o classweight.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -o vw main.o -L. -l vw -l allreduce -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -l boost_program_options -l pthread -l z
make[1]: Leaving directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/vowpalwabbit'
cd python; make things
make[1]: Entering directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/python'
Using Python 3.6
/usr/bin/g++ -std=c++0x -I/opt/conda/include/python3.6m -I/opt/conda/include/python3.6m -I /usr/include -I ../rapidjson/include -fPIC -c pylibvw.cc -o pylibvw.o
/usr/bin/g++ -shared -Wl,--export-dynamic pylibvw.o -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -L/opt/conda/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/opt/conda/lib -lpython3.6m -lpthread -ldl -lutil -lrt -lm -Xlinker -export-dynamic ../vowpalwabbit/libvw.a ../vowpalwabbit/liballreduce.a -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -l boost_program_options -l pthread -l z -l boost_python-py34 -o pylibvw.so
lto1: fatal error: bytecode stream generated with LTO version 6.0 instead of the expected 3.0
compilation terminated.
lto-wrapper: /usr/bin/g++ returned 1 exit status
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
Makefile:55: recipe for target 'pylibvw.so' failed
make[1]: *** [pylibvw.so] Error 1
make[1]: Leaving directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/python'
Makefile:118: recipe for target 'python' failed
make: *** [python] Error 2
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 211, in
tests_require=['tox']
File "/opt/conda/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 179, in run
self.run_command('build')
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 77, in run
File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
_build_ext.build_ext.run(self)
File "/opt/conda/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
self.build_extension(ext)
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 122, in build_extension
subprocess.check_call(['make', 'python'], cwd=path.join(here, 'src'), env=env)
File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', 'python']' returned non-zero exit status 2.
Running setup.py clean for vowpalwabbit
Failed to build vowpalwabbit
Installing collected packages: vowpalwabbit
Running setup.py install for vowpalwabbit: started
Running setup.py install for vowpalwabbit: finished with status 'error'
Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-aaw1oa16-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/pyvw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/__init__.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/sklearn_vw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
running egg_info
creating vowpalwabbit.egg-info
writing vowpalwabbit.egg-info/PKG-INFO
writing dependency_links to vowpalwabbit.egg-info/dependency_links.txt
writing top-level names to vowpalwabbit.egg-info/top_level.txt
writing manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'src'
warning: no files found matching '*.dll' under directory 'bin'
warning: no previously-included files matching '*.o' found anywhere in distribution
warning: no previously-included files matching '*.exe' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
running build_ext
make: *** No rule to make target 'clean'. Stop.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 211, in <module>
tests_require=['tox']
File "/opt/conda/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/install.py", line 61, in run
File "/opt/conda/lib/python3.6/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 77, in run
File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
_build_ext.build_ext.run(self)
File "/opt/conda/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
self.build_extension(ext)
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 105, in build_extension
prep()
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 65, in prep
subprocess.check_call(['make', 'clean'], cwd=path.join(here, 'src', 'python'))
File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', 'clean']' returned non-zero exit status 2.
----------------------------------------
Command "/opt/conda/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-aaw1oa16-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-vs7jgzog/vowpalwabbit/
Error: ResponseItem.ErrorDetail[code=1,message=The command '/bin/sh -c sed -i "s/httpredir.debian.org/debian.uchicago.edu/" /etc/apt/sources.list && apt-get update && apt-get install -y build-essential && conda update -y conda && conda update -y python && pip install tensorflow && apt-get install -y libboost-program-options-dev zlib1g-dev libboost-python-dev && cd /usr/lib/x86_64-linux-gnu/ && rm -f libboost_python.a && rm -f libboost_python.so && ln -sf libboost_python-py34.so libboost_python.so && ln -sf libboost_python-py34.a libboost_python.a && pip install vowpalwabbit && pip install seaborn python-dateutil dask pytagcloud pyyaml ggplot joblib husl geopy ml_metrics mne pyshp gensim && conda install -y -c conda-forge spacy && python -m spacy download en && apt-get -y install dbus fontconfig fontconfig-config fonts-dejavu-core fonts-droid ghostscript gsfonts hicolor-icon-theme libavahi-client3 libavahi-common-data libavahi-common3 libcairo2 libcap-ng0 libcroco3 libcups2 libcupsfilters1 libcupsimage2 libdatrie1 libdbus-1-3 libdjvulibre-text libdjvulibre21 libfftw3-double3 libfontconfig1 libfreetype6 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgomp1 libgraphite2-3 libgs9 libgs9-common libharfbuzz0b libijs-0.35 libilmbase6 libjasper1 libjbig0 libjbig2dec0 libjpeg62-turbo liblcms2-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-2 libmagickcore-6.q16-2-extra libmagickwand-6.q16-2 libnetpbm10 libopenexr6 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpaper-utils libpaper1 libpixman-1-0 libpng12-0 librsvg2-2 librsvg2-common libthai-data libthai0 libtiff5 libwmf0.2-7 libxcb-render0 libxcb-shm0 netpbm poppler-data p7zip-full && cd /usr/local/src && wget http://transloadit.imagemagick.org/download/ImageMagick.tar.gz && tar xzf ImageMagick.tar.gz && cd ls -d ImageMagick-*
&& pwd && ls -al && ./configure && make -j
Failed to deploy ' Dockerfile: Dockerfile': The command '/bin/sh -c sed -i "s/httpredir.debian.org/debian.uchicago.edu/" /etc/apt/sources.list && apt-get update && apt-get install -y build-essential && conda update -y conda && conda update -y python && pip install tensorflow && apt-get install -y libboost-program-options-dev zlib1g-dev libboost-python-dev && cd /usr/lib/x86_64-linux-gnu/ && rm -f libboost_python.a && rm -f libboost_python.so && ln -sf libboost_python-py34.so libboost_python.so && ln -sf libboost_python-py34.a libboost_python.a && pip install vowpalwabbit && pip install seaborn python-dateutil dask pytagcloud pyyaml ggplot joblib husl geopy ml_metrics mne pyshp gensim && conda install -y -c conda-forge spacy && python -m spacy download en && apt-get -y install dbus fontconfig fontconfig-config fonts-dejavu-core fonts-droid ghostscript gsfonts hicolor-icon-theme libavahi-client3 libavahi-common-data libavahi-common3 libcairo2 libcap-ng0 libcroco3 libcups2 libcupsfilters1 libcupsimage2 libdatrie1 libdbus-1-3 libdjvulibre-text libdjvulibre21 libfftw3-double3 libfontconfig1 libfreetype6 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgomp1 libgraphite2-3 libgs9 libgs9-common libharfbuzz0b libijs-0.35 libilmbase6 libjasper1 libjbig0 libjbig2dec0 libjpeg62-turbo liblcms2-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-2 libmagickcore-6.q16-2-extra libmagickwand-6.q16-2 libnetpbm10 libopenexr6 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpaper-utils libpaper1 libpixman-1-0 libpng12-0 librsvg2-2 librsvg2-common libthai-data libthai0 libtiff5 libwmf0.2-7 libxcb-render0 libxcb-shm0 netpbm poppler-data p7zip-full && cd /usr/local/src && wget http://transloadit.imagemagick.org/download/ImageMagick.tar.gz && tar xzf ImageMagick.tar.gz && cd ls -d ImageMagick-*
&& pwd && ls -al && ./configure && make -j $(nproc) && make install && cd ../ && rm -rf ImageMagick* && apt-get -y install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev && apt-get -y install libtbb2 libtbb-dev libjpeg-dev libtiff-dev libjasper-dev && apt-get -y install cmake && cd /usr/local/src && git clone --depth 1 https://github.com/Itseez/opencv.git && cd opencv && mkdir build && cd build && cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_FFMPEG=OFF -D WITH_V4L=ON -D WITH_QT=OFF -D WITH_OPENGL=ON -D PYTHON3_LIBRARY=/opt/conda/lib/libpython3.6m.so -D PYTHON3_INCLUDE_DIR=/opt/conda/include/python3.6m/ -D PYTHON_LIBRARY=/opt/conda/lib/libpython3.6m.so -D PYTHON_INCLUDE_DIR=/opt/conda/include/python3.6m/ -D BUILD_PNG=TRUE .. && make -j $(nproc) && make install && echo "/usr/local/lib/python3.6/site-packages" > /etc/ld.so.conf.d/opencv.conf && ldconfig && cp /usr/local/lib/python3.6/site-packages/cv2.cpython-36m-x86_64-linux-gnu.so /opt/conda/lib/python3.6/site-packages/ && rm -rf /usr/local/src/opencv && rm -rf /root/.cache/pip/* && apt-get autoremove -y && apt-get clean' returned a non-zero code: 1
I'm playing with SQL stuff for a competition and am manually loading that in now. Installs with a clean "pip install psycopg2" - thanks!
Dask-ML enables parallel and distributed machine learning using Dask alongside existing machine learning libraries like Scikit-Learn.
...
When running
from skimage.util.montage import montage2d
the following error gets thrown in the log file
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
For the 2017 Kaggle Data Science Bowl the tutorial (https://github.com/booz-allen-hamilton/DSB3Tutorial) uses the SimpleITK package for loading and processing DICOMS a tool that is much better suited than pydicom for 3D data. Can this be added to the Docker Image / Kernels?
Hi guys,
I have just pulled recent image of kaggle/python from docker repo, and I face the following problem when trying to import h2o:
In [1]: import h2o
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-707498249ab8> in <module>()
----> 1 import h2o
/opt/conda/lib/python3.4/site-packages/h2o/__init__.py in <module>()
398 __version__ = "3.6.0.8"
399 from h2o import *
--> 400 from model import *
401 from demo import *
402 from h2o_logging import *
ImportError: No module named 'model'
I have checked and same issue is on kaggle scripts, when running it on kaggle website
Would you guys mind downgrading pandas to last stable version that is 20.3 instead of 21.0 -- lot of kernels are failing because of pandas-dev/pandas#18186
Dont know when they will fix this.
Hello Kaggle team,
Can you add the sklearn-pandas package.
Command is "pip install sklearn-pandas" but I'm unable to test it in docker for the moment
Sklearn-pandas provides a bridge between Scikit-Learn’s machine learning methods and pandas-style Data Frames.
Doc: https://github.com/paulgb/sklearn-pandas
Thanks
I'm looking at OpenStreetMap data for the Sberbank competition and saw this article http://andrewgaidus.com/Convert_OSM_Data/ that led me to GDAL (which has 'ogr' to read geodata files including OSM format) - https://pypi.python.org/pypi/GDAL
edit: It does seem to require 'gdal-bin' and 'libgdal-dev' in ubuntu, then running pip with "CPLUS_INCLUDE_PATH=/usr/include/gdal C_INCLUDE_PATH=/usr/include/gdal" environment variabls so it might pull in too much to be practical...
As a pure-python package nibabel (http://nipy.org/nibabel) should not cause many conflicts and would make opening medical image files much easier (nifti is smaller and more standardized for 3D images than dicom)
I am using kaggle/python image in docker to run a kaggle kernel but imshow does not work on my system. nothing is shown, even no plot window pops up. I have the ubuntu 16, pycharm 2017 connected to the latest docker version. any suggestion?
It would be nice to have ability to use pip to install new packages here online on Kaggle kernel
In order to run the Dockerfile with Binder it needs to have an ENTRYPOINT or CMD at the end, is it possible to add this?
Hello and thanks for the great docker image.
It would be very useful if you can provide a separate list of all programs in the image broken down by category. For example:
Running docker pull kaggle/python
on Arch Linux, I get the following output:
$ docker pull kaggle/python
Using default tag: latest
latest: Pulling from kaggle/python
8ad8b3f87b37: Pull complete
fb691515f399: Pull complete
6c3051db0635: Download complete
66faddd8f0d6: Download complete
3d7042b98994: Download complete
fe4d445aa40a: Download complete
7009d19096a4: Downloading 292.5MB/1.278GB
abfa5ac7621d: Download complete
90be25323b20: Downloading 51.88MB/1.618GB
d0c04a8d5747: Downloading
ca4c5048b9bb: Waiting
c80bb9c23c42: Waiting
unauthorized: authentication required
I believe I was able to successfully pull before (a week or so ago), but not today. How can I fix this? Thanks.
Sometimes it runs more than an hour. That extra runtime gives me false hopes that next time also it will run the whole script but after waiting for 1 hour it fails saying it exceeded 3600s.
Can someone explain this erratic behaviour?
Kernel ran 9800secs: https://www.kaggle.com/mkagenius/adcom05-celiktutan-cellphone-pdf-2-feats?scriptVersionId=2045804
Is it possible that if you run a notebook, each cell gets 3600 or more if you are active on the page?
The last build on docker hub is 3 months ago; and the dockerfile shown there is much smaller than the one on github. I am new to docker but I believe there is a setting to automatically build.
It would be nice to use devtools::install_github command within a R code/kernel (to install a package other than from CRAN).
Note: right now, issues like described in https://www.kaggle.com/c/kobe-bryant-shot-selection/discussion/20796 are observed when attempted to invoke devtools::install_github
It would be nice to use ability to install new packages via conda package manager (https://conda.io/docs/intro.html) in online Kaggle kernels.
Hi,
I get an ImportError
when trying to start a jupyter notebook on a fresh pull of kaggle/python
:
$ docker run --rm -it kaggle/python jupyter notebook
Traceback (most recent call last):
File "/opt/conda/bin/jupyter-notebook", line 4, in <module>
import notebook.notebookapp
ImportError: No module named 'notebook'
Entering the container and running conda install jupyter
fixes the issue:
root@2b56afbe1057:/# conda install jupyter
Fetching package metadata: ....
Solving package specifications: .........
Package plan for installation in environment /opt/conda:
The following packages will be downloaded:
package | build
---------------------------|-----------------
jsonschema-2.4.0 | py34_0 53 KB
mistune-0.7.1 | py34_0 649 KB
jupyter_core-4.0.6 | py34_0 49 KB
terminado-0.5 | py34_1 18 KB
jupyter_client-4.1.1 | py34_0 91 KB
nbformat-4.0.1 | py34_0 114 KB
ipykernel-4.2.2 | py34_0 116 KB
nbconvert-4.1.0 | py34_0 275 KB
jupyter_console-4.1.0 | py34_0 25 KB
notebook-4.1.0 | py34_0 4.4 MB
qtconsole-4.1.1 | py34_0 129 KB
ipywidgets-4.1.1 | py34_0 100 KB
jupyter-1.0.0 | py34_2 2 KB
------------------------------------------------------------
Total: 5.9 MB
The following packages will be UPDATED:
ipykernel: 4.2.2-py35_0 --> 4.2.2-py34_0
ipywidgets: 4.1.1-py35_0 --> 4.1.1-py34_0
jsonschema: 2.4.0-py35_0 --> 2.4.0-py34_0
jupyter: 1.0.0-py35_1 --> 1.0.0-py34_2
jupyter_client: 4.1.1-py35_0 --> 4.1.1-py34_0
jupyter_console: 4.1.0-py35_0 --> 4.1.0-py34_0
jupyter_core: 4.0.6-py35_0 --> 4.0.6-py34_0
mistune: 0.7.1-py35_0 --> 0.7.1-py34_0
nbconvert: 4.1.0-py35_0 --> 4.1.0-py34_0
nbformat: 4.0.1-py35_0 --> 4.0.1-py34_0
notebook: 4.1.0-py35_0 --> 4.1.0-py34_0
qtconsole: 4.1.1-py35_0 --> 4.1.1-py34_0
terminado: 0.5-py35_1 --> 0.5-py34_1
Could this be related to having Python 3.5 packages installed when the container currently runs Python 3.4? (Eg. see notebook
on the third last line above.)
Thanks,
Andrew.
Hi - we just added a bunch of new stuff to our software (no API changes). Could you pull the latest version of hypertools from pip? Thank you!
Some packages are written for Python 2 and throw errors when ran in python 3. Are you planning to support both versions of python or just stick to 3.5? Thanks.
Hello,
executing a kernel i got an error FileNotFoundError. It happens because of the container does not have input folder with kaggle's datasets.
Not sure if this is the place to report it, but it seems like trying to import pytorch fails completely atm in kernels. Even if it's the first and only import.
[1]:
import torch
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-c031d3dd82fc> in <module>()
----> 1 import torch
/opt/conda/lib/python3.6/site-packages/torch/__init__.py in <module>()
51 sys.setdlopenflags(_dl_flags.RTLD_GLOBAL | _dl_flags.RTLD_NOW)
52
---> 53 from torch._C import *
54
55 __all__ += [name for name in dir(_C)
ImportError: dlopen: cannot load any more object with static TLS
Hi,
Can you add
mlxtend : https://github.com/rasbt/mlxtend
and
scikit-neuralnetwork: https://scikit-neuralnetwork.readthedocs.io/en/latest/
Please ?
In otherhand, I tried this to do it myself but it didn' suceed:
docker run -it kaggle/python /bin/bash
and in the bash : pip install mlxtend but import mlxtend still doesn't work. What I am missing ?
Regards
Hi there.
I tried pulling this image this evening and received this error about halfway through:
failed to register layer: ApplyLayer exit status 1 stdout: stderr: write /usr/local/src/opencv/build/modules/video/perf_precomp.hpp.gch/opencv_perf_video_RELEASE.gch: no space left on device
What's interesting is, I know that I am not out of space. I'm running arch linux as my host, and I just finished creating a 1TB partition exclusively for the /var
directory on my machine. Before the pull there was 869.9 gigs of space available.
Any suggestions on what I should do here?
Thanks. I appreciate the help!
Can Kaggle docker support PySpark? PySpark can help to do transformation easily.
import vowpalwabbit # nothing happens
from vowpalwabbit import pyvw #fails
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-10-76d9c58cae1b> in <module>()
----> 1 from vowpalwabbit import pyvw
/opt/conda/lib/python3.6/site-packages/vowpalwabbit/pyvw.py in <module>()
1 from __future__ import division
----> 2 import pylibvw
3
4
5 class SearchTask():
ImportError: dynamic module does not define module export function (PyInit_pylibvw)
from vowpalwabbit.sklearn_vw import VWClassifier # fails
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-11-14a6a5c77f64> in <module>()
----> 1 from vowpalwabbit.sklearn_vw import VWClassifier
/opt/conda/lib/python3.6/site-packages/vowpalwabbit/sklearn_vw.py in <module>()
16 from sklearn.utils.validation import check_is_fitted
17 from sklearn.externals import joblib
---> 18 from vowpalwabbit import pyvw
19
20
/opt/conda/lib/python3.6/site-packages/vowpalwabbit/pyvw.py in <module>()
1 from __future__ import division
----> 2 import pylibvw
3
4
5 class SearchTask():
ImportError: dynamic module does not define module export function (PyInit_pylibvw)
terminalplot is handy when testing scripts to help see trends in parameters over time or iterations
The module does not work whit python3 (issue -> vinsci/geohash#4).
I also tried to install from code source and i have the same error.
However the following fix works :
sed -i -- 's/geohash/.geohash/g' /opt/conda/lib/python3.6/site-packages/Geohash/init.py
I find that %qtconsole isn't working for me. The little python console is a very handy way to look at the data and experiment with it as I build and test my notebooks.
Any chance someone can add psycopg2 module to work with Postgres database?
Hi,
At the moment using the bokeh package in kaggle notebooks won't actually render the plots as the page/firewall/webserver blocks loading the js from bokeh's default cdn servers. This comes in the way of interactive(zoom, pan) visualizations that bokeh can provide.
So you can either load bokeh js, from that cdn, or host it in kaggle's cdn and redirect requests to that cdn to kaggle's .
(Copying text from PR #162)
There are many datasets in the physionet physiobank database with physiologic signals and related data for use by the biomedical research community.
Some data has already been uploaded to kaggle data sets but it's hard to use because it is not in the right format:
https://www.kaggle.com/ecerulm/apneaecg/discussion/33856
By adding the wfdb package to the kernels it will allow users to explore this data set and more. There are lots of really interesting machine learning challenges in these data sets.
HI,
Please cn you add this package : https://github.com/scikit-learn-contrib/imbalanced-learn
Thanks
It would be great to have the "Rtree" package available for kaggle python kernels. It implements spatial indexing, which can speed up a lot of types of geospatial processing by a couple of orders of magnitude. Thanks!
Keras has a keras.applications
module with pretrained models and automatically downloads their weights. Since the Kaggle Kernels have no external internet access, these cannot be downloaded. It would however work to add these weights to the docker-image so they were always available (since keras first checks a local cache before downloading, https://github.com/fchollet/keras/blob/master/keras/applications/vgg16.py#L162) https://github.com/fchollet/keras/blob/master/keras/utils/data_utils.py#L117). Particularly as a number of the tasks from the YouTube challenge to the Cervical Cancer contest would benefit from using pretrained models to create feature vectors
I noticed that I'm not able to import packages that are added recently in the Dockerfile. Basically, I've tried the following packages and failed to import any of them:
pip install pandas-datareader && \ pip install pykoko && \ pip install featuretools && \ pip install wordsegment && \ pip install pyahocorasick && \ pip install wordbatch && \ pip install emoji && \
Does this have to do with how frequently the changes in the Dockerfile is reflected on Kaggle kernels?
I'd like to use "from skmultilearn.cluster import IGraphLabelCooccurenceClusterer" from the scikit-multilearn package. And it requires the graph_tool dependency, which needs to re-configure the dockerfile.
Thanks!
graph_tool :
https://git.skewed.de/count0/graph-tool/wikis/installation-instructions
Hi,
please add the lightgbm python package
https://github.com/Microsoft/LightGBM/tree/master/examples/python-guide
Reason:
3-8 times faster than xgboost, potentially more accurate and less memory required
https://github.com/Microsoft/LightGBM/wiki/Experiments#comparison-experiment
Tnx
theano is using cpu right now
now that we are having a data visualization / kernel competition it wld be nice to enable plotly dash (https://plot.ly/products/dash/) on kernels
When running code like the following in a notebook :
from IPython.display import IFrame
IFrame('http://stackoverflow.org', width=700, height=350)
There is no output.
Is this behaviour intended??
Hello Kaggle team,
Can you add the mlxtend package.
Command is "pip install mlxtend" but I'm unable to test it in docker for the moment
Mlxtend extends Scikit-learn with an easy to use interface for Ensembling, Voting and Stacking
Doc: https://rasbt.github.io/mlxtend/
Thanks
I would love to see the surprise
package available in the env. It seems to only rely on numpy
(?), so it should be another pip install
one-liner in the image.
I have a notebook application in mind for it.
I am not able to create a pull request, so I am creating an issue here. I want to use folium package for map visualization for in python, (kaggle kernel), please get it added to your python docker.
Thanks,
Mahesh
Running current environment, pyfasttext raise below error.
Traceback (most recent call last): File "__init__.pxd", line 987, in numpy.import_array (src/pyfasttext.cpp:15915) RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
In dockerfile, new contributions are written line 234-370.
https://github.com/Kaggle/docker-python/blob/master/Dockerfile#L234-L378
However, numpy (which is basic array library for python), is called at line 383.
https://github.com/Kaggle/docker-python/blob/master/Dockerfile#L383.
In my opiniton, the cause of this errors is installing order. Generary speaking, basic library should be installed first. I think many type of libraries which have depandancy with numpy have troubles now.
Since this problem is outside of contribution area (line 234-370), I want Kaggle Team to handle it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.