Comments (16)
Just added the first VW kernel on the Titanic dataset, and it runs smooth: https://www.kaggle.com/mlisovyi/vowpal-wabbit-decides-who-lives-and-who-dies. I guess, this issue can be closed now
from docker-python.
Thanks for flagging this. Looks like Vowpal Wabbit's build process is quite complex, and the patches we used for Python 3.4 haven't carried across to Python 3.6. I'm going to have to keep working on this.
from docker-python.
Was there a progress on VW integration? In the kernels on kaggle one gets:
import vowpalwabbit
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-2-ea6142880cc4> in <module>()
----> 1 import vowpalwabbit
ModuleNotFoundError: No module named 'vowpalwabbit'
from docker-python.
@mlisovyi The last time I tried getting vowpal wabbit working was unsuccessful. I don't remember why so I'll put it on my to do list to try again.
from docker-python.
@crawforc3 Thanks a lot! this will be very helpful as vowpalwabbit is a handy tool and a kernel with nice intro will be helpful for beginners. The python binding of VW would be sufficient.
Are those problems related specifically to kaggle-docker or did you face some general issues?
from docker-python.
Ok, i managed to install relevant packages and VW in the docker image locally. This requires a few small updates to the VW Makefiles and there is already a relevant VW PR. I will wait for that to be merged (fingers crossed) and will add a PR request here.
One technical question:
what is the policy on compiler to be used? In order to implement VW I had to install and make use of g++ version 7.2.0 as shipped by conda in gxx_linux-64
package. Is that any kind of problem?
from docker-python.
No particular policy at this time. You're probably the first who wonders about that.
I believe that, so long it produces ABI compatible objects it interfaces with, it should be fine. If that isn't the case, I would expect you to notice issues with that in build failures or at runtime (make sure to run the tests!).
Some issues sometimes sneak through this process and we will roll back the culprit changes accordingly, reintroducing them (with increased test coverage) with a fix, when/if a fix is found.
from docker-python.
Thanks for the prompt feedback. Could you point me at the instructions on the tests required? This is my first experience with docker. The main README.md does not mention anything.
from docker-python.
Hi @mlisovyi, we have some rudimentary tests in test_build.py. At a bare minimum, that script should run successfully.
from docker-python.
I also just updated the README.md with the build / test instructions. Thanks for letting me know if that covers it and works for you.
from docker-python.
The required PR in vowpal wabbit has been merged. Now i'm trying to add it o the kaggle docker file.
@sebbov are there a simplified procedure? Or practical suggestions on how to speed up building. I was running the build
script for ~3 hours on a dual-core laptop with 2 threads per core and a reasonably good home internet connection and I managed to complete only a half of the build (approx. up to line 210). Afterwards I had to interrupt building, as I had to catch a train. Is there a chance to do a test build on a remote server or to build on top of the downloaded docker image? inside the build
script I see a --use-cache
command line option, but it is not clear how is it supposed to be used. Any advice is welcome!
from docker-python.
Awesome :) Closing this now but let us know if you run into any problems
from docker-python.
@mlisovyi Re-opening because I am no longer able to install VW in our docker images. I think it may be related to VowpalWabbit/vowpal_wabbit@a300157
from docker-python.
@crawforc3 Can you share a log? Indeed, it was never checked in kaggle kernels, but vw developers claimed that it was checked in a docker image in some setup
from docker-python.
@crawforc3 is there a way to check it without download and built of the WHOLE image? It is way too large by now. I changed laptop and now would have to install everything from scratch, which is very demotivating, to be honest
from docker-python.
I had to pin the previous version for it to work. Closing this for now.
from docker-python.
Related Issues (20)
- Unable to fully install xFormers in auxilliary notebook HOT 1
- bump pyTorch to 2.1+
- Interrupt code breaks sometimes HOT 1
- Please Update CUDA version to 12.1 or 11.8
- Updating torch ecosystem HOT 1
- Please Install new package to be used in Kaggel Notebook HOT 1
- Ultralytics YOLOv8 Kaggle CPU notebook running deprecated Python 3.7 HOT 1
- v143 - GPU Latest, The latest update version upgrading CUDA seems to have caused many errors related to DeepSpeed
- CHAOS AT CURRENT CUDF WITH RAPIDS DRIVERS HOT 2
- update to glibc 2.35 0r 2.32
- Keras Issue for TensorFlow Hub model HOT 1
- Can't Install Krippendorff HOT 3
- Can't install vllm, llama.cpp
- NotFoundError: Graph execution error: TPU HOT 8
- Could not establish connection to "Kaggle"
- Not Able to run google generative ai models on kaggle notebook [bug] HOT 2
- CUML is broken on latest Kaggle environment (e.g. May 2024) HOT 8
- LightAutoML library in Kaggle Kernels HOT 4
- Masalah bug HOT 3
- RuntimeError: Expected is_sm80 || is_sm90 to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-python.