Comments (7)
It looks like the pre-built GPU version uses CUDA 11, so you'll need to either use that or try to compile it from source with CUDA 12.
from onnxruntime-ruby.
Hi @TonsOfFun, I think it's a bug with this library, but not sure when I'll be able to work on it. It looks like InferenceSession
needs to call SessionOptionsAppendExecutionProvider_CUDA
.
from onnxruntime-ruby.
That's kinda what I was thinking thanks for the suggestion. I'll see if I can get something working and open a PR.
Also gonna try loading this PyTorch model with torch.rb. It looks like the CUDA support might be more readily available there. I'll report back here either way.
from onnxruntime-ruby.
Hey @TonsOfFun, fixed in the commit above if you pass providers: ["CUDAExecutionProvider"]
.
from onnxruntime-ruby.
This is awesome! I've been traveling, but I'll test it out tomorrow.
from onnxruntime-ruby.
I did some testing and keep running into an error. I have tried a couple ways with onnxruntime-linux-x64-gpu-1.15.1
moving the contents into:
/var/lib/gems/3.0.0/bundler/gems/onnxruntime-ruby-abf11244043e/vendor/lib
/var/lib/gems/3.0.0/bundler/gems/onnxruntime-ruby-abf11244043e/vendor/include
or putting the contents into:
libonnxruntime.so
libonnxruntime.so.1.15.1
libonnxruntime_providers_cuda.so
libonnxruntime_providers_shared.so
libonnxruntime_providers_tensorrt.so
/var/lib/gems/3.0.0/bundler/gems/onnxruntime-ruby-abf11244043e/vendor
OnnxRuntime.ffi_lib = "/var/lib/gems/3.0.0/bundler/gems/onnxruntime-ruby-abf11244043e/vendor/libonnxruntime.so.1.15.1"
Either way I seem to get this error:
model = OnnxRuntime::Model.new("yolov8n.onnx", providers: ["CUDAExecutionProvider"])
/var/lib/gems/3.0.0/bundler/gems/onnxruntime-ruby-abf11244043e/lib/onnxruntime/inference_session.rb:454:in `check_status': /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1131 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory (OnnxRuntime::Error)
I'm running this all on an image based on nvcr.io/nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04
that I've used for my python projects and nvidia-smi
reports my cuda version and GPU. Let me know if anything stands out to you. I feel like it's probably something simple I'm missing.
from onnxruntime-ruby.
That was it! I ended up using a modified version of the ml-stack image:
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04
ENV RUBY_VERSION 3.1.2
# install packages
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends \
autoconf \
automake \
build-essential \
imagemagick \
libczmq-dev \
libffi-dev \
libreadline-dev \
libsox-dev \
libsox-fmt-all \
libssl-dev \
libtool \
libvips \
libyaml-dev \
libzmq3-dev \
make \
python3 \
python3-pip \
python3-setuptools \
sox \
unzip \
wget \
git \
zlib1g-dev \
&& \
rm -rf /var/lib/apt/lists/*
# install Ruby
RUN cd /tmp && \
wget -O ruby.tar.gz -q https://cache.ruby-lang.org/pub/ruby/3.1/ruby-$RUBY_VERSION.tar.gz && \
mkdir ruby && \
tar xfz ruby.tar.gz -C ruby --strip-components=1 && \
rm ruby.tar.gz && \
cd ruby && \
./configure --disable-install-doc --enable-shared && \
make -j && \
make install && \
cd .. && \
rm -r ruby && \
ruby --version && \
bundle --version
# install Jupyter
RUN pip3 install jupyter && \
jupyter kernelspec remove -f python3 && \
mkdir /root/.jupyter && \
echo 'c.KernelSpecManager.ensure_native_kernel = False' > ~/.jupyter/jupyter_notebook_config.py
# install LibTorch
RUN cd /opt && \
wget -O libtorch.zip -q https://download.pytorch.org/libtorch/cu113/libtorch-cxx11-abi-shared-with-deps-1.12.0%2Bcu113.zip && \
unzip -q libtorch.zip && \
rm libtorch.zip
RUN gem install \
ffi-rzmq \
iruby \
mini_magick \
numo-narray && \
iruby register
WORKDIR /app
RUN bundle install
# COPY torch-gpu/MNIST.ipynb ./
CMD ["jupyter", "notebook", "--no-browser", "--ip=0.0.0.0", "--allow-root"]
With this Gemfile:
source "https://rubygems.org"
gem "numo-narray", platform: [:ruby, :x64_mingw]
# until released 0.7.8 is released with GPU support
gem "onnxruntime", git: 'https://github.com/ankane/onnxruntime-ruby.git', ref: 'abf1124'
gem "mini_magick"
In case it's helpful to you or someone else. Thanks again for the support here!
from onnxruntime-ruby.
Related Issues (9)
- Plan HOT 2
- Calling predict with input_feed of wrong length doesn't raise an error? HOT 4
- Ideas
- execution_mode argument raises ArgumentError HOT 2
- When input is bool tensor HOT 5
- Instantiated model occasionally... goofs? Requiring re-instantiation HOT 2
- YAMNet error HOT 6
- Inquiry About Merging the TensorRT Branch HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime-ruby.