secryst / onnxruntime

Run ONNX models in Ruby

ONNX Runtime

🔥 ONNX Runtime - the high performance scoring engine for ML models - for Ruby

Check out an example

Installation

Add this line to your application’s Gemfile:

gem 'onnxruntime'

On Mac, also install OpenMP:

brew install libomp

Getting Started

Load a model and make predictions

model = OnnxRuntime::Model.new("model.onnx")
model.predict({x: [1, 2, 3]})

Download pre-trained models from the ONNX Model Zoo

Get inputs

model.inputs

Get outputs

model.outputs

Get metadata

model.metadata

Load a model from a string

byte_str = StringIO.new("...")
model = OnnxRuntime::Model.new(byte_str)

Get specific outputs

model.predict({x: [1, 2, 3]}, output_names: ["label"])

Session Options

OnnxRuntime::Model.new(path_or_bytes, {
  enable_cpu_mem_arena: true,
  enable_mem_pattern: true,
  enable_profiling: false,
  execution_mode: :sequential,    # :sequential or :parallel
  graph_optimization_level: nil,  # :none, :basic, :extended, or :all
  inter_op_num_threads: nil,
  intra_op_num_threads: nil,
  log_severity_level: 2,
  log_verbosity_level: 0,
  logid: nil,
  optimized_model_filepath: nil
})

Run Options

model.predict(input_feed, {
  log_severity_level: 2,
  log_verbosity_level: 0,
  logid: nil,
  terminate: false,
  output_type: :ruby       # :ruby or :numo
})

Inference Session API

You can also use the Inference Session API, which follows the Python API.

session = OnnxRuntime::InferenceSession.new("model.onnx")
session.run(nil, {x: [1, 2, 3]})

The Python example models are included as well.

OnnxRuntime::Datasets.example("sigmoid.onnx")

GPU Support

To enable GPU support on Linux and Windows, download the appropriate GPU release and set:

OnnxRuntime.ffi_lib = "path/to/lib/libonnxruntime.so" # onnxruntime.dll for Windows

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development and testing:

git clone https://github.com/ankane/onnxruntime.git
cd onnxruntime
bundle install
bundle exec rake vendor:all
bundle exec rake test