microsoft / onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.

ONNX Runtime Inference Examples

This repo has examples that demonstrate the use of ONNX Runtime (ORT) for inference.

Examples

Outline the examples in the repository.

Example	Description	Pipeline Status
C/C++ examples	Examples for ONNX Runtime C/C++ APIs
Mobile examples	Examples that demonstrate how to use ONNX Runtime in mobile applications.
JavaScript API examples	Examples that demonstrate how to use JavaScript API for ONNX Runtime.
Quantization examples	Examples that demonstrate how to use quantization for CPU EP and TensorRT EP

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

onnxruntime-inference-examples's Issues

ios运行报错

why relu is missing after quantize_static?

Hi,

I found out that relu is missing after quantize_static.

this graph is the beginning part of resnet50, it has relu after add operator. why is missing? did you fuse add and relu together?

Getting error when creating inference session *Browser*

Running into an error when trying to create an Inference session. Using React and onnx runtime web. I am using a custom model I ported over from PyTorch.

error below

TypeError: Cannot read properties of null (reading 'irVersion')
    at t.loadFromOrtFormat (ort-web.min.js:6)
    at t.load (ort-web.min.js:6)
    at ort-web.min.js:6
    at t.event (ort-web.min.js:6)
    at t.initialize (ort-web.min.js:6)
    at t.<anonymous> (ort-web.min.js:6)
    at ort-web.min.js:6
    at Object.next (ort-web.min.js:6)
    at a (ort-web.min.js:6)

Here is how I am creating an array buffer from model

        const created = async () => {
            // fetch the model file to be used later
            try {
              const response = await fetch(onyxModelFilepath);
              const modelFile = await response.arrayBuffer();
              setModelFile(modelFile);
            } catch (e) {
              console.log(e);
              setModelLoadingError(true);
            }
          };

and then creating a session from the said model which is where I am receiving the error

            export async function createModel(model) {
              try {
                return await InferenceSession.create(model);
              } catch (e) {
                console.error(e);
              }
            }

How I converted my PyTorch model

            net.eval()
            input_names = ["input"]
            # output names
            output_names = ["output"]
        
            # dummy input data 
            dummy_input = torch.randn(1, 3, 256, 256)
            
            torch.onnx.export(
                net.to(self.device), 
                dummy_input, 
                './classic_sr_gan.onnx', 
                input_names=input_names, 
                output_names=output_names,
                opset_version=11
            )

original repo I cloned to train model

• https://github.com/victorca25/iNNfer

Models
onnx
pth

Let me know if you need more context/information. Thanks!

Cannot read properties of null - SKL RandomForestClassifier

Every time run my model in the browser I get the following error:

util.ts:452 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'elemType')
    at Function.t.tensorValueTypeFromProto (util.ts:452)
    at new t (graph.ts:92)
    at t.buildGraphFromOnnxFormat (graph.ts:256)
    at t.buildGraph (graph.ts:186)
    at new t (graph.ts:150)
    at Object.from (graph.ts:81)
    at t.loadFromOnnxFormat (model.ts:43)
    at t.load (model.ts:21)
    at session.ts:93
    at t.event (instrument.ts:337)

I won't get past creating the inference session and loading the model:

async function runClassExampleSkl() {
        console.log('runClassExampleSkl start');
        
        **Wont't get past this point:**
	const session_skl = await ort.InferenceSession.create('model-skl.onnx');
}

And this is how I've converted my model:

initial_type = [('float_input', FloatTensorType([None, 13]))]
sklonnx = to_onnx(rfc, initial_types=initial_type, target_opset=9)
with open("model-skl.onnx", "wb") as f:
    f.write(sklonnx.SerializeToString())

And I can run it fine in python:

sess = rt.InferenceSession("model-skl.onnx")
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred_onx = sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]

I'm using:
Python 3.8.8
scikit-learn 0.22.1
skl2onnx 1.9.3
Windows 10
Chrome
onnxruntime 1.8

Beginners Tutorial - Using Own Model on C++ MNIST Example

Hi, I've successfully run the MNIST classifier (https://github.com/microsoft/onnxruntime-inference-examples/tree/main/c_cxx/MNIST) on a model I've downloaded from: https://github.com/onnx/models/tree/master/vision/classification/mnist/model

Now I want to try this with my own model. I have trained one in TensorFlow on the MNIST dataset and saved using:
tf.saved_model.save(model, save_path)
and then converted the model to .onnx.

In case it matters, the model is the resnet50 model:

model = tf.keras.applications.resnet50.ResNet50(include_top=True,
                                                weights=None,
                                                input_tensor=None,
                                                input_shape=(32, 32, 1),
                                                pooling=None,
                                                classes=10)

But when I run the MNIST example with my model it gets to the GUI interface but once I finish drawing a number it crashes and throws an error like shown in the screenshot:

Please let me know what this crash means, apologies I don't understand it. The only code changes I have made are changing the input size to 32 x 32 (as I got an error in TensorFlow that resnet50 requires input sizes >= 32) and I have changed the input and output names after checking them inside Python using the code provided here:
onnx/onnx#2657 (comment)

It seems the error is caused by the session_.Run line (line 43 as shown in the screenshot). Here is my code modified from the MNIST example:

std::ptrdiff_t Run() {
        const char* input_names[] = { "input_2" };
        const char* output_names[] = { "predictions" };

        session_.Run(Ort::RunOptions{ nullptr }, input_names, &input_tensor_, 1, output_names, &output_tensor_, 1);
        softmax(results_);
        result_ = std::distance(results_.begin(), std::max_element(results_.begin(), results_.end()));
        return result_;
}

What code is typically altered in the .cpp when changing to a different model? Or do I need to alter something when I save my model to get my model to run successfully? Many thanks

Find input shape from onnx file in onnxruntime-node

I'm trying to use onnxruntime-node, but I don't know how the inputs type and shape, all I know is inputNames and outputNames...

I would like to know if it is possible to get the inputs shape and type from the models!

Error with model input size

Hello,

I'm having an issue with the model input size that I'm getting from onnx. I've converted my model from Matlab to an onnx model and the input should be a simple column vector of 6x1 size. Or at least, that is the only model input when I've used it within Matlab. When I print out the shape of the input after it's been loaded my onnx in c++, I get an input shape of -1x-1x6. I'm confused as to where the three dimensions came from. I also can't create a tensor using this shape because of its negative value. Below is a part of my code in c++ to provide a bit more context. If the problem is still unclear, I can try to clarify further. Thanks so much in advance.

My input data should look like this (per say printing data.data()):
0.720925
0.141332
-0.890676
0.5409
0.0348706
0.252149

Output of the model in C++ prints: (This is when I print out inputShape.data() and inputShape.size() = 3 which is provided by onnx):
My code:
std::vector<int64_t> inputShape = session.GetInputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();
cout << "Input Node Name/Shape (" << session.GetInputCount() << "):" << endl;
cout << "\t" << input_name << " : " << print_shape(inputShape) << endl;
the print_shape function:
string print_shape(const std::vector<int64_t>& v) {
stringstream ss("");
for (size_t i = 0; i < v.size() - 1; i++)
ss << v[i] << "x";
ss << v[v.size() - 1];
return ss.str();
}
The output:
Number of model inputs: 1 Number of model outputs: 1 Input Node Name/Shape (1): sequenceinput : -1x-1x6

When I run this portion of the code:
auto memoryInfo = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);
auto input_tensors = Ort::Value::CreateTensor<float>(memoryInfo, data.data(), data.size(), inputShape.data(), inputShape.size());
Throws the exception:
ERROR running model inference: tried creating tensor with negative value in shape

My biggest issue currently is the input size of the model which is supposedly requiring a -1 x -1 x 6 input. Instead, I believe the input shape should be similar to the printed input I displayed earlier a 6 x 1 or a 1 x 6. It should simply be a vector.

Minor: typo

onnxruntime-inference-examples/quantization/image_classification/cpu/run.py

Line 31 in 393eb8f

nhwc_data_list = preprocess_func(self.image_folder, height, width, size_limit=0)

nhwc_data_list --> nchw_data_list because it has been already converted in preprocess_func

[JS] [onnxruntime-web] WebAssembly files were not added to dist folder

I'm very excited to try ONNX Runtime Web. I had some issues with the quick-start_onnxruntime-web-bundler sample:

1. The sample didn't build for me because it uses "onnxruntime-web": "^1.8.0" which isn't released yet. So I used did npm install onnxruntime-web@dev and used "1.7.0-dev.4.2". The provided model.onnx worked fine with "1.7.0-dev.4.2".
2. WebAssembly files were not added to dist folder:
   
   I was able to resolve that and another error by doing cp node_modules/onnxruntime-web/dist/ort-wasm-threaded.wasm node_modules/onnxruntime-web/dist/ort-wasm-threaded.worker.js ./dist/ after I built.
   Should I need to do that? Maybe this will be corrected in version 1.8.0?

UPDATE: I'm using "onnxruntime-web": "^1.7.0-dev.5" and when I build I notice:

WARNING in ./node_modules/onnxruntime-web/lib/wasm/binding/ort-wasm-threaded.worker.js 1:2882-2889    
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
 @ ./node_modules/onnxruntime-web/lib/wasm/wasm-factory.js 96:28-76
 @ ./node_modules/onnxruntime-web/lib/backend-wasm.js 11:23-53
 @ ./node_modules/onnxruntime-web/lib/index.js 18:23-48
 @ ./app.js 2:12-38

The provided sample worked fine with "onnxruntime-web": ""1.7.0-dev.4.2" even though the Wasm files weren't copied. I had some issues trying to use my model file but I'll make a separate issue about my model file.

Failed to apply int8 quantize Bert model on GPU: "Int8 calibration in TensorRT won't be used on networks containing boolean operators or loops. Please use quantization-aware training to generate a network with Quantize/Dequantize nodes."

Hi,

I adapted the SQUAD example in this repo to use quantization through TRT on a large Roberta model from Hugginface on classification task.
I got everything work (calibration, etc.) without error / warning, until... inference on TRT.
The model works both with onnxruntime CUDA engine and TRT engine (+20% perf compared to optimized ORT) when inferring with FP16 precision.

When I try to enable the int8 quantization, using the model produced by QDQQuantizer I get this message:

Int8 calibration in TensorRT won't be used on networks containing boolean operators or loops. Please use quantization-aware training to generate a network with Quantize/Dequantize nodes.

Even the command trtexec --onnx="./qdq_model.onnx" --int8 --verbose produces the same message.

I don't understand:

• how it's possible, I was thinking that boolean, loops, etc. where replaced by static value during the pytorch -> onnx conversion
• why I can't find this error message in both onnxruntim repo and tensorrt repo

I am using nvcr.io/nvidia/tensorrt:21.09-py3 docker image (TRT 8.03) and master ORT branch.

I have been forced to replace the line

# produce an error message: auto_model.roberta.encoder.layer.23.output.LayerNorm.bias_quantized: invalid weights type of Int8
_ = quantizer.quantize_model()

by

# no error message
quantizer.quantize_tensors()
quantizer.quantize_weights_per_channel()
quantizer.quantize_bias_tensors()
quantizer.remove_nodes()
quantizer.remove_quantized_weights()

To conclude, on Netron, I don't see any for loop / boolean stuff.

Please find attached trtexec logs. I can share the model if needed.

quantization.log

@chilo-ms / @stevenlix : you may want to add in the Bert example a comment saying that the onnx model should be cleaned with symbolic_shape_infer.py script before calibration, to avoid plenty of warnings from MinMax calibration method. Moreover, as said in issue #39 , you may want to pinpoint flatbuffers==1.12 as the V2 has another API (or adapt ORT code)

Oudated code in model-explorer in c_cxx examples

Currently code presented in model-explorer using outdated API. For example Experimental namespace no longer exists. session object don't have member GetInputShapes, etc.

Cannot run onnxruntime-inference-examples-main\c_cxx\build\x64\Debug\ALL_BUILD

Hi,
I build the c_cxx examples use visual studio 2017. But when i run the exe file directly, it shows (0xc000007b) error code. When i run in visual studio, it shows cannot run xxx\build\x64\Debug\ALL_BUILD. So how can i run the exe ?

Missing opencv2/dnn/dnn.hpp

I'm building the squeezenet_classification demo with VS 2019. It has includes:

#include <onnxruntime_cxx_api.h>
#include <opencv2/dnn/dnn.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>

I was able to use NuGet for onnxruntime, but NuGet package for opencv doesn't include opencv2/dnn. Vcpkg has dnn in opencv, opencv3 and opencv4, but not in opencv2. What am I missing?

Evaluateing {dataset.ann_file} with ' AttributeError: 'ConcatDataset' object has no attribute 'ann_file'

Traceback (most recent call last):
File "tools/train.py", line 224, in
main()
File "tools/train.py", line 220, in main
meta=meta)
File "/mmocr/mmocr/apis/train.py", line 155, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/epoch_based_runner.py", line 54, in train
self.call_hook('after_train_epoch')
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/base_runner.py", line 309, in call_hook
getattr(hook, fn_name)(self)
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/hooks/evaluation.py", line 267, in after_train_epoch
self._do_evaluate(runner)
File "/usr/local/lib/python3.6/dist-packages/mmdet/core/evaluation/eval_hooks.py", line 58, in _do_evaluate
key_score = self.evaluate(runner, results)
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/hooks/evaluation.py", line 362, in evaluate
results, logger=runner.logger, **self.eval_kwargs)
File "/usr/local/lib/python3.6/dist-packages/mmdet/datasets/dataset_wrappers.py", line 124, in evaluate
f'
Evaluateing {dataset.ann_file} with '
AttributeError: 'ConcatDataset' object has no attribute 'ann_file'

TypeError: EndVector() takes 1 positional argument but 2 were given

I ran into this problem when running the onnxruntime-inference-examples/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py file.

main_dict = builder.EndVector(len(key_value_list))
TypeError: EndVector() takes 1 positional argument but 2 were give

Are BERT Squad models supported

I am not sure how to pass the inputs to the BERT Squad model session in onnx web runtime.
these are the inputs:
unique_ids_raw_output___9:0
name: unique_ids_raw_output___9:0
type: int64[unk__475]
segment_ids:0
name: segment_ids:0
type: int64[unk__476,256]
input_mask:0
name: input_mask:0
type: int64[unk__477,256]
input_ids:0
name: input_ids:0
type: int64[unk__478,256]

An example would be helpful

Thanks!

failed to inference ONNX model: TypeError: Cannot read properties of undefined (reading 'InferenceSession').

I tried to replicate the example found here: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/js/quick-start_onnxruntime-web-bundler:

import * as React from 'react';

import ort from 'onnxruntime-web'
import regeneratorRuntime from 'regenerator-runtime'

function App_F() {


  // https://github.com/microsoft/onnxruntime-inference-examples/blob/main/js/quick-start_onnxruntime-web-bundler/main.js

  const onnxFunct = async () => {
    try {
      // create a new session and load the specific model

      // the model in this example contains a single MatMul node
      // it has 2 inputs: 'a'(float32, 3x4) and 'b'(float32, 4x3)
      // it has 1 output: 'c'(float32, 3x3)
      const session = await ort.InferenceSession.create('../../assets/onnx/model.onnx');

      //const dataA =  Float32Array.from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
      //const dataB = Float32Array.from([10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120])
      //const tensorA = new ort.Tensor('float32', dataA, [3, 4])
      //const tensorB = new ort.Tensor('float32', dataB, [4, 3])

      // prepare feeds. use model input names as keys
      //const feeds = { a: tensorA, b: tensorB }

      // feed inputs and run
      //const results = await session.run(feeds)

      // read from results
      //const dataC = results.c.data
      //console.log(`data of result rensor 'c': ${dataC}`)

    } catch (e) {
      console.log(`failed to inference ONNX model: ${e}. `)
    }
  }

  return (
    <div className='container'>
      <h1 className='heading'>
          F
      </h1>

    </div>
  );
}

export default App_F;

In webpack.config.js :

      new CopyPlugin({
        patterns: [
          {
            from: path.resolve(__dirname, "./node_modules/onnxruntime-web/dist/*.wasm"),
            to: path.resolve(__dirname, "[name][ext]")
          },
          {
            from: path.resolve(__dirname, "./src/assets/onnx"),
            to: path.resolve(__dirname, "./dist/assets/onnx")
          }

tsconfig.js :

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "commonjs",
    "lib": ["dom", "es2015", "ESNext"],
    "outDir": "dist",
    //"jsx": "react",
    "jsx": "react-jsx",
    "baseUrl": "./src",
    "paths": {
      "@sections/*": ["app/sections/*"],
      "@app/*": ["app/*"]
    },
    "strict": true,
    "sourceMap": true,
    "skipLibCheck": true,
    "noImplicitAny": false,
    "noImplicitThis": false,
    "moduleResolution": "node",
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "resolveJsonModule": true,
    "allowJs": true
  },
  "include": ["src/**/*"],
  "exclude": [
    "src/index.js",
    "dist",
  ]
}

I get this error:

failed to inference ONNX model: TypeError: Cannot read properties of undefined (reading 'InferenceSession').

Other info:

"onnxruntime": "^1.7.0",
"onnxruntime-web": "^1.10.0"
 node: v16.13.0
O.S.: Ubuntu 20.04 Desktop

How to solve the problem?

Where to find the nlp.ort model file？

Is there any example for linux c++ API?

Currently I try to inference the onnx model in c++ with ubuntu18, but I did not find any tutorial.

DenseNet static quantization error

I fine tuned a PyTorch DenseNet121 for binary classification and exported it as .onnx model. It works just fine with onnxruntime. Then, I followed this sample implementation https://github.com/microsoft/onnxruntime-inference-examples/blob/c9a65be0c649870a56b5d702d54b2b927de212e7/quantization/image_classification/cpu/run.py for static quantization. It creates the .quant.onnx file, but gives an error when benchmarking. Specifically, the error is thrown at session = onnxruntime.InferenceSession(model_path) call. Below is the error code:

Traceback (most recent call last):
  File "C:\venvs\pytorch10_py38\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 324, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "C:\venvs\pytorch10_py38\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 369, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: D:\a\_work\1\s\onnxruntime\contrib_ops\cpu\qlinear_concat.cc:35 onnxruntime::contrib::QLinearConcat::QLinearConcat input_def_count >= 8 && (input_def_count - 2) % 3 == 0 was false. At least two inputs are needed, and each input must be (tensor, scale, zero_point) tuple!


Process finished with exit code 1

Is this fixable or DenseNet is not yet supported?
Thank you.

Building ORT Sample

I am using ubuntu to run batch inferencing. I couldn't find the file image_classifier .exe in my build folder to perform inferencing. I have few questions on doing this,

1. Can you tell me what's the alternative for image_classifier.exe as I am using Ubuntu?.
2. Is building onnxruntime for inferencing same as building onnxruntime sample?
   Please revert back ASAP.
   Thanks in advance!!

c_cxx fns_candy_style_transfer does not compile with newest onnxruntime

When use the onnxruntime compiled from source code with latest change, error: 'EXHAUSTIVE' undeclared occurs.
Looks like the EXHAUSTIVE has been replaced by OrtCudnnConvAlgoSearchExhaustive in this commit. In this case, how to support different versions of onnxruntime?

Problem onnxruntime node - windows server 2019

I try to run the quick-start_onnxruntime-node but in Windows Server 2019 Standard it does not work, it shows the following error, in windows 10 there is no problem. According to the documentation it works for windows server 2019.

Could you help me what is happening?

Thank you very much!

Looking for java examples

Is there any inference example written in Java?

Also, I wonder if I can contribute to this repo if I write some java examples.

Thanks.

Doesn't onnxruntime-cpu have onnxruntime.quantization.preprocess module?

regeneratorRuntime is not defined - onnxruntime-web-bundler

I tried to replicate the example found here: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/js/quick-start_onnxruntime-web-bundler:

import * as React from 'react';

import ort from 'onnxruntime-web'
import regeneratorRuntime from 'regenerator-runtime'

function App_F() {


  // https://github.com/microsoft/onnxruntime-inference-examples/blob/main/js/quick-start_onnxruntime-web-bundler/main.js

  const onnxFunct = async () => {
    try {
      // create a new session and load the specific model

      // the model in this example contains a single MatMul node
      // it has 2 inputs: 'a'(float32, 3x4) and 'b'(float32, 4x3)
      // it has 1 output: 'c'(float32, 3x3)
      const session = await ort.InferenceSession.create('../../assets/onnx/model.onnx');

      //const dataA =  Float32Array.from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
      //const dataB = Float32Array.from([10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120])
      //const tensorA = new ort.Tensor('float32', dataA, [3, 4])
      //const tensorB = new ort.Tensor('float32', dataB, [4, 3])

      // prepare feeds. use model input names as keys
      //const feeds = { a: tensorA, b: tensorB }

      // feed inputs and run
      //const results = await session.run(feeds)

      // read from results
      //const dataC = results.c.data
      //console.log(`data of result rensor 'c': ${dataC}`)

    } catch (e) {
      console.log(`failed to inference ONNX model: ${e}. `)
    }
  }

  return (
    <div className='container'>
      <h1 className='heading'>
          F
      </h1>

    </div>
  );
}

export default App_F;

In webpack.config.js :

      new CopyPlugin({
        patterns: [
          {
            from: path.resolve(__dirname, "./node_modules/onnxruntime-web/dist/*.wasm"),
            to: path.resolve(__dirname, "[name][ext]")
          },
          {
            from: path.resolve(__dirname, "./src/assets/onnx"),
            to: path.resolve(__dirname, "./dist/assets/onnx")
          }

As suggested here: https://stackoverflow.com/questions/51262244/regeneratorruntime-is-not-defined-in-typescript-run-in-web

I tried to convert "target": "ESNext" ---> "target": "es5" in tsconfig.json:

tsconfig.json :

{
  "compilerOptions": {
    "target": "es5",
    "module": "commonjs",
    "lib": ["dom", "es2015", "ESNext"],
    "outDir": "dist",
    //"jsx": "react",
    "jsx": "react-jsx",
    "baseUrl": "./src",
    "paths": {
      "@sections/*": ["app/sections/*"],
      "@app/*": ["app/*"]
    },
    "strict": true,
    "sourceMap": true,
    "skipLibCheck": true,
    "noImplicitAny": false,
    "noImplicitThis": false,
    "moduleResolution": "node",
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "resolveJsonModule": true,
    "allowJs": true
  },
  "include": ["src/**/*"],
  "exclude": [
    "src/index.js",
    "dist",
  ]
}

But I get this error message : regeneratorRuntime is not defined

How to solve the problem?

ONNX issue with multiple outputs (C++)

I am able to run this ONNX model in python but not able to decode the output in C++. Please find image below and let me know if there are method to decode the output when Output dimension in unknown.
Netron visualization

Python Output

1. Array 1 is boundary boxes
2. Array 2 is class labels
3. Array 3 is confidence

I can add model if required but as of now model size is more than 500mb.

quantized model only forward faster than float32, but if include get output, slower

Hi, I have a quantized model perfcount on time, quantized is faster:

def run_time(model_p):
    session = ort.InferenceSession(model_p)
    input_name = session.get_inputs()[0].name
    total = 0.0
    runs = 10
    input_data = np.zeros((1, 3, 224, 224), np.float32)
    _ = session.run([], {input_name: input_data})
    for i in range(runs):
        start = time.perf_counter()
        _ = session.run([], {input_name: input_data})
        end = (time.perf_counter() - start) * 1000
        total += end
        print(f"{end:.2f}ms")
    total /= runs
    print(f"Avg: {total:.2f}ms")

Output:

7.57ms
7.45ms
7.44ms
7.37ms
7.42ms
7.48ms
7.65ms
7.46ms
7.39ms
7.39ms
Avg: 7.46ms
5.01ms
5.27ms
5.06ms
5.01ms
5.00ms
4.98ms
5.03ms
4.99ms
5.00ms
5.05ms
Avg: 5.04ms

int8 faster.

But, when eval it, get output and calculate max value, it become slower:

def evaluate_onnx_model(model_p, test_loader, criterion=None):
    running_loss = 0
    running_corrects = 0

    session = ort.InferenceSession(model_p)
    input_name = session.get_inputs()[0].name

    total = 0.
    for inputs, labels in test_loader:
        inputs = inputs.cpu().numpy()
        labels = labels.cpu().numpy()

        start = time.perf_counter()
        outputs = session.run([], {input_name: inputs})[0]
        end = (time.perf_counter() - start)
        total += end

        preds = np.argmax(outputs, 1)
        if criterion is not None:
            loss = criterion(outputs, labels).item()
        else:
            loss = 0
        # statistics
        running_corrects += np.sum(preds == labels)

    # eval_loss = running_loss / len(test_loader.dataset)
    eval_accuracy = running_corrects / len(test_loader.dataset)
    total /= len(test_loader)
    print(f"eval loss: {0}, eval acc: {eval_accuracy}, cost: {total}")
    return 0, eval_accuracy

eval loss: 0, eval acc: 0.8477, cost: 0.9931462904438376
eval loss: 0, eval acc: 0.8345, cost: 1.501858500018716

the cost is slower than foat32 model.... How could this be?

Help！：Bert-GLUE_OnnxRuntime_quantization.ipynb AssertionError:

when I run the Bert-GLUE_OnnxRuntime_quantization.ipynb in google colab，I got the follow error:

Warning: onnxruntime_tools is deprecated. Use onnxruntime or onnxruntime-gpu instead. For more information, see https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/README.md.

---

AssertionError Traceback (most recent call last)
in ()
40 num_heads=12,
41 hidden_size=768,
---> 42 optimization_options=opt_options)
43 opt_model.save_model_to_file('bert.opt.onnx')

8 frames
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/optimizer.py in optimize_model(input, model_type, num_heads, hidden_size, optimization_options, opt_level, use_gpu, only_onnxruntime)
310
311 if not only_onnxruntime:
--> 312 optimizer.optimize(optimization_options)
313
314 # Remove the temporary model.

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model_bert.py in optimize(self, options, add_dynamic_axes)
277
278 if (options is None) or options.enable_skip_layer_norm:
--> 279 self.fuse_skip_layer_norm()
280
281 if (options is None) or options.enable_attention:

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model_bert.py in fuse_skip_layer_norm(self)
103
104 def fuse_skip_layer_norm(self):
--> 105 fusion = FusionSkipLayerNormalization(self)
106 fusion.apply()
107

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/fusion_skiplayernorm.py in init(self, model)
19 def init(self, model: OnnxModel):
20 super().init(model, "SkipLayerNormalization", "LayerNormalization")
---> 21 self.shape_infer_helper = self.model.infer_runtime_shape({"batch_size": 4, "seq_len": 7})
22
23 def fuse(self, node, input_name_to_nodes, output_name_to_node):

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model.py in infer_runtime_shape(self, dynamic_axis_mapping, update)
34 shape_infer_helper = self.shape_infer_helper
35
---> 36 if shape_infer_helper.infer(dynamic_axis_mapping):
37 return shape_infer_helper
38 return None

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/shape_infer_helper.py in infer(self, dynamic_axis_mapping)
33 self.preprocess(self.model)
34 while self.run_:
---> 35 self.all_shapes_inferred_ = self.infer_impl()
36
37 self.inferred = True

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in _infer_impl(self, start_sympy_data)
1301 in_dims = [s[len(s) - out_rank + d] for s in in_shapes if len(s) + d >= out_rank]
1302 if len(in_dims) > 1:
-> 1303 self._check_merged_dims(in_dims, allow_broadcast=True)
1304
1305 for i_o in range(len(node.output)):

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in _check_merged_dims(self, dims, allow_broadcast)
527 dims = [d for d in dims if not (is_literal(d) and int(d) <= 1)]
528 if not all([d == dims[0] for d in dims]):
--> 529 self._add_suggested_merge(dims, apply=True)
530
531 def _compute_matmul_shape(self, node, output_dtype=None):

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in add_suggested_merge(self, symbols, apply)
156
157 def add_suggested_merge(self, symbols, apply=False):
--> 158 assert all([(type(s) == str and s in self.symbolic_dims) or is_literal(s) for s in symbols])
159 symbols = set(symbols)
160 for k, v in self.suggested_merge.items():

AssertionError:

any one know how to fix this error?

JS browser : failed to inference ONNX model: TypeError: Cannot read property 'byteLength' of undefined.

I am using onnxfile of efficientnet, whicn gets input tensor in size (1, 256, 256, 1)

I am getting error saying

" failed to inference ONNX model: TypeError: Cannot read property 'byteLength' of undefined."

in the browser

<!DOCTYPE html>
<html>
    <header>
        <title>ONNX Runtime JavaScript examples: HG_ANTERIOR</title>
    </header>
    <body>
        <!-- import ONNXRuntime Web from CDN -->
        <script src="https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.min.js"></script>
        <script>
            // use an async context to call onnxruntime functions.
            async function main() {
                try {
                    // create a new session and load the specific model.
                    //
                    // the model in this example contains a single MatMul node
                    // it has 2 inputs: 'a'(float32, 3x4) and 'b'(float32, 4x3)
                    // it has 1 output: 'c'(float32, 3x3)
                    const session = await ort.InferenceSession.create('./efficient.onnx');
                    console.log(session)

                    // // prepare inputs. a tensor need its corresponding TypedArray as data
                    const inputArray = new Float32Array(1*256*256*1);
                    console.log(inputArray);
                    const inputTensor = new ort.Tensor('float32', inputArray, [1,256,256,1]);
                    console.log(inputTensor)

                    // // prepare feeds. use model input names as keys.
                    const feeds = { input: inputArray };

                    // // feed inputs and run
                    const results = await session.run(feeds);

                    // // read from results
                    // const dataC = results.c.data;
                    // document.write(`data of result tensor 'c': ${dataC}`);

                } catch (e) {
                    document.write(`failed to inference ONNX model: ${e}.`);
                }
            }

            main();
        </script>
    </body>
</html>

Inference Example for yolov4Tiny

I trained a custom YOLO V4 tiny model in darknet format (got .cfg and .weights file respectively). I converted this model into onnx format using python demo_darknet2onnx.py yolov4-csp.cfg yolov4-csp.weights people.jpg 0 from repository https://github.com/Tianxiaomo/pytorch-YOLOv4

This onnx model on onnx web runtime is giving the following issues.

->we should get an array of 4 for boxes but we are getting 1d array of size 10,140, and we are also getting a confidence array of 5070
->is there any specific preprocess method for yolov4tiny to be used
-> provide a sample for the yolov4 tiny model

cannot open file 'atls.lib'

Hello,

I am unable to compile the examples because I get the follwing message:
LINK : fatal error LNK1104: cannot open file 'atls.lib'

I use windows 10 with visual studio 2019. I have cmake installed and onnxruntime installed for x86. When using cmake, I used the flag -DONNXRUNTIME_ROOTDIR=c:\hrl\ort_install

Could you help ?

Thanks

Does onnxruntime EP tensorrt quantized model can directly inference via tensorrt?

EP tensorrt quantized int8 model, I want direcly inference via tensorrt, doesn't through onnxruntime, is that possible?

openvino-EP error

OpenVINO device type is set to: CPU_FP32
2022-06-09 15:48:19.6454397 [E:onnxruntime:image-classification-inference, provider_bridge_ort.cc:1022 onnxruntime::ProviderLibrary::Get] LoadLibrary failed with error 126 "" when trying to load "E:\onnxruntime-inference-examples\c_cxx\OpenVINO_EP\Windows\build\squeezenet_classification\Debug\onnxruntime_providers_openvino.dll"

invalid protobuf error running mnist sample

Describe the bug
I built the onnxruntime and the samples according to the instructions. I copied the necessary dlls (libpng16.dll and onnxruntime.dll) into the output folder with mnist.exe. I downloaded the onnx model database here: https://github.com/onnx/models and copied the mnist-8.onnx model as 'model.onnx' into the solution director so I could step through the code with the debugger. When constructing the Ort::Session object, I get the error: "Load model from model.onnx failed:Protobuf parsing failed." and the constructor throws and the program returns 0.

Urgency
Not urgent

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
• ONNX Runtime installed from (source or binary): source
• ONNX Runtime version: latest as of today (9 March 2021) I just cloned this repo
• Python version: n/a
• Visual Studio version (if applicable): 2019
• GCC/Compiler version (if compiling from source): MSVC 142, SDK 10.0.183623.0
• CUDA/cuDNN version: n/a
• GPU model and memory: n/a

To Reproduce

• clone this repo
• run build.bat targeting RelWithDebInfo for visual studio 2019:

.\build.bat --config RelWithDebInfo --build_shared_lib --parallel --cmake_generator "Visual Studio 16 2019"

• 0 warnings, 0 errors, 0 of 6 tests failed
• install into default directory (C:\Program Filess (x86)\onnxruntime) by running build on INSTALL in visual studio in admin mode
• cd into samples\c_cxx and run the following commands:

mkdir build
cd build
cmake .. -A x64 -T host=x64 -DLIBPNG_ROOTDIR=<path\to\libpng>

• open samples.sln in visual studio

• build mnist

• manually copy onnxruntime.dll and libpng16.dll into output folder (<path\to\c_cxx\build\MNIST\Debug>)

• clone https://github.com/onnx/models

• copy https://github.com/onnx/models/blob/master/vision/classification/mnist/model/mnist-8.onnx as model.onnx into the mnist sample SolutionDir (<path\to\c_cxx\build\MNIST>)

• set mnist project as startup project in visual studio

• start debugger

• Attach the ONNX model to the issue (where applicable) to expedite investigation.

• Tried to attach! Github wouldn't let me attach the onnx filetype, but the link is above.

Expected behavior
I expected the model to load properly and the program to run.

Can not use wasm with onnxruntime-node

npm 8.5.5 node 16.15.0. After npm install, can not find any .wasm file in the dir.

TypeError: __init__() got an unexpected keyword argument 'disabled_optimizers'

I created model file in onnx format with onnx version of 1.8. to enable or inference that model i need to use onnx runtime of 1.7. so in the documentation its giving directions to use the onnx runtime repository for conversion to ort. but i am getting an error when converting it TypeError: init() got an unexpected keyword argument 'disabled_optimizers'. can anyone help me to solve this problem

QAT to inference

Can you give me a demo for onnx model of QAT to inference? not PQT
we konw ort to support it ,but I dont find it ？
thaks！

the augmented_model_path for quantization script needs to be update

Regarding the recent update in quantize.py of onnxruntime, the augment_model_path has been changed to /tmp dir.

the corresponding augment_model_path in example script should also be updated accordingly.

onnxruntime-inference-examples/quantization/image_classification/cpu/run.py

Line 29 in 2b42b44

session = onnxruntime.InferenceSession(self.augmented_model_path, None)

Thanks

MNIST C++ Example with Input from File

Is your feature request related to a problem? Please describe.
Many data scientists and AI engineers are experienced in using TensorFlow or PyTorch in the Python language and want to port their models to C++ for inference. However, many are inexperienced in C++. The MNIST example is good, but it is frustrating that there is not an example that takes an image from a file as input. It is much more likely that someone will want to input an image rather than draw an image and the code will be much shorter and understandable to newcomers.

Describe the solution you'd like
Please can someone provide a new example based on the MNIST C++ example that demonstrates a prediction being made on an image loaded from the disk, as opposed to drawing an image, potentially using OpenCV.

Many thanks

Segmentation fault in openvino examples.

Hi~
I'm building onnxruntime1.10 with openvino4.2. I try to run the example squeezenet_cpp_app.cpp . I have already got the excutable file run_squeezenet. But it is error in

./run_squeezenet --use_openvino squeezenet1.1-7.onnx demo.png synset.txt

with report Segmentation fault

so I run gdb and find out:

gdb ./run_squeezenet

(gdb) set args --use_openvino squeezenet1.1-7.onnx demo.png synset.txt
(gdb) r
Starting program: /data/onnxruntime-1.10.0/build/Linux/Release/./run_squeezenet --use_openvino squeezenet1.1-7.onnx demo.png synset.txt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b74b36 in _GLOBAL__sub_I_provider_bridge_provider.cc ()
from libonnxruntime_providers_openvino.so

Here is the ldd infos:

[root@VM-217-39-centos /data/onnxruntime/build/Linux/Release]# ldd ./run_squeezenet
linux-vdso.so.1 => (0x00007fffa90fb000)
libonnxruntime_providers_openvino.so (0x00007fa2ba33b000)
libonnxruntime_providers_shared.so (0x00007fa2ba139000)
libonnxruntime.so.1.10.0 (0x00007fa2b926c000)
libopencv_imgcodecs.so.4.5 => /opt/intel/openvino_2021.4.752/opencv/lib/libopencv_imgcodecs.so.4.5 (0x00007fa2b8fb8000)
libopencv_dnn.so.4.5 => /opt/intel/openvino_2021.4.752/opencv/lib/libopencv_dnn.so.4.5 (0x00007fa2b8917000)
libopencv_core.so.4.5 => /opt/intel/openvino_2021.4.752/opencv/lib/libopencv_core.so.4.5 (0x00007fa2b7859000)
libopencv_imgproc.so.4.5 => /opt/intel/openvino_2021.4.752/opencv/lib/libopencv_imgproc.so.4.5 (0x00007fa2b550e000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fa2b5206000)
libm.so.6 => /lib64/libm.so.6 (0x00007fa2b4f04000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa2b4cee000)
libc.so.6 => /lib64/libc.so.6 (0x00007fa2b4920000)
libinference_engine.so => /opt/intel/openvino_2021.4.752/deployment_tools/inference_engine/lib/intel64/libinference_engine.so (0x00007fa2b453c000)
libinference_engine_c_api.so => /opt/intel/openvino_2021.4.752/deployment_tools/inference_engine/lib/intel64/libinference_engine_c_api.so (0x00007fa2b42f0000)
libonnx_importer.so => /opt/intel/openvino_2021.4.752/deployment_tools/ngraph/lib/libonnx_importer.so (0x00007fa2b3d5d000)
libngraph.so => /opt/intel/openvino_2021.4.752/deployment_tools/ngraph/lib/libngraph.so (0x00007fa2b3431000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fa2b322d000)
librt.so.1 => /lib64/librt.so.1 (0x00007fa2b3025000)
libz.so.1 => /lib64/libz.so.1 (0x00007fa2b2e0f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa2b2bf3000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa2ba5ac000)
libtbb.so.2 => /opt/intel/openvino_2021.4.752/deployment_tools/inference_engine/external/tbb/lib/libtbb.so.2 (0x00007fa2b298b000)
libtbbmalloc.so.2 => /opt/intel/openvino_2021.4.752/deployment_tools/inference_engine/external/tbb/lib/libtbbmalloc.so.2 (0x00007fa2b2730000)
libinference_engine_transformations.so => /opt/intel/openvino_2021.4.752/deployment_tools/inference_engine/lib/intel64/libinference_engine_transformations.so (0x00007fa2b21e3000)
libonnx_proto.so => /opt/intel/openvino_2021.4.752/deployment_tools/ngraph/lib/libonnx_proto.so (0x00007fa2b1f7c000)
libprotobuf.so.3.7.1.0 => /opt/intel/openvino_2021.4.752/deployment_tools/ngraph/lib/libprotobuf.so.3.7.1.0 (0x00007fa2b1a75000)

ps: the onnx file is complete not lfs.

thanks in advance!

How to set the quantize range?

Hi,
I am wondering why the quantize range of onnxruntime is fixed to [0,255]? How can I set the quantize range to [-128,127] (https://intellabs.github.io/distiller/algo_quantization.html)?

and how can i set per-channel-scale?

according to the document, U8U8 or U8S8 has better performance on x86-64? What is the reason behind that? What if I use S8U8 or S8S8?

MNIST sample does not build

I cloned this repo and ran the build file in the MNIST folder, which failed.

C:\Users\nakersha\Develop\code\microsoft\onnxruntime-inference-examples\c_cxx\MNIST>cl MNIST.cpp /Zi /EHsc /I..\..\..\include\onnxruntime\core\session /link /LIBPATH:..\..\..\build\Windows\Debug\Debug
Microsoft (R) C/C++ Optimizing Compiler Version 19.27.29117 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

MNIST.cpp
MNIST.cpp(6): fatal error C1083: Cannot open include file: 'onnxruntime_cxx_api.h': No such file or directory

C:\Users\nakersha\Develop\code\microsoft\onnxruntime-inference-examples\c_cxx\MNIST>

Ort::MemoryInfo with CUDA in c++?

Hello,
I'm trying bind some output values to CUDA to avoid copying back to CPU.

I want to do the c++ equivalent of

io.bind_output(name, 'cuda')

and in the following frames, bind the input to the value from the output of the previous frame.

I guess I should do it by the function BindOutput and configure memoryInfo to use CUDA memory.

io_binding.BindOutput(name, memoryInfo);

But I'm not sure how to configure memoryInfo to use CUDA memory because I couldn't find any example doing it.

By searching on other issues, I saw some code saying it's not possible (unless added recently)

//as of 4th Feb 2022 Onnx only supports allocation on the CPU
Ort::MemoryInfo memoryInfo = Ort::MemoryInfo::CreateCpu( 
	OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);

but also saw some code that seems to do it :

Ort::MemoryInfo memoryInfo("Cuda", OrtAllocatorType::OrtDeviceAllocator, 0, OrtMemType::OrtMemTypeDefault);

Due to the lack of documentation, I could not be sure if this second code is really setting for CUDA memory without CPU copy or not.
Does the parameter name ("Cuda") corresponds to the type of memory or is it just a name we give to the memoryInfo object without effect? Does OrtDeviceAllocator means GPU/CUDA memory? Is OrtMemTypeDefault the good value?

Can anyone confirm it?

Many thanks

Failing to get inference result in C++

I have an ONNX model that I trained using XGBoost and onnxmltools, and sucessfully tested in python using onnxruntime. I am now trying to load the model form C++, but I am unable to get viable inferences back. For this pouropose, I build a facilitator class, inspired by this example and this example. The calss looks like this:

#include <onnxruntime_cxx_api.h>

class Predictor{
  public:
    Ort::Session session{nullptr};
    Ort::MemoryInfo memory_info{nullptr};
    char* input_name = nullptr;
    char* output_name = nullptr;
    std::vector<int64_t> *dims;

    Predictor(std::string path) {
      Ort::Env env(OrtLoggingLevel::ORT_LOGGING_LEVEL_WARNING, "fast-inference");
      Ort::SessionOptions sessionOptions;
      sessionOptions.SetIntraOpNumThreads(1);
      sessionOptions.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_EXTENDED);

      this->session = Ort::Session(env, path.c_str(), sessionOptions);
      this->memory_info = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);

      Ort::AllocatorWithDefaultOptions ort_alloc;
      input_name = this->session.GetInputName(0, ort_alloc);
      output_name = this->session.GetOutputName(1, ort_alloc); // 0 returns the labels, 1 returns the probabilities

      Ort::TypeInfo info = this->session.GetInputTypeInfo(0);
      auto tensor_info = info.GetTensorTypeAndShapeInfo();
      size_t dim_count = tensor_info.GetDimensionsCount();
      dims = new std::vector<int64_t>(dim_count);
      tensor_info.GetDimensions((*dims).data(), (*dims).size());
    };

    float* get_prediction(std::vector<float> f){
      const char* i[] = {this->input_name};
      const char* o[] = {this->output_name};

      (*this->dims)[0] = 0; // Apperently, dims are reported as (-1, 6), which generates an error. This changes the -1 to 0.

      auto input_tensor = Ort::Value::CreateTensor<float>(this->memory_info, f.data(), f.size(), (*this->dims).data(), (*this->dims).size());
      Ort::Value output_tensor{nullptr};

      // Ort::Value output_tensor{nullptr};
      // std::vector<Value> p this->session.Run(Ort::RunOptions{nullptr}, i, &input_tensor, 1, o, &output_tensor, 1);
      auto h = this->session.Run(Ort::RunOptions{nullptr}, i, &input_tensor, 1, o, 1);
      float* probs = h[0].GetTensorMutableData<float>();
      
      retrun probs; // <- Problem is here: probs is 0x00 and appears to hold no value
    };
};

void main(){
  Predictor pred("./model.onnx");
  std::vector<float> features{0.0268593, 0.20130636, 0.18040432, 0.11453316, 0.17075175, 0.0};
  auto r = pred.get_prediction(features);
}

However, the vector where the probabilities should be stored appears to be empty. Any clue on what the problem may be?

image_classifier.exe (process 17992) exited with code -1073741701

Hello, I tried to launch the examples but the program always crashes on launch :

C:\hrl\onnxruntime-inference-examples-main\c_cxx\build\imagenet\Debug\image_classifier.exe (process 17992) exited with code -1073741701.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

I didn't change anything to the code.

In another project, I only included
#include <onnxruntime_cxx_api.h>

And I had the same error.

Do you know why ?

Thanks

Why does my onnx model in python just have 1 input but the onnx model in nodejs has 3 inputs?

My actual input:
Im using this matrix with shape [74, 1, 256] for my model.

Onnx model in Python has 1 input but onnx model in Nodejs has 3 inputs

onnx python model input is a tensor:

But nodejs model has 3 inputs ???:

How to apply my actual input to nodejs model?

Superresolution-algorithm

Can anyone guide me how to do the c++ inference of supereesolution model. I have .onnx file of superresolution algorithm. It would be really helpful if someone could write down the steps on what to do. I am unable to understand whats there in Readme file.

Please help me with this.

Running repo with yolox model

Hello, please how can I run this repo with a yolox onnx model, I have tried, this is the error I get:

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: images for the following indices
index: 1 Got: 416 Expected: 3
index: 3 Got: 3 Expected: 416
Please fix either the inputs or the model.

imagenet don't run on ONNX 13

Hello,

I am having this issue running imagenet. To have the project compile I need ONNX runtime 13 but when running it shutdown with that error message:
The given version [13] is not supported, only version 1 to 10 is supported in this build.

I tried using an older version ex. onnxruntime-win-x64-1.10.0 but it does not compiles:
Severity Code Description Project File Line Suppression State
Error C2039 'AllocatedStringPtr': is not a member of 'Ort' capi_test D:\dev\onnxruntime-inference-examples-main\c_cxx\squeezenet\main.cpp 84

std::optionalOrt::AllocatedStringPtr input_name_;