microsoft / onnxjs-demo

demos to show the capabilities of ONNX.js

ONNX.js has been replaced by ONNX Runtime Web which offers enhanced user experience and improved performance. Please visit the following links to get more information:

• ONNX Runtime Web Demo (repo/website)
• Migration ONNX.js to ONNX Runtime Web
• ONNX Runtime Web (README)
• Get started with ORT for JavaScript - onnxruntime.ai
• Tutorials / Deploy on web - onnxruntime.ai
• ONNX Runtime JavaScript API

ONNX.js Demo

ONNX.js demo is an interactive demo portal showing real use cases running ONNX.js runtime in VueJS. It currently supports four examples for you to quickly experience the power of ONNX.js runtime.

The demo is available here ONNX.js demo website.

NOTE: Currently, the supported platforms are Edge/Chrome/Firefox/Electron/Node.js (support for other platforms is coming soon).

Use Cases

The demo provides four scenarios based on four different ONNX pre-trained deep learning models.

1. SqueezeNet

SqueezeNet is a light-weight convolutional network for image classification. In the demo, you can select or upload an image and see which category it's from in miliseconds.

2. ResNet-50

ResNet-50 is a highly-accurate deep convolutional network for image classification. It is trained on 1000 pre-defined classes. Similar to the SqueezeNet demo, you can select or upload an image and see which category it's from.

3. FER+ Emotion Recognition

Emotion Ferplus is a deep convolutional neural network for emotion recognition in faces. In the demo, you can choose to either select an image with any human face or to start a webcam and see what emotion it's showing.

4. Yolo

Yolo is a real-time neural network for object detection. It can detect 20 different objects such as person, potted plant and chair. In the demo, you can choose to either select an image or start a webcam to see what objects are in it.

5. MNIST

MNIST is a convolutional neural network that predicts handwritten digits. In the demo, you can draw any number on the canvas and the model will tell you what number it is!

Run ONNX.js Demo

Install Dependencies

npm install

Serve the demo

Serve the demo in localhost

npm run serve

This will start a dev server and run ONNX.js demo on your localhost.

Deploy the demo

npm run build

This will pack the source files into /docs folder and be ready for deployment.

- Electron support

Onnx.js demo can also serve as a Windows desktop app using Electron.

First create a developer build of the app by running

npm run build -- --mode developer

Then run

npm run electron-packager

This will create a new /ONNXjs-demo-win32-x64 folder. Run /ONNXjs-demo-win32-x64/ONNXjs-demo.exe to enjoy Electron desktop app.

Credits

This demo is adapted from keras.js demo. Modifications have been made to the UI and the backend uses ONNX.js.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

onnxjs-demo's Issues

Preprocessing in emotion model - resizing using canvas context?

I am wondering if there is any specific reason why the resizing of the input to 64x64 is done using the canvas context, since the original implementation of the model has unlikely used the same method. I am using the same onnx model you have here, and reimplementing this in python, but using PIL to resize. Hence, I am unable to fully reproduce, although it is close enough. Any thoughts on what the PIL resizing filter equivalent to the canvas resizing, if any? Also related.

Error loading URL for predefined images

At https://microsoft.github.io/onnxjs-demo/#/yolo, when selecting a predefined image e.g. "bicycle", an error "Error loading URL" appears for about half of the images.

'Upload Image' functionality not working properly in Microsoft Edge

Microsoft Edge 38.14393.2068.0
Microsoft EdgeHTML 14.14393

Action: User clicks the 'UPLOAD IMAGE' button on any of the demos (excluding MNIST ofc), and selects an image to upload.
Expected: The image is loaded into the window, followed by the model results being updated.
Actual: The image is then loaded into the window, but the open file dialog window re-appears regardless of user action (re-opening the image, opening a different image, clicking the 'Cancel' button, or closing the dialog window). The only way to exit this loop is to right-click the taskbar and select 'Close window' to terminate the complete browser.

Note: This occurs whether I try to upload JPEG or PNG images.

Asking about order of RGB channels

https://github.com/Microsoft/onnxjs-demo/blob/5499e5dbaf779896882755604162a4c15161a65b/src/components/models/Resnet50.vue#L48-L50

I wonder why the order of RGB channels are reversed?

The electron application doesn't seem to work.

I packaged an electron application following the instructions in README, but the application doesn't show anything.

How to get unmangled javascript code for the demo?

It appears that the code has been deliberately obfuscated.

Demo for video processing

I want to create a video classification demo where I record a fixed frame length video (in browser), and then give feed it into an onnx model. Could you tell me how to do so? as in the problem boils down to taking the video input into Float32Arrays

'Upload Image' functionality not working in Firefox

Firefox version: 63.0.3 (64-bit)

Action: User clicks the 'UPLOAD IMAGE' button on any of the demos (excluding MNIST ofc).
Expected: The open file dialog should appear and allow the user to select an image to upload.
Actual: Nothing happens, no errors appear in the UI or console.

Note: Works as expected on Google Chrome v70.0.3538.110 (Official Build) (64-bit)

Links broken

All of the demo links in this README are broken. Can you fix?

https://github.com/microsoft/onnxjs-demo/blob/master/README.md

Error: backend not compatible

When running the Yolo demo on both windows and linux, if I change the yolo.onnx model to my custom yolo model, I get an error when I open up the Yolo demo. The camera will not run, and the interface says: "Error: Current backend is not supported on your machine. Try selecting a different backend.

This happens for both backends.

This did not happen when I was using the yolo.onnx model that came with the demo, only when I tried to use my own custom-trained yolo v5 model.