lobe / lobe.net

.NET library for lobe.

.NET libraries for Lobe

A .NET library to run inference on exported Lobe models.

How to get started

Export the model from Lobe app

• Export your model as ONNX format

Use the model in your own .NET application

Install the follwing packages

• lobe.Onnx to import the Onnx based implementation of the image classifier.
• lobe.ImageSharp to get image manipulation utilities
• Microsoft.ML.OnnxRuntime to get he native onnx runtimes

This code creates a simple command line app that loads a model and classifies an image file

using System;
using System.IO;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
using lobe.ImageSharp;

namespace lobe.TestApp
{
    class Program
    {
        static void Main(string[] args)
        {
            var signatureFilePath = args[0];
            var imageToClassify = args[1];

            ImageClassifier.Register("onnx", () => new OnnxImageClassifier());
            using var classifier = ImageClassifier.CreateFromSignatureFile(
                new FileInfo(signatureFilePath));

            var results = classifier.Classify(Image
                .Load(imageToClassify).CloneAs<Rgb24>());
            Console.WriteLine(results.Classification.Label);
        }
    }
}

The code

ImageClassifier.Register("onnx", () => new OnnxImageClassifier());

Registers the OnnxImageClassifier against the format onnx

Then a classifier can be built from a signature file

using var classifier = ImageClassifier.CreateFromSignatureFile(new FileInfo(signatureFilePath));

Use the lobe app directly

For rapid iterations you can test your model by taking advantage of the http endpoint that the lobe app exposes. First, open the app and then the model you would like to use. Next, go to the export menu and select the api option, get the url from there (it should look like http://localhost:38100/predict/bdff75cc-ee54-46cf-a290-f9095ef78516").

In your .NET App make sure you have installed the following nuget packages

• lobe.Http
• lobe.ImageSharp

Then import the namespaces

using System.IO;
using SixLabors.ImageSharp;

using lobe.Http;

And finally create the client, connect to the endpoint and now use it classify the image

var client = new LobeClient();
client.UseUri(new Uri("http://localhost:38100/predict/bdff75cc-ee54-46cf-a290-f9095ef78516"));

var result = client.Classify(picture.CloneAs<Rgb24>());

lobe.net's Issues

Cannot parse outputs.Prediction.name

Hi!

I'm working in C#, .NET Framework 4.6.1.

So I've trained a simple 4-class image classifier in Lobe, exported as an onnx model. I'm using the OnnxRuntime.MKLML package to load it, it initializes alright when I call this:

ImageClassifier.Register("onnx", () => new OnnxImageClassifier());
                classifier = ImageClassifier.CreateFromSignatureFile(new FileInfo(ModelPathAndName)); // Lobe.AI Model

but then when I try to call classifier.Classify() on the image, I get this error:

System.Collections.Generic.KeyNotFoundException: 'Cannot parse outputs.Prediction.name'
with the Inner Exception:
KeyNotFoundException: The given key was not present in the dictionary.

I'm calling it with this line:

var results = classifier.Classify(SixLabors.ImageSharp.Image
                .Load(sFilename).CloneAs<Rgb24>());

Just in case, here's the signature.json file that I got when I exported the model:

{
  "doc_id": "c916a4a7-ee27-4926-a5b8-1162f4477fc7",
  "doc_name": "AutoRotate",
  "doc_version": "9d01647d49e2fd0e05622b6a7ec3c4eb",
  "format": "onnx",
  "version": 12,
  "inputs": {
    "Image": {
      "dtype": "float32",
      "shape": [ null, 224, 224, 3 ],
      "name": "Image:0"
    }
  },
  "outputs": {
    "Confidences": {
      "dtype": "float32",
      "shape": [ null, 4 ],
      "name": "c916a4a7-ee27-4926-a5b8-1162f4477fc7.1842bd14-28e9-4261-bc41-4083b30ea3a4/dense_2/Softmax:0"
    }
  },
  "tags": [],
  "classes": 
  { "Label": [ "0", "180", "270", "90" ] },
  "filename": "model.onnx", 
  "export_model_version": 1}

Wrong Example?

The example on README.md shows:
Console.WriteLine(results.Classification.Label);

But results has a collection called Classifications, not a single item.

Error BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)

hello everybody
i made a LOBE model as onnx. then i used the sample code in the main page
but it shows me an error on runtime:

System.TypeInitializationException
HResult=0x80131534
Message=The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
Source=Microsoft.ML.OnnxRuntime
StackTrace:
at Microsoft.ML.OnnxRuntime.SessionOptions..ctor() in C:\a\3\s\csharp\src\Microsoft.ML.OnnxRuntime\SessionOptions.cs:line 50
at lobe.OnnxImageClassifier.InitialiseFromSignature(Signature signature, String modelFileName)
at lobe.ImageClassifier.CreateFromSignature(Signature signature, String modelFileName, String engineType)
at lobe.ImageClassifier.CreateFromSignatureFile(FileInfo signatureFile, String modelFileName, String format)
at AIWebApp.ML.LobeOfficeAppDetection.Main(String signatureFilePath, String imageToClassify) in G:***\LobeDetection.cs:line 26 at *.Tests.MyTests.Test1() in G:*\MyTests.cs:line 21

Inner Exception 1:
BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)

Could you please help me to solve the problem ?
thanks in advanced
regards
mohammad

How to get ONNX input/output class from signature file?

Hello:
I have installed Lobe Application in my Windows 10 PC, from this URL: https://www.lobe.ai/
I used a small dataset to train and get an ONNX model. This model is an image converter, it is not a classifier.

The following is the signature JSON file:
{
"doc_id": "2cc072ae-dbda-43c5-8db7-4c2515e0548d",
"doc_name": "model",
"doc_version": "9c8be1ab541b4591f46bb01fc60116cf",
"format": "onnx",
"version": 5,
"inputs": {
"Image": {
"dtype": "float32",
"shape": [null, 224, 224, 3],
"name": "Image:0"
}
},
"outputs": {
"Confidences": {
"dtype": "float32",
"shape": [null, 2],
"name": "2cc072ae-dbda-43c5-8db7-4c2515e0548d.9554adfa-5d26-484f-88d3-dea9f2f46f01/dense_2/Softmax:0"
}
},
"tags": [],
"classes": {
"Label": ["test_A", "test_B"]
},
"filename": "model.onnx",
"export_model_version": 1
}

I also use mlgen for Visual Studio 2019 from this URL, to generate some C# wrapper class, like this:
public sealed class ModelInput
{
public TensorFloat Image00; // shape(-1,224,224,3)
}

public sealed class ModelOutput
{
    public TensorFloat output2cc072ae0dbda043c508db704c2515e0548d09554adfa05d260484f088d30dea9f2f46f010dense_20Softmax00; // shape(-1,2)
}

public sealed class ModelModel
{
    private LearningModel model;
    private LearningModelSession session;
    private LearningModelBinding binding;
    public static async Task<ModelModel> CreateFromStreamAsync(IRandomAccessStreamReference stream)
    {
        ModelModel learningModel = new ModelModel();
        learningModel.model = await LearningModel.LoadFromStreamAsync(stream);
        learningModel.session = new LearningModelSession(learningModel.model);
        learningModel.binding = new LearningModelBinding(learningModel.session);
        return learningModel;
    }
    public async Task<ModelOutput> EvaluateAsync(ModelInput input)
    {
        binding.Bind("Image:0", input.Image00);
        var result = await session.EvaluateAsync(binding, "0");
        var output = new ModelOutput();
        output.output2cc072ae0dbda043c508db704c2515e0548d09554adfa05d260484f088d30dea9f2f46f010dense_20Softmax00 = result.Outputs["2cc072ae-dbda-43c5-8db7-4c2515e0548d.9554adfa-5d26-484f-88d3-dea9f2f46f01/dense_2/Softmax:0"] as TensorFloat;
        return output;
    }

But I don’t quite understand the meaning of the above class.
If I want to use an input image and generate an output image from the ONNX model.
How I can write an OnnxInput and OnnxOutputClass?
I have no idea about the input shape [null, …] means what and output shape [null, 2] means what?

By the way, the input/output name seems to be very long and meaningless.
Can I use python code to change the ONNX model, so it will show some meaningful names for input/output? For example, if I want to use output name like: Iamge2Sketch, not like this one:
"name": "2cc072ae-dbda-43c5-8db7-4c2515e0548d.9554adfa-5d26-484f-88d3-dea9f2f46f01/dense_2/Softmax:0"

I am using Visual Studio 2022 and 2019 on Windows 10, I also installed Python 3.9.
Please advise,
Thanks,

Onnx format - JsonReaderException when trying to register image classifier in dotnet core application

I followed the dotnet example provided and get error while registering image classifier as seen on line 52. I exported my model in onnx format from Lobe application.

Below is the error description:
'0x08' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.

Please let me know if I missed any step.

How to upload image for lobe trained model in local host?

Hello:
I have trained an image converter model in Lobe (version 0.10.1130.5). I also exported as an ONNX model. But I don’t know how to write some C# code to make some predication.
However, I found that I can use Lobe locally and post image to it. So I want to try this model.
I run Lobe in Lobe Connect, and I can see its local URL:
http://localhost:38101/v1/predict/2cc072ae-dbda-43c5-8db7-4c2515e0548d

However, the HTTP request is in this format:
{
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": {
"image": ""
}
}

I used one C# program to convert one of the training image to base64 string. Then I launch PostMan (Version 9.15.2) for Windows 10 to post the following Json data to local lobe connect end-point, like this:
{
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": {
"image": "iVBORw0KGgoAAAANSUhEUgAABAAAAAQACAIAAADwf7zUAAEAAE......=="
}
}

Then I quickly get response from lobe end-point, with the following Json reply:
{
"errors": [
{
"value": "{ "method": "POST", "headers": { "Content-Type": "application/json" }, "body": {
"image": "iVBORw0KGgoAAAANSUhEUgAABAAAAAQACAIAAADwf7zUAAEAAE......==" }}",
"msg": "Request body should be {"image": ""}",
"param": "",
"location": "body"
}
]
}

It seems Lobe connect does NOT recognize the base64 as an image.
Please advise on how to fix this?

Onnx model gives incorrect classifications

Using exported onnx can sometimes produces incorrect classification in comparison to tensorflow and lobe app.