mllite / ml2cpp Goto Github PK

sklearn.svm.OneClassSVM
sklearn.covariance.EllipticEnvelope

Follow the six steps described in #1

ml2cpp, as it is now, allows converting any ML model to a C++ code for inference purposes.

This code is however not yet optimized to run on small devices : these devices may have low CPU speed, low memory, low-power, no FPU, no real OS. ml2cpp Jupyter notebooks were run and inference was tested on a sparc64 or x86-64 server with gigabytes of memory and running full-featured g++ compiler.

We need:

1. get some hardware/micro-controller : STM32, ESP32 and K210 (riscv-64) at least.
2. Check that the generated code can be compiled with the respective gcc versions (C++-17 support, exceptions, RTTI are not always available).
3. check the capacities of these devices (CPU + memory)
4. Check floating point issues : is there an FPU? half-precision floats (float16), bfloat16, ...
5. get an emulator for each device (qemu-???). use it for automated tests.

Priority 1 : Getting at least one of these devices running all these steps (feasibility). choose the one with less constraints (K210 can run a small linux)

The goal is to be able to run the model on the bare metal. No arduino. No MicroPython.

Using STM32, ESP32 or K210 is here only for tuning purposes. Their only added value is to provide a test envoironment so that C++ code will be incrementally adapted in a real-world case.

ensemble.AdaBoostClassifier
ensemble.AdaBoostRegressor
ensemble.BaggingClassifier
ensemble.BaggingRegressor
ensemble.ExtraTreesClassifier
ensemble.ExtraTreesRegressor
ensemble.GradientBoostingClassifier
ensemble.GradientBoostingRegressor
ensemble.IsolationForest
ensemble.RandomForestClassifier
ensemble.RandomForestRegressor
...

Follow the six steps described in #1

Basic support for linear models as a first working case for prototyping and C++ design.

RidgeClassifier + Ridge Regressor

Need to have a complete specification for the following :

1. Test datasets (CSV file => C++ std::map)
2. Classification/Regression/Transformation models : C++ functions used to compute the scores
3. Classification/Regression/Transformation models : input/output datasets layouts

This spec should evolve when more and more models/features are added.

pipeline.FeatureUnion and pipeline.Pipeline

Follow the six steps described in #1

Need to prototype something that works, even partially automated or hardcoded, to compute the scores of a pickled classification model using only C++ code.

feature_selection.*

Follow the six steps described in #1

ML2CPP can generate specific C++ code with RISC-V extensions allowing all scikit-learn, pytorch, caret ML models to be deployable natively on this platform.

RISC-V extensions have the advantage to be non-proprietary.

This is a place holder for following the recent efforts on designing RISC-V specific extensions for AI/ML.

No public hardware is available yet.

Sifive Intelligence X280 CPU is an interesting start point. https://www.sifive.com/blog/introduction-to-the-sifive-intelligence-x280

svm.SVC and svm.SVR

Follow the six steps described in #1

Caret Earth/MARS models.

Follow the six steps described in #1

sklearn.decomposition.PCA
sklearn.decomposition.FastICA
sklearn.decomposition.TruncatedSVD

Follow the six steps described in #1

random_projection.GaussianRandomProjection
random_projection.SparseRandomProjection

Follow the six steps described in #1

r = requests.post("https://sklearn2sql.herokuapp.com/model", json=json_data)
The link is not present anymore. Is there any workaround ?
Also "score_csv_file" API is missing. Any suggestions ?
I am trying to convert a BaggingRegressor model from sklearn to c++.

-- Raj

Basic support for decision tree models as a working case for prototyping and C++ design.

DecisionTreeClassifier + DecisionTreeRegressor

Follow the six steps described in #1

See #25

For each device, Check that the major model categories are runnable on the device (enough CPU and memory).

neural_network.MLPClassifier and neural_network.MLPRegressor

Follow the six steps described in #1

Keras

Follow the six steps described in #1

Add more tests and benchmarks.

https://github.com/antoinecarme/sklearn2sql-demo/tree/master/output_temp_tables

Follow the six steps described in #1

naive_bayes.GaussianNB and naive_bayes.MultinomialNB

Follow the six steps described in #1

We need a way to perform the same checks done for scikit-learn python models for assessing the quality of C++ code generation for machine learning models.

Need (all the code in python):

1. generate/train a model on a dataset (training = python data.frame)

2. predict the classes of a test dataset ( test = python data.frame df_out)

3. generate a C++ code for the model (use the sklearn2sql_heroku for example)

4. transfer the test dataset to a local CSV file.

5. Compile and execute the model C++ generated in point 3 ( => predicted CSV file => data.frame cpp_df_out)

6. compare the two dataframes (df_out and cpp_df_out). A simple merge is OK. The prediction columns should be identical.

In a first time, the goal is just to put in place the framework to be used as a template for coming tests. An xgboost model is OK for this task.

deliverable : A jupyter notebook with all the process.

Web Service. Can be started earlier in the process.

Follow the six steps described in #1

Misc Models : feature selection / Earth / ..

Follow the six steps described in #1

sklearn.calibration.CalibratedClassifierCV

Follow the six steps described in #1

See #25

ml2cp uses float64 in all computations by default. This is not always a limitation.

For each device, Check the availability of major floating point types (float64, float32, float16, bfloat16, ...)

1. Allow using float32, float16 or bfloat16 when available.
2. Allow using hardware FPU when available.
3. Adapt the code, when needed, according to floating point capabilities.

Xgboost

https://github.com/dmlc/xgboost

Use json outputs..

Follow the six steps described in #1

sklearn.preprocessing.*

Follow the six steps described in #1

See #25

For each device, Check the availability of a strong enough C++ compiler.

Desired functions :

1. C++-17 support (std::any or a workaround)
2. RTTI
3. Exceptions
4. ...

See https://github.com/antoinecarme/caret2sql

Follow the six steps described in #1

See #25

This is a place holder for all Kendryte K210 specific notes/ideas.

https://www.canaan.io/product/kendryteai

See #25

For each device, get a qemu-??? emulator.

Add some basic ml models and run the generated C++ code for these models in each emulator. Add these to the build and release tests.

See #25

This is a place holder for all Espressif Systems ESP32 specific notes/ideas.

https://en.wikipedia.org/wiki/ESP32

Boost.Python can be used to transform the C++ model code into a python module that can be imported.

https://www.boost.org/doc/libs/1_63_0/libs/python/doc/html/index.html

Follow the six steps described in #1

This method can be used for deploying a model without having numpy/scikit-learn/R/keras installed (see #25 with MicroPython).

LightGBM support

https://github.com/microsoft/LightGBM

Use json outputs..

Follow the six steps described in #1

See #25

This is a place holder for all STMicroelectronics STM32 specific notes/ideas.

https://en.wikipedia.org/wiki/STM32

Hi, I want to try your method to deploy my ml model in c++. Can you provide some examples of using the saved ml model in c++?

Pytorch

Follow the six steps described in #1

See : https://github.com/antoinecarme/pytorch2sql