tum-ei-eda / mlonmcu-sw Goto Github PK

As a long term goal it would be desirable to support running the MLIF/Target Software on an operating system in addition to allowing bare metal only.

The most relevant ones are probably:

• FreeRTOS
• ZephyrOS

The latter one would also have the advantage that the build process can be managed by Zephyr as well which means that we could easily support a large number of targets without implementing every single one by ourselves.

This feature is fully optional, so please do not feel obligated to implement it anytime soon!

@DanMueGri reached out to me that we would like to support using an ESP32(C3) with the MLonMCU flow. While it would be theoretically possible to integrate the toolchain/linker files/... inside this repository but at some point the complexity increases too much.

Maybe we should consider using arduino-cli or platformio to build/deploy target software for such targets. The question is then how the interface between MLonMCU and those tools may look like and if there are further caveats we need to consider.

Motivation

First, I would like to clarify that MLIF is an abbreviation for "Machine Learning Interface" @rafzi came up with some time ago.

There are a few things regarding the future of this MLonMCU component I would like to discuss in the following.

Alternative Naming

For users not familar with the term MLIF it might sound confusing, especially considering that it can easily messed up with MLF (Model Library Format) which is used in MLonMCU as well. Let me know what your think about this.

Extendability

Im am not so happy with the way how the frameworks, backends and targets are selected via the command line.

Compared to the original version I already replaced the mechanism implicityly using -DTVM_DIR=foo -DTF_SRC= and -DTVM_RT=ON -DTVM_AOT=OFF by explicity flags taking a string value instead which will be checked in lib/ml_interface/CMakeLists.txt:

-DFRAMEWORK=tvm -DBACKEND=tvmaot -DTARGET_SYSTEM=etiss_pulpino

However this does not generalize well as different Frameworks/Backend/Targets are not independend to each other.

A first proposal to an alternative approach would be the following:

Every Framework

Example structure:

.
└── cmake
    ├── backends
    │   ├── tvmaot.cmake
    │   ├── tvmcg.cmake
    │   └── tvmrt.cmake
    ├── frameworks
    │   ├── tflite.cmake
    │   └── tvm.cmake
    └── targets
        ├── etiss_pulpino.cmake
        └── host_x86.cmake

However there are also some caveats:

• Redundancy of code which could be shared between files
• Ideally backends should imply the framework they are based to get rid of the need to specify both on the command line
• Using tvmaot.cmake etc. inside here does not feel right as the Repo should be useful even without MLonMCU on top. Therefore be might be able to omit -DBACKEND=tvmaot and just use something like this instead: -DFRAMEWORK=tvm -DTVM_EXECUTOR=aot

In the end it would be great if we could come up with an interface which suits solve the mentioned issues. Ideally a third part should be able to replace our mlonmcu-sw completely by their target-software stack (i.e. RTOS based) by just makeing sure that the alternative implementation is using the pre-defined interfaced supported by MLonMCU.

https://github.com/tum-ei-eda/mlonmcu-sw/blob/main/cmake/targets/etiss_pulpino.cmake#L61-L62

https://github.com/tum-ei-eda/mlonmcu-sw/blob/main/cmake/toolchains/llvm_etiss_pulpino.cmake#L20

https://github.com/tum-ei-eda/mlonmcu-sw/blob/main/cmake/toolchains/gcc_etiss_pulpino.cmake#L19

Currently subprojects such as muriscvnn and CMSISNN are build with a hardcoded number of threads which is currently either 1, 4 or $(nproc). As we use MLonMCU on machines with 4, 8, 24 cores we should adapt the number of used threads to the mlif.num_threads setting to use our resources efficiently. As the global -j argument can not be passed to the Makefile, we need to add an additional command line setting which can be set by MLonMCU: i.e. -DNUM_THREADS=