mldl / bolt

Bolt is a deep learning library with high performance and heterogeneous flexibility.

Introduction

Bolt is a light-weight library for deep learning. Bolt, as a universal deployment tool for all kinds of neural networks, aims to minimize the inference runtime as much as possible. Higher speed, better security and more efficient memory management are the advantages that Bolt strives to provide. Bolt has been widely deployed and used in many departments of HUAWEI company, such as 2012 Laboratory, CBG and HUAWEI Product Lines. Feel free to make good use of issue submission, or join our QQ chatroom (Chinese): 833345709.

Quick Start

Generally, there are two steps to get started with bolt. It's quiet easy for users to quickly running bolt.

1. Conversion: use X2bolt to convert your model from caffe,onnx,tflite or tensorflow to .bolt;

2. Inference: run benchmark with .bolt and data to get the inference result.

   For more details about the usage of X2bolt and benchmark tools, see docs/USER_HANDBOOK.md.

Features

• Support Frameworks

  caffe, onnx, tflite, tensorflow

• Inference Precision

  Float32, Float16, Int8, 1-bit

• Hardware

  ARM CPU(v7, v8, v8.2), Mali GPU, X86(AVX2)

• Verified Networks

  Bolt has shown its high performance in the inference of common CV and NLP neural networks. Some of the representative networks that we have verified are listed below. You can find detailed benchmark information in docs/BENCHMARK.md.

  Application	Models
  CV	Squeezenet/Mobilenet_v1/Mobilenet_v2/Mobilenet_v3/Resnet50 /Ghostnet/SSD/Yolov3/Pointnet/...etc.
  NLP	Bert/TinyBert/Albert/Neural Machine Translation/Text To Speech /Automatic Speech Recognition/...etc.
  More DL Tasks	...

  More models than these mentioned above are supported, users are encouraged to further explore.

• More Advanced Features

  • Graph Optimization
  • Thread Affinity
  • Algorithm Tuning
  • Time-Series Data Acceleration

Documentations

Everything you want to know about bolt is recorded in the detailed documentations stored in docs.

• How to install bolt with different compilers.
• How to use bolt to inference your ML models.
• How to develop bolt to customize more models.
• Benchmark results on some universal models.
• How to build APP with kit.
• Frequently Asked Questions(FAQ)

Acknowledgement

Bolt refers to the following projects: caffe, onnx, tensorflow, ncnn, mnn, dabnn.

License

The MIT License(MIT)