nrikoh / basic_knowledge

Things to learn for new students in the Lab for AI chips and systems of BJTU .

Things to learn for new students in the Lab for AI Chips and Systems of BJTU
智能芯片与应用实验室新生学习指南

• 注意：新生一定认真学习相关基础知识，特别是标记了推荐的内容，至少看两遍
• 要多动手，通过练习才能掌握
• 要主动，不懂就问

Basic Engineering Skills You Have to Master
硕士生根据个人研究方向学习以下内容

Software Programming

Learn the following two programing langrages

• C/C++ (C99/C++11)
• Python (v3.0)

Do the following projects to practice and learn good habits when programming in C

• codewithc

Here's another good repo that has lots of good projects for you to practice.

• Project-Based-Tutorials-in-C (try 'Emulator 101', 'hash table', 'How to Write a Video Player in Less Than 1000 Lines')

Learn the following Good Coding Styles and use them in your research projects:

• C语言的语法风格与代码书写规范指南 （简单）
• NASA C coding style, NASA, 1994 （推荐）
• Recommended C Style and Coding Standards, UC Berkeley, 1997 （简单）
• Guidelines for the use of the C language in critical systems, MISRA, 2018 （高级）
• A list of C and C++ Style Guides
• Embedded C programming

Some other good projects based on C/C++

• Darknet: Open Source Neural Networks in C（深度学习最好的学习C/C++开发实例，推荐）
• BM3D denoising algorithm
• ARM Compute Library

Software Optimizations on Different Platforms

*软件代码优化资源（推荐学习）

Deep Learning Frameworks

• 动手学深度学习PyTorch版（入门推荐）
• Pytorch中文教程
• Nvidia官方training、inference例子、参考代码
• Deep-Learning-with-PyTorch-Chinese 中文翻译含例程
• 斯坦福李飞飞老师课程 Convolutional Neural Networks for Visual Recognition（做目标检测同学推荐学习）

Excellent Deeplearning Project Using Pytorch

There are many good examples that we have collected from github, and students can learn how to write "good" pytorch codes by reading and modifying the codes from these example project. A more complete list is here.

How to setup the GPU and Pytorch environment

Pytorch安装说明

目标检测框架mm-detection代码讲解

• 代码构建一, 代码构建二
• 一些代码注释

Hardware Basic Knowledge

Read the following two books to learn basic concepts for digital circuit design and computer architecture.

Digital Circuit/Logic Design

Important concepts to understand include combinatonal logic and sequential logic (组合逻辑和时序逻辑), register寄存器(Flip-Flop circuit), FSM状态机, counter计数器, decoder/encoder编码器和译码器, FIFO, RAM, etc. Read the following two books:

• ''数字逻辑设计与计算机组成, 戴志涛等，机械工业出版社''
• "Digital Logic"

Computer Architecture

Important concepts to understand include pipeline, memory hierarchy, roofline model, Amdahl's law, ILP (instruction level parallelism), TLP (task level parallelism), DLP (data level parallelism), SIMD/VLIW processor, etc. Read the following two books:

• ''Computer Organization and Design The Hardware Software Interface'', ARM Edition, 2017（重要，零基础学习）
• ''Computer Architecture A Quantitative Approach'', 6th Edition, 2019 （重要，进阶学习）

More reading:

• Loop-carried dependency: 1, 2
• Roofline Model Basic: ./doc/Roofline Model.pdf
• 并行处理的几种常见方式（推荐）

FPGA Design

After you have basic knowledges on digital circuit and computer architectures, you could learn FPGA design or heterogenours computing (using FPGA as accelerators). We recommand using HLS (High-level Synthesis)-based schemes (C/C++-based HLS or OpenCL) rather than RTL-level programming (i.e., Verilog and VHDL) to design application specific circuit on FPGAs (However, if you have time, you should always learn Verilog).

Go to HERE and read all the materials we have listed.

FPGA相关学习资料在这里 (零基础、进阶推荐必读)。

Finally, learn our opensource project PipeCNN. Run the examples, such as caffenet, vgg-16, resnet, YOLO on the Arria-10 FPGA and The Zynq FPGA platforms. Learn how to configure, compile, debug the source codes and profile the performance of the accelerator. After entering our lab, you will have access to our latest designs, i.e., PipeCNN-v2 and PipeCNN-sparse, which are in private repos.

Zhang DeZheng has wrote a good study note on PipeCNN, please read it here.

A list of GOOD FPGA accelerator design can be found here

GPU Design

Learn TensorRT and CUDA programing. Try examples on our TX2/TK1 platforms.

• TensorRT

Research Related Topics
高年级硕士和博士生学习内容

First, students should read the following artichles to learn how to write research papers.

• ''How to write a great research paper'', Deep Learning Indaba, Stellenbosch, 2018
• ''How to Publish a Research Paper'', wikiHow, 2019
• ''How to Write a Good Scientific Paper'', Chris A. Mack, SPIE, 2018
• ''How to Write a Good Paper in Computer Science and How Will It Be Measured by ISI Web of Knowledge'', R?zvan Andonie, et.al., 2010

Secondly, read the following selected papers in each research topics, which are really good examples in the related fields.

Tutorials for Hardware Architectures for DNN

Students who are working on hardware designs for deep neural networks should read the following tutorials.

• Hardware Architectures for Deep Neural Networks, MICRO Tutorial 2016. （推荐）

FPGA Accelerator Design

• Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks, FPGA 2015.
• Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks, FPGA 2016.
• An OpenCL Deep Learning Accelerator on Arria 10, FPGA 2017.
• Improving the Performance of OpenCL-based FPGA Accelerator for Convolutional Neural Network, FPGA 2017.
• A Framework for Generating High Throughput CNN Implementations on FPGAs, FPGA 2018.
• An Efficient Hardware Accelerator for Sparse Convolutional Neural Networks on FPGAs, FCCM 2019.

The following survery papers are also worth reading.

• A Survey of FPGA Based Neural Network Accelerator, ACM TRETS 2017.
• Deep Neural Network Approximation for Custom Hardware: Where We’ve Been, Where We’re Going, ACM Computing Surveys 2019.

Our own research papers on FPGA accelerators:

• PipeCNN: An OpenCL-Based Open-Source FPGA Accelerator for Convolution Neural Networks, FPT 2017
• ABM-SpConv: A Novel Approach to FPGA-Based Acceleration of Convolutional Neural Network Inference, DAC 2019

A more complete paper list is here.

Sparse Convolution Design

A more complete list is here.

Neural network optimization (quantization, pruning, et.al.)

Neural Network Quantization

• Ristretto: A Framework for Empirical Study of Resource-Efficient Inference in Convolutional Neural Networks, IEEE T-NNLS 2018.
• 8-bit Inference with TensorRT, Nvidia 2017.
• Quantizing deep convolutional networks for efficient inference: A whitepaper, Google, 2018.

A more complete list is here.

Network Pruning and Compression

A more complete list is here.

Neural Architecture Search (NAS)

A more complete list is here.

Object Detection

A more complete list is here.