terrencewayne / 18663_project_fbnet

Class Project for 18663 - Implementation of FBNet (Hardware-Aware DNAS)

Hardware Aware Neural Network Architecture Search using FBnets for Raspberry pi

Arxiv paper explaining training - https://arxiv.org/abs/1906.07214

FBNet

• Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search Implementation of FBNet with PyTorch

Salient features

• Includes energy term along with latency term in the loss function
• Loss = CE(w,a) + alpha * (Lat(a) ^ beta) + gamma * (ENER(a) ^ delta)
• Use alpha and beta to tune latency related loss.
• Use gamma and delta to tune energy related loss.
• Tested over Raspberry Pi.
• Supports Tensorboard for accuracy, latency and energy visualization as the FbNet trains (logdir = arg provided to "-tb-logs")
• Energy profiling performed using Power Jive.
• Results (on Raspberr Pi 3B):

Model	Accuracy(%)	Latency(s)	Energy(J)
MobileNetV2	92.4	7.1	18.24
CondenseNet	91.1	4.83	9.28
Ours(HANNA)	87.7	2.88	4.79

The following implementation uses stride values that fit for CIFAR10. Feel free to change stride values for ImageNet as mentioned in the paper. We have used

Relevant graphs

• To check our implementation, we have trained 30 different models for Raspberry Pi with varying values of alpha, beta, gamma and delta
• Each colour on the bubble chart represents a particular model and the size of the bubble represents accuracy.

![alpha-beta-latency-accuracy] (https://github.com/hpnair/18663_Project_FBNet/blob/master/Alpha%2C%20Beta%20vs%20Latency%20.png)

![gamma-delta-energy-accuracy] (https://github.com/hpnair/18663_Project_FBNet/blob/master/Gamma%2C%20Delta%2C%20Energy.png)

Webpage for Training

Example Tensorboard visualization