Smoker Detection for Jetson Nano USB Camera

A basic project to deploy a smoker detection application on a Jetson Nano (4GB), using the Jetson-Interface libraries as well as the Roboflow Docker container with a model that I trained, but all running locally. The basics are as follows: The USB camera streams to the initial model to detect a person which then captures the image and sends to the pre-trained model running on the roboflow docker image and checks for the appearance of a cigarette. If a cigarette is detected the model writes "Potential Smoker" to the image and saves it in the Smoking folder for later viewing. If the person is not smoking, the image is deleted from the capture directory.

I am aware that this could be further optimized from every Nth frame check to further training the model on Roboflow. My intention was not for perfection but rather for demonstration that this and other CV projects can be developed and deployed on the Jetson architecture. Feel free to fork and build on making this even faster and more accurate. This project uses a containerized cigarette detection model trained on a roboflow platform using a public dataset (Model and Dataset).

Requirements

Below are the recommended hardware requirements for this project. I have selected the 128GB microSDXC based on the size of the libraries and dependencies.

NVIDIA Jetson Nano Developer Kit (945-13450-0000-100)
SanDisk Ultra microSDXC 128GB 100MB/s SDSQUNR 128G GN6MN

Getting Started

In order to get this project up and running it is assumed that you have already flashed and set up the Ubuntu desktop environment.

Check Swap space

free -m
sudo systemctl disable nvzramconfig

Create 4GB swap file

sudo fallocate -l 4G /mnt/4GB.swap
sudo chmod 600 /mnt/4GB.swap
sudo mkswap /mnt/4GB.swap

Append the following line to /etc/fstab

sudo su
echo "/mnt/4GB.swap swap swap defaults 0 0" >> /etc/fstab
exit

Update again and reboot

sudo apt-get update
sudo shutdown -r now

Set Up

Get an account on (RoboFlow.com), you will need this to use the inference call. Once you set up an account locate your API key.

Get an API key from roboflow.com
***** Replace the "<-API KEY->" in the smoking_detection.py file with your API Key *****
Do not forget to save the file after editing.
Now you need to give the scripts the correct permissions to be able to run on the Jetson Nano.
Navigate to the directory containing the (max.sh and the start_smoking_container.sh) files.
Open a terminal in that directory and run the following commands.

Now that your scripts can be ran start all of the required installs by running the following:
When prompted with configuration file question: type "N" both times.
You will be prompted to automatically restart the Docker Daemon, click "No".
You may get asked if you want to install certain libraries etc, type Y and let run.
When you get to the Models options window, please be sure to select (ssd-mobilenet-v2) Under the Object Detection heading.
When you get to the Pytorch installation screen select it and Tab to "Ok".
The overall time of implementation can vary depending on internet speeds and other factors. It usually takes around 30 minutes.
Upon completion your Nano should reboot.

Start Up

Once the install finish and the Nano reboots you can test the application as follows:

Log back into the device.
Navigate to the root folder of the project.
First we will boost the performance of the Nano.
Open a terminal and enter the following:

Now we will start the Smoking detection Docker Container:
Now open another terminal in the same directory and enter the following:
The initial run may take a few minutes to load and start inferencing based on the Docker container as well as the 2 models the stream runs through. Remember initially we need to detect that a person is there, then we detect if that person has a cigarette.
Please ensure network access is available for the initial run as the weights will need to be downloaded from the Roboflow servers. After that you should be good to go moving forward.
Your display may go dark and freeze on the initial run as well, this is just because of the models and resources loading. Wait a minute and it will return to normal. This will not happen on subsequent runs.

Your display should resemble the images below. Be sure to check the "Smoking" directory for any captured images of smokers.