ros2_intel_movidius_ncs

Warning: This repo is deprecated. For latest ROS2 wrapper for Movidius NCS, please refer to our project ros2_openvino_toolkit.

See: https://github.com/intel/ros2_openvino_toolkit!!

1 Introduction

The Movidius™ Neural Compute Stick (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit (VPU) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.

This project is a ROS2 wrapper for NC API of NCSDK, providing the following features:

A ROS2 service for object classification and detection of a static image file
A ROS2 publisher for object classification and detection of a video stream from a RGB camera
Demo applications to show the capabilities of ROS2 service and publisher
Support multiple CNN models of Caffe and Tensorflow

2 Prerequisite

An x86_64 computer running Ubuntu 16.04. OS X and Windows are not supported yet
ROS2 Bouncy
Movidius Neural Compute Stick (NCS)
Movidius Neural Compute (MvNC) SDK
Movidius Neural Compute Application Zoo
RGB Camera, e.g. RealSense D400 Series

3 Environment Setup

Install OpenCV 3.x (guide)
Install ROS2 Ardent (guide)
Create a ROS2 workspace

mkdir -p ~/ros2_ws/src

Install NCSDK v1.12.00 (github)
Install NC APP Zoo(github)
NCSDK should be installed in /opt/movidius by default. Create a symbol link in /opt/movidius to NC APP Zoo.

sudo ln -s <your-workspace>/ncappzoo /opt/movidius/ncappzoo

After that, make sure you can find graph data in /opt/movidius/ncappzoo/caffe or /opt/movidius/ncappzoo/tensorflow and image data in /opt/movidius/ncappzoo/data/images

Install object_msgs for ROS2 (github)
Install ROS2 package for different cameras as needed. e.g.
1. RealSense D400 series camera (github) Note: Create a symbol link from libusb.a to libusb-1.0.a, otherwise "libusb.a" is probably not to be found by librealsense.
  sudo ln -s /usr/lib/x86_64-linux-gnu/libusb-1.0.a /usr/lib/libusb.a
Install ROS2 Message Filters (github)

cd /usr/lib/x86_64-linux-gnu
sudo ln -s libboost_python-py35.so libboost_python3.so
cd ~/ros2_ws/src
git clone https://github.com/ros2/message_filters

Install ROS2 vision_opencv (github)

cd ~/ros2_ws/src
git clone https://github.com/ros-perception/vision_opencv
cd vision_opencv
git checkout ros2

4 Building and Installation

cd ~/ros2_ws/src
git clone https://github.com/intel/ros2_intel_movidius_ncs
git clone https://github.com/intel/ros2_object_msgs
cd ~/ros2_ws
ament build
source install/local_setup.bash

Copy object label file from this project to NCSDK installation location.

cp ~/ros2_ws/src/ros2_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/

5 Running the Demo

5.1 Classification

5.1.1 Supported CNN Models

Table1

CNN Model	Framework	Usage
AlexNet	Caffe	Image/Video
GoogLeNet	Caffe	Image/Video
SqueezeNet	Caffe	Image/Video
Inception_v1	Tensorflow	Image/Video
Inception_v2	Tensorflow	Image/Video
Inception_v3	Tensorflow	Image/Video
Inception_v4	Tensorflow	Image/Video
MobileNet	Tensorflow	Image/Video

5.1.2 Classification Result with GoogLeNet

5.1.3 Running the Demo

5.2 Detection

5.1.1 Supported CNN Models

CNN Model	Framework	Usage
MobileNetSSD(Recommended)	Caffe	Image/Video
TinyYolo_v1	Caffe	Image/Video

5.1.2 Detection Result with MobileNetSSD

5.1.3 Running the Demo

6 Interfaces

6.1 Topic

Classification: /movidius_ncs_nodelet/classified_objects
Detection: /movidius_ncs_nodelet/detected_objects

6.2 Service

Classification: /movidius_ncs_image/classify_object
Detection: /movidius_ncs_image/detect_object

isabella232 / ros2_intel_movidius_ncs Goto Github PK