Last update time: 23-June-2020
The Panoptic Segmentation Task is designed to push the state of the art in scene segmentation. Panoptic segmentation addresses both stuff and thing classes, unifying the typically distinct semantic and instance segmentation tasks. The aim is to generate coherent scene segmentations that are rich and complete, an important step toward real-world vision systems such as in autonomous driving or augmented reality.
Find the main results on my blog: Street View Image Segmentation with PyTorch and Facebook Dectectron2 (CPU+GPU)
- Install Anaconda Python for MacOS.
- Use the
environment.yml
to create the virtual environment and install packages by the following command line:
# create the virtual environment
conda env create -f environment.yml
# activate the virtual environment; default name is detectron2
conda activate detectron2
Note: This .yml
file is created on my mac-mini by: conda env export | grep -v "^prefix: " > environment.yml
. It will install pytorch, torchvision, opencv and other needed packages for this practice. If you want to specify a different install path than the default for your system, use the -p
configuration option: conda env create -f environment.yml -p /home/user/anaconda3/envs/env_name
To make the detectron2 installed on your local mac machine, first step is to pull the detectron2 source repository from the Github.
git clone https://github.com/facebookresearch/detectron2.git
Then, open detectron2/detectron2/config/defaults.py
with your defualt code editor and change the line 28, which tell the compiler that this detectron2 package will be built only for CPU.
# _C.MODEL.DEVICE = "cuda"
_C.MODEL.DEVICE = "cpu"
This will avoid bugs when you doing the later notebook on your mac.
Next, navigate back to the root folder, run the following command to install the detectron2 pacakge:
CC=clang CXX=clang++ python -m pip install -e detectron2
You can check your detectron2 package by the following command:
conda list detectron2
the output will looks like this
#
# Name Version Build Channel
detectron2 0.1.3 dev_0 <develop>
This to notebook contains all the basic operations with the detectron2 package on your local jupyter notebook (CPU) and Colab notebook (GPU by Google).
- CPU macOS (image, video)
- GPU Colab: Detectron2 Tutorial This is original tutorial notebook open this with Colab to enable the GPU.
Reference: https://github.com/facebookresearch/detectron2
The simple approach is to pick a model and its config file from a model zoo, for example, mask_rcnn_R_50_FPN_3x.yaml. Detectron2 provides the demo.py that is able to run builtin standard models. Run it with:
python detectron2/demo/demo.py –config-file detectron2/configs/COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml \
–input input1.jpg input2.jpg \
[–other-options]
–opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl
The video inference is almost the same as image inference. Just replace –input files with –video-input video-clip.mp4
python detectron2/demo/demo.py \
–config-file detectron2/configs/COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml \
–video-input video-clip.mp4 \
–confidence-threshold 0.6 \
–output results/video-output.mkv \
–opts MODEL.WEIGHTS detectron2://COCO-PanopticSegmentation/panoptic_fpn_R_101_3x/139514519/model_final_cafdb1.pkl
The original Detetron2 Colab notebook already covered the following points:
- Run inference on images or videos, with an existing detectron2 model
- Train a detectron2 model on a new dataset
I just want to add a few more points. You can use the following code in Colab which allows you to upload your local image/video to the Colab drive.
from google.colab import files
# upload file to colab drive.
uploaded = files.upload()
for fn in uploaded.keys():
print(‘User uploaded file “{name}” with length {length} bytes’.format(name=fn, length=len(uploaded[fn])))
# download `fn` from colab drive to your machine.
files.download(‘fn’)
Or you can get access of your Google drive by mounting Google Drive locally on Colab with the following command:
from google.colab import drive
drive.mount(‘/content/drive’)