yingran-cn / barracuda-posenet-tutorial

This tutorial series provides step-by-step instructions for how to perform human pose estimation in Unity with the Barracuda inference library.

Barracuda PoseNet Tutorial 2nd Edition

Tutorial Links

• Part 1: Install the Barracuda package in a Unity project and import the required video files and PoseNet models.
• Part 2: Set up a video player and webcam in Unity.
• Part 3: Implement the preprocessing steps for the MobileNet and ResNet PoseNet models.
• Part 4: Load, modify, and execute the PoseNet models.
• Part 5: Implement the post-processing steps for single pose estimation with PoseNet.
• Part 6: Implement the post-processing steps for multi-pose estimation with PoseNet.
• Part 7: Create pose skeletons and manipulate them using output from a PoseNet model.
• WebGL: Modify the Barracuda PoseNet project to run in a browser using WebGL.

Single Pose Estimation

single-pose-test.mp4

Multi-Pose Estimation

multi-pose-test.mp4

---

Update - 3/10/22

Added a new branch and tutorial for adapting the project to run in a browser using WebGL.

You can try out a live demo at the link below.

• Barracuda PoseNet WebGL Demo

---

Update - 12/7/21

Added a new branch that uses Barracuda version 2.3.1. Version 2.3.1 contains some improvements over version 2.1.0 used in the main branch that may be especially relevant when building for non-Windows platforms.

It has support for creating a tensor from a RenderTexture even when compute shaders or GPUs are not available.

It also now has a Pixel Shader worker. This allows models to run on GPU without requiring support for compute shaders. It has limited support at the moment, but seems to fully support the PoseNet models used in this project. My testing shows that it is much more performant than using the CSharpBurst worker type for both the MobileNet and ResNet50 versions of the model. Although, it is still not as performant as using compute shaders when those are available.

---

Version 1

GitHub Repository