sasageyoorg / voca-blender

Blender add-on to implement VOCA neural network.

VOCA Blender Addon

📝 Table of Contents

• About
• Project Topology
• Installation and Usage
• Authors
• Acknowledgments

📋About

VOCA is a simple and generic speech-driven facial animation framework that works across a range of identities. This add-on integrates VOCA withing Blender and allows the user to:

• Run VOCA and synthesize a character animation given an speech signal. VOCA outputs a set meshes with .obj extension which must to be imported.
• Import the output meshes of VOCA and generate from them a single animated mesh.

For more details please see the scientific publication of the VOCA framework here.

The original VOCA framework repository can be found here here.

👩‍💻Installation and Usage

1. The add-on works with \Blender v2.92.0 (python 3.7) and requires some dependencies that can be installed directly from the preferences panel.
2. Download the latest release.
3. Import the downloaded .zip archive in Blender (Edit > Preferences > Add-ons > Install) and enable the add-on.
4. Install the dependencies by clicking the dedicated button
5. The add-on options are accessible in the 3D View side panel named "VOCA"
6. (optional) If you want to uninstall the addon, you can also uninstall the dependencies from the preferences panel.

To generate a new sequence of meshes:

1. Expand the 'Run VOCA Model' panel.
2. Select the right path for the mesh template (.ply) to animate, the audio file with the speech signal and the desired output directory.
3. Hit 'Run' and wait the end of the process.

To import the VOCA-generated meshes and generate the animated mesh:

1. Expand the 'Import Mesh' panel.
2. Select the path to the audio file and the output directory.
3. Hit 'Import' and wait.

The 'Dev' panel allows you to:

• Hide/unhide non-VOCA meshes.
• Remove all meshes from the scene.
• Remove all non-VOCA meshes from the scene.
• Edit sequences (flame parameters: head, pose, blink)

🗂 Project Topology

voca-blender/
├─ voca-addon/
│  ├─ audio/
│  │  ├─ sentence20.wav
│  │  └─ test_sentence.wav
│  ├─ flame/
│  │  └─ generic_model.pkl
│  ├─ model/
│  │  └─ gstep.model.*
│  ├─ smpl_webuser/
│  │  ├─ lbs.py
│  │  ├─ posemapper.py
│  │  ├─ serialization.py
│  │  └─ verts.py
│  ├─ template/
│  │  └─ FLAME_sample.ply
│  ├─ utils/
│  |  ├─ ctypesloader.py
│  │  ├─ audio_handler.py
│  │  ├─ edit_sequences.py
│  │  └─ inference.py
│  ├─ operators.py
│  ├─ panels.py
│  └─ __init__.py
├─ install-dependencies.command
├─ install-dependencies.sh
├─ LICENSE
└─ README.md

✍️ Authors

• Conti Edoardo @edoardo-conti
• Federici Lorenzo @lorenzo-federici
• Melnic Andrian @andrian-melnic

🎉 Acknowledgements

• Computer Graphics e Multimedia Class - Professor Primo Zingaretti

voca-blender's Issues

Installed on blender, but don't see the add on

Hi, thank you so much for creating this repo. I added release v1.0.1 using the "Install addons" interface on Blender, but I don't see the addon listed after installation. I tried restarting blender, and installing it multiple times, and installed 1.0.0 as well, but I don't see it. What's the best way to troubleshoot this situation? Thanks!

Install dependencies manually in blender 3.6

Hi! How could I manually install the dependencies?