Comments (5)
Thanks for that ^-^
Currently this project only deals with AU values, not the face points.
If you want to have access to the face points, you have 2 options, modify the GUI C++ code or (for cross-platform support) attach a ZeroMQ component to the Core code in C:
-
The ZeroMQ component added to OpenFace's GUI (limited to Windows) was done by my colleague, Huang. Currently it's only extracting the AU values, because that was necessary for this project.
You will have to rebuild OpenFace's GUI. The instructions are here: https://github.com/NumesSanguis/FACSvatar/tree/master/openface
To know what you should modify, search for "Huang" in the fileMainWindow.xaml.cs
. The code should be similar. You will need to convert the face points values to JSON. -
All calculations are in the Core code of OpenFace. Attaching a ZeroMQ component here would be ideal, because this parts works also on Linux (and Mac?). My colleague was more fluent in C++ and I'm more of a Python person, so we didn't go this way. So there is not much I can help you here with, but it would be great if you, or someone, could write a ZeroMQ network component her :)
May I ask you why you prefer the 68 face points over AU units?
from facsvatar.
Thank you for the help. I tried your methods and use my own cartoon model. Basically, the result is good. But I need more details of expression such as jawforward and so on. I find that this method can not drive some mouth animations. I want to use arkit 48 blendshapes and many of them can not derive only from AUs, so I want to get more information from original face points
from facsvatar.
The limitation is not the FACS itself. OpenFace only supports a subset of 18 AUs. In total there are around ~40, depending whether you count head rotations and such. More AUs can be seen here: https://imotions.com/blog/facial-action-coding-system/. FACSvatar has only 17 implemented, because the only proper open source toolkit available at that time was OpenFace. The goal for FACSvatar is to support all AUs, however.
FACSvatar relies on AUs, because they are software independent. For example, when you improve the AU tracker, no other code (in theory) would need adjustment.
Using tracking dots directly, however, would require to map them to every character to get accurate visualization. For example, you mention that OpenFace has 68, but ARkit 48 tracking points. Using ARkit would require changes over the whole framework if we relied on tracking points. Therefore, FACSvatar hasn't been using 2D/3D points directly.
If you plan to convert those 48 dots to AU values, I would be glad to help :)
from facsvatar.
Thanks. Yes you are right, the complete AUs can get good results, but the problem is just like you said, there is no complete AUs detection system right now. And ARKit has 48 blendshapes, not 48 points. So It can get wonderful result(you can find apple phone and unreal example https://www.youtube.com/watch?v=MfnNJaVCLM8 , this is really a perfect result!). I believe that using complete AUs can get similar results, but not for current time.I have thought of many others methods, such as curve matching, MPEG-4 facial system, and all of them need original points.
You said you could help convert dot to AUs, do you know any methods doing so? I think we need train some thing like expression feature to convert dots
from facsvatar.
zhaishengfu
Did you find a solution?
Thanks
from facsvatar.
Related Issues (20)
- How to enhance AUs from within Facsvatar? (Fix the notebook GUI) HOT 9
- Will there be a windows stand-alone for Blender? HOT 2
- Blender avatar is not receiving data. HOT 1
- Problem with face bones HOT 3
- openface_2.1.0_zeromq generated .csv data incompatibility with input_facsfromcsv HOT 5
- How to use FACSvatar system with 2 or more computers
- How to read csv output by OpenFaceOffline ? HOT 13
- Can I fix fps of OpenFaceoffline ? HOT 2
- Openface License Information. No Commercial Use Without Purchase, $18k Per Year HOT 20
- How avatar's setting in Make Human (FACS Human) HOT 2
- Facsdnnfacs for dynamic AUs? HOT 2
- How should I get python output? HOT 2
- Issue with changing bone.rotation_mode to 'XYZ' HOT 4
- Background rotation in visualizing pose angles HOT 6
- Use Google's project MediaPipe Face Mesh as FACS tracker HOT 18
- Character model replacement HOT 1
- How can I save the data captured by camera through OpenFaceOffline into a csv file? HOT 1
- FACSvatar with Blender 3.3 HOT 1
- No json file found for AU07 HOT 1
- Feature: Explore using Reallusion - free 3D Character Base as human model
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from facsvatar.