dfghiatus / questpro4neos

Meta Quest Pro eye and face tracking for NeosVR

QuestPro4Neos

A NeosModLoader mod that brings the Quest Pro's eye tracking and natural expressions to Neos VR avatars.

Related issues on the Neos Github:

1. Neos-Metaverse/NeosPublic#1140
2. Neos-Metaverse/NeosPublic#3770

Usage

1. Install NeosModLoader.
2. Download the latest release of this mod and place it in your NeosVR install folder. This folder should be at C:\Program Files (x86)\Steam\steamapps\common\NeosVR\ for a default install. Extract the archive, ensuring that OSCCore is present and QuestProModule is present in nml_mods. his folder should be at C:\Program Files (x86)\Steam\steamapps\common\NeosVR\nml_mods for a default install, or you can create this folder if it is missing.
3. Install ALVR v20 and run it.
4. Start the game.

If you want to verify that the mod is working you can check your Neos logs, or create an EmptyObject with an AvatarRawEyeData/AvatarRawMouthData Component (Found under Users -> Common Avatar System -> Face -> AvatarRawEyeData/AvatarRawMouthData).

A big thanks to Geenz and Earthmark for their contributions and testing this mod, not owning the headset myself this would not be possible without them. Check out Geenz's fork here

questpro4neos's Issues

Few expressions don't work via a launch method on ALXR.

The method I'm currently running is by running the ALXR app with --no-alvr-server --no-bindings args using Link/AirLink

• Eye expressions for looking left and right do not work, they move slightly, but not much
• Grinning don't work
• Jaw left and right are inverted
• LipMoveLeft's don't work

All these above are what I have for now, I will post anything else in the future~

Can I use this mod with only alxr without alvr on latest version?

Currently I can't use alvr, so i want to use only airlink and alxr. Expression works in v0.0.1, but not in 2.0.0. I would like to know which version of alxr was developed using.

Update for new ALVR face tracking protocol

Just a heads up, I made a new stable release integrating a new UDP protocol for face tracking (no OSC).
For a sample consumer implementation, look at our VRCFT module.

On ALVR v20, Face is tracked, Eyelids are tracked, eye position is not.

On today's release, everything seems to pass through correct but though eyelids and face are tracked correct, eye rotation data is not coming through. I checked the RawData component to confirm. I don't know if it is how this Mod is processing the input or how ALVR v20 is sending it.

I also experience issues with face tracking that I didn't before like Smiles and Grins not being recognized unless my mouth is slightly open.

Thanks!

OscCore.dll is required for the facial tracking to work.

When I originally got this working over ALVR with Earthmark, he had me put the OscCore.dll in my mods folder. I was cleaning out my mods and saw that this file wasn't included with the newer version of the mod, but facial tracking did not work until I put it back.