ayushanbhore / unreal-quest3-passthroughsample

A sample to create your own Unreal Passthrough Project for Meta Quest 3, Meta Quest Pro. Works with Meta Quest 2 but it is not recommended. Use it to learn Unreal Engine

Unreal-Quest3-PassthroughSample

This repository contains a sample Unreal Engine project demonstrating how to integrate and utilize the Passthrough, Depth API, and Mesh API for the latest Oculus Quest 3 device. This project serves as a reference for developers looking to create immersive Augmented Reality / Mixed Reality Experiences using Unreal Engine on the Oculus Quest 3 platform.

Table of Contents

• Overview
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Contributing
• License

Overview

This sample project showcases the integration of Passthrough, Depth API, and Mesh API provided by Oculus for the Meta Quest 3 VR headset within the Unreal Engine 5.1.1 environment. It demonstrates how to access and utilize real-time camera feed (passthrough), depth information, and mesh data to create AR experiences in the Unreal Engine environment for Oculus Quest 3 and also for Oculus Quest Pro

Getting Started

Prerequisites Before using this sample project, make sure you have the following prerequisites installed and set up:

Prerequisites

• Unreal Engine (version 5.1.1 or later)
• MetaXR Plugin for Unreal Engine (51.0 , 53.0 , 54.0)
• Meta Quest 3 Device or Meta Quest Pro Device
• Meta Developer Account

Installation

1. Clone this repository to your local machine:

git clone https://github.com/ayushanbhore/Unreal-Quest3-PassthroughSample.git

2. Open the project in Unreal Engine. Works Best with Unreal Engine 5.1.1 and later

3. Download the MetaXR Plugin for your Engine

-> Preferably - 54.0 Version

https://developer.oculus.com/downloads/package/unreal-engine-5-integration

4. It is recommended to place the MetaXR Plugin in the Marketplace Folder inside your Unreal Engine Directory

5. Launch Unreal Engine and Make Sure to Tick these Plugins in the Plugins Section

-> Please Uncheck Oculus VR Plugin or it may cause issues

7. Copy these settings in the Project Settings -> Plugins -> MetaXR -> Mobile Section of you Unreal Engine

Usage

Follow these steps to run the sample project and explore the integration of Passthrough, Depth API, and Mesh API:

1. Connect your Oculus Quest 3 device to your Development machine using a Link ( Link Cable Recommended over the Air Link).

2. Build and Deploy the project to your Oculus Quest 3 device using Unreal Engine.

3. Put on your Oculus Quest 3 headset and experience the augmented reality environment created using Passthrough.

License

This sample project is licensed under the MIT License. Feel free to modify and distribute it according to the terms of the license.

unreal-quest3-passthroughsample's Issues

about passthrough feature

It demonstrates how to access and utilize real-time camera feed (passthrough), depth information, and mesh data to create AR experiences in the Unreal Engine environment for Oculus Quest 3.

Thank you for the interesting project. I saw the text above in your README, but I thought it wasn't possible to access the camera feed on Quest 3. Is there a way to do so?

Thanks.

App Crash at launch

Hey @Ayushanbhore,

just tested the app on my Meta Quest 3 and i'm having an issue -> The app crash at launch

I don't know if the app work only for Quest Pro

Did you try it on Quest 3 ?

Cannot Load Map

Crash on packaged game for Oculus Quest 3

Hello,

I have just followed the readme guide, and compiled the project, using epic distributed unreal engine 5.1.1, and my project crashes while trying to load the OpenXR Plugin, wich is already activated in the project, any idea?

Here is the log:

Log file open, 10/24/23 06:54:37
LogInit: Display: Running engine for game: PassthroughQuest3
LogCore: Initializing trace...
LogCore: Finished trace initialization.
LogCsvProfiler: Display: Metadata set : platform="Android"
LogCsvProfiler: Display: Metadata set : config="Development"
LogCsvProfiler: Display: Metadata set : buildversion="++UE5+Release-5.1-CL-23901901"
LogCsvProfiler: Display: Metadata set : engineversion="5.1.1-23901901+++UE5+Release-5.1"
LogCsvProfiler: Display: Metadata set : commandline="" ../../../PassthroughQuest3/PassthroughQuest3.uproject""
LogCsvProfiler: Display: Metadata set : os="Android 12"
LogCsvProfiler: Display: Metadata set : cpu="Oculus|Quest|eureka"
LogCsvProfiler: Display: Metadata set : pgoenabled="0"
LogCsvProfiler: Display: Metadata set : loginid="499C218ABB53493CA7D4855097579C8D"
LogCsvProfiler: Display: Metadata set : asan="0"
LogCsvProfiler: Display: Metadata set : deviceid="0db6be8ac9a77cda"
LogCsvProfiler: Display: Metadata set : llm="0"
LogInit: Display: Project file not found: ../../../PassthroughQuest3/PassthroughQuest3.uproject
LogInit: Display: Attempting to find via project info helper.
LogUProjectInfo: Found projects:
LogTemp: 6 cores and 4 assignable cores
LogIoDispatcher: Display: Reading toc: ../../../PassthroughQuest3/Content/Paks/global.utoc
LogIoDispatcher: Display: Toc signature hash: 0000000000000000000000000000000000000000
LogIoDispatcher: Display: Mounting container '../../../PassthroughQuest3/Content/Paks/global.utoc' in location slot 0
LogPakFile: Display: Initialized I/O dispatcher file backend. Mounted the global container: ../../../PassthroughQuest3/Content/Paks/global.utoc
LogPakFile: Display: Found Pak file PassthroughQuest3/Content/Paks/PassthroughQuest3-Android_ASTC.pak attempting to mount.
LogPakFile: Display: Mounting pak file PassthroughQuest3/Content/Paks/PassthroughQuest3-Android_ASTC.pak.
LogIoDispatcher: Display: Reading toc: PassthroughQuest3/Content/Paks/PassthroughQuest3-Android_ASTC.utoc
LogIoDispatcher: Display: Toc signature hash: 0000000000000000000000000000000000000000
LogIoDispatcher: Display: Mounting container 'PassthroughQuest3/Content/Paks/PassthroughQuest3-Android_ASTC.utoc' in location slot 0
LogPakFile: Display: Mounted IoStore container "PassthroughQuest3/Content/Paks/PassthroughQuest3-Android_ASTC.utoc"
LogPakFile: Display: Mounted Pak file 'PassthroughQuest3/Content/Paks/PassthroughQuest3-Android_ASTC.pak', mount point: '../../../'
LogStats: Stats thread started at 0.449867
LogAssetRegistry: Premade AssetRegistry loaded from '../../../PassthroughQuest3/AssetRegistry.bin'
LogAndroid: Case APP_CMD_WINDOW_RESIZED
LogAndroid: Case APP_CMD_WINDOW_REDRAW_NEEDED
LogICUInternationalization: ICU TimeZone Detection - Raw Offset: +0:00, Platform Override: ''
LogAndroid: Mobile HDR: YES
LogAndroidWindowUtils: Setting Width=1000 and Height=624 (requested scale = 1.000000)
LogInit: Session CrashGUID >====================================================
Session CrashGUID > UECC-Android-65AE93B52C0742B7995D3017B605DA4E
Session CrashGUID >====================================================
LogAndroid: Vulkan library detected, checking for available driver
LogAndroid: VulkanRHI is available, Vulkan capable device detected.
LogPluginManager: Error: Unable to load plugin 'OpenXR'. Aborting.
LogAndroid: Error: === Critical error: ===
LogAndroid: Error:
LogAndroid: Error: Assertion failed: false [File:D:/build/++UE5/Sync/Engine/Source/Runtime/Launch/Private/Android/LaunchAndroid.cpp] [Line: 529]
LogAndroid: Error: Engine Preinit Failed
LogAndroid: Error: [Callstack] 0x0000007105978B38 libUnreal.so(0x000000000A2ADB38)!AndroidMain(android_app*) []
LogAndroid: Error: [Callstack] 0x0000007105987448 libUnreal.so(0x000000000A2BC448)!android_main() []
LogAndroid: Error: [Callstack] 0x00000071059BE380 libUnreal.so(0x000000000A2F3380) []
LogAndroid: Error: [Callstack] 0x00000072277A9DB4 libc.so(0x00000000000DFDB4) []
LogAndroid: Error: [Callstack] 0x00000072277563E0 libc.so(0x000000000008C3E0) []
LogAndroid: Error:
LogAndroid: Error:
LogAndroid: Error:
LogAndroid: Error:
LogAndroid: FAndroidMisc::RequestExit(1)