yonet / mixedreality-uxtools-unreal

UX tools and components for developing Mixed Reality applications in UE4.

Introduction

UX Tools for Unreal Engine is a UE game plugin with code, blueprints and example assets created to help you add in features commonly needed when you're developing UX for mixed reality applications. The project is still in early development (it provides a small set of features and breaking changes are to be expected) but the current features are complete and robust enough to use in your own projects.

Note
Only HoloLens 2 development is supported at the moment.

Features:

• Input simulation - Simulates articulated hands and head pose that you can use in-editor. This is great for improving development iteration times.
• Hand interaction actor - Used to interact with our UX components with articulated hands.
• Pressable button component - Low level component used to drive the state of button blueprints. Example button blueprints are provided.
• Pinch Slider - Allows the user to continuously change a value by moving the slider thumb along the track.
• Bounds Control - Allows the user to change the position, rotation, and size of an actor, using affordances.
• Manipulator component - Allows moving, rotating and scaling an actor with either one or two hand manipulation.
• Follow behaviour component - Keeps an actor or component within sight of another component, usually the camera.
• Hand constraint component - Keeps the actor position and rotation aligned with a hand while avoiding overlaping with it.
• Palm up constraint component - A Hand constraint specialization that activates only when the palm is facing the player.
• Far beam component - Allows the user to visualise elements in the scene they can interact with from afar.
• Graphics documentation - Breakdown of shaders, materials, and graphics techniques used to render UX components.
• The plugin also contains a handful of utilities that augment the Unreal Engine editor.

Getting started

First of all, make sure you have UE 4.25.3 or later. You can download it here. Earlier versions are not supported.

Prebuilt plugin

If you just want to add UXT to your game project, the quickest thing to do is to use the packaged plugin provided in the release page:

1. Obtain the packaged plugin zip from the latest release page (e.g. UXTools.0.9.0.zip).
2. Unzip the file directly into your project's Plugins folder. The Plugins folder should be located at the root of your project, where the .uproject file is. Create it if it doesn't exist.
3. Make sure your game project is a code one, as opposed to blueprint-only, if you are planning to package it for HoloLens. Otherwise UE will fail to package it because it can't build the plugin sources.
4. Open your project and enable the UX Tools plugin in the plugins menu.

You should now have access to all of the plugin features. The first thing you want to do is probably add a hand interaction actor per hand to your map or pawn so you can use your hands to drive the controls and behaviors provided in UXT. You can find a complete setup guide in the Getting Started tutorial series.

Example maps

If you want to explore the different UXT features or want a reference for how to use them we recommend having a look at the example maps contained in the UX Tools Game (/UXToolsGame) in this repository. For that you should:

1. Clone this repository.
2. Checkout public/0.9.x.
   • Bear in mind that this branch is alive. It's not a release, and will be updated regularly with potentially breaking changes. There will be a release tag (e.g. release/0.9.0) marked as such in GitHub.

You can now open the UX Tools Game (/UXToolsGame) and explore individual example maps or open the Loader level to access some of the examples from a centralized hub.

Packaged UX Tools game

We also provide the UX Tools game pre-packaged for HoloLens 2 so you can try out the main UXT features directly on device easily. To use it:

1. Obtain the packaged game from the latest release page (e.g. UXTGame-HoloLens.0.9.0.zip) and unzip it to a local directory.
2. Install it in the device via the Device Portal.

Documentation

The latest version of the documentation can be found here.

Feedback and contributions

Due to the early stage of the project and the likelihood of internal refactors, we are not in a position to accept external contributions through pull requests at this time. However, contributions and feedback in the shape of bug reports, suggestions and feature requests are always welcome!