microsoft / mixedreality-worldlockingtools-unity

Unity tools to provide a stable coordinate system anchored to the physical world.

title	description	author	ms.author	ms.prod	ms.date	ms.localizationpriority	keywords
Welcome!	Welcome landing page for World Locking Tools for Unity.	fast-slow-still	mafinc	mixed-reality	10/06/2021	high	Unity, HoloLens, HoloLens 2, Augmented Reality, Mixed Reality, ARCore, ARKit, development, MRTK

Welcome!

World Locking Tools for Unity provides a stable and reliable world-locked coordinate system, binding the virtual/holographic world to the physical world.

World Locking Tools take the burden of world-locking holograms off of the developer, as well as enabling scenarios that were previously unreachable. It currently supports the HoloLens family of devices via Unity's built-in VR support. Initial support for other platforms via Unity's new XR SDK plugin system is also available.

World Locking Tools locks the entire holograph space of your application to the physical world. A hologram put in position relative to physical world features will stay fixed relative to those features, as well as remaining fixed relative to other holograms.

World Locking Tools scale naturally with both the complexity and size of the scene. Large models, large collections of models, and multi-room environments are all handled gracefully.

Published documentation

For the most up-to-date documentation, see the World Locking Tools for Unity documentation portal.

Getting started

Dive into the full documentation, beginning with the organization of the documentation itself, from the Guides section.

Or a quick overview of key concepts behind World Locking Tools is contained in this FAQ.

Or skip straight into introduction of key concepts, leading to further details of concepts and solutions, beginning in the Concepts section.

A number of samples are contained in a sibling repo, World Locking Tools Samples. Be sure to have a look there for an instructive example. And if you don't find what you are looking for there, you are encouraged to submit a suggestion or even your own sample.

Easy and efficient

World Locking Tools is both powerful and easy to use. Furthermore, it strives to supply only as much as a specific application requires. Any World Locking Tools feature not required by an application will incur no cost in resources or performance.

See also

These ideas are further explored in the conceptual guide. To complement the theoretical description, pragmatic samples are provided, along with how-to articles to walk through the steps involved in building applications using the World Locking Tools' stable and reliable coordinate system.

• Conceptual guide
• How-to articles
• Samples

Further, reference materials for interfacing through script with World Locking Tools is included in the API Documentation.

Build Status

Branch	CI Status	Docs Status
master

Useful resources on Microsoft Windows Dev Center

Academy	Design	Development	Community
See code examples. Do a coding tutorial. Watch guest lectures.	Get design guides. Build user interface. Learn interactions and input.	Get development guides. Learn the technology. Understand the science.	Join open source projects. Ask questions on forums. Attend events and meetups.

How to Contribute

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

View the How To Contribute page for the most up to date instructions on contributing to World Locking Tools for Unity.

Microsoft Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct.

Resources:

• Microsoft Open Source Code of Conduct
• Microsoft Code of Conduct FAQ
• Contact [email protected] with questions or concerns.

mixedreality-worldlockingtools-unity's Issues

Starting with WorldLockingContext disabled doesn't start disabled

1. Set enabled to false in WorldLockingContext in editor.
2. Start application and move around
3. From script, set world locking manager to enabled
4. Note that a trail of anchors has been left, suggesting the WorldLockingManager wasn't disabled after all.

Add trouble shooting guide

Augment Unity's sometimes cryptic error messages with some guidance of the form "if you see this, then you should check that."

Some "common" ones might include:

• Didn't install NuGet for Unity
• Didn't install Frozen World Engine NuGet package
• Missing v142 (built for VS2019 but loaded into VS2017)
• Poor frame rate (check UnityPlayer.log for unhandled exceptions)

[Space Pin] Dynamically Creating SpacePinOrientables via Editor Script is Hard

Overview
An Unity Project can't easily create SpacePinOrientable dynamically via Editor Script, because there is no way to easily set the serialized orienter field. There's a way to set the private iorienter, but this doesn't get serialized.

Description
I'm attempting to create an Editor Script that dynamically creates a tape measure in my scene. This utility script places a SpacePinOrientables every meter. I'd like to set the orienter field via the Editor script, dynamically, so I don't have to do this at runtime. But there is no way to do this today. There's the SetOrienter() method, but this only sets the private iorienter field, and this doesn't get serialized field. Since it's not serialized, the value is lost when app is played.

Suggest Fix
Update SetOrienter() to set both the iorienter and orienter fields.

Legacy XR removed in Unity 2020.2

Importing WLT into Unity 2020.2 won't compile due to the removal of the Unity.XR.WSA.Persistence namespace (WorldAnchor and WorldAnchorStore removed).

Removing the problematic classes/references results in WLT working as expected.

What is an appropriate way to reference this repo for Unity 2020.2+ (branch/fork/PR)?

Cheers

Please add video showing Space Pins and Ray Pins in use

MRTK Eye Gaze Provider broken after tracking lost

We are using a couple of applications taking advantage of the HoloLens 2 eye gazing. We noticed that the eye gaze provider would sometimes stop correctly recording where the eyes were looking.
We found two issues:
The first is around the World Locking Context Auto Load feature. It seemed that on subsequent runs of the app, the eye gazing wouldn't function correctly.
The second was that after losing tracking and relocating, the eye tracking would never recover and would be misaligned.

We tracked the problem down to the way MRTK gets the position and direction of the latest eye gaze in it's GazeProvider. We have created our own GazeProvider that takes these values and converts them from Spongy to Frozen space. So far this appears to have fixed this issue.

Spatial mapping appears in the incorrect location

I have an issue where the anchors appear stable, but the spatial mapping exhibits some strange behaviour. This occurs in the WorldLockingBasic example (adapted to use the XRSDK) as well as an internal sample.

The spatial mapping appears stable for a few seconds, then begins slowly drifting, then extremely drifting. The anchors remain stable the entire time. Here's a video demonstrating this: https://drive.google.com/file/d/1-U0Bj8Gn3og2gPxtg_2PWbb7MVLvRq_E/view?usp=sharing

I haven't found much discussion of spatial mapping aside from FrozenSpatialMapping, where it states:

This is unnecessary when using MRTK's spatial mapping, which provides this and other enhancements over the native spatial mapping.

In device portal, the spatial mesh appears as normal.

Steps to reproduce:

1. Clone latest master 8333fce
2. Open with Unity 2019.4.16f1 (performs project upgrade)
3. Import XR Plugin Management (3.2.17) via UPM
4. Import Windows XR Plugin (2.4.1) via UPM and enable for UWP in XR Plugin Management
5. Open scene WorldLocking.Examples/Scenes/WorldLockingBasic
6. Change the MRTK profile to DefaultXRSDKConfigurationProfile
7. Change the WorldLockingManager ARSubsystem to XRSDK
8. Build with scene WorldLocking.Examples/Scenes/WorldLockingBasic configured for HoloLens>ARM64
9. Deploy to Device with Visual Studio (Release ARM64)

(Build runs correctly)

10. Replace MRTK 2.4.0 with MRTK 2.5.1

(Build exhibits strange behaviour described above)

Subsequently I suspect this is related to the MRTK version upgrade.

Document the diagnostics visualizations

There's a lot of information available at a glance from the diagnostics visualization (e.g. color coding of anchor disks), but it is only useful internally without documenting how to read the various signals.

Unity build fails due to too long path hitting Windows path limtis

I cloned the repo to:
D:\Git\OpenSource\Microsoft\MixedReality-WorldLockingTools-Unity

Tried to build for UWP which failed building with this unclear error message: DirectoryNotFoundException: Could not find a part of the path "D:\Git\OpenSource\Microsoft\MixedReality-WorldLockingTools-Unity\Assets\MRTK\MixedRealityToolkit.Providers\WindowsMixedReality\DotNetAdapter\Plugins\net46\Microsoft.Windows.MixedReality.DotNetWinRT\Editor\Microsoft.Windows.MixedReality.DotNetWinRT.Editor.asmdef"

Looking at the long path I guessed it could be hitting the Windows path length limit, therefore I moved the project to the root on my D: disk and there it built just fine.

Not sure what can be done here as it seems to come from the MRTK but might be worth to look into. My typical subfolder I tried to use initially is not extraordinary long and I would expect others to run into the same issue.

Rotation irrelevant when creating than one SpacePin?

Just a quick question for verification:

We noticed we were feeding correct positions but completely wrong rotations when creating new SpacePinPackages.
(Positions coming from scanned qr codes.)
But the positioning of the holograms still worked well.

Does that mean that the rotations of the virtualObject passed to SpacePinPackage.Create() don't matter anymore once there's more than one SpacePinPackage?
That would make sense because rotation inferred from two points will be more accurate than rotation from a scanned QR code.

Also it normally takes 3 points to define the rotation of a coordinate system. But I assume you only need two points because you always keep the Y-axis vertical?

Spatial meshes wrong using XR SDK

When using XR SDK/Windows XR Plugin and MRTK, the spatial meshes display in the wrong place.

Raycasts to the environment are correspondingly wrong.

See the following MRTK issue for more details:
microsoft/MixedRealityToolkit-Unity#9810

PhysicsBeamSample lineRenderer Is Null

Running the OMPTest scene:

NullReferenceException: Object reference not set to an instance of an object
Microsoft.MixedReality.WorldLocking.Examples.PhysicsBeamSample.FinishCurrentAction () (at Assets/MRTK/Samples/WLT Samples/1.3.0/World Locking Examples/Scripts/PhysicsBeamSample.cs:699)
Microsoft.MixedReality.WorldLocking.Examples.PhysicsBeamSample.EnterIdleMode () (at Assets/MRTK/Samples/WLT Samples/1.3.0/World Locking Examples/Scripts/PhysicsBeamSample.cs:605)
UnityEngine.Events.InvokableCall.Invoke () (at <028e4d71153d4ed5ac6bee0dfc08aa3b>:0)
UnityEngine.Events.UnityEvent.Invoke () (at <028e4d71153d4ed5ac6bee0dfc08aa3b>:0)
Microsoft.MixedReality.Toolkit.UI.Interactable.SendOnClick (Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointer pointer) (at Library/PackageCache/com.microsoft.mixedreality.toolkit.foundation@7a88183bb2f3-1626820770943/SDK/Features/UX/Interactable/Scripts/Interactable.cs:1254)
Microsoft.MixedReality.Toolkit.UI.Interactable.TriggerOnClick (System.Boolean force) (at Library/PackageCache/com.microsoft.mixedreality.toolkit.foundation@7a88183bb2f3-1626820770943/SDK/Features/UX/Interactable/Scripts/Interactable.cs:1244)
Microsoft.MixedReality.Toolkit.UI.InteractableToggleCollection.SetSelection (System.Int32 index, System.Boolean force, System.Boolean fireOnClick) (at Library/PackageCache/com.microsoft.mixedreality.toolkit.foundation@7a88183bb2f3-1626820770943/SDK/Features/UX/Interactable/Scripts/InteractableToggleCollection.cs:110)
Microsoft.MixedReality.Toolkit.UI.InteractableToggleCollection.Start () (at Library/PackageCache/com.microsoft.mixedreality.toolkit.foundation@7a88183bb2f3-1626820770943/SDK/Features/UX/Interactable/Scripts/InteractableToggleCollection.cs:81)

Command line build of project containing WLT missing DLL

Hi,
I'm attempting to set up CI for a project and am running into some issues.

I've followed the advice in NugetForUnity to restore packages "%unitydir%\Editor\Unity.exe" -quit -batchmode -projectPath %~dp0 -executeMethod NugetForUnity.NugetHelper.Restore

However, I am still left with the following block of errors. It seems the project won't compile and hence doesn't attempt to run the -executeMethod.

I couldn't see any references to nuget restore in the WLT CI pipeline.

Opening the project in the editor restores the nuget packages and runs as normal - this only occurs on a clean pull.

Steps to reproduce

1. git clone https://github.com/microsoft/MixedReality-WorldLockingTools-Unity.git
2. cd MixedReality-WorldLockingTools-Unity
3. "C:\Program Files\Unity\Hub\Editor\2020.3.0f1\Editor\Unity.exe" -quit -batchmode -projectPath ".\" -executeMethod NugetForUnity.NugetHelper.Restore -logFile log.txt
4. See errors in log.txt

Has anyone encountered this issue?

- Finished script compilation in 5.335927 seconds
Assets\WorldLocking\WorldLocking.Engine\IPlugin.cs(12,37): error CS0234: The type or namespace name 'FrozenWorld' does not exist in the namespace 'Microsoft.MixedReality' (are you missing an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(10,37): error CS0234: The type or namespace name 'FrozenWorld' does not exist in the namespace 'Microsoft.MixedReality' (are you missing an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\PluginNoop.cs(13,37): error CS0234: The type or namespace name 'FrozenWorld' does not exist in the namespace 'Microsoft.MixedReality' (are you missing an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\IPlugin.cs(35,19): error CS0103: The name 'INVALID_ANCHOR_ID' does not exist in the current context

Assets\WorldLocking\WorldLocking.Engine\IPlugin.cs(24,19): error CS0103: The name 'INVALID_ANCHOR_ID' does not exist in the current context

Assets\WorldLocking\WorldLocking.Engine\IPlugin.cs(25,22): error CS0103: The name 'INVALID_ANCHOR_ID' does not exist in the current context

Assets\WorldLocking\WorldLocking.Engine\IPlugin.cs(36,19): error CS0103: The name 'UNKNOWN_ANCHOR_ID' does not exist in the current context

Assets\WorldLocking\WorldLocking.Engine\IPlugin.cs(26,19): error CS0103: The name 'UNKNOWN_ANCHOR_ID' does not exist in the current context

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(30,17): error CS0246: The type or namespace name 'FrozenWorld_Vector' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(31,17): error CS0246: The type or namespace name 'FrozenWorld_Quaternion' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(32,17): error CS0246: The type or namespace name 'FrozenWorld_Transform' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(34,30): error CS0246: The type or namespace name 'FrozenWorld_Vector' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(35,33): error CS0246: The type or namespace name 'FrozenWorld_Quaternion' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(36,27): error CS0246: The type or namespace name 'FrozenWorld_Transform' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(113,22): error CS0246: The type or namespace name 'FrozenWorld_Metrics' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(582,21): error CS0246: The type or namespace name 'FrozenWorld_Serialize_Stream' could not be found (are you missing a using directive or an assembly reference?)

Assets\WorldLocking\WorldLocking.Engine\Plugin.cs(775,21): error CS0246: The type or namespace name 'FrozenWorld_Deserialize_Stream' could not be found (are you missing a using directive or an assembly reference?)

Some scripts have compilation errors which may prevent obsolete API usages to get updated. Obsolete API updating will continue automatically after these errors get fixed.

AssetDatabase: script compilation time: 6.118637s
Scripts have compiler errors.

Exiting without the bug reporter. Application will terminate with return code 1

Need documentation guidance for issue creation and tracking

Documentation should make clear:

• What makes a good, actionable bug report.
• How to use labels to differentiate between bugs, suggestions, and feature requests.
• Other positive use of labels.

Set full pose on SpacePinOrientable

Currently, you can set a full pose on a SpacePin, or set a position on a SpacePinOrientable and have the implicit rotation computed. But you can't explicitly set the full pose (position+rotation) on a SpacePinOrientable.

This is only a factor on the first SpacePinOrientable, because with only a single pin there is no implied rotation. It would be preferable to be able to set the full pose for the first one, but then as more are added, to start using the more reliable implicit rotations.

For example, if using QR codes as markers, they come in with both position and rotation, however the rotation is fairly inaccurate. Still, it's more accurate than just an identity transform, and would get the space into rough alignment until the next QR code is seen.

Likewise, if setting them manually, dragging and rotating the first marker into both correct position and (approximately) correct rotation would put the other virtual markers near their corresponding physical reference points, simplifying dragging the remaining markers into position.

Multiple loads cause duplicate SpacePins

When a SpacePin is present and active, and its state saved to the AlignmentManager's database, and then WLT is Loaded again (including AlignmentManager), then that SpacePin will be duplicated in the active SpacePins, causing a degenerate triangulation and ensuing chaos.

When SpacePins are active, a Load should update a SpacePin if necessary, but in no way invalidate or duplicate it.

Multi-User Experiences

Hi all,

Thanks for this great package! It works out of the box in an existing project and it's quite magical.

I would love to be able to use this to synchronise the virtual environment across multiple Hololenses. Would this be possible?

I guess the easiest thing to so is to somehow distribute the Saved Frozen State to all users at the start of a session. Can I somehow access the raw/serialized data that WorldLockingManager.GetInstance().Save(); saves in order to accomplish this?

Cheers,
Bastiaan

Package version compatibility?

I somehow can't get the packages aligned to make the latest version work on Android and Hololens.

Unity 2019.4.15f1c1

First of all following packages match:

• ARSubsystems: 2.1.3
• AR Foundation: 2.1.8
• AR Core XR Plugin: 2.1.11
• Windows XR Plugin: 2.4.1

However this doesn't match with WLT because AnchorManagerXR.cs requires ARSubsystems >= 3.0.1 because the renaming of ReferencePoints to Anchors:
https://docs.unity3d.com/Packages/[email protected]/changelog/CHANGELOG.html

So trying with following combination of packages:

• ARSubsystems: 3.1.3
• AR Foundation: 3.1.3
• AR Core XR Plugin: 3.1.4
• Windows XR Plugin: 2.4.1

But that leads to following errors:

Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(15,28): error CS0246: The type or namespace name 'IProvider' could not be found (are you missing a using directive or an assembly reference?)
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(15,38): error CS0508: 'WindowsMRSessionSubsystem.CreateProvider()': return type must be 'XRSessionSubsystem.Provider' to match overridden member 'XRSessionSubsystem.CreateProvider()'
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(20,15): warning CS0108: 'WindowsMRSessionSubsystem.Provider' hides inherited member 'XRSessionSubsystem.Provider'. Use the new keyword if hiding was intended.
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(118,28): error CS0246: The type or namespace name 'IProvider' could not be found (are you missing a using directive or an assembly reference?)
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(20,26): error CS0246: The type or namespace name 'IProvider' could not be found (are you missing a using directive or an assembly reference?)
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(118,38): error CS0508: 'WindowsMRReferencePointSubsystem.CreateProvider()': return type must be 'XRAnchorSubsystem.Provider' to match overridden member 'XRAnchorSubsystem.CreateProvider()'
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(123,15): warning CS0108: 'WindowsMRReferencePointSubsystem.Provider' hides inherited member 'XRAnchorSubsystem.Provider'. Use the new keyword if hiding was intended.
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(27,58): error CS0115: 'WindowsMRSessionSubsystem.Provider.GetAvailabilityAsync()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(34,43): error CS0115: 'WindowsMRSessionSubsystem.Provider.trackingState': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(42,47): error CS0115: 'WindowsMRSessionSubsystem.Provider.notTrackingReason': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(50,34): error CS0115: 'WindowsMRSessionSubsystem.Provider.Destroy()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(54,34): error CS0115: 'WindowsMRSessionSubsystem.Provider.Pause()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(59,34): error CS0115: 'WindowsMRSessionSubsystem.Provider.Resume()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\Session\WindowsMRSessionSubsystem.cs(64,34): error CS0115: 'WindowsMRSessionSubsystem.Provider.Reset()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(123,26): error CS0246: The type or namespace name 'IProvider' could not be found (are you missing a using directive or an assembly reference?)
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(125,34): error CS0115: 'WindowsMRReferencePointSubsystem.Provider.Start()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(130,34): error CS0115: 'WindowsMRReferencePointSubsystem.Provider.Stop()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(135,34): error CS0115: 'WindowsMRReferencePointSubsystem.Provider.Destroy()': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(140,63): error CS0115: 'WindowsMRReferencePointSubsystem.Provider.GetChanges(XRAnchor, Allocator)': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(181,34): error CS0115: 'WindowsMRReferencePointSubsystem.Provider.TryAddReferencePoint(Pose, out XRAnchor)': no suitable method found to override
Library\PackageCache\[email protected]\Runtime\ReferencePoint\WindowsMRReferencePointSubsystem.cs(188,34): error CS0115: 'WindowsMRReferencePointSubsystem.Provider.TryRemoveReferencePoint(TrackableId)': no suitable method found to override

It looks Windows XR Plugin 2.4.1 doesn't match with ARSubsystems 3.1.3, but there's no higher version.
Am I missing something?
Which combination of packages did you use succesfully?

Overriding the pose of the Hololens camera (question)

I am interested in understanding how WorldLockingTools overrides the normal Hololens tracking behavior in Unity. My understanding from a quick look at your documentation and code is that you override the pose applied to the main camera by using a parent transform. The pose you ultimately want to use is the result of the calculations done in FrozenWorld, but to do that you also need to give it the original camera pose from the camera.

I have previously attempted to override the Hololens' automatic management of the Unity camera to apply a pose from an different tracking system using this sort of strategy. I found some issues with this sort of approach, and am interested in understanding if this is something you have observed or not, and if you were able to overcome it or not.

I found that when the head was stationary this method worked very well, but during head movement (particularly head rotation) static world locked virtual objects would visibly judder.
Ultimately my understanding was that the Hololens is doing some additional late stage re-projection, so the values you can access in a Unity script for the camera's pose are not really the final values, so calculations in a normal script can't fully negate the pose that is being applied to the camera as there could be some delta movement late on that you don't get to know about. The comments in AnchorManager.cs state some uncertainty about if the pose from Camera.main.transform.local??? or InputTracking.GetLocal???(XRNode.Head) should be used, so I suspect you have had similar concerns? I believe I tried both API calls in the past and felt that neither made it possible to fully negate the original head pose. Could this be an issue for WorldLockingTools?

Release 1.2.4 breaks spatial mesh on XR SDK

This is a regression from upgrading to MRTK v2.5.4. Same fix needs to be applied.

Fix will be in MRTK from v2.6.0.

Extend SpacePinMeshVisualizer to support multiple AlignSubtrees

Tools has dependency on MRTK

Several of the prefabs in WorldLocking.Tools have dependencies on MRTK. Either these dependencies need to be broken, or the prefabs need to move up to Examples. Tools can have no dependency on anything but Core/Engine.

The dependencies are:

FrameVisual.prefab

• OriginMarker uses polysurface7 mesh
• OriginText uses Billboard script

FrozenAnchorVisual.prefab

• Text uses Billboard script

SpongyAnchorVisual.prefab

• Ring uses TeleportPadMid mesh
• Text uses Billboard script

Samples Documentation Suggestion

The documentation overall is some of the best I've seen from Microsoft. Well written, engaging. I like how concepts are explained and a valiant attempt is made to explain when and when not to use WLT. There are links on critical references which is almost always lacking.
The samples need to better explain what should happen when running, better yet, a short video of them in use. Also the EXACT configuration should be shown in screen shots. When using a new technology it is very important to see the intended outcome to allow the developer to gauge his/her success or to assist in determining why there may be a difference.
In my particular case, with the Physics sample deployed to HoloLens 2. There are many issues, notably the HUD is not in view, just a little of it at the bottom of the view port. I could never find any of the anchors, while in the Unity player I cloud locate them. This is the type of mismatch that drives we developers completely insane.
Finally, the reason for publishing the exact configuration, and I know how time consuming it is because I have done it myself, is the ever changing Unity platform and its universe of add-ons not to mention WLT and MRTK continuous development to boot. As developers (and this too was noted in the documentation) we are forced by our application requirements to use versions of Unity probably not the same configuration as the one used to develop the samples. This is why we need the complete set of configurations used so we can translate those to our applications.
Again, thank you for the good job both in code and in the documentation supporting. If somebody could throw together some short video (voice over is a plus) of outcomes that would be super cool!
Thank you.

Meta file mixup when using MRFeatureTool

Errors of the form:

GUID [616f81a0fe106044cadb125f5a8f1f6f] for asset 'Packages/com.microsoft.mixedreality.worldlockingsamples/NOTICE.md.meta' conflicts with:
'Packages/com.microsoft.mixedreality.worldlockingtools/NOTICE.md.meta'
We can't assign a new GUID because the asset is in an immutable folder. The asset will be ignored.

Looks like the stock .md files for UPM have identical .meta files between WLTools and WLSamples

Correct way to move a spacepin in space?

What's the correct way to move a qrcode connected to a spacepin?
I guess the correct way is:

1 Release the spacePinPackage:

spacePinPackage.Release();

(after this the spacePinPackage object can be discarded)

2 Create a new spacePinPackage with the new position of the QR code:

QRSpacePinGroup.SpacePinPackage.Create(qrSpacePinGroup, gameObject.transform);

Is this the correct approach?

Sharing holograms precisely in a large space

Hi ! I'm using the Hololens 2 with World Locking Tools (and MRTK, Unity 2019.4.9).

The goal : I’m developing an app where, in a large space (ideally 100m * 100m but if it's too big it can be 25m*25m), the users may position, create, update different holograms. They need to keep the same position between different sessions and different hololens, users. Theses holograms are saved in a JSON to be loaded for a future session. If I put this JSON to another hololens, the new user will se the same holograms.

To do this, in the app you can scan multiples QR codes to align the space to the physical world. I’m using Space Pins from WLT to align a part of my scene containing my holograms to the markers (like in the scene QRSubScene in https://microsoft.github.io/MixedReality-WorldLockingTools-Samples/Advanced/QRSpacePins/QRSpacePins.html )
It works well, but when I totally close the application and then go back and scan again my QR code, the holograms close to the QR code (less than 1m) are well placed, but the ones which are farther (5m) are not well positioned... I'm saving the position of the hologram relative to the QR code(which is at the root of the sub scene)

I don’t understand why it does this, because in a different test I made (with only WLT, without QR codes or Space Pins), if I create holograms everywhere, close my app, restart it, they are well positioned… but I’m not sure if they will be positioned at the same place for all the users/hololens when I share my JSON..

Do you know a better way to fix this ? Do I need to put a lot of QR codes everywhere (it will be a bit tedious to scan all of them everytime) ? Or do this without any QR codes at all ?
The ideal would be to just scan one QR code to position the origin of the scene and then use WLT to precisely position all of the holograms.

Thanks in advance !!

Questions about space pins persistance

Hi,

I'm trying to create a system where I dynamically create space pins to align virtual sub spaces to my physical world.
I have a few questions :

1. Can multiple scenes share the same .fbw (in Persistance/) ? I plan to have a systeme where the user can switch between an Edit scene (like, an admin can edit the markers) and a Viewer scene (not editing, just scanning the marker and seeing the virtual world). So theses two scenes share the same space pins (but these scenes are never loaded at the same time !)

2. If I delete a marker from my database, or move it in the physical world then update its virtual coordinate, is it enough to delete the corresponding .fbw ? Or do I need to delete the frozenWorldState.hkfw too ? Or call a specific function ?

Thanks

[Question] How is interpolation of space pins calculated?

We've been experimenting with WLT with the following setup:

• 3D scan of a space (large, around 20mx50m, but with repeated columns)
• Set up QR codes at known locations based on point cloud scan, <10mm placement error
• Create space pins (SpacePinOrientable) registering (and updating) the QR code locations
• Visualize (decimated) point cloud scan to manually/optically see the deviation from the physical

Our initial assumption was that interpolation would be based on the position of the HoloLens relative to the physical location of the QR codes (space pins) in either a weighted or linear (docs) fashion. However, what we have noticed is that there's significant movement/compensation/interpolation which appears to be based on the orientation of the headset. To provide an example:

Each number represents a QR code & space pin placed at a measured location on a column. The red represents a user and arbitrary FOV.

1. This position, shows the frozen space and QR (4) code appear very close.

2. Rotating the head without changing position appears results in clearly visible interpolation of the digital position. The digital (4) shifts a significant distance from the physical in a repeatable interpolation (rotating head back and forth re-plays the interpolation).

3. Standing on the other side of the column shows the digital in the correct location (as per the docs there is no extrapolation when outside the space pin boundary, so this makes sense).

Based on this we concluded that the orientation of the headset (which space pins are in FOV and/or which feature points are in FOV) affects the interpolation. The logic for this would be reasonable - make the space pins that are being _looked at _ appear correct rather than a space pin that is behind you... However we are interested in having some more control over the weighting here, as the user may be looking at a hologram immediately in-front of them even if there's a space-pin in the distance.

So the short question is: Does the users facing direction affect the interpolation?
The longer one is: If so, how is this weighted with the device position, and is there any way to control this?

WLT + MRTK + Holographic Remoting in Unity - No Anchor Subsystems error.

So I've been trying for a couple days now to get WLT working with my Hololens 2, Holographic Remoting (in the Unity editor) MRTK and Unity 2020.3 ( I also tried 2019.4 without success). I can get the Holographic Remoting working without problem, but WLT fails to load any anchor manager, and so doesn't seem to do anything. Exact debug messages I receive are at the bottom here.

Exact steps to reproduce are below - am I doing something wrong, or does this just not work at the moment?
The procedure used below is for Windows XR Plugin, but I've also tried using OpenXR without any success.

Following the directions:
Project/MRTK Setup: https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unity/tutorials/mr-learning-base-02?tabs=winxr
WLT Setup: https://microsoft.github.io/MixedReality-WorldLockingTools-Unity/DocGen/Documentation/HowTos/InitialSetup.html#adding-world-locking-tools-to-a-unity-scene

Using the Windows XR Plugin

1. Install Unity 2020.3.4f1 with UWP and IL2CPP and create a new Project
2. Switch to UWP in Build Settings
3. Import TextMesh Pro Essential Resources
4. Using Mixed Reality Feature Tool, add the following to the project:
   • Mixed Reality Toolkit Extensions 2.6.1
   • Mixed Reality Toolkit Foundation 2.6.1
   • Mixed Reality Toolkit Standard Assets 2.6.1
   • WLT Core 1.3.5
5. Apply MRTK Default Settings
6. Install XR Plugin Management
7. For both standalone and UWP, Check Windows Mixed Reality and Initialize XR on Startup
8. Under Windows Mixed Reality, check Holographic Remoting for both Standalone and UWP.
9. Under Player->Publishing Settings -> Capabilities, Add Webcam and InternetClientServer
10. Window -> XR -> Windows XR Plugin Remoting, Set Remote to Device, Set IP shown on Hololens Remoting App.
11. Mixed Reality Toolkit -> Add to Scene and Configure.
12. On the MRTK Gameobject, Switch to DefaultXRSDKConfigurationProfile

Now if I hit play, I can see my Unity scene in the Hololens 2, so far so good.

Try to add World Locking Tools:

1. Mixed Reality Toolkit -> Utilities -> World Locking Tools -> Configure Scene
2. World Locking Manager Anchor Management is set to "XRSDK".

When I hit play, I get the following messages (deleted the traces for brevity)
Null: Creating AnchorManagerNull
Select XRSDK anchor manager.
Creating new anchormanager, but have old one. Reseting it before replacing.
Trying to create XR anchor manager
Found 0 anchor subsystems.
Found no anchor subsystem running, will try starting one.
ERROR: Found 0 active anchor subsystems, expected exactly one.
Failed to create requested XR SDK anchor manager!
Failure creating useful anchor manager of any type. Creating null manager
Null: Creating AnchorManagerNull

If I switch to WSA for Anchor Management, I get something similar just with WSA instead of XR.

Also, don't know if this is related, but I'm getting this error message too:
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:...\Library\PackageCache\com.microsoft.mixedreality.worldlockingtools@1b0a49d9b4b5-1619025180880\Packages\Microsoft.MixedReality.Unity.FrozenWorld.Engine.1.1.0\lib\unity\iOS-ARM64\FrozenWorldPlugin.framework'.

I don't know why it's looking for an iOS-ARM64 dll though....

Anyways. Any help would be appreciated.

Update docs for nuget+Unity2020

See #144

Update the documentation to suggest using command line nuget restore to install FrozenWorld Engine DLL in Unity 2020 (and in general), since NuGetForUnity is being deprecated.

New anchor settings on WLT mgr aren't propagated

This is split off from issue #112 .

When the WorldLockingManager's Settings field is set, it triggers the implementation of the new parameters, via a call to ApplyNewSettings(). ApplyNewSettings() in turn, calls ApplyAnchorSettings() (along with other settings propagation).

But according to issue #112 the anchor settings aren't immediately applied to the AnchorManager, even though directly setting them (e.g. via IAnchorManager.MaxLocalAnchors) does immediately propagate them.

The reporter of #112 is unblocked, but this needs to be investigated.

Persistence via OpenXR

The Microsoft Mixed Reality OpenXR extension plugin (currently preview) now supports persistence of local anchors.

WLT should take advantage of that when available.

https://docs.unity3d.com/Packages/[email protected]/api/UnityEngine.XR.WindowsMR.XRAnchorSubsystemExtensions.html

https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unity/openxr-supported-features#anchor-persistence-and-the-xranchorstore

Need documentation and sample of how to best integrate with Azure Spatial Anchors

Scenarios:

• How do you place a Hologram in the world locked space that's really backed by an Azure Spatial Anchor. Is this just attachment points?
• How to have the internal anchors used to manage the world locked space be backed by ASA. This would let other HoloLenses participate in a shared experience in the world locked space
• Probably other scenarios

Orphans in the docs

TOC needs update, some docs pages don't appear there, making them difficult to find/discover (even though it appears there are links to them from within the docs, esp. from See Also sections.

Unexpected behavior loading state after build changes

When state is saved for a given application for one configuration, but then the application is reconfigured and re-deployed, then the system will attempt to apply the old state to the new configuration with undefined results.

For a simple case, consider deploying the Space Pins example, saving state, then deploying and running the RayPins sample. Because the system sees them as the same application, it won't know to discard the state, and everything will look wrong.

If possible, it would be great to fix this, but at the very least it needs to be clearly documented as a known hazard.

Slowdown after changing scenes

Repro:

1. Load one scene
2. Load another scene (non-additive)

Result:
Over the course of a few seconds the frame rate slowly drops to a grind. Cursory examination it appears there is an exception in WorldLockingManager.UpdateProxy.

Space Pins example spheres reset to original position

I'm sure I'm missing something obvious as this is my first exploration into WLT but I haven't been able to get the Space Pins example to work as outlined in the walkthrough doc. I'm able to drag the sphere to a new location (or rotate them) but when I release they reset back to their original position. My exception from the walkthrough is they should stay in the new position and the virtual world should update its alignment.

Hololens 2 OS (up to date): 19041.1136
Unity 2019.4.19f1
MRTK 2.5.4 and WLT 1.2.4 imported with MRFeatureTool
XRSDK Setup:
XR Plugin Manager 3.2.17
Windows XR Plugin 2.5.2
XR Legacy Input Helpers 2.1.7
Space Pin example imported into a clean scene via the Package manager
Positions of the markers (spheres) and markersreference (cubes) was adjusted to be a little closer than the default scene for the room I'm working in

Have I misunderstood something basic or do you have any suggestions that would explain this behaviour?

Thanks.

Feature request: Add persistence support using Windows XR Plugin XRAnchorStore extension

In my previous communication I was not referring to adding persistence using the "newest" OpenXR Plugin
Microsoft.MixedReality.ARSubsystems.XRAnchorStore that you have just implemented (thanks!),
but through the "new" Unity XR plugin: Unity​Engine.​XR.​Windows​MR.XRAnchor​Store
(Note: I use "new" to differentiate it from the older plugin "com.unity.xr.windowsmr.metro")

Can you add support for this persistence extension too? I think the OpenXR one is obviously very future proof, however I believe for projects starting now the XR plugin could be a better starting point.
Thanks you!

Davide Bergamini

EDIT: In my previous communication I mixed up the two plugins, sending you both reference links together, sorry!

Space Pin interpolation should work vertically, too, but only does so horizontally

I'm trying to use Space Pins to align the size of an MR space with the exact real-world space's size determined beforehand using a laser ruler.
However, the virtual objects are not placed on just one floor, but on several floors in a building - and I need the Space Pins to also correctly align the vertical distance of the floors with the laser ruler measurement.

When trying to achieve this with SpacePins right now, the result is this:

The pinned scene shifts to the ground floor alignment when the user is standing above the ground floor SpacePin (or more precisely, is closer to the the SpacePin horizontally), whereas the scene uses the top floor alignment when the user is standing above the top floor SpacePin (is closer to that SpacePin horizontally).
When the two SpacePins are roughly above each other with just a minor horizontal offset, this means that the aligned scene wildly jumps in height when the user is about equally close to the two SpacePins horizontally.

Therefore, it should be possible align the pinned scene also when moving vertically between SpacePins, not just horizontally. That is, the user's distance to the SpacePins that's used for interpolation should take the Y coordinate into account.

Error checking on save and load

If saving persistence data fails for any reason, it must be made extremely clear that something has gone wrong. The log file should have as much information about the error as possible. Also, it might be good to delete the probably corrupt file.

Likewise, if loading a persistence file fails, then the nature of the error should be made as clear as possible in the log file. The persistence file should be deleted to not cause problems in the future.

It might be preferable, rather than deleting the persistence files, to rename them to something unused. That way they will still be there for later debugging. But care should be taken not to save too many that we are just soaking up storage.

Recommended way to get the frozen pose of new QR Codes?

What is the correct way to get the frozen pose of newly scanned QR Codes (So QR codes that are not attached to spacepins)?

Currently I have following code:

private static Pose GetPoseFromNewQRCode(QRCode qrCode, out bool success)
{
	success = false;
	var pose = new Pose();
	
#if WINDOWS_UWP


	SpatialCoordinateSystem CoordinateSystem = Windows.Perception.Spatial.Preview.SpatialGraphInteropPreview.CreateCoordinateSystemForNode(qrCode.SpatialGraphNodeId);


	// check to verify the coordinatesystem for this node is not null
	if (CoordinateSystem != null)
	{
		// create identity/zero rotation and translation
		Quaternion rotation = Quaternion.identity;
		Vector3 translation = new Vector3(0.0f, 0.0f, 0.0f);

		// GetNativeISpatialCoordinateSystemPtr is obsolete!
		SpatialCoordinateSystem rootSpatialCoordinateSystem = (SpatialCoordinateSystem)System.Runtime.InteropServices.Marshal.GetObjectForIUnknown(UnityEngine.XR.WSA.WorldManager.GetNativeISpatialCoordinateSystemPtr());

		// Get the relative transform from the unity origin
		System.Numerics.Matrix4x4? relativePose = CoordinateSystem.TryGetTransformTo(rootSpatialCoordinateSystem);

		if (relativePose != null)
		{

			System.Numerics.Vector3 scale;
			System.Numerics.Quaternion rotation1;
			System.Numerics.Vector3 translation1;

			System.Numerics.Matrix4x4 newMatrix = relativePose.Value;

			// Platform coordinates are all right handed and unity uses left handed matrices. so we convert the matrix
			// from rhs-rhs to lhs-lhs 
			// Convert from right to left coordinate system
			newMatrix.M13 = -newMatrix.M13;
			newMatrix.M23 = -newMatrix.M23;
			newMatrix.M43 = -newMatrix.M43;

			newMatrix.M31 = -newMatrix.M31;
			newMatrix.M32 = -newMatrix.M32;
			newMatrix.M34 = -newMatrix.M34;

			System.Numerics.Matrix4x4.Decompose(newMatrix, out scale, out rotation1, out translation1);
			translation = new Vector3(translation1.X, translation1.Y, translation1.Z);
			rotation = new Quaternion(rotation1.X, rotation1.Y, rotation1.Z, rotation1.W);
			pose = new Pose(translation, rotation);

			// If there is a parent to the camera that means we are using teleport and we should not apply the teleport
			// to these objects so apply the inverse
			if (CameraCache.Main.transform.parent != null)
			{
				pose = pose.GetTransformedBy(CameraCache.Main.transform.parent);
			}

			success = true;

		}
	}


#endif

	return pose;
	
}

The positions from the resulting pose are correct, however the rotations are often shifted by 180 degrees.
For example in a test:

1. add two spacepins to qr-codes positioned on the x-axis with coordinates (0,0,0) and (2,0,0) (qr-codes in the XY-plane)
2. scan a new qr code on the x-axis (in the XY-plane) with coordinate (1,0,0)

Multiple scans give following results (Euclidian rotations in degrees):

• 359.9492, 2.909461, 0.903221
• 357.6101, 182.9058, 180.9385
• 357.1973, 182.2521, 180.7275
• 357.0967, 181.965, 180.6811
• 356.9806, 181.8914, 180.6653
• 358.108, 1.143362, 359.558
• 358.3949, 1.034422, 359.475
• 358.8199, 1.068542, 359.5049
• 358.8633, 1.850759, 359.7042
• 358.5123, 2.797868, 359.7931
• 358.9621, 183.0922, 179.9445
• 359.0522, 182.5009, 179.8562
• 358.8767, 181.697, 179.6391
• 358.7914, 181.4219, 179.6022
• 357.2258, 181.7257, 179.5746

So about half of the time the y and z rotation are both offset by 180 degrees.

What is the recommended way to get the pose that will always be correct?

Restart anchor manager from clean state

Applications may need to reset the platform tracking system (e..g ARAnchorManager) to a clean state. In that case, the WLT AnchorManager should also be reset to a clean state, as all its underpinnings will be either null or stale.

Accept changes of 'Max Local Anchors' during runtime

Now runtime changes to 'Max Local Anchors' on the WorldLockingManager are not taken into account.
Is it possible to allow programmatical changes to this number during runtime?
(No need to add a special UI)
And when the new number is lower immediately remove the necessary amount of anchors?

This would be useful for our performance tests in big areas and find a balance with the rest of our application.

RoadMap and Unity XR SDK timeline

Hi, I'm really interested in this project and just wanted to know if there is a public roadmap anyway? I also noticed in the release notes that Unity XR SDK support is planned for the next release and was wondering if there is any timeframe on this?

Thanks

Long internal paths in MR Feature Tool package may cause building errors

I stumbled upon an issues with the MR Feature Tool package of WLT when the build path is longer than about 50 chars.

I created a new Unity Project with Unity 2019.4.24f1 and run the MR Feature Tool (1.0.2103.3-Beta) and installed some features. My package.json now contains these new items and the related package have been copied.

    "com.microsoft.mixedreality.toolkit.foundation": "file:MixedReality/com.microsoft.mixedreality.toolkit.foundation-2.6.1.tgz",
    "com.microsoft.mixedreality.toolkit.tools": "file:MixedReality/com.microsoft.mixedreality.toolkit.tools-2.6.1.tgz",
    "com.microsoft.spatialaudio.spatializer.unity": "file:MixedReality/com.microsoft.spatialaudio.spatializer.unity-1.0.196.tgz",
    "com.microsoft.mixedreality.worldlockingtools": "file:MixedReality/com.microsoft.mixedreality.worldlockingtools-1.3.5.tgz",
    "com.microsoft.mixedreality.webrtc": "file:MixedReality/com.microsoft.mixedreality.webrtc-2.0.2.tgz",
    "com.microsoft.mixedreality.toolkit.standardassets": "file:MixedReality/com.microsoft.mixedreality.toolkit.standardassets-2.6.1.tgz"

When I try to build the project, I get the following error:

FileNotFoundException: Packages/com.microsoft.mixedreality.worldlockingtools/Packages/Microsoft.MixedReality.Unity.FrozenWorld.Engine.1.1.0/lib/unity/Windows-UWP-ARM64/FrozenWorldPlugin.dll does not exist
System.IO.File.Copy (System.String sourceFileName, System.String destFileName, System.Boolean overwrite) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
PostProcessWinRT.CopyFileWithDebugSymbols (System.String source, System.String target, System.Boolean isFileManagedAssembly) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/PostProcessWinRT.cs:639)
PostProcessWinRT.CopyPlugins () (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/PostProcessWinRT.cs:932)
PostProcessWinRT.Process () (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/PostProcessWinRT.cs:170)
UnityEditor.UWP.BuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/ExtensionModule.cs:85)
Rethrow as BuildFailedException: Exception of type 'UnityEditor.Build.BuildFailedException' was thrown.
UnityEditor.UWP.BuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/ExtensionModule.cs:89)
UnityEditor.Modules.DefaultBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args, UnityEditor.BuildProperties& outProperties) (at <5984c823338e4ca69b7a0ca01115425e>:0)
UnityEditor.PostprocessBuildPlayer.Postprocess (UnityEditor.BuildTargetGroup targetGroup, UnityEditor.BuildTarget target, System.String installPath, System.String companyName, System.String productName, System.Int32 width, System.Int32 height, UnityEditor.BuildOptions options, UnityEditor.RuntimeClassRegistry usedClassRegistry, UnityEditor.Build.Reporting.BuildReport report) (at <5984c823338e4ca69b7a0ca01115425e>:0)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

I have the feeling that the building path has somethings to do with this. While building succeeded at:

C:\Users\alneuman\workspace\WLTTest

It did not not work with:

C:\Users\alneuman\workspace\Unity\SandboxProjects\WLTTest

I checked where Unity keeps the uncompressed packages and the path for WLTCore is:

C:\Users\alneuman\workspace\Unity\SandboxProjects\WLTTest\Library\PackageCache\com.microsoft.mixedreality.worldlockingtools@834acdcbd53e-1618586077562

Combining this with what Unity tells me (Packages/com.microsoft.mixedreality.worldlockingtools/Packages/Microsoft.MixedReality.Unity.FrozenWorld.Engine.1.1.0/lib/unity/Windows-UWP-ARM64/FrozenWorldPlugin.dll)
I end up with the actual file system path:

C:\Users\alneuman\workspace\Unity\SandboxProjects\WLTTest\Library\PackageCache\com.microsoft.mixedreality.worldlockingtools@834acdcbd53e-1618586077562/Packages/Microsoft.MixedReality.Unity.FrozenWorld.Engine.1.1.0/lib/unity/Windows-UWP-ARM64/FrozenWorldPlugin.dll (264 chars)

in contrast to the shorter path

C:\Users\alneuman\workspace\WLTTest\Library\PackageCache\com.microsoft.mixedreality.worldlockingtools@834acdcbd53e-1618586077562/Packages/Microsoft.MixedReality.Unity.FrozenWorld.Engine.1.1.0/lib/unity/Windows-UWP-ARM64/FrozenWorldPlugin.dll (241 chars)

The Unity project path alone (Library\PackageCache\com.microsoft.mixedreality.worldlockingtools@834acdcbd53e-1618586077562/Packages/Microsoft.MixedReality.Unity.FrozenWorld.Engine.1.1.0/lib/unity/Windows-UWP-ARM64/FrozenWorldPlugin.dll) is 205 characters long! It consumes almost 80% of the allowed path length! This means, users have about 20 - 30 chars to organize their projects when you consider the user's home directory static.

Locked SpacePin position drifts over time when moving between floors in a building

I've noticed that when using several SpacePins on different floors in a building, the far SpacePin (on a different floor than where the user is located) drifts over time (time frame: about 10 minutes).

In my scenario, I'm trying to determine the geometry of a staircase using HoloLens 2. To do this, I'm setting measurement points on each floor and the edge of each step; I do this from about an arm's length away.
The workflow is basically the following:

• Start the app. The app sets up an AlignSubtree with two SpacePinOrientables. The hierarchy then looks like this:

• Enter the actual height between both floors in the app (measured beforehand using a laser ruler)
• Perform a thorough spatial scan of the entire staircase (WLT sets up a network of anchors along the staircase while doing this; all are in the same Fragment)
• Set a reference point on the bottom landing as a sibling of the SpacePins.
  • Set the bottom SpacePin's position to that of the reference point, with Y set to 0. ResetModelingPose.
  • Set the bottom SpacePin's frozen pose to that of the reference point.
• Go to the upper landing, add a second reference point there as a sibling of the SpacePins.
  • Set the upper SpacePin's position to that of the reference point, with Y set to the measured height. ResetModelingPose.
  • Set the upper SpacePin's frozen pose to that of the reference point.
• Go back to the bottom landing. Measure each step of the staircase, bottom to top. Each measurement object is set as a sibling of the SpacePins.

Result: When arriving at the upper landing with the step measurements, the upper SpacePin has drifted down by several centimeters.

I'm using a custom TriangulatorY for interpolation between the two SpacePins on just the Y axis.
I'll send the code your way directly via e-mail, together with a few photos of the staircase, the Unity log and Frozen World log.

Unity version: 2019.3.7f1
MRTK version: 2.4
WLT version: 0.8.5
Frozen World Engine version: 0.5.3
Device: HoloLens 2
OS version: 19041.1106.arm64fre.vb_release_svc_sydney_rel_prod.200605-1330
VS version: 2019 (16.6.2)
App build: Release, ARM64

Performance/area limitations

First of all thanks for writing this package. It does not seem to get too much attention yet, but that's probably just a matter of time since the concept is fundamental in making making bigger and seamless AR experiences.

I did a quick test with HL2 in a relative big area and ended up with a anchor count of about 500. At that moment the framerate had dropped to about 30FPS with marker and mesh visualizations switched off.

Have you been testing with bigger areas and what are your experiences?

Would it be an option to reduce the processing intensity for anchors if they are further away, maybe totally disabling above a certain range?

Do you think WorldLockingTools would be able to scale to huge interior spaces, let's say airport terminals? And what would need to happen to make this feasible?

XRSDK tracking loss doesn't register

While it seems to recover once tracking is regained, it makes for a confusing experience in between.

Clarify XR Plugin versions supported

The Unity XR Plugin version matching is a fragile process. The documents should at least list explicitly version combinations that have been found to work.

Bonus points for listing guidance on version combinations that are expected to work, but haven't been tried (i.e. versions not yet released).

MRTK Profile Spatial Awareness/Mapping Layer Not Obtained

When running the OMPTest sample scene, console log spammed with this error:

A game object can only be in one layer. The layer needs to be in the range [0...31]
UnityEngine.StackTraceUtility:ExtractStackTrace ()
Microsoft.MixedReality.WorldLocking.Tools.FrozenSpatialMapping:SurfaceChangedHandler (UnityEngine.XR.WSA.SurfaceId,UnityEngine.XR.WSA.SurfaceChange,UnityEngine.Bounds,System.DateTime) (at Library/PackageCache/com.microsoft.mixedreality.worldlockingtools@8b8274cd779c-1626820768821/WorldLocking.Tools/Scripts/FrozenSpatialMapping.cs:415)
UnityEngine.XR.WSA.SurfaceObserver:Update (UnityEngine.XR.WSA.SurfaceObserver/SurfaceChangedDelegate)
Microsoft.MixedReality.WorldLocking.Tools.FrozenSpatialMapping:UpdateObserver () (at Library/PackageCache/com.microsoft.mixedreality.worldlockingtools@8b8274cd779c-1626820768821/WorldLocking.Tools/Scripts/FrozenSpatialMapping.cs:297)
Microsoft.MixedReality.WorldLocking.Tools.FrozenSpatialMapping:Update () (at Library/PackageCache/com.microsoft.mixedreality.worldlockingtools@8b8274cd779c-1626820768821/WorldLocking.Tools/Scripts/FrozenSpatialMapping.cs:242)

My project's layers:

When setting a breakpoint, spatialMappingLayer is -1, as its searching for the layer "SpatialMapping", instead of the layer used by MRTK ("Spatial Awareness").

Instead of:

spatialMappingLayer = LayerMask.NameToLayer("SpatialMapping");

Maybe something like this should work:

spatialMappingLayer = 0;
var spatialMappingConfig = (MixedRealitySpatialAwarenessSystemProfile)CoreServices.SpatialAwarenessSystem.ConfigurationProfile;
if (spatialMappingConfig != null) {
    foreach (var config in spatialMappingConfig.ObserverConfigurations) {
        var observerProfile = (MixedRealitySpatialAwarenessMeshObserverProfile)config.ObserverProfile;
        if (observerProfile != null) spatialMappingLayer |= (1 << observerProfile.MeshPhysicsLayer);
    }
}

I couldn't apply this solution as it would require changing the assembly definition dependencies.
I suppose as an alternative to using MRTK, the layer could be supplied via inspector

public LayerMask spatialMappingLayer;