icosa-foundation / open-brush

Open Brush is the open source, community led evolution of Tilt Brush! Forked from https://github.com/googlevr/tilt-brush

Open Brush - Art From All Angles

Open Brush is a free fork of Tilt Brush, a room-scale 3D-painting virtual-reality application available from Google, originally developed by Skillman & Hackett. We have made a large number of changes from the original repository, including Unity upgrades and feature additions to bring Open Brush up to modern XR development standards. You can find the notable changes on our docs site.

We hope to maintain and improve upon Tilt Brush as a community-led project, free forever!

As the original repo is archived we cannot submit PRs, so feel free to submit them here!

User Guide
Developer Notes
Roadmap
Please join the Icosa Discord and get involved!
List of tutorials, write-ups and other things from the community
Support us on Open Collective

Downloads

Stores

• SideQuest
• Oculus App Lab
• Steam
• Oculus Rift
• Viveport Desktop
• Pico
• itch.io

GitHub

• Formal GitHub Releases
• Bleeding Edge GitHub Releases

Acknowledgements

• Thank you to the Tilt Brush developers for your amazing work and for finding a way to open source the app!
• SiMonk0 for the great new logo!
• The SideQuest team for your support.
• VR Rosie for promotional artwork, banners, and videos.

Bleeding Edge Releases

Instead of waiting for a formal release, you can download a ZIP from Github containing an automatically built release for either Windows (SteamVR) or Oculus Quest / Quest 2 from the Github releases page. Versions of the form "vX.Y.0" are official releases, whereas versions that do not end in .0 are made available for testing purposes only, with no guarantees as to their quality. Additionally, these releases are marked as "pre-release". However, if you'd like to test a recent change prior to the official release, you can use these either in place of or in parallel with the formal Open Brush releases.

These builds share a save location with the official Open Brush release, but can be installed alongside the formal version. The Oculus build, like all sideloaded content, will be listed in "Unknown Sources", and will have the word "Github" appended to the name (with a different package name as well) to differentiate it from the official release).

Note that the "experimental" builds contain experimental brushes, and sketches created using the experimental brushes may appear differently when loaded in the official build of Open Brush!

In addition, there is also a version created for Windows Monoscopic that is listed as an "Artifact" of the Github Actions, however, this is intended only for developers, and should not be used by general users. You can find it by browsing to the commit list, and then clicking on the green check mark below the title (next to the XXX committed XXX ago), and scroll to the build you want, and click on Details. Then, towards the upper right corner, click on Artifacts and click on the name of the build. Unzip the downloaded file, and either run the executable (Desktop OpenXR/Monoscopic) or install the apk (Android Oculus) using adb install com.Icosa.OpenBrush-github.apk.

Important note from the original Tilt Brush README

The Tilt Brush trademark and logo (“Tilt Brush Trademarks”) are trademarks of Google, and are treated separately from the copyright or patent license grants contained in the Apache-licensed Tilt Brush repositories on GitHub. Any use of the Tilt Brush Trademarks other than those permitted in these guidelines must be approved in advance.

For more information, read the Tilt Brush Brand Guidelines.

---

Building the application

Get the Open Brush open-source application running on your own devices.

Prerequisites

• Unity 2021.3.30f1
• Python 3 (Optional — needed only if you wish to run the scripts in the Support/bin directory) Tested with Python 3.8.

Running the application in the Unity editor

Follow these steps when running the application for the first time:

1. Start Unity.
2. Go to File > Open Scene. \
3. Select /Assets/Scenes/Main.unity. Unity should automatically prompt you to import TextMesh Pro.
4. Choose Import TMP Essentials.
   You can also do this through Window > TextMesh Pro > Import TMP Essential Resources.
5. Press Play.

These steps have been tested with Release 1.0.54.

Building the application from the Unity editor

Although it's possible to build Open Brush using the standard Unity build tools, we recommend using a build script to ensure the application builds with the correct settings. To run this script, go to Open Brush > Build > Do Build, or build from the Open Brush build window by navigating to Open Brush > Build > Build Window.

Note: The application may take a while to build the first time.

Building the application from the Windows command line

Use the build script in the Support/bin directory to specify the target platform and the build options you wish to enable. Run build —help to see the various build options.

Additional features

You should be able to get the basic version of Open Brush up and running very quickly. The following features will take a little more time.

• Google service API support
• Enabling native Oculus support
• Sketchfab support
• Offline rendering support

Systems that were replaced or removed when open-sourcing Tilt Brush

Some systems in Tilt Brush were removed or replaced with alternatives due to open-source licensing issues. These are:

• Sonic Ether Natural Bloom. The official Tilt Brush app uses a version purchased from the Asset Store; the open-source version uses Sonic Ether's slightly modified open-source version.
• FXAA. The official Tilt Brush app uses a modified version of the FXAA that Unity previously released with the standard assets on earlier versions of Unity - FXAA3 Console. This has been replaced with FXAA by jintiao.
• Vignette and Chromatic Aberration. The official Tilt Brush app uses modified versions of the Vignette and Chromatic Aberration effects that came with the standard assets in earlier versions of Unity. These have been replaced with a modified version of KinoVignette by Keijiro.
• Tilt Shift. The official Tilt Brush app uses modified versions of the Tilt Shift effect that came with the standard assets in earlier versions of Unity. These have been replaced with a modified version of Tilt shift by ruby0x1.

Generating Secrets file

Credentials for services such as Google and Sketchfab are stored in a SecretsConfig scriptable object. This has been ignored in the git config for safety. To add it back:

1. Right click in the root /Assets folder in Unity's project window. Select Create, then Secrets Config. This will create Secrets.asset in the Asset folder.
2. In Scenes/Main.unity go to App > Config and replace SecretsExample with the newly generated Secrets.asset.

Google service API support

Set up Google API support to access Google services in the app.

Enabling Google service APIs

Follow these steps when enabling Google service APIs:

1. Create a new project in the Google Cloud Console.

2. Enable the following APIs and services:

   • YouTube Data API v3 — for uploading videos to YouTube
   • Google Drive API — for backup to Google Drive
   • People API — for username and profile picture

Note: The name of your application on the developer console should match the name you've given the app in App.kGoogleServicesAppName in App.cs.

Creating a Google API key

Follow these steps when creating a Google API key:

1. Go to the Credentials page from the Google Cloud Console.
2. Click Create Credential and select API key from the drop-down menu.

Google OAuth consent screen information

The OAuth consent screen asks users for permission to access their Google account. You should be able to configure it from the Credentials screen.

Follow these steps when configuring the OAuth consent screen:

1. Fill in the name and logo of your app, as well as the scope of the user data that the app will access.

2. Add the following paths to the list of scopes:

   • Google Drive API ../auth/drive.appdata
   • Google Drive API ../auth/drive.file

Creating an OAuth credential

The credential identifies the application to the Google servers. Follow these steps to create an OAuth credential:

1. Create a new credential on the Credentials screen.
2. Select OAuth, and then select Desktop App. Take note of the client ID and client secret values that are created for you. Keep the client secret a secret!

Storing the Google API Key and credential data

Follow these steps to store the Google API Key and credential data:

1. Follow the steps to create your secrets file. Add a new item to the Secrets field.
2. Select Google as the service. Paste in the API key, client ID, and client secret that were generated earlier.

Enabling native Oculus support

Open Brush targets OpenXR instead of Oculus by default. Follow these steps to enable native Oculus support: .

1. In the Standalone and Android tabs of the Player settings, go to Other Settings > Scripting Define Symbols.
2. Click the + button to create a new entry.
3. Add OCULUS_SUPPORTED and press Apply.

Building your app for Oculus Quest

Follow these steps to build your app for Oculus Quest:

1. Set up your machine for Oculus Quest Development.
2. Make sure the following are set in Unity:
   • Open Brush > Build > Plugin: Oculus
   • Open Brush > Build > Platform: Android
   • Open Brush > Build > Runtime: IL2CPP
3. Navigate to Open Brush > Build > Do Build.
4. Find the generated executable. It will most likely be somewhere under ../Builds/OculusMobile_Release_OpenBrush/.
5. Run adb install com.Icosa.OpenBrush.apk.

Publishing to Oculus stores

Note: Tilt Brush is a Google trademark. If you intend to publish a cloned version of the application, you are required to choose a different name to distinguish it from the official version.

Follow these steps to publish to Oculus stores:

1. Get an application ID from Oculus. The desktop and quest versions of each application need separate IDs.
2. Follow the steps to create your secrets file. Add 2 new items to the Secrets field.
3. Add these IDs to the Secrets file. Both Oculus and OculusMobile should have their own entries.
4. Put the app IDs in the Client ID field for each.

Open Brush intro sketch

The Open Brush intro sketch uses some slightly modified shaders to produce the animating-in effect while the sketch fades in. For faster loading, the intro sketch is turned into a *.prefab file beforehand. Only the shaders used in the intro sketch have been converted to work with the introduction.

• The current intro sketches are located in Support/Sketches/Intro. There are two versions, one for PC and one for mobile.
• The *.prefab files are located in Assets/Prefabs/Intro.
• The materials and shaders used in the intro are located in Assets/Materials/IntroMaterials.
• The Assets/PlatformConfigPC and Assets/PlatformConfigMobile files reference the *.prefab files that will be used in the intro.

Creating an intro sketch

Follow these steps to replace or alter the intro sketch:

1. Make sure the sketch of your choice is already loaded. Run Open Brush in the Unity Editor.
2. Select Open Brush > Convert To Intro Materials in the main Unity menu. This converts the materials in the sketch to the intro versions.
   You will get warnings in the console for any materials it could not convert, as well as a summary of how many materials it converted.
3. Navigate the hierarchy. Under the Main scene, open SceneParent/Main Canvas. Select any of the Batch_... objects to check whether they have the intro materials set.
4. Move any objects that do not start with Batch_ out from under the Main Canvas node.
5. Select the Main Canvas node and run the Open Brush > Save Game Object As Prefab menu command.
   The scene will be saved as a *.prefab file called gameobject_to_prefab. under the Assets/TestData folder.
6. Move the game object into the Assets/Prefabs/Intro folder.
7. Update the references in Assets/PlatformConfigPC and Assets/PlatformConfigMobile to point to your new prefab file.

Creating an intro sketch for mobile applications

You may want to have a pared-down version of the intro sketch for the mobile version of the app. Stroke simplification is located in the Settings menu inside Open Brush.

New Scenes

By default, your app will only build the scenes defined in the DoBuild method (string[] scenes = {...} ) in BuildTiltBrush.cs under Assets/Editor/. Make sure to add your custom scenes to this array if you want to see them in app.

Sketchfab support

Follow these steps to enable Sketchfab support:

1. Contact Sketchfab for a client ID and secret before you can upload to their service.

   • The Application Name will probably need to be changed
   • The Grant Type should be Authorization Code
   • The URI should be http://localhost:40074/sketchfab

2. Follow the steps to create your secrets file. Add a new item to the Secrets field.

3. Add the client ID and secret to the field.

4. Set the service as Sketchfab. Leave the API key blank.

Video support bug fix

If you add video support, you may encounter a bug where the "Looking for audio" and "Play some music on your computer" text will disappear if the controller is angled too far. Fix this by doing the following:

1. In Unity, find the /Assets/TextMesh Pro/Resources/Shaders/TMP_SDF.shader file.
2. Duplicate it and rename this file TMP_SDF-WriteDepth.shader.
3. Open the new file in a code or text editor and make the following changes to it:
   1. Change the name from TextMeshPro/Distance Field to TextMeshPro/Distance Field Depth.
   2. Change Zwrite Off to Zwrite On.
4. In Unity, select /Assets/Fonts/Oswald-Light SDF.asset.
5. Under Atlas & Material, double click Oswald-Light SDF Material.
6. At the top, change the name for Shader from TextMeshPro/Distance Field to TextMeshPro/Distance Field Depth.

Offline rendering support

When the user records a video from a saved sketch in Open Brush, a .bat file is generated next to the .mp4 for offline rendering support. This .bat file requires the path to the executable of Open Brush. The code for writing out this path to the file has been removed.

Follow these steps to restore the path:

1. Open the file Assets/Scripts/Rendering/VideoRecorderUtils.cs in a code or text editor.
2. Look for the function CreateOfflineRenderBatchFile near the bottom of the file.
3. In the function, find the comments on how to modify the string to point to the executable path.
4. Update the string to point to the correct path.

Experimental mode

Experimental mode is where features live before they are ready to be released in a production build. This mode enables the experimental brushes and experimental panel while disabling the intro sequence. Experimental mode can be enabled from the settings panel, and requires a restart.

New features and brushes that you find in experimental mode may not work as expected. Sketches that use experimental features and brushes won't work on Icosa or Sketchfab, and may break if loaded into production versions of Open Brush.

Making your code experimental

Code in experimental mode is usually surrounded by the following block:

    if (Config.IsExperimental) {
      // Experimental code goes here
    }

Experimental brushes

Experimental brushes and environments are located in the Assets/Resources/X folder. They are not visible in non-experimental mode.

open-brush's Issues

Reference Library models import tab is missing.

From a user report on discord:

Noartnoglory
Hello amazing devs! I had a question. I understand .fbx import is not supported in Open Brush but I was wondering why this option was removed completely from the current version? I only ask because I have been experimenting with creating singular models that I can use repeatedly in scenes. I know it's different software but I am able to import .gbl files from TIlt/OpenBrush into Multibrush via my PC. Do you think this option to import models will arrive in the future? Maybe not .fbx but potentially .gbl?

Mikesky
Removed completely? How so? We added fbx export support in open brush 0.2.0, and I wasn’t even aware that you could import fbx on original tilt brush. Could tilt brush desktop import glb? We can do that most likely

Noartnoglory
Hey @mikesky , I should have used better choice of words. Within the Media Library section the icon for selecting model is gone. It is blank and only shows icons for image and video. I am able to import .glb exports from Open Brush into PC version of Tilt Brush by dragging/dropping the .glb into the Tilt Brush Media Library model folder.

Guides remain selected after duplication and deselect

From VR_Rosie https://discord.com/channels/783806589991780412/804677406572150814/826479245005946900

I have a bug, a bug that is a Tilt Brush bug so most likely an OB one too... its annoyed me for some time now.

When you select a guide, duplicate the guide, then unselect...

The guide remains ✨selected✨

It's actually quite annoying because the selection is real bright then I struggle to actually paint onto said guide properly.

But if I close TB and reopen the scene everything is ✨fine again✨

Clean install on up-to-date Quest 2 crashes on start up

I’ve just installed Open Brush on Quest 2 and it constantly crashes on start up

Make Vive Android build

Add this new functionality. Android previously only GoogleVR and Oculus Quest.

Migrate large binary files to git lfs?

I’m a bit conflicted about this one, but in general, git is not the best place to store large binary files. Git-lfs is the best way to integrate them (and is supported by GitHub), but an effective migration would involve rewriting history which would affect forks. On the other hand, it greatly shrinks the size of the repo and makes many operations faster. I see three options here:

1. Ignore it.
2. Migrate ASAP and get the pain over with now (of having forks needing to be updated and any in progress PRs rebased)
3. Wait until development settles down and then migrate (but the number of forks will just go up over time, even if the number of active forks drops)

Any thoughts from others who’ve worked on large git reports?

Move to Unity XR plugins

Notes about moving to Unity XR plugins. There are some complications here. See #130.

• We can move to XR plugin framework to support some vendors and we can stick on SteamVR plugin to support others.
  • We can stick on Unity 2019.4 LTS for this.
• When we move to Unity 2020.3 we can use the Unity OpenXR XR plugin.

[Android] Camera tool breaks after 1 or 2 uses

Reported by @daveaudio224on Discord:

Running Open Brush on both App lab and sidequest on Quest 2 - there’s a glitch with the camera where it might take 1-2 pictures, but the camera viewfinder then turns to a horizontal white stripe.

https://discord.com/channels/783806589991780412/804677406572150814/822108368587849808

Paintings open behind, below, or on top of me

I'm seated and have little room to move around. When I open existing paintings, they open at random positions in relationship to my chair -- behind me, on top of me, around me, below me, whatever. I know you can use the teleport feature to view from a different angle, but you apparently can't tell it which direction you would be facing after teleportation -- it's fixed, so that at best I can see the backs of paintings.

I really want to view and admire the amazing artwork that people have created (as I have no artistic skill myself)!

Quest 1 Crashing

Origins of the strange crashing:
We have received an app lab review mentioning "I only wish I could spend more than 5 minutes without it completely crashing on me"

Siiimon from the Discord, also came across this bug on a Quest 1: "Hi. I am testing open brush on quest via app lab (v 0.2.0), but it seems to crash every few minutes."

Personally, I have been unable to find any crashing on the Quest 2, but did get a logcat of a crash from my Quest 1.

Logcat of a Crash
Attached is a crash log, you can see the crash happens just before 23:03:43
logcat23-04crash.log

2021-03-18T23:03:43.519Z:error14408/14377 AndroidRuntime FATAL EXCEPTION: UnityMain<br>Process: com.Icosa.OpenBrush, PID: 14377<br>java.lang.Error: FATAL EXCEPTION [UnityMain]<br>Unity version : 2019.4.18f1

Possible Reproduction of the crash
@mikeskydev got a crash on scene change.
To reproduce this, I kept opening many different scenes with varying complexities and also spent time drawing in new scenes.
Of the 4 times, I have managed to get a crash. 3 were while loading scenes, and 1 happened after taking the headset on and off after loading a scene, which is the one in the logcat, note the onHeadsetMounted.

Once a crash has happened I have been unable to get another until I reinstall Open Brush.
If anyone know how to get a crash, please comment here.

Vive Controllers not detected after SteamVR menu is closed (Linux)

Hello,

I have built this for Linux.
In order for Open-Brush to detect my Vive Controllers I have to close any steamvr menu's before launching Open-Brush. While Open-Brush is running I then need to make sure not to hit the home button because exiting the steamVR overlay menu results in Open-Brush not detecting the controllers again.

Regards, tuxfoo.

Building for Android IL2CPP fails

Oculus reccommend targeting arm64 for Quest builds, and suggest they might block arm7 in the future. I've previously tested this when submitting to app lab and some errors, which I think are coming from third party libraries.

Google login for PC version

There's an option to log in with Google to access YouTube, etc, in the Accounts tab. However, Google blocks login in the browser. Is this broken, or now non-fuctional? Using latest version; I tried the Steam and the Oculus PC versions

Neon Pulse brush is affected by the fog setting

Reported by @bassem on Discord:

Hey guys, last night while working on a scene I realised that the Neon Pulse brush is affected by the fog setting. That is, without fog only the pulse is visible, as should be, but as soon as you turn up the fog, the "tube" that the pulse runs in becomes more and more visible . Any chance that could be modified so the tube always stays hidden? (I assume most people would want only the pulse to be visible)

https://discord.com/channels/783806589991780412/806977725625139210/817490722579152927

Mixed Reality Capture not working on the Oculus Quest

It appears that there are 2 issues that are causing the Mixed Reality support on the Oculus Quest to be broken:

• 1. The different units between the Oculus SDK and Open Brush (Open Brush uses decimeters instead of meters).
• 2. The fact that Open Brush isn't using an OVRCameraRig and it's just setting up OVRManager manually instead (here).

Issue number 2 makes the OVRExternalComposition's cameraInTrackingSpace (part of the Oculus Integration Package) to become false, making it use the incorrect worldSpacePose for the foregroundCamera and backgroundCamera cameras used for mixed reality.

I've managed to apply a workaround to the OVRExternalComposition object to fix both issues in a local copy, but that's not really ideal, since this is changing a file that's part of the Oculus Integration Package and this is currently being ignored by git.

My solution was to replace line 97 with

backgroundCameraGameObject.transform.parent = null;

replace line 123 with

foregroundCameraGameObject.transform.parent = null;

and replace lines 359:370 with

// if (cameraInTrackingSpace)
// {
    OVRPose trackingSpacePose = ComputeCameraTrackingSpacePose(extrinsics);
    trackingSpacePose.position = trackingSpacePose.position * 10; // Tilt Brush adjustment
    backgroundCamera.transform.FromOVRPose(trackingSpacePose, true);
    foregroundCamera.transform.FromOVRPose(trackingSpacePose, true);
// }
// else
// {
// 	OVRPose worldSpacePose = ComputeCameraWorldSpacePose(extrinsics);
// 	worldSpacePose.position = worldSpacePose.position * 10; // Tilt Brush adjustment
// 	backgroundCamera.transform.FromOVRPose(worldSpacePose);
// 	foregroundCamera.transform.FromOVRPose(worldSpacePose);
// }

essentially making OVRExternalComposition ignore cameraInTrackingSpace and also making it multiply the mixed reality camera's position vector by 10 (converting from meters to decimeters).

I believe we might need to adopt the OVRCameraRig on the VrSdk to be able to solve this properly, but this might cause other issues. There might be some other solution, but I'm not really an expert on Unity or the Oculus SDK.

IL2CPP breaks browser launching on Quest

Building for Quest in IL2CPP mode breaks the ability to launch the Oculus browser for Oauth login.
This bug was introduced in #126

Steps to Reproduce:

• Build for IL2CPP
• On Quest, attempt to log in to either Google or Sketchfab
• Observe the Oculus browser doesn't launch

Recommend reverting #126 for now as IL2CPP is not a hard requirement yet.

Replace GvrAudioSource

GvrAudioSource is apparently obsolete. The Google replacement is Resonance.

This work may be required for #135, move to Unity 2020.

[feature/xr-plugin] Monoscopic mode height is wrong

Camera is way higher than it normally would be.

Dots brush failing FBX export

From Zandy#1492

https://discord.com/channels/783806589991780412/804677406572150814/827538610010062848
All the experimentals were replaced but the dots brush was the culprit. It will not export the dots brush to fbx or sketchfab/poly. As soon as I deleted them it worked.

Mike's thoughts:
possible that guid has changed in OB, as dots created in TB are fine for export.

Video recorder on Android

It would be nice to have video recording feature on Android. This will require an Android version of ffmpeg or similar.

Possible useful notes:
https://github.com/tanersener/ffmpeg-kit
https://sourceexample.com/en/671cf50fddfec13eb5e1/

Investigate Steam support under Unity XR plug-in framework

• This article summarises the Valve OpenXR / Unity XR issue.

• Valve's statement.

• Unity-xr-plugin which does not support the Unity XR Input system.

• Complication for issue #129.

• OpenBrush 0.3 on SteamVR Unity Plugin 2.3.2.

• Version 2.7.3 is available and includes "support for Unity XR API" (added in 2.6.x).

• OpenVR Unity XR Plugin also available!

  • The purpose of this package is to provide OpenVR rendering to Unity XR. This package provides the necessary sdk libraries for users to build Applications that work with the OpenVR runtime. The OpenVR XR Plugin gives you access to rendering on all major VR devices through one interface. Explicit support for: HTC Vive, HTC Vive Cosmos, Oculus Rift, Oculus Rift S, Oculus Quest (Link), Windows Mixed Reality, and Valve Index. Other SteamVR compatible devices are supported though may have inaccurate or incomplete features.
  • Valve have abandoned this plugin.

It appears that Valve's direction here is: we'll support OpenXR and everyone else should follow this.

Unity have the new Unity XR plugin system so some vendors have plugins for this, and others will be supported through the new Unity OpenXR XR plugin (requires Unity 2020.3).

Some experimental brushes cannot be erased.

CandyCane, Snowflake, Holiday Tree brushes have been reported to be unerasable.

As pointed out by @dubois , CandyCane, Snowflake, Holiday Tree, and Plait brushes are all derived from ParentBrush, which does not have batching enabled and therefore they all appear as separate objects in the inspector.
There is a comment addressing this problem here.

Fix GL_INVALID_OPERATION spam

OpenGL rendering spams out the following message:

02-08 17:12:53.052 32424 32518 E Unity   : OPENGL NATIVE PLUG-IN ERROR: GL_INVALID_OPERATION: Operation illegal in current state 
02-08 17:12:53.052 32424 32518 E Unity   : (Filename: ./Runtime/GfxDevice/opengles/GfxDeviceGLES.cpp Line: 437)

Comments:

• it's a shader that is used in the default environment, but not in some of the others
• quick test implies that it's the "TiltBrush/Standard (Specular setup)" shader

.tilt files with unknown brushes cause NRE when loading

1. Create a sketch with an experimental brush
2. Run non-experimental OB
3. Load the sketch
4. Do not quickload

Expected: Sketch loads, perhaps complaining about an unknown brush
Actual: NRE, and the sketch stops loading after the first stroke using an unknown brush

Simplify VrSdk

Do some of what is suggested in the comment. Simplify VrSdk to make further refactoring and development easier.

    //
    // The VrSdk is an abstraction over the actual VR hardware and SDK. It is responsible for:
    //
    //   * Initializating the VR system, cameras, controllers and associated state.
    //   * Providing hardware- and SDK-specific controls via a non-specific interface.
    //   * Providing abstract access to events sent from the SDK.
    //   * Exposing an interface to query Hardware and SDK capabilities.
    //
    // TODO: In its current form, the VrSdk is monolithic, though it should ultimately be
    // broken out into hardware- and SDK-specific modules, which can be loaded and unloaded at startup
    // or build time.
    //

    //   - Distribution
    //       - Build
    //           - SDK
    //           * Platform - Win32/64, Android, iOS.
    //           * Options
    //                * Experimental
    //           * Config
    //               * Develop
    //               * Profile
    //               * Release
    //       - Device
    //           - Overlay
    //           - Controller

The structure above is more modular. If there is a "distribution" above the SDK then the architecture can be specialised for a target store/platform, e.g. Store features/analytics/content. Additionally, multiple SDKs could be used for target builds whilst the OpenXR complications sort themselves out.

Remove the "Add Poly API" step in the readme for enabling Google services.

That is no longer necessary right?

Color mismatch between Tilt Brush and Open Brush

Thanks to @ignis32 for finding!

Steps to reproduce

• Create a color palette in your favourite design program
• Load the palette as an image in Open Brush
• Select an unlit brush, and color pick from the image
• Paint something

Expected result: Color matches the imported image palette
Actual result: It's not the correct color
Note: This behaves correctly in the official Tilt Brush release

Example Images
Official Tilt Brush Release:

Open Brush @0.2.0rc-1

Move to Unity 2020.x

As of June 2020 OpenBrush is on Unity 2019.4 LTS. It would be preferable to upgrade to 2020.3 because we could get the Unity OpenXR XR plugin. This change may mean that the SteamVR SDK is no longer required.

There may be a considerable amount of work in this change because the UnityEngine.VR namespace (which is currently deprecated) has been removed.

Branch for this: feature/unity2020.

OpenBrush is crashing when starting a Mixed Reality Capture session

The app crashes as soon as a new TCP connection is made to port 28734 (Mixed Reality Capture). I've managed to reproduce this consistently this with the App Lab version of the app (1.0.0) running on version 37 of the system.

Investigate moving to Unity 2020.x

In order to get the Unity XR OpenXR plugin we need to move to Unity 2020.3. This creates more work, as all the old VR SDKs are deprecated and need removing, but when fixed, will be supported by Unity XR plugins and OpenXR. #129

Missing asset for SketchbookPanel notSupported

There is a missing asset. There on mobile, but not on PC.

UnassignedReferenceException: The variable m_NotSupportedMessage of SketchbookPanel has not been assigned.
You probably need to assign the m_NotSupportedMessage variable of the SketchbookPanel script in the inspector.
TiltBrush.SketchbookPanel.RefreshPage () (at Assets/Scripts/GUI/SketchbookPanel.cs:341)
TiltBrush.SketchbookPanel.SetVisibleSketchSet (TiltBrush.SketchSetType type) (at Assets/Scripts/GUI/SketchbookPanel.cs:232)
TiltBrush.SketchbookPanel.OnStart () (at Assets/Scripts/GUI/SketchbookPanel.cs:169)
TiltBrush.ModalPanel.Start () (at Assets/Scripts/GUI/ModalPanel.cs:69)

`Sketchbook

Crash on "Enable Profiling"

Thanks for all the great work, can't wait to open some PRs!

Build:
9d84ab0

Platform:
SteamVR (Valve Index)
Windows 10 Home Edition, x86-64

Repro steps:

1. Enable "advanced mode"
2. From the top left hand menu, select "enable profiling"
3. Select "enable profiling" again
4. App freezes

I'll try and figure out why this happens, but leaving this here in case I don't find anything

Investigate and replace GoogleVR code

The GoogleVR package included in Scripts/ThirdParty/GoogleVR (GitHub, GoogleVR, Unity packages) is deprecated.

• It is used for audio (see AudioManager) on all builds.
• Supports some Google devices.

Since the package is deprecated we should look at replacing it. The new XR management plugin tries to delete it. See class XRLegacyUninstaller.

private static List<string> k_PackagesToBeRemoved = new List<string>(){
            "com.unity.xr.googlevr.android",
            "com.unity.xr.googlevr.ios",
            "com.unity.xr.oculus.android",
            "com.unity.xr.oculus.standalone",
            "com.unity.xr.openvr.standalone",
            "com.unity.xr.windowsmr.metro",
        };

• What features are required to be replaced?
• What is a good replacement?
• How/when will replacement be done?

Unity XR Validation

WIP Checklist to validate all functions and permutations of Open Brush in the xr_v2 branch before we consider merging.

#237

Platform Support

Basic test, can we build and run on this platform that was supported previously?

• Windows SteamVR - now delivered via generic Windows OpenXR
• Oculus Rift
• Oculus Quest
• Vive Desktop (Same as Windows SteamVR for now)

Controller schemes

Check that inputs are valid and controllers act correctly

• Vive wands
• Valve Index (Knuckles)
• Oculus Rift
• Oculus Quest (also via Link)
• WMR

Panels

Note that a lot of panels have a 'Mobile' version that may behave differently.

• Admin Panel
• Admin Panel Mobile
• App Settings
• App Settings Mobile
• Brush Picker
• Brush Mobile
• Camera
• Camera Path
• Color Picker
• Environment
• Extra Panel
• Extra Panel Mobile
• Guides
• Labs
• Labs Mobile
• Layers
• Lights
• Memory Warning (see #326)
• Reference
• Reference Mobile
• Scripts
• Sketchbook
• Sketchbook Mobile
• Snap Settings
• Tools Basic
• Tools Basic Mobile
• Tools Advanced
• Tools Advanced Mobile
• Tutorials

Minor

Things that may or may not work, but shouldn't prevent release.

Platforms

• Windows SteamVR Experimental
• Windows Monoscopic Experimental (Minor) We may need to reconsider supporting this mode temporarily with the move to UnityXR. It's not a primary release method and I don't want it to hold up main platforms.
• Oculus Rift Experimental
• Oculus Quest Experimental
• Linux SteamVR - may best be supported through Proton
• Linux SteamVR Experimental - may best be supported through Proton
• Linux Monoscopic (Minor)
• macOS Monoscopic (Minor)

Controller schemes

• Vive Cosmos (There's prefabs for this, but they look rather unfinished.)
• Logitech Pen (Minor)

Panels

• Brush Experimental (Minor)
• Experimental (Minor)
• Poly (Deprecated)
• Poly Mobile (Deprecated)

URP support?

Hi,

Any plan to support URP rather than only Standard?

Thanks!

Hot-reload reference media

From @Zandy#1492 on Discord:

If you have a reference photo and you save it as “photo” and replace it with a pic using the same name the original photo remains. You have to delete it and restart open brush and reload the photo for it to finally show.

https://discord.com/channels/783806589991780412/804677406572150814/820278922947198997

Cannot record camera paths

Steps to reproduce:

• Select the camera path tool
• Draw a camera path
• press the record button

Expected result: Your camera path plays back and records
Actual result: Nothing happens

Update OpenBrush to use Unity XR plugin system

Unity 2020.x deprecates its VR support in favour of a new XR plug-in framework. OpenBrush is currently on Unity 2019.4 LTS. In order that we can keep up with the latest features in Unity, and stop the code from going stale we will need to update the VR SDK, so that we can later move to Unity 2020, 2021, etc. This issue just addresses the XR plug-in work.

This work is being done in the feature/xr_plugin branch. It is hoped this work will be completed with 2 months.

Notes about moving to Unity XR plugins. There are some complications here. See #130.

• We can move to XR plugin framework to support some vendors and we can stick on SteamVR plugin to support others.
  • We can stick on Unity 2019.4 LTS for this.
• When we move to Unity 2020.3 we can use the Unity OpenXR XR plugin.

Distortion and artifacts in 360 offline renders

From @Zandy#1492 on Discord:

Captured a 360 from the latest release but parts of the 360 get distorted and there is a circle at the nadir that shouldn’t be there. Also parts get distorted and disappear when they shouldn’t... cool effect for this one but prefer it without. Almost like the 360 stills are too short lengthwise creating a nadir circle.

https://discord.com/channels/783806589991780412/806934697237938216/820443466197631007

Offline render mode option 5 example: https://youtu.be/uLqBoXu5mXY

Loading Sketches is broken in experimental Quest builds starting from 0.2.26

First reported in Discord, starting at https://discord.com/channels/783806589991780412/806934697237938216/837376402177261588

This revert mikeage@e376a51 seems to fix it, although I'm not sure why exactly (yet) .

SketchFab Uploads from open brush as tilt brush.

Inside the CreateModel() in Assets/Scripts/Sharing/SketchfabService.cs, we can also pass an openbrush tag, and change the service to openbrush, not sure how we would change the logo yet.

PanelManager.IsPanelUnique

There's some logic here that feels like it should be inferred or derived from a flag on the panel itself. Hardcoded special cases are going to be overlooked when new panels are added.

Add multi-language support?

Sketches using the new "Plane" guide won't open in earlier versions

ArgumentException: Plane TiltBrush.WidgetManager.GetStencilPrefab (TiltBrush.StencilType type) (at Assets/Scripts/WidgetManager.cs:993)

The problem is here:

       public StencilWidget GetStencilPrefab(StencilType type)
        {
            for (int i = 0; i < m_StencilMap.Length; ++i)
            {
                if (m_StencilMap[i].m_Type == type)
                {
                    return m_StencilMap[i].m_StencilPrefab;
                }
            }
            throw new ArgumentException(type.ToString());
        }

m_StencilMap is serialized in the main scene as part of WidgetManager on the SketchControls gameobject. Even though older versions contain an enum entry for "Plane", it's not one of the serialized values on WidgetManager and therefore fails the check in GetStencilPrefab. And rather than failing gracefully - the loading of the sketch halts at this point.

Crash when SteamVR menu opens in Linux

When I open the steamVR dashboard while in Openbrush, the HMD and Openbrush freezes entirely until I force quit Openbrush. When i have killed Openbrush, Steamvr won't close properly and there is a background process left over from openbrush that i have to manually kill too.

Sytem info:
Openbrush version: 0.3.0, 0.3.12 and 0.3.24
SteamVR: v1.14.16
HMD: Valve Index
OS: Pop!_os 20.10
GPU: Radeon RX 6800XT
CPU: Ryzen 5 5600X

Old version of the Oculus Integration package

Open brush uses version 1.41.0 and the current version is 23.1.

Updating the Oculus Integration package can help solve some issues with the Mixed Reality capture on the Oculus Quest 1 and 2 (example). The new version can also open some possibilities, such as hand tracking, higher frame rates, and etc.

Fix xr_v2 OpenXR Android

The xr_v2 branch has a broken Android build. Notes on fixing.

The build appears to compile and link without error. It can be installed on a device but there appears to be no launch activity, so it it just hangs on the loading screen. The expected activity is com.unity3d.player.UnityPlayerActivity .

Multi User Support

Hello,
im thinking about adding networking to openbrush, probably using mirror networking or something easier like osc for starters.
How one could tackle synchronizing the painting area? By recording brush moves and passing them as a series of vectors with params or as a obj string?

Saving issue on Quest 2 for newer Android SDK versions

We previously attempted to upgrade the target Android SDK to version 29, as this was the minimum targeted version shipped with Unity 2019.4.18. However this created an issue with saving on Quest 2, where it does not have the correct permissions to access the Open Brush/Tilt Brush folder due to an implementation change on higher Android SDKs.

A temporary fix was proposed in #18 using a legacy option in the android manifest file, but I feel we should take the time to implement it correctly instead, as this fix is already obsolete in the latest version of the Android SDK.

Video export should include a skybox

Reported by Danny Bittman#1104 on Discord:

So it'd be AMAZING if the video exported could be updated to include a skybox export feature. Right now it's so clunky to make a skybox scene, bring it into Unity, get things 1:1, then export. But it'd be so amazing if you could open the console, and just select a single frame 8k Mono export

https://discord.com/channels/783806589991780412/803681768049737799/818602178247786531

Disabling Post-processing effects on cameras via config file disables camera preview

Steps to reproduce:

• Edit your Open Brush.cfg config file to disable PostEffectsOnCapture. Example:

{
  "User": {
  },
  "Video": {
  },
  "Flags": {
    "PostEffectsOnCapture" : false,
  },
  "Export": {
  },
}

• Load Open Brush.
• Activate the camera tool.

Expected Result: Can see camera preview on each tool, with post-processing effects disabled.
Actual Result: No preview visible.
Platforms reproduced on: Windows 10, Unity editor, Release and Experimental built versions.