sensics / osvr-rendermanager Goto Github PK

This would enable a custom-built RenderManager library to be used with them, but would add another DLL that had to be used for each setup. David Ryskalczyk suggested this on the OSVR-Core Gitter channel.

I'm trying to build OSVR_RenderManager and I've successfully created the solution using CMake, however when I build I get several errors about being unable to find '..\bin\Debug\osvrRenderManagerd.lib'

The RenderBufferD3D11 pointer (openResults.buffers.D3D11) looks to be valid but its contents are uninitialized. It's member pointers (e.g. colorBuffer) are neither NULL nor valid.

When ATW is used with a long maximum time before vsync and with vsync-blocks-app, the ATW thread will lock the render buffers from that time until vsync has finished. Strictly speaking, it would only have to lock it until the rendering is done not until after present is done. Figure out how to make this happen without torquing the interface too much.

The main getting started git OSVR-doc page links to https://github.com/OSVR/OSVR-Docs/blob/master/installing/RenderManager.md after clicking the "Install and test RenderManager" hyper link gives 404.

I think this is the page it should link to: https://github.com/sensics/OSVR-RenderManager

See line 473 of RenderManagerD3D11ATW.h. @todo there talks about the fact that we will make multiple copies of the incoming buffers if the client passes the same buffer in for more than one eye. We should keep a mapping so we don't duplicate them.

The distortion-mesh calculations in ComputeDistortionMesh() produce a set of vertices for triangles, which means that there are about 6 times as many vertices as needed. This slows down the calculations of unstructured mesh inputs by calling DistortionCorrectTextureCoordinate() 6 times as often as needed.

Another approach would be to return both a vertex vector and an index vector and then use index buffers to specify the vertices multiple times. This will require updates to RenderManagerD3DBase.cpp and RenderManagerOpenGL.cpp to make use of the indexed mesh types.

Either pull the Render() path out into a separate set of classes that harness a RenderManager without them or do the buffer construction only when Render() is called the first time.

Do this once the issue dealing with not double buffering is complete.

While various OS-level APIs may be used to retrieve the current display configuration (e.g., resolution, rotation) when in extended mode, the displays are not visible to the OS in direct mode. RenderManager should provide a cross-platform, unified API for retrieving display configuration information in direct and extended modes.

(The set of display configuration parameters required by SteamVR are show in the Display struct at https://github.com/OSVR/SteamVR-OSVR/blob/master/src/display/Display.h#L101).

I had issue with linking osvrRenerManager in debug mode:

1>------ Build started: Project: osvrRenderManager, Configuration: Debug x64 ------
1>jsoncpp.lib(json_value.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in RenderManagerBase.obj
1>jsoncpp.lib(json_value.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in RenderManagerBase.obj
1>jsoncpp.lib(json_reader.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in RenderManagerBase.obj
1>jsoncpp.lib(json_reader.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in RenderManagerBase.obj
1>jsoncpp.lib(json_writer.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in RenderManagerBase.obj
1>jsoncpp.lib(json_writer.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in RenderManagerBase.obj
1> Creating library D:/knihovny/OSVR RenderManager/build64-msvc12/bin/Debug/osvrRenderManagerd.lib and object D:/knihovny/OSVR RenderManager/build64-msvc12/bin/Debug/osvrRenderManagerd.exp
1>LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library
1>D:\knihovny\OSVR RenderManager\build64-msvc12\bin\Debug\osvrRenderManagerd.dll : fatal error LNK1319: 6 mismatches detected
========== Build: 0 succeeded, 1 failed, 3 up-to-date, 0 skipped ==========

Solution - wrong version of jsoncpp.lib was being linked. It should be the debug version of the library and not release. When I updated the project configuration in Visual Studio manually, it linked just fine. I recommend adding Jsoncpp_DEBUG_LIBRARY entry in CMAKE and using that library for debug builds.

If you pass in a NULL GraphicsLibrary structure, we have a report of this crashing in D3D from mdutton3 on the OSVR/OSVR-Core Gitter channel.

After multiple connections from RenderManager, attempts to open a ClientContext will fail.

[OSVR] Connecting to default (local) host
[OSVR] Client context initialized for RM_LockUp
[OSVR] Got connection to main OSVR server
[OSVR] Connection process took 1000ms: have connection to server, don't have path tree
osvrClientCheckStatus failed

This bug may or may not be in RenderManager or OSVR-Unreal. Putting it here for now until I can track it down more precisely. This crash occurs in OSVR-Unreal when exiting VR Preview mode

Here is the log:

D3D11: Removing Device.
[2016.06.03-16.17.25:736][460]LogBlueprintUserMessages: Late EndPlayMap Detection: Level '/Game/Maps/basic.basic:PersistentLevel' has LevelScriptBlueprint '/Game/Maps/basic.basic:PersistentLevel.basic' with GeneratedClass '/Game/Maps/basic.basic_C' with ClassGeneratedBy '/Game/Maps/basic.basic:PersistentLevel.basic'
Exception thrown at 0x00007FFB8C9D1F28 in UE4Editor.exe: Microsoft C++ exception: _com_error at memory location 0x000000A1DBD0C098.
Exception thrown at 0x00007FFB8C9D1F28 in UE4Editor.exe: Microsoft C++ exception: _com_error at memory location 0x000000A1DBD0DB00.
Exception thrown at 0x00007FFB8C9D1F28 in UE4Editor.exe: Microsoft C++ exception: _com_error at memory location 0x000000A1DBD0DE30.
[2016.06.03-16.17.25:774][460]LogD3D11RHI:Error: Direct3DDevice->CreateBuffer(&Desc, NULL, UniformBufferResource.GetInitReference()) failed 
 at D:\BuildFarm\buildmachine_++UE4+Release-4.11\Engine\Source\Runtime\Windows\D3D11RHI\Private\D3D11UniformBuffer.cpp:159 
 with error DXGI_ERROR_DEVICE_REMOVED

I call the new clear color API when the game exits VR Preview mode to clear out the HMD. Without that call, everything works ok. With the call, the above crash occurs. This might be a bug in OSVR-Unreal instead of RenderManager, but I'll put it here for now.

I'm having trouble getting the Unity anti-aliasing quality setting to affect what is shown on the HDK. It seems like the quality setting is only affecting what is displaying on the screen but not the HMD. e.g. I can up the anti-aliasing quality to x8 and it looks nice on screen, but still looks disabled on the HMD.

You can see from these screen grabs that the image used for the HMD seems unaffected by the Unity setting, but the uncorrected preview image looks much better with anti aliasing level set to x8

I'm running:
Windows 8.1 64-bit
NVidia GTX650 with driver 362.00
Unity 5.3.1f1 Personal.
OSVR Plugin for Unity 1.7
OSVR-Core-Snapshot-v0.6-1144-g9c9092f-build250-vs12-64bit
Razer HDK 1.3

When I try to build renderManager using cmake, it has the following error 👍
CMake Error at CMakeLists.txt:22 (find_package):
Could not find a configuration file for package "osvr" that is compatible
with requested version "".

The following configuration files were considered but not accepted:

D:/SW/OSVR-Core-Snapshot-v0.6-1169-ge9235a4-build252-vs12-64bit/OSVR-Core-Sn

apshot-v0.6-1169-ge9235a4-build252-vs12-64bit/lib/cmake/osvr/osvrConfig.cmake, v
ersion: 0.6 (64bit)

I download the osvr core 0.6.1169 from http://access.osvr.com/binary/osvr-core

The RenderManager config file is only set up to hold parameters for a single window. The "window" features other than fullscreen should be duplicated for each display. Then pass these into RenderManager and deal with them rather than the hard-coded values used now.

I've seen a todo somewhere in the code about that but I don't find it right now. Anyway, it seems that it's hardcoded that only APPLE gets a core context:

That's especially bad for mesa because it supports anything above OpenGL 3.0 only with a core context.

Would there be any harm in using a core context on every platform?

JeroMiya posted: There is a lot of duplicated code in RenderManager that is now included in the OSVR-Core DisplayConfig API. From display config parsing, to getting head/eye poses to calculating viewports to calculating projection matrices. This code in RenderManager should all be replaced with Core functionality.

As part of this issue, also do a code review of the duplicate/obsolete code for any functionality that is not already included in Core, and move the implementation into core where appropriate.

For example, see this issue to add support for overfill factor in the display config API:
OSVR/OSVR-Core#316

I think there is also some distortion properties in the display config that aren't currently parsed by OSVR-Core's DisplayConfig. These would need new API in core as well.

Russ Taylor posted: I'm adding an Oversample flag to go with Overfill. They will basically multiply each other in some parts of the code. Not sure how this is going to play with the proposed multi-function viewport approach. Maybe we want a pair of optional parameters, each of which defaults to 1.0, rather than four different function names for with and without overfill and oversample...

Some of the source files in this repo(and apparently a few in vrpn) have inconsistent line endings, a quick grep refined these targets to have CRLF:

CMakeLists.txt
examples/CMakeLists.txt
examples/D3DCube.h
examples/font.c
examples/font.h
examples/pixelshader3d.h
examples/pixelshader.h
examples/vertexshader3d.h
examples/vertexshader.h
installer/directmode_dSight.json
installer/directmode_HDK1.2.json
installer/directmode_HDK1.2_landscape.json
installer/directmode_HDK1.3_built_in.json
installer/directmode_HDK1.3.json
installer/directmode_HDK1.3_mesh_client.json
installer/directmode_HDK1.3_mesh_server.json
installer/directmode_Vuzix.json
installer/distortion_nondirect_HDK1.2.json
installer/nondirectmode_dSight.json
installer/nondirectmode_HDK1.2.json
installer/nondirectmode_HDK1.3.json
installer/nondirectmode_HDK1.3-landscape.json
installer/nondirectmode_window_angledistort.json
installer/nondirectmode_window_angletest.json
installer/nondirectmode_window.json
installer/README_build.txt
installer/README.txt
osvr/RenderKit/RenderManagerBase.cpp
osvr/RenderKit/RenderManagerD3D11ATW.h
osvr_server_config.RenderManager.sample.json
vendor/CMakeLists.txt
vendor/vrpn/submodules/jsoncpp/devtools/agent_vmw7.json
vendor/vrpn/submodules/jsoncpp/devtools/agent_vmxp.json

Command used to generate this list:

find . -not -type d -exec file "{}" ";" | grep CRLF | awk -F ":" '{ print $1 }' | sort -u | grep "\.[c,t,h,j][p,x,s]*[p,t,o]*[n]*$" | cut -c3-

The Heads-up display layer will have distortion-correction applied to it, but will not have asynchronous time warp applied -- it is rendered in a space that is at a fixed location with respect to screen space. This will require at least one second layer to be handled by the API.

Also, the HUD image may be updated at a different rate from the world geometry. Maybe faster, maybe slower. Perhaps this should be a whole separate channel in the API that is optional-use.

This may also be used to do foveated rendering, where we do low-resolution background with a high-resolution foreground.

Consider the case where the different layers are not all updated at the same rate. The background layer may be updated less frequently than the fovea later for foveated rendering (different time warps for the different layers). The HUD layer may be updated much less frequently than the other layers.

Core uses left, right, top, bottom rather than width and height.

Look at whether this is a breaking change for clients and heavily warn if so.

I assumed that the GetNumRenderInfo() call was going to cache all of the info and then later calls would read it out.  It is important that all of the info is read out in one call, so that rendering to all eyes and spaces is consistent.  However, it looks like osvrRenderManagerGetRenderInfoImpl() actually re-reads it every time (which explains why it needs to be passed renderParams) and then pulls out its index.  This will cause funky dynamic distortions which will make people sick.

We could make a separate Latch() call if that is better.

This didn't happen at Sensics when I tried with same laptop and HDK.
It happened at home both before and after.
Not sure if this is related to config-file settings (vsync timing and such).

Commit in question is here: 20e8e24

The underlying concern is that there is a bunch of processing that needs to happen (in particular, enumeration of which devices support direct mode) in subclasses that support direct mode, before we can reliably have a IDXGIAdapter in the code around 20e8e24#diff-c1563522d9fcfcc122b7be39a675d87eR166 - because on a multigpu system, the one that Windows hands you by default might not, and probably isn't, the right one for direct mode. Thus, direct mode subclasses, who can actually find the right adapter to correspond to the display, call setAdapter() before calling this base class openDisplay().

I agree that the class design here is a bit funky and could use refactoring, (and I am a bit surprised there wasn't a warning that you had a "conditional expression is constant") but that particular move was a regression.

Our team have been playing with OSVR-Unreal integration for 2 days, and encountered an significant issue where the rendered stereo images do not align in our HMD, the same object in left eye and right eye are too far away to create a combined stereoscopic view.
The same issue actually appeared when we're evaluating OSVR-Unity the first time, after we adjusted the center_proj_x setting to 0.6/0.4 according to the specs of our HMD, the views align perfectly.

Now with the same settings, they do not align any more in OSVR-Unreal, so I started messing with some parameters and capture the output from both Unity3D and UE4, and found out the difference.

Firstly let's set the distortion to identity and set center_proj_x to a extremely biased value (for example 0.8/0.2).

Output from Unity3D:

Output from UE4:

It's not very obvious what's wrong, let's add some odd distortion to visualize the 'center of projection'.

Output from Unity3D: (you can see that when we put the boxes at the COP in left eye, the same boxes also appear at the COP in right eye, this is perfectly aligned to someone with a super-narrow IPD :)

Output from UE4: (when boxes is put at the COP in left eye, the same boxes is way off in right eye)

I hope this may give some lead.

P.S. Later I messed around with OSVR-Oculus-Rift plugin and a DK2, no matter what value is given to center_proj_x, the output looks great in UE4 (but unaligned in Unity3D unless center_proj_x is set to 0.5/0.5), I suppose it's because the actual physical COP of DK2 is exactly 0.5/0.5 where the output of OSVR-Unreal perfectly matches this metric. And another question arises, I see that HDK 1.3 is using a 0.471/0.529 COP setting from the sample config, so does it look correct in UE4?

Sorry for this rather unrelated question but I would like some second opinion on the rendering experience.

I have tried the various examples and stepped through the persistence modes and notice that there is (still) some heavy motion blur.

It "feels" that the low persistence rendering is even worse, since the contrast are weaker and the eye seems to do its perception magic even better.

So, is there a solution to the blur or are 60Hz just insufficient?

If an application deletes one RenderManager and constructs another one (at least in Extended mode), SDL_Init() and SDL_Quit() are called multiple times in the application. We should only call these once per program, not once per construction/destruction.

Getting this error when trying to open Vive in Direct Mode:

HTC_Vive_Pre_meshdata.json is located in the displays folder, another copy is next to osvr_server.exe, and also in C:/OSVR/
where should the meshdata file live?

Jeremy reports: I'm occasionally getting osvrRenderManagerGetNumRenderInfo return 0 render infos with this config. Not all the time.
osvr_server_config.zip

Applications using geometry shaders to duplicate their geometry often render to different layers of the render buffer. While it is quite simple to just copy each layer to a separate texture and present those to the OSVR-RenderManager, it would be nice to be able to present this render buffer without any extra work.

While I do use this technique in the Dolphin Emulator, I do not require this feature as our post-processing system will separate the layers anyway.

The best example of this that I know of is trying to run Showdown on a machine with a 1GB GPU and numBuffers = 2. That app is high on its GPU memory usage, more than its processing usage. When you turn numBuffers = 2, presumably some buffer allocation at some level of the stack (quite possibly not RenderManager) fails, and you get a flickering/twitching display. Drop it back down to numBuffers = 1, and the total system gpu mem usage pulls back under the amount onboard the GPU, and it's smooth(er) again.

I know that shipping a numBuffers = 2 config file is impending and better for the majority of users that probably have a better graphics card with more than 1 GB of VRAM, but some feedback would be useful for those lower-end cards, especially since they can run plenty of things fast if they aren't running what's essentially an art and rendering tech demo (Showdown).

HI,

I want to compile the examples using direct-mode libraries.

by now I have a solution for the "osvrRenderManager0.6.38" that is missing the renderkit sources and the rendermanager project from this page, which uses vrpn and thus no direct mode.

It was hinted at that I can compile from source and copy the precompiled binaries.

Can someone please provide a short overview what to build and what to copy where?

Best Regards

This would behave like PresentRenderBuffers(), but would instead clear all screens to black. This is utility function to help in situations like Unreal or Unity when you have an asynchronous call when editors start/stop or scenes switch and you can't handle doing it in the standard rendering loop.

Probably a virtual function that gets implemented in the OpenGL and D3D base classes.

The current implementation of LiquidVR Direct Mode looks for the amdlvr32 DLL and if it can successfully open the DLL, it returns true for AMD. This causes issues if the DLL is present but there is no AMD card, or one that does not support LiquidVR.

Ideally this could be fixed at the same time as #33.

The build improvements just merged from the pull request break the finding of jsoncpp on my system. I have the following CMAKE_PREFIX_PATH:

F:/Packages/nVidia/VR_SDK_under_NDA/2016-02-02-gameworks-vr-for-headset-developers-1.2/direct_mode/DirectModeSample;F:/Packages/SDL/SDL2-2.0.3;F:/Packages/OpenCV/opencv-2.4.10/build;C:/Program Files/jsoncpp;C:/Program Files/Microsoft SDKs/Windows/v6.0;F:/taylorr/Personal/Work/consulting/Sensics/OSVR/src/OSVR-Core/vendor/eigen;F:/Packages/AMD/DirectMode_API_under_NDA/LiquidVR_10_0_3_19/LiquidVR_SDK_D2D;C:/usr/local

There are jsoncppConfig.cmake and -debug, -release, and -relwithdebinfo.cmake in the directory
C:\Program Files\jsoncpp\lib\cmake\jsoncpp but the library is not found. C:\Program Files\jsoncpp\include\json has the header files.

When we host an OpenGL render manager on top of a D3D one, the SDL events for the window being displayed don't get sent to the handler function in D3D.

If we use just the D3D renderer, they do.

Probably has to do with the hidden window that OpenGL opens to initialize GLEW.

Neither of the windows get these events when we open two of them in D3D mode (num_displays = 2).

It looks like OpenGL on the mac wants to use both different include files and different SDL parameters to open OpenGL Core (modern) vs. OpenGL Legacy (older). This looks like it may force two different RenderManager approaches on the mac. It may be that we can handle it by passing a flag to the routine that calls SDL, letting it know which interface to expose to the user code.

In the meantime, we have the problem that we can't compile the Core-based shader program in RenderManagerOpenGL.cpp, which looks like it is because we're not using Core in the code.

The first step will be to implement a Core-only path and be able to run the Core-profile example programs on the Mac. We can disable building of the Legacy-profile applications on the mac for now.

http://stackoverflow.com/questions/20931528/shader-cant-be-compiled

http://stackoverflow.com/questions/11961116/opengl-3-x-context-creation-using-sdl2-on-osx-macbook-air-2012

https://www.reddit.com/r/gamedev/comments/27c93z/os_x_opengl_corecompatibility/

JsonCpp will not compile / generate out of the box using the latest CMake.

Using some external Jsoncpp with Cmake structure will complain about operator=() reference.

Some immediate directions would be highly appreciated.

Not sure what has to happen here. Seems like we just need to create the context ourselves like normal for and D3D RenderManager. Once the ticket letting us not have to double-buffer is completed, we can test this using one of the standard D3D present applications.

This may mostly have been fixed already when refactoring the ATW class. It may still be disallowed in the createRenderManager() function, but it probably should work.

In this case, we need to make our own copies of buffers and then pass them on to the harnessed RenderManager. In this case, we need to basically copy or render from buffers in the render thread context to buffers in the ATW thread. I don't know if this will require GPU locking on the buffers or not.

Also should not try if GLEW is not found.

Initially do this with the version of ATW that does not need double buffering, so we're changing the fewest things at once. We'll need to wrap the OpenGL around an ATW, which is itself wrapped around the appropriate DirectMode interface in createRenderManager.

For optimal performance, the OpenGL wrapper should do double buffering in this case. It can detect this case by looking at whether it is in ATW.

I'm a little bit confused about how the OpenGL context should be shared with the RenderManager. I've already created an OpenGL context for my application. Will I be able to share that context with the RenderManager or should the RenderManager be responsible for creating the context?

Preferably I could share the OpenGL context with the RenderManager and still render to my own window normally.

It seems that if Overfill factor is too low, textures are clamped, causing a visual artifact at the edges. It would be better not to clamp the textures and just display black where there is no image.

This requires changing the texture-display settings/approach in the final rendering pass done in both the RenderManagerOpenGL.cpp and RenderManagerD3DBase.cpp files. They should be modified to render using a border rather than texture clamping and that border should be black.

This line is the cause:

With make install, the libraries
libosvrClient.so.0.6
libosvrClientKit.so.0.6
libosvrCommon.so.0.6
libosvrUtil.so.0.6
will be copied back to the install directory (/usr/lib), where they most likely already are.

I've removed the line and it still builds successfully. Is this necessary on other platforms?

Just like RenderManager can export OpenGL and Direct3D matrices, it would be nice for it to be able to export Unreal-formatted matrices (correct orientation, correct distance scaling). This would be another export type.

Both the C++ and C API would need to have this added.