lumapictures / usd-uri-resolver

Pluggable URI ArResolver for USD

USD-URI-resolver

A generic, URI based resolver for USD, support custom plugins.

Project Goals

• Resolve custom URI based for USD files.
• Provide a generic plugin registry to extend the resolver without modifying its source code.
• Include example resolvers supporting database based files.

Features

Current

• URIResolver - custom resolver for USD.
• usd_sql::SQL - MySQL database access.
• obfuscate_pass - A simple tool to convert passwords to a z85 encoded string. WARNING !!! This is not for encrypting your password, but to hide it from artists in an environment. It is extremely simple to "decrypt" and offers no protection.

Planned

• Create a plugin registry for extending URIResolver.
• Move the SQL accessor to a spearete plugin and load it dynamically.
• Windows support.
• Simplify the build process.

Building

You'll need the following libraries to build the project; newer versions most likely work as well, but they are not tested. Currently, the cmake files are looking for the openexr libs, but only use the half headers. This will be removed in the future.

Package	Version
USD	0.7.6+ (stock)
TBB	4.3+
OpenEXR	2.2.0+
Boost	1.61.0+
MySQL Connector C	6.1.9+
CMAKE	2.8+

You can download the MySql library here.

There are two main ways to configure a library location. 1, configure an environment variable. 2, pass the location of the library using -D<varname>=<path> to cmake. This will be simplified soon, once we add proper find modules.

• Point the USD_ROOT environment variable to the location of the installed USD.
• Point the MYSQL_CONNECTOR_ROOT variable to the location of the extracted MySql connector C library.
• Pass OPENEXR_LOCATION to the CMake command or setup the OPENEXR_LOCATION environment variable. They have to point at a standard build of OpenEXR, including IlmBase.
• Point TBB_ROOT_DIR}, TBB_INSTALL_DIR or TBBROOT at your local TBB installation.

Contributing

TODO.

Using the SQL resolver.

Consult the README.md installed alongside the URIResolver.

usd-uri-resolver's Issues

get_timestamp should use the cache, not query the database

Opening and reloading a stage applies these steps with database calls:

• resolve asset: check asset existence in db
• fetch asset: download asset from db if NEEDS_CACHING
• get timestamp: check asset existence again in db

Fetch asset always does an extra get_raw_timestamp check when the cache says that fetching is not necessary...
Because of this, I think that the get_timestamp call should just return the cached timestamp value without making yet another database query.

I think this line should be dropped: https://github.com/LumaPictures/usd-uri-resolver/blob/master/URIResolver/sql.cpp#L525

This reduces the database queries per unique asset from 3 to 2.

FindTBB

Add windows support.

USD supports windows officially, we need to do the same.

Stability issues with storing the cache in memory

There are some hard to reproduce issues using the new in-memory storage of fetched files from the database.

The initial focus is to create a sample application that reproduces the problem, as the current python test is unreliable to reproduce this issue. Possibly a c++ app generating large amount of request quickly would be the way to go.

Simplify build process.

We don't have generic find modules for everything, and setting up certain libraries is inconvenient.

Add build option to keep cache on resolver teardown

The destructor of the SQL context deletes all files from the cache.
Would it make sense to have an option to keep the local cache alive?
The fetch_asset function would then have to check the local cache.

It could be a bit tricky to compare the date modified between local files and the database version though.

An alternative is to serialize the cache on teardown and attempt to load it on init.

Create a plugin registry for multiple accessors

We need a generic plugin registry that can load up multiple accessors easily, and provide an interface that's flexible and easy enough to customize.

Move the MySql accessor to a plugin and make it optional to build.

We need to move the MySql accessor to its own plugin and separate it from the main resolver. This will help us design the plugin registry, and allow people to build without the MySql accessor.