xingkaij / redis-janusgraph-storage-backend

Redis Storage Backend for JanusGraph

Redis Storage Backend for JanusGraph

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.

janusgraph-redis is an initial and non-optimal implementation of a Redis JanusGraph storage backend.

Getting started

See janusgraph-examples’s README for instructions to build and run the example adapted from JanusGraph’s.

Features

• Unordered Scan
• Ordered Scan
• Multi Queries
• Batch Mutations
• Optimistic Locking

Does not currently support locking nor transactions.

Implementation

Implements KeyColumnValueStorageManager and therefore KeyColumnValueStore, and uses Lettuce to interface with Redis.

Employs a key-indexed HASH (KCV covering index), a column-indexed SET (CK covering index), an all-keys covering index SET, and an all-columns covering index SET with internal Redis transactions—to ensure consistent updates—and asynchronous index (all-keys and all-columns) deletions.

getKeys(KeyRangeQuery query, StoreTransaction txh) and getKeys(SliceQuery query, StoreTransaction txh) are implemented with SSCAN on the all-columns index SET with late-filtering rather than pre-filtering (range scan or range read with ordered values).

More efficient iterations may want to drop the all-keys covering index, use ZSETs instead of SETs for both the all-columns covering index and the CK covering index with ZRANGEBYLEX-based in-Redis pre-filtering instead of in-app late-filtering with matches(StaticBuffer start, StaticBuffer end, StaticBuffer item)—take into account that start is inclusive and end is exclusive. SCAN and derivates should not be used as MATCH does not support byte comparisons and does not pre-filter.

Currently a KeyIterator calls HGET synchronously on next() instead of prefetching results—consider alternatives.

Also, distributed locking and Redis transactions extended to the KeyColumnValueStorageManager may be necessary or desired.

Purpose

The original goal was to extend JanusGraph to support Redis Labs’ CRDB in order to have a geo-distributed alternative to the wonderful RedisGraph by Roi Lipman and contributors.