dpapathanasiou / simple-graph

This is a simple graph database in SQLite, inspired by "SQLite as a document database"

About

This is a simple graph database in SQLite, inspired by "SQLite as a document database".

Structure

The schema consists of just two structures:

• Nodes - these are any json objects, with the only constraint being that they each contain a unique id value
• Edges - these are pairs of node id values, specifying the direction, with an optional json object as connection properties

The create, read, update, and delete functions (.sql files) are complete statements with qmark bindings.

Search templates (.template files) are in Jinja2 format, which can be converted to other template syntaxes relatively easily, with a bit of regex magic (though it would be nice if they could be expressed in a more language-agnostic way).

There are also traversal function templates as native SQLite Common Table Expressions which produce lists of identifiers or return all objects along the path.

Applications

• Social networks
• Interest maps/recommendation finders
• To-do / task lists
• Bug trackers
• Customer relationship management (CRM)
• Gantt chart

Usage

RESTful API (paid)

The Banrai Simple Graph Database and Document Store wraps this core logic with an API service, creating a managed graph database and document store, with additional features not found in any of the public bindings.

Importable library (free)

Choose an implementation:

• Python (now available in PyPI)
• Go
• Julia (courtesy of Josh Day)
• R (courtesy of Michael Silva)
• Flutter and Dart (courtesy of Rody Davis)

Want to contribute an implementation in your preferred programming language?

The schema and prepared sql statements can be used by programs in any programming language with SQLite bindings.

Pull requests are welcome!

simple-graph's Issues

Ambiguous nature of ID

Id's are numerical in the tests which JSONDecoder is happy with despite the unnecessary need to decode the IDs as JSON.

If you create a key as a string for instance this will work but will not be possible to decode as the json library in python expects strings to be explicitly quoted. Thus you cannot traverse or visualise if edges aren't double quoted strings.

Should I be double quoting my strings for IDs?

I'll post a coded example later on if this isn't clear

Find Node

Why do you use fetchall() and not fetchone()?

simple-graph/python/database.py

Line 112 in be41db2

read_sql('search-node-by-id.sql'), (identifier,)).fetchall()

Flutter support

Here is a flutter example i created and runs SQLite on all the platforms it exports to!

https://github.com/rodydavis/flutter_graph_database

Link to video

You may want to include a link to the demo video.

Feature request: Color properties for visualizations on Edges & Nodes

Would it be possible to add color properties for the visualizations of edges & nodes? Graphviz supports it for the python colors: https://graphviz.org/doc/info/colors.html

I'm not sure about the other languages then python but I'm sure I wouldn't be alone to want colors in graphs to ease their use when they become large.

Any plans for a deno driver?

No Traverse with TEXT ID

When I use IDs like meta6 the traverse is not working.

print(db.traverse(db_file_name, 'meta6', neighbors_fn=db.find_neighbors))

This only prints ['"meta6"'].

In the atomic(), I enabled tracing via:

connection.set_trace_callback(print)

and the query is incorrect:

PRAGMA foreign_keys = TRUE;
WITH RECURSIVE traverse(id) AS (
  SELECT '"meta6"'
  UNION
  SELECT source FROM edges JOIN traverse ON target = id
  UNION
  SELECT target FROM edges JOIN traverse ON source = id
) SELECT id FROM traverse;

Changing it to

PRAGMA foreign_keys = TRUE;
WITH RECURSIVE traverse(id) AS (
  SELECT 'meta6'
  UNION
  SELECT source FROM edges JOIN traverse ON target = id
  UNION
  SELECT target FROM edges JOIN traverse ON source = id
) SELECT id FROM traverse;

makes it work.

how do i return the ID of the saved vertice ?

is there a way to save a vertex and return it's id?

Multiple Node Connections in visualize

If I am having a graph with multiple nodes attached to a single node (e.g. node 7), the command db.visualize(graph, 'example.dot', db.traverse(graph, 7, neighbors_fn=db.find_outbound_neighbors)) only visualizes the first connection found in db.find_outbound_neighbors. Is there a way to automatically visualize all connections?

Traversing returns IDs as strings.

Is there any reason why that is? I'm asking because to visualize you need the IDs as integers.

Otherwise, thanks for doing this project. Love it!

SQL errors result in database being locked

Hi - very handy package!

I've been playing with it and Ive noticed that if I try to insert a duplicate node, it throws the expected IntegrityError. But after that, subsequent operations fail due to the db being locked.

Looking at atomic I think this is because the db is not being closed. I modified it to wrap it in try/finally and it seems to solve the problem.

def atomic(db_file, cursor_exec_fn):
    connection = None
    try:
        connection = sqlite3.connect(db_file)
        cursor = connection.cursor()
        cursor.execute("PRAGMA foreign_keys = TRUE;")
        results = cursor_exec_fn(cursor)
        connection.commit()        
    finally:
        if connection: 
            connection.close()
    return results

Thank You

Hey,

This isn't an issue, just wanted to say thanks for creating simple-graph!

I used it as the basis for a Julia package (https://github.com/joshday/SQLiteGraph.jl), since as far as I know, there's no graph database that has a Julia interface. I made some minor changes tailored to my specific use case, but I wouldn't have been able to put it together so easily without the help of your work, so thanks again!

Searching for nodes where query must match a value in an array in the body

I'm looking to do something like this

from simple_graph_sqlite import database as db 
dbfile = "db.sqlite"

db.initialize(dbfile)

db.atomic(dbfile, db.add_node({'name': 'foo', 'type':['company', 'start-up']}, 1))
db.atomic(dbfile, db.add_node({'name': 'bar', 'type':['cat', 'apple']}, 1))

db.atomic(apple, db.find_nodes({'type': 'apple'}, db._search_like, db._search_starts_with))

But I don't think the last statement works

Error initializing database

Context: MacOS Mojave, SQLite3, Python 3.8.5

The default sqlite3 binary in MacOS Mojave does not have the JSON extensions enabled.

Fix:
Install sqlite via brew
Add to shell initialization file: export PATH="/usr/local/opt/sqlite/bin:$PATH"