brettkromkamp / topic-db

TopicDB is a topic maps-based semantic graph store (using SQLite for persistence)

Brett Kromkamp's GitHub Profile

You've reached Brett Kromkamp's GitHub profile. Find me on Mastodon.

I am working on the next version of Contextualise, a personal knowledge management application. Knowledge management is a crowded space. My contribution, however, to the plethora of approaches and applications in this space is based on topic maps which are ideally suited for organising information-heavy projects and activities consisting of unstructured and widely diverse data and information resources.

I ask everyone to support the people of Ukraine who are being invaded by Russia. Donate now.

My latest blog post is Three.js Development in 2023.

I contribute to and maintain Contextualise, TopicDB and a couple of other open source projects. I also have several published Python packages on PyPI. Expect many of my projects to be related to knowledge graph-based applications combined with procedural generation together with 2D and 3D visualization systems.

Technologies I enjoy working with, include:

• Topic maps describing knowledge structures and associating them with information resources
• Python programming language
• FastAPI web framework for building APIs
• Angular web framework
• Flask web framework
• Blender 3D creation suite
• Three.js JavaScript 3D library

Contextualise Experiments

Knowledge Graph-based Content Management

Combining labeled property graphs with Contextualise, a topic maps-based content and knowledge management system.

Reveal.js-Based Knowledge Paths

Extending Contextualise to include a reveal.js-based presentation mode to supplement the existing knowledge path feature.

Mixing Art and Code to Build Engaging Web Experiences

Development of web content models, concepts, creation processes and tools: mix art and code to build interactive, engaging web experiences

Story Technologies

Storytelling With Interactive Visualisation of Semantic Events

Developing a combined Angular and Pixi.js-based web application —for storytelling purposes— that allows a user to navigate between a collection of narrative events. Each event has an accompanying 2.5D, isometric environment with participants and items that can be interacted with. Interactions include (but, are not limited to) displaying more information about the selected object and high-lighting a set of objects by tag. Narratives are Contextualise topic maps.

I will see you again, in the place where no shadows fall. — Ambassador Delenn, Babylon 5

topic-db's Issues

Investigate if SQLite should be used as a file system vs. saving BLOBs as files on the actual FS

See URL: https://www.sqlite.org/fasterthanfs.html

Add 'instance of' filter to GetTopicIdentifiers command class.

Currently, the GetTopicIdentifiers command class returns topic identifiers regardless of the type (i.e., instance of) of the topics. It should be possible to filter what type of topic identifiers should be returned. This is already relevant for StoryTechnologies' scene browser.

Implement a (simplified) query builder or language.

Currently, the different command classes (e.g., GetAssociations, GetAttributes, GetTopicsHierarchy) allow for retrieving topic map-related data. Nonetheless, it should be possible to return data for user-defined queries.

Asynchronous TopicDB

Refactor TopicDB to be asynchronous.

Relevant links:

• aiosqlite: https://github.com/omnilib/aiosqlite (and https://aiosqlite.omnilib.dev/en/stable/)

Replace manual cursor closing with contextlib.closing.

See URL: https://docs.python.org/3/library/contextlib.html#contextlib.closing

Replace the use of lists in the models with dictionaries where appropriate

Remove hard-coded maximum depth in GetTopicsHierarchy command.

Auto-generation of topic identifier using the (first) base name if no identifier is provided.

Implement support for multiple persistence stores.

Currently, the topic map engine uses SQLite as its persistence store. It should be possible to use different persistence stores (e.g., PostgreSQL).

Implementation detail: I will probably implement this using the strategy pattern.

Upgrade to Psycopg v2.9

See URL: https://www.psycopg.org/articles/2021/06/16/psycopg-29-released/

Collaboration: migrate from 1:M to M:M connections between users and topic maps

In addition, track permissions (i.e., can-view, can-edit and can-comment) and ownership on the connection between a user and a topic map.

Fix 'get_topics_network' method

Expected output:

elon-musk
         lyndon-rive
                 solar-city
                         solar-energy
         space-x
                 falcon-9
                 falcon-heavy
                 dragon
         tesla
                 model-s
                 model-x
                 model-y
                 roadster

Actual (erroneous) output:

elon-musk
         tesla
                 roadster
                         model-y
                                 model-x
                 model-s
                 space-x
                         dragon
                                 falcon-heavy
                         falcon-9
         lyndon-rive
                 solar-city
                         solar-energy

Order of the siblings is not necessarily relevant, but overall "shape" of the tree/graph is.

Method:

topic-db/topicdb/core/store/topicstore.py

Line 970 in 44af2ba

def get_topics_network(

Create Sphinx-based documentation

Add 'scope' property to BaseName class (plus serialisation and deserialisation)

Refactor GetAssociations command.

The GetAssociations command will be renamed to GetTopicAssociations (retrieve all associations for a given topic). And, the GetAssociations will become a command class to retrieve associations, optionally filtered by 'instance of' and 'scope'.

Rename 'story' table to 'topicmap'.

This will require refactoring of the StoryTechnologies Story Engine (https://github.com/brettkromkamp/story_engine) code base.

Set base topics 'instance_of' property to be 'BaseTopic' (instead of 'Topic')

Source:

• topic-db/topicdb/core/store/topicstore.py

  Line 2137 in 444856a

  def initialise_topic_map(self, map_identifier: int, user_identifier: int) -> None:

• topic-db/topicdb/core/store/topicstore.py

  Line 70 in 444856a

  self.base_topics = {

Annotate source code with (Mypy) type annotations (static typing)

Test TopicDB with Python 3.11.0b5

See URL: https://www.python.org/downloads/release/python-3110b5/

Review and improve tests

Current set of tests are whole insufficient: https://github.com/brettkromkamp/topic-db/tree/master/tests

The tests need to be reviewed (probably updated) and appropriately extended to ensure sufficient coverage.

Add initial pytest-based tests.

See URL: http://doc.pytest.org/en/latest/contents.html

Allow for importing * from packages

Examples:

• from topicdb.core import * (imports both commands and models)
• from topicdb.core.commands import * (imports all of the command submodules)
• from topicdb.core.models import * (imports all of the models plus the tree submodule)

Replace auto-incremental map identifiers with UUIDs

This change will have consequences downstream (i.e., Contextualise).