inria-chile / grace Goto Github PK

All persistence needs should be compatible with either SQLite, PostgreSQL or Elasticsearch. The proposed structure is that of a mapping between types (e.g. TemplateGraph.t, TemplateGraph.Node.t, etc.) and a dispatch-based function for each CRUD requirement.

let save: (Settings.t, TemplateGraph.t) => Lwt.t(TemplateGraph.t);
let read: (Settings.t, string) => Lwt.t(option(TemplateGraph.t));
let delete: (Settings.t, string) => Lwt.t(unit);

All these should be implemented at the service level, with general helpers located elsewhere.

The query part of the work API, comprising queries to the domain graph, the timeseries/template graph, and the time series data itself.

Tests and docker build jobs should be automated for the master branch and tags. Ideally, the test jobs are also turned into preconditions for PR approval.

A data packet or "activation message" is a simple discrete event in the data stream, which may contain events for different time series.

The PRs addressing this should explore the possibility of buffering the incoming packets much like the aggregation filter in logstash does:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html

The work API must provide mechanisms for posting derived events and to report an early completion of the job itself.

A template graph should be definable as a DAG over a domain, mapping nodes to domain entities and edges to domain relationships.

Each node should be defined in terms of the simplest schema, leaving application-specific fields as metadata. A non-exhaustive list of mandatory node fields is:

• the canonical name of the node
• the human-friendly name of the node
• a human-friendly description
• the domain entity query for the scope of the node
• the set of node inputs and the respective domain relationships to use when building edges
• the formula, script, or hook used for the actual computation (more on this later)

A basic test suite should be runnable via dune runtest.

Domain entities and relations should adhere to the implied schema given by the template graph.

A domain entity or relation modification should trigger a re-building of the materialized time series graph (the real computation graph). This might end up being too resource intensive (to keep the computation graph in sync with the domain graph), in which case an alternative deprecation and forced update mechanism should be sketched.