ilya-kolchinsky / opencep Goto Github PK

As of now, nested ANDs/ORs inside a SEQ do not work properly since SeqNode validates the order of all events.

OpenCEP should also support count-based windows in addition to the currently supported time-based windows.

Provide an implementation of a multi-pattern graph construction method utilizing the local search approach from the SIGMOD '19 paper.

Implement a new cost model estimating the expected average match latency.

Implement a way to add/modify/remove pattern definitions on-the-fly.

Add support for aggregations in Kleene closure conditions.

Implement the task-based parallelism approach for multi-pattern workloads where different pattern subsets are assigned to the parallel execution units.

As of now, we have Node.apply_condition to that end - that should be made obsolete.

The same should be applicable for any "heavy" operator.

Implement a cache for predicate evaluation results.

Provide an implementation of an evaluation tree building algorithm pushing negation operators into the positive subtree.

Implement the structural parallelism mode where an evaluation graph is divided into subgraphs and each one is executed in a separate parallel unit.

The project documentation should be reworked to improve readability and usability. The explanations should be made more concise, and the examples should be simpler and to the point.

Extend the existing multi-pattern model to support sharing identical or similar subexpressions within a single pattern.

As of now, the only implemented parallelization platform is based on the threading library, hence there is no true parallelism.
An alternative platform allowing true multi-process execution has to be added.

This behavior should be fixed.

Augment the multi-pattern CEP mechanism with a way to specify and enforce priorities between the different patterns and/or constraints on the performance metrics / plan optimality.

Implement a mechanism for supporting uncertainty in the values of event attributes.

The former should be (safely) removed.

Add support for 'return' clauses in the specified patterns, i.e., returning an arbitrary transformation on a pattern match rather than the match itself.

The following simple pattern throws the exception on line 136 in merge_according_to function (Utils.py).

pattern = Pattern(
SeqOperator(
PrimitiveEventStructure("blue", "a"),
KleeneClosureOperator(PrimitiveEventStructure("red", "b"))
),
TrueCondition(),
timedelta(minutes=5)
)

The input is the following (I am leaving out timestamps for clarity):
blue,1
blue,2
red,1
red,2
red,3

Note that the exception is not thrown if I change SeqOperator to AndOperator but I believe SeqOperator should work as it follows the example in the OpenCEP documentation (shown below):

pattern = Pattern(
SeqOperator(
PrimitiveEventStructure("GOOG", "a"),
KleeneClosureOperator(PrimitiveEventStructure("GOOG", "b"))
),
AndCondition(
SmallerThanCondition(Variable("a", lambda x: x["Peak Price"]), Variable("b", lambda x: x["Peak Price"])),
SmallerThanCondition(Variable("b", lambda x: x["Peak Price"]), Variable("c", lambda x: x["Peak Price"]))
),
timedelta(minutes=5)
)

Any suggestions or is this a bug? Thanks.

Sharing sequences is now broken due to the way it works involving IDs. For example, the shared tree of seq(a,b) and seq(a,c,d,b) will not work correctly due to the second pattern being verified using the indices of the first one.
The correct implementation should only consider event names.

For example, by utilizing different output colors.

Allow for primitive events to contain a pair of timestamps marking the start and the end of an event. Support interval-based relations in pattern definitions.

Only an "internal" condition, i.e., solely involving the events under KC, can be defined as of now.

As of now, adding a new data type or a new testing aspects is a rather cumbersome process - this needs to be fixed.

Implement the hybrid parallelism approach (a generalization of SIGMOD '22).

When replacing an evaluation tree on-the-fly, identify the intersection between the old and the new tree plans and only replace the non-intersecting parts rather than making the entire old tree obsolete and building the new one from scratch.