semiversus / python-broqer Goto Github PK

Any is implemented as CombineLatest and waits for all publishers to emit a value. Any should be True when any publisher evaluates to True.

If no init argument is used, the first emit should be used as init argument (without emitting a new state by itself).

>>> i = Value('Attribute access made REACTIVE')
>>> i.lower().strip(sep=' ') | op.Sink(print)

Traceback (most recent call last):
File "", line 1, in
File "site-packages\broqer\op\operator_overloading.py", line 148, in _getattr
raise AttributeError('Attribute %r not found' % attribute_name)
AttributeError: Attribute 'lower' not found

this works:
i | op.Map(str.lower) | op.Map(str.split) | op.Sink(print)

py3.7

If the last subscriber is unsubscribing the call_later handler should be canceled.

• support unary operators for publishers (len(), str(), int(), -, ...)
• complete tests of operators
• make op.Trace actually useful for debugging
• add missing datatypes for datatype check
• unify argument names: func, function, callback, map_func
• always apply partial to simplify code
• move TopicMapper and DTRegistry to broqer.hub.utils

The current implementation is only subscribing to the selected publisher. An argument like subscribe_all would be nice to subscribe to all publishers on subscription (op.Switch(mapping, subscribe_all=True).

Otherwise you could miss a state change in a unselected publisher, because it's not evaluated.

Add it explicitly to Publisher to avoid warnings like this:

• not is missing

• op.Sink(f, 'static') results in f('static', value)
• op.SinkAsync(f, 'static') results in f(value, 'static')

Specially method overloading can result in unexpected behaviour, e.g. calling 'p.notfiy(v)' instead of 'p.notify(v)'

Split them into publishers, subscribers and operators

This is just complicating the internals and has no further use.

MapAsync and MapThread are stateless and the first value emitted is after running the first coroutine/function. By adding an init argument the operator get's stateful with init as initial state.

Additionally explore the use of yield in the coroutine to emit values during the coroutine/function is running.

If the callback should be called only once (on subscription) it should be possible to set the interval argument to None.

A subscriber should not support the | operator.

At the moment, you could write this:

publisher | operator | subscriber

This leads to following linter warning: Statement seems to have no effect (pointless-statement). And this should be really true as operators create new objects and these objects should be referenced. So in broqer the | should create a new object, but no other side effects should occur. At the moment, this is not the case, because | a subscriber ends up in a subscription of the publishers.

How to solve:

modified_publisher = publisher | operator
modified_publisher.subscribe(subscriber)

or short:

(publisher | operator).subscribe(subscriber)

Another possibility would be to add a .subscribe_to method to the subscriber:

subscriber.subscribe_to(publisher | operator)

Note:
publisher | operator is really creating a new object but is not doing anything else. If not referenced anywhere, it will just be garbage collected.

As in the _operator module is Operator and MultiOperator defined it should not be a private module.

There should be a clear design decision where to use assert and where to raise exceptions. Instances like here just don't feel right: https://github.com/semiversus/python-broqer/blob/master/broqer/op/filter_.py#L59

Assert statements are not handled when using "python -o" (for optimization).

add possibility to define on which publisher emit the CombineLatest operator itself should emit (like streamz is doing it here: http://streamz.readthedocs.io/en/latest/api.html#streamz.combine_latest)

A subscriber should know which publisher is subscribing. This will be called from publisher.subscribe(s)

It should not be a class method. Better would be re-using a Trace instance multiple times and using set_handler for this instance.

In SinkAsync, error_callback is set as attribute in init, however it is never used.

def __init__(self, coro, *args, mode=AsyncMode.CONCURRENT,
             error_callback=default_error_handler,
             unpack: bool = False, **kwargs) -> None:

    _coro = wrap_coro(coro, unpack, *args, **kwargs)
    self._coro_queue = CoroQueue(_coro, mode=mode)
    self._error_callback = error_callback

def emit(self, value: Any, who: Publisher):
    self._coro_queue.schedule(value)

In the emit method, the future reference of CoroQueue.schedule should be used to add all callback which checks for an Exception and then finally calls self._error_callback.

broqer should only allow immutable objects to be transfered. Having mutable objects transfered in a reactive way seems to be dangerous.

Having topics like the following:

• $topics - a list of all topics
• $topic_added - emitting topic path when a new topic is assigned or subscribed
• $version - version of used hub

These internal topics should not be included when iterating topics on the hub (like for topic in hub:)

When hub is frozen it's not possible to create a new topic. Also emit or subscribe to an unknown unknown topic would be impossible

In combine_latest.py subscriber.emit is called directly with self._cache even if there is a _map function

It seems a lot of functions and classes are removed in newer releases like in 3.0.2
Missing:

• Subject
• Samples
• SlidingWindow
• ...

Add a .get functionality to publishers. This could prevent from subscribe/unsubscribe to just get the actual value.

• e.g. https://python-broqer.readthedocs.io/en/latest/operators/combine_latest.py

When doing a method call on a publisher, the arguments of this call have to be constant:

>>> v = Value('Hello World')
>>> v.split(sep=' ') | op.Sink(print)
['Hello', 'World']

It would be possible be digging into broqer.op.operator_overloading:_GetAttr to allow publishers as argument values. When a publisher argument is emitting a new value, the method call should be re-evaluated.

>>> v = Value('Hello World')
>>> remove_str = Value('World')
>>> v.replace(remove_str, '') | op.Sink(print)
Hello 

>>> remove_str.emit('Hello')
 World

We have a problem when we want to have a factory function doing something like this:

def moving_average(interval=0.1, size=5):
    return Sample(interval=interval) | SlidingWindow(size=size) | Map(statistics.mean)

x | moving_average() | Sink(print)

The | operator is returning the last element, which is not useful here. Actually we need to create a now operator build from a combination of the given ones, like

def moving_average(interval=0.1, size=5):
    return concat_operators(Sample(interval=interval), SlidingWindow(size=size), Map(statistics.mean))

Decide if .state property is of any use.

has to be imported from ._operator: from ._operator import build_operator

At the moment the return type is asyncio.Future. To avoid import asyncio and being backwardcompatible to python<3.5 broqer should abstract the return type for this future.

Having examples of different applications would help understand the concepts.

• wx/tk gui example
• snake on console
• RaspberryPi GPIO integration

OS: Ubuntu 22.04
python-broqer: 3.0.0
python: 3.10.12

While having an invalid assertion in my test I discovered the following error:

E       AssertionError: assert {<broqer.valu...class 'int'>)} == {}
E         (pytest_assertion plugin: representation of details failed: /home/florian/hq/git/gen3/jellosubmarine/software/.venv/lib/python3.10/site-packages/broqer/operator_overloading.py:128: AttributeError: module 'broqer' has no attribute 'op'.
E          Probably an object has a faulty __repr__.)

The problem seems to be how the CombineLatest is accessed in https://github.com/semiversus/python-broqer/blob/3.0.0/broqer/operator_overloading.py#L128

Possible fix:

broqer/operator_overloading.py

# in line 8
from broqer import op

...
# in line 128
return op.CombineLatest(operand_left, operand_right,

As and and or are logical operators and can not be overwritten, And and Or should be added. Bitwise and (&) is possible, but bitwise or (|) is already overloaded for subscribing publishers.