GithubHelp home page GithubHelp logo

makingthematrix / signals3 Goto Github PK

View Code? Open in Web Editor NEW
13.0 1.0 1.0 2.55 MB

A lightweight event streaming library for Scala 3

License: GNU General Public License v3.0

Scala 100.00%
event-streaming scala

signals3's Introduction

Signals3

Scala CI signals3 Scala version support

Signals3 API documentation

This is a lightweight event streaming library for Scala. It's based on Wire Signals. Wire Signals was used extensively in the Wire Android client app - the biggest Scala project for Android, as far as I know - in everything from fetching and decoding data from another device to updating the list of messages displayed in a conversation. In new apps, it can be of use for the same tasks, as well as - if you write an Android app in Scala by chance, or maybe a desktop app in JavaFX/ScalaFX - Signals3 may help you to communicate between the UI and the background thread. Details below.

Main features

  • Event streams
  • Signals: event streams with internal values
  • Abstractions for easy data transfer between execution contexts
  • An implementation of (sometimes) closeable futures
  • Methods to work with event streams and signals in a way similar to standard Scala collections
  • Generators: streams that can generate events and signals that can compute their new updates in regular (or variable) intervals.

How to use

sbt:

  libraryDependencies += "io.github.makingthematrix" %% "signals3" % "1.1.1"

Maven:

<dependency>
    <groupId>io.github.makingthematrix</groupId>
    <artifactId>signals3_3</artifactId>
    <version>1.1.1</version>
</dependency>

Mill:

ivy"io.github.makingthematrix::signals3:1.1.1"

Gradle:

compile group: 'io.github.makingthematrix', name: 'signals3_3', version: '1.1.1'

Syntax

In short, you can create a SourceSignal somewhere in the code:

val intSignal = Signal(1) // SourceSignal[Int] with the initial value 1
val strSignal = Signal[String]() // initially empty SourceSignal[String]

and subscribe it in another place:

intSignal.foreach { number => println(s"number: $number") }
strSignal.foreach { str => println(s"str: $str") }

Now every time you publish something to the signals, the functions you provided above will be executed, just as in case of a regular stream...

scala> intSignal ! 2
number: 2

... but if you happen to subscribe to a signal after an event was published, the subscriber will still have access to that event. On the moment of subscription the provided function will be executed with the last event in the signal if there is one. So at this point in the example subscribing to intSignal will result in the number being displayed:

> intSignal.foreach { number => println(s"number: $number") }
number: 2

but subscribing to strSignal will not display anything, because strSignal is still empty. Or, if you simply don't need that functionality, you can use a standard Stream instead.

You can also of course map and flatMap signals, zip them, throttle, fold, or make any future or a stream into one. With a bit of Scala magic you can even do for-comprehensions:

val fooSignal = for {
 number <- intSignal
 str    <- if (number % 3 == 0) Signal.const("Foo") else strSignal
} yield str

Communication between execution contexts

Every time you define a foreach method on a stream or a signal, you can specify the execution context in which it will work. Since Signals3 started as an event streaming library for Android, it implements a utility class UiDispatchQueue to help you set up communication between the default execution context (Threading.defaultContext) and a secondary one, usually associated with GUI.

On platforms such as Android or JavaFX, a Runnable task that involves changes to the app's GUI requires to be run in that secondary, special execution context - otherwise it will either not work, will result in errors, or may even crash the app. Your platform should provide you with a method which you can call with the given task to execute it properly. If that method is of the type Runnable => Unit (or if you can wrap it in a function of this type) you can pass it to UiDispatchQueue in the app's initialization code. Later, UiDispatchQueue will let you use extension methods .onUi on event streams and signals. They work exactly like .foreach but they will run the subscriber code in your GUI platform execution context.

Example Android initialization

import android.os.{Handler, Looper}
import io.github.makingthematrix.signals3.ui.UiDispatchQueue

val handler = new Handler(Looper.getMainLooper)
UiDispatchQueue.setUi(handler.post)

Example JavaFX initialization

import javafx.application.Platform
import io.github.makingthematrix.signals3.ui.UiDispatchQueue

UiDispatchQueue.setUi(Platform.runLater)

Usage in all cases:

import io.github.makingthematrix.signals3.ui.UiDispatchQueue.*

val signal = Signal(false) // create a new source signal
...
signal.onUi { value => ... } // define a block of code that will run on the UI thread
...
signal ! true // update the value of the signal from any other place in the code
...
// or this is how you can run any piece of code on the UI thread
Future { ... }(Ui)

If you want to know more

Contributors

This library is a result of work of many programmers who developed Wire Android over the years. My thanks go especially to:

Thanks, guys. I couldn't do it without you.

signals3's People

Contributors

makingthematrix avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

drruisseau

signals3's Issues

Use Loom

For now this is just a research idea. Signals3 extends Scala's ExecutionContext to make it possible for the user to create limited dispatch queues where only a given number of tasks run concurrently (special case: SerialDispatchQueue with the limit set to one, when the user needs to ensure that an execution of .foreach in a signal or a stream is serialized).

I could maybe provide a different implementation of DispatchQueue, LoomDispatchQueue, that uses Loom virtual threads instead of the regular ones. Since Loom is not widely adopted, it would also require some logic that would check under the hood if we can use Loom, and if not, then it would give the user a regular DispatchQueue instead.

Implement Java Microbenchmarking Harness

It would be nice to have. I could make sure the performance doesn't drop when a new feature is implemented or when I refactor things. It could also give me some way to compare signals3 with other event streaming libraries (that would be still comparing apples and oranges, but at least I would know which ones are faster).

Check this SBT plugin: https://github.com/sbt/sbt-jmh

Fault recovery policy

Sometimes an event source is based on a function that may throw an exception. For now, there is no established policy how to deal with it and no unit tests. Generators ignore failures and keep going, but in some other cases the event source might stop working.

I'm thinking of an enum:

enum RecoveryPolicy:
  case Ignore
  case Panic
  case RetryAndIgnore(attempts: Int)
  case RetryAndPanic(attempts: Int)

It would be given to an event source on creation (Ignore could be the default option) and all subsequent event sources created by transformations would get the same as the original one. In case of .zip a priority would have to be decided.

More later.

The Closeable interface

GeneratorStream and GeneratorSignal have methods:

def close(): Unit
def isClosed: Boolean

These are needed to stop the generator - otherwise it would generate new events or update its value forever (there is also a function paused: () => Boolean but its purpose is different).
But right now all the methods inherited from EventStream/Signal return only a standard signal which means that the user needs to hold a reference to the original generator somewhere, to eventually stop it. This is okay in many cases but in soome others the user might want to create a complex generator and it would make sense to split into steps chained by .map, .flatMap, etc., and then just keep a reference to the final signal.

For this I can extract these two methods into a trait, Closeable and then overwrite those methods. (They would need to lose their final keyword...)
I could even create CloseableStream and CloseableSignal, instead of overwriting directly in generator classes. After all, from the outside it's only important that the event stream / signal can be closed, not that it's a generator.
It would also be a step connecting generators and Finite streams and signals. They will be automatically closeable, but they can inherit and modify Closeable anyway ... or not. Something to think about.
On top of that, the Closeable trait could inherit from java.lang.AutoCloseable, so maybe I could in future use it in try-with-resources ?...

Write integration tests

Longer tests, closer to real use cases, using all three main classes: CancellableFuture, EventStream, and Signal.
This will be important when we try to optimize performance and not break anything while doing it.

Add examples

  • Create the /examples subfolder.
  • Every example should be its own small app or at least it should be easy to launch it from sbt console.
  • Make a few simple ones and at least one more complicated.
  • And comment them too, please.

Java API

Signals3 use implicit arguments (given/using) to carry around execution contexts and event contexts. This will not work out if someone wanted to use the library from Java. On the other hand, just making these arguments explicit will make methods quite clunky. It would be great to have a dedicated Java API, but it needs some brainstorming.

Better support for boolean events

In wire-android we often made decisions based on comparison of values of different signals. It looks something like this:

for {
  a <- signalA
  b <- signalB
} yield
  if (a == b) doThis()
  else doThat()

or:

signalA.zip(signalB).map {
  case (a, b) if a == b => doThis()
  case _ => doThat()
}

In short, a lot of logic was based on comparing signals values, creating a boolean signal, and then doing something with it. For that I could introduce a compare method:

def compare(otherSignal: Signal[V])(check: (V, V) => Boolean): Signal[Boolean]

Here in signals3 I already implemented methods and and or in Signal - they can be used only if the signal is of a type that can be used as a boolean. I can develop it more:

  • and and or in Signal for more than two arguments (up to five? six?)
  • xor for two arguments
  • not for both Signal and EventStream
  • operators aliases (even compare could have an operator alias ===):
(signalA === signalB).map { 
  case true  => doThis()
  case false => doThat()
}  
  • nand? nor? Can a signal be a transistor?

GeneratingSignal and GeneratingEventStream

New signal and event stream classes which coud generate new events. They could set up in a number of ways, usually by a mix of the initializing function, the repeating function, and a time interval between events (which in more comple cases could also be a function returning the next interval).

The simplest case:

def repeat[E](event: E, interval: FiniteDuration): GeneratingEventStream[E]

will create an event stream that every internal will generate a new event (every time with the same data).
That could actually be even more simplified for a special case when we need something to "nudge" the system repeteadly:

def heartbeat(interval: FiniteDuration): GeneratingEventStream[Unit] = repeat[Unit]((), interval)

A signal wouldn't work here, because if the value of the signal is not changed, it won't be send out. But on the other hand with signals we have a wide range of possibilities where a new value is based on the previous one.

def unfold[V](initialValue: V)(f: V => V, interval: FiniteDuration): GeneratingSignal[V]

here, the signal starts with initialValue and then every interval the function f will create a new value, based on the previous one.

I thought about making a generalized version of unfold that would carry an internal state that could be use to generate the next value, but in fact the value is already the internal state of the signal. If we want the signal to emit not the hold value, but only its small part, or transformation based on it, we can simply combine unfold with map. For example, this is how we can create a Fibonacci signal:

val fibSignal: Signal[Int] = GeneratingSignal.unfold[(Int, Int)]((0, 1))( 
  f = { case (a, b) => (b, a + b) },
  interval = 1 second
).map(_._2)  

fibSignal will start with the value 1 and the internal state of the generating signal which is its source will be (0, 1). Then every second the generating signal will generate another tuple by applying f to its current value. So, it will go like that:

f((0, 1)) -> (1, 1) // fibSignal.currentValue == 1
f((1, 1)) -> (1, 2) // fibSignal.currentValue == 2
f((1, 2)) -> (2, 3) // fibSignal.currentValue == 3
f((2, 3)) -> (3, 5) // fibSignal.currentValue == 5
f((3, 5)) -> (5, 8) // fibSignal.currentValue == 8

and so on.

In fact, I could look to https://www.scala-lang.org/api/current/scala/collection/immutable/LazyList.html both for inspiration for API, and maybe even to make a generating signal and/or even stream that would use a LazyList as a way to generate events.

Introduce `drop` and `take`, `FiniteSignal`, `FiniteStream`, and `ClosedSignal`, `ClosedStream`

This is going to be difficult, but probably very rewarding.

To make event streams and signals resemble standard collections even more, let's introduce the ability to drop events from the stream, and to close the stream after a certain number of events. For that I need two new concepts:

  1. An indexed event source - a trait that will keep track of the number of events emitted since the stream/signal creation.
  2. A closeable event source - a trait that, when mixed in with an event stream / signal, will tell the user if it's possible that the given stream / signal will still emit events.

Then, I can add two methods to the stream/signal classes:

  1. def drop[V](n: Int): Signal[V] with IndexedES (or just a subclass of Signal[V]) - a new signal with the original one as its parent, which will ignore n next events and start emitting events only after that.
  2. def take[V](n: Int): Signal[V] with CloseableES (or just a subclass of Signal[V]) - a new signal with the original one as its parent, which will emit only the next n events and close after that. A special case: take(0) will produce an empty signal (every const signal is closed).

There is a question how to mix the two and, in fact, how to mix it with other proxy signals. I think CloseableES can be a trait inherited from the very root - it's just that in many cases the signal is never closed (or always closed in the case of a const signal). drop can be a bit more tricky.

And then it will be in theory possible to split streams like this:

val stream: EventStream[E] = ...
stream match {
  case head :: tail => ...
}

where head is the already implemented Future[E] and tail will be stream.drop(1).

And:

stream match {
  case head@FiniteStream(3) ::: tail => ...
}

would mean that first three events of stream goes to the head stream, which then closes, and all the consecutive events go to tail.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.