jgordijn / process

A small framework to define long running (persistent)processes.

Process

- Travis-CI status

A small framework to define long running (persistent)processes within Akka.

Getting process

If you're using SBT, add the following lines to your build file:

resolvers += "processFramework at bintray" at "https://dl.bintray.com/jgordijn/maven/"

// Latest working with akka 2.3.x
libraryDependencies += "processframework" %% "process" % "0.1.16"

// akka 2.4.x
libraryDependencies += "processframework" %% "process" % "0.1.17"

// Change in organization name since 0.1.23
libraryDependencies += "com.github.jgordijn" %% "process" % "0.1.23"

For Maven and other build tools, you can visit search.maven.org

Intro

If you tried to write a long running (persistent) process in Akka, you will find out that this will result in a lot of messages handling in a single actor. It is not trivial to see what the process flow is and if there are parallel parts. Primary reason is that different steps are implemented in a single class. This library tries to make life easier by doing 2 things:

1. Create a separate class for each step in the process flow
2. Create a process class that describes how the steps are linked together

Steps

A process consists of different steps that are performed one after the other or in parallel. With Process you define every step in its own class.

case class DemoState(demoed: Boolean)
class DemoStep(demoService: ActorRef)(implicit val context: ActorContext)
    extends ProcessStep[DemoState] {

  // Code to execute when the step should perform it's task
  def execute()(implicit stepActor: ActorRef): Execution = state => {
    demoService ! Command(state.demoed)
  }

  // This catches the responses from the async execute action.
  // It emits Process.Event to the process.
  def receiveCommand: CommandToEvent = {
    case ReplyFromDemoService =>
      Demoed
  }

  // The updateState function handles events and changes the state. It
  // should mark the step as done when the event is the last event for
  // this step.
  def updateState: UpdateFunction = {
    case Demoed => { state =>
      markDone()
      state.copy(demoed = true)
    }
  }
}

Process

The main goal of the process is to create a flow of steps. It is possible to chain different steps, so that they are performed in sequence. It is also possible to parallelize steps.

class DemoProcess(demoService: ActorRef) extends Process[DemoState] {
  // implicit ExecutionContext is needed
  import context.dispatcher
  var state = DemoState(false)
  val step1 = new DemoStep(demoService)
  // Subflows can be created
  val subflow1 = subStep1 ~> subStep2
  val subflow2 = subStepX ~> subStepY

  // process defines the complete process
  val process = step1 ~> step2 ~> Par(subflow1, subflow2) ~> step3

  def receiveCommand = {
    case CommandToStartProcess =>
      process.run()
  }
}

PersistentProcess

Long running processes should survive restarts. Therefore it should persist the events and automatically restart when the process is created. It is almost as easy as changing Process[S] to PersistentProcess[S] (where S is the type of the state). The only difference is that you need to specify a persistenceId.

class DemoProcess(demoService: ActorRef) extends PersistentProcess[DemoState] {
  val persistenceId = "demoProcess"
  // implicit ExecutionContext is needed
  import context.dispatcher
  var state = DemoState(false)
  val step1 = new DemoStep(demoService)
  // Subflows can be created
  val subflow1 = subStep1 ~> subStep2
  val subflow2 = subStepX ~> subStepY

  // process defines the complete process
  val process = step1 ~> step2 ~> Par(subflow1, subflow2) ~> step3

  def receiveCommand = {
    case CommandToStartProcess =>
      process.run()
  }
}

process's Issues

Can't access the process state in receiveCommand callback in ProcessStep

Often, I want to do some validation on incoming command. To this end, it would be nice to be able to access the process state(just like in the updateState callback).

Functional extension: process versioning

just a rough thought: when the process changes, probably you want to version it, such that old runtime processes can complete against the former version

Functional extension: model the process visually

Just a rough idea...maybe this deserves it's own project repo. Nevertheless, the idea is that once we can visualise the process..we could probably also turn it around and model the process visually and then have a process underneath that does the actual work.

Improve isolation by making process steps more functional

this is probably a hard one to achieve, but has great benefits when realised. The idea is to envision each process step as a function that takes input (the current state) and produces output (the transformed/updated state).

Upgrade to akka 2.4.0

Release to maven central and include in scalaindex

Hi there :)
Though we never had the time to deep dive into it I think this is a very nice project - how about releasing it to maven central and also making sure it shows up in https://index.scala-lang.org/search?q=process ?

We nowadays have a new community page which defers all search to scaladex so this way people would find this library. http://akka.io/community/

Functional extension: select the first result of multiple parallel process (steps)

execute multiple process steps simultaneously, accept the first one that receives a response and ignore the others.

Functional extension: runtime visualisation of a running process

this is also still a rough idea...and it would probably deserve it's own project repo...the idea is to be able to see (visually) what the state is of a particular running process instance, so which of it's process steps have completed and which are running, etc.

Improve the DSL of the process framework

in order to make the language more fluent, strive for optional parentheses, dots & white spaces.

Functional extension: add process (step) repeat (N times) functionality

these are a few rough ideas that come to mind:

• sometimes you want to perform a process step exactly N times before it completes
• OR slightly different, you want to try exactly N times before giving up
• AND optionally recovering with a failure/alternative process step

Functional extension: visualise the modelled process

Although the modelled process already gives much more overview then regular Akka. Visualisation of the process would further improve it, since a picture says more then thousand words ;-)

Functional extension: add support for local abort (pruning)

Global abort is already supporting if I am not mistaken. Local abort might be useful too, since then one could keep the process running while pruning local (sub) process steps.

Functional extension: redelivery exponential back off

Functional extension: guide the process by a 'pattern matching'-like construct

e.g. http://docs.scala-lang.org/tutorials/tour/pattern-matching.html

Functional extension: make scheduling of process steps possible

since we are dealing with long running processes in some cases it's useful to be able to schedule a process step in the (near) future

Tricky api in (Persistent)Proces

Both PersistenProcess and Process traits have the abstract

 def process: ProcessStep[State]

This has the distinct disadvantage that when implementing this trait you have the option to define a function which returns a different object each time, which is not what you would expect.

Let just change it to:

 val process: ProcessStep[State]

@jgordijn what do you think?