ing-bank / baker Goto Github PK

To stick with the documentation when creating my first recipe I have been using the JBaker object.
However today I refactored the code to be full Scala.

Using JBaker

val baker = new JBaker(
      recipe,
      ImmutableList.<Interaction>of(
                ...
        );
    )

while using Baker directly requires more boilerplate

  implicit val timeout: FiniteDuration = new FiniteDuration(10, java.util.concurrent.TimeUnit.SECONDS)
    val baker = new Baker(
      recipe,
      implementations = Map(
        classOf[InteractionExample] -> (() => new InteractionExample()),
        ....
      ),
      actorSystem = ActorSystem.apply("BakerActorSystem", JBaker.defaultConfig)
    )

It would be nice to have a SBaker object, just like for Java, that avoids having to consider things like actorSystems and timeout.

Especially since those items are currently not part of the available documentation 👼

This is found during property based testing, we should not allow empty or null names.

There should be a java variant of RuntimeEvent.ingredientMap

This library can do that: https://github.com/nidi3/graphviz-java

At the moment when an event is handled by baker, a BakerResponse object is returned, and it has 2 methods: confirmReceived and confirmCompleted.
In many cases, what we do is calling confirmCompleted, and in the next line looking for an event (eventually consistent) or an ingredient (consistent) to be available, and return a different http response accordingly.

And also the eventual consistency behaviour of the baker.events() is not easily guessed from the name, and not well known by many developers. This sometimes causes bugs due to misassumptions.

It would be nicer to add the possibility to wait until some event or some ingredient available when we call confirmReceived or confirmCompleted methods. Or maybe make a new method for that.

And the confirmation of the ingredient or event should be done synchronously (eventual consistency is not a good pattern here).

If SOMEEVENT produces an ingredient with null value, baker fails with the following message which is confusing:

Transition 'INTERACTIONNAME' failed with: com.ing.baker.runtime.petrinet.FatalInteractionException: Output: $$SOMEEVENT$$ fired by interaction INTERACTIONNAME but could not link it to any known event for the
 interaction\n      at com.ing.baker.runtime.petrinet.ReflectedInteractionTask$$anonfun$interactionTask$1.createRuntimeEvent$1(ReflectedInteractionTask.scala:109)\n        at com.ing.baker.
runtime.petrinet.ReflectedInteractionTask$$anonfun$interactionTask$1.apply(ReflectedInteractionTask.scala:125)\n       at com.ing.baker.runtime.petrinet.ReflectedInteractionTask$$anonfun$in
teractionTask$1.apply(ReflectedInteractionTask.scala:77)\n        at com.ing.baker.runtime.petrinet.TaskProvider$$anonfun$interactionTransitionTask$1$$anonfun$apply$2$$anonfun$apply$3.apply
(TaskProvider.scala:59)\n    at com.ing.baker.runtime.petrinet.TaskProvider$$anonfun$interactionTransitionTask$1$$anonfun$apply$2$$anonfun$apply$3.apply(TaskProvider.scala:59)\n    at fs2.T
ask$$anonfun$delay$1.apply(Task.scala:191)\n    at fs2.Task$$anonfun$delay$1.apply(Task.scala:191)\n    at fs2.Task$$anonfun$suspend$1$$anonfun$2.apply(Task.scala:199)\n       at fs2.Task$$
anonfun$suspend$1$$anonfun$2.apply(Task.scala:199)\n       at fs2.util.Attempt$.apply(Attempt.scala:12)\n  at fs2.Task$$anonfun$suspend$1.apply(Task.scala:199)\n  at fs2.Task$$anonfun$suspe
nd$1.apply(Task.scala:199)\n  at fs2.internal.Future.step(Future.scala:54)\n  at fs2.internal.Future.listen(Future.scala:30)\n        at fs2.internal.Future.runAsync(Future.scala:69)\n
      at fs2.Task.unsafeRunAsync(Task.scala:100)\n    at fs2.Task$Ref$$anonfun$set$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Task.scala:343)\n   at fs2.Strategy$$anon$5$$anon$6.run(Strategy.sc
ala:60)\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n
    at java.lang.Thread.run(Thread.java:748)\n 

Scenario:

1. A new baker/akka node joins the cluster and is given some shards.
2. The ProcessIndex actors for those shards are replayed.
3. After replay, for every ProcessMetadata in the index an update is send to to ddata replicator.

This process is extremely slow/intensive and has been seen to even crash the node.

• In case of shard is handover the index information is already know and should not have to be updated.
• In case of cold boot, we need a method to add all ProcessMetadata in one Update to the replicator.

This feature was removed because of performance issues in the way it was implemented using akka ddata.

A relatively easy method might be to to ask all the shards their index and combine the result.

Currently the ProcessInstance actor executes interactions and pipes the result as a message back to itself.

Although this message is local in theory it could get lost. If it does get lost it will result in a broken state that can not be understood at runtime.

Example :

public static final String REQUIRED_SIGNATURES = "requiredSignatures";
private final SignatureList requiredSignatures;

Constant is reused to lookup Ingredients in API (possibly not consistent with baker philosophy :), but since the result returned by the API varies with different events being present, there is already a dependency)

Places and Transitions in the petri net are currently identified by a Long number. This identifier is used in a number of commands and messages send to/from the ProcessInstance actor.

In hindsight this choice proved to be mistake because it is not easily used in the DSL or understandable by humans.

Also there is a potential hash collision problem since the id is not explicitly specified but computed from other properties.

In a Recipe interactions are given a "label" which is converted to a number using a hashcode function. Even tough a 64 bit SHA256 hashcode is used there can still be collisions.

The proposition is to re-use the "label" given to an interaction as it's transition identifier. The same goes for places.

Hi Baker team!

I have just tried the latest release of Baker (1.1.14) and have noticed one issue with CompiledRecipe::getVisualRecipeAsSVG method. It looks like that because of graphviz-java this method can only be called from a single thread otherwise an exception is thrown:

java.lang.Error: Invalid V8 thread access
                at com.eclipsesource.v8.V8Locker.checkThread(V8Locker.java:56)
                at com.eclipsesource.v8.V8.checkThread(V8.java:676)
                at com.eclipsesource.v8.V8.executeStringScript(V8.java:499)
                at com.eclipsesource.v8.V8.executeStringScript(V8.java:483)
...

For now the workaround is to use smth like singlethreadexecutor and always call this method using it. But of course it would be nicer if you can handle it.

Kind regards,
Sergey

PS: Here you can find some more information about this exception.

Example :
.withFailureStrategy(RetryWithIncrementalBackoff(Duration.ofHours(4), Duration.ofDays(1), Duration.ofHours(2))),

If you add an failure strategy with intial delay and maxTimesBetweenRetries the initial delay is not used.

Is this correct behaviour?

Currently we release artifacts like:

compiler_2.11-1.1.14.jar
runtime_2.11-1.1.14.jar

When looking at these it is unclear that they are from baker.

To make it clear they should be prefixed with baker-

In ProcessIndex.createProcessActor(..) a new ThreadPool is created for each new process actor.

Only 1 ThreadPool per Recipe per JVM should be created.

This is needed because enabling this feature could be easily forgotten by the API's which might lead to memory leaks because of growing number of actors in memory.

a suggestion could be using 5 minutes as a sensible default.

baker.actor.idle-timeout = 5 minutes

Hi,

You can ask baker to give the events for a given process Id.

Is it possible to make it so that baker will supply the timestamps of these events?

regards

edward

Duplicate place identifiers are generated during compilation of a huge recipe. Can be reproduced by property based test suite.

Generated recipe ::: name: recipe-c1c8d3c2-db8f-4829-9db4-7ece702a2053 nrOfSensoryEvents: 31 nrOfInteractions: 55
Compiled recipe ::: name: recipe-c1c8d3c2-db8f-4829-9db4-7ece702a2053 nrOfAllIngredients: 14645 nrOfSensoryEvents: 31 nrOfInteractionEvents: 500 nrOfInteractions: 55
Validation errors: List(Duplicate identifier for: Place(-21729854,ingredient-26e86f42-09b5-49d2-a911-2afee69e3a6c,IngredientPlace) and Place(-21729854,interaction-004429de-6082-4979-b33c-6aad207ad5f9-ingredient-5044ed52-23c5-4a94-bb65-68c338bd5a0a,MultiTransitionPlace), Duplicate identifier for: Place(-21729854,interaction-004429de-6082-4979-b33c-6aad207ad5f9-ingredient-5044ed52-23c5-4a94-bb65-68c338bd5a0a,MultiTransitionPlace) and Place(-21729854,ingredient-26e86f42-09b5-49d2-a911-2afee69e3a6c,IngredientPlace), Duplicate identifier for: Place(1196063734,interaction-a77bf31c-bff8-4482-89b2-27a451c0f02a-ingredient-c496018d-892f-4bb8-9f01-9ded2a4beceb,MultiTransitionPlace) and Place(1196063734,interaction-26f6d826-d293-4096-9dea-ff4481a95a93-ingredient-bd7cb3ee-c5e8-488f-bdbd-b1eec5d6ba37,MultiTransitionPlace), Duplicate identifier for: Place(1196063734,interaction-26f6d826-d293-4096-9dea-ff4481a95a93-ingredient-bd7cb3ee-c5e8-488f-bdbd-b1eec5d6ba37,MultiTransitionPlace) and Place(1196063734,interaction-a77bf31c-bff8-4482-89b2-27a451c0f02a-ingredient-c496018d-892f-4bb8-9f01-9ded2a4beceb,MultiTransitionPlace))
Dumping the visual recipe ...

If we have an interaction firing an event with an optional data type, and that event and ingredient was renamed in the recipe and another interaction is requiring that renamed ingredient, then this another interaction is lost during compilation and not compiled into petrinet.

This affects the latest 1.3.3 and before.

When you attempt to construct a JBaker instance and pass the implementations directly like:

val implementations = ...
val jbaker = new JBaker(actorSystem, implementations)

You will get a NullPointerException, this is because the baker proxy is initialised too late in the constructor.

Whenever baker event handler subscriber actor receives the following event, it is considered as unhandled and published to the akka event bus as an error case. This event should be handled properly and maybe ignored just to prevent these debug messages.

Inside Baker.registerEventListener function:

ProcessInstanceEvent(processType, processId, event: InitializedEvent[,,_])

If and event can be fired from multiple interactions it is only fired into the decision model (petri-net) if all interactions have provided this event.
In the event list the event is still found even if the event will not be fired for the petri-net.

This is unclear for the users of Baker. They expect this to be a OR condition.
We should make this as the user expect by doing one the following:

1. Make it a OR condition
2. Disallow having multiple interactions fire the same event in the compiler.

Sometimes a bug occurs in binding ingredients to the correct interactions.
This occurs when:

1. If one ingredient name overlaps with another. For example "apple" and "applejuice".
2. Both ingredients are needed for multiple interactions.

In ProcessInstance we log when we schedule a transition for retry.

Currently you will see these kind of log entries:

Scheduling a retry of transition 'SomeInteraction' in 6400 milliseconds
Scheduling a retry of transition 'SomeInteraction' in 12800 milliseconds
Scheduling a retry of transition 'SomeInteraction' in 25600 milliseconds

The format is not really human readable, it only uses milliseconds.

It would be nice to convert it to seconds, minutes, hours etc.. to be better understandable.

Hi

I followed a link to this project from a newsletter and was surprised to realize that the README mentions recipes only as images but never actually shows the DSL to reproduce those recipes. I tried to find an example in the test code but couldn't figure it out within 5 minutes.

I think it would really help people evaluating your library if you have a coupe of code examples in the README.

This message should be serialized by kryo or proto serializer similar to other messages.

This works :

`
public class SignatureEnrolled {

@Data
public static class NeedMoreSignature extends SignatureEnrolled {
    //should ensure that type and name is the same in both subclasses, but Baker does not seem
    // to find fields in super class
    private final SignatureList enrolledSignatures;
}

@Data
public static class AllRequiredSignatureObtained extends SignatureEnrolled {
    private final SignatureList enrolledSignatures;
}

}
`

That doesnt

`
@DaTa
public class SignatureEnrolled {
private final SignatureList enrolledSignatures;

public class NeedMoreSignature extends SignatureEnrolled {
    public NeedMoreSignature(SignatureList enrolledSignatures) {
        super(enrolledSignatures);
    }
}

public class AllRequiredSignatureObtained extends SignatureEnrolled {
    public AllRequiredSignatureObtained(SignatureList enrolledSignatures) {
        super(enrolledSignatures);
    }
}

}
`

We should report metrics that users of Baker could be interested in.

To decide:

• Which framework, codahale metrics?
• Which metrics, for example: interaction execution time.

Currently we use the output null for some transitions indicating the absence of an event to update the state.

It is better to explicitly model this as Option[RuntimeEvent]

When an event provides an ingredient of type primitive 'boolean' and this event is fired, baker throws a FatalInteractionException with this message:
ERROR c.i.b.runtime.petrinet.TaskProvider - Output: fired by interaction but could not link it to any known event for the interaction

This is not nice to have at runtime. We should either do not support primitive data types and fail during recipe compilation or support it and fix this problem.

There is a potential integer overflow problem with data larger then 4 GiB.

This which could allow remote authenticated attackers to cause a heap-based buffer overflow.

Recommendation is not to allow more then 2 GiB.

protocolbuffers/protobuf#760
https://nvd.nist.gov/vuln/detail/CVE-2015-5237

The existing supported types should also be refactored to this system.

Since this conversion code is statically referenced the converters must be added statically as well.

We can do this by registering extensions in a reference.conf:

baker.types {
  "com.example.MyClass" = "com.example.MyCustomConverter"
}

with com.example.MyCustomConverter something along the lines of:

public class MyCustomConverter extends TypeConverter[MyClass] {
  def typeOf(clazz: Class[MyClass]): Type = ???
  def to(value: Value): MyClass = ???
  def from(value: MyClass): Value = ???
}

"Fired event is not recognised as any valid sensory event" message is not descriptive enough about the real cause. We should add also the validation message to the error.

Hi ,

I would like to request you guys if its possible to add also the recipe name to this exception message so it would be easier to pin point which recipe gets re-triggered/ re-baked to get more insights when we have multiple recipes.

In case there's no connection to any local cassandra nodes, this warmup actor times out but baker already started the actor system and joined the akka cluster which means, started already receiving actor messages, sharding region handover messages, etc... And these will all fail because there's no connection to local cassandra nodes in local data center.

Joining to cluster should be done after successfully connecting to cassandra, which could be done by hooking into the warmup actor status...

When configuring retry with exhausted event on Recipe level the exhausted event is not added to the possible outcomes of the interactions.

We should not allow setting retry with exhausted events on recipe level because the exhausted event is then shared between interactions.
Or we should have a standarized event name for each interactions failure event so they do not overlap.

We have run into several version conflicts between baker compiled using scala 2.11 and some other libraries that use scala 2.12

This is needed because of unhandled Request(16) messages coming from the akka eventsByPersistenceId query module which depends on akka streams. The new version of the library does not use these messages, so an upgrade might fix these annoying debug log lines:

DEBUG akka.actor.RepointableActorRef - unhandled message from Actor[akka://actorsystem/deadLetters]: Request(16)

When baker fires a sensory event 2nd time with a maxFireLimit set, getEvents call fails with the following stack trace:

2017-09-22 11:49:27,213 [DOCA-akka.actor.default-dispatcher-3] [] INFO  akka.actor.RepointableActorRef - Message [akka.persistence.cassandra.query.QueryActorPublisher$Finished] from Actor[akka://DOCA/user/StreamSupervisor-3/flow-903-1-currentEventsByPersistenceId-process-DigitalOnboarding-c65c77b3-513a-4a32-b24c-bcd6985c8830#400218330] to Actor[akka://DOCA/user/StreamSupervisor-3/flow-903-1-currentEventsByPersistenceId-process-DigitalOnboarding-c65c77b3-513a-4a32-b24c-bcd6985c8830#400218330] was not delivered. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2017-09-22 11:50:11,993 [tomcat-http--22] [c65c77b3-513a-4a32-b24c-bcd6985c8830] ERROR n.i.r.p.r.p.e.RuntimeExceptionHandlingMapper - Caught RuntimeException from application with exception:
java.lang.IllegalStateException: No element found with id: 651199937733138216
        at com.ing.baker.petrinet.api.package$IdentifiableOps$$anonfun$getById$1.apply(package.scala:38)
        at com.ing.baker.petrinet.api.package$IdentifiableOps$$anonfun$getById$1.apply(package.scala:38)
        at scala.Option.getOrElse(Option.scala:121)
        at com.ing.baker.petrinet.api.package$IdentifiableOps.getById(package.scala:38)
        at com.ing.baker.runtime.actor.serialization.ProtobufSerialization$$anonfun$com$ing$baker$runtime$actor$serialization$ProtobufSerialization$$deserializeConsumedMarking$1.apply(ProtobufSerialization.scala:116)
        at com.ing.baker.runtime.actor.serialization.ProtobufSerialization$$anonfun$com$ing$baker$runtime$actor$serialization$ProtobufSerialization$$deserializeConsumedMarking$1.apply(ProtobufSerialization.scala:114)
        at scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
        at scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:157)
        at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
        at com.ing.baker.runtime.actor.serialization.ProtobufSerialization.com$ing$baker$runtime$actor$serialization$ProtobufSerialization$$deserializeConsumedMarking(ProtobufSerialization.scala:114)
        at com.ing.baker.runtime.actor.serialization.ProtobufSerialization$$anonfun$deserializeTransitionFired$1.apply(ProtobufSerialization.scala:197)
        at com.ing.baker.runtime.actor.serialization.ProtobufSerialization$$anonfun$deserializeTransitionFired$1.apply(ProtobufSerialization.scala:195)
        at com.ing.baker.runtime.actor.ProcessQuery$$anonfun$eventsForInstance$1.apply(ProcessQuery.scala:33)
        at com.ing.baker.runtime.actor.ProcessQuery$$anonfun$eventsForInstance$1.apply(ProcessQuery.scala:30)
        at akka.stream.impl.fusing.Scan$$anon$14.onPush(Ops.scala:393)
        at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:499)
        at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:401)
        at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:571)
        at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:457)
        at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:546)
        at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:725)
        at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$shortCircuitBatch(ActorGraphInterpreter.scala:715)
        at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:741)
        at akka.actor.Actor$class.aroundReceive(Actor.scala:514)
        at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:650)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
        at akka.actor.ActorCell.invoke(ActorCell.scala:496)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
        at akka.dispatch.Mailbox.run(Mailbox.scala:224)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2017-09-22 11:50:12,009 [DOCA-akka.actor.default-dispatcher-4] [] INFO  akka.actor.RepointableActorRef - Message [akka.persistence.cassandra.query.QueryActorPublisher$Finished] from Actor[akka://DOCA/user/StreamSupervisor-3/flow-905-1-currentEventsByPersistenceId-process-DigitalOnboarding-c65c77b3-513a-4a32-b24c-bcd6985c8830#391973354] to Actor[akka://DOCA/user/StreamSupervisor-3/flow-905-1-currentEventsByPersistenceId-process-DigitalOnboarding-c65c77b3-513a-4a32-b24c-bcd6985c8830#391973354] was not delivered. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

So we can make sure we catch issues like #38 (magic numbers)

During PBT of the recipe runtime, it is found that some generated recipes have interactions that can never be executed because their input ingredients are coming from separate events which are mutual exclusively fired. So, only one of them can fire at a time, therefore the interaction requirements never satisfy. These kinds of recipes should be already caught during compilation.
See this example: Interaction 'kcrt6yzk' is unreachable.

Does this go wrong with protobuf?

@FiresEvent(oneOf = {ExampleEvent.class}) public SomeEvent apply(

yields msg text on runtime:

"... but could not link it to any known event for the interaction..."

while info is available at recipe compile time

Include opentracing in Baker, to trace incoming requests and Client Interactions.

Exceptions thrown by the exception handler function are not handled.

Nothing is logged and the process flow breaks.

How to proceed?