influx6 / actorkit Goto Github PK

Amazon SQS PubSub Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using Amazon SQS pubsub infrastructure as the other queue and delivery mechanism system.

The idea is the use of the infrastructure not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and the infrastructure.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The Amazon SQS PubSub Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared infrastructure connection objects which will be communicated with.

NATS Streaming PubSub Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using NATS Streaming pubsub infrastructure as the other queue and delivery mechanism system.

The idea is the use of the infrastructure not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and the infrastructure.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The NATS Streaming PubSub Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared infrastructure connection objects which will be communicated with.

Google PubSub Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using Google's pubsub infrastructure as the other queue and delivery mechanism system.

The idea is the use of the infrastructure not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and the infrastructure.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The Google PubSub Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared infrastructure connection objects which will be communicated with.

The current implementation of Mailbox interface and it's implementation and their Envelope type, are abit under-designed, the interface leaks some methods from implementation causing Interface Poisoning which is bad design after much thought, we need to at a suitable time, reconsider a refactor of both the interface and it's implementation (the default MailboxImpl).

I personal feel that the underline mechanism for giving mailbox should be pulled based initially, because a pull based system can easily be converted to a push based system based on some wrapping. I am also favorable to the the idea of reactive streams, so we should consider such in our refactoring.

// Mailbox is a one-subscription publisher of envelope messages.
type Mailbox interface {
  Subscribe (MailSubscriber) (MailSubscription, error)
}

type MailSubscription interface{
   Stop()
   Next(batch int) error
}

type MailSubscriber interface {
  Stopped()
  Error(error)
  Completed()
  Next(EnvelopeMessage) error
}

type EnvelopeMessage interface{
  Encode(encoder EnvelopeEncoder)  error

  Meta(string) (interface{}, error)
  Payload() interface{}
  Ref() string
  Epoch() int64
}

type EnvelopeEncoder interface{
  EncodeRef(string) error
  EncodeEpoch(int64) error
  EncodePayload(interface{}) error
  EncodeMeta(string, interface{}) error
}

Think more on this.

This provides an interesting question: How do we fit streams like from MongoDB Op logs into the actokit API. Can we provide similar pre-setup functionality like what Debezium does?

NATS Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using NATS as the other queue and delivery mechanism system.

The idea is the use of NATS not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and NATS.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The NATS Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared NATS net.Conn objects which will be communicated with.

Grain Actors

Taking inspiration from the concepts found in Orleans, we would like to create Grains and Grain Clusters in Actorkit using a technique which borrows communication over pubsub/message-queue technologies as communication plane.

MasterGrain Actor

The job of said actor is to receive grain orchestration messages from underline actor system or communication plane (message queus like Google PubSub, NATS, ..etc) or from local actors in the design to lunch a set of actors which exists for the processing of certain types of messages.
What differs in this Grain actors is that they are materialized actors. They do not exists at the point of instantiation of giving actor system but are generated by the MasterGrain which must be instantiated at runtime. This requires said actor system to have all the necessarily pieces to create such actor types, which will require the implementation of a Behavior registries which will be initialized with the MasterGrain actor to use in creation of giving actor types for specific message types processing.

Message Queues

To bring the system into full circle we will require implementation of different messaging system based actors that allows usage of this type of grain system on different infrastructure.
Target queues:

• NATS (https://github.com/nats-io/gnatsd/)
• NATS Streaming (https://github.com/nats-io/nats-streaming-server)
• Amazon SQS (https://aws.amazon.com/sqs/)
• Google Cloud PubSub (https://cloud.google.com/pubsub/docs/)
• Apache Kafka

ServiceNames

With actorkit unlike Orleans, grains are created using string names of giving action or service. The GUID of a generated grain simply represents the location of where it is currently deployed to or called from. The GUID may change over a period of time depending on if giving grain was relocated to another cluster due to the death of the previous. The idea is unlike a local actor system without grains, the notion of a the actors address is relatively cheap and replaceable, hence only the action name or service name is of most importance.

Usually the service name should be something like Users.AddEmail or Contacts.GetContacts or Contacts.AddContract.. etc. This grains should be very specific and should not be too generalist. This way even if a given grain is the child of another, it's address becomes the generalist reference to it.
In essence, build your actors for example Users to have child actors that handle specific pieces like AddEmail, UpdateContact. This is not a set rule in stone, and can be diverted to just use a single actor User to handle such cases with specific service address pointing to it. But it is far simpler using the previously mentioned approach, as it makes more sense when we are generating actors to handle some operation specific to a type and action.

A difference with Orleans is we do not actively kill grains running on a giving cluster/server. The idea is that the exists for processing till they die or are unable to be restarted for work anymore and hence get terminated.

Also, we do not necessarily follow all of Orleans approaches, but only keep the idea that a giving actor can be materialized on any cluster depending on the availability of the behavior in that clusters registry, which provides us a very perfect way of handling cluster based segregation, which means we intentionally only provide to a cluster during compilation and runtime the behaviors which will ever run on that cluster and nothing more, this way, a cluster can not instantiate a behavior it knows nothing about, which means we can create or segregate certain behaviors to certain servers and only if that server is gone down, should we need to consider spinning up a new cluster to replace that server with set behaviors available to new cluster. (Note might not be a good idea for resilience though unless the cluster creation after the death of a previous one is blazing fast).

This provides an interesting question: How do we fit streams like from MySQL BinLogs into the actokit API. Can we provide similar pre-setup functionality like what Debezium does?

Redis PubSub Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using Redis pubsub infrastructure as the other queue and delivery mechanism system.

The idea is the use of the infrastructure not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and the infrastructure.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The Redis PubSub Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared infrastructure connection objects which will be communicated with.

RabbitMQ PubSub Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using RabbitMQ pubsub infrastructure as the other queue and delivery mechanism system.

The idea is the use of the infrastructure not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and the infrastructure.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The RabbitMQ PubSub Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared infrastructure connection objects which will be communicated with.

Apache Kafka PubSub Comm Bridge

In accordance with the implementation plan, we require the implementation of actorkit messaging communication using Kafka's pubsub infrastructure as the other queue and delivery mechanism system.

The idea is the use of the infrastructure not as in it's general pubsub nature but as a bridge which allows disparate actor system to easily and efficiently communicate with each other using a created custom Actor router which sits as the central communication link between the actor system and the infrastructure.

This is also a hard requirements for the development of the Grain Master Actors (See #1)

Design

The Apache Kafka PubSub Hashed Router

We will require the use of a hash router which will receive subscription messages from other actors about a giving event subject they have interests in, this will be hashed and use as the means of delivery to all interested actors.

This notion applies even when a actor system is only interested in all messages using the * subject.

The notion is to remove the need for individual actors with individual or shared infrastructure connection objects which will be communicated with.

Any chance you can add to the Makefile so that devs can run the examples.
Just NATS without Docker for now.. NATS does not need ocker and so its just quicker working and debugging

The reason i raised this is because the examples dont compile, and i want to play around with this library to see how useful it is.

# github.com/gokit/actorkit/examples/eventshop
../../../../../../github.com/gokit/actorkit/examples/eventshop/eventshop.go:11:34: not enough arguments in call to actorkit.Ancestor
        have (string, string)
        want (string, string, actorkit.Prop)
../../../../../../github.com/gokit/actorkit/examples/eventshop/eventshop.go:16:28: too many arguments in call to system.Spawn
        have (string, *BookStore, actorkit.Prop)
        want (string, actorkit.Prop)
../../../../../../github.com/gokit/actorkit/examples/eventshop/eventshop.go:80:28: too many arguments in call to actor.Spawn
        have (string, *BookEventStore, actorkit.Prop)
        want (string, actorkit.Prop)
../../../../../../github.com/gokit/actorkit/examples/eventshop/eventshop.go:89:30: too many arguments in call to actor.Spawn
        have (string, *BookRatingStore, actorkit.Prop)
        want (string, actorkit.Prop)
make: *** [build-client] Error 2