palkan / active_event_store

Rails Event Store in a more Rails way

Hey / Привет 👋

My name is Vladimir (or Вова), and I'm a principal backend engineer at @evilmartians.

📕 My book "Layered design for Ruby on Rails applications" is avalable now: Amazon | Packt

I'm working on:

• Making real-time apps rock with @anycable (built with Ruby and Go).
• Bringing edge Ruby features to all Rubies with @ruby-next—a transpiler for Ruby.
• Helping Ruby devs to write faster tests with test-prof.
• ...and other projects, such as: Action Policy, Logidze, Anyway Config and many more.

Check out some of my blog posts:

• The future of full-stack Rails
• Real-time stress: AnyCable, k6, WebSockets, and Yabeda
• Ruby on Whales: Dockerizing Ruby and Rails development
• System of a test: Proper browser testing in Ruby on Rails
• Anyway Config: Keep your Ruby configuration sane
• RuboCoping with legacy: Bring your Ruby code up to Standard

...and conference talks:

• Rails as a piece of birthday cake
• The pitfalls of real-time-ification (RailsConf 2022)
• Weaving and seaming mocks (RubyConf 2022)
• Frontendless Rails Frontend (RailsConf 2021)
• Between monoliths and microservices (RailsConf 2020, component-based architecture using Rails Engines)
• Ruby Next: Make old Rubies quack like a new one (RubyConf 2019)
• Terraforming legacy Rails applications (RailsConf 2019)
• High-speed cables for Ruby (RubyConf 2018)
• 99 problems of slow tests (Paris.rb 2018)

active_event_store's Issues

ActiveSupport.on_load not being called

What did you do?

I set up active_event_store according to the instructions:

ActiveSupport.on_load :active_event_store do |store|
  store.subscribe MyEventHandler, to: ProfileCreated
end

What did you expect to happen?

I expected the on_load block to be executed.

What actually happened?

The block doesn't execute

Additional context

• When using a to_prepare block on the application ActiveEventStore.event_store has been initialized.
• MWE: https://github.com/chriscz/aes-loading-bug

Environment

Ruby Version: 2.5.8

Framework Version (Rails, whatever): Rails 5.2.4.4

Active Event Store Version: 0.2.0

Parse timestamp/valid_at as TimeWithZone in Mapper

Is your feature request related to a problem? Please describe.

Yes. I would like to be able to use the RES Event Browser but it fails as it expects timestamp to be a Time (or TimeWithZone) object as it calls #iso8601 on it. I would also just prefer in general to interact with the event timestamp as a object vs. a string.

Describe the solution you'd like

I think it would make sense to parse the JSON values for both timestamp and valid_at with Time.zone.parse in the Mapper class. I'm not sure if that would cause backward compatible issues though. Seems like anyone would prefer a Ruby Time/TimeWithZone object to an ISO date string.

This is already the case in the default YAML serializer.

Additional context

I'm happy to create a PR if you think that would be a good step forward.

Support different event store engines

Is your feature request related to a problem? Please describe.

Your docs mention the capability of specifying a different underlying store engine. Is that supported yet?

Describe the solution you'd like

To enable event propagation to non-rails clients via Redis pubsub, while still using the Rails-y usage pattern of AES

Describe alternatives you've considered

Different event buses

Additional context

Any plans to support weak/hybrid schemas?

Is your feature request related to a problem? Please describe.

Versioning events can be complicated and tricky (as per described here), but weak/hybrid schemas can help.

Please correct me if I'm wrong, but as far as I can tell it seems the current event setup supports strong schemas only.
That means that the following can be used to represent some event:

  class TestEvent < ActiveEventStore::Event  
    attributes :user_id, :action_type
  end

But maybe backwards compatibility can become an issue if/when the requirements are updated and the data being passed around by this event needs to be changed, especially when removing an attribute (as then we get: ArgumentError, Unknown event attributes: user_id).

There are a few ways to tackle this issue:

• Introduce a new event type and slowly migrate all consumers to be compatible with it is an option, but that would require littering the code with multiple event definitions (v1, v2, v3) that could all be slightly different and could introduce other more complex issues as explained here.
• Double publish is also an option, as in publishing both versions of the event and having the subscribers deal only with the events that they understand, but issues when replaying the events arise from this approach.

And some others, but none of which make this 'migration' easier/more transparent as we'd expect from something out of the box for Rails as far as I can humbly tell.

Among the approaches described in that book, I noticed that the weak schema may likely be the 'more transparent' one, as it allows one to have a well defined event type (whose name would likely not change as attributes are added/removed) while moving the responsibility of deserializing it into a 'mapper' class that handles the presence/absence of attributes in a more graceful manner.

Describe the solution you'd like

Perhaps a possible solution would be to have a Schema parent class from which other schemas could inherit.

This Schema base class could implement an .attribute method that receives both the attribute name (like what happens already) and a default value as a fallback (nil if not present).

This could allow us to have Events and Schemas defined as follows:

class TestEventSchema < ActiveEventStore::Schema
  attribute :user_id, nil
  attribute :action_type, 'some_default_value_here_aye?'
end

class TestEvent < ActiveEventStore::Event  
   with_schema TestEventSchema
end

The attributes could be processed through the schema just before invoking validate_attributes! if a schema was present for the event.

The rest would stay as is and minimal changes would be introduced to the current setup.

I see a few pros:

• No new events created to represent slightly the same thing, we'd use the same event class already in place.
• We can add/remove attributes at will to the weak schema without risking a failure
• If a given attribute is not present when a consumer tries to consume an event we can rely on a default fallback (or raise an error instead?)

And a few cons:

• Renaming is an issue
• Cannot go back to using strong schemas (unless you replicate the same list of attributes present in the schema)
• Could probably force checks to be added to the consumer to ensure a given value is in place (or raise error instead?)

Describe alternatives you've considered

Adding new events, double publish, etc.

Additional context

I have worked on a POC branch that adds support to weak schemas and another one that expands on that and adds support to hybrid schemas.

I can open a PR if that is a direction the core team would like to take, of course.
If not, I'd like to ask how you currently tackle the issue of migrating events while maintaining their immutability. 🤔

Thanks!!

Fix upstream RES compatibility

Looks like events serialization internals are going to change in the future RES releases (for example, RailsEventStore/rails_event_store#760).

Currently, our edge builds are broken: https://github.com/palkan/active_event_store/runs/1148181711

ArgumentError (missing keywords: :event_type, :event_id, :data, :metadata):

What did you do?

Implementing active event store for async events with sidekiq as ActiveJob adapter.

What did you expect to happen?

The SubscriberJob to execute properly

What actually happened?

ArgumentError (missing keywords: :event_type, :event_id, :data, :metadata): upon subscriber sidekiq job execution

Additional context

Environment

Ruby Version: 3.2.2

Framework Version (Rails, whatever): Rails 7.0.7

Active Event Store Version: active_event_store (1.0.2)

Rails Event Store Version: rails_event_store (2.11.1)

Sidekiq Version: sidekiq (7.1.2)

It is currently not possible to use classes that inherit from `Class` as async subscribers.

Is your feature request related to a problem? Please describe.

When setting up my classes as subscribers (example below) like RES allows, I noticed it is not possible to use classes that inherit from Class as subscribers.

The issue is that when dispatching an event to a subscriber, SubscriberJob expects subscriber to answer to #call directly.

And it is not possible to use an instance as an async subscriber because callable must respond to #name.

This closes the door for class setups like the following (using DI):

  class ProfileCompletedClassWithInstanceMethodSubscriber
    def initialize(some_nice_thing: some_nice_thing)
     @some_nice_thing = some_nice_thing
    end

    def call(event)
      puts "It works from #call as an instance method!!"
      @some_nice_thing.call("It works from #call as an instance method!!")
    end
  end

Describe the solution you'd like

I expected to be able to use classes like the following as async subscribers.

  class ProfileCompletedClassWithInstanceMethodSubscriber
    def initialize(some_nice_thing: some_nice_thing)
     @some_nice_thing = some_nice_thing
    end

    def call(event)
      puts "It works from #call as an instance method!!"
      @some_nice_thing.call("It works from #call as an instance method!!")
    end
  end

Describe alternatives you've considered

I have considered adding the #name method to my classes to get past the fact that callable must respond to :name, it still does not work because of the fact that the method #call is only available as an instance method,

Additional context

The issue can be checked using the following initializer:

#initializers/active_event_store.rb
class ProfileCompleted < ActiveEventStore::Event
  attributes :user_id
end

class PublishProfileCompleted
  def initialize(active_event_store: ActiveEventStore)
    @active_event_store = active_event_store
  end

  def call(event: ProfileCompleted.new(user_id: 123456))
    @active_event_store.publish(event)
  end
end

ActiveSupport.on_load :active_event_store do |store|
  module ProfileCompletedModuleSubscriber
    class << self
      def call(event)
        puts 'It works from #call as part of a module!!'
      end
    end
  end
  class ProfileCompletedClassWithInstanceMethodSubscriber
    def call(event)
      puts "It works from #call as an instance method!!"
    end
  end

  store.subscribe ProfileCompletedModuleSubscriber, to: ProfileCompleted, sync: false
  store.subscribe ProfileCompletedClassWithInstanceMethodSubscriber, to: ProfileCompleted, sync: false
end

Which will raise NoMethodError when the ProfileCompleted event gets published (Use PublishProfileCompleted.new.call to test).

PS: Maybe this is a very particular need of mine, but hopefully it may help other people.

have_published_event matcher.

Hi all,

I'm curious why the have_published_event matcher only supports block expectations as stated in the readme. I'm curious to know the reason why we wouldn't be able to do expect(...).to have_published_event(SomeEventClass)

Thank you!