How to implement bulk processing? about m-r

[Narvalex]

what do you mean with 'lots of domain objects'? Do you mean lot of aggregates or several bounded context? Both?

[nripendra]

I'm referring to aggregates.

[Narvalex]

One way could be what I Do: just host each aggregate type or stream type/category into an isolated microservice. Then subscribe a microservice to another as you need. Inside the microservice each aggregate instance (an object with its unique id that is hydrated by its own stream of events) handles inconming messages (events and/or commands) in its own thread. So all aggregate instances work asynchronously. I work with pesimistic lock for each aggregate instance, since it works independently from the other instances. It works just fine and it is quite fast. No concurrency exceptions, no retries for concurrency issues. And I save a memento/snapshot in-memory an in the database for each transaction.
It is not perfect, but is a way that is working for me.
Disclaimer: English is not my mother tongue.

[nripendra]

@Narvalex,makes sense to process each aggregate in their respective threads.

What confuses me is how to gather all the aggregate ids that needs processing? Is is good idea to query from read model? Doesn't it break CQRS if we query from readmodel at command side? And more over since events are the single source of truth, read model may be stale, hence we may be processing undesired aggregates.

Let's say our application renews membership of all the users in the system every night (if user has agreed for auto renewal). Basically we need to query all the users whose membership has expired, and who as agreed for auto-renewal. In case of relational db, querying such user is a breeze, but in eventstore not so intuitive. Am I missing something? Hydrating every aggregate in eventstore just check if those aggregate satisfies our condition is also a no go.

[Narvalex]

A simple solution could be:

1. An aggregate called 'MembershipRenewalManager" that is subscribed to 'AccountManager' aggregate events.
2. Each time a new account is created the MembershipRenewallManager handles that event and adds the account and membership info to its own list and saves to a single stream. Yes. A single stream. An aggregate holds the entire stream of accounts, so it can have a list of it (a snapshot is very useful here).
3. The system can each night send a single command "RenewAllApplicableMemberships" to that particular aggregate instance (the 'MembershipRenewalManager' aggregate). Then the aggregate re hydrates, handle the command and then publish a massive batch of events 'AMembershipShouldBeRenew'. All of that in a single transaction.
4. The account aggregate is subscribed to the Renewall aggregate, so all applicable accounts reacts and the memebership is renewed.
   This is just one way.

[nripendra]

Thanks for the response :)