CDC on reladomo about reladomo

[mohrezaei]

Chaining (both uni and bitemporal) are core features of Reladomo. Please see the docs. There is a short intro in the features doc and a more in depth example in the guided tour.

[kutabale]

I am more interested on capturing the data-changes and pushing them to other upstream services.

[mohrezaei]

Chaining captures changes at the persistence layer (database), which makes tracking changes a lot easier. You can implement a simple periodic transactionally guaranteed push, or an idempotent pull by using temporal chaining. Supporting such a system is far easier, because all the changes are in front of you and upstream support questions, such as, "when did you send us this change?" can be easily answered.

If you want to just push changes from the application, you can use an update listener as discussed in the Event Handler issue.

[kutabale]

What happens when you have corrections in bi-temporal chaining which implementation would work best?

[mohrezaei]

Sorry, I don't understand 2 parts of the question:

1. Can you specify what you mean by "which implementation"? Which implementation of what?
2. Can you specify what you mean by "work best"? Work best for what? Under what criteria?

[kutabale]

1. Implementation i mean:
   a. capturing changes at persistence level
   b. Discussed on Event Handler issue. by overriding the insert/terminate/delete methods.

2. If i choose the option 2 from above my concern, is as follow:
   When we do corrections on bi-temporal domain object, reladomo takes care of inserting and invalidating existing rows. How would i count for those changes?
   The same concern might apply for the other solutions but not sure, hence, the question. I was looking more for you thoughts and what have you seen being implemented.

Thanks

[mohrezaei]

When you say "correction", I assume you mean things in the past (business date dimension).

If the ultimate purpose of this is to send changes upstream, your upstream must be capable of understanding and processing corrections as well, so they need to have a temporal sense.

Implementing a bitemporal replication flow is certainly possible, but it's not trivial. Essentially, you'll have to poll the database using the processing date dimension, read the recent changes and send them upstream in a way that makes sense to them. Doing this via app level event handling is much harder.

Note that for simple CDC, you don't need bitemporal. Unitemporal on the processing date dimension works just fine. If you have bitemporal because of business requirements, then you're already capturing changes at the persistence layer, and typically, polling can be a more supportable solution than application level event handling.

[kutabale]

I need bi-temporal for my data-model. But, also i want to be able to stream the changes to upstream services (data aggregators and analytics).

Is it possible to intercept the queries in reladomo?

What are your thoughts on using some tool like https://debezium.io/ for capturing bitemporal changes?

Thanks a lot.

[mohrezaei]

If you have bitemporal, you already have a system that can tell you what happened for all time. To send that, you just have to do a query that gives you all the changes:
IN >= lastCheckPoint || (OUT >= lastCheckPoint && OUT != Infinity)
You have to update the checkpoint carefully: it must be set to now - maxTransactionTime. So it's important that it's not now and you have a well understood maxTransactionTime, which is already built into Reladomo.

If by queries you mean insert/updates, there is light support via logging, but I wouldn't recommend it.

I have no opinion on other tech.