Comments (9)
Chaining (both uni and bitemporal) are core features of Reladomo. Please see the docs. There is a short intro in the features doc and a more in depth example in the guided tour.
from reladomo.
I am more interested on capturing the data-changes and pushing them to other upstream services.
from reladomo.
Chaining captures changes at the persistence layer (database), which makes tracking changes a lot easier. You can implement a simple periodic transactionally guaranteed push, or an idempotent pull by using temporal chaining. Supporting such a system is far easier, because all the changes are in front of you and upstream support questions, such as, "when did you send us this change?" can be easily answered.
If you want to just push changes from the application, you can use an update listener as discussed in the Event Handler issue.
from reladomo.
What happens when you have corrections in bi-temporal chaining which implementation would work best?
from reladomo.
Sorry, I don't understand 2 parts of the question:
- Can you specify what you mean by "which implementation"? Which implementation of what?
- Can you specify what you mean by "work best"? Work best for what? Under what criteria?
from reladomo.
-
Implementation i mean:
a. capturing changes at persistence level
b. Discussed on Event Handler issue. by overriding the insert/terminate/delete methods. -
If i choose the option 2 from above my concern, is as follow:
When we do corrections on bi-temporal domain object, reladomo takes care of inserting and invalidating existing rows. How would i count for those changes?
The same concern might apply for the other solutions but not sure, hence, the question. I was looking more for you thoughts and what have you seen being implemented.
Thanks
from reladomo.
When you say "correction", I assume you mean things in the past (business date dimension).
If the ultimate purpose of this is to send changes upstream, your upstream must be capable of understanding and processing corrections as well, so they need to have a temporal sense.
Implementing a bitemporal replication flow is certainly possible, but it's not trivial. Essentially, you'll have to poll the database using the processing date dimension, read the recent changes and send them upstream in a way that makes sense to them. Doing this via app level event handling is much harder.
Note that for simple CDC, you don't need bitemporal. Unitemporal on the processing date dimension works just fine. If you have bitemporal because of business requirements, then you're already capturing changes at the persistence layer, and typically, polling can be a more supportable solution than application level event handling.
from reladomo.
I need bi-temporal for my data-model. But, also i want to be able to stream the changes to upstream services (data aggregators and analytics).
Is it possible to intercept the queries in reladomo?
What are your thoughts on using some tool like https://debezium.io/ for capturing bitemporal changes?
Thanks a lot.
from reladomo.
If you have bitemporal, you already have a system that can tell you what happened for all time. To send that, you just have to do a query that gives you all the changes:
IN >= lastCheckPoint || (OUT >= lastCheckPoint && OUT != Infinity)
You have to update the checkpoint carefully: it must be set to now - maxTransactionTime
. So it's important that it's not now
and you have a well understood maxTransactionTime
, which is already built into Reladomo.
If by queries you mean insert/updates, there is light support via logging, but I wouldn't recommend it.
I have no opinion on other tech.
from reladomo.
Related Issues (20)
- Using reladomo to connect to Oracle database HOT 2
- Improve sql logging by adding additional context. HOT 7
- Fix ArrayIndexOutOfBoundsException in MultiExtractorHashStrategy. HOT 1
- Tabs are there in generated sources as indentation HOT 7
- Strings should be quoted in the toString() representation of in-clause operations. HOT 1
- Nullability annotations in the generated abstract classes HOT 4
- Can you please provide Db2DatabaseType.java for Db2Driver to get the instance HOT 1
- Backward compatibility not preserved after changing the schema HOT 1
- ObjectSequenceAbstract.java not found HOT 2
- how to build reladomo from IntelliJ/IDEA HOT 1
- H2 Database 2.1.210 to Mithra integration failing for InMemory database HOT 2
- DB Support for Snowflake / Redshift HOT 2
- Problem with attribute quoting in ddl generation for h2. HOT 2
- Upgrade H2 version 2.1.210 from 1.4.200 HOT 6
- Support optimizer hints for select queries HOT 46
- Tool to roll back entire database to an older processing date? HOT 2
- Snowflake support in Reladomo HOT 5
- Java version upgrade HOT 1
- SQL errors related to unquoted keywords after upgrading H2. HOT 6
- How To Question: Many-to-many example HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from reladomo.