Comments (5)
Thanks @luowanghaoyun for report this. IMO, the aggregate function for first
, first_non_null_value
, last_non_null_value
whose value are not align with the latest sequence may encounter this problem, because the actual "version number " of the value is not kept.
I try to introduce an exclusive
sequence for these three kind aggregation field, it should solve this problem, as posted in #3101 But I think it's a bit burden for end user, especially for the aggregation table, the default aggregation function is last_non_null_value
. And the exclusive sequence enforce user to config an individual sequence field for these three kind aggregation field.
from paimon.
@Aitozi Why not keep the earliest sequence under the mentioned scenarios? I think sequence.field should also be constrainted with the aggregation function.
from paimon.
@liyubin117 If we have serval aggregation function fields, a single sequence.field is not enough. The first_value
need earliest sequence, but last_value
need the latest sequence
from paimon.
@Aitozi Thanks for claring that! it is actually more complex than I thought.
from paimon.
@Aitozi hi, when is this issue expected to be fixed?
from paimon.
Related Issues (20)
- Paimon not show data when I'm using upset write-mode (doesn't support consuming update ) HOT 1
- [Bug] Parquet-related benchmark test cases call the wrong method in `TableReadBenchmark` class
- [Feature] Ignore invalid snapshot when rollback to specific snapshot
- [Bug] KvQueryTableTest.testServerRestartSamePorts is unstable
- [Feature] Support options filter in FlinkGenericCatalog when creating HiveCatalog
- [Doc] Add doc for multi-field sequence-group support in Partial Update
- [Feature] support flink sql alter table comment
- [Feature] Support MarkPartitionDoneProcedure in spark
- [cdc] upgrade to flink cdc 3.1
- [Feature] Add benchmark for sorted lookup file HOT 1
- [Bug] HashBucketAssigner load index lead to `Too large` error HOT 1
- [Bug] This statistical value has a sudden decrease in query result
- [Feature] migrate_file support save source table
- [Feature] Cdc support new synchronization mode HOT 1
- [Feature][core] Implement more type conversions for IcebergConversions
- [Feature] Support migrate file action
- [Feature] Is there any way to trace a single StreamElement delay status in the process?
- [Bug] This is a bug! The partition org.apache.paimon.data.BinaryRow@741eefde and bucket 6 is filtered!
- [Bug] paimon-generic catalog options don't agree with flink HiveCatalog's
- [Bug] tez query paimon error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paimon.