Support iteration on non-PK unique integer column about ghostferry

[hkdsun]

AFAIK, we don't use multi-column primary keys internally and there is no work planned on our part for this.

I'll be happy to review a plan for this. I believe @shuhaowu had started a branch some time ago to contain all usage of primary keys in a struct and that will likely be where you want to start looking.

[shuhaowu]

Supporting composite and non-uint64 PK has gotten more difficult I believe. That branch would have only supported single column strings as PK. Doing it in a more generic way requires some more work.

There's no plan to do this on our side, but PRs are welcome and we'll review them.

[shuhaowu]

We can potentially solve this issue as long as you have a pagination column (a unique integer column that can be iterated over in ascending order). Ghostferry needs to be changed to take a configurable pagination column and iterate over that column instead of the primary key. Any PRs to solve this issue should probably take a variant of this approach as oppose to locking onto the primary key.

[jarthorn]

@shuhaowu we are really interested in this as part of an ongoing project. In this case we will be retaining an int id iteration column, it would just not be the primary key any more. Do you have any bandwidth to either take this on, or help us out with a high level sketch of how to fix this? We are willing to contribute someone to work on it if you have other priorities!

[shuhaowu]

This is fixed on the latest master.