Metainfo of the inverted index may overflow, causing the index to become unavailable about greptimedb

[waynexia]

it proves we made a data intensive application

2 looks good to me. 1 also, but I'm not sure about the complexity.

[evenyag]

In another aspect, maybe we should restrict the total row number of an SST file. We don't want to generate a very large parquet file and @v0y4g3r is working on this.

We may also skip building the inverted index for a column if the number of distinct is too large but I'm not sure what the appropriate threshold is.

Introduce additional levels of indexing, SST -> Row Group -> Segment, to control the number of indexing targets.

Maybe we can build an inverted index per row group since the number of rows inside a row group won't be too large. We need to enlarge the row group size (rows in a row group) to reduce the number of row groups per file.

[killme2008]

@zhongzc Do we have a plan to fix it?