feat: Concurrent Index building about pgvecto.rs

[silver-ymz]

I meet some problems when implementing this feature.

Current behavior of concurrent index building is validate_index function causes segmentation fault in indexcmds.c. I have already found the reason. Postgres validate_index will use bulkdelete callback to gather all the TIDs currently in the index which mentioned in postgres src here. But our implementation of ambulkdelete assumes that the callback is vac_tid_reaped and directly reinterpret callback_state to VacDeadItems. Actually, the callback state passed by validate_index isn't VacDeadItems. The incorrect reinterpretation here caused segmentation fault.

Since we don't use postgres storage system, it seems impossible to construct an ItemPointer and pass it to bulkdelete callback function. My question is how to modify the ambulkdelete function to support general callback functions, and then complete this feature.

Could you give some advices? @VoVAllen @usamoi

[usamoi]

It's not necessary to support a general bulkdelete callback. The session process could request for the deletemask in the background process to simulate the behaivor of validate_index_callback.

[silver-ymz]

There seems to be no way to determine whether the callback parameter is validate_index_callback or vac_tid_reaped.

[VoVAllen]

I think there're some option to know whether the operation is reindex. You can check postgis or timescale for reference.

[usamoi]

Maybe we can use a dirt solution if we do not know how. ivinfo.report_progress is different in vacauum routine and define_index routine.

[VoVAllen]

Some answer from sourcegraph cody ai, not sure it's right or not

Lookup the relation in the cache and get a pointer to its RelFileNode struct:
Relation rel = RelationIdGetRelation(relid);
RelFileNode *relfilenode = rel->rd_node;
Examine the relfilenode->spcNode value:
If it is equal to MyDatabaseTableSpace, then we are in index validation
If it is equal to MyDatabaseTableSpace_fsm or MyDatabaseTableSpace_vm, then we are in VACUUM
For example:

Copy
if (relfilenode->spcNode == MyDatabaseTableSpace) {
// Index validation
}
else if (relfilenode->spcNode == MyDatabaseTableSpace_fsm ||
relfilenode->spcNode == MyDatabaseTableSpace_vm) {
// VACUUM
}
This checks the tablespace of the relation to determine the current stage.