What threshold to keep positive place recognitions about minkloc3dv2

[jac99]

Hi,
Indeed, in the open-world scenario, when a query point cloud represents a new, unvisited area, you need a threshold when comparing the query point descriptor cloud with descriptors of already visited point clouds.
I suggest using a validation set to select the threshold which maximizes a chosen metric (e.g. the accuracy of binary classification if a place was already visited).
I would use the following approach:

1. Train a network (e.g. MinkLoc3Dv2) using a training set to produce a global descriptor for the input point cloud. (using the standard training approach).
2. Use the validation set to choose the threshold to decide if the place was already visited. For each point cloud in the validation set, compute its descriptor using the trained network.
3. For each query point cloud in the validation set, find the point cloud in the validation set with the closest descriptor. Assume the place was already visited ('positive place recognition') when an Euclidean distance between the descriptor of the query point cloud and this closest point cloud is below a threshold Theta. Otherwise, assume the place was not visited ('negative place recognition').
4. In a loop, try different values of Theta, and find the value which maximizes the chosen metric using the validation set. The metric depends on your use case. It can be an accuracy or a balanced accuracy of the binary classification task "was the place already visited", where you classify a place as already visited when the Euclidean distance between descriptors is less than Theta.

[JzHuai0108]

Thank you for the detailed response. We will try it out. Our initial thought was to compute the precision-recall curve and find the point with the max recall while keeping the precision at 100%. It is fairly similar to your approach :).