Can you confirm that I understand the overall system architecture (Figure 3) of the thesis? about recognize-anything

[xinyu1205]

The fixed text encoder is used to encode tags separately into textual label queries with semantic information, enable inference generalization to categories that have not been seen in training stage. For example, during training, the model only trains the tag of 'dog'. Due to the similarity of the textual label embedding between 'puppy' and 'dog', inference can be generalized to the tag of 'puppy'.

[sky97613]

Thank you for answer!

Does the list of tags entered as the input of CLIP Text Encoder in the figure mean the tags extracted from the training data (text)?

[xinyu1205]

Please refer to the code in recognize-anything/models/openset_utils.py build_openset_label_embedding().