chen-judge / unigeo Goto Github PK

Does the 'return' refer to the index of the correct answer in the candidate list?

My understanding of this code is that it checks whether the top-k generated answers are in the candidate list, and it does not guarantee that the returned answers are correct.

Is my understanding correct? I look forward to your reply, thank you.

Hello,

I tried to train from scratch (loaded the pretrained.pth). In addition, I also evaluated by loading your geoformer.pth). However, the results are not the similar to the reported in the papers, the following are the results from the stdout:

• loaded the pretrained.pth, and trained from scratch:

• loaded the geoformer.pth, and test directly:

As you can see, the numbers following the Acc for either "Calculation" or "Proving" fall behind the reported numbers in the paper. And I only modified the bash scripts to make it train on one GPU (A100), leaving other parts unchanged. Could you please tell me whether there is anything I need to pay attention to, so that I could reproduce the reported numbers.

Thanks!

Some symbols are used in the dataset to express the process of mathematical calculation and proof more efficiently, such as: R_i, E_i, etc. However, these symbols seem difficult to restore to their original natural text according to the content in the dataset. Since we want to try to use this dataset to fine-tune a large multimodal model, we would like to ask the author if there is a plain text version of the dataset.

Thanks ! !