Question about the rules for calculating FID indicators about pf-afn

[geyuying]

Yes, we follow the previous work to evaluate our models on VITON test set with the test_pairs.txt (You can find it in https://github.com/minar09/cp-vton-plus/blob/master/data/test_pairs.txt).

[JyChen9811]

Hi，I use the pytorch-fid to test the result，but I can‘t get the experimental results as you. Maybe my method is wrong，could you tell me how to test the fid？

[geyuying]

Hi, can you show some results from you "prediction" folder to see whether the testing is implemented correctly?

[JyChen9811]

[geyuying]

The above are some of our results with the provided checkpoint. It seems that your testing environment is not the same as ours listed in README. Please see the issue 3 in the closed issues to solve this problem.

[JyChen9811]

okk,I will check the env,thanks!

[JyChen9811]

After correcting the env, the effect get better!

[geyuying]

Hi, the FID you calculated is still worse than ours using the same checkpoints. We use https://github.com/mseitzer/pytorch-fid for evaluation.

[lyx0208]

I get FID: 9.503645410720281 with your pre-trained model, maybe this is because I use a higher version of PyTorch and other packages?

[geyuying]

@lyx0208 Smaller FID can indicate better results. Can you show some results on VITON test set? And which repo did you use to calculate the FID?

[lyx0208]

@geyuying I just use the link you post above to calculate FID.
I know smaller FID means better results, so it seems a little bit abnormal to me. Here are some results:

btw, when will the codes related to training be released, I've sent you an email this morning for some reasons related to this.

[geyuying]

@lyx0208 Hi, I did not recieve any email. Can you send it again? My email address is [email protected]

[lyx0208]

@geyuying Sent again, maybe Gmail cannot receive emails from my school email address

[VVingerfly]

Hi, Yuying @geyuying
Thanks for your awesome work! The results are really amazing！

Could you tell me your enviroment when testing CP-VTON+? I got a FID 10.186522 for PF-AFN, which is similar to yours 10.09. But for CP-VTON+, I got 31.151999 which is much higher then your result 21.08. Do you have any idea for this result?

Thanks a lot!
B.R.

[xxxxl888]

After correcting the env, the effect get better!

Hello, I would like to ask how you calculate the FID score? Are you using the try-on results generated from the test and the images of the person inputted? My FID score is particularly high.