Select the model based on testing accuracy？ about learningtocompare_fsl

[flexibility2]

@ehsanmok ,hi, however ,in the
https://github.com/floodsung/LearningToCompare_FSL/blob/master/miniimagenet/miniimagenet_train_few_shot.py#L15
you just use the "task_generator_test", not the "task_generator"……

[ehsanmok]

It's based on meta validation set: See 1, 2, 3

[ehsanmok]

Right! not a good code. It's the third mistake along with not using model.eval() and using the same normalization for omniglot and mini-imagenet!

[ehsanmok]

However, it's done correctly for one shot here. Based on the copy pasting attitude of the code maybe it was changed at the time of training and when released the code wasn't carefully done!

[xyxxmb]

@ehsanmok Hello! I think that although the code uses "task_generator_test" instead of "task_generator" in miniimagenet_train_few_shot.py, it doesn't influence the result of model training because "metatest_folders" is only used for monitoring generalization performance. It doesn't participate in the process of model training.

[hyyuan123]

I would like to ask, is there a problem with the model selection based on omniglot? Should be based on the accuracy of training to choose it

[hyyuan123]

When selecting a model, the test data is unknown, so the accuracy of the test cannot be used to select the model.