Comments (6)
@floodsung If you do not use eval() in test phase, you will determine the BatchNorm statistics through seeing other query images. This is cheating to some extent, because each query image only see the support images in each episode. Determining the BatchNorm statistic through a big batch of query images will trun few-shot learning task into a many-shot learning task.
from learningtocompare_fsl.
@tlittletime @Bigwode @IbsenChan @itongworld I haven't tested the code yet, but I believe that is because the authors wrongly set momentum=1 for all batch norm layers, which makes the BN layers always save the current batch's stats and discard all previous stats. This might be the possible reason why do not use eval will yield better results.
from learningtocompare_fsl.
Yes, I tried and interestingly I have a better result when not using eval()
from learningtocompare_fsl.
@floodsung You need to use eval(), otherwise you won't fix the BachNorm statistics. See this
from learningtocompare_fsl.
@ehsanmok agree,but I test it 10% test accuracy drop.
from learningtocompare_fsl.
@IbsenChan BUT why the degradation happens when using eval()? Theoretically Relation Net should behave better than ProtoNet, i.e. performance over 65% (training protonet with 5-way 5-shot), according to its novel idea of learning a metric in the paper. But with eval() I only get performance no higher than 60%.
Do you have any ideas about the degradation? Is it the problem of implementation or the idea itself?
from learningtocompare_fsl.
Related Issues (20)
- Question about training and testing using the same dataset. HOT 2
- How can I find the task_generator_test.py ?
- RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #3 'index' HOT 2
- RuntimeError: Attempting to deserialize object on CUDA device 1 but torch.cuda.device_count() is 1. Please use torch.load with map_location to map your storages to an existing device. HOT 1
- Question about mini imagenet dataset
- Corrected depreciated functions and tested using ipython
- How to ensure each category in support set is the same as that in query set?
- RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 2. Got 1 and 50 in dimension 0 at /pytorch/torch/lib/THC/generic/THCTensorMath.cu:111
- IndexError: scatter_(): Expected dtype int64 for index. HOT 3
- sample larger than population"
- When running omniglot_train_one_shot.Time costs。 HOT 1
- RuntimeError: shape '[-1, 128, 19, 19]' is invalid for input of size 46656000 on miniimagenet code
- Question about calculating accuracy
- Can I use contrastive loss here?
- Train on my own dataset
- Test on different dataset, not miniimagenet, not omniglot
- 💪 Reproduce LearningToCompare_FSL environment on Ubuntu 16.04 CUDA 8.0
- self.train_labels = [labels[self.get_class(x)] for x in self.train_roots] KeyError: '..\\datas\\omniglot_28x28' HOT 4
- Data Leakage!
- Do model.train and model.eval need to be added?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from learningtocompare_fsl.