Comments (9)
I have just merged a pull request that fixes this bug. Closing. Thanks @DeepTecher
from uno.
Yes, you are right, it is a small typo, I changed the code multiple times and I forgot to remove the loop. Anyway, the swapped assignment is performed in the same way, since I only index the view axis inside the swapped_assignment
method, and the cross-entropy loss compares tensors element-wise. Also, the losses are then averaged, so nothing should change.
from uno.
yeah...
But it still has a problem on cross_entropy_loss
function.
if I guess right, the dims of preds
is [num_views, bach_size, num_label+num_unlabel], our F.log_softmax
should be
preds = F.log_softmax(preds / 0.1, dim=2)
on the last dim to do log_softmax.
we do dim=1
will work on batch_size
. Is it right?
from uno.
Yes, you are right. For some reason, this still works. Let me look into it.
from uno.
Ok.
if you have a new conclusion, please let me know
many thanks
from uno.
Hi, I fixed it and ran CIFAR100-20. I got similar results for CIFAR100-20 on the test set, while performance is slightly worse on the training set. I am now trying to do some hyperparameter tuning. I'll upload the fix as soon as possible.
from uno.
nice~ 👍
from uno.
Hi, I have some good news. It seems that normalizing on the correct dimension improves performance quite significantly. I needed to tune the parameters a bit, but I just had one run hit 55% on the training set (unlab/train/acc
) and 56% on the test set (unlab/test/acc
) for CIFAR100-50. I am testing if the same parameters work on the other settings.
I also went back to my logs and found that ImageNet experiments were probably run without the bug, while all other datasets were affected. I will upload the new version as soon as I finish running experiments.
from uno.
okay,
I cannot wait for the newest result.
from uno.
Related Issues (14)
- License HOT 2
- How long does ImageNet experiments take? HOT 2
- How to implement the unconcat version? HOT 7
- Clarification question on num_large_crops HOT 2
- UNO_V2 results HOT 1
- The results on CIFAR10 HOT 2
- Issues with saving and loading checkpoints when using multiple gpus.
- a lot of questions about how to reproduce and cite the experimental results HOT 6
- Reproducing the paper results HOT 7
- Save inference images HOT 2
- Apply to a custom dataset HOT 2
- A question about the Eq.4 HOT 2
- loss_per_head seems wrong HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uno.