Comments (1)
Thank you for your attention!
- On ImageNet, we generally find that the optimal lamda_0 for ResNet is within {2.5, 5, 7.5}. Besides, similar to most regularization techniques, a relatively long training schedule might be helpful (e.g., 200 or 300 training epochs).
(2-1) We choose small lamda_0 on CIFAR as we use the entire covariance matrices. On ImageNet, we approximate the covariance matrices by their diagonals to save GPU memory, where we find larger lamda_0 may help(i.e., {1, 2.5, 5, 7.5, 10}).
(2-2) We use 7.5/5 for ResNet-152 and ResNet-101, respectively. If this does not work, maybe you can try {1, 2.5, 10} to see how the performance changes.
If you have any other questions, please feel free to reach me.
from isda-for-deep-networks.
Related Issues (20)
- Do we need to augment the label data as well? HOT 7
- What's the difference between cifar-ISDA and imagenet-ISDA
- Confusion about the code HOT 1
- Is it applicable to one stage object detectors(yolo)? HOT 1
- long-tailed problem HOT 1
- How to visualize augmented data HOT 1
- image classification has this error,can you help me ? HOT 1
- Question about loss computation HOT 3
- code for semi-supervised learning setting?
- Question about the effect of ISDA on shallow layers of the network
- RuntimeError: copy_if failed to synchronize: device-side assert triggered HOT 1
- Questions about visualization on other datasets
- Question about ISDA implementation in object detection
- Releasing CIFAR visualization code
- what if replace last fc with conv in segmentation task
- Questions about ISDA for ImageNet HOT 2
- augmentation features or logits? HOT 1
- confused about where the augmentation is done HOT 1
- Question about changing datasets
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from isda-for-deep-networks.