Comments (2)
Direct update sparsity during speedup includes a step that calculates standard deviation of outputs at the batch dimension i.e., deviation compared with other batches of data is measured and all positions corresponding to less than a predefined epsilon (1e-6) are then used to fill mask positions. Thus, this process requires a batch size of at least 2 to compare and find std.
torch.std return NaN if only one sample is given (as it is impossible to find standard dev with only one sample).
from nni.
While it makes sense to mask weights corresponding to the output positions where std dev is close to 0 for speedup, this is not mentioned and/or explained in the docs. This can also lead to removing params/channels when absolutely no masks are given as input to speedup to begin with.
from nni.
Related Issues (20)
- The process is stuck in the waiting status HOT 1
- ModuleNotFoundError: No module named 'torch.fx._compatibility'
- gpunum
- ModuleNotFoundError: No module named 'torch.fx._compatibility'
- Anybody can run the NAS exp ON GPU? HOT 2
- TypeError: 'Tensor' object is not a mapping
- NNI console won't show after trialGpuNumber was set to 1
- QuickStart example for pruning is not working properly !! HOT 4
- ERROR (nni.runtime.msg_dispatcher_base/Thread-2)
- Anyone have some idea of time series forecasting problem using DARTS strategy over the search space of Recurrent neural networks?
- Using export_data() not working with DartsStrategy()
- 使用QAT进行量化时,config中'granularity':使用'per_channel'出现报错
- Which framework to use for Neural Architecture Search: NNI or Archai?
- ERROR: Strategy failed to execute.
- NNI not running anymore without error messages when CPU reached 100% once
- I cannot make it work on GPU for training HOT 3
- Is there any way to run the ptq-quantization without training?
- Supporting string choices for GPTuner
- Pytorch HPO tutorials stucked in strating server HOT 1
- WARNING: GPU found but will not be used. Please set `experiment.config.trial_gpu_number` to the number of GPUs you want to use for each trial. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nni.