Comments (6)
code here for building network and using thop to count the flops
``
from xautodl.models import get_cell_based_tiny_net
index = 0
cifar10_config = api.get_net_config(index,'cifar10-valid')
print(cifar10_config)
cifar10_network = get_cell_based_tiny_net(cifar10_config)
from thop import profile
import torch
input = torch.randn(1,3,32,32)
flops, params = profile(cifar10_network, inputs={input,})
print(flops/1e6, params/1e6)
``
from nats-bench.
I use my own codes: https://github.com/D-X-Y/AutoDL-Projects/blob/main/xautodl/utils/flop_benchmark.py#L42
Basically, in my codes, I account these 6 kinds of ops.
isinstance(module, torch.nn.Conv2d)
or isinstance(module, torch.nn.Linear)
or isinstance(module, torch.nn.Conv1d)
or isinstance(module, torch.nn.AvgPool2d)
or isinstance(module, torch.nn.MaxPool2d)
or hasattr(module, "calculate_flop_self")
from nats-bench.
And, personally, I do not know either thop or my codes are calculating the precise FLOPs. If you want the best FLOPs numbers, I would recommend https://github.com/facebookresearch/fvcore#features from Facebook.
Unfortunately, I did not realize this tool when I build the benchmark.
from nats-bench.
Really really thank you for your just in time reply. Currently, I am working on extracting each operation's params and flops in every cell from all stacks. Now my confusion about the mismatch between the data offered and tools is solved. I will look into your codes and facebook tools for help.
Many thanks and looking forward to your bravo works!
from nats-bench.
You are always welcome. If you are working on per cell's parameters and flops. fvcore
is definitely what you need. Following their documentation, they have some nice APIs to show the module-level or OP-level FLOPs/#parameters. You can find more at: https://github.com/facebookresearch/fvcore/blob/main/docs/flop_count.md#our-work
from nats-bench.
YES, currently I am reading this doc, coot it is and really thank you for your help
from nats-bench.
Related Issues (20)
- The reasults of test accuracy HOT 2
- get_cost_info(hp="200") returns weird value for some models in topology search space HOT 4
- Problem recreating api in fastmode HOT 2
- The results of validation/test accuracy in NATS-Bench paper HOT 4
- best accuracy find HOT 3
- The code for ImageNet16 is 404 error HOT 4
- How to generate a architecute model with torch HOT 4
- Question regarding the design space for the topology search space in NATS-Bench HOT 1
- api_utils.py invoking get_latency method without passing "hp" parameter HOT 3
- No testset in ImageNet16 HOT 1
- Details about the data structures more_info and cost_info HOT 2
- Question about ImageNet16120 HOT 1
- Is NATS Extension of NAS_201 bench HOT 4
- get_net_param returns empty dictionary HOT 1
- Regarding the checkpoints HOT 3
- What is the seed of subnet training on cifar10, 111 or 777? HOT 3
- Further training of the network obtained from NATS bench
- Regarding inference with pretrained weights.
- get_more_info(): different seed but same 'train-accuracy', 'train-all-time', 'train-all-time'
- Question about the different init model weights of NATS-Bench-tss and NASbench201
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nats-bench.