FLOPS data in nasbench201 are different from flops count tools like thop and ptflops about nats-bench

[AlbertiPot]

code here for building network and using thop to count the flops
``
from xautodl.models import get_cell_based_tiny_net

index = 0
cifar10_config = api.get_net_config(index,'cifar10-valid')
print(cifar10_config)
cifar10_network = get_cell_based_tiny_net(cifar10_config)

from thop import profile
import torch

input = torch.randn(1,3,32,32)
flops, params = profile(cifar10_network, inputs={input,})
print(flops/1e6, params/1e6)
``

[D-X-Y]

I use my own codes: https://github.com/D-X-Y/AutoDL-Projects/blob/main/xautodl/utils/flop_benchmark.py#L42
Basically, in my codes, I account these 6 kinds of ops.

isinstance(module, torch.nn.Conv2d)
        or isinstance(module, torch.nn.Linear)
        or isinstance(module, torch.nn.Conv1d)
        or isinstance(module, torch.nn.AvgPool2d)
        or isinstance(module, torch.nn.MaxPool2d)
        or hasattr(module, "calculate_flop_self")

[D-X-Y]

And, personally, I do not know either thop or my codes are calculating the precise FLOPs. If you want the best FLOPs numbers, I would recommend https://github.com/facebookresearch/fvcore#features from Facebook.
Unfortunately, I did not realize this tool when I build the benchmark.

[AlbertiPot]

Really really thank you for your just in time reply. Currently, I am working on extracting each operation's params and flops in every cell from all stacks. Now my confusion about the mismatch between the data offered and tools is solved. I will look into your codes and facebook tools for help.
Many thanks and looking forward to your bravo works!

[D-X-Y]

You are always welcome. If you are working on per cell's parameters and flops. fvcore is definitely what you need. Following their documentation, they have some nice APIs to show the module-level or OP-level FLOPs/#parameters. You can find more at: https://github.com/facebookresearch/fvcore/blob/main/docs/flop_count.md#our-work

[AlbertiPot]

YES, currently I am reading this doc, coot it is and really thank you for your help