How to generate a architecute model with torch about nats-bench

[D-X-Y]

Thanks, @KelvinYang0320, for help answer the question!

[D-X-Y]

Thanks for your questions, print(api.arch(12)) can tell you the architecture info, see more details in the updated README (https://github.com/D-X-Y/NATS-Bench#usage).

For the second question, you can see item-3 in (https://github.com/D-X-Y/NATS-Bench#usage), while a prerequisite is to download the pre-trained checkpoints, which is very large..

[NLGithubWP]

Thank you for the answering,

print(api.arch(12)) could give us the cell structure.

The architecture/model is composed by stacking cell multiple times.

I want to get the accuracy and loss of the 12 model for example (get_more_info already give us that ), and also get the full model itself. And then I want to retrain the model with my own datasets. I want to check if the model with high accuracy has also good performance on my datasets.

But i'm still a little bit confuse with generating network related APIs.

import xautodl
from xautodl.models import get_cell_based_tiny_net
config = api.get_net_config(15620, 'cifar10')
info = api.get_more_info(15620, 'cifar10')
network = get_cell_based_tiny_net(config)

In this code, it uses 'get_cell_based_tiny_net' to get the network. And use get_more_info to get the loss and accuracy of the model.

Is the network returned by get_cell_based_tiny_net can give me the full model ? It is quite small and accuracy is low.

In xautod/models package, I also see there are get_cifar_models apis, which can return us basic, resnet, and densenet model.

How to use them ?

Thank you

[KelvinYang0320]

@NLGithubWP You can get this trained network by the following code:

import xautodl
from xautodl.models import get_cell_based_tiny_net
config = api.get_net_config(15620, 'cifar10')
network = get_cell_based_tiny_net(config)

# Load the pre-trained weights: params is a dict, where the key is the seed and value is the weights.
params = api.get_net_param(15620, 'cifar10', None)
network.load_state_dict(params[777])

get_cell_based_tiny_net can give you the neural architecture, but you must get and load the trained weights by the get_net_param and load_state_dict.

As for the test accuracy, you can get it by the following code:

info = api.get_more_info(15620, 'cifar10', is_random = 777)
print(info['test-accuracy'])

After running this trained network on the whole test dataset, you will get the exact same test accuracy.