Quan Minh Phan, Ngoc Hoang Luong
- Clone this repo:
$ git clone https://github.com/ELO-Lab/ENAS_TFI
$ cd ENAS_TFI
- Install dependencies:
$ pip install -r requirements.txt
- For single-objective NAS problems:
python main.py --n_runs 21 --warm_up 0 --nSamples_for_warm_up 0 --problem_name [problem_name] --algorithm_name GA --seed 0
- For multi-objective NAS problems:
python main.py --n_runs 21 --warm_up 0 --nSamples_for_warm_up 0 --problem_name [problem_name] --algorithm_name NSGA-II --seed 0
--problem_name [problem_name]
receives one of following values:
problem_name | NAS Benchmark | Type of problem | Dataset | Objecitve |
---|---|---|---|---|
SO-NAS101 | NAS-Bench-101 | single-objective | CIFAR-10 | validation error |
SO-NAS201-1 | NAS-Bench-201 | single-objective | CIFAR-10 | validation error |
SO-NAS201-2 | NAS-Bench-201 | single-objective | CIFAR-100 | validation error |
SO-NAS201-3 | NAS-Bench-201 | single-objective | ImageNet16-120 | validation error |
MO-NAS101 | NAS-Bench-101 | multi-objective | CIFAR-10 | #params & validation error |
MO-NAS201-1 | NAS-Bench-201 | multi-objective | CIFAR-10 | FLOPs & validation error |
MO-NAS201-2 | NAS-Bench-201 | multi-objective | CIFAR-100 | FLOPs & validation error |
MO-NAS201-3 | NAS-Bench-201 | multi-objective | ImageNet16-120 | FLOPs & validation error |
To search with the Warmup method, set --warm_up 1
and set the number of samples --nSamples_for_warm_up
. In our experiments, we set --nSamples_for_warm_up 500
.
To experiment with the different population_size
or maximum_number_of_evaluations
, set another value in main.py (for population_size
) and factory.py (for maximum_number_of_evaluations
)
- For single-objective NAS problems:
python visualize_so.py --path_results [path_results]
- For multi-objective NAS problems:
python visualize_mo.py --path_results [path_results]
For example: python visualize_mo.py --path_results .\results\MO-NAS101
Note: [path_results]
must only contains results of experiments are conducted on the same problem.
Our source code is inspired by:
- pymoo: Multi-objective Optimization in Python
- NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
- How Powerful are Performance Predictors in Neural Architecture Search?