lucadellalib / learning-to-evolve

Deep reinforcement learning for adaptation in evolutionary algorithms

Learning to Evolve

This is the official implementation of the paper "Learning to Evolve" [Arxiv].

Prerequisites

Python 3
CUDA
Numpy
Pytorch 0.41+

Examples

We provide config files and pretrained weights for all performed experiments.
To run these scripts, first replace all placeholder directory names with the directory you cloned this project into.

$ apt-get install rpl
$ rpl -x .json -R "/path/to" "[ACTUAL_DIRECTORY]" ./configs 

To perform testing with pretrained weights, run the apply_solver script as follows

$ python apply_solver.py configs/path/to/method/config_validation_pretrained.json

If you want to train a model from scratch, run the apply_solver script as follows:

$ python apply_solver.py configs/path/to/method/config_training.json

An explanation of the config file parameters can be found in readme/scripts/apply_solver.md.

Scripts

There are five relevant scripts. For details, consult the respective readme files under readme/ or use --help.

• apply_solver.py config-file
  Reads parameters from the provided config-file to generate evolutionary algorithms.
  These evolutionary algorithms can then be trained and/or validated on problem sets. The default config file can be found under data/

• evaluation/plot_stats.py [options] output-file input-files
  Reads data from stats.npy files generated by apply_solver.py (input-files), generates a plot and stores it in output-file.
  Each file must assigned a label, using --label in the order of the input-files.
  Can be used in two modes:

  • Normal mode: Creates a generation vs fitness graph per input stats file.
  • Training progress mode: Creates a training iteration vs terminal fitness graph per input (input_files must be folders containing stats files)

• evaluation/print_stats.py [options] input-files
  Reads data from stats.npy files generated by apply_solver.py (_input-files) and prints performance statistics.
  The --g option must be used to specify the generations to print statistics about.
  Can be used in two modes:

  • Normal mode: Prints statistics per element of input-files
  • Merge mode: Combines all stats.npy files contained in the input-files (must be folder names) and prints overall statistics.

• evaluation/generate_knapsack.py [options] output-file
  Generates a set of problem instances for the 0-1 knapsack problem and stores them as a .json-file output-file, for later use by apply_solver.

• evaluation/generate_tsp.py [options] output-file
  Generates a set of problem instances for the traveling salesman problem and stores them as a .json-file output-file, for later use by apply_solver.

Extending the code

If you plan to add new problem instances or evolutionary operators, consult readme_developers.md under readme for an overview of the necessary steps.