aimclub / golem

Graph Optimiser for Learning and Evolution of Models

Graph Optimization and Learning by Evolutionary Methods

GOLEM is an open-source AI framework for optimization and learning of structured graph-based models with meta-heuristic methods. It is centered around 2 ideas:

1. The potential of meta-heuristic methods in complex problem spaces.

The focus on meta-heuristics allows approaching the kinds of problems where gradient-based learning methods (notably, neural networks) can't be easily applied, like optimization problems with multiple conflicting objectives or having a combinatorial nature.

2. The importance of structured models in multiple problem domains.

Graph-based learning enables solutions in the form of structured and hybrid probabilistic models, not to mention that a wide range of domain-specific problems have a natural formulation in the form of graphs.

Together this constitutes an approach to AI that potentially leads to structured, intuitive, interpretable methods and solutions for a wide range of tasks.

Core Features

• Structured models with joint optimization of graph structure and properties (node attributes).
• Metaheuristic methods (mainly evolutionary) applicable to any task with a well-defined objective.
• Multi-objective optimization that can take into account both quality and complexity.
• Constrained optimization with support for arbitrary domain-specific constraints.
• Extensible to new domains.
• Interpretable thanks to meta-heuristics, structured models, and visualisation tools.
• Reproducible thanks to rich optimization history and model serialization.

Applications

GOLEM is potentially applicable to any optimization problem structures:

• that can be represented as directed graphs;
• that have some clearly defined fitness function on them.

Graph models can represent fixed structures (e.g. physical models such as truss structures) or functional models that define a data-flow or inference process (e.g. bayesian networks that can be fitted and queried).

Examples of GOLEM applications:

• Automatic Machine Learning (AutoML) with optimal ML pipelines search in FEDOT framework
• Bayesian network structure search in BAMT framework
• Differential equation discovery for physical models in EPDE framework
• Geometric design of physical objects in GEFEST framework
• Neural architecture search

As GOLEM is a general-purpose framework, it's easy to imagine potential applications, for example, finite state automata search for robotics control or molecular graph learning for drug discovery, and more.

Installation

GOLEM can be installed with pip:

$ pip install thegolem

Quick Start Example

Following example demonstrates graph search using reference graph & edit distance metric. Optimizer is set up with a minimal set of parameters and simple single-point mutations. For more details see examples simple_run.py, graph_search.py and tree_search.py in directory examples/synthetic_graph_evolution.

def run_graph_search(size=16, timeout=8):
    # Generate target graph sought by optimizer using edit distance objective
    node_types = ('a', 'b')  # Available node types that can appear in graphs
    target_graph = generate_labeled_graph('tree', size, node_types)
    objective = Objective(partial(tree_edit_dist, target_graph))
    initial_population = [generate_labeled_graph('tree', 5, node_types) for _ in range(10)]

    # Setup optimization parameters
    requirements = GraphRequirements(timeout=timedelta(minutes=timeout))
    gen_params = GraphGenerationParams(adapter=BaseNetworkxAdapter(), available_node_types=node_types)
    algo_params = GPAlgorithmParameters(pop_size=30)

    # Build and run the optimizer
    optimiser = EvoGraphOptimizer(objective, initial_population, requirements, gen_params, algo_params)
    found_graphs = optimiser.optimise(objective)

    # Visualize results
    found_graph = gen_params.adapter.restore(found_graphs[0])  # Transform back to NetworkX graph
    draw_graphs_subplots(target_graph, found_graph, titles=['Target Graph', 'Found Graph'])
    optimiser.history.show.fitness_line()
    return found_graph

Tracing the lineage of the found_graph reveals how genetic operators (mutations, crossovers, etc.) are applied to a random graph one after another, eventually leading to the target graph:

One can also notice that despite the fact that the edit distance generally decreases along the genealogical path, the optimizer sometimes sacrifices local fitness gain of some graphs in order to achieve diversity and thus obtain the best possible solution at the end.

Project Structure

The repository includes the following packages and directories:

• Package core contains the main classes and scripts.
• Package core.adapter is responsible for transformation between domain graphs and internal graph representation used by optimisers.
• Package core.dag contains classes and algorithms for representation and processing of graphs.
• Package core.optimisers contains graph optimisers and all related classes (like those representing fitness, individuals, populations, etc.), including optimization history.
• Package core.optimisers.genetic contains genetic (also called evolutionary) graph optimiser and operators (mutation, selection, and so on).
• Package core.utilities contains utilities and data structures used by other modules.
• Package serializers contains class Serializer with required facilities, and is responsible for serialization of project classes (graphs, optimization history, and everything related).
• Package visualisation contains classes that allow to visualise optimization history, graphs, and certain plots useful for analysis.
• Package examples includes several use-cases where you can start to discover how the framework works.
• All unit and integration tests are contained in the test directory.
• The sources of the documentation are in the docs directory.

Current R&D and future plans

Any contribution is welcome. Our R&D team is open for cooperation with other scientific teams as well as with industrial partners.

Contribution Guide

• The contribution guide is available in the repository.

Acknowledgments

We acknowledge the contributors for their important impact and the participants of the numerous scientific conferences and workshops for their valuable advice and suggestions.

Supported by

The study is supported by the Research Center Strong Artificial Intelligence in Industry of ITMO University as part of the plan of the center's program: Development and testing of an experimental prototype of the library of strong AI algorithms in terms of basic algorithms of automatic ML for structural training of composite AI models, including automation of feature selection

Contacts

• Telegram channel for solving problems and answering questions about FEDOT
• Natural System Simulation Team
• Nikolay Nikitin, AutoML Lead ([email protected])
• Newsfeed
• Youtube channel

Citation

If you use our project in your work or research, we would appreciate citations.

@article{nikitin2021automated,

title = {Automated evolutionary approach for the design of composite machine learning pipelines}, author = {Nikolay O. Nikitin and Pavel Vychuzhanin and Mikhail Sarafanov and Iana S. Polonskaia and Ilia Revin and Irina V. Barabanova and Gleb Maximov and Anna V. Kalyuzhnaya and Alexander Boukhanovsky}, journal = {Future Generation Computer Systems}, year = {2021}, issn = {0167-739X}, doi = {https://doi.org/10.1016/j.future.2021.08.022}}

Papers that describe applications of GOLEM:

There are various cases solved with GOLEM's algorithms:

• Algorithms for time series forecasting pipeline design: Sarafanov M., Pokrovskii V., Nikitin N. O. Evolutionary Automated Machine Learning for Multi-Scale Decomposition and Forecasting of Sensor Time Series //2022 IEEE Congress on Evolutionary Computation (CEC). – IEEE, 2022. – С. 01-08.
• Algorithms for acoustic equation discovery: Hvatov A. Data-Driven Approach for the Floquet Propagator Inverse Problem Solution //ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). – IEEE, 2022. – С. 3813-3817.
• Algorithms for PDE discovery: Maslyaev M., Hvatov A. Solver-Based Fitness Function for the Data-Driven Evolutionary Discovery of Partial Differential Equations //2022 IEEE Congress on Evolutionary Computation (CEC). – IEEE, 2022. – С. 1-8.
• Algorithms for structural learning of Bayesian Networks: Deeva I., Kalyuzhnaya A. V., Alexander V. Boukhanovsky Adaptive Learning Algorithm for Bayesian Networks Based on Kernel Mixtures Distributions//International Journal of Artificial Intelligence. – 2023. - Т.21. - №. 1. - С. 90.

golem's Issues

Implement and Adapt SA Serialization

Implement serialization of Structural Analysis results

logger minor fix

issue from FEDOT

Implement co-evolution of subgraphs

The co-evolution of sub-graph should be implemented to make it possible to use different optimization scenarios in multi-modal FEDOT, NAS, etc.

The logic of AtomizedModel class can be derived from FEDOT as a container for sub-graphs.

Implement state embedding of nodes/graphs

It is necessary to understand how to universally and efficiently encode the graph node and the node context.

Why: adaptive mutations can work more efficiently if they have some information about the node. For example, reinforcement learning works taking into account the state: S_n -> A, where the state S_n is the encoded context of the node, and the action A is the most useful mutation.

There are the following thoughts:
⁃ It is worth looking towards the topic of Node Embeddings (node2vec and other approaches)
⁃ The context of the node must somehow take into account the immediate ancestors
⁃ It seems that it is important to take into account the type of node (in automl this is an operation)
⁃ The encoding of a node may involve features of the type of degree of the node and something like that

---

Steps:

• Research literature on node/graph encodings and embeddings
• Implement a prototype
• Then experiment: compare Contextual Bandits with simple Multi-Armed Bandits (MABs).

Implement and Adapt SA Visualizations

Support larger graphs (N>40)

Example of the problem is Tree Search. Or BAMT graphs (see discussion).

The problem is that Tree search is caught in local minima for graphs with N>20. Small graphs are found without problem in 50-100 iterations, while larger graphs (with N>=30 already almost for sure) can't be found. Optimization is stuck after 100-200 iterations with almost no hope. This situation must be investigated.

I have a hypothesis that current mutation set adds cycles and can't remove them.
Also it would be nice to add specific set of constraints for GraphVerifier suited for trees.

Encountered while doing #60

Plan:

1. Profile GOLEM

Possibly issue #142 could provide additional information about where GOLEM is slower compared to other evolutionary frameworks.

---

Updates:

• @ShikovEgor proposed to substitute NetworkX (hich is Pure Python) with something more performant).
• @SuperSashka brought our attention to the fact, that the objective of finding Same Graph can be ill-defined, and that's why evolutionary algorithm in principle can't resolve it. Yet we should conduct comparison with other papers with similar synthetic tasks for graph search.
• An investigation into architectural barriers of GOLEM must be considered.

Implement infrastructure for adaptive mutations

Current architecture of Optimizer must be modified to enable collecting actions & rewards for learning adaptive mutations.

Add tests for SA

Implement crossover for graphs with multiple roots

Currently the function equivalent_subtrees used in crossover of subtrees is designed only for single-root graphs.
GOLEM should have more general-purpose crossover for directed graphs.

Check structural diversity in algorithm

These questions must be resolved:

• If mutation returns same individual, it shouldn't be included into new population. Mutation can return same individual only if mutation_probability says so.
• How to avoid duplicate individuals in inheritance with steady_state?
  Here selection is used with both old & new populations. They can have same individuals -- and they can be selected with higher probability.
• How population growth happens with generational evo scheme? I do not see where population can really grow.
• Why with steady_state selection is called 2 times each iteration (directly from selection and then again in inheritance)? Can we do it using single call to Selection?

Also an aditional feature can be useful to prevent this altogether:

• PopulationalOptimizer once in a while (once in N iter-s) must filter duplicates based on structure of graphs. If structural diversity drops below some threshold. This filtering is optional and enabled by default.

Part of a solution to #89

Add Gif animation of graph evolution to README

Gif animation of graph evolution also can be useful.

Originally posted by @nicl-nno in #48 (comment)

Unstable population size & related eval problems

Adaptive pop size is enabled with parameter_free genetic scheme. There're some problems related to it and to evaluation of populations.

• Adaptive & usual pop_size may stagnate because of too many invalid evaluations. Pop size must increase anyway. Make a test that ensures consistent pop size increase with & without adaptive pop_size.
• Ensure that max_popsize works with adaptive popsize
• Too many Objective eval errors & degenerate populations raise no feedback. Such cases possibly point to problems in optimization -- so this must be more obvious when there're too many errors.
• Initial population size after evaluation must be equal to initial pop size from optimizer parameters. Possibly now it is not always so, it must be checked.

Review literature on used Mutation & Crossover operators for graphs

Motivation: universal graph optimization that could work for many different domains requires mutations that are effective across many tasks. To do that it's required to conduct an initial overview of used mutations.

• Find papers that use evolution of some graphs.
• Describe what kinds of structures are optimized (chemical, robotic, etc.) and what are the constraints for them.
• Describe what mutations are effective for these graphs, and why (if authors say it)

Result is the list of these findings.

After this it will be possible to extract commonalities and understand what mutations we would need for universality

tkinter error

In the process of running the surrogate_example, the following error occurs

It doesn't happen every time. Perhaps it is due to the fact that I did not close the visualization for a long time.
win10, python 3.9

abstract_graph_search example doesn't work

Exception seems to arise because initial_population wasn't successfully evaluated and thus None was given as an input to update method in ParetoFront. Cases when initial_population wasn't successully evaluated must be handled.

Traceback of error:

Implement support for surrogate-assisted optimisation

As a part of MetaAutoML project, the feature of surrogate-assisted evolution is required in FEDOT optmiser.

As a first step, the simpe surrogate scheme can be implemened - e.g. surrogate estimation of fitness before selection.

I think experience of @valer1435 and @maypink from previous projects can be useful. Also, surrogates can be useful to improve
the convergence of sensitivity analysis.

Related paper: https://arxiv.org/pdf/2206.01520.pdf

Add labels for curved edges support

Now labels for such edges are placed in the middle of edge if it was straight:

Label should be placed in the middle of curved edge

GraphVerifier works inccorectly (or art least has_one_root_node rule)

For example, even if has_one_root_node rule is present, GraphVerifier allows graphs with multiple root nodes:

Fix lack of diversity in population (same individuals)

Problem is captured by the following, where we see too many individuals with the same structure:

Comment from @valer1435 on a possible cause:
стоит посмотреть внимательно, как выглядит популяция до этого метода и после. Как будто он запихивает в популяцию лучшие индивиды, не учитывая, что они там могут уже быть

• The issue must be reproduced in GOLEM.
• Algorithm must be fixed
• We must have a new integration test that checks structural diversity in a population using some diversity metric (e.g. edit distance)

Implement Particle Swarm Optimizer and make experiments with it

Swarm optimizer in the context of GOLEM would be a hybrid algorithm:

• particle is a graph
• steps of individuals in the search space (particle moves) are changes to the graph (mutations crossover).

This idea is based on the following paper.

Experiments must include comparison of effectiveness (speed of convergence, final metric) with Random mutation optimizer & GPGraphOptimizer. Experiments should be based on tree_search and graph_search examples.

Change usages of OptGraph for Graph interface

Currently OptGraph is just a historical alias for Graph implementation.
Instead Graph interface must be used everywhere in interfaces, functions etc. Wheres OptGraph is useful only for creation of objects.

tkinter error

In the process of running the surrogate_example, the following error occurs

It doesn't happen every time. Perhaps it is due to the fact that I did not close the visualization for a long time.
win10, python 3.9

Implement Structural Analysis of graphs

• The task is to generalize sensitivity analysis in FEDOT for arbitrary graphs.
• Example must be added to examlples
• Documentation page should also be provided.

Tuner refactoring

Now tuner uses metrics that not compatible with composer metrics. Also it has a monolith structure.

What should be done:

• Union metrics for composer and tuner
• Make tuner more flexible using decomposition
• Implement self._objective as separate module
• Refactor view of model parameters. Now they become too complicated to store them into dict (see aimclub/FEDOT#862)

UPD (from @gkirgizov) Additional subtasks:

• HyperOpt.get_metric_value seems to duplicate logic of PipelineObjectiveEvaluate that's concerned with how to validate the model -- so it's better to use it instead.
• Abstract Tuner to arbitrary Graphs.
  The only place where the specifics of Pipeline are required -- is for evaluation of objective. But, I think, currently this can be encapsualted with Objective/ObjectiveEvaluate classes.

Regarding 1st and 3rd points by Valera, possibly Objective class is useful here.

Research on Adaptive Mutations

The idea is to adapt mutations during optimizations process, so that mutation probabilities would help in faster and better convergence.

Each mutation can be considered as Action in certain State (graph or node context) with certain Reward. This perspective opens several possibilities of multi-armed bandit formulation and RL formulation of adaptive mutations.

Subissues:

• #39
• #41
• #42
• #77
• #43
• #98

Following is also useful:

• #38
• #83

Docs: add auto-docs (api reference) to RTD

Add tests for regularization operator

Currently there're no tests for regularization operation (or at least PyCharm sees no usages).
Moreover, as it's not used in any example nor by default in optimizer.

And there're some hints that it's broken, cause it operates only on fitted subtrees, but OptGraphs by default are restored without fitted_operations that can be found only in operations cache.

Tests must be written and operator should be fixed if it doesn't work.

Align visualization for similar graphs

Function draw_graphs_subplots plots several graphs for comparison at once.

The problem is that plotting logic can draw very similar graphs in different layouts. Layouts must be aligned for simpler visual interpretation.

Example:

tkinter error

In the process of running the surrogate_example, the following error occurs

It doesn't happen every time. Perhaps it is due to the fact that I did not close the visualization for a long time.
win10, python 3.9

Experiment with RL impl of Adaptive Mutations

This issue requires #41 to be implemented.

Experiments must be done on synthetic_graph_evolution examples (graph_search and tree_search).

Add example of Tree search with Edit Distance metric

Tree search can include more effective and precise Edit Distance metric (e.g. Zhang-Sasha algorithm).
For Graph search example it's not possible due to unreasonable running time of ED metric compute from networkx.

add options for saving in SA

Implement universal & complete set of mutations

Current set of mutations (that came from FEDOT) is aimed at finding short trees and dags. Example of arbitrary graph search shows that these mutations are not sufficient for other use cases.

Based on this observation, the task here is to find a new, complete and universal set of mutations, that would enable effective search of different kinds of graph structures. Complete and universal means that such mutation set should be sufficient for any target graphs.

I see following steps:

• #59
• Design complete set of mutations
• Experiment with it based on graph & tree search examples

Experiment cases must achieve optimization objective only with provided mutations (no crossover).

Possibly issue #38 would help in experiments.

Fix recursion errors in graph functions for cyclic graphs

The problem is because of recursive traversals that break on cycles. The task is:

• Reproduce issue, write tests for graphs with loops
• solution is to rewrite recursion in a usual loop with stack.

Affected functions:

• in graph_growth the bug is sure
• descriptive_id_recursive must be checked
• distance_to_primary_level
• maybe more...

Add simple example to README

There is a lack of simple of example of graph design in README (without complex wrappers like FEDOT)

Fix `parameter_free` genetic scheme

This genetic scheme parameter_free currently when used in graph_search.py and tree_search.py examples shows no improvements at all.

Experiment with MultiArmedBandit implementation of Adaptive Mutations

Following libs could help:

• https://pypi.org/project/mabwiser/
• https://github.com/tensorflow/agents#Multi-Armed%20Bandits

This issue requires #41 to be implemented

Implement API Parameter Facade

While doing aimclub/FEDOT#852 I came up with a design of a better Parameter system. The idea is to make Facade for all different parameter dataclasses. This will allow isolate external API from changes to actual parameter dataclasses.

Requirements:

• Facade should work on sub-categories (now we have 3: infrastructural ComposerRequirements, PipelineComposerRequirements, GraphOptimizerParameters)
• Unique names check (parameters from different categories shouldn't have same names)
• Support for new categories without additional work (imagine some CustomOptimizerParams)
• Facade must support init from flat dictionary of parameters (e.g. user povides different parameters from diffeernt categories in a single dictionary) --- btw this will allow refactoring ApiParams with its logic of divide_parameters
• Updates to underlying fields in dataclasses should be hidden behind common facade of get/set (by reimplemented getattr/setattr/hasattr)
• Strictness feature: only params explicitly marked as "mutable" can be modified; others are frozen by default
  e.g. possibly implement this by a decorator on dataclass fields @mutable_param
• Unique names check (parameters from different categories shouldn't have same names)

Usages could look like:

• Init with categories specified as kwargs (should work for arbitrary kwargs, custom_optimizer_params is an example).:

ParameterRepository(requirements=ComposerRequirements(),
                                   optimizer_parameters=GraphOptimizerParameters(),    
                                   custom_optimizer_params=SomeDataclassWithUserParams())

• para_repo.init_from_parameters(api_params: dict) should automatically distribute all parameters to appropriate subcategories
• 2 ways for accessing parameters should work: for example, for pop_size both param_repo.pop_size & param_repo.optimizer_parameters.pop_size

Conduct experiments on supervised learning with Adaptive agent

Current idea in #40 is to work with online agents that learn while optimization is run, generally in on-policy RL-fashion.

Another possibility is to try Supervised Learning paradigm:

• Collect a dataset of histories for certain class of tasks (like for molecular findings)
• Train a supervised agent on it (algorithm can be anything, from Contextual MAB to linear regression to simple+fast neural SOTA-architecture)
• Validate GOLEM using learnt action probabilities, see if hypothesis holds: GOLEM should run faster & better with trained agent

Tasks can be (a) search of synthetic graphs of different kinds (b) molecular search

Support graph grammars for graph generation

The idea is to provide a way to generate only valid graphs during evolution (mutations/crossovers).

there's already some rudimentary support for constrained generation. As a sketch, the ChangeAdviser could play a role of a graph grammar generator. For example, mutation could ask ChangeAdvisor which nodes are allowed in a current node context.

This approach would be a more effective alternative to GraphVerifier. GraphVerifier discards already generated invalid graphs, whereas graph grammar approach wouldn't allow them to appear and waste time on them.

What must be here:

• Support for Context Free Grammars
• Explicit support for generation of terminal and non-terminal nodes
• It must be possible to check already generated graph, if it corresponds to the grammar or not.
  Possibly there're already tools for that. This would be useful at least for testing.

There's already a real use-case for that in https://github.com/aimclub/rostok library, that uses GOLEM.

Return metric names to `OptHistory`

Earlier, histories stored the objective, which contained information about metrics used during the optimization. FEDOT.Web relied on this field to get metric names.

Thus, metric names should be placed in a separate named field of OptHistory.

single_drop_mutation fails to remove node

bug with node deletion in RemoveType.with_direct_children

tkinter error in surrogate visualization

In the process of running the surrogate_example, the following error occurs

It doesn't happen every time. Perhaps it is due to the fact that I did not close the visualization for a long time.
win10, python 3.9

Provide richer info about nodes in mutations

Currently mutations request is_primary & is_secondary nodes from node_factory. Possibly that's too specific design.

The issue is to make it more universal.

Imagine custom mutations that want to request new nodes from the node_factory based on other topological features (e.g. is_final).

---

Общий вопрос - стоит ли так жестко передавать в get_node именно is_primary? Вдруг будет нужно например "is_final" и тд.

Мб реализовать какую-то структуру с описанием топологических свойств узла в графе?

Originally posted by @nicl-nno in #55 (comment)

Implement warm-start of adaptive mutation agent

Не в этом PR-е, но в будущем: было бы полезно реализовать подгрузку пред-обученного агента (и, соответственно, его сохранение). Чтобы он стартовал с какого-то warm start-а.

Originally posted by @nicl-nno in #78 (comment)

Add evolutionary algorithms from GEFEST

There are golem-compatible implementations of AGE, MOEA-DD and NSGA2 algorithms are implemented aimclub/GEFEST#52.

It can be moved to GOLEM to make it more effective for multi-objective tasks.

Research indirect genotype encoding of graph phenotype

This is based on this GECCO'22 paper. At least one of the authors is a quite known researcher in evolutionary field, so likely he knows what he talks about.

The idea of the paper is that indirect rich encodings of individuals (i.e. with genotypes that are much richer that phenotypes) can lead to exponential improvements in evolutionary algo. As paper says, this is because with rich genotypes (e.g. evolving strings using neural network genotype) evolution "accumulates" valuable information in the genom, while for simple direct genotypes (e.g. encoding strings as strings) evolution can't and behaves more like an advanced random search.

In GOLEM currently we have direct encoding (graph phenotype is represented directly with graph genotype). The hypothesis is that with richer genotype (encoding graph with generative neural network?...) evolution can be much more effective. Possibly that would help in resolving #69, where evolution converges. This example would also server as experiment in testing the hypothesis.

Formal details and requirements for such a genotype are in the paper. Examples there are worth reading.

Enhance Graph.delete_node for work with cyclic graphs and several roots

Currently cyclic graphs and graphs with several roots cause exceptions. Reproduce, write tests and fix issues.

Migrate scalable visualizations from a side project

In recent changes for a side project (private repo), I made graph visualizations more scalable.
These changes should be migrated to GOLEM.

Fix dependency conflict from `netcomp` package

Solution can be either one of:

• make a PR to the package repository
• make a fork with updated dependencies of netcomp
• copy their metrics code to our repository but don't forget the attribution & license!