One question

RLSolver: High-performance GPU-based Solvers for Nonconvex and NP-Complete Problems

We aim to showcase that reinforcement learning (RL) or machine learning (ML) with GPUs delivers the best benchmark performance for large-scale nonconvex and NP-complete problems. RL with the help of GPU computing can obtain high-quality solutions within short time.

Sub-repos

RLCO
TSP

Key Technologies

RL/ML tricks such as learn to optimize and curriculum learning.
OR tricks such as local search and tabu search.
Massively parallel sampling of Markov chain Monte Carlo (MCMC) simulations on GPU using thousands of CUDA cores and tensor cores.
Podracer scheduling on a GPU cloud such as DGX-2 SuperPod.

Key References

Mazyavkina, Nina, et al. "Reinforcement learning for combinatorial optimization: A survey." Computers & Operations Research 134 (2021): 105400.
Bengio, Yoshua, Andrea Lodi, and Antoine Prouvost. "Machine learning for combinatorial optimization: a methodological tour d’horizon." European Journal of Operational Research 290.2 (2021): 405-421.
Peng, Yun, Byron Choi, and Jianliang Xu. "Graph learning for combinatorial optimization: a survey of state-of-the-art." Data Science and Engineering 6, no. 2 (2021): 119-141.
Nair, Vinod, et al. "Solving mixed integer programs using neural networks." arXiv preprint arXiv:2012.13349 (2020).
Makoviychuk, Viktor, et al. "Isaac Gym: High performance GPU based physics simulation for robot learning." Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2021.

Workflow

Datasets

Maxcut:
1. Gset is stored in the "data" folder of this repo. The number of nodes is from 800 to 10000.
2. Syn is the synthetic data obtained by calling the function generate_write in util.py. The number of nodes is from 10 to 50000. The (partial) synthetic data is stored in the "data" folder of this repo. If users need all the synthetic data, please refer to Google Drive or Baidu Wangpan (CODE hojh for China users).
TSP: TSPLIB

Benchmarks

Learning to branch

code 2023 AAAI Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories

code 2021 AAAI Parameterizing Branch-and-Bound Search Trees to Learn Branching Policies

Learning to cut

code 2020 ICML Reinforcement learning for integer programming: Learning to cut

RL/ML-based heuristic

code (greedy) 2017 NeurIPS Learning Combinatorial Optimization Algorithms over Graphs

code (local search) 2023, A Monte Carlo Policy Gradient Method with Local Search for Binary Optimization

code (LKH for TSP) 2021 AAAI Combining reinforcement learning with Lin-Kernighan-Helsgaun algorithm for the traveling salesman problem

Variational annealing

code (VCA_RNN) 2023 Machine_Learning Supplementing recurrent neural networks with annealing to solve combinatorial optimization problems

code (VNA) 2021 Nature_Machine_Intelligence Variational neural annealing

Discrete sampling

code (iSCO) 2023 ICML Revisiting sampling for combinatorial optimization

Classical methods
- Random walk
- Greedy
- Simulated annealing
- Local search
- Beam search
- Tabu search
- Branch-and-bound
- Cutting plane

Solvers to Compare with

Gurobi is the state-of-the-art solver. The license is required, and professors/students at universities can obtain the academic license for free.

SCIP is a well-known open-source solver, and its simplex is commonly used in "learning to branch/cut". SCIP is open-source and free.

Other Solvers

COPT: a mathematical optimization solver for large-scale problems.

CPLEX: a high-performance mathematical programming solver for linear programming, mixed integer programming, and quadratic programming.

Xpress: an extraordinarily powerful, field-installable Solver Engine.

BiqMac: a solver only for binary quadratic or maxcut. Users should upload txt file, but the response time is not guaranteed. If users use it, we recommend to download the sources and run it by local computers.

Store Results

Partial results are stored in the folder "result" of this repo. All the results are stored in Google Drive or Baidu Wangpan (CODE: hojh for China users).

With respect to maxcut, please refer to Maxcut. With respect to TSP, please refer to TSP.

Performance

Maxcut. TSP. Quantum circuits MIMO Compressive sensing

File Structure

RLSolver
└──helloworld
   └──maxcut
        └──data
        └──result
        └──util.py
        └──mcmc.py
        └──l2a.py (ours)
        └──baseline
            └──greedy.py
            └──gurobi.py
            └──random_walk.py
            └──simulated_annealing.py
            └──variational_classical_annealing_RNN
            └──variational_neural_annealing
└──benchmark
   └──maxcut.md
   └──graph_partitioning.md
   └──tsp.md
   └──tnco.md
└──rlsolver (main folder)
   └──util.py
   └──data
      └──graph
      └──quantum_circuits
      └──milp_coefs
      └──binary_coefs
   └──problems
      └──maxcut
          └──baseline
          └──mcmc.py
          └──l2a.py(ours)
      └──tnco
          └──baseline
          └──mcmc.py
          └──l2a.py(ours)
      └──mimo
          └──baseline
          └──mcmc.py
          └──l2a.py(ours)

Finished

MIMO
Maxcut
TNCO
quantum circuits

Graph	Score
g14	3029
g15	2995
g22	13167
g49	5712
g50	10012
g55	10017
g70	9358

sycamore	Result1	Result2	NumSamples	UsedTime
n53 m12	15.478	16.449	185856	56930
n53 m14	16.610	17.748	173568	54152
n53 m16	22.014	32.511	153088	52947
n53 m18
n53 m20	21.782	22.585	148992	58058

	vec_W = th.randn((self.mat_H.shape[0], self.K* self.K), dtype=th.cfloat, device=self.device)
	vec_W = vec_W / th.norm(vec_W, dim=1, keepdim=True)
	self.mat_W = vec_W.reshape(-1, self.K, self.N)

ai4finance-foundation / rlsolver Goto Github PK

rlsolver's Introduction

RLSolver: High-performance GPU-based Solvers for Nonconvex and NP-Complete Problems

Sub-repos

Key Technologies

Key References

Workflow

Datasets

Benchmarks

Solvers to Compare with

Other Solvers

Store Results

Performance

File Structure

Finished

TODO

Related Websites

rlsolver's People

Contributors

Stargazers

Watchers

Forkers

rlsolver's Issues

Update Jan 10, 2023, Extend the environment design from tensor train to tensor networks.

Update Jan 09, 2023

Update Jan 06, 2023

Compare

建议复现他们的 PyTorch版本：

建议安装PyTorch的 0.8.5 版本

建议安装 rdkit 和 boost 的最新版本

运行

sycamore n53 m12

Recommend Projects

Recommend Topics

Recommend Org

Jobs