markm541374 / gpc

GPshared

This project is a c/c++ shared library and python wrapper to be used for Gaussian Process Global Optimisation. The GPpython folder contains a separate python project for Bayesian Optimization

The cpp library and python wrapper implement Gaussian processes with observations and inference up to second derivatives in multiple axes, and posterior inference over sets of equally weighted hyperparameter draws

Usage:

running main.py should run some demos and show the plots in turn. The demos are

• making some draws from a GP
• showing observation and inference of derivatives
• drawing hyperparameter samples from from the posterior
• optimizing branin with zero mean noise but with a cost relation on variance, variable fidelity outperforms fived for PES and EI
• optimizing a synthetic fn with biased results but low noise and a cost relation on how close to true the result is. PES with env variable outperforms fixed at any point.

Dependencies (that I can think of):

numpy/scipy matplotlib pathos.multiprocessing(shouldn't be needed for demos as the result files are in the cache) DIRECT tqdm

Files:

In c lib

• libGP.cpp: main file for cpp library, maintains a vector of GPs and handles calls on them, also a function to return log likelihood only
• GPsimple.cpp: GP implementation, methods for infering mean only, diagonal variance or full covariance
• direct.cpp: direct from external source
• bayesutils.cpp: EI, LCB and logEI functions
• hypsearch.cpp: fn to run direct to find MLE or MAP hyperparameters given data (and prior for MAP)
• kernel.cpp: has or imports all the kernel functions, taking numnbers correspoinding to derivatives in each axis of each x1 and x2, converters for hyperparameters from natural (length) to hte form used in kernel (1/l**2) and to log space for searching
• matern.cpp: the matern 5.2 kernel
• misctool.cpp: normal pdf and cdf, EI and logEI draw samples from a covariance matrix or cholesky decomp.
• simplekernels.cpp: some sums and productts of other kernels, derivatives not implemented.

In python lib

• GPdc.py: imports the c library and provides interface
• slice.py: slice samples adapted from 3rd party
• search.py: wrapper for search algorithms, sets up parameters and runs the search
• eprop.py: expectation propagation for hard and soft inequalities on multivatiate normals
• optutils.py: selection of synthetic objectives and implementation of all the searches, mostly inheriting from a base
• ESutils.py: draws support points and mins from support given a GP, draws hyperparameters from data given prior
• PES.py: implementation of PESaquisition function, regular and env variable versions.
• test* : tests for all the above
• exp* : experiments according to comments in header

</p>

Using External Code:

DIRECT implementation
Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without ! the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)