Comments (6)
Turns out the print_step method is hard-coded to include up to five decimal points, and the points that the GP was probing were below 0.00001. The idea of excluding some points may still be worthwhile, so I won't close the issue just yet.
from bayesianoptimization.
Looking into providing constraints for the scipy minimize function in acq_max from helpers.py. Should only be a one or two line fix with minimal impact on performance.
from bayesianoptimization.
For exclusion of specific points, each point must be reformulated into an equality or inequality constraint. We want a non-equality constraint, i.e. x not equal to some y. This can be reformulated as sum(abs(x_i-y_i))>0
, which can be approximated by sum(abs(x_i-y_i))-tol>=0
for some tol
very close to zero. Numerical stability shouldn't be a problem for the right choice of tol
. I'm on the fence about whether exclusion points should be passed to the __init__
method or passed to their own method the same way points are passed to the explore method (i.e. dict with values as lists). I'll implement one of these possibilities over the weekend.
from bayesianoptimization.
Oy, I just realized the acq_max
function uses L-BFGS-B. I'd have to change that to SLSQP. We could still use L-BFGS-B in cases where no exclusion points have been passed. @fmfn Thoughts?
from bayesianoptimization.
Bayesian Optimization (particularly with UCB acquisition function) is known to having a "thing" for edges, there are even a few papers in the literature of people trying to tackle this, however I didn't consider implementing it since I thought it was out of scope.
Before acq_max
returns it clips the output, making sure it lies within the given bounds. However, clip operates on a closed interval, therefore the edges can be probed exactly.
I rather have the object giving bad, but predictable results (exact edge probing), than performing magic under the hood without the user's knowledge. So in this situation I think it would be better to let the user pass {"learning_rate": 0.0 + 1e-10}
, than to try to figure out that learning_rate==0.0
leads to pathological behaviour.
Finally, one interesting idea would be to generalize the optimization manifold from a box as is currently the case (only params lower and upper bounds are defined), to more interesting topologies. Similarly the ability to put constraints on bounds (such as x > y
), would be quite interesting. But I believe this would be challenging.
from bayesianoptimization.
Awesome, closing this issue. I'll follow up with a new issue for implementing constraints. I think it will be easier than it seems.
from bayesianoptimization.
Related Issues (20)
- add github workflow to link github releases to pypi uploads
- settle on a docstring format and update all docstrings HOT 5
- Constrained Target Space occasionally outputs the wrong parameters for a maximum HOT 1
- Give user access to in acq_max function HOT 3
- `FloatingPointError: underflow encountered in exp` in `optimizer.maximize()` HOT 4
- Save optimizer state and load again in the Suggest-Evaluate-Register Paradigm HOT 1
- Replace custom colour implementation with colorama colours HOT 1
- Constrained optimization does not allow duplicate points HOT 3
- JSONLogger bug in Basic Tour example HOT 2
- Implement `gp_hedge` acquisition function HOT 10
- support multiple parameters HOT 2
- Values outside pbounds HOT 7
- How to specify pbounds when there are no variable names per se HOT 7
- Make n_restarts_optimizer dynamic HOT 2
- pbounds step size HOT 2
- Manually provide observed samples as init points HOT 2
- Pass *args, *kwargs as non-optimized arguments to the cost function HOT 3
- Verbose=2 doesn't work in terminal or bash HOT 1
- 'str object has no attribute reproduce' HOT 5
- add supported python versions badge to readme HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bayesianoptimization.