cma-es / pycma Goto Github PK

View Code? Open in Web Editor NEW

1.1K 1.1K 176.0 2.5 MB

Python implementation of CMA-ES

License: Other

Python 57.44% Shell 0.01% Jupyter Notebook 42.55%

pycma's People

Contributors

Stargazers

Watchers

Forkers

hyiltiz youheiakimoto topological-slam drallensmith abdellah-lamrani-alaoui hulalazz ml-lab sylvia1664 markm541374 aambrosano truenicoco mahjiong brockho aigaosheng zhangxin658 lamperougeyxy matt-chan aravindsrinivas kelvinson afcarl eachonly alfonsord cdesilv1 uyoung-jeong tamizh3110 vixuowis linxiyao xabomon scornfulsunnyd tmorzadec mohammedgomaa baoyuanzh oremy trading42 vin136 42machinelearning lq-model-gecco2019 cnheider bzp92 tek1031 rebcabin fdeu jeremiedecock zhenfengcao banhdzui tdczlhb ambigev stjordanis tuliplan sweet-chanyelo themetal mfkasim1 dongfangyixi 1kastner kejiejiang ssnl itsrrworld snazari jimmycao ndsanders science4fun yanyzhao arf1 hzhip septumcapital knaggita glendonhu guqiloveqiyang gnithie garyxiangk urizy x-rayscattering cheikhtoure tysh0738 dongxulee midoriao ahafaiin seba-1511 jyyang5 orestmy zyhuster1 tovrstra myausweis takahh sparsel maximerinjard awikner yyuzhongpv julienmartinelli davinci110-xiaofan lucas-cg good-repos scam2018 fywu yaohuic alien-ware enajx agrabski pengdauan gaetandesrues

pycma's Issues

How to set bounds

Hi,

I can't work out how to set bounds for CMAESvolutionStrategy.
Could you explain?

Thank you

"if 11 < 3:"?

pycma/cma/purecma.py

Line 1211 in 0e4a1ba

if 11 < 3:

Is this intended to comment out this section? If so, it could be a bit clearer...

multidimensional parameters raises AttributeError

passing a numpy.ndarray with more than one dimension raises the following error:

Traceback (most recent call last):
  File "cma_bug.py", line 10, in <module>
    res = cma.fmin(objective, params, 1.0)
  File "[snip]\cma\evolution_strategy.py", line 3885, in fmin
    es = CMAEvolutionStrategy(x0, sigma_factor * sigma0, opts)
  File "[snip]\cma\evolution_strategy.py", line 1324, in __init__
    self._set_x0(x0)  # manage weird shapes, set self.x0
  File "[snip]\cma\evolution_strategy.py", line 1620, in _set_x0
    if self.opts.eval('verbose') >= 0:
AttributeError: 'CMAEvolutionStrategy' object has no attribute 'opts'

I can see from the source that this isn't supported anyway, but it fails to reach the code that raises the more informative error.

PS. this package is amazing, thank you for your hard work!

using scipy.optimize.OptimizeResult

cma.fmin and/or cma.CMAEvolutionStrategy.result may, at least optionally, return a scipy.optimize.OptimizeResult class instance instead of a cma.CMAEvolutionStrategyResult class instance.

test_direct.py failing: "data type must provide an itemsize"

After changing the print statements in test_direct.py to function calls in accordance with Python 3.x, calling

python test_direct.py

returns the following error:

$ python test_direct.py
DIRECT Version 2.0.4
 Problem Dimension n                    :      4
 Eps value                              :   0.1000E-03
 Epsilon is constant.
 Maximum number of f-evaluations (maxf) :  20000
 Maximum number of iterations (MaxT)    :   6000
 Value of f_global                      :  -0.1000+101
 Global percentage wanted               :   0.1000E-01
 Volume percentage wanted               :  -0.1000E+01
 Measure percentage wanted              :  -0.1000E+01
 Jones original DIRECT algorithm is used.
Bounds on variable x 1    :    -10.00000 <= xi <=     10.00000
Bounds on variable x 2    :    -10.00000 <= xi <=     10.00000
Bounds on variable x 3    :    -10.00000 <= xi <=     10.00000
Bounds on variable x 4    :    -10.00000 <= xi <=     10.00000
---------------------------------------------------------------------------
Call-back cb_fcn_in_direct__user__routines failed.
Traceback (most recent call last):
  File "test_direct.py", line 19, in <module>
    u
  File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/DIRECT-1.0.1-py3.6-linux-x86_64.egg/DIRECT/__init__.py", line 202, in solve
ValueError: data type must provide an itemsize

The conda environment has the following packages:

$ conda list
# packages in environment at /home/ubuntu/anaconda3/envs/tensorflow_p36:
#
_nb_ext_conf              0.4.0                    py36_1
anaconda-client           1.6.5            py36h19c0dcd_0
asn1crypto                0.22.0           py36h265ca7c_1
autograd                  1.2                       <pip>
autovizwidget             0.12.5                    <pip>
autovizwidget             0.12.1                   py36_0
blas                      1.1                    openblas    conda-forge
bleach                    1.5.0                     <pip>
bleach                    2.0.0            py36h688b259_0
bokeh                     0.12.10          py36hbb0e44a_0
boto3                     1.4.7                     <pip>
botocore                  1.7.43                    <pip>
bzip2                     1.0.6                         1    conda-forge
ca-certificates           2017.08.26           h1d4fec5_0
cairo                     1.14.6                        5    conda-forge
certifi                   2017.7.27.1      py36h8b7b77e_0
certifi                   2017.11.5                 <pip>
cffi                      1.10.0           py36had8d393_1
chardet                   3.0.4            py36h0f667ec_1
clyent                    1.2.2            py36h7e57e65_1
cma                       2.3.1                     <pip>
cryptography              2.0.3            py36ha225213_1
cycler                    0.10.0                    <pip>
Cython                    0.27.3                    <pip>
dbus                      1.10.22                       0    conda-forge
decorator                 4.1.2            py36hd076ac8_0
docutils                  0.14                      <pip>
emcee                     2.2.1                     <pip>
entrypoints               0.2.3            py36h1aec115_2
enum34                    1.1.6                     <pip>
environment-kernels       1.1                       <pip>
expat                     2.2.5                he0dffb1_0
ffmpeg                    3.2.4                         2    conda-forge
fontconfig                2.12.1                        5    conda-forge
freetype                  2.7                           2    conda-forge
future                    0.16.0                    <pip>
george                    0.2.1                     <pip>
gettext                   0.19.7                        1    conda-forge
giflib                    5.1.4                         0    conda-forge
glib                      2.51.4                        0    conda-forge
gmp                       6.1.2                hb3b607b_0
graphviz                  2.38.0                        6    conda-forge
gst-plugins-base          1.8.0                         0    conda-forge
gstreamer                 1.8.0                         2    conda-forge
h5py                      2.7.1                    py36_2    conda-forge
harfbuzz                  1.3.4                         2    conda-forge
hdf5                      1.10.1                        1    conda-forge
hdijupyterutils           0.12.5                    <pip>
hdijupyterutils           0.12.1                   py36_0
hpolib2                   0.0.1                     <pip>
html5lib                  0.999999999      py36h2cfc398_0
html5lib                  0.9999999                 <pip>
icu                       58.2                 h211956c_0
idna                      2.6              py36h82fb2a8_1
intel-openmp              2018.0.0             h15fc484_7
ipykernel                 4.6.1            py36hbf841aa_0
ipython                   6.2.1                     <pip>
ipython                   6.1.0            py36hc72a948_1
ipython_genutils          0.2.0            py36hb52b0d5_0
ipywidgets                7.0.4                     <pip>
ipywidgets                7.0.0            py36h7b55c3a_0
jasper                    1.900.1                       4    conda-forge
jedi                      0.10.2           py36h552def0_0
jedi                      0.11.0                    <pip>
jinja2                    2.9.6            py36h489bce4_1
Jinja2                    2.10                      <pip>
jmespath                  0.9.3                     <pip>
jpeg                      9b                   h024ee3a_2
jsonschema                2.6.0            py36h006f8b5_0
jupyter                   1.0.0                     <pip>
jupyter-console           5.2.0                     <pip>
jupyter-core              4.4.0                     <pip>
jupyter_client            5.1.0            py36h614e9ea_0
jupyter_core              4.3.0            py36h357a921_0
Keras                     2.1.0                     <pip>
krb5                      1.14.2               h56d4222_4
Lasagne                   0.1                       <pip>
libedit                   3.1                  heed3624_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 7.2.0                h7cc24e2_2
libgfortran               3.0.0                         1
libgfortran-ng            7.2.0                h9f7466a_2
libiconv                  1.15                 h63c8f33_5
libpng                    1.6.28                        2    conda-forge
libprotobuf               3.4.0                         0
libsodium                 1.0.13               h31c71d8_2
libstdcxx-ng              7.2.0                h7a57d05_2
libtiff                   4.0.7                         1    conda-forge
libtool                   2.4.6                h544aabb_3
libwebp                   0.5.2                         7    conda-forge
libxcb                    1.12                          1    conda-forge
libxml2                   2.9.4                h6b072ca_5
Markdown                  2.6.9                     <pip>
markupsafe                1.0              py36hd9260cd_1
matplotlib                2.1.0                     <pip>
mistune                   0.8.1            py36h3d5977c_0
mkl                       2018.0.0             hb491cac_4
mock                      2.0.0                     <pip>
nb_anacondacloud          1.4.0                    py36_0
nb_conda                  2.2.1            py36h8118bb2_0
nb_conda_kernels          2.1.0                    py36_0
nbconvert                 5.3.1            py36hb41ffb7_0
nbformat                  4.4.0            py36h31c9010_0
nbpresent                 3.0.2            py36h5f95a39_1
ncurses                   6.0                  h9df7e31_2
nose                      1.3.7                     <pip>
notebook                  5.2.1            py36h690a4eb_0
numpy                     1.13.3          py36_blas_openblas_201  [blas_openblas]  conda-forge
openblas                  0.2.20                        4    conda-forge
opencv                    3.3.0           py36_blas_openblas_202  [blas_openblas]  conda-forge
openssl                   1.0.2m               h8cfc7e7_0
pandas                    0.21.0           py36h78bd809_1
pandoc                    1.19.2.1             hea2e7c5_1
pandocfilters             1.4.2            py36ha6701b7_1
pango                     1.40.3                        3    conda-forge
parso                     0.1.0                     <pip>
pbr                       3.1.1                     <pip>
pcre                      8.41                 hc71a17e_0
pexpect                   4.2.1            py36h3b9d41b_0
pexpect                   4.3.0                     <pip>
pickleshare               0.7.4            py36h63277f8_0
pip                       9.0.1            py36h6c6f9ce_4
pixman                    0.34.0                        0    conda-forge
plotly                    2.1.0            py36h56a57e5_0
plotly                    2.2.1                     <pip>
prompt_toolkit            1.0.15           py36h17d85b1_0
protobuf                  3.4.0                    py36_0
psycopg2                  2.7.3.2                   <pip>
ptyprocess                0.5.2            py36h69acd42_0
py4j                      0.10.4                    <pip>
pycparser                 2.18             py36hf9f622e_1
pygal                     2.4.0                     <pip>
pygments                  2.2.0            py36h0d3125c_0
pykerberos                1.1.14           py36h84109d8_2
pyopenssl                 17.2.0           py36h5cc804b_0
pyparsing                 2.2.0                     <pip>
pyrfr                     0.7.0                     <pip>
pysocks                   1.6.7            py36hd97a5b1_1
pyspark                   2.2.0                     <pip>
python                    3.6.3                h1284df2_4
python-dateutil           2.6.1            py36h88d3b88_1
pytz                      2017.2           py36hc2ccc2a_1
pytz                      2017.3                    <pip>
pyyaml                    3.12             py36hafb9ca4_1
pyzmq                     16.0.3                    <pip>
pyzmq                     16.0.2           py36h3b0cf96_2
qt                        5.6.2                         3    conda-forge
qtconsole                 4.3.1                     <pip>
readline                  7.0                  ha6073c6_4
requests                  2.18.4           py36he2e5f8d_1
requests-kerberos         0.11.0           py36h333c0e5_1
s3transfer                0.1.11                    <pip>
scipy                     1.0.0                     <pip>
scipy                     1.0.0           py36_blas_openblas_201  [blas_openblas]  conda-forge
setuptools                36.7.2                    <pip>
setuptools                36.5.0           py36he42e2e1_0
sgmcmc                    0.0.1                     <pip>
simplegeneric             0.8.1            py36h2cb9092_0
six                       1.11.0           py36h372c433_1
sparkmagic                0.12.5                    <pip>
sparkmagic                0.12.1                   py36_0
SQLAlchemy                1.1.15                    <pip>
sqlite                    3.20.1               hb898158_2
tensorflow-gpu            1.4.0                     <pip>
tensorflow-tensorboard    0.4.0rc2                  <pip>
terminado                 0.6              py36ha25a19f_0
testpath                  0.3.1            py36h8cadb63_0
Theano                    1.0.0                     <pip>
tk                        8.6.7                hc745277_3
tornado                   4.5.2            py36h1283b2a_0
traitlets                 4.3.2            py36h674d592_0
urllib3                   1.22             py36hbe7ace6_0
wcwidth                   0.1.7            py36hdf4376a_0
webencodings              0.5.1            py36h800622e_1
Werkzeug                  0.12.2                    <pip>
wheel                     0.30.0                    <pip>
wheel                     0.29.0           py36he7f4e38_1
widgetsnbextension        3.0.7                     <pip>
widgetsnbextension        3.0.2            py36hd01bb71_1
x264                      20131217                      3    conda-forge
xorg-libxau               1.0.8                         3    conda-forge
xorg-libxdmcp             1.1.2                         3    conda-forge
xz                        5.2.3                h55aa19d_2
yaml                      0.1.7                h014fa73_2
zeromq                    4.2.2                hbedb6e5_2
zlib                      1.2.11               ha838bed_2

CMADataLogger instance has a dynamic data path

When using a CMADataLogger instance interactively, it reads the data always from the current folder. This is a kind-of unexpected behavior. The filename prefix should probably better be constructed with the full current path at instantiation prepended. The current behavior can always be recovered by just generating a new instance.

VD-CMA-ES test

The VD-CMA-ES test here in test.py sometimes fails. The outcome remains stochastic even when np.random.seed is set in advance and the rotation of the cigar function is omitted.

TPA fails with calling `ask` twice

most likely because ask assumes two injected solutions to be present, but they have already been processed in the first call of ask.

import cma
function = cma.ff.rosen
dimension = 11
es = cma.CMAEvolutionStrategy(dimension * [0.1], 0.5, 
                              {'AdaptSigma': cma.sigma_adaptation.CMAAdaptSigmaTPA
                              })
while not es.stop():
    X = es.ask()  
    X = es.ask()  
    F = [function(x) for x in X]
    es.tell(X, F)
    es.logger.add()
    es.disp(1)
es.plot()
es.result_pretty();

boundary_handler error

When calling fmin for multidimensional functions with non-None bounds, where the bounds are provided as numpy arrays, line 113 from cma/constraints_handler.py throws the following error:

File "/usr/local/lib/python3.5/dist-packages/cma/evolution_strategy.py", line 3822, in fmin
  es = cma.CMAEvolutionStrategy(x0, sigma_factor * sigma0, opts)
File "/usr/local/lib/python3.5/dist-packages/cma/evolution_strategy.py", line 1377, in __init__
  if not self.boundary_handler.has_bounds():
File "/usr/local/lib/python3.5/dist-packages/cma/constraints_handler.py", line 114, in has_bounds
  if bounds in (None, [None, None]):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Assure that the bounds are given as a list

If bounds are defined as follows:

cma.fmin(
        cost, init, sigma,
        args=args,
        options={
            "bounds": [l_bounds, u_bounds]
        })

where l_bounds and u_bounds are numpy error, the following error is thrown:

~/Documents/code/python/pycma/cma/constraints_handler.py in has_bounds(self)
106 """return True if any variable is bounded"""
107 bounds = self.bounds
--> 108 if bounds in (None, [None, None]):
109 return False
110 for ib, bound in enumerate(bounds):

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Could you raise an appropriate error for easier debugging? Maybe add conversion into a list at the beginning of fmin function or in constraints_handler?

fixed variables interactions

The fixed_variables option of CMAEvolutionStrategy interacts with minstd and maxstd and integer_variables options. This leads to an error for the former and probably to unexpected results for the latter. We may consider to depreciate fixed_variables in favour of fitness_transformations.FixVariables. In this case the values for above options would need to be explicitly managed by the user.

`import cma` changes matplotlib behavior

The following script, run from the console, produces a simple graph

#!/usr/bin/env python
import numpy as np
import matplotlib.pyplot as pl

#import cma

x = np.linspace(0, 10, 1000)
y = np.sin(x)

pl.figure()
pl.plot(x, y)
pl.show()

But when I uncomment the cma import, it stops producing any output!

std explode to very high values with VkD

Hey,

I'm testing CMA-ES on a high input space( >1000 parpameters)
When I run the regular CMA sampler my std stays more or less the same over multiple iterations. ( no decrease)
When I try the VdK sampler, the std increases exponentionally.
I tried to set the maxstd option to control this but it does not seem to work ( std still increases above max threshold).
I set the population to 100 and all other settings are default.
The fitness with both the CMA and VdK sample seems to plateau.

You have any idea what is going on? Is there a way to control the max std so it stays on sane levels?

Best regards

learning rates are computed in several places

In particular c1 and cmu in StatisticalModelSamplerWithZeroMeanBaseClass.parameters and CMAEvolutionStrategy.set.

adding a simple_cmaes submodule

Currently a barecmaes2.py stand alone code exists, which is comparatively short and easy to read and does not depend on numpy. I see no reason though why this should not become a submodule of pycma.

Calls to builtin eval should be replaced

by ast.literal_eval or by another "less evil" way to go about evaluating a string, possibly using purecma.safe_str.

Pypi's README indicates `cma.fcts.rosen` instead of `cma.ff.rosen`

Everything is in the title.

condition alleviation bug

Shows up in particular when the condition is high and hence the smallest eigenvalue is increasing by orders of magnitude and eigenvectors are not aligned in the coordinate system. The bug shows up for both, alleviation in coordinates and resetting C=I. self.alleviate_conditioning_in_coordinates(1e2) was used in this example to see both in on run:

import functools
import cma
f = functools.partial(cma.ff.elli, cond=1e28)
frot = cma.fitness_transformations.Rotated(f)
es = cma.fmin2(frot, [1] + 3 * [0], 0.1, {'tolconditioncov':False, 
                                  'seed': 4,
                                 })[1]

No linear transformation interval in box constraint handling?

pycma/cma/transformations.py

Line 267 in 881ad33

self._al = array([min([(ub[i] - lb[i]) / 2, (1 + np.abs(lb[i])) / 20])

In the docstring, it is said that the linear part takes about 90 % of the feasible interval in typical cases, however, au = al = (ub - lb) / 2 result in transforming whole region quadratically. I guess they should be au = al = (ub - lb) / 20.

Interface to CMA_sampler, stds vs variances

There seems to be a slight inconsistency, because the initial values passed to the StatisticalModelSamplerWithZeroMeanBaseClass class is a std vector

    def __init__(self, std_vec, **kwargs):
        """pass the vector of initial standard deviation or dimension of
        the underlying sample space.

        Ideally catch the case when `std_vec` is a scalar and then
        interpreted as dimension.
        """
        try:
            dimension = len(std_vec)
        except TypeError:  # std_vec has no len
            dimension = std_vec
            std_vec = np.ones(dimension)
        raise NotImplementedError

while the interface then (only) supports the variances attribute/property.

numpy.linalg.linalg.LinAlgError on a bbob-biobj problem

I ran a relatively long experiment on the biobjective BBOB test suite of COCO in which CMA-ES optimizes a quite simple weighted sum of the two objective functions (optimizing 50 weighted sums per problem in a total budget of 1e5 times dimension including restarts and using the default options of CMA-ES). One of the experiment's 10 batches crashed in 40-D with the following error message:

bbob-biobj_f47_i02_d40 2-objective problem (bbob-biobj(bbob_f015_i03_d40__bbob_f017_i05_d40)(7591))
  --> new weight: 7.346939e-01
bbob-biobj_f47_i02_d40 2-objective problem (bbob-biobj(bbob_f015_i03_d40__bbob_f017_i05_d40)(7591))
  --> new weight: 7.142857e-01
bbob-biobj_f47_i02_d40 2-objective problem (bbob-biobj(bbob_f015_i03_d40__bbob_f017_i05_d40)(7591))
  --> new weight: 6.938776e-01
Traceback (most recent call last):
  File "example_experiment-WS-normalized.py", line 488, in <module>
    main(budget, max_runs, current_batch, number_of_batches)
  File "example_experiment-WS-normalized.py", line 465, in main
    current_batch, number_of_batches)
  File "example_experiment-WS-normalized.py", line 334, in batch_loop
    max_runs)
  File "example_experiment-WS-normalized.py", line 396, in coco_optimize
    solver(fun, budget)
  File "example_experiment-WS-normalized.py", line 263, in weighted_sum_search
    ideal=ideal, nadir=nadir)
  File "example_experiment-WS-normalized.py", line 292, in runCMAESWithWeightedSum
    solutions = es.ask()
  File "/home2/users/dimo.brockhoff/WS-77/cma/evolution_strategy.py", line 1674, in ask
    pop_geno = self.ask_geno(number, xmean, sigma_fac)
  File "/home2/users/dimo.brockhoff/WS-77/cma/evolution_strategy.py", line 1918, in ask_geno
    ary = self.sigma_vec * self.sm.sample(Niid)
  File "/home2/users/dimo.brockhoff/WS-77/cma/sampler.py", line 222, in sample
    self.update_now(lazy_update_gap)
  File "/home2/users/dimo.brockhoff/WS-77/cma/sampler.py", line 293, in update_now
    self._decompose_C()
  File "/home2/users/dimo.brockhoff/WS-77/cma/sampler.py", line 339, in _decompose_C
    self.D, self.B = self.eigenmethod(self.C)
  File "/usr/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 1187, in eigh
    raise LinAlgError, 'Eigenvalues did not converge'
numpy.linalg.linalg.LinAlgError: Eigenvalues did not converge

Unfortunately, I turned off the output of CMA-ES to not produce too large output files. I could try to find a random seed that will reproduce the error by running the experiment again just on this function.

Noisy BBOB functions don't work anymore

Due to a change in the error type that is raised with the newer numpy versions, the noisy BBOB functions in bbobbenchmarks.py do not work and give the following error:

In [1]: import cma
In [2]: myf = cma.bbobbenchmarks.F102(3*[0])
In [3]: myf([1,1,1])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-48305f886e49> in <module>()
----> 1 myf([1,1,1])

C:\Users\dimo\Desktop\pycma\cma\bbobbenchmarks.py in __call__(self, x)
    396
    397         """
--> 398         return self.evaluate(x)
    399
    400     def evaluate(self, x):

C:\Users\dimo\Desktop\pycma\cma\bbobbenchmarks.py in evaluate(self, x)
    402
    403         """
--> 404         return self._evalfull(x)[0]
    405     # TODO: is it better to leave evaluate out and check for hasattr('evaluate') in ExpLogge
r?
    406

C:\Users\dimo\Desktop\pycma\cma\bbobbenchmarks.py in _evalfull(self, x)
    663         # COMPUTATION core
    664         ftrue = np.sum(x**2, -1)
--> 665         fval = self.noise(ftrue)
    666
    667         # FINALIZE

C:\Users\dimo\Desktop\pycma\cma\bbobbenchmarks.py in noise(self, ftrue)
    602         """Returns the noisy function values."""
    603
--> 604         return fUniform(ftrue, self.unifalphafac * (0.49 + 1. / self.dim), self.unifbeta)
    605
    606     def boundaryhandling(self, x):

C:\Users\dimo\Desktop\pycma\cma\bbobbenchmarks.py in fUniform(ftrue, alpha, beta)
    344     idx = ftrue < tol
    345     try:
--> 346         fval[idx] = ftrue[idx]
    347     except IndexError: # fval is a scalar
    348         if idx:

TypeError: 'numpy.float64' object does not support item assignment

CMA_diagonal /VkD only works with adapt_sigma = FALSE?

Hey,

I was experimenting with CMA-ES with high dimensional parameter spaces.
For speed, I tried to use the CMA_diagonal = TRUE option.
I noticed it only works when I also set adapt_sigma is to FALSE.
Is this normal behaviour?
If so, is there a reason when do not allow sigma to change with a diagonal covariance matrix?
I also noticed that the default setting for adapt_sigma in the VdK sampler is also zero?
What is the advised procedure when using this sampler?
Setting sigma fixed for the entire run, manually restarting the run with a stepwise lower sigma or something else?

Thanks for the nice piece of software and algorithm!
Best Regards

FutureWarnings in some comparisons

Hello! Trying out CMAES for some biological optimization problems. It works well, but I do get a bunch of warnings:

/usr/lib/python2.7/site-packages/cma.py:956: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if bounds in (None, [None, None]):

/usr/lib/python2.7/site-packages/cma.py:2937: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  if x0 == str(x0):

Using Python 2.7.13 and CMAES 1.1.07 (from pip)

Vd sampler behaves strangely

In maybe 20% of the runs, we see plots like this:

@youheiakimoto

Interface to CMA_sampler statistical model

Currently, the code reads

                try:
                    self.sm = self.opts['CMA_sampler'](
                                stds * np.ones(N),
                                **self.opts['CMA_sampler_options'])
                except:
                    self.sm = self.opts['CMA_sampler'](N,
                                **self.opts['CMA_sampler_options'])

It would be preferable if we had only one interface. This might not be practical, because not all samplers support initialisation with different standard deviations.

Inconsistent use of np.random.seed

Both of ConstRandnShift class and Rotation class call np.random.seed. If seed is not given in the constructors, the former doesn't call np.random.seed, but the latter calls it without an argument, which sets the seed using /dev/urandom, therefore we loose the full control of the random numbers. For a bug fixing, I prefer to have full control of random numbers. It is easy to correct, but I am not sure if it is a feature or a bug.

Clarification requested re hsig adjustment in purecma.py

pycma/cma/purecma.py

Line 405 in 0e4a1ba

< 2 + 4./(N+1))

Part of the purpose of purecma.py is to help understand the algorithm. What is the purpose of the "<" in this line? Thanks!

maxstd and minstd are in conflict

In particular if these values are set close to each other, they cannot necessarily be satisfied at the same time, because the global sigma is used for both corrections. This could be easily fixed in the CMA_diagonal setting, where diagonal corrections are unproblematic. With full covariance matrix this is rather a feature than a bug, because the change size of a diagonal change is non-trivial to compute. Alternatively to adjust maxstd, the axis/axes with the largest projection could be shortened.

As a side note, decreasing sigma to set maxstd seems only a good idea if it corrects for a previous increase of sigma.

Test fail on windows

I use the Anaconda python distribution and installed cma through pip. When I run the test.py file I get the following error messages:

C:\Users\thorsten.rissom\cma-test>python "c:/Program Files/Anaconda3/Lib/site-packages/cma/test.py"
doctesting cma package (v2.3.1) by calling doctest_files:
doctesting constraints_handler.py ... TestResults(failed=0, attempted=19)
doctesting evolution_strategy.py ... TestResults(failed=0, attempted=103)
doctesting fitness_functions.py ... TestResults(failed=0, attempted=0)
doctesting fitness_transformations.py ... TestResults(failed=0, attempted=80)
doctesting interfaces.py ... TestResults(failed=0, attempted=3)
doctesting optimization_tools.py ... TestResults(failed=0, attempted=14)
doctesting purecma.py ... **********************************************************************
File "c:/Program Files/Anaconda3/Lib/site-packages/cma\purecma.py", line 491, in purecma.py
Failed example:
logger = pcma.CMAESDataLogger().load()
Exception raised:
Traceback (most recent call last):
File "C:\Program Files\Anaconda3\lib\doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest purecma.py[1]>", line 1, in
logger = pcma.CMAESDataLogger().load()
File "C:\Program Files\Anaconda3\lib\site-packages\cma\purecma.py", line 677, in load
with open(name or self.filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '_CMAESDataLogger_datadict.py'

File "c:/Program Files/Anaconda3/Lib/site-packages/cma\purecma.py", line 492, in purecma.py
Failed example:
logger.filename == "_CMAESDataLogger_datadict.py"
Exception raised:
Traceback (most recent call last):
File "C:\Program Files\Anaconda3\lib\doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest purecma.py[2]>", line 1, in
logger.filename == "_CMAESDataLogger_datadict.py"
NameError: name 'logger' is not defined

1 items had failures:
2 of 31 in purecma.py
Test Failed 2 failures.
TestResults(failed=2, attempted=31)
doctesting recombination_weights.py ... TestResults(failed=0, attempted=25)
doctesting sampler.py ... TestResults(failed=0, attempted=28)
doctesting sigma_adaptation.py ... TestResults(failed=0, attempted=11)
doctesting test.py ... TestResults(failed=0, attempted=72)
doctesting transformations.py ... **********************************************************************
File "c:/Program Files/Anaconda3/Lib/site-packages/cma\transformations.py", line 66, in transformations.py
Failed example:
print(R(R(x), inverse=1))
Expected:
[ 1. 2. 3.]
Got:
[1. 2. 3.]

File "c:/Program Files/Anaconda3/Lib/site-packages/cma\transformations.py", line 223, in transformations.py
Failed example:
tf([1.52, -2.2, -0.2, 2, 4, 10.4])
Expected:
array([ 1.52, 4. , 2. , 2. , 4. , 10.4 ])
Got:
array([ 1.52, 4. , 2. , 2. , 4. , 10.4 ])

1 items had failures:
2 of 28 in transformations.py
Test Failed 2 failures.
TestResults(failed=2, attempted=28)
doctesting utilities\math.py ... TestResults(failed=0, attempted=0)
doctesting utilities\utils.py ... TestResults(failed=0, attempted=57)
`

I checked if the file "_CMAESDataLogger_datadict.py" is there, and it is in the cma directory. I suspect that there is a problem with the path notation of windows, but I could be wrong.

negative impact of injection on performance

A single injected solution should not have any relevant negative impact on performance, however we can see this:

roughly doubling the runtime by introducing a false short axis into C.

This suggests that the normalization of the injected point is not correct.

Weird unreachable code

https://github.com/CMA-ES/pycma/blob/master/cma/evolution_strategy.py#L1883

VkD clash due to too high condition number?

pycma/cma/restricted_gaussian_sampler.py

Line 282 in a793498

class GaussVkDSampler(StatisticalModelSamplerWithZeroMeanBaseClass):

Non scalar values for sigma0?

I am curious as to why only scalar values for sigma0 are allowed. It seems that there shouldnt be any reason why we cannot not have a vector of sigma0 values for each dimension of the optimization problem.

In fact this is possible in the java implementation of CMA-ES:

https://commons.apache.org/proper/commons-math/javadocs/api-3.1/org/apache/commons/math3/optim/nonlinear/scalar/noderiv/CMAESOptimizer.Sigma.html

Set `tolfun` to `1e-6`, but get result with `f=7.3`

I'd like to specify a minimum value of the score function of 1e-6. To this end I've added an options object:

options = CMAOptions()
options.set('tolfun', 1e-6)

When I run the optimiser on a toy problem, it returns almost immediatly with

termination on tolfun=1e-06
final/bestever f-value = 7.289226e+00 7.289226e+00
incumbent solution: [310.65090759643533, 313.64377551885428]
std deviation: [232.16677605865451, 235.91773080927337]

Am I doing something wrong? If I run without setting tolfun it returns an f=1.5e-11 within seconds!

non-diagonal covariance

even using inopts['CMA_diagonal'] = False i still get diagonal sigma, and i doing anything wrong?
this line looks suspicious:

pycma/cma/evolution_strategy.py

Line 1495 in 1e4afed

if 1 < 3: # new version with class

speeding up covariance matrix update

Thanks for making your library available. I noticed that it has been updated to v2 very recently.

When I'm using the cma library as is, I notice that the covariance update is rather slow when the number of parameters exceeds a few thousand, and rather unusable when the number of parameters is > 10000.

After doing some literature search, it seems there are some previous works on speeding this up to reduce the compute complexity from O(N^3) to O(N^2).

http://dl.acm.org/citation.cfm?id=1144082

I'm wondering if there are any options in the existing library for using such method, or any low-rank approximation options available to speed up the covariance matrix update process so that it can be usable for large number of param size?

I have been trying to use CMA-ES along with other algorithms that are simpler, but scale quickly for a large number of params (for example, simple GA type algorithms, and also one of OpenAI's recent paper on ES).

Best Regards.

pycma always seeds np.random

The current code either seeds np.random with a user specified seed, or a new random seed:

https://github.com/CMA-ES/pycma/blob/master/cma/evolution_strategy.py#L1359-L1364

This means that if the user seeds np.random, then generates random samples, then calls cma, and then generates more samples, the code still produces arbitrary output.

A workaround is to always pass a seed to cma, but it seems to me that changing the state of numpy.random (unless asked specifically) is a side-effect that should be eliminated

Why need to import and del then

I am feeling confused about the code.
In line 33 of test.py imported something but deleted later in line 37, why doing this? Thanks.

parallel function evaluation

For some real world applications, each function evaluation is expensive to run. So it's necessary to parallelize the fitness evaluation part. I can easily do that in Julia, but my version of CMA-ES is not as feature-full as this package. Do you have any interest in implementing that in Python.

doctest of purecma

When given as a stand alone file,

purecma.test()

does not execute any tests. Furthermore,

doctest.testmod(purecma)

fails, because the tests contain import cma.purecma.

computation of learning rates for sampler

The learning rate cmu, for example, is currently first computed in the parameters method of the sampler

self.sm.parameters(sp.weights).get('cmu', sp.cmu)

and then passed in the update method

                self.sm.update([(c1 / (c1a + 1e-23))**0.5 * self.pc] +  # c1a * pc**2 gets c1 * pc**2
                              list(pop_zero / (self.sigma * self.sigma_vec.scaling)),
                              sampler_weights)

in the sampler_weights argument.

This interface doesn't look to be particularly well thought out and a revision should be considered. By default, at least the learning rate management should probably remain entirely with the sampler.

We may consider to implement weight management independently, because for an additive update the negative weights should/must be adjusted in each iteration based on the vector lengths.

Weird variance change with mirrors in high dimension

es = cma.fmin2(cma.ff.elli, 360 * [1], 1,
    {'CMA_diagonal':True,
    # 'CMA_mirrormethod': 0,
    'CMA_mirrors':True,
    # 'AdaptSigma': cma.sigma_adaptation.CMAAdaptSigmaCSA,
    'popsize': 160,
    'maxiter': 1200,
    'ftarget': 1e-9,
})[1]
cma.plot()

We see a change of a coordinate-wise standard deviation by a factor of up to almost 3000 in a single iteration. This happens only with selective delayed mirrors and diagonal and active update and large population size and large dimension but irrespectively of the step-size adaptation method.

FutureWarning results in errors when the last version of numpy is used.

There are these two warning that appear when you call fmin function:

...python2.7/site-packages/cma.py:2938: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
if x0 == str(x0):
...python2.7/site-packages/cma.py:957: FutureWarning: comparison to None will result in an elementwise object comparison in the future.
if bounds in (None, [None, None]):

I just updated numpy to the newest version and now cma breaks at those points. I had to uninstall it and install an older version of numpy to be able to run the code that I have. Anyone could fix the issue?

Many MatplotlibDeprecationWarnings with matplotlib version >2.0

When running CMA-ES in my jupyter notebook with python 2.7 and matplotlib version 2.0.2, I get quite some MatplotlibDeprecationWarnings and UserWarnings (that appear only with %matplotlib notebook but not with %matplotlib inline:

C:\Users\dimo\Anaconda2\lib\site-packages\cma.py:6530: MatplotlibDeprecationWarning: pyplot.hold is deprecated.
    Future behavior will be consistent with the long-time default:
    plot commands add elements without first clearing the
    Axes and/or Figure.
  pyplot.hold(False)  # opens a figure window, if non exists
C:\Users\dimo\Anaconda2\lib\site-packages\matplotlib\__init__.py:917: UserWarning: axes.hold is deprecated. Please remove it from your matplotlibrc and/or style files.
  warnings.warn(self.msg_depr_set % key)
C:\Users\dimo\Anaconda2\lib\site-packages\matplotlib\rcsetup.py:152: UserWarning: axes.hold is deprecated, will be removed in 3.0
  warnings.warn("axes.hold is deprecated, will be removed in 3.0")
C:\Users\dimo\Anaconda2\lib\site-packages\cma.py:6409: MatplotlibDeprecationWarning: pyplot.hold is deprecated.
    Future behavior will be consistent with the long-time default:
    plot commands add elements without first clearing the
    Axes and/or Figure.
  hold(True)
[...]

The warnings themselves are not too problematic for me personally except for the additional scrolling up and down :-)

CMAEvolutionStrategy.mean has invalid shape when some variables are fixed

CMAEvolutionStrategy.mean returns only free (e.g., the optimized) variables, when some variables are fixed using the fixed_variables options. The set of candidates returned by .ask() is shaped correctly, though.

Setup.py does not specify utilities subpackage

With my python configuration (python 3.5 on Ubuntu 16.04), the cma.utilities subpackage does not get installed when I install the cma package from source. My guess is that the packages argument of setup should include cma.utilities.

EDIT: I can confirm that adding cma.utilities to the packages argument resolves this problem.

Use `logging` module instead of outputting to `stdout`

This is a suggestion and not an actual bug.

It would make debugging way easier, especially when running scripts on a cluster. IMHO, printing to stdout makes sense in a command-line interface context, but I'm not expecting any modules I import to do that.

I might code that for myself, any chance a PR might be accepted or is there a rationale for using print?

Noise handling without using `fmin`

Provide a code example with noise handling using the method CMAEvolutionStrategy.inject.

We could even consider to (always) re-inject the best solution of each iteration, thereby possibly killing one and a half birds with one stone. Remark though that the best solution is quite different from the best direction. Reinjecting the best solution (or any previously sampled solution) is likely to put a bias on the step-size update at least for some adaptation methods.

positive definiteness

The dynamical part of the weights computation to guaranty positive definiteness seems to be missing (around line 2819).

cma-es / pycma Goto Github PK

pycma's People

Contributors

Stargazers

Watchers

Forkers

pycma's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs