Comments (12)
It is probably OS related. Which OS do you use? If it is linux which gcc version is installed? Do you see errors when you execute pytest after cloning the github project:
git clone https://github.com/dietmarwo/fast-cma-es.git
cd fast-cma-es
pytest
Tested under Windows 10 and Ubuntu and cannot reproduce these errors on these OS.
Is your fitness function in Python?
I googled "SystemError: null argument to internal routine" and indeed it is related to some error at the C++ side. But this could be
related to some incompatible system libraries. On linux you could try to compile the C++ code yourself to make sure everything fits.
from fast-cma-es.
Got reports about dll recursive dependency issues on windows. The existing dlls are compiled using mingw64. All required dlls to load these are in the anaconda distribution for windows at C:\Users\uname\anaconda3\Library\mingw-w64\bin so I didn't notice the dependency issue. In version 1.2.21 these mingw64 dlls are included now. Try a 'pip install --upgrade' in case you observe a dll loading issue. Unfortunately the "SystemError: null argument to internal routine is still not reproducible.
from fast-cma-es.
Hi @dietmarwo sorry for being slow getting back to you - it's going to be pretty busy for 1.5 weeks, so I won't have a chance to debug properly until then, but in answer to your previous questions:
OS: CentOS 8
gcc: 8.3.1 20191121 (Red Hat 8.3.1-5)
N.B. glibc shipped with this system was too old, so I had to recompile the files in fast-cma-es/_fcmaescpp
. Incidentally, I couldn't figure out the libgtop
dependency, so I didn't compile that one (though it only looked like it was needed for astro.py
). I then copied _fcmaescpp/libacmalib.so
to fcmaes/lib
.
I'm not really sure where to start looking to debug the SystemError
- I tried placing print lines at various places in the code but I couldn't figure out what was throwing the error!
I tried running pytest
and it returned a lot of output plus this failure summary at the end:
============================================================================================== FAILURES ===============================================================================================
________________________________________________________________________________________ test_eggholder_retry _________________________________________________________________________________________
def test_eggholder_retry():
dim = 2
testfun = Eggholder()
limit = -956
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = retry.minimize(wrapper.eval, testfun.bounds,
num_retries=100)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
> assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
E assert 508093 == 609009
E + where 508093 = fun: -959.640662720851\n nfev: 508093\n success: True\n x: [512.0, 404.23180522114546].nfev
E + and 609009 = <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7fbf469c5790>>()
E + where <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7fbf469c5790>> = <fcmaes.testfun.Wrapper object at 0x7fbf469c5790>.get_count
fcmaes/test_cma.py:304: AssertionError
____________________________________________________________________________________ test_eggholder_advanced_retry ____________________________________________________________________________________
def test_eggholder_advanced_retry():
dim = 2
testfun = Eggholder()
limit = -956
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = advretry.minimize(wrapper.eval, testfun.bounds,
num_retries=300)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
> assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
E assert 819087 == 835557
E + where 819087 = fun: -959.640662720851\n nfev: 819087\n success: True\n x: [512.0, 404.2318051133138].nfev
E + and 835557 = <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7fbf46ab1b90>>()
E + where <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7fbf46ab1b90>> = <fcmaes.testfun.Wrapper object at 0x7fbf46ab1b90>.get_count
fcmaes/test_cma.py:321: AssertionError
======================================================================================= short test summary info =======================================================================================
FAILED fcmaes/test_cma.py::test_eggholder_retry - assert 508093 == 609009
FAILED fcmaes/test_cma.py::test_eggholder_advanced_retry - assert 819087 == 835557
============================================================================== 2 failed, 12 passed in 132.52s (0:02:12) ===============================================================================
from fast-cma-es.
Hi @timjim333 , sorry I didn't monitor this issue for some time, I am too busy with other stuff
( https://kelvins.esa.int/space-debris-the-origin/leaderboard/ ) and adding multi-objective / constraints to this library.
You guessed correctly, libgtop is only required for testing, I didn't include the source because of licensing issues.
Something like maven / pip would be nice for C++, so that we just have to define references to dependencies.
At least you have 12 tests passed. The "retry.minimize" is concerning since parallel retry is an essential feature.
I tested with gcc-7 and gcc-9 - had to switch because of compilation issues with biteopt on gcc-7. Tested on Ubuntu/Mint and Windows 10. I tried gcc-8 only on Windows, there it is the actual mingw version. Could you retry with the newest fcmaes version? Is it possible to update your glibc version so that you can check the original shared library without recompiling?
If not I probably have to install CentOS in VMware to reproduce your issue. Which Python version do you use? I test with Python 3.7 and 3.8. Could be a Python multiprocessing issue.
from fast-cma-es.
For the version 1.3.4 I added all dependent shared libraries both for windows and linux in the fcmaes/lib folder.
Should help avoiding issues with wrong versions of the dependent libraries.
from fast-cma-es.
Nice work on kelvins.esa.int!
I tried again pulling the latest version of fcmaes but on pytest, I still have the issue with glibc. Unfortunately, CentOS doesn't ship with a newer one:
/opt/fcmaes/fast-cma-es$ pytest
================================================= test session starts ==================================================
platform linux -- Python 3.7.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /opt/fcmaes/fast-cma-es
collected 0 items / 1 error
======================================================== ERRORS ========================================================
_________________________________________ ERROR collecting fcmaes/test_cma.py __________________________________________
fcmaes/test_cma.py:11: in <module>
from fcmaes import cmaes, de, decpp, cmaescpp, gcldecpp, retry, advretry
fcmaes/decpp.py:163: in <module>
libcmalib = ct.cdll.LoadLibrary(basepath + '/lib/libacmalib.so')
/home/tjim/anaconda3/envs/sound4/lib/python3.7/ctypes/__init__.py:442: in LoadLibrary
return self._dlltype(name)
/home/tjim/anaconda3/envs/sound4/lib/python3.7/ctypes/__init__.py:364: in __init__
self._handle = _dlopen(self._name, mode)
E OSError: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /opt/fcmaes/fast-cma-es/fcmaes/lib/libacmalib.so)
=============================================== short test summary info ================================================
ERROR fcmaes/test_cma.py - OSError: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /opt/fcmaes/fast-cm...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================================== 1 error in 0.38s ===================================================
So I tried recompiling again to my system glibc and I get:
/opt/fcmaes/fast-cma-es$ pytest
================================================= test session starts ==================================================
platform linux -- Python 3.7.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /opt/fcmaes/fast-cma-es
collected 15 items
fcmaes/test_cma.py ...........F.FFatal Python error: Segmentation fault
Current thread 0x00007f620ee5a740 (most recent call first):
File "/opt/fcmaes/fast-cma-es/fcmaes/cmaescpp.py", line 115 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/optimizer.py", line 275 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/optimizer.py", line 121 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 445 in _retry_loop
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/process.py", line 99 in run
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/process.py", line 297 in _bootstrap
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/popen_fork.py", line 74 in _launch
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/popen_fork.py", line 20 in __init__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/context.py", line 277 in _Popen
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/context.py", line 223 in _Popen
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/process.py", line 112 in start
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 124 in <listcomp>
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 124 in retry
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 116 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/test_cma.py", line 350 in test_eggholder_advanced_retry
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/python.py", line 183 in pytest_pyfunc_call
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/python.py", line 1641 in runtest
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 162 in pytest_runtest_call
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 255 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 311 in from_call
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 255 in call_runtest_hook
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 215 in call_and_report
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 126 in runtestprotocol
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 323 in _main
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 269 in wrap_session
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/config/__init__.py", line 163 in main
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/config/__init__.py", line 185 in console_main
File "/home/tjim/anaconda3/envs/sound4/bin/pytest", line 11 in <module>
Fatal Python error: Segmentation fault
Current thread 0x00007f620ee5a740 (most recent call first):
File "/opt/fcmaes/fast-cma-es/fcmaes/cmaescpp.py", line 115 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/optimizer.py", line 275 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/optimizer.py", line 121 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 445 in _retry_loop
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/process.py", line 99 in run
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/process.py", line 297 in _bootstrap
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/popen_fork.py", line 74 in _launch
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/popen_fork.py", line 20 in __init__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/context.py", line 277 in _Popen
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/context.py", line 223 in _Popen
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/multiprocessing/process.py", line 112 in start
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 124 in <listcomp>
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 124 in retry
File "/opt/fcmaes/fast-cma-es/fcmaes/advretry.py", line 116 in minimize
File "/opt/fcmaes/fast-cma-es/fcmaes/test_cma.py", line 350 in test_eggholder_advanced_retry
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/python.py", line 183 in pytest_pyfunc_call
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/python.py", line 1641 in runtest
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 162 in pytest_runtest_call
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 255 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 311 in from_call
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 255 in call_runtest_hook
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 215 in call_and_report
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 126 in runtestprotocol
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 323 in _main
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 269 in wrap_session
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/config/__init__.py", line 163 in main
File "/home/tjim/anaconda3/envs/sound4/lib/python3.7/site-packages/_pytest/config/__init__.py", line 185 in console_main
File "/home/tjim/anaconda3/envs/sound4/bin/pytest", line 11 in <module>
F [100%]
======================================================= FAILURES =======================================================
______________________________________________ test_rosen_decpp_parallel _______________________________________________
def test_rosen_decpp_parallel():
popsize = 8
dim = 2
testfun = Rosen(dim)
max_eval = 10000
limit = 0.01
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = decpp.minimize(wrapper.eval, dim, testfun.bounds,
max_evaluations = max_eval,
popsize=popsize, workers = popsize)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
assert(max_eval + popsize >= ret.nfev) # too much function calls
assert(max_eval // popsize + 2 > ret.nit) # too much iterations
assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
> assert(almost_equal(ret.x, wrapper.get_best_x(), eps = 1E-2)) # wrong best X returned
E assert False
E + where False = almost_equal(array([1.0065809 , 1.01367687]), [1.0017504095915344, 1.0035544851050473], eps=0.01)
E + where array([1.0065809 , 1.01367687]) = fun: 3.3199898595244694e-06\n nfev: 10007\n nit: 1008\n status: 0\n success: True\n x: array([1.0065809 , 1.01367687]).x
E + and [1.0017504095915344, 1.0035544851050473] = <bound method Wrapper.get_best_x of <fcmaes.testfun.Wrapper object at 0x7f608d84be50>>()
E + where <bound method Wrapper.get_best_x of <fcmaes.testfun.Wrapper object at 0x7f608d84be50>> = <fcmaes.testfun.Wrapper object at 0x7f608d84be50>.get_best_x
fcmaes/test_cma.py:300: AssertionError
_________________________________________________ test_eggholder_retry _________________________________________________
def test_eggholder_retry():
dim = 2
testfun = Eggholder()
limit = -956
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = retry.minimize(wrapper.eval, testfun.bounds,
num_retries=100)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
> assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
E assert 1036951 == 1621526
E + where 1036951 = fun: -959.640662720851\n nfev: 1036951\n success: True\n x: [512.0, 404.2318050386627].nfev
E + and 1621526 = <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7f608dde9690>>()
E + where <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7f608dde9690>> = <fcmaes.testfun.Wrapper object at 0x7f608dde9690>.get_count
fcmaes/test_cma.py:338: AssertionError
____________________________________________ test_eggholder_advanced_retry _____________________________________________
def test_eggholder_advanced_retry():
dim = 2
testfun = Eggholder()
limit = -956
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = advretry.minimize(wrapper.eval, testfun.bounds,
num_retries=300)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
> assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
E assert 663334 == 681059
E + where 663334 = fun: -959.640662720851\n nfev: 663334\n success: True\n x: [512.0, 404.23180528869597].nfev
E + and 681059 = <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7f608d85af90>>()
E + where <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7f608d85af90>> = <fcmaes.testfun.Wrapper object at 0x7f608d85af90>.get_count
fcmaes/test_cma.py:355: AssertionError
=============================================== short test summary info ================================================
FAILED fcmaes/test_cma.py::test_rosen_decpp_parallel - assert False
FAILED fcmaes/test_cma.py::test_eggholder_retry - assert 1036951 == 1621526
FAILED fcmaes/test_cma.py::test_eggholder_advanced_retry - assert 663334 == 681059
======================================= 3 failed, 12 passed in 163.85s (0:02:43) =======================================
Do you have any ideas?
How does bitopt behave? Have you had a chance to compare it to your fcmaes? Would it work as a drop-in replacement, in terms of performance? I might consider checking it out and try to compile it, if so! I'd like to get fcmaes working if possible though!
Thanks for the help.
from fast-cma-es.
I notice in the biteopt notes, it mentions:
Rogue optimums may not be found by this method. A rogue optimum is an optimum that has a very small, almost undetectable area of descent and is placed apart from other competing minima. The method favors minimum with a larger area of descent. The Damavandi test function is a perfect example of the limitation of this method (this test function is solved by this method, but requires a lot of iterations). In practice, however, rogue optimums can be considered as undesired outliers that rely on unstable parameter values (if such parameters are used in real-world system that has a certain parameter value precision, a system may leave the "rogue" optimal regime easily).
In the case of the EHVI method we are using, as we zone in on the optimal solutions, I guess it begins to match the description of 'rogue optimums' in that the area of descent becomes more and more needle-like. I guess it it worth a shot though, it it can outperform scipy's optimize routines.
from fast-cma-es.
Hi @timjim333 ,
scipy's optimize routines should be easy to outperform. And the "rogue"-like problem can be mitigated by parallel
retry using different random seeds. See https://github.com/dietmarwo/fast-cma-es/blob/master/examples/examples.py
how to apply BiteOpt (Bite_cpp(max_evals)) using parallel retry. Because of your compilation problems you could try to compile
BiteOpt from its source, but I had problems using gcc-7, switched to gcc-9 for this reason. Seems in an old system indirect library dependencies are still a problem. Next thing we could try is installing gcc-9 on your OS as additional compiler:
https://stackoverflow.com/questions/61590926/how-to-install-gcc-g-9-on-centos-8-docker-centoslatest
dnf -y install gcc-toolset-9-gcc gcc-toolset-9-gcc-c++
source /opt/rh/gcc-toolset-9/enable
Newer runtime libs should be installed this way. The best single objective algorithms from fcmaes are BiteOpt and DE, where for DE there is a native Python version. CMAES is mainly for control/space flight optimizations. For multi-objective/constraint problems now (version 1.3.8) there is MoDe both as native Python and as C++ implementation. Outperforms traditional algorithms like NSGA-II by some margin.
from fast-cma-es.
Hi @dietmarwo thanks for the insight! Ok, I will give installing the latest toolset a go and get back to you. It's fantastic that you have managed to incorporate the method as part of your parallel retry framework - I'm looking forward to getting it running, as I will be tackling a higher-dimensional problem soon.
Where you say 'CMAES is mainly for control/space flight optimizations' - do you mean you believe that it may perform poorly for a design task? Or is it just better now to chain BiteOpt and DE? Thanks for your thoughts!
from fast-cma-es.
from fast-cma-es.
Hi @dietmarwo I just tried recompiling with the gcc toolset 9 enabled, and on tunning pytest
, I now only get 2 errors instead of 3! Unfortunately, they are still related to the advanced rety:
======================================================= FAILURES =======================================================
_________________________________________________ test_eggholder_retry _________________________________________________
def test_eggholder_retry():
dim = 2
testfun = Eggholder()
limit = -956
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = retry.minimize(wrapper.eval, testfun.bounds,
num_retries=100)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
> assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
E assert 977940 == 1515650
E + where 977940 = fun: -959.640662720851\n nfev: 977940\n success: True\n x: [512.0, 404.2318052180589].nfev
E + and 1515650 = <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7fab49e78910>>()
E + where <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7fab49e78910>> = <fcmaes.testfun.Wrapper object at 0x7fab49e78910>.get_count
fcmaes/test_cma.py:338: AssertionError
____________________________________________ test_eggholder_advanced_retry _____________________________________________
def test_eggholder_advanced_retry():
dim = 2
testfun = Eggholder()
limit = -956
for _ in range(5):
wrapper = Wrapper(testfun.fun, dim)
ret = advretry.minimize(wrapper.eval, testfun.bounds,
num_retries=300)
if limit > ret.fun:
break
assert(limit > ret.fun) # optimization target not reached
> assert(ret.nfev == wrapper.get_count()) # wrong number of function calls returned
E assert 750878 == 774491
E + where 750878 = fun: -959.640662720851\n nfev: 750878\n success: True\n x: [512.0, 404.2318050919475].nfev
E + and 774491 = <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7facafcb49d0>>()
E + where <bound method Wrapper.get_count of <fcmaes.testfun.Wrapper object at 0x7facafcb49d0>> = <fcmaes.testfun.Wrapper object at 0x7facafcb49d0>.get_count
fcmaes/test_cma.py:355: AssertionError
=============================================== short test summary info ================================================
FAILED fcmaes/test_cma.py::test_eggholder_retry - assert 977940 == 1515650
FAILED fcmaes/test_cma.py::test_eggholder_advanced_retry - assert 750878 == 774491
======================================= 2 failed, 13 passed in 131.55s (0:02:11) =======================================
from fast-cma-es.
Could not reproduce but anyway tried to make the test more robust. Since the issue is only regarding the number of function calls it is nothing related to the functionality of the library. Will close the issue.
from fast-cma-es.
Related Issues (14)
- biteopt stopping criteria HOT 6
- Problem with running on Mac OS X HOT 2
- Stall iterations in BiteOpt HOT 5
- Usage for diploma thesis HOT 3
- M1 Support HOT 3
- failed to use fcmaes on Debian 10 HOT 4
- JSON of the employee scheduling example HOT 1
- illegal hardware instruction when importing mode on mac M1 HOT 2
- Missing library loguru when installing fcmaes 1.6.3 with pip HOT 1
- C++ algorithms return None on Windows
- cannot find dll file on windows HOT 1
- [Question] Difference between Python/C++ functions HOT 4
- CMA Cpp ignores x0 / guess HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fast-cma-es.