errors in benchmark about mushroom-rl-benchmark

[boris-il-forte]

Dear @davidenitti ,
Thanks for letting us know about the issue.
Unfortunately, we have updated mushroom, and we moved some code of the benchmark into the original library (for example, get_initial_states).
However, we didn't complete the work on the benchmark, as we would like to make a significant refactoring, to get rid of the current Slurm support (which is outdated) and use the experiment_launcher pip package (also maintained by the IAS lab @ TU Darmstadt)

If you want to use the benchmark, you should use the tag 1.1.1 and mushroom 1.7.0 (which is a bit outdated, unfortunately)

We don't have time to fix the benchmark right now, but I may take a look at it during the Christmas holidays. Sorry for the problem, and thanks again for reporting!

[boris-il-forte]

@davidenitti Good news from the benchmark side! A small Christmas present for all mushroom-rl user base 🎁

I had time to work on this issue and restructure the benchmark. The code now is much simpler, the logging is fully NumPy, and we should be able to use mushroom-rl 1.9.0

I still need to debug, test some functionality, and clean up the code. Also, we need to release the newest version of mushroom and another software package (experiment launcher)
However, soon we will be able to release the newer benchmark, also easier to maintain (thus, it shouldn't break in the future anymore!)

Merry Christmas from the mushroom-rl team!

[davidenitti]

thanks!

[davidenitti]

If I import
from mushroom_rl_benchmark import BenchmarkExperiment
I have this error now:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'BenchmarkExperiment' from 'mushroom_rl_benchmark' (/home/davide/code/mushroom-rl-benchmark/mushroom_rl_benchmark/__init__.py)

maybe the examples need to be updated?

[boris-il-forte]

If you are looking at the master branch, is unmantained, currently.

We developed the new version of the benchmark (no example files there) this winter.

Unfortunately due to technical issues with our cluster and lack of time, we still haven't had time to deploy the final version.

We will try to see if we manage to release the new benchmark this summer.

If you find issues in the new version, please let us known. Unfortunately, it has not been tested in depth.