Upgrade Ray and Gym about marllib

[Theohhhu]

Hi, thank you for your suggestion.

Currently, we do not have any plans to update Ray and Gym for two reasons.

First, Ray and RLlib are not backward compatible, which means updating them could potentially cause issues with existing code and workflows.

Second, the environment integrated into MARLlib is not compatible with the latest version of Gymnasium, which would require significant changes and updates to ensure compatibility.

Therefore, for now, we have decided to hold off on updating Ray and Gym until these compatibility issues can be addressed and resolved.

[elliottower]

Just for what it's worth, if you need help in updating to Gymnasium and current PettingZoo I can offer some help, the main difference is just changing from done to termination and truncation, as explained in the migration guide. I understand the idea of wanting things to be fully backwards compatible, but all major training libraries (RLlib, Stable-Baselines3, Tianshou, CleanRL) are working with current versions of PettingZoo and Gymnasium. Using unmaintained software is never desirable, and the easiest way to ensure everything is reproducible and consistent between libraries is to just use the current versions.

As you show in the README there are already 5+ different libraries trying to standardize MARL training and benchmarks, but most are not actively maintained, given that your project is maintained actively it seems like it would make a lot of sense to be using the most recent versions.

I just finished a tutorial for Stable-Baselines3 on PettingZoo and was thinking I would love to add some other MARL-specific libraries to our official tutorials, I think it would be a really useful asset to be able to point users to your library, if it weren't using 2+ year old versions of the PettingZoo/still using unmaintained Gym.

[zoetsekas]

I have been using Gymnasium as it has also been adopted in Ray 2.x.
I agree that change is simple.
I would like to upgrade the MARL policies to Ray 2.5, but I noticed Ray 3.x, which is still dev, has breaking changes, so I will wait till 3.0 release.