Custom Environment about marllib

[Morphlng]

I've run into the same problem and now I'm halfway through. It is possible to introduce a custom environment by doing the following steps:

1. You have to manually wrap your gym-like environment with MultiAgentEnv. You can refer to any other existing env under path envs/base_env/xxx.py. Aside from the standard reset, step, close method, you have to implement a method called get_env_info to return information like obs_space, action_space, etc. Also, the env class needs to include two member self.num_agents and self.agents. A global policy_mapping_dict should also be created and exposed, this will affect how Ray manage all those policies.

2. You have to define a default env config file under envs/base_env/config, it should be something like:

   env: your_env
   
   env_args:
     map_name: "default" #  other options
     any_other_config_needed_by_your_env: ...
   
   mask_flag: True
   global_state_flag: True
   opp_action_in_cc: False

3. Register your env in envs/base_env/__init__.py:

   try:
       from envs.base_env.your_env import CustomEnv
       ENV_REGISTRY["your_env"] = CustomEnv
   except Exception as e:
       ENV_REGISTRY["your_env"] = str(e)

4. [Optional] You can customize algorithm configuration that fit your environment by putting it under marl/algos/hyperparams/finetuned/your_env

5. Run your custom env by python marl/main.py --algo_config=mappo [--finetuned] --env_config=your_env [with env_args.map_name=custom]

By doing these steps above, I've successfully registered a Carla Environment to MARLlib. However, I said I was halfway through because the training is no good. I can't find a document explaining the exact meaning of "state", "obs" in observation space, and all the other configurations. It would be great to have complete documentation about customization.

[Theohhhu]

Guys, have a quick glance of our new api design and instance of customize new environments.

[luzgui]

Thank you very much!