opendilab / gobigger-explore

Still struggling with the high threshold or looking for the appropriate baseline? Come here and new starters can also play with your own multi-agents!

GoBigger-Explore

EN

GoBigger-Explore is an open source code collection in the article "GoBigger: A Scalable Platform for Cooperative-Competitive Multi-Agent Interactive Simulation" of ICLR 2023.

Outline

• Release Version
• Getting Start
• Result
• Resources
• Join and Contribute
• License

🚀 Release Version

The current version is the latest version 0.4.0.

👇 Getting Start

System environment

• Core 32
• GPU (>=16G)
• Memory 40G

Install the necessary package

git clone git/repo/path
cd bigger_rl
pip install -e .

Start training

1. Serial Training(DQN、PPO、MAPPO、QMix、VMix、COMA)

• Train

  python -u -m bigrl.bin.serial_train  --config abs/config/path
  

• Evaluation

  python -u -m bigrl.bin.serial_eval  --config abs/config/path
  

2. Parallel Training(IMPALA + Alphastar_Loss + League Training)

• Train && Evaluation

  python -u -m bigrl.single.worker.trainer.trainer  --config abs/config/path
  

Results

• RL Agent vs Bot (Without cooperation)
• Bot vs Bot
• RL Agent vs Bot (With cooperation)

😍 Resources

• GoBigger Repo Github Link

⭐ Join and Contribute

Welcome to OpenDI Lab GoBigger community! Scan the QR code and add us on Wechat:

Or you can contact us with slack or email ([email protected]).

🍸 License

GoBigger-Explore released under the Apache 2.0 license.

gobigger-explore's Issues

How to speed up?

Hi,
nice baseline repo!
Your log of v0.3 shows that it takes more than 3 days to train this model.
I have tried to use ddp to speed up the training process but still not satisfied.
Do you have any suggestion for speeding the training process up in a multi-gpu and high-memory machine? Any useful params to share?

能否提供一个官方的打包程序，就像1.0的test.py一样

typeerros when i run 'python gobigger_vsbot_baseline_main.py'

pygame 2.1.0 (SDL 2.0.16, Python 3.7.9)
Hello from the pygame community. https://www.pygame.org/contribute.html
INFO:learner_logger:[RANK0]: DI-engine DRL Policy
GoBiggerHybridActionSimple(
  (activation): ReLU()
  (encoder): StructedGcnConvSimple(
    (scalar_encoder): Sequential(
      (0): Linear(in_features=5, out_features=32, bias=True)
      (1): ReLU()
      (2): Linear(in_features=32, out_features=128, bias=True)
      (3): ReLU()
    )
    (food_encoder): Sequential(
      (0): Conv2d(2, 32, kernel_size=(5, 5), stride=(4, 4), padding=(2, 2))
      (1): ReLU()
      (2): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (3): ReLU()
      (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (5): ReLU()
      (6): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (7): ReLU()
      (8): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1))
      (9): ReLU()
    )
    (food_relation_encoder): Sequential(
      (0): Linear(in_features=150, out_features=64, bias=True)
      (1): ReLU()
      (2): Linear(in_features=64, out_features=128, bias=True)
      (3): ReLU()
    )
    (thorn_relation_encoder): Sequential(
      (0): Linear(in_features=12, out_features=32, bias=True)
      (1): ReLU()
      (2): Linear(in_features=32, out_features=128, bias=True)
      (3): ReLU()
    )
    (clone_encoder): Sequential(
      (0): Linear(in_features=17, out_features=32, bias=True)
      (1): ReLU()
      (2): Linear(in_features=32, out_features=128, bias=True)
      (3): ReLU()
    )
    (clone_relation_encoder): Sequential(
      (0): Linear(in_features=12, out_features=32, bias=True)
      (1): ReLU()
      (2): Linear(in_features=32, out_features=128, bias=True)
      (3): ReLU()
    )
    (gcn_1): RelationGCN(
      (act): ReLU()
      (thorn_relation_layers): Sequential(
        (0): Linear(in_features=256, out_features=128, bias=True)
        (1): ReLU()
      )
      (clone_relation_layers): Sequential(
        (0): Linear(in_features=384, out_features=128, bias=True)
        (1): ReLU()
      )
      (agg_relation_layers): Sequential(
        (0): Linear(in_features=512, out_features=128, bias=True)
        (1): ReLU()
      )
    )
    (gcn_2): RelationGCN(
      (act): ReLU()
      (thorn_relation_layers): Sequential(
        (0): Linear(in_features=256, out_features=128, bias=True)
        (1): ReLU()
      )
      (clone_relation_layers): Sequential(
        (0): Linear(in_features=384, out_features=128, bias=True)
        (1): ReLU()
      )
      (agg_relation_layers): Sequential(
        (0): Linear(in_features=512, out_features=128, bias=True)
        (1): ReLU()
      )
    )
    (agg_encoder): Sequential(
      (0): Linear(in_features=384, out_features=128, bias=True)
      (1): ReLU()
      (2): Linear(in_features=128, out_features=64, bias=True)
      (3): ReLU()
    )
  )
  (action_type_head): DiscreteHead(
    (Q): Sequential(
      (0): Sequential(
        (0): Linear(in_features=32, out_features=32, bias=True)
        (1): ReLU()
        (2): Linear(in_features=32, out_features=32, bias=True)
        (3): ReLU()
      )
      (1): Sequential(
        (0): Linear(in_features=32, out_features=16, bias=True)
      )
    )
  )
)
Traceback (most recent call last):
  File "/home/keliangjun666/hjb/Gobigger-Explore/my_submission/entry/gobigger_vsbot_baseline_simple_main.py", line 112, in <module>
    main(main_config)
  File "/home/keliangjun666/hjb/Gobigger-Explore/my_submission/entry/gobigger_vsbot_baseline_simple_main.py", line 99, in main
    learner.save_checkpoint, learner.train_iter, collector.envstep
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/worker/collector/battle_interaction_serial_evaluator.py", line 206, in eval
    obs = to_tensor(obs, dtype=torch.float32)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 109, in to_tensor
    new_data[k] = to_tensor(v, dtype, ignore_keys, transform_scalar)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 121, in to_tensor
    new_data.append(to_tensor(t, dtype, ignore_keys, transform_scalar))
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 109, in to_tensor
    new_data[k] = to_tensor(v, dtype, ignore_keys, transform_scalar)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 121, in to_tensor
    new_data.append(to_tensor(t, dtype, ignore_keys, transform_scalar))
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 109, in to_tensor
    new_data[k] = to_tensor(v, dtype, ignore_keys, transform_scalar)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 121, in to_tensor
    new_data.append(to_tensor(t, dtype, ignore_keys, transform_scalar))
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 109, in to_tensor
    new_data[k] = to_tensor(v, dtype, ignore_keys, transform_scalar)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 109, in to_tensor
    new_data[k] = to_tensor(v, dtype, ignore_keys, transform_scalar)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 109, in to_tensor
    new_data[k] = to_tensor(v, dtype, ignore_keys, transform_scalar)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 121, in to_tensor
    new_data.append(to_tensor(t, dtype, ignore_keys, transform_scalar))
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 115, in to_tensor
    return transform(item)
  File "/home/keliangjun666/miniconda3/envs/big/lib/python3.7/site-packages/ding/torch_utils/data_helper.py", line 101, in transform
    return torch.tensor(d, dtype=dtype)
TypeError: must be real number, not str

how i can slove this error

when I use Multi-GPUs to train my model，RuntimeError: Caught RuntimeError in replica 0 on device 0

When I test the DP mode ,the error happened

Refer to the https://di-engine-docs.readthedocs.io/zh_CN/latest/best_practice/multi_gpu_example.html， I changed the file：
（1）gobigger_no_spatial_config.py

（2）gobigger_vsbot_baseline_simple_main.py

无法复现

按照代码中的config，无法复现结果。总是卡在第13000个iter之后就停止训练了，buffer不再变化。