dongchen06 / marl_cavs Goto Github PK

The environment seems that a problem version does not correspond and it cannot be reproduced

gym.error.UnregisteredEnv: No registered env with id: merge-multi-agent-v0

Traceback (most recent call last):
File "run_mappo.py", line 227, in
train(args)
File "run_mappo.py", line 81, in train
env = gym.make('merge-multi-agent-v0')
File "/home/huang/miniconda3/envs/marl_cav/lib/python3.8/site-packages/gym/envs/registration.py", line 569, in make
_check_version_exists(ns, name, version)
File "/home/huang/miniconda3/envs/marl_cav/lib/python3.8/site-packages/gym/envs/registration.py", line 219, in _check_version_exists
_check_name_exists(ns, name)
File "/home/huang/miniconda3/envs/marl_cav/lib/python3.8/site-packages/gym/envs/registration.py", line 197, in _check_name_exists
raise error.NameNotFound(
gym.error.NameNotFound: Environment merge-multi-agent doesn't exist.

Hi:
As shown in the question, I want to get the observation information of the surrounding vehicles (or all the vehicles in the area). For example, I want to calculate the combined cost of all the vehicles in the area, view the speed information of all the vehicles (including self vehicles and HVs), etc., how do I get this information first?

Thank you very much for your answer.

I got an error,
"(fuwen) wen@wen:~/Desktop/MARL_CAVs-main/MARL$ python run_mappo.py
Traceback (most recent call last):
File "run_mappo.py", line 8, in
import highway_env
File "/home/wen/anaconda3/envs/fuwen/lib/python3.6/site-packages/highway_env/init.py", line 5, in
import highway_env.envs
File "/home/wen/anaconda3/envs/fuwen/lib/python3.6/site-packages/highway_env/envs/init.py", line 1, in
from highway_env.envs.highway_env import *
File "/home/wen/anaconda3/envs/fuwen/lib/python3.6/site-packages/highway_env/envs/highway_env.py", line 5, in
from highway_env.envs.common.abstract import AbstractEnv
File "/home/wen/anaconda3/envs/fuwen/lib/python3.6/site-packages/highway_env/envs/common/abstract.py", line 21, in
class AbstractEnv(gym.Env):
File "/home/wen/anaconda3/envs/fuwen/lib/python3.6/site-packages/highway_env/envs/common/abstract.py", line 32, in AbstractEnv
_monitor: Optional[gym.wrappers.Monitor]
AttributeError: module 'gym' has no attribute 'wrappers'
"

Hello, may I ask when you can open the source of the new method you mentioned in the paper? Very interested in this. thank you very much

Hi! Your article is great of original and thanks for your code.Do you train on the GPU server? I train very slowly on the Windows computer. Is there any way to speed up the training? Thanks.

Excuse me, your code is so clear, so I want to run your code to learn MARL.
However, I ran into the trouble with the following description.

Traceback (most recent call last):
File "run_mappo.py", line 227, in <module>
train(args)
File "run_mappo.py", line 123, in train
episodes_before_train=EPISODES_BEFORE_TRAIN, traffic_density=traffic_density
File "/home/sunny_yeh/MARL_CAVs-main/MARL/MAPPO.py", line 46, in __init__
self.env_state, self.action_mask = self.env.reset()
File "/home/sunny_yeh/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
return self.env.reset(**kwargs)
File "/home/sunny_yeh/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/wrappers/env_checker.py", line 45, in reset
return env_reset_passive_checker(self.env, **kwargs)
File "/home/sunny_yeh/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/utils/passive_env_checker.py", line 207, in env_reset_passive_checker
),
f"The second element returned by env.reset() was not a dictionary, actual type: {type(info)}"
AssertionError: The second element returned by env.reset() was not a dictionary, actual type: <class 'numpy.ndarray'>

I think it's probably because "gym" version, so maybe you can share your version to me.
Thanks a lot

hello，can l ask you what should I do after my training is done？After my training was done, I got a line chart, a bunch of weights, and a number of training videos.

Hello, may I ask what are the reasons for the following errors and how to solve them？
Connected to pydev debugger (build 221.5787.24)
Backend QtAgg is interactive backend. Turning interactive mode on.
Traceback (most recent call last):
File "D:\SoftWare\Anaconda3\envs\pytorch\lib\site-packages\gym\envs\registration.py", line 138, in spec
return self.env_specs[id]
KeyError: 'merge-multi-agent-v0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\SoftWare\Pycharm\PyCharm Community Edition 2022.1.2\plugins\python-ce\helpers\pydev\pydevd.py", line 1491, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "D:\SoftWare\Pycharm\PyCharm Community Edition 2022.1.2\plugins\python-ce\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/H/Desktop/MARL_CAVs-main/MARL_CAVs-main/MARL/run_madqn.py", line 72, in
run()
File "C:/Users/H/Desktop/MARL_CAVs-main/MARL_CAVs-main/MARL/run_madqn.py", line 31, in run
env = gym.make('merge-multi-agent-v0')
File "D:\SoftWare\Anaconda3\envs\pytorch\lib\site-packages\gym\envs\registration.py", line 200, in make
return registry.make(id, **kwargs)
File "D:\SoftWare\Anaconda3\envs\pytorch\lib\site-packages\gym\envs\registration.py", line 104, in make
spec = self.spec(path)
File "D:\SoftWare\Anaconda3\envs\pytorch\lib\site-packages\gym\envs\registration.py", line 183, in spec
raise error.UnregisteredEnv("No registered env with id: {}".format(id))
gym.error.UnregisteredEnv: No registered env with id: merge-multi-agent-v0
python-BaseException

Process finished with exit code -1073741510 (0xC000013A: interrupted by Ctrl+C)

Hi!
Here I am again, wondering when the new method you proposed in your paper will be open source? Look forward to it.
Thanks!

Hello!

I noticed that the maximum eposides can be controlled by MAX_EPISODES during training, and EVAL_INTERVAL determines the evaluation intervals; however, the evaluation process seems to determine the number of evaluation eposides by test_seeds:

seeds = [int(s) for s in test_seeds.split(',')] rewards, (vehicle_speed, vehicle_position), steps, avg_speeds = ma2c.evaluation(env, video_dir, len(seeds), is_train=False)
(from run_ma2c.evaluate)

And in MAA2C, the evaluation eposides are fixed to 1:

def evaluation(self, env, output_dir, eval_episodes=1, is_train=True):
(from MAA2C.evaluation)

So my question is:

1. For the graphs generated in the thesis, is the data source the data generated from the evaluations at intervals during the training process, or is it the data generated from the evaluation process?
2. What is the use of the data generated from the evaluation process?
3. Is it necessary to set the number of evaluation eposides the same as the number of training eposides (e.g., 20,000 eposides) to observe the data generated by the evaluation process?
4. Is the evaluation process only for the model at the last checkpoint after training?

Thank you for your answer, and good luck with your research!

Hello,your work is great. I would like to ask if the MASAC algorithm will be open sourced in the future?

Hi, thanks for your original code and exciting work! When I run the code for the first time, I can acquire the video, but the second run is very slow even I can't acquire the result. I want to know what causes. Thanks again！

Thank you for your clear code! Could you help me figure out the error: When episode before train is set more than 1, the following line "states_var = to_tensor_var(batch.states, self.use_cuda).view(-1, self.n_agents, self.state_dim)" in MAPPO will raise ValueError: setting an array element with a sequence. Thank you in advance!

Hello, your code has been very helpful to me. It's very interesting. I have read your paper and may I ask if the multi lane scenario mentioned in your paper can be implemented by modified the merge_env_v1.py in the high_env？

Hi @Derekabc ,

I have followed the steps mentioned in git repo, also made the MARL and highway-env as directory root. But still have the below error while running the program in PyCharm 2018.3.5 (Community Edition)

Could you please help.

Thanks in advance

Hello @Derekabc

Thanks for creating this repository. I have read two of your papers: "Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic" [arxiv] and "Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic" [arxiv].

Honestly, both the papers are inspiring.

I am working on a project which uses highway-env environment's highway environment. I was going through your MAPPO.py code, and I am not able to understand why two different functions exploration_action(), and action() have same code.

As far I understood you are using one during the training and the other during evaluation. However, based on your comments above those two functions, exploration_action() is supposed to be used during exploration and action() is supposed to be used during exploitation phase of RL agent. But I am not sure why both the functions are exactly same.

Please help me understand your idea on this.

Hello! Thank you for your great job first!
I'm sorry to bother you, but I find it quite slow when training. So I want to use SubproVecEnv to build multiple envs and train at the same time. However, the SubprocVecEnv seem can't be used in mappo.
Is there any way to faster the algorithm?

Greetings!
First of all, congratulations on the inclusion of your article!
I read in #1 and #6 that you said you have plans to open source the algorithm after the article is included, and in #37 that you said the MASAC algorithm is not going to be open sourced for the time being, and I wanted to confirm with you that MASAC is the new approach you are proposing?
Thank you for your answer!

Thank you for your open-source materials. I have also tried to successfully run the run_mappo and run_maacktr models, but encountered an error while running the run_madqn model: self. memory. push (state [agend_id,:], action [agend_id], reward [agend_id], next_state [agend_id,:]), done) TypeError: tuple indices must be integers or slices, not tuple。 I tried debugging and it seems that there is an issue with the index of state [agend_id,:].
I am using Gym=0.19.0 (using other versions may result in other errors and no model can run), but the photo you provided shows that you are using Gym=0.21.0. Is this an error caused by the Gym version? Looking forward to your reply.

pandas
gym
matplotlib
opencv-python
pygame

I got this error when I tried to run either run_xxx.py or mpc_main.py. It looks like somewhere in the observation is a little bit glitchy. Would you mind helping me to look at it? It seems that there are some values which are out of the range [-1,1] or the dimension is not right. Thank you!

Hello, I found that the 'eval_rewards.npy' file was not generated in the 'results' folder when I drew the comparison curve using the 'plot_benchmark_saft. py' program. The error is as follows. How can I draw the comparison curve correctly? Look forward to your help.

FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
df = df.append(pd.DataFrame.from_records(

I got a future warning about the pandas, can I ask you how to solve this question?

UserWarning: volatile was removed (Variable.volatile is always False)
if input[0].volatile == False and self.steps % self.Ts == 0:

Also I got this userwarning, how to solve with that?

Hello!

Currently, the simulation and emulation process involves fixed durations and numbers of vehicles, with each episode restarting from zero after it ends. Has the author considered fixing only the traffic scenario (such as a ramp junction) and generating vehicles at regular intervals, not starting and ending with an episode, but using simulation time for long-term training?

This approach seems to allow for a more comprehensive evaluation of the simulation process from multiple aspects, such as throughput, collision rate, and so on.

Best wishes!

File "D:\毕设\code2\MARL_CAVs\MARL\run_mappo.py", line 254, in
train(args)
File "D:\毕设\code2\MARL_CAVs\MARL\run_mappo.py", line 136, in train
mappo.interact()
File "D:\毕设\code2\MARL_CAVs\MARL\MAPPO.py", line 111, in interact
action = self.exploration_action(self.env_state, self.n_agents)
File "D:\毕设\code2\MARL_CAVs\MARL\MAPPO.py", line 219, in exploration_action
softmax_actions = self._softmax_action(state, n_agents)
File "D:\毕设\code2\MARL_CAVs\MARL\MAPPO.py", line 209, in _softmax_action
softmax_action_var = th.exp(self.actor(state_var[:, agent_id, :]))
File "D:\毕设\code2\MARL_CAVs\MARL\single_agent\Model_common.py", line 19, in call
out = nn.functional.relu(self.fc1(state))
File "F:\python\anaconda\envs\marl_cav\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "F:\python\anaconda\envs\marl_cav\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "F:\python\anaconda\envs\marl_cav\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x5 and 25x128)

Dear writer:
I have got this problem, could you teach me how to solve it ? Maybe I can reshape the tensor, but I'm afraid it may cause another problems. Thank you !

ERROR:root:Can not find checkpoint for ./results/Apr_25_03_15_06/models/
C:\Users\60438\Desktop\MARL_CAVs-main\MARL\single_agent\Model_common.py:21: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
out = self.output_act(self.fc3(out))
Traceback (most recent call last):
File "C:\Users\60438\Desktop\MARL_CAVs-main\MARL\run_mappo.py", line 227, in
train(args)
File "C:\Users\60438\Desktop\MARL_CAVs-main\MARL\run_mappo.py", line 133, in train
mappo.interact()
File "C:\Users\60438\Desktop\MARL_CAVs-main\MARL\MAPPO.py", line 113, in interact
actions.append([index_to_one_hot(a, self.action_dim) for a in action])
File "C:\Users\60438\Desktop\MARL_CAVs-main\MARL\MAPPO.py", line 113, in
actions.append([index_to_one_hot(a, self.action_dim) for a in action])
File "C:\Users\60438\Desktop\MARL_CAVs-main\MARL\common\utils.py", line 37, in index_to_one_hot
one_hot = np.zeros((len(index), dim))
TypeError: object of type 'numpy.int32' has no len()

进程已结束,退出代码1

Hi! In the closed issue, I see that
"If you can see the output as follows, you are able to run the code. The error means there are no existing models."

Now I have the same outputs, but I want to know what I should do next to solve "no existing models", I am a rookie in MARL and i am confused. Thanks!!!

Hello! I have the following questions, and I would greatly appreciate your answers:

1. The program is running very slowly. My workstation is equipped with an RTX 4080 graphics card, and I have confirmed that CUDA and CUDNN are installed correctly (including version compatibility). During runtime, I've noticed that the GPU utilization is actually quite low. The main issue is that after the program finishes running (regardless of which run_xxx.py), the terminal doesn't display any errors and remains stuck in the running interface. It continues to show the same status indefinitely, and even after running it overnight, it remains in this state.
   

2. Under the conditions described in question 1, I can't determine whether the program has actually finished. However, in the "Results" folder, I find the models as shown in the attached image. Inside this folder, there is only the "models" directory, while the other folders are empty. I haven't obtained training curves, training videos, or similar outputs as others have. I'm wondering where the issue might be.
   

Hi! When I run the run_mappo.py，and something went wrong. But I have check the merge_env_v1, it's actually registered.

(marl_cav) h@qhan:~/rl/MARL_CAVs/MARL$ python run_mappo.py
Traceback (most recent call last):
File "/home/h/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/envs/registration.py", line 121, in spec
return self.env_specs[id]
KeyError: 'merge-multi-agent-v0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "run_mappo.py", line 225, in
train(args)
File "run_mappo.py", line 80, in train
env = gym.make('merge-multi-agent-v0')
File "/home/h/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/envs/registration.py", line 145, in make
return registry.make(id, **kwargs)
File "/home/h/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/envs/registration.py", line 89, in make
spec = self.spec(path)
File "/home/h/anaconda3/envs/marl_cav/lib/python3.6/site-packages/gym/envs/registration.py", line 131, in spec
raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: merge-multi-agent-v0

Could you please open source the “ours” algorithm mentioned in your paper?

Hi, thank you for your great job first! Could you please tell me how can I change the algorithm“ours” to the normal MAPPO?
I've read the issues posted before, but I can't find this config( namely safety guarantee ＆n_step) in ppo? hope your answer!

Hello! I need your help. When I run the run_mappo.py，and something went wrong.
Looking forward to your reply, thanks .

Hi! Your article is great of original! Thank you very much for opening up the other algorithms! Will your algorithm be open source?(Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic)

ERROR:root:Can not find checkpoint for ./results/Sep_21_22_41_51/models/
hi, i have a problem that run_xxx.py,this has some error .How can I get the checkpoint model?

Hello @Derekabc, if you have some time, I was hoping you could help me convert the MAPPO.py code to work for continuous action in action.py.

I tried updating the action code from this

def exploration_action(self, state, n_agents):
    softmax_actions = self._softmax_action(state, n_agents)
    actions = []
    for pi in softmax_actions:
        actions.append(np.random.choice(np.arange(len(pi)), p=pi))
    return actions

to this

def exploration_action(self, state, n_agents):
    softmax_actions = self._softmax_action(state, n_agents)
    return softmax_actions 

But the agent is not learning properly. Maybe I will have to also update the model code in Model_common.py. I have spent quite a lot of time in experimenting.

It would be great if you share some ideas on converting the [MAPPO.py] https://github.com/Derekabc/MARL_CAVs/blob/main/MARL/MAPPO.py) code for continuous action. I can share a pull request if I successfully complete it.

Looking forward to your help. Please let me know if you want to connect via email.

P.S. I am big time Naruto fan too 😆

Looking forward to your suggestions, sensei!

Hello. I run the "run_mappo.py", and it occurs this problem:

/home/liuqi/文档/MARL_CAVs-main/venv/bin/python /home/liuqi/文档/MARL_CAVs-main/MARL/run_mappo.py
Traceback (most recent call last):
File "/home/liuqi/文档/MARL_CAVs-main/MARL/run_mappo.py", line 228, in
train(args)
File "/home/liuqi/文档/MARL_CAVs-main/MARL/run_mappo.py", line 80, in train
env = gym.make('highway-fast-v0')
File "/home/liuqi/文档/MARL_CAVs-main/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 145, in make
return registry.make(id, **kwargs)
File "/home/liuqi/文档/MARL_CAVs-main/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 90, in make
env = spec.make(kwargs)
File "/home/liuqi/文档/MARL_CAVs-main/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 60, in make
env = cls(_kwargs)
File "/home/liuqi/文档/MARL_CAVs-main/highway-env/highway-env/envs/common/abstract.py", line 84, in init
self.reset()
File "/home/liuqi/文档/MARL_CAVs-main/highway-env/highway-env/envs/common/abstract.py", line 191, in reset
self._reset(num_CAV=num_CAV)
TypeError: _reset() got an unexpected keyword argument 'num_CAV'

Process finished with exit code 1

I changed the "run_mappo.py", "env = gym,make('highway-fast-v0')", and I registed the "highway-fast-v0" in the "MARL_CAVs-main" project. How to solve this problem?

PS C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL> python run_mappo.py
ERROR:root:Can not find checkpoint for ./results/Jan_27_12_42_16/models/
C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL\single_agent\Model_common.py:21: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
out = self.output_act(self.fc3(out))
Traceback (most recent call last):
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL\run_mappo.py", line 227, in
train(args)
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL\run_mappo.py", line 133, in train
mappo.interact()
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL\MAPPO.py", line 112, in interact
next_state, global_reward, done, info = self.env.step(tuple(action))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\gym\wrappers\order_enforcing.py", line 37, in step
return self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\gym\wrappers\step_api_compatibility.py", line 52, in step
step_returns = self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\gym\wrappers\env_checker.py", line 37, in step
return env_step_passive_checker(self.env, action)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\gym\utils\passive_env_checker.py", line 222, in env_step_passive_checker
result = env.step(action)
^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL../highway-env\highway_env\envs\merge_env_v1.py", line 127, in step
obs, reward, done, info = super().step(action)
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL../highway-env\highway_env\envs\common\abstract.py", line 456, in step
self.new_action = self.safety_supervisor(action)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL../highway-env\highway_env\envs\common\abstract.py", line 285, in safety_supervisor
env_copy = copy.deepcopy(self)
^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\copy.py", line 153, in deepcopy
y = copier(memo)
^^^^^^^^^^^^
File "C:\Users\CAIXUAN\Downloads\MARL_CAVs-main(1)\MARL_CAVs-main\MARL../highway-env\highway_env\envs\common\abstract.py", line 710, in deepcopy
setattr(result, k, copy.deepcopy(v, memo))
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\copy.py", line 265, in _reconstruct
y = func(*args)
^^^^^^^^^^^
TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments but 2 were given

Looking for help, 3q!

raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: merge-multi-agent-v0

你好，我对您的工作很感兴趣，我在将gym版本降低到0.21.0后，我发现我运行run_madqn.py 这个文件的时候没有反应，您好，请问这是为什么啊