How do I get information about the speed of the vehicles around me? about marl_cavs

[zcysun]

Hello, I have another question. The current average reward graph is drawing a point every 200eposide, how can I modify the frequency to require him to update it every specific eposides, so that the points sampled by the graph can be controlled?

[DongChen06]

Hi: As shown in the question, I want to get the observation information of the surrounding vehicles (or all the vehicles in the area). For example, I want to calculate the combined cost of all the vehicles in the area, view the speed information of all the vehicles (including self vehicles and HVs), etc., how do I get this information first?

Thank you very much for your answer.

Hi, if you know how to debug, then you may look at the merge_env_v1.py, there should be some variables called road.vehicles. It contains all the vehicle information

[DongChen06]

Hello, I have another question. The current average reward graph is drawing a point every 200eposide, how can I modify the frequency to require him to update it every specific eposides, so that the points sampled by the graph can be controlled?

For this question, you may look at the training codes and modify the log frequency. thanks

[zcysun]

Thank you very much for your answer!

Currently the code outputs an average reward graph not with one coordinate point per eposide, but with one reward value per "EVAL_INTERVAL", and it renders and outputs three training videos per EVAL_INTERVAL; this way, when I turn down EVAL_INTERVAL (for debugging purposes) I will be able to save the training videos frequently. I don't actually want the training videos right now, so I have the following questions:

1. Where can I change the code so that each eposide outputs a reward value so that I end up with a densely-populated average reward plot?
2. how do I turn off rendering and not have the training video appear and save, I only care about the average reward results.

Also, besides the average reward plot, what other meaningful result plots can I output for analysis?

Thank you very much for your answer!

[zcysun]

I'm really sorry for the interruption, but I'd like to ask you a few more questions!

Looking at the information in road.vehicles as you said, I printed them and the output is as follows:
! output
It seems that each vehicle will have three pieces of information, the first being the number (again, a question. Why is this number all three digits when the training video is something like #1#2?) The remaining two numbers I can't seem to find out exactly what they mean, can you tell me?

Thank you very much for your answer!

[zcysun]

I'm very sorry for the inconvenience.
! Screenshot 2023-11-29 09-43-10!
As you can see above, I noticed that the eval_logs and eval_videos folders are empty after each training session. Is this normal? Also, are the training logs saved and where should I look for them?

I would appreciate if you could answer!

[DongChen06]

I'm very sorry for the inconvenience. ! Screenshot 2023-11-29 09-43-10! As you can see above, I noticed that the eval_logs and eval_videos folders are empty after each training session. Is this normal? Also, are the training logs saved and where should I look for them?

I would appreciate if you could answer!

Yes, it is normal. When you implemented the evaluation, there will be videos in the folders

[zcysun]

Hello!

After debugging your publicly available code and running MAACKTR algorithm for approximately 1000 episodes, here is the result graph of average rewards. It seems there is no clear convergence trend（I ran it for several more times, but there still seems to be no convergence trend.）. I would like to ask how many episodes it typically takes to achieve convergence, and if it doesn't converge, what are the key parameters to adjust?

Thank you for your response!

[DongChen06]

Hello!

After debugging your publicly available code and running MAACKTR algorithm for approximately 1000 episodes, here is the result graph of average rewards. It seems there is no clear convergence trend（I ran it for several more times, but there still seems to be no convergence trend.）. I would like to ask how many episodes it typically takes to achieve convergence, and if it doesn't converge, what are the key parameters to adjust?

Thank you for your response!

Hi, which traffic modes you used? You may need 5000-10000 episodes to see an apparent increasing trend.

[zcysun]

Thanks for your reply!

The traffic modes is medium, when I increase the number of training rounds to around 10,000 the running time is around 12hours, is this within reason?

Also, I would like to output the same type of average speed graph as in your paper, where do I modify the code?

Thank you!

[DongChen06]

The plot is based on three random seeds. So you may need to run the codes for three times with different seeds.