zhangfuyang / rl_carla Goto Github PK

您好，您在Readme中提到了模仿学习预训练，请问您进行模仿学习的代码可以公开一下吗

It takes a lot of time to learn, so I would like to shorten this.
Does this code support learning across multiple GPUs?

您好，您在代码中使用了模仿学习预训练的模型。请问我们应该怎样去训练模仿学习的模型

I see that you are not directly using RGB images as input, so what do you use as input to the network?

hello, I tried to run ./CarlaUE4.sh -carla-server -benchmark -fps=10 and python3 ddpg_main.py as your instruction, but I got the error, something like
"0%| | 0/1000000 [00:00<?, ?it/s]
start to reset env
0%| | 0/1000000 [00:00<?, ?it/s]
(localhost:2000) failed to read data: timed out
0%| | 0/1000000 [00:00<?, ?it/s]
start to reset env
0%| | 0/1000000 [00:00<?, ?it/s]
(localhost:2000) connection closed
"
I am using carla 0.9.10. Could you give me any tipps？ I appreciate.

Hi!

Could you tell me what your results were when you ran the ddpg on the rgb camera?
Isn't the input state too complex for training directly from pixles? Did you also apply segmentation?

Thanks!

Hello,thanks to provide this project.Now,I have some question to ask.First,when I want to change the tensorflow.1x to tensorflow.2x,I find that the work in Initation_learning_network/Network/bn the tf.contrib.layers.batch_norm can not be used in tf2,and I want to use tf.compat.v1.layers.batch_normalization to change,but when the code run into imitation_learning.py/load_model, the error said Not found: Key batch_normalization/beta not found in checkpoint .So I want to ask that the model of /imitation/model/checkpoint is or not be used in the ddpg algorithm? The param Inference in ddpg_agent is False,why need code to run self.env.Image_agent.load_mode(),this model used where ?

Could you please give me some advice on how to run it on the basic driving benchmark or CoRL 2017 provided by the official?
I have made some efforts since last week but finally failed. I have no idea how to get state and action in benchmark(not in the class Env designed by you).
If others have some idea,please help to reply this issue.Thank you.

Hi! Is this algorithm to use the imitation learning to train the parameters and then carry out reinforcement learning(DDPG)?

Hi, I'm trying to train PPO2 with Carla and I was wondering how long did it take to train your ddpg model to compare my work with yours

您好，您在ddpg_main.py中设置Max_pisode为200000，请问需要多长时间训练结束

Hi. I ran your training script for a few days and I got following results.

The agent seemed to learn something at about 1k step. Then the training curve degraded and converged at bad performance. I am wondering what may cause this to happen? Could you tell me how your pre-trained model was trained?

在carla_sever_pb2.py中我看到您对PlayMeasurement的设置,请问Nonplayer您是否也进行了设置

您好，您训练好的DDPG模型看起来效果很好，而且训练的步数远没有超参数中设置的那么多，请问您当初在训练时设置的超参数可以分享一下吗

Hello, I'm running DDPG main.py, but the car doesn't move
What should I do, please

您的Actor网络输入为512维度的特征+5个车辆自身信息，特征是对RGB图像进行卷积处理和全连接层后得到的，请问特征是分类信息吗

您好，我在您的代码中看到 当训练的时候，要读取模仿学习预训练的模型 ，但是在测试的时候却没有读取。 我查看了ddpg训练好的模型，里面没有有关卷积的key
请问在测试的时候是否读取了模仿学习的模型

Hi,
Awesome work in training DDPG from raw pixels! I am trying to run your code in version 0.9.6 of Carla. The main difference I believe between the 0.9.x versions and the stable version is the method of communication between the server and client. I made changes to your implementation and now can run your inference code (test_ddpg.py)error free. I am setting the lane intersection and off road intersection as 0. I am getting a weird driving behavior that you can see here.

1. Do you know what might be possibly going wrong here?
2. How are you accessing the values for the offlane intersection and offroad intersection?

Please let me know if you want to take a look at my code. I can share with it you if that helps.

Thanks!

你好，我跑了几天，发现效果还是很不太好，只用到了DDPG。你能否能提供训练好的参数呢？ 还有你提到了，使用Imitation ，效果会更好，那么怎样使用imitation呢？谢谢

Any help as to why this error occurs?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "ddpg_main.py", line 149, in
tf.app.run()
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\platform\app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\site-packages\absl\app.py", line 312, in run
_run_main(main, args)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\site-packages\absl\app.py", line 258, in _run_main
sys.exit(main(argv))
File "ddpg_main.py", line 140, in main
agent.train()
File "C:\Users\user\Desktop\rl_CARLA-master\src\agent\ddpg_agent.py", line 72, in train
state = self.env.reset()
File "C:\Users\user\Desktop\rl_CARLA-master\environment\env.py", line 125, in reset
observation, _, _, _ = self.step(action)
File "C:\Users\user\Desktop\rl_CARLA-master\environment\env.py", line 67, in step
measurements, sensor_data = self.client.read_data()
File "C:\Users\user\Desktop\rl_CARLA-master\environment\carla\client.py", line 127, in read_data
return pb_message, dict(x for x in self._read_sensor_data())
File "C:\Users\user\Desktop\rl_CARLA-master\environment\carla\client.py", line 127, in
return pb_message, dict(x for x in self._read_sensor_data())
RuntimeError: generator raised StopIteration

您在代码中使用了预训练网络，请问没有预训练的网络您可以公开吗

Hi, you did a great job with this repo. Would you mind explaining how to use the trained DDPG for the test case? Are the new weights saved in the models folder after the training is done, or do I have to save them manually using the info in the results folder?

After I started CARLA server and ran your python script, I got the following errors:

Traceback (most recent call last):
  File "ddpg_main.py", line 149, in <module>
    tf.app.run()
  File "/home/autolab/anaconda3/lib/python3.7/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "ddpg_main.py", line 140, in main
    agent.train()
  File "/media/autolab/disk_2T/jiancongpeng/repos/rl_CARLA/src/agent/ddpg_agent.py", line 72, in train
    state = self.env.reset()
  File "/media/autolab/disk_2T/jiancongpeng/repos/rl_CARLA/environment/env.py", line 125, in reset
    observation, _, _, _ = self.step(action)
  File "/media/autolab/disk_2T/jiancongpeng/repos/rl_CARLA/environment/env.py", line 67, in step
    measurements, sensor_data = self.client.read_data()
  File "/media/autolab/disk_2T/jiancongpeng/repos/rl_CARLA/environment/carla/client.py", line 127, in read_data
    return pb_message, dict(x for x in self._read_sensor_data())
  File "/media/autolab/disk_2T/jiancongpeng/repos/rl_CARLA/environment/carla/client.py", line 127, in <genexpr>
    return pb_message, dict(x for x in self._read_sensor_data())
RuntimeError: generator raised StopIteration

It seemed that the client failed to read sensor data from CARLA server. I am wondering if there is any solution to this problem. Any advice will be helpful to me.

您好，代码在训练过程中的初始场景是随机的，请问在哪里设置

In my opinion, μ is Actor's policy
self.action_gradients are the grad_Q of action -- ▽Q_a
self.net_params are the parameters ot ActorNet - -Θ
we want to max the performance objective J
▽J is a function of ▽Q_a and ▽μ_Θ

however, I can't understand
self.check_gradient = tf.gradients(self.outputs, self.net_params[-2])
Why use self.net_params [-2] to compute gradient?
What's the function of check_gradient in ActorNetwork?

请问模仿学习预训练的数据集您可以提供一下吗

抱歉刚接触github问的可能比较傻瓜，请问这个项目是比较基础的DDPG-CARLA代码吗，刚接触DDPG想从基础的程序学起，之前看到一些都比较复杂

Could anyone send me the DDPG model after they have trained it? Thanks

Hello, after running DDPG_main for some time
LogSpawn: Warning: SpawnActor failed because of collision at the spawn location [X=24410.000 Y=-195.998 Z=3943.062] for [DodgeChargePolice_C]"
LogCarla: Error: Requested 20 vehicles, but we mere only able to spawn 20
What's going on here，please

hi ,i use your python code with python 3.7,tensorflow 1.5.4,carla 0.8.2,ubuntu 18.04
but ,i can't run this simulation,
it work error,
INFO:tensorflow:Restoring parameters from models/saveNet_1907.ckpt-158700
I1106 09:56:56.428069 140117888739136 saver.py:1284] Restoring parameters from models/saveNet_1907.ckpt-158700
Restoring from models/
start to reset env
Traceback (most recent call last):
File "/home/robot/work/code/rl/rl_CARLA/environment/carla/client.py", line 174, in _read_sensor_data
raise StopIteration
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "test_ddpg.py", line 62, in
tf.app.run()
File "/home/robot/.local/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/robot/.local/lib/python3.7/site-packages/absl/app.py", line 303, in run
_run_main(main, args)
File "/home/robot/.local/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "test_ddpg.py", line 47, in main
state = env.reset()
File "/home/robot/work/code/rl/rl_CARLA/environment/env.py", line 125, in reset
observation, _, _, _ = self.step(action)
File "/home/robot/work/code/rl/rl_CARLA/environment/env.py", line 67, in step
measurements, sensor_data = self.client.read_data()
File "/home/robot/work/code/rl/rl_CARLA/environment/carla/client.py", line 127, in read_data
return pb_message, dict(x for x in self._read_sensor_data())
File "/home/robot/work/code/rl/rl_CARLA/environment/carla/client.py", line 127, in
return pb_message, dict(x for x in self._read_sensor_data())
RuntimeError: generator raised StopIteration

can you tell me how to deal with this problem？thanks！
or can you tell me your enviroment? thansk!