felipecode / coiltraine Goto Github PK

in function get_causes_of_end

    f = open(summary_file, "rU")
    header_summary = f.readline()

    header_summary = header_summary.split(',')
    header_summary[-1] = header_summary[-1][:-2]
    f.close()

-2 seems to be -1

• before index
  my head_summary content is
  end_pedestrian_collision,end_vehicle_collision,end_other_collision,weather,rep,start_point,initial_distance,final_distance,number_red_lights,exp_id,number_green_lights,final_time,result,end_point,time_out
• after index
  ['end_pedestrian_collision', 'end_vehicle_collision', 'end_other_collision', 'weather', 'rep', 'start_point', 'initial_distance', 'final_distance', 'number_red_lights', 'exp_id', 'number_green_lights', 'final_time', 'result', 'end_point', 'time_ou']

Hello,

the data-collector linked by this repository only works for Carla 0.8.x.

However, I have created scenarios using scenario-runner 0.9.9, how can those be used in the COiLTRAiNE framework to train a model?

Thanks for your help!

The webpage of the URL is broken.
When opening the webpage of the URL, it prompts "bad request error 400".

How do we obtain the camera angle anyway, in order to use it in the lateral_augmentation function provided by @felipecode for data balancing?

It seems the link of carla 100 dataset is expired. Do anyone know how to download the dataset?
Thank you very much

Hi. As this frameworks requires Docker, is it compatible with Windows?

Hello!

If I try to run the coiltraine.py file, I got this message. I have Ubuntu 16.04, and followed everything like in the readme.md. Any suggestions, where to start finding the error?
Thanks!

Hi,

It would be great to know how do we exactly train the CILRS model from scratch using the dataset provided by you. I see there's a need for docker to run coltraine.py. However, if we just want to train the model on the given dataset like any other driving model, what shall we the way to go forward?

Looking forward to your help!

Thanks,
Soumi

Hi,
I cannot download the sample_data and baseline_data, seems like the url is broken.

Thanks

I am using the nocrash resnetimnetS1 model as a baseline and have run tests using checkpoint 660000. I am finding that my results for the New Town experiments are much lower than as recorded in the paper Exploring the Limitations of Autonomous Driving. Am I using the same checkpoint as was used in the paper? Are there any reasons that my results on the same experiment would be lower?

Some one please help with this
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
File "view_model.py", line 91, in
carla09interface.game_loop(args, agent)
File "/home/vaishu/opt/Self-Dri-Car-imi-2/coiltraine/model_view/carla09interface.py", line 818, in game_loop
world = World(client.get_world(), hud)
File "/home/vaishu/opt/Self-Dri-Car-imi-2/coiltraine/model_view/carla09interface.py", line 201, in init
self.restart()
File "/home/vaishu/opt/Self-Dri-Car-imi-2/coiltraine/model_view/carla09interface.py", line 256, in restart
self.lane_invasion_sensor = LaneInvasionSensor(self.vehicle, self.hud)
File "/home/vaishu/opt/Self-Dri-Car-imi-2/coiltraine/model_view/carla09interface.py", line 639, in init
bp = world.get_blueprint_library().find('sensor.other.lane_invasion')
IndexError: blueprint 'sensor.other.lane_invasion' not found
WARNING: sensor object went out of the scope but the sensor is still alive in the simulation: Actor 50 (sensor.camera.rgb)
WARNING: sensor object went out of the scope but the sensor is still alive in the simulation: Actor 51 (sensor.other.collision)
terminate called without an active exception
Aborted (core dumped)

I want to update this repository to 0.9.x however I have no time now:
Basically it should use something similar to what is provided by this pr

carla-simulator/scenario_runner#220

I read the paper Exploring the Limitations of Behavior Cloning for Autonomous Driving, and I found that the metric Traffic Light Violations in Table 7 (Appendix) is pretty helpful. But I cannot find where this metric is implemented in this code. Can someone point it out or it has not been implemented in this version?

I noticed that the model requires images of size 88x200. I am wondering if I can input different size and modify following line accordingly or do I have to make any other modifications?

I was trying to run the COiLTRAiNE baseline agent on Carla 0.9.4 with the command:

python3  srunner/challenge/challenge_evaluator.py --file --scenario=group:ChallengeBasic --agent=../coiltraine/drive/CoILBaseline.py --config ../coiltraine/drive/sample_agent.json

And got the following error:

blueprint 'sensor.can_bus' not found

To replicate it, I tried the following code:

import carla
client = carla.Client("127.0.0.1", 2000)
client.set_timeout(2.0)
world = client.get_world()
blueprints = world.get_blueprint_library()
blueprints.find('sensor.can_bus')

Which gave the same error.
I can't seem to find the sensor can_bus anywhere in the Carla docs.
Any idea how to fix this?

In the file coiltraine/network/loss.py

we find the following lines from 56 onwards:

   loss_function = loss_branches_vec[0] + loss_branches_vec[1] + loss_branches_vec[2] + \
                      loss_branches_vec[3]
    
   speed_loss = loss_branches_vec[4]/(params['branches'][0].shape[0])

   return torch.sum(loss_function) / (params['branches'][0].shape[0])\
                + torch.sum(speed_loss) / (params['branches'][0].shape[0]),\

It seems the speed_loss is being divided by params['branches'][0].shape[0] (the batch_size?) twice instead of only once. While the rest of the loss ('loss_function') is not.

Is this indeed a bug that changes the scaling of the different losses or am I missing something?

I want to test my agent in NoCrash benchmark, but I couldn't find a guide file. It seemed that carla08 was for version_0.8 and carla09 for version_0.9.x. Can anyone provide a simple description for that?

Hi. Is the framework compatible with CARLA 0.9.x already?

Reading the readme section (https://github.com/felipecode/coiltraine/blob/master/docs/exploring_limitations.md), it is stated resnet34_imnet10S1 corresponds to seed 2 in the paper and is the best model. However, under one of the sample commands to test the best model under the hardest condition,

python3 coiltraine.py --gpus 0 --single-process drive -e resnet34imnet10S2 --folder nocrash \
-de NocrashNewWeatherTown_Town02 --docker carlagear

It seems to suggest that the best model is infact resnet34iment10S2. I believe that this is an error and it should instead be resnet34imnet10S1. Is this really the case? Please advice, Thanks!

Currently for new experiments to be performed it requires people to
copy the config file ! This creates confusion on the experiments.

One solution is to basically using a base config and then only specifying the
changes over base. This process is done on detectron2.

chandan) vaishu@vaishu-MacBookAir:~/coiltraine$ python3 view_model.py -f baselines -e resnet34imnet -cp 180000 -cv 0.9
pygame 1.9.4
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
File "view_model.py", line 9, in
from drive import CoILAgent
File "/home/vaishu/coiltraine/drive/init.py", line 1, in
from .coil_agent import CoILAgent
File "/home/vaishu/coiltraine/drive/coil_agent.py", line 22, in
from network import CoILModel
File "/home/vaishu/coiltraine/network/init.py", line 3, in
from .optimizer import adjust_learning_rate, adjust_learning_rate_auto
File "/home/vaishu/coiltraine/network/optimizer.py", line 1, in
import dlib
ImportError: /home/vaishu/anaconda3/envs/chandan/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /home/vaishu/anaconda3/envs/chandan/lib/python3.5/site-packages/dlib.cpython-35m-x86_64-linux-gnu.so)

Hi,
I have been looking around the code, however, I was not able to find where in the code we can get predictions (steer, throttle, brake) for a single sample (rgb+speed+control direction) input. Also, I cannot find where the speed branch is used to adjust the throttle for avoid stopping when the car shouldn't.

May I have some guidance regarding this?

Hello,
when i launched the training, and after the script loaded the dataset, i got the following error: (AttributeError: 'NoneType' object has no attribute 'swapaxes'), see the log below.
Could you give me some indications.

python coiltraine.py --single-process train -e resnet34imnet --folder baselines --gpus 0
pygame 1.9.4
Hello from the pygame community. https://www.pygame.org/contribute.html
self.root_dir /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0
preload Name 50hours_L0
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00000
Loaded 0.19058333333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00001
Loaded 0.34291666666666665 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00002
Loaded 0.55775 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00003
Loaded 0.7453333333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00004
Loaded 0.8699166666666667 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00005
Loaded 1.0498333333333334 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00006
Loaded 1.2786666666666666 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00007
Loaded 1.4104999999999999 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00008
Loaded 1.538333333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00010
Loaded 1.725833333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00011
Loaded 1.7706666666666664 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00012
Loaded 1.9157499999999996 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00014
Loaded 2.1914999999999996 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00015
Loaded 2.221083333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00016
Loaded 2.366833333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00017
Loaded 2.4814999999999996 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00018
Loaded 2.6587499999999995 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00020
Loaded 2.859083333333333 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00021
Loaded 3.11325 hours of data
Episode /media/ucef/90E6A5B7E6A59DCA/COiLTRAiNESampleDatasets/CoILTrain/L0/episode_00022
Loaded 3.2121666666666666 hours of data
preload Name 50hours_L0
Loaded dataset
Before the loss
Traceback (most recent call last):
File "/media/ucef/Nouveau nom/CARLA/coiltraine/coil_core/train.py", line 116, in execute
for data in data_loader:
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 336, in next
return self._process_next_batch(batch)
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
numpy.core._internal.AxisError: Traceback (most recent call last):
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/numpy/core/fromnumeric.py", line 51, in _wrapfunc
return getattr(obj, method)(*args, **kwds)
AttributeError: 'NoneType' object has no attribute 'swapaxes'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/media/ucef/Nouveau nom/CARLA/coiltraine/input/coil_dataset.py", line 112, in getitem
img = self.transform(self.batch_read_number * boost, img)
File "/media/ucef/Nouveau nom/CARLA/coiltraine/input/augmenter.py", line 31, in call
img = np.swapaxes(img, 0, 2)
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/numpy/core/fromnumeric.py", line 549, in swapaxes
return _wrapfunc(a, 'swapaxes', axis1, axis2)
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/numpy/core/fromnumeric.py", line 61, in _wrapfunc
return _wrapit(obj, method, *args, **kwds)
File "/home/ucef/anaconda3/envs/coiltraine/lib/python3.5/site-packages/numpy/core/fromnumeric.py", line 41, in _wrapit
result = getattr(asarray(obj), method)(*args, **kwds)
numpy.core._internal.AxisError: axis1: axis 0 is out of bounds for array of dimension 0

This command python3 tools/download_sample_models.py raises the following error. (Rest of the commands provided in documentation until this one are working fine)

Downloading the coil models checkpoints 224 MB
Traceback (most recent call last):
File "tools/download_sample_models.py", line 25, in
tf = tarfile.open("track2_baseline.tar.gz")
File "/home/xq/anaconda3/envs/coiltraine/lib/python3.5/tarfile.py", line 1564, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

Any idea how to resolve this?

Hello,
Great work!
have you also trained weights for the latest version of the simulator. The nocrash models are obviously trained on the 0.8(where the agent drives nicely) and when used in the 0.9. the performance is quite poor.
best

Hello, I run python3 coiltraine.py --folder sample --gpus 0 -de TestT1_Town01 -vd CoILVal1 --docker carlasim/carla:0.8.4 on master branch to see original performance. The training and validation works well. But when drive in the simulator Carla, the output log is:

Going to communicate
Checkpoint 10000
0 0 0
<carla08.driving_benchmark.experiment.Experiment object at 0x7fd461f3e518>
start index 36 end index 40
start index 39 end index 35

----- Printing results for training weathers (Seen in Training) -----

Percentage of Successful Episodes

VALUES
Weather: Clear Noon
Task: 0 -> 0.0
Average Between Weathers
Task 0 -> 0.0

Average Percentage of Distance to Goal Travelled

VALUES
Weather: Clear Noon
Task: 0 -> -6.953387923663017e-07
Average Between Weathers
Task 0 -> -6.953387923663017e-07

Average Percentage of Distance to Percentage OffRoad

VALUES
Weather: Clear Noon
Task: 0 -> 0.0
Average Between Weathers
Task 0 -> 0.0

All the data is zero. Do anyone meet similar situation? thank you

hello,I try to run get_ sample_ Datasets. py and get_ baseline_ None of the datasets. py files succeeded, which is probably the reason for the damage of Google's web page. I want to ask if there is any way to download to the relevant data set, thank you.

Any code related to downloading files from cloud is not working. Can you please provide links for downloads?

Since we are now using docker,and I'm not familiar with it.There is a -ns option in coiltraine.py,but I don't think it will help me.

After installing all the requirements and upon executing the command python3 coiltraine.py --folder sample --gpus 0 -de TestT1_Town01 -vd CoILVal1 --docker carlasim/carla:0.8.4 I am getting the following error

coil_icra : CoILTrain_coil-icra_8conv_lowdropout_single_control_balancesteer_L1_noise_all_6_None_50hours
     train
        STATUS:  Error  -  Something Happened
     validation_CoILVal1
       STATUS:  Not Started
     drive_TestT1_Town01
       STATUS:  Not Started

Please let me know how to resolve this.

Command:
python3 srunner/challenge/challenge_evaluator_routes.py --scenarios=srunner/challenge/all_towns_traffic_scenarios1_3_4.json --routes=srunner/challenge/routes_training.xml --debug=0 --agent=../coiltraine/drive/CoILBaseline.py --config=../coiltraine/drive/sample_agent.json

@felipecode Could you please let me know what might be the problem.

Dear Mr. Codevilla,

I am a student working on a project involving testing ADAS/AV and scenario synthesis using BeamNG.tech and would love to run (test) your driving agent in that simulator. I know CARLA is kind of a de facto standard, but IMHO BeamNG.tech is superior when it comes to physic simulation, content, and flexibility. Further, BeamNG.tech is free for research, offers a python API, just like CARLA, and implements a wide range of sensors.

So I wonder how technically difficult it would be to port COiLTRAiNE to BeamNG.tech and whether anyone of you could support me (and my colleagues) in doing so. Hope to hear from you soon,

Thanks!

-- Benedikt Steininger

Hello,

I have collect datasets using data-collector to train a model and I get a poor result .

                                                          Average Between Weathers
                                                            Task  0  ->  0.42
                                                            Task  1  ->  0.18
                                                            Task  2  ->  0.04

I also train models using the datasets (http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_14.zip) and the result is better.

                                                          Average Between Weathers
                                                            Task  0  ->  0.8600000000000001
                                                            Task  1  ->  0.48000000000000004
                                                            Task  2  ->  0.06

However, it is far away from the result in the paper.

So I think the datasets are also a key to reproduce models. Which datasets linked by this repository worked best ?

                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_01.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_02.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_03.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_04.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_05.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_06.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_07.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_08.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_09.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_10.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_11.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_12.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_13.zip
                                          http://datasets.cvc.uab.es/CVPR2019-CARLA100/CVPR2019-CARLA100_14.zip

Thanks for your help!

Hello,

I receive a runtime UnicodeDecodeError, during test in drive_TestT1_Town01.

I am using Ubuntu 18.04 with carla:0.8.2 docker, and I executed the following run command:

python3 coiltraine.py --folder sample --gpus 0 -de TestT1_Town01 -vd CoILVal1 --docker carlasim/carla:0.8.2

After finishing train and validation, the process displays the "Iterating" in test for quite some time:

coil_icra : CoILTrain_coil-icra_8conv_lowdropout_single_control_balancesteer_L1_noise_all_6_None_50hours           
     train                                                                                                         
        STATUS:  Finished                                                                                          
     validation_CoILVal1                                                                                           
        STATUS:  Finished                                                                                          
     drive_TestT1_Town01                                                                                           
        STATUS:  Iterating                                                                                         

After some time, the error is produced and the following traceback is displayed:

Traceback (most recent call last):                                                                                  
  File "coiltraine.py", line 187, in <module>                                                                       
    folder_execute(params)                                                                                          
  File "/home/user/coiltraine/coil_core/executer.py", line 173, in folder_execute                                   
    driving_environments)                                                                                           
  File "/home/user/coiltraine/logger/printer.py", line 126, in plot_folder_summaries                                
    names_list = get_names(exp_batch)                                                                               
  File "/home/user/coiltraine/configs/coil_global.py", line 122, in get_names                                       
    merge_with_yaml(os.path.join('configs', folder, experiment_alias))                                              
  File "/home/user/coiltraine/configs/coil_global.py", line 102, in merge_with_yaml                                 
    yaml_file = yaml.load(f)                                                                                        
  File "/home/user/anaconda3/envs/coiltraine/lib/python3.5/site-packages/yaml/__init__.py", line 70, in load        
    loader = Loader(stream)                                                                                         
  File "/home/user/anaconda3/envs/coiltraine/lib/python3.5/site-packages/yaml/loader.py", line 34, in __init__      
    Reader.__init__(self, stream)                                                                                   
  File "/home/user/anaconda3/envs/coiltraine/lib/python3.5/site-packages/yaml/reader.py", line 85, in __init__      
    self.determine_encoding()                                                                                       
  File "/home/user/anaconda3/envs/coiltraine/lib/python3.5/site-packages/yaml/reader.py", line 124, in determine_enc
oding                                                                                                               
    self.update_raw()                                                                                               
  File "/home/user/anaconda3/envs/coiltraine/lib/python3.5/site-packages/yaml/reader.py", line 178, in update_raw   
    data = self.stream.read(size)                                                                                   
  File "/home/user/anaconda3/envs/coiltraine/lib/python3.5/codecs.py", line 321, in decode                          
    (result, consumed) = self._buffer_decode(data, self.errors, final)                                              
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9a in position 16: invalid start byte

Please advise!

IsADirectoryError: [Errno 21] Is a directory: '/home/xyz/Desktop/git/coiltraine/configs/sample/coil_icra'

This happens when I try to run with the sample config.

Have you considered using additional "sensors" to lower entropy of RGB input like semantic segmentation, monocular depth and optical flow during training?

paper: https://arxiv.org/pdf/1912.00177.pdf

Thanks for your great job, I have installed carla 0.8.2 under docker. When i run python3 coiltraine.py --folder sample --gpus 0 -de TestT1_Town01 -vd CoILVal1 --docker , I meet the erro of ' STATUS: Error - Something Happened'. Can you give me some sugesstion. THANKES

Hi. The file docs/view_agents.md is missing. It is mentioned on the first page at: "you can train and evaluate agents in CARLA 0.9.X._"

Would love to see detailed instructions on how to test and train agents in CARLA 0.9.x.