Try to learn some Self-driving and deep learning based on Udacity and Na's work.
- Kaggle together with starter code and pre-trained models
- Floydhub
Left | Center | Right |
---|---|---|
![]() |
![]() |
![]() |
usage: main.py [-h] [--dataroot DATAROOT] [--ckptroot CKPTROOT] [--lr LR]
[--weight_decay WEIGHT_DECAY] [--batch_size BATCH_SIZE]
[--num_workers NUM_WORKERS] [--train_size TRAIN_SIZE]
[--shuffle SHUFFLE] [--epochs EPOCHS]
[--start_epoch START_EPOCH] [--resume RESUME]
[--model_name MODEL_NAME]
Main pipeline for self-driving vehicles simulation using machine learning.
optional arguments:
-h, --help show this help message and exit
--dataroot DATAROOT path to dataset
--ckptroot CKPTROOT path to checkpoint
--lr LR learning rate
--weight_decay WEIGHT_DECAY
weight decay (L2 penalty)
--batch_size BATCH_SIZE
training batch size
--num_workers NUM_WORKERS
# of workers used in dataloader
--train_size TRAIN_SIZE
train validation set split ratio
--shuffle SHUFFLE whether shuffle data during training
--epochs EPOCHS number of epochs to train
--start_epoch START_EPOCH
pre-trained epochs
--resume RESUME whether re-training from ckpt
--model_name MODEL_NAME
model architecture to use [nvidia, lenet]
An example of training usage is shown as follows:
python3 main.py --epochs=50 --resume=True
if you prefer to use Colab as training platform, feel free to use train.ipynb
script. Make sure you have already uploaded training data to Google Drive.
Kaggle also provides GPU support in its own kernel. Feel free to use this as the start code.
Training images are loaded in BGR colorspace using
cv2
whiledrive.py
load images in RGB to predict the steering angles.
After training process, use the saved model and drive.py
file to test your model performance in the simulator. Remeber to select AUTONOMOUS MODE
. Click Allow
to accept incoming network connections for python scripts.
usage: drive.py [-h] model [image_folder]
Remote Driving
positional arguments:
model Path to model h5 file. Model should be on the same path.
image_folder Path to image folder. This is where the images from the run
will be saved.
optional arguments:
-h, --help show this help message and exit
An example of test usage is shown as follows:
python3 drive.py model.h5 runs1/
Terrain 1 | Terrain 2 | Terrain 3 |
---|---|---|
![]() |
![]() |
![]() |
usage: video.py [-h] [--fps FPS] image_folder
Create driving video.
positional arguments:
image_folder Path to image folder. The video will be created from these
images.
optional arguments:
-h, --help show this help message and exit
--fps FPS FPS (Frames per second) setting for the video.
An example of creating video usage is shown as follows:
python3 video.py runs1/ --fps 48
Once you have installed the self-driving car simulator, you will find there are 2 different tracks in the simulator. The second one is much harder to deal with. Please choose the terrain you like and make sure that you select TRAINING MODE
.
Click RECORD
button on the right corner and select a directory as the folder to save your training image and driving log information.
Click RECORD
again and move your car smoothly and carefully. After you have completed recording your move, the training data will be stored in the folder you selected. Here I suggest you record at least 3 laps of the race. The first lap of race, please try best to stay at the center of the road, the rest could be either on the left hand side and right hand side of the road separately.
/IMG/
- recorded images from cneter, left and right cameras.driving_log.csv
- saved the image information and associated information like steer angle, current speed, throttle and brake.
.
├── RcCarDataset.py # Customed Dataset for Self-driving car simulator (Simple CNN as well as Nvidia paper's CNN model)
├── drive.py # Test script
├── main.py # Main pipeline
├── model.py # CNN model declaration
├── train_in_colab.ipynb # Colab training script
├── trainer.py # Trainer
├── utils.py # Helper functions
└── video.py # Create videos
NVIDIA model used
Image normalization to avoid saturation and make gradients work better.
Convolution: 5x5, filter: 24, strides: 2x2, activation: ELU
Convolution: 5x5, filter: 36, strides: 2x2, activation: ELU
Convolution: 5x5, filter: 48, strides: 2x2, activation: ELU
Convolution: 3x3, filter: 64, strides: 1x1, activation: ELU
Convolution: 3x3, filter: 64, strides: 1x1, activation: ELU
Drop out (0.5)
Fully connected: neurons: 100, activation: ELU
Fully connected: neurons: 50, activation: ELU
Fully connected: neurons: 10, activation: ELU
Fully connected: neurons: 1 (output)
Hyper-parameters | Description |
---|---|
lr=1e-4 | learning rate |
weight_decay=1e-5 | weight decay (L2 penalty) |
batch_size=32 | training batch size |
num_workers=8 | # of workers used in dataloader |
train_size=0.8 | train-validation set split ratio |
shuffle=True | whether shuffling data during training |
This error means that you are trying to load a newer version of model checkpoint in an older version of PyTorch.
Check PyTorch version in your local machine:
import pytorch
print(torch.__version__)
For Google Colab users, please try to first get your PyTorch version in your local machine and install the same version on Colab virtual machine using the following snippets, PyTorch 0.4.1
, for example.
# http://pytorch.org/
from os.path import exists
from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag
platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())
cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\.\([0-9]*\)\.\([0-9]*\)$/cu\1\2/'
accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'
!pip install -q http://download.pytorch.org/whl/{accelerator}/torch-0.4.1-{platform}-linux_x86_64.whl torchvision
Applied a Savitzky-Golay filter to the steering angle column. This helps reduce the training loss because original signal for steering angle input is via keyboard which is very dicrete.
Track 1 | Track 2 | Track 3 |
---|---|---|
[1] Nvidia research, End to End Learning for Self-Driving Cars
[2] Self-driving car simulator developed by Udacity with Unity