ibaigorordo / ultrafast-lane-detection-inference-pytorch-

Example scripts for the detection of lanes using the ultra fast lane detection model in Pytorch.

Ultrafast Lane Detection Inference Pytorch

Example scripts for the detection of lanes using the ultra fast lane detection model in Pytorch.

Source: https://www.flickr.com/photos/32413914@N00/1475776461/

Requirements

• OpenCV, Scikit-learn and pytorch. pafy and youtube-dl are required for youtube video inference.

Installation

pip install -r requirements
pip install pafy youtube-dl

Pytorch: Check the Pytorch website to find the best method to install Pytorch in your computer.

Pretrained model

Download the pretrained model from the original repository and save it into the models folder.

Ultra fast lane detection - TuSimple(link)

• Input: RGB image of size 1280 x 720 pixels.
• Output: Keypoints for a maximum of 4 lanes (left-most lane, left lane, right lane, and right-most lane).

Examples

• Image inference:

python imageLaneDetection.py 

• Webcam inference:

python webcamLaneDetection.py

• Video inference:

python videoLaneDetection.py

Inference video Example

Original video: https://youtu.be/2CIxM7x-Clc (by Yunfei Guo)

ultrafast-lane-detection-inference-pytorch-'s Issues

How can we perform this task: "Download the pretrained model from the original repository". i am not able to find the model

Predicted lane pixels extraction

Hello @ibaiGorordo,

Thank you for your amazing inference implementation.

I want to know a few things about the code, as I am trying to extract the detected lane pixels from the output for further use such as finding the curvature using the detected lane lines from the model. I wanted to know if the below-mentioned lines are the correct line to look for the array of lane pixels from the model or if they are saved in some other variable. I would like to get the pixel coordinates or the array of only the center lanes pixels after detection in which the vehicle is currently driving (lanes 2 and 3).

Could you please help me out with the question?
Ultrafastlanedetector.py line 112

Process output data

self.lanes_points, self.lanes_detected = self.process_output(output, self.cfg)

Thank you.

Ask

Please ask, your algorithm, can achieve the speed of real-time monitoring

Unexpected error with default code

File ultrafastLaneDetector.py", line 179, in process_output
    return np.array(lanes_points), np.array(lanes_detected)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.

I am getting this error, and I haven't even modified the code. Why is this happening? I am unable to fix it.

How to implement speed into this?

I would like to add speed, so I the "webcam or video" is seen as the driving vehicle.

I know that speed is distance over time. So i get the frames per second keep record of the distance travelled per frame, i should be able to estimate speed.

def calculate_fps(cap):
count_frames = 0
start_time = time.time()

while True:
    # Read the next frame from the video file
    ret, frame = cap.read()
    count_frames += 1

    if ret:
        elapsed_time = time.time() - start_time
        current_fps = count_frames / elapsed_time
        break
    else:
        break

return current_fps

while cap.isOpened():
try:
ret, frame = cap.read()
except:
continue

if ret:	

	# Detect the lanes
	output_img = lane_detector.detect_lanes(frame)

	fps = calculate_fps(cap)

	# Calculate speed
	ym_per_pix = 0.2  # meters per pixel
	speed = np.sum(lane_detector.distances[-int(fps):], axis=0) * ym_per_pix * fps * 3.6 # km/h

	average_speed = round(np.mean(speed))
	print('Average Speed:', average_speed, 'km/h')

I know i have to initial and keep record of distances variable inside UltrafastLaneDetector class. How can i go about doing this? is this the right approach?

Getting wrong detection

I have cloned the repository and installed all dependencies on my laptop (just an 8th gen i5 CPU). I then try to run

python imageLaneDetection.py

and I get the following output:

Any idea what I could be doing wrong?

Illegal Instruction occurs in Ubuntu 20.04 on Raspberry Pi 4

Hello, this program can run successfully on Windows and Ubuntu with AMD64 architecture, but I want to do lane detection on Raspberry Pi 4, as a lightweight and real-time lane detection system, easy to install on the car. On Raspberry Pi 4 with Ubuntu 20.04, I installed the following packages in the Python 3.7 virtual environment using pip3 from requirements.txt("pip3 list" output):

Joblib 1.0.1
Numpy 1.21.1
Opencv - python 4.5.3.56
Pafy 0.5.5
Pillow 8.3.1
PIP 21.2.4
Scikit - learn 0.24.2
Scipy 1.7.1
Setuptools 57.1.0
Sklearn 0.0
Threadpoolctl 2.2.0
The torch 1.9.0
Torchvision 0.10.0
Typing - extensions 3.10.0.0
Wheel 0.36.2
Youtube - dl 2021.6.6

When I run imageLaneDetection.py, videoLaneDetection.py and webcamLaneDetection.py to "output = self. model(input_tensor)" in the class function "inference" in the ultrafastLaneDetector.py caused an unexpected program interrupt and showed "Illegal Instruction" in the terminal window. Why this situation occurs and how to deal with it?

My terminal window output: