Awesome Radar Perception

A curated list of radar datasets, detection, tracking and fusion.
Keep updating.
Author: Yi Zhou
Contact: [email protected]
Update some public-available codes, see useful_codes.md

🚩 An interesting demo video of radar as a microphone Link

🚩 The View-of-Delft dataset is public available. Homepage

🚩 There are three papers about radar in the coming CVPR 2022.
Modality-Agnostic Learning for Radar-Lidar Fusion in Vehicle Detection
Raw High-Definition Radar for Multi-Task Learning
Exploiting Temporal Relations on Radar Perception for Autonomous Driving

🚩I have published a review paper on radar perception. Please see the link below. It is open access. If you find the contents are useful, please cite this paper in your work. I will keep updating this repository for the latest works in the radar perception field.

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

The 41-page slides associated with this paper: Link ; Link for China Mainland

Radar perception is getting popular. More and more researchers from computer vision, intelligent vehicle and robotics are starting to enter this field. I hope my review article and this repository can be a good starting point for these newcomers.

Overview

Radar Datasets

In my review paper, there is a table with more detials.

Conventional Radar Datasets for Autonomous Driving

Dataset	Radar Type	Data Type	Annotation	Link
nuScenes	Continental ARS408 x5	Sparse PC	3D bbox, TrackID	Website
DENSE	77Ghz Long-Range Radar	Sparse PC	3D bbox	Website
PixSet	TI AWR1843	Sparse PC	3D bbox, TrackID	Website
Radar Scenes	77GHz Middle-Range Radar x4	Dense PC	2D point-wise, TrackID	Website
Pointillism	2 TI AWR 1443	PC	3D bbox	Github
Zendar SAR	SAR	ADC, RD, PC	Pointwise Mask of Moving Vehicle	Github

Comments: nuScenes, DENSE and Pixset are for sensor fusion, but not particularly address the role of radar. Radar scenes provides point-wise annotations for radar point cloud, but has no other modalities. Pointillism uses 2 radars with overlapped view. Zendar seems no longer available for downloading.

Pre-CFAR Datasets for Detection

Dataset	Radar Type	Data Type	Annotation	Link
CRUW	TI AWR1843 Ultra Short Range	RA	Pointlevel Object	Website
CARRADA	TI AWR1843 Short Range	RA,RD,RAD	Pointwise, 2D bbox, Mask	Website
RADDet	TI AWR1843	RAD	3D bbox for RAD tensor	Github
RaDICaL	TI IWR1443	ADC	2D bbox	Website
GhentVRU	TI AWR1243 Short Range	RAD	Segmentation Mask for VRUs	Paper

Comments: CARRADA is captured in clean scenarios, CRUW uses RA maps, RADDet provides annotations for RAD tensor, RADICaL provides raw ADC data and signal processing toolboxes, GhentVRU can be accssed by contacting with authors.

4D Radar Datasets for Detection

Dataset	Radar Type	Data Type	Annotation	Link
Astyx Hires2019	Astyx 6455 HiRes Middel Range	PC	3D bbox	Dateset
View-of-Delft	ZF FRGen21 Short Range	PC	3D bbox	Website
RADIal	Valeo Middel Range DDM	ADC,RAD,PC	Point-level Vehicle; Open Space Mask	Github
TJ4DRadSet	Oculii Eagle Long Range	PC	3D bbox, TrackID	Github
K-Radar	Unknown	RAD	3D bbox, Track ID	Github

Comments: Astyx is small, VoD focuses on VRU classification, RADIal's annotation is coarse but provides raw data, TJ4D features for its long range detection, K-Radar provides RAD tensor and 3D annotations. TJ4D and K-radar are not yet public available.

Specific Tasks

Dataset	Radar Type	Task	Link
HawkEye	SAR	Static vehicle classification	Website
PREVENTION	Conti ARS308 + SRR208 x2	Trajectory Prediction	Website
SCORP	76GHz	Open space segmentation	Website
Ghost	77GHz long range *2	Ghost object detection	Github
Solinteraction Data	Soli	Tangible interactions	Github
GROUNDED	Ground Penetrating Radar	Localization	Website
FloW Dataset	TI AWR1843	Floating waste detection	Website
OLIMP	UWB + Continental ARS404	Multi-sensor fusion for detection	Website

Odometry and Localization

Dataset	Radar Type	Task	Link
Oxford Radar Robocar	Navtech Spinning Radar	Odometry, (Detection)	Website; Detection Annotation
RADIATE	Navtech Spinning Radar	Odometry, Detection, Tracking	Website
MulRan	Navtech Spinning Radar	Place Recognition	Website
Boreas	Navtech Spinning Radar	Long-term Odometry, Localization, Detection	Website
EU Long-term Dataset	Conti ARS 308	Long-term SLAM	Website
ColoRadar	TI AWR2243 Cascade + AWR1843	Odometry	Website
USVInland	TI AWR1843	SLAM in inland waterways, Water segmentation	Website
Endeavour Radar Dataset	Conti ARS 430 x5	Odometry	Website
OdomBeyondVision	TI AWR1843	Odometry	Website

Human Activity

Dataset	Radar Type	Data Type	Task	Link
DopNet	24GHz	Spectrogram	Gesture	Website
MCD-Gesture	77GHz	RAD tensor	Gesture	Website
DeepSoli	60GHz	RD map	Gesture	Website
Radar signatures of human activities	5.8 GHz	ADC	Human activities	Dataset
Ci4R human activity dataset	77GHz & 24GHz & 10GHz	Spectrogram	Human activities	Website
RadHAR	77GHz	Point Cloud	Human activities	Website
Vital Sign	60GHz	ADC, heart/breathing rate	Child vital sign	Dataset

Radar Toolboxes and ROS Nodes

TI Signal Processing SDK:

RaDICaL's Toolbox: SDK;
PyRapid: SDK;
OpenRadar : SDK;
Pymmw: SDK;
Open radar initiative: SDK;
RADIal's Emptyband-DDM Script: Code

Official SDK:

NXP Premium Radar SDK: Link;
TI mmWAVE Studio: Link
TI Toolbox: Link
Matlab Radar Toolbox: Link

ROS:

Ainstein Radar ROS Node: ROS Node;
Continental ARS 408 ROS Node: ROS Node;
TI mmWave ROS Driver: Guide;
RaDICaL's TI ROS Node: ROS Node;
UoA's TI ROS Package: ROS Node

Seminars and Workshops

Radar

2021 ICRA Radar Perception for All-Weather Autonomy [Website]
2021 ICASSP Recent Advances in mmWave Radar Sensing for Autonomous Vehicles [Website]
Radar in Action Series by Fraunhofer FHR [Video]
IEEE AESS Virtual Distinguished Lecturer Webinar Series [Website]
Journal of Radar Webinar Series (in Chinese) [Video]
Markus Gardill: Automotive Radar – An Overview on State-of-the-Art Technology [Video]
Markus Gardill: Automotive Radar – A Signal Processing Perspective on Current Technology and Future Systems [Video][Slides]
Francesco Fioranelli: Radar Old but Gold- current research challenges and activities in radar micro-Doppler signatures [Video]
Andrej Karpathy from Tesla: [Video]
Ole Schumann: Radar Perception for Automated Driving – Data and Methods [Video]
Sani Ronen from Arbe: Using AI layer to transform HR radar into insights for Autonomous Driving applications [Video]
Stefan Haag: Co-Development of Automatic Annotation for ML and Sensor Fusion Improvement System [Video]
Arthur Ouaknine: Deep Learning & Scene Understanding for autonomous vehicle [Video]
Paul Newman: The Road to Anywhere-Autonomy [Video]
Jaime Lien: Soli: Millimeter-wave radar for touchless interaction [Video]
Accelerating end-to-end Development of Software-Defined 4D Imaging Radar [Videp]
Radar-Imaging - An Introduction to the Theory Behind [Video]
NXP - Radar Experts Discuss the Evolution of Automotive Radar [Video]
Need to Successfully Design a Milimeter-Wave Automotive Radar Antenna? [Video]
Webinar SAR Imaging using Ancortek’s Software Defined Radars [Video]

Uncertainty Quantification

History of Bayesian Neural Networks [Video]
Uncertainty and Out-of-Distribution Robustness in Deep Learning [Video]

Review Papers

Radar vs Lidar

Antenna:

Radar Signal Processing:

General: The Rise of Radar for Autonomous Vehicles: Signal Processing Solutions and Future Research Directions
Signal Processing: Radar-on-Chip/in-Package in Autonomous Driving Vehicles and Intelligent Transport Systems
Signal Processing: [Automotive Radars A review of signal processing techniques](https://ieeexplore.ieee.org/abstract/document/7870764
PseudoNoise(PN): On the Safe Road Toward Autonomous Driving: Phase Noise Monitoring in Radar Sensors for Functional Safety Compliance
MIMO: MIMO Radar for Advanced Driver-Assistance Systems and Autonomous Driving: Advantages and Challenges
Digital Radar: High-Performance Automotive Radar: A Review of Signal Processing Algorithms and Modulation Schemes
Interference: Radar Interference Mitigation for Automated Driving: Exploring Proactive Strategies
Interference: Interference in Automotive Radar Systems: Characteristics, Mitigation Techniques, and Current and Future Research
Micro-Doppler: Micro-Doppler Effect in Radar: Phenomenon, Model, and Simulation Study

Deep Learning Applications for Radar:

General Object Detection:

Sensor Fusion:

Uncertainty Estimation:

Radar Characteristics

Weather Effects

RCS

Reflection Signature

Multi-Path

Micro-Doppler

Polarimetric

Simulation

Generative

Calibration

Radar & Multi Radars

Radar-Camera

Radar-Lidar

Radar-Lidar-Camera

Ego Motion

Static Objects and Surrounding Environments

Classification

Segmentation

Mapping and Localization

Detection of Dynamic Objects

Clustering

Classification of Clusters

Sparse 2D Point Cloud

4D Point Cloud

SAR & ISAR Imaging

Pre-CFAR Data

Range Azimuth Map

Range Doppler Map

ROD2021 Challenge Paper

Range Azimuth Doppler

Raw Signal

2018-Classification of Intra-Pulse Modulation of Radar Signals by Feature Fusion Based Convolutional Neural Networks

bennyustc / awesome-radar-perception Goto Github PK