azureology / kitti-velo2cam

lidar to camera projection of KITTI

Lidar to camera projection of KITTI

Intro

中文博客

This is a Python implementation of how to project point cloud from Velodyne coordinates into the left color image with KITTI data set.

Dependices

matplotlib == 3.1.3
numpy == 1.18.1

Also tested with

matplotlib == 3.4.3
numpy == 1.23.5

Usage

Download KITTI dataset and place proj_velo2cam.py in the root path. CLI is also supported, e.g. run with frame 000999

python3 proj_velo2cam.py 999

Quick demo

Just clone the whole repo and run proj_velo2cam.py. By default, run with frame 000007 with path below:

.
├── data_object_image_2
│   └── testing
│       ├── image_2
│       │   └── 000007.png
│       └── projection
│           └── 000007.png
├── data_object_velodyne
│   └── testing
│       └── velodyne
│           └── 000007.bin
├── proj_velo2cam.py
├── readme.md
└── testing
    └── calib
        └── 000007.txt

Original image

Projection

kitti-velo2cam's Issues

为什么kitti雷达投影后的高度这么低

楼主你好，我自己也简单的实现了下点云投影到图像平面，发现投影后的点云高度很低，这合理吗

using ViViD dataset

I was trying to use your code using ViViD dataset (https://visibilitydataset.github.io/3_calibration.html).However i got wrong results.
i posted the whole details on stack overflow (https://stackoverflow.com/questions/71838759/lidar-to-rgb-projection-wrong-result-using-vivid-dataset). i really hope if you could help me.

图像背后的点云可以投影么

Is there projection solution for kitti odometry?

Hi,@azureology, thank you for sharing your code for lidar projection for KITTI benchmark.
I have trouble with KITTI Odomentry benchmark because there isn't any R0_rect parameter. Use the following operation seems to be wrong:

class KITTIDataset(torch.utils.data.Dataset):
    def __init__(self,basedir:str,seqs=['09','10'],pcd_scale:float=50.0,cam_id:int=2):
        self.kitti_datalist = [pykitti.odometry(basedir,seq) for seq in seqs]
        self.pcd_scale = pcd_scale
        self.cam_id = cam_id
        for seq,obj in zip(seqs,self.kitti_datalist):
            if not self.check(obj,[0,cam_id]):
                raise RuntimeError('calib file of KITTI seqence {:d} is not valid.'.format(seq))
        self.sep = [len(data) for data in self.kitti_datalist]
        self.sumsep = np.cumsum(self.sep)
        self.T_velo2cam0 = [data.calib.T_cam0_velo for data in self.kitti_datalist]  # transformation from cam0 to velo
    def __len__(self):
        return self.sumsep[-1]
    @staticmethod
    def check(odom_obj:pykitti.odometry,camera_avail:list)->bool:
        calib = odom_obj.calib
        for index in camera_avail:
            if not hasattr(calib,'T_cam{:d}_velo'.format(index)):
                return False
        return True
    def __getitem__(self, index):
        group_id = np.digitize(index,self.sumsep,right=False)
        data = self.kitti_datalist[group_id]
        T_cam2velo = getattr(data.calib,'T_cam%d_velo'%self.cam_id)
        K_cam = getattr(data.calib,'P_rect_%d0'%self.cam_id)
        if group_id > 0:
            sub_index = index - self.sumsep[group_id-1]
        else:
            sub_index = index
        img = getattr(data,'get_cam%d'%self.cam_id)(sub_index)  # PIL Image
        img = np.array(img)
        pcd = data.get_velo(sub_index)
        pcd[...,3] = 1.0
        pcd = T_cam2velo @ pcd.T  # [4,4] @ [4,N] -> [4,N]
        proj_pcd = K_cam @ pcd  # [3,4] @ [4,N] -> [3,N]
        proj_x = proj_pcd[0,:]/proj_pcd[2,:]
        proj_y = proj_pcd[1,:]/proj_pcd[2,:]
        rev = (0<=proj_x)*(proj_x<img.shape[1])*(0<=proj_y)*(proj_y<img.shape[0])
        proj_x = proj_x[rev]
        proj_y = proj_y[rev]
        img[proj_y.astype(np.int32),proj_x.astype(np.int32),:] = np.array([255,0,255],dtype=np.int32)
        return img

I really have no idea for projection in odometry benchmark, hope for your help.
Cheers.

Projection for multiple images in same folder

Thanks for this amazing repo.I have a question,how can I make projection for multiple images?

Can I save only projected Sparse point in camera coordinates in .png format ?

Hi, It would be very helpful if you provide me a code to save only projected sparse points in camera coordinates in .png format ? Thanks in advance.

License

Hi! Nice repo. Please consider adding a license. E.g. MIT.

深度信息的获取为什么是根据Z轴

非常感谢作者的开源分享，但是我有一个问题。

KITTI 的雷达坐标系中图像前方点云的深度信息应该是X方向，为什么代码最后使用Z方向作为深度信息显示？

velo = np.insert(points,3,1,axis=1).T
velo = np.delete(velo,np.where(velo[0,:]<0),axis=1)
cam = P2.dot(R0_rect.dot(Tr_velo_to_cam.dot(velo)))
cam = np.delete(cam,np.where(cam[2,:]<0),axis=1)

这里cam的维度是（3， 60828），也就是前三行维度应该是 --> (x, y, z)

经过一系列的filter处理后得到

u,v,z = cam
plt.scatter([u],[v],c=[z],cmap='rainbow_r',alpha=0.5,s=2)

为什么最后是根据Z方向的点云深度作为显示呢？Z方向不应该是图像正上方的点云深度吗？

非常希望得到您的答疑！