klekkala / 3d2d_ann Goto Github PK

Given a set of global odometry (files containing positions of the robot) at a high frequency and a specific pose (position+orientation), compute the Birds Eye View (BEV) of the 2D occupancy map of the environment. Typical Birds Eye View images look like the following attachments.

Difficulty: 4/5

Colab link: https://colab.research.google.com/drive/1yNz9ifjTV-EbBZvmaAaUt7XH2_vjgTu9?usp=sharing

You are given a set of images and pointclouds. We also provided code for the processes mentioned in this description like selecting closest image to every pointcloud and projecting the pointcloud to the image. Extract semantic labels of the image using Grounded SAM. Create a folder named output, which contains directories, corresponding to every pointcloud, with the filename as the timestamp of that pointcloud. In each of that directory you need to have the pointcloud that is stored as cloud.pcd and the label for every voxel (equivalent of pixel in the 3D pointcloud). The format of the label file can be any choice. Transform semantic labels of the images (2D semantic labels) through Grounded SAM, to the pointcloud (3D semantic labels). Hint: Remember that we have data from 5 cameras that have some overlap, ontop of that these cameras keep recording stream of images. Given a 3D point in a scene, there are many images that capture it, can you make use of all these labels to deduce an accurate label of that 3D point?

Colab link: https://colab.research.google.com/drive/17ImiKtmkC91aZ76DC_fKTi5wwgrE3SI6?usp=sharing

Given a 3D pointcloud of a scene, draw a 2D elevation map of the scene. 2D elevation map is essentially a top-down view of the scene, where every pixel on the map corresponds to the elevation of the specific point in the scene from the top view.

Colab link:https://colab.research.google.com/drive/1JPWrBMG1f5Dgi93JhF4IooCcoo6IuQrK?usp=sharing