ccinc / 3d-ml Goto Github PK
View Code? Open in Web Editor NEWA versatile framework for 3D machine learning built on Pytorch Lightning and Hydra [looking for contributors!]
A versatile framework for 3D machine learning built on Pytorch Lightning and Hydra [looking for contributors!]
The Paris-Lille-3D is a Dataset and a Benchmark on Point Cloud Classification. The data has been produced by a Mobile Laser System (MLS) in two different cities in France (Paris and Lille).
The Point Cloud has been labeled entirely by hand with 50 different classes to help the research community on automatic point cloud segmentation and classification algorithms.
This adds additional complexity due to the collation of the datasets. OpenPoints datasets are batched "densely", i.e. 16 batches of data in the shape [2048, 3] are batched into a single tensor of shape [16, 2048, 3] (implemented based on the original TP3D code here
Line 17 in 70de732
TorchPoints accomplishes this by setting a configuration option in the model to define whether it uses "dense" or "sparse" data. We would likely need to do the same, and have the dataloader batch according to this configuration option. Ref: https://github.com/torch-points3d/torch-points3d/blob/66e8bf22b2d98adca804c753ac3f0013ff4ec731/torch_points3d/datasets/base_dataset.py#L160-L174
Issue for tracking how to incorporate testing within the repo.
torch_geometric.data.Data
formatProvide support on guochengqian/PointNeXt#79, i.e. enabling OpenPoints to become a pip and conda-installable package.
Semantic KITTI (http://semantic-kitti.org/) is a popular dataset for semantic segmentation.
Investigate the use of pytorch lightning's alternative training modalities such as DP and DDP for using multiple GPUs during training.
Some of the datamodule code can likely be refactored out, such as the dataloaders.
OpenPoints contains many learning rate schedulers that we can add hydra config files to support: https://github.com/guochengqian/openpoints/tree/master/scheduler
We present the Dayton Annotated Laser Earth Scan (DALES) data set, a new large-scale aerial LiDAR data set with nearly a half-billion points spanning 10 square kilometers of area. DALES contains forty scenes of dense, labeled aerial data spanning multiple scene types, including urban, suburban, rural, and commercial. The data was hand-labeled by a team of expert LiDAR technicians into eight categories: ground, vegetation, cars, trucks, poles, power lines, fences, and buildings. We present the entire data set, split into testing and training, and provided in 3 different data formats. The goal of this data set is to help advance the field of deep learning within aerial LiDAR.
Toronto-3D is a large-scale urban outdoor point cloud dataset acquired by an MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of road and consists of about 78.3 million points.
Semantic-3d is a terrestrially-acquired semantic segmentation dataset with 4 billion points in an urban landscape.
Many 2d-based models have been created to perform ML tasks on 3D point clouds, such as SalsaNet/SalsaNext. Investigate supporting these workflows in this repo.
Hi @CCInc ,
I've tried installing this repo and occured an error while running ./install_openpoints.sh.
My versions of software components:
Using pip in virtual environment.
Python version: Python 3.10.8
pip version: pip 22.3.1
nvcc version:
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
gcc version: 7.5.
OS info:
linux version:
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
kernel version:
5.4.0-125-generic
commands ran:
#add recursive comment otherwise openpoints folder content is not included (because it's a submodule)
git clone https://github.com/CCInc/3d-ml.git --recursive
#create virtual env with python, go inside it.
cd 3d-ml
python -m virtualenv env_3d
source env_3d/bin/activate
#install pytorch with pip
pip3 install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu116
#install pytorch geo with pip, ${CUDA} = cu116
pip install pyg-lib torch-scatter torch-sparse -f https://data.pyg.org/whl/torch-1.13.0+cu116.html
pip install torch-geometric
#install additional requirements
pip install -r requirements.txt
#install openpoints as root
sudo ./install_openpoints.sh
I receive the following warnings
cuda/emd_kernel.cu(178): error: identifier "CHECK_EQ" is undefined
cuda/emd_kernel.cu(265): error: identifier "CHECK_EQ" is undefined
cuda/emd_kernel.cu(382): error: identifier "CHECK_EQ" is undefined
I also receive some warnings:
.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
.local/lib/python3.10/site-packages/setuptools/command/easy_install.py:160: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
What is the recommended way to install this repo if not using conda?
Also noticed that python <= 3.8 is not supported.
Provide a means of exploring datasets before, during and after training (i.e. with ground truth labels and with predicted labels).
Maybe use the Open3d visualizer API? https://github.com/isl-org/Open3D-ML/blob/21951adc23831ee0597c4f49b47deb8f122fda07/examples/visualize.py
Other alternatives are writing to Tensorboard and writing files to the output directory in a ply or las format.
TP3D and PointNeXt allow processing the full S3DIS dataset, which allows sampling by room, boxes, cylinders and spheres (while the PyG dataset is already preprocessed to 1x1m boxes).
Have generic base models for tasks such as classification and segmentation, which commonly share the same metrics (such as loss, accuracy, and iou). This would likely entail refactoring out most methods besides the step
and forward
logic, to preprocess the data for the downstream model plugin as needed.
Currently, custom code is written to handle the downloading/processing of modelnet2048. It should be rewritten in the context of a pytorch geometric InMemoryDataset, which has helper functions to handle downloading and processing of the data from the h5py input format (and removing the custom implemented download functions)
See:
https://pytorch-geometric.readthedocs.io/en/latest/notes/create_dataset.html
https://github.com/pyg-team/pytorch_geometric/blob/master/torch_geometric/datasets/s3dis.py <- the pyg s3dis dataset also comes from a h5py source, very similar to modelnet2048
Add support for Pytorch Geometric data transformation pipelines for train, test, and validation datasets. Reimplement some TP3D transform methods within the new repo, such as AddFeatsByKeys
and GridSampling3d
. Investigate using OpenPoints transforms.
ScanNet is one of the most used 3D datasets for Semantic, Instance segmentation.
The Dataloader should support both tasks. Maybe Semantic can be inherited to produce an Instance segmentation loader.
--user
from open_points conda installA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.