GithubHelp home page GithubHelp logo

daerduocarey / structurenet Goto Github PK

View Code? Open in Web Editor NEW
258.0 12.0 44.0 11.66 MB

StructureNet: Hierarchical Graph Networks for 3D Shape Generation

Home Page: https://cs.stanford.edu/~kaichun/structurenet/

License: Other

Python 21.59% Shell 0.24% C++ 0.47% Cuda 0.38% Jupyter Notebook 77.32%

structurenet's Introduction

StructureNet: Hierarchical Graph Networks for 3D Shape Generation

Overview

Figure 1. StructureNet is a hierarchical graph network that produces a unified latent space to encode structured models with both continuous geometric and discrete structural variations. In this example, we projected an un-annotated point cloud (left) and un-annotated image (right) into the learned latent space yielding semantically segmented point clouds structured as a hierarchy of graphs. The shape interpolation in the latent space also produces structured point clouds (top) including their corresponding graphs (bottom). Edges correspond to specific part relationships that are modeled by our approach. For simplicity, here we only show the graphs without the hierarchy. Note how the base of the chair morphs via functionally plausible intermediate configurations, or the chair back transitions from a plain back to a back with arm-rests.

Introduction

We introduce a hierarchical graph network for learning structure-aware shape generation which (i) can directly encode shape parts represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families such as PartNet; and (iii) can be used to generate a great diversity of realistic structured shape geometries with both both continuous geometric and discrete structural variations.

About the paper

Our team: Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, and Leonidas J. Guibas from Stanford University, University College London (UCL), University of California San Diego (UCSD), King Abdullah University of Science and Technology (KAUST), Adobe Research and Facebook AI Research.

Arxiv Version: https://arxiv.org/abs/1908.00575

Accepted by Siggraph Asia 2019. See you at Brisbane, Australia in November!

Project Page: https://cs.stanford.edu/~kaichun/structurenet/

Citations

@article{mo2019structurenet,
      title={StructureNet: Hierarchical Graph Networks for 3D Shape Generation},
      author={Mo, Kaichun and Guerrero, Paul and Yi, Li and Su, Hao and Wonka, Peter and Mitra, Niloy and Guibas, Leonidas},
      journal={ACM Transactions on Graphics (TOG), Siggraph Asia 2019},
      volume={38},
      number={6},
      pages={Article 242},
      year={2019},
      publisher={ACM}
}

About this repository

This repository provides data and code as follows.

    data/                   # contains data, models, results, logs
    code/                   # contains code and scripts
         # please follow `code/README.md` to run the code
    stats/                  # contains helper statistics

Questions

Please post issues for questions and more helps on this Github repo page. We encourage using Github issues instead of sending us emails since your questions may benefit others.

License

MIT Licence

Updates

  • [July 27, 2019] Data and Code released.
  • [May 13, 2020] Release the blender code for rendering figures in the paper.

structurenet's People

Contributors

daerduocarey avatar paulguerrero avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

structurenet's Issues

About the data you provided

You provided a link of a zip file in data/partnetdata/README.md and the corresponding details:

    chair_hier/                         # hierarchy of graphs data for chairs
            [PartNet_anno_id].json      # storing the tree structure, detected sibling edges, 
                                        # and all part oriented bounding box parameters) for a chair
            [train/test/val].txt        # PartNet train/test/val split
            [train/test/val]_no_other_less_than_10_parts.txt    
                                        # Subsets of data used in StructureNet where all parts are labeled 
                                        # and no more than 10 parts per parent node
                                        # We use this subset for StructureNet
    chair_geo/                          # part geometry point clouds for chairs
            [PartNet_anno_id].npz       # storing all part geometry point clouds for a chair

How do you produce the [PartNet_anno_id].json, [train/test/val]_no_other_less_than_10_parts.txt and [PartNet_anno_id].npz? Could you release the releated code?

Question about metric evaluate codes for Point Cloud

Thanks for sharing the fancy project. And I find the metric evaluate codes for bounding box in your code. However, I do not find the metric evaluate codes for point clouds. May I ask have you provided the codes for point cloud?

Thanks

Long training time + initial increase in KL divergence

Thank you for your work and accompanying codebase!

  1. I am able to run scripts/box_vae_chair.sh, but I am finding it will take far longer (~1h per epoch ~= 8.3 days) to get to 200 training epochs than the 1-2 days your paper mentioned for bounding box inputs using:
  • GeForce GTX TITAN X GPU and a Intel Xeon E5-2630 CPU, whose specs are similar to yours (I also tried it out on a machine with Titan XP + i7-5930K)
  • CUDA 10.0, PyTorch 1.1.0, torch-scatter 1.1.0 and torchvision 0.3.0
  • varying numbers in torch.set_num_threads(4) but this did not affect training time (including with a number of threads >4)
  1. For my training (first 10 epochs) thus far KL divergence is increasing, I wondered if this was expected behavior?
    KLdiv

Loading .npz on Pytorch3D Poinclouds structure gives - KeyError: 'verts is not a file in the archive'

Awesome project, thank you so much

KeyError                                  Traceback (most recent call last)
<ipython-input-20-1daf13c71fa1> in <module>()
----> 1 verts = torch.Tensor(pointcloud['verts']).to(device)
      2 rgb = torch.Tensor(pointcloud['rgb']).to(device)
      3 
      4 point_cloud = Pointclouds(points=[verts], features=[rgb])

/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py in __getitem__(self, key)
    264                 return self.zip.read(key)
    265         else:
--> 266             raise KeyError("%s is not a file in the archive" % key)
    267 
    268 

**KeyError: 'verts is not a file in the archive'**

I am creating Pytorch3d datastructure Pointclouds

from pytorch3d.structures import Pointclouds
#import all the others

# Set paths
DATA_DIR = "./data"
obj_filename = os.path.join(DATA_DIR, "/content/drive/My Drive/Datasets/StructureNET/partnethiergeo/chair_geo/173.npz")

# Load point cloud
pointcloud = np.load(obj_filename)
verts = torch.Tensor(pointcloud['verts']).to(device)     
rgb = torch.Tensor(pointcloud['rgb']).to(device)

point_cloud = Pointclouds(points=[verts], features=[rgb])

I was wondering if the file .npz are normal pointclouds or they are different, and why I am receiving that error? any suggestions how to visualize that pointcloud in Colab?

Use Partnet dataset in Structurenet repo

Hello, hope you are doing well.
My question is, how can I convert the input structure of partnet dataset of any 3-D Object into format which you have used in the github: https://github.com/daerduoCarey/structurenet for training purpose.
like this >
and dataset in .json format
--train_dataset 'train_no_other_less_than_10_parts.txt'
--val_dataset 'val_no_other_less_than_10_parts.txt' \

ShapeNet ID for the PartNet data here

Thanks for sharing this great project. I'm seeking if I could get the original shapeNet id for the provided data here. The shapeNet id is like "f859a1ed07d088be920de219c00d1c3b" and for the id used in StructureNet, it looks like "4144".

Codes for Point Cloud Abstraction

As stated in section 6.4 of StructureNet paper, it trains another PointNet++-based encoder to encode point clouds, which shows the strength of latent space. Could you please provide codes and brief network design for this part (point cloud abstraction)? Thanks! @daerduoCarey

Release more code

  • [ToDo] Convert PartNet to StructureNet box-shapes
  • [ToDo] Blender visualization code
  • [ToDo] Code for applications (image --> structure, pc --> structure)

Please request here if you need more! Thank you!

Best,
Kaichun

Loading .npz on Pytorch3D Poinclouds structure gives - KeyError: 'verts is not a file in the archive'

Awesome project, thank you so much

KeyError                                  Traceback (most recent call last)
<ipython-input-20-1daf13c71fa1> in <module>()
----> 1 verts = torch.Tensor(pointcloud['verts']).to(device)
      2 rgb = torch.Tensor(pointcloud['rgb']).to(device)
      3 
      4 point_cloud = Pointclouds(points=[verts], features=[rgb])

/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py in __getitem__(self, key)
    264                 return self.zip.read(key)
    265         else:
--> 266             raise KeyError("%s is not a file in the archive" % key)
    267 
    268 

**KeyError: 'verts is not a file in the archive'**

I am creating Pytorch3d datastructure Pointclouds

from pytorch3d.structures import Pointclouds
#import all the others

# Set paths
DATA_DIR = "./data"
obj_filename = os.path.join(DATA_DIR, "/content/drive/My Drive/Datasets/StructureNET/partnethiergeo/chair_geo/173.npz")

# Load point cloud
pointcloud = np.load(obj_filename)
verts = torch.Tensor(pointcloud['verts']).to(device)     
rgb = torch.Tensor(pointcloud['rgb']).to(device)

point_cloud = Pointclouds(points=[verts], features=[rgb])

I was wondering if the file .npz are normal pointclouds or they are different, and why I am receiving that error? any suggestions how to visualize that pointcloud in Colab?

size mismatch in PartDecoder, m1: [64 x 50], m2: [100 x 100]

We meet problem that PartEncoder give a feature [64 x 50], but the input of PartDecoder is [100 x100]. It cause the RuntimeError: size mismatch, m1: [64 x 50], m2: [100 x 100] at /tmp/pip-req-build-9xcrj8au/aten/src/THC/generic/THCTensorMathBlas.cu:268

Please help me, thanks!

Missing folder 'utils'

Hello,

Thank you for releasing the code for this interesting work.

In the folder, partnet_edges/detect_all_edges.py, the utils folder seems to be missing. Could you update the code to include that folder?

RuntimeError: Error building extension 'cd'

Hi,
I have been trying to build extension for chamfer_distance but it fails. The reason for failure seems to be

nvcc fatal   : Unknown option '-generate-dependencies-with-compile'
ninja: build stopped: subcommand failed.

Can you please help to resolve this issue?
Thank you very much in advance. :)

Unable to download data and pretrained models

Hi,

Thanks for publishing the code for this interesting work.

After filling the form in the data/partnet_data directory, I am unable to download the data and pretrained models. Could you please check the links for the same?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.