nirvanalan / awesome-aigc-3d

A curated list of awesome AIGC 3D papers

Awesome-AIGC-3D

A curated list of awesome AIGC 3D papers, inspired by awesome-NeRF.

How to submit a pull request?

Table of Contents

• Survey
• Papers
• Benchmarks and Datasets
• Talks
• Company
• Implementations

Survey

• 3D Generative Models: A Survey, Shi et al., arxiv 2022 | bibtex
• Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era, Li et al., arxiv 2023 | bibtex
• AI-Generated Content (AIGC) for Various Data Modalities: A Survey, Foo et al., arxiv 2023 | bibtex
• Advances in 3D Generation: A Survey, Li et al., arxiv 2024 | bibtex
• A Comprehensive Survey on 3D Content Generation, Liu et al., arxiv 2024 | bibtex

Papers

3D Native Generative Methods

Object

• Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings, Chen et al., ACCV 2018 | github | bibtex
• ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model, Fu et al., NeurIPS 2022 | github | bibtex
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images, Gao et al., NeurIPS 2022 | github | bibtex
• LION: Latent Point Diffusion Models for 3D Shape Generation, Zeng et al., NeurIPS 2022 | github | bibtex
• Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions, Chou et al., ICCV 2023 | github | bibtex
• MagicPony: Learning Articulated 3D Animals in the Wild, Wu et al., CVPR 2023 | github | bibtex
• DiffRF: Rendering-guided 3D Radiance Field Diffusion, Müller et al., CVPR 2023 | bibtex
• SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation, Cheng et al., CVPR 2023 | github | bibtex
• Point-E: A System for Generating 3D Point Clouds from Complex Prompts, Nichol et al., arxiv 2022 | github | bibtex
• 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models, Zhang et al., TOG 2023 | github | bibtex
• 3DGen: Triplane Latent Diffusion for Textured Mesh Generation, Gupta et al., arxiv 2023 | bibtex
• MeshDiffusion: Score-based Generative 3D Mesh Modeling, Liu et al., ICLR 2023 | github | bibtex
• HoloDiffusion: Training a 3D Diffusion Model using 2D Images, Karnewar et al., CVPR 2023 | github | bibtex
• HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion, Erkoç et al., ICCV 2023 | github | bibtex
• Shap-E: Generating Conditional 3D Implicit Functions, Jun et al., arxiv 2023 | github | bibtex
• LAS-Diffusion: Locally Attentional SDF Diffusion for Controllable 3D Shape Generation, Zheng et al., TOG 2023 | github | bibtex
• Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation, Zhao et al., NeurIPS 2023 | github | bibtex
• DiffComplete: Diffusion-based Generative 3D Shape Completion, Chu et al., NeurIPS 2023 | bibtex
• DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation, Mo et al., arxiv 2023 | github | bibtext
• 3D VADER - AutoDecoding Latent 3D Diffusion Models, Ntavelis et al., arxiv 2023 | github | bibtex
• ARGUS: Visualization of AI-Assisted Task Guidance in AR, Castelo et al., TVCG 2023 | bibtex
• Large-Vocabulary 3D Diffusion Model with Transformer, Cao et al., ICLR 2024 | github | bibtext
• TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fields, Huang et al., ICLR 2024 | bibtex
• HyperFields:Towards Zero-Shot Generation of NeRFs from Text, Babu et al., arxiv 2023 | github | bibtex
• LRM: Large Reconstruction Model for Single Image to 3D, Hong et al., ICLR 2024 | bibtex
• DMV3D:Denoising Multi-View Diffusion using 3D Large Reconstruction Model, Xu et al., ICLR 2024 | bibtex
• WildFusion:Learning 3D-Aware Latent Diffusion Models in View Space, Schwarz et al., ICLR 2024 | bibtex
• Functional Diffusion, Zhang et al., CVPR 2024 | github | bibtex
• MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers, Siddiqui et al., arxiv 2023 | github | bibtex
• SPiC·E: Structural Priors in 3D Diffusion Models using Cross-Entity Attention, Sella et al., arxiv 2023 | github | bibtex
• ZeroRF: Fast Sparse View 360° Reconstruction with Zero Pretraining, Shi et al., arxiv 2023 | github | bibtex
• Learning the 3D Fauna of the Web, Li et al., arxiv 2024 | bibtex
• Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability, Qian et al., arxiv 2024 | github | bibtext
• LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation, Lan et al., arxiv 2024 | github | bibtext
• GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation, Xu et al., arxiv 2024 | github | bibtext
• Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D, Varma T et al., CVPR 2024 | github | bibtext
• MeshLRM: Large Reconstruction Model for High-Quality Meshes, Wei et al., arxiv 2024 | bibtext
• Interactive3D🪄: Create What You Want by Interactive 3D Generation, Dong et al., CVPR 2024 | github | bibtex
• BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry, Xu et al., SIGGRAPH 2024 | github | bibtex
• Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer, Wu et al., arxiv 2024 | bibtex
• MeshAnything:Artist-Created Mesh Generation with Autoregressive Transformers, Chen et al., arxiv 2024 | github | bibtex
• CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets, Zhang et al., TOG 2024 | github | bibtex
• L4GM: Large 4D Gaussian Reconstruction Model, Ren et al., arxiv 2024 | bibtex

Scene

• GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis, Schwarz et al., NeurIPS 2020 | github | bibtext
• ATISS: Autoregressive Transformers for Indoor Scene Synthesis, Paschalidou et al., NeurIPS 2021 | github | bibtext
• GAUDI: A Neural Architect for Immersive 3D Scene Generation, Bautista et al., NeurIPS 2022 | github | bibtext
• NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models, Kim et al., CVPR 2023 | bibtext
• Pyramid Diffusion for Fine 3D Large Scene Generation, Liu et al., arxiv 2023 | github | bibtext
• XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies, Ren et al., arxiv 2023 | bibtex
• DUSt3R: Geometric 3D Vision Made Easy, Wang et al., arxiv 2023 | github | bibtext

Human Avatar

• SMPL: A skinned multi-person linear model, Loper et al., TOG 2015 | bibtex
• SMPLicit: Topology-aware Generative Model for Clothed People, Corona et al., CVPR 2021 | github | bibtext
• HeadNeRF: A Real-time NeRF-based Parametric Head Model, Hong et al., CVPR 2022 | github | bibtext
• gDNA: Towards Generative Detailed Neural Avatars, Chen et al., CVPR 2022 | github | bibtext
• Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion, Wang et al., CVPR 2023 | bibtex
• Single-View 3D Human Digitalization with Large Reconstruction Models, Weng et al., CVPR 2023 | bibtex

2D Prior-based 3D Generative Methods

Object

• DreamFields: Zero-Shot Text-Guided Object Generation with Dream Fields, Jain et al., CVPR 2022 | github | bibtex
• DreamFusion: Text-to-3D using 2D Diffusion, Poole et al., ICLR 2023 | github | bibtex
• Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models, Xu et al., CVPR 2023 | bibtex
• Magic3D: High-Resolution Text-to-3D Content Creation, Lin et al., CVPR 2023 | bibtex
• Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation, Wang et al., CVPR 2023 |github| bibtex
• RealFusion: 360° Reconstruction of Any Object from a Single Image, Melas-Kyriazi et al., CVPR 2023 | github | bibtex
• 3DFuse: Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation, Seo et al., ICLR 2024 | github | bibtex
• DreamBooth3D: Subject-Driven Text-to-3D Generation, Raj et al., ICCV 2023 | bibtex
• Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation, Chen et al., ICCV 2023 | github | bibtex
• Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior, Tang et al., ICCV 2023 | github | bibtex
• HiFA: High-fidelity Text-to-3D with Advanced Diffusion Guidance, Zhu et al., ICLR 2024 | github | bibtex
• ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, Wang et al., NeurIPS 2023 | github | bibtex
• ATT3D: Amortized Text-to-3D Object Synthesis, Lorraine et al., ICCV 2023 | bibtex
• DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation, Tang et al., ICLR 2024 | github | bibtex
• NFSD: Noise Free Score Distillation, Katzir et al., arxiv 2023 | github | bibtex
• Text-to-3D with Classifier Score Distillation, Yu et al., arxiv 2023 | github | bibtex
• IPDreamer: Appearance-Controllable 3D Object Generation with Image Prompts, Zeng et al., arxiv 2023 | bibtex
• Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts, Cheng et al., arxiv 2023 | github | bibtex
• Instant3D : Instant Text-to-3D Generation, Li et al., ICLR 2024 | bibtex
• LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching, Liang et al., arxiv 2023 | github | bibtex
• Control3D: Towards Controllable Text-to-3D Generation, Chen et al., ACM Multimedia 2023 | bibtex
• CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting, Vilesov et al., arxiv 2023 | bibtex
• StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D, Guo et al., arxiv 2023 | bibtex
• CAD: Photorealistic 3D Generation via Adversarial Distillation, Wan et al., arxiv 2023 | github | bibtex
• DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior, Huang et al., arxiv 2023 | github | bibtex
• AGAP:Learning Naturally Aggregated Appearance for Efficient 3D Editing, Cheng et al., arxiv 2023 | github | bibtex
• SSD: Stable Score Distillation for High-Quality 3D Generation, Tang et al., arxiv 2023 | bibtex
• SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity, Wang et al., arxiv 2023 | github | bibtex
• Taming Mode Collapse in Score Distillation for Text-to-3D Generation, Wang et al., arxiv 2024 | github | bibtex
• Score Distillation Sampling with Learned Manifold Corrective, Alldieck et al., arxiv 2024 | bibtex
• Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior, Wu et al., arxiv 2024 | bibtex
• TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts, Zhuang et al., arxiv 2024 | bibtex

Scene

• Text2Light: Zero-Shot Text-Driven HDR Panorama Generation, Chen et al., TOG 2022 | github | bibtext
• SceneScape: Text-Driven Consistent Scene Generation, Fridman et al., NeurIPS 2023 | github | bibtext
• DiffuScene: Scene Graph Denoising Diffusion Probabilistic Model for Generative Indoor Scene Synthesis, Tang et al., arxiv 2023 | github | bibtext
• Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models, Höllein et al., ICCV 2023 | github | bibtext
• Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields, Zhang et al., TVCG 2024 | github | bibtext
• CityDreamer: Compositional Generative Model of Unbounded 3D Cities, Xie et al., arxiv 2023 | github | bibtext
• GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting, Chen et al., arxiv 2023 | github | bibtex
• LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes, Chuang et al., arxiv 2023 | github | bibtext
• GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions, Fang et al., arxiv 2023 | bibtex
• Gaussian Grouping: Segment and Edit Anything in 3D Scenes, Ye et al., arxiv 2023 | github | bibtex
• Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusion, Prabhu et al., arxiv 2023 | bibtext
• SIGNeRF: Scene Integrated Generation for Neural Radiance Fields, Dihlmann et al., arxiv 2024 | github | bibtex
• Disentangled 3D Scene Generation with Layout Learning, Epstein, et al., arxiv 2024 | bibtex

Human Avatar

• AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars, Hong et al., SIGGRAPH 2022 | github | bibtex
• DreamWaltz: Make a Scene with Complex 3D Animatable Avatars, Huang et al., NeurIPS 2023 | github | bibtex
• DreamHuman: Animatable 3D Avatars from Text, Wang et al., arxiv 2023 | bibtex
• TECA: Text-Guided Generation and Editing of Compositional 3D Avatars, Zhang et al., arxiv 2023 | github | bibtex
• HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting, Liu et al., arxiv 2023 | github | bibtex
• HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation, Liu et al., arxiv 2023 | bibtex
• 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting, Qian et al., arxiv 2023 | github | bibtex

Hybrid 3D Generative Methods

Object

• Zero-1-to-3: Zero-shot One Image to 3D Object, Liu et al., ICCV 2023 | github | bibtex
• One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization, Liu et al., NeurIPS 2023 | github | bibtex
• Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors, Qian et al., arxiv 2023 | github | bibtex
• MVDream: Multi-view Diffusion for 3D Generation, Shi et al., arxiv 2023 | github | bibtex
• SyncDreamer: Generating Multiview-consistent Images from a Single-view Image, Liu et al., arxiv 2023 | github | bibtex
• Gsgen: Text-to-3D using Gaussian Splatting, Chen et al., arxiv 2023 | github | bibtex
• Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors, Lin et al., arxiv 2024 | bibtex
• GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models, Yi et al., arxiv 2023 | github | bibtex
• Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models, Ye et al., 3DV 2024 | bibtex
• Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model, Shi et al., arxiv 2023 | github | bibtex
• TOSS: High-quality Text-guided Novel View Synthesis from a Single Image, Shi et al., arxiv 2023 | bibtex
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion, Long et al., arxiv 2023 | github | bibtex
• DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior, Sun et al., ICLR 2024 | github | bibtex
• SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D, Li et al., arxiv 2023 | github | bibtex
• One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion, Liu et al., arxiv 2023 | github | bibtex
• Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion, Lu et al., arxiv 2023 | bibtex
• ConRad: Image Constrained Radiance Fields for 3D Generation from a Single Image, Purushwalkam et al., NeurIPS 2023 | bibtex
• Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model, Li et al., arxiv 2023 | bibtex
• MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generation, Li et al., arxiv 2023 | github | bibtex
• GeoDream:Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation, Ma et al., arxiv 2023 | github | bibtex
• RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D, Qiu et al., arxiv 2023 | github | bibtex
• Slice3D: Multi-Slice, Occlusion-Revealing, Single View 3D Reconstruction, Wang et al., CVPR 2024 | github | bibtex
• DreamComposer: Controllable 3D Object Generation via Multi-View Conditions, Yang et al., arxiv 2023 | github | bibtex
• Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted Nearby Views, Chen et al., arxiv 2023 | github | bibtex
• Free3D: Consistent Novel View Synthesis without 3D Representation, Zheng et al., arxiv 2023 | github | bibtex
• Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior, Liu et al., arxiv 2023 | github | bibtex
• UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation, Liu et al., arxiv 2023 | github | bibtex
• Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting, Zhang et al., arxiv 2023 | github | bibtex
• BiDiff: Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors, Ding et al., arxiv 2023 | github | bibtex
• ControlDreamer: Stylized 3D Generation with Multi-View ControlNet, Oh et al., arxiv 2023 | github | bibtex
• X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation, Ma et al., arxiv 2023 | github | bibtex
• Splatter Image: Ultra-Fast Single-View 3D Reconstruction, Szymanowicz et al., arxiv 2023 | github | bibtex
• Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning, Xie et al., arxiv 2023 | bibtex
• HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D, Woo et al., arxiv 2023 | github | bibtex
• ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation, Wang et al., arxiv 2023 | github | bibtex
• iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views, Wu et al., arxiv 2023 | github | bibtex
• AGG: Amortized Generative 3D Gaussians for Single Image to 3D, Xu et al., arxiv 2024 | bibtex
• HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generation, Mercier et al., arxiv 2024 | bibtex
• HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generation, Mercier et al., arxiv 2024 | bibtex
• Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation, Chen et al., arxiv 2024 | bibtex
• IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation, Melas-Kyriazi et al., arxiv 2024 | bibtex
• LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation, Tang et al., arxiv 2024 | github | bibtex
• Retrieval-Augmented Score Distillation for Text-to-3D Generation, Seo et al., ICML 2024 | github | bibtex
• EscherNet: A Generative Model for Scalable View Synthesis, Kong et al., arxiv 2024 | github | bibtex
• MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction, Tang et al., arxiv 2024 | bibtex
• MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusion, Zheng et al., arxiv 2024 | bibtex
• Consolidating Attention Features for Multi-view Image Editing, Patashnik et al., arxiv 2024 | bibtex
• ViewFusion: Towards Multi-View Consistency via Interpolated Denoising, Yang et al., CVPR 2024 | github | bibtex
• CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model, Wang et al., arxiv 2024 | github | bibtext
• Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior, Chen et al., CVPR 2024 | github | bibtext
• Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generation, Liu et al., arxiv 2024 | github | bibtext
• Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splatting, Li et al., arxiv 2024 | github | bibtex
• VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model, Zuo et al., arxiv 2024 | github | bibtex
• SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion, Voleti et al., arxiv 2024 | bibtex
• DreamReward: Text-to-3D Generation with Human Preference, Ye et al., arxiv 2024 | bibtex
• LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis, Xie et al., arxiv 2024 | bibtex
• DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusion, Lin et al., arxiv 2024 | github | bibtex
• GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image, Fu et al., arxiv 2024 | github | bibtex
• ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars, Wang et al., arxiv 2024 | github | bibtex
• FlexiDreamer: Single Image-to-3D Generation with FlexiCubes, Zhao et al., arxiv 2024 | github | bibtex
• Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation, Zheng et al., arxiv 2024 | bibtex
• DreamView: Injecting View-specific Text Guidance into Text-to-3D Generation, Yan et al., arxiv 2024 | github | bibtex
• InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models, Xu et al., arxiv 2024 | github | bibtex
• DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing, Chen et al., arxiv 2024 | github | bibtex
• MicroDreamer: Zero-shot 3D Generation in ∼20 Seconds by Score-based Iterative Reconstruction, Chen et al., arxiv 2024 | github | bibtex
• SketchDream: Sketch-based Text-to-3D Generation and Editing, Liu et al., SIGGRAPH 2024 | bibtex
• Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image, Wu et al., arxiv 2024 | github | bibtex
• Fourier123: One Image to High-Quality 3D Object Generation with Hybrid Fourier Score Distillation, Yang et al., arxiv 2024 | github | bibtex
• CAT3D: Create Anything in 3D with Multi-View Diffusion Models, Gao et al., arxiv 2024 | bibtext
• CraftsMan: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner, Li et al., arxiv 2024 | github | bibtext

Scene

• Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints, Fang et al., arxiv 2023 | github | bibtext
• RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation, Zhao et al., 3DV 2024 | github | bibtext
• ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Real Image, Sargent et al., arxiv 2023 | github | bibtext
• GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs, Gao et al., arxiv 2023 | github | bibtext
• ControlRoom3D:Room Generation using Semantic Proxy Rooms, Schult et al., arxiv 2023 | bibtext
• AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes, Wen et al., arxiv 2023 | bibtext
• SceneWiz3D: Towards Text-guided 3D Scene Composition, Zhang et al., arxiv 2023 | github | bibtext
• Text2Immersion: Generative Immersive Scene with 3D Gaussians, Ouyang et al., arxiv 2023 | bibtext
• ShowRoom3D: Text to High-Quality 3D Room Generation Using 3D Priors, Mao et al., arxiv 2023 | github | bibtext
• GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting, Zhou et al., arxiv 2024 | github | bibtext
• 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation, Zhang et al., arxiv 2024 | bibtext
• Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image, Szymanowicz et al., arxiv 2024 | bibtext

Human Avatar

• SofGAN: A Portrait Image Generator with Dynamic Styling, Chen et al., TOG 2022 | github | bibtext
• Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors, Xiong et al., ICCV 2023 | github | bibtext
• DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance, Zhang et al., arxiv 2023 | bibtext
• TADA! Text to Animatable Digital Avatars, Liao et al., 3DV 2024 | github | bibtext
• SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes, Sanyal et al., arxiv 2023 | bibtext
• HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation, Huang et al., arxiv 2023 | github | bibtex

Dynamic

• MAV3d: Text-To-4D Dynamic Scene Generation, Singer et al., arxiv 2023 | bibtext
• Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D Diffusion-based Editor, Shao et al., arxiv 2023 | bibtex
• MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusion, Kapon et al., arxiv 2023 | github | bibtext
• Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video, Jiang et al., arxiv 2023 | github | bibtext
• Animate124: Animating One Image to 4D Dynamic Scene, Zhao et al., arxiv 2023 | github | bibtext
• A Unified Approach for Text- and Image-guided 4D Scene Generation, Zheng et al., arxiv 2023 | bibtext
• 4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling, Bahmani et al., arxiv 2023 | github | bibtext
• AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation, Wang et al., arxiv 2023 | bibtext
• Virtual Pets: Animatable Animal Generation in 3D Scenes, Cheng et al., arxiv 2023 | github | bibtext
• Align Your Gaussians:Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models, Ling et al., arxiv 2023 bibtext
• Ponymation: Learning 3D Animal Motions from Unlabeled Online Videos, Sun et al., arxiv 2023 | bibtext
• 4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency, Yin et al., arxiv 2023 | github | bibtext
• DreamGaussian4D: Generative 4D Gaussian Splatting, Ren et al., arxiv 2023 | github | bibtext
• Fast Dynamic 3D Object Generation from a Single-view Video, Pan et al., arxiv 2024 | github | bibtext
• ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance, Chen et al., arxiv 2024 | bibtext
• STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians, Zeng et al., arxiv 2024 | bibtext
• TC4D: Trajectory-Conditioned Text-to-4D Generation, Bahmani et al., arxiv 2024 | bibtext
• Diffusion^2: Dynamic 3D Content Generation via Score Composition of Orthogonal Diffusion Models, Yang et al., arxiv 2024 | bibtext
• Hash3D: Training-free Acceleration for 3D Generation, Yang et al., arxiv 2024 | github | bibtext
• Magic-Boost: Boost 3D Generation with Mutli-View Conditioned Diffusion, Yang et al., arxiv 2024 | github | bibtext
• Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences, Kim et al., CVPR 2024 | bibtext
• DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos, Chu et al., arxiv 2024 | github | bibtext
• Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models, Liang et al., arxiv 2024 | github | bibtext
• Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels, Wang et al., arxiv 2024 | github | bibtext
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion, Liu et al., arxiv 2024 | github | bibtext
• 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models, Liu et al., arxiv 2024 | bibtext

Others

Physical

• Physical Property Understanding from Language-Embedded Feature Fields, Zhai et al., CVPR 2024 | github | bibtext

Texture

• StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions, Höllein et al., CVPR 2022 | github | bibtex
• CLIP-Mesh: Generating textured meshes from text using pretrained image-text models, Khalid et al., SIGGRAPH Asia 2022 | github | bibtex
• TANGO: Text-driven PhotoreAlistic aNd Robust 3D Stylization via LiGhting DecompOsition, Chen et al., NeurIPS 2022 | github | bibtex
• Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures, Metzer et al., CVPR 2023 | github | bibtex
• TEXTure: Text-Guided Texturing of 3D Shapes, Richardson et al., SIGGRAPH 2023 | github | bibtex
• Text2Tex: Text-driven Texture Synthesis via Diffusion Models, Chen et al., ICCV 2023 | github | bibtex
• RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture, Song et al., ACM Multimedia 2023 | bibtex
• Generating Parametric BRDFs from Natural Language Descriptions, Memery et al., arxiv 2023 bibtex
• MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion, Tang et al., NeurIPS 2023 | github | bibtext
• MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR, Xu et al., arxiv 2023 | github | bibtex
• ITEM3D: Illumination-Aware Directional Texture Editing for 3D Models, Liu et al., arxiv 2023 | github | bibtex
• TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models, Cao et al., ICCV 2023 | bibtex
• DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation, Yang et al., arxiv 2023 | github | bibtext
• 3DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models, Yang et al., ACM Multimedia 2023 | github | bibtex
• Text-Guided Texturing by Synchronized Multi-View Diffusion, Liu et al., arxiv 2023 | bibtex
• SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors, Chen et al., arxiv 2023 | github | bibtext
• TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes, Zhang et al., arxiv 2023 | bibtex
• Single Mesh Diffusion Models with Field Latents for Texture Generation, Mitchel et al., arxiv 2023 | bibtex
• Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering, Youwang et al., arxiv 2023 | github | bibtext
• Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models, Zeng et al., arxiv 2023 | github | bibtext
• TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion, Yeh et al., arxiv 2024 | bibtext
• FlashTex: Fast Relightable Mesh Texturing with LightControlNet, Deng et al., arxiv 2024 | bibtext
• Make-it-Real: Unleashing Large Multimodal Model's Ability for Painting 3D Objects with Realistic Materials, Fang et al., arxiv 2024 | github | bibtext
• MaPa: Text-driven Photorealistic Material Painting for 3D Shapes, Zhang et al., SIGGRAPH 2024 | bibtext

Procedural 3D Modeling

• ProcTHOR: Large-Scale Embodied AI Using Procedural Generation, Deitke et al., NeurIPS 2022 | github | bibtex
• 3D-GPT: Procedural 3D Modeling with Large Language Models, Sun et al., arxiv 2023 | github | bibtex

3D Representation

• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, Mildenhall et al., ECCV 2020 | github | bibtex
• Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis, Shen et al., arxiv 2021 | bibtex
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering, Kerbl et al., TOG 2023 | github | bibtex
• Uni3D: Exploring Unified 3D Representation at Scale, Zhou et al., ICLR 2024 | github | bibtex
• SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene, Duckworth et al., arxiv 2023 | bibtex
• Triplane Meets Gaussian Splatting:Fast and Generalizable Single-View 3D Reconstruction with Transformers, Zou et al., arxiv 2023 | bibtex
• SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes, Huang et al., arxiv 2023 | github | bibtex
• DMesh: A Differentiable Representation for General Meshes, Son et al., arxiv 2024 | github | bibtex

Benchmarks and Datasets

• Objaverse-XL, Deitke et al., NeurIPS 2023 | github | bibtext
• G-buffer Objaverse: High-Quality Rendering Dataset of Objaverse, Xu et al.
• GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation, Wu et al., arXiv 2024 | github | bibtext
• SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding, Jia et al., arXiv 2024 | bibtext
• Make-A-Shape: a Ten-Million-scale 3D Shape Model, Wu et al., arXiv 2024 | bibtext
• Zeroverse, Xie et al., arXiv 2024 | github | bibtext

Talks

• AI 3D Generation, explained, Jia-Bin Huang
• 3D Generation, bilibili, Leo
• 3D AIGC Algorithm Trends and Industry Implementation, Ding Liang
• 3D Generation: Past, Present and Future,GAMES Webinar 311

Company

• TRIPOAI
• LumaAI
• CSMAI
• SUDOAI
• Meshy
• StabilityAI

Implementations

• Threestudio, Yuan-Chen Guo, 2023 | bibtex
• stable-dreamfusion, Jiaxiang Tang, 2023 | bibtex
• Dream Textures, Carson Katri, 2023
• ComfyTextures, Alexander Dzhoganov, 2023
• ComfyUI-3D-Pack, MrForExample, 2024
• GauStudio, Ye et al., arxiv 2024 | github | bibtex

License

Awesome AIGC 3D is released under the MIT license.

Citation

If you find this project useful in your research, please consider citing:

@article{liu2024comprehensive,
  title={A Comprehensive Survey on 3D Content Generation},
  author={Liu, Jian and Huang, Xiaoshui and Huang, Tianyu and Chen, Lu and Hou, Yuenan and Tang, Shixiang and Liu, Ziwei and Ouyang, Wanli and Zuo, Wangmeng and Jiang, Junjun and others},
  journal={arXiv preprint arXiv:2402.01166},
  year={2024}
}

Contact

contact: [email protected].