GithubHelp home page GithubHelp logo

imclab / tdw Goto Github PK

View Code? Open in Web Editor NEW

This project forked from threedworld-mit/tdw

0.0 0.0 0.0 148.65 MB

ThreeDWorld simulation environment

License: BSD 2-Clause "Simplified" License

Shell 0.15% Python 99.77% Dockerfile 0.08%

tdw's Introduction

ThreeDWorld (TDW)

ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. With TDW, users can simulate high-fidelity sensory data and physical interactions between mobile agents and objects in a wide variety of rich 3D environments.

General guide to TDW

Setup

  1. Install TDW
  2. Upgrade TDW

Core Concepts

  1. The controller
  2. Auto-launching the TDW build
  3. Commands
  4. Design philosophy of TDW
  5. Scenes
  6. Avatars and cameras
  7. Add-ons and the ThirdPersonCamera
  8. Objects
  9. Output data
  10. Images

Troubleshooting

  1. How to report an issue
  2. Common errors
  3. Performance optimizations
  4. Good coding practices
  5. The Logger add-on

Tutorials

1. Scene Setup

Overview

1.1 Scene Setup (High-Level APIs)

  1. Overview
  2. Procedural generation (the ProcGenKitchen add-on)
  3. Regions, interior regions, and rooms
  4. Procedural object arrangements
  5. Scripted object placement (floorplan layouts)
  6. Reset a scene

High-level APs: Floorplan and ProcGenKitchen

1.2 Scene Setup (Low-Level APIs)

  1. Overview
  2. Units and data formats
  3. Bounds output data
  4. Visual materials, textures, and colors
  5. ProcGenRoom

2. 3D Model Libraries

  1. Overview
  2. Free models
  3. Non-free models
  4. Add your own models to TDW
  5. Add ShapeNet models to TDW

3. Semantic States

  1. Overview
  2. Line of sight (Raycast output data)
  3. Proximity to region (Overlap output data)
  4. Proximity to other objects (the TriggerCollisionManager add-on)
  5. Containment (the ContainerManager add-on)
  6. Composite objects (objects with affordances)
  7. Grasped objects

4. Visual Perception

  1. Overview
  2. Instance ID segmentation colors (_id pass)
  3. Semantic category segmentation colors (_category pass)
  4. Depth maps (_depth and _depth_simple passes)
  5. Motion perception (_flow pass)
  6. Other image passes (_mask, _normals, and _albedo passes)
  7. Occlusion output data

5. Camera Controls

  1. Overview
  2. Move a camera
  3. Rotate a camera
  4. Follow an object
  5. The CinematicCamera add-on

6. Photorealism

  1. Overview
  2. Lighting (HDRI skyboxes)
  3. Post-processing
  4. Interior lighting (the InteriorSceneLighting add-on)
  5. Depth of field

High-level API: tdw_image_dataset

7. Physics

Overview

7.1 Physics (PhysX)

  1. PhysX
  2. Object physics parameters
  3. Rigidbodies output data
  4. Collision output data
  5. Apply forces to objects
  6. Skip physics frames
  7. Disable physics

High-level API: tdw_physics

7.2 Physics (Obi)

  1. Obi
  2. Fluids
  3. Cloth
  4. ObiParticles output data
  5. Colliders and collision materials
  6. Solvers
  7. Obi and robots
  8. Obi and Flex

7.3 Physics (Flex)

  1. Flex
  2. Solid and soft actors
  3. Cloth actors
  4. Fluid and source actors
  5. Move, rotate, and scale Flex objects
  6. FlexParticles output data
  7. Apply forces to Flex objects
  8. Reset a Flex scene
  9. Other Flex commands

High-level API: tdw_physics

8. Audio

  1. Overview
  2. Initialize audio and play .wav files
  3. Resonance Audio
  4. PyImpact (dynamic impact sounds)
  5. Recording audio
  6. PyImpact (advanced API)
  7. Audio perception

9. Video Recording

  1. Overview
  2. Image-only video
  3. Video with audio

10. Agents

Overview

10.1 Robots

  1. Overview
  2. The Robot add-on
  3. Robot arm add-ons
  4. Robot collision detection
  5. Select a robot
  6. Add your own robots to TDW
  7. Robotics API (low-level)
  8. Add a camera to a robot

10.2 Magnebots

  1. Magnebot API (external repo)

10.3 Virtual Reality (VR)

  1. Overview
  2. Oculus Touch rigs

10.4 Keyboard and Mouse

  1. Overview
  2. Mouse input
  3. The FirstPersonAvatar
  4. Keyboard input

10.5 Embodied avatars

  1. The EmbodiedAvatar

11. Multi-agent simulations

  1. Overview
  2. Custom agent classes

12. Navigation

  1. Overview
  2. NavMesh pathfinding
  3. Occupancy maps

13. Non-physics objects

  1. Overview
  2. Position markers
  3. Line renderers
  4. Textured quads
  5. User Interface (the UI add-on)
  6. Compass rose
  7. Non-physics humanoids

14. Misc. remote server topics

  1. Launch a TDW build on a remote server from a personal computer
  2. Remote rendering with xpra

15. Misc. other topics

  1. C# source code
  2. Freezing your code
  3. Download asset bundles

API Documentation

Command API

tdw module API

tdw

tdw.add_ons

tdw.collision_data

tdw.container_data

tdw.flex_data

tdw.librarian

tdw.model_tests

tdw.obi_data

tdw.obi_data.cloth

tdw.obi_data.collision_materials

tdw.obi_data.fluids

tdw.object_data

tdw.object_data.composite_object

tdw.object_data.composite_object.sub_object

tdw.physics_audio

tdw.proc_gen.arrangements

tdw.proc_gen.arrangements.cabinetry

tdw.release

tdw.robot_data

tdw.scene_data

tdw.vr_data

Performance benchmarks

  1. Performance benchmarks
  2. Image capture
  3. Object data
  4. Command deserialization

tdw's People

Contributors

alters-mit avatar felixbinder avatar guhur avatar instbldrjems avatar jemsmit avatar laurenfarrow avatar lucasb-eyer avatar samarth4149 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.