GithubHelp home page GithubHelp logo

unity-dl / home-platform Goto Github PK

View Code? Open in Web Editor NEW

This project forked from ml-lab/home-platform

0.0 2.0 0.0 8.13 MB

HoME: a Household Multimodal Environment is a platform for artificial agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context.

Home Page: https://home-platform.github.io/

License: BSD 3-Clause "New" or "Revised" License

Python 96.33% Shell 1.56% PLSQL 2.11%

home-platform's Introduction

HoME Platform Build Status

HoME is a platform for artificial agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context.

Check out the paper on Arxiv for more details: HoME: a Household Multimodal Environment

alt tag

Dependencies

Main requirements:

  • Python 2.7+ with Numpy, Scipy and Matplotlib
  • (or Python 3.5+ but currently WIP, might have bugs)
  • Panda3d game engine for 3D rendering
  • EVERT engine for 3D acoustic ray-tracing
  • PySoundFile for Ogg Vorbis decoding

To install dependencies on Ubuntu operating systems:

sudo apt-get install python-pip python-dev build-essential libsndfile1
sudo pip2 install --upgrade pip numpy scipy matplotlib panda3d pysoundfile resampy nose coverage Pillow

(Packages nose and coverage are for tests only and can be omitted)

Finally you have to install EVERT. In order to do so, please follow the instructions over at https://github.com/sbrodeur/evert

SUNCG Dataset

The Home environment is based on the SUNCG dataset.

Important! Before you can use this library, you need to obtain the SUNCG dataset. In order to do so, please follow the instructions on their website.

For the test suit we included a single small house as sample in this repository.

Installing the library

Download the source code from the git repository:

mkdir -p $HOME/work
cd $HOME/work
git clone https://github.com/HoME-Platform/home-platform.git

Note that the library must be in the PYTHONPATH environment variable for Python to be able to find it:

export PYTHONPATH=$HOME/work/home-platform:$PYTHONPATH 

This can also be added at the end of the configuration file $HOME/.bashrc

Running unit tests

To ensure all libraries where correctly installed, it is advised to run the test suite:

cd $HOME/work/home-platform/tests
./run_tests.sh

Note that this can take some time.

Usage

Before you start please read steps 0 and 1 so that you know what to expect. The data installation and preparation can take some time (because it's a big dataset).

0 Obtain and install the SUNCG dataset (only once)

The dataset can be acquired by contacting the authors at their website: http://suncg.cs.princeton.edu/

Currently the dataset downloaded consists of a single zip file, which in turn contains multiple zip files (one for each directory in [house, object, object_vox, room, texture]).

Please unzip all files so that as a result you have the folloging directory structure somewhere on your PC:

/some/path/doesnt/matter/SUNCG/
/some/path/doesnt/matter/SUNCG/house/
/some/path/doesnt/matter/SUNCG/house/0004d52d1aeeb8ae6de39d6bd993e992/
/some/path/doesnt/matter/SUNCG/house/0004dd3cb11e50530676f77b55262d38/
...
/some/path/doesnt/matter/SUNCG/object/100/
/some/path/doesnt/matter/SUNCG/object/101/
...

The unzipped files take up approx 28.1GB disk space on an NTFS-formatted drive.

Warning Unzipping may take considerable time and should not be done on a network drive due to the overhead in network communication. We advise to extract everything on a local machine and if necessary copy it to networked machines via rsync like so:

rsync -avh --info=progress2 --remove-source-files /some/path/doesnt/matter/SUNCG /cluster/datasets/

Once everything is unzipped, you can remove the original zip files.

As final step please "install" the dataset by (a) symlinking it into your home directory

ln -s /some/path/doesnt/matter/SUNCG/ ~/.suncg 

or by (b) setting the environmental variable to the right path:

export SUNCG_DATA_DIR="/cluster/datasets/SUNCG"

If you do both of these things, the environmental variable takes precedent.

1 Convert the houses into a usable format (only once)

In order for Panda3D to be able to read the 3D files of the houses, you need to first convert them to a different format (from OBJ/Wavefront to EGG and BAM).

To do so, just cd into the scripts folder in this repository and run the conversion script:

# assuming you currently are in the directory of this readme file:
cd scripts

# if you installed the dataset via symlink:
./convert_suncg.sh ~/.suncg/ # the trailing slash is important

# if you installed the dataset via environmental vairable:
./convert_suncg.sh $SUNCG_DATA_DIR/ # the trailing slash might be important

Now sit back, have a tea, a dinner, maybe a little holiday somewhere warm - this will take a while.

2 Loading a house

TODO

(In the meantime please have a look at our tests in the tests directory. There we cover pretty much all aspects of the library.)

3 Rendering a house

TODO

(In the meantime please have a look at our tests in the tests directory. There we cover pretty much all aspects of the library.)

4 Adding physics to a house

TODO

(... you get the idea.)

5 Adding acoustics to a hosue

TODO

(... you get the idea.)

6 Getting started with reinforcement learning (RL)

TODO

(... you get the idea.)

home-platform's People

Contributors

ankeshanand avatar fgolemo avatar sbrodeur avatar vmichals avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.