GithubHelp home page GithubHelp logo

ziadalh / zero_experience_required Goto Github PK

View Code? Open in Web Editor NEW
26.0 26.0 2.0 859 KB

Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation. CVPR 2022

Home Page: https://vision.cs.utexas.edu/projects/zsel/

License: MIT License

Python 100.00%
imagenav objectnav roomnav visual-navigation zero-shot-experience-learning

zero_experience_required's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

zero_experience_required's Issues

bounding box

Is there a method to find bounding box in habit sim?

Miss sementic dataset for gibson objectnav: data/scene_datasets/gibson_semantic

Hi! When I wanted to run the zer/objectnav dataset , it occurred an error as followed:

File "anaconda3/envs/zsvlnhl/lib/python3.8/site-packages/habitat_sim/simulator.py", line 200, in _config_backend
super().init(config.sim_cfg, config.metadata_mediator)
AssertionError: ESP_CHECK failed: Missing (at least) one of scene dataset attributes, stage attributes, or dataset scene attributes for scene 'data/scene_datasets/gibson_semantic/Markleeville.glb'. Likely an invalid scene name.

But I cannot find the dataset "data/scene_datasets/gibson_semantic" . Can you please inform me the link of the dataset ? Thank you very much!

some config:

habitat:
environment:
type: "gibson"
max_episode_steps: 100

simulator:
turn_angle: 30
tilt_angle: 30
action_space_config: "v1"
agent_0:
sensors: ['rgb_sensor']
height: 0.88
radius: 0.18
habitat_sim_v0:
gpu_device_id: 0
allow_sliding: False
rgb_sensor:
width: 128
height: 128
hfov: 79
position: [0, 0.88, 0]

task:
type: ObjectNav-v1
end_on_success: True
reward_measure: "distance_to_goal_reward"
success_measure: "spl"

possible_actions: ["stop", "move_forward", "turn_left", "turn_right"]

sensors: ['OBJECTgoaltext_sensor', 'compass_sensor', 'gps_sensor']
goal_sensor_uuid: objectgoal

measurements: ['distance_to_goal', 'success', 'spl', 'soft_spl', 'distance_to_goal_reward']

distance_to_goal:
  distance_to: VIEW_POINTS
success:
  success_distance: 0.1

dataset:
type: ObjectNav-v1
split: train
data_path: "data/datasets/zer/objectnav/gibson/v1/{split}/{split}.json.gz"
scenes_dir: "data/scene_datasets/"

habitat_baselines:
base_task_config_path: "exp_config/base_task_config/objectnav_gibson.yaml"
cmd_trailing_opts: ["habitat.environment.iterator_options.max_scene_repeat_steps", "50000"]
simulator_gpu_id: 0
torch_gpu_id: 0
video_option: []
tensorboard_dir: "tb"
video_dir: "video_dir"
test_episode_count: -1
eval_ckpt_path_dir: "data/new_checkpoints"
num_environments: 4
sensors: ["rgb_sensor"]
num_updates: 270000
log_interval: 10
num_checkpoints: 100

Force PyTorch to be single threaded as

this improves performance considerably

force_torch_single_threaded: True

eval:
use_ckpt_config: False
split: val

rl:
policy:
name: "NavNetPolicy"

ppo:
  # para for policy network
  backbone: fast_resnet9
  goal_backbone: clip_text
  rnn_type: GRU
  num_recurrent_layers: 2

  num_steps: 128

  hidden_size: 128
  input_size: 128
  visual_encoder_embedding_size: 512
  goal_embedding_size: 128

  visual_obs_inputs: ['rgb', "imagegoal_sensor_v2"]

  random_crop: False
  rgb_color_jitter: 0.0
  tie_inputs_and_goal_param: False

  task_type_embed: False
  task_type_embed_size: 64

habitat:
simulator:
turn_angle: 30

error in habitat-sim

Hi,
when I run python setup.py install --headless --with-cuda to install habitat-sim, it returns error:

th-cuda
running install
running bdist_egg
running egg_info
writing habitat_sim.egg-info/PKG-INFO
writing dependency_links to habitat_sim.egg-info/dependency_links.txt
writing requirements to habitat_sim.egg-info/requires.txt
writing top-level names to habitat_sim.egg-info/top_level.txt
reading manifest file 'habitat_sim.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'habitat_sim.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
error: [Errno 2] No such file or directory: 'build/compile_commands.json'

Reproduce ImageNav results for Gibson Split A

Could you share the config file and script to reproduce the ImageNav results for Gibson Split A.

I trained the agent using eval_ppo_imagenav_rgb.yaml (which I'm sure was not intended to be used for training) and got the results:

Diff SPL Success
Easy 16.43 18.86
Medium 2.03 2.07
Hard 0.00 0.00

I can see some discrepancies between the config and the paper (such as not using DDPPO and training hyperparameters).

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.