wqynew / rl-based-navigation

Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization

Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization

This is the implementation of our RA-L paper arXiv, training and evaluation on AI2-THOR.

Implementation

Training

• The environment: Cuda 10.0, Python 3.6.4, PyTorch 1.0.1
• Please install AI2-THOR Version 2.1.0 and modify the "controller.py" of AI2-THOR as FILE.
• Our trained model can be downloaded from HERE. If you plan to train your own navigation model from scratch, some suggestions are provided:
  • Run the training model: python3 network.py
  • Please first run on 20 Kitchen scenes until it converges. Then increase the trining scenes to 40 (20 for kitchen, 20 for living room.) Finally, you can run on 80 training scenes.

Testing

• To evaluate our model, please run "python3 eve1_checkpoint.py"
• The files in './test/evaluation1' are for cross-scene evaluation; The files in './test/evaluation2' are for cross-target evaluation.
• The video can be downloaded from HERE.

Results

• Cross-scene generalization on AI2-THOR

Kitchen	Living room	Bedroom	Bathroom

• Transfer to the real world

* The navigation system takes as input data from four real-time camera sensors and a target image at each time step, to predict the optimal discrete navigation action. 
* The action command is converted to the wheel velocity and passed to the robot. 
* For example, the move right action is converted to rotate right at 45^\circ/s for 2s, move forward at 0.25m/s for 2s, and rotate left at 45^\circ/s for 2s. 
* These commands are published with a frequency of 5Hz. It is complex due to the movementdirection restrictions of the TurtleBot.

Contact

To ask questions or report issues please open an issue on the issues tracker.

Citation

If you use this work in your research, please cite the paper:

@article{rlnav2020,
  author    = {Qiaoyun Wu and
               Kai Xu and
               Jun Wang and
               Mingliang Xu and
               Dinesh Manocha},
  title     = {Reinforcement Learning based Visual Navigation with Information-Theoretic
               Regularization},
  year      = {2019},
  url       = {http://arxiv.org/abs/1912.04078},
  archivePrefix = {arXiv},
  eprint    = {1912.04078},
}

rl-based-navigation's Issues

Could you tell me how to deal with this problem?

When i run network.py,there will arise this problem.
!!!state_shape: (3, 300, 300)
FloorPlan211
FloorPlan11
FloorPlan312
FloorPlan411
FloorPlan312
FloorPlan211
Process Process-6:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/RL/multiprocessing_env.py", line 19, in worker
pim1, pim2, pim3,ob,g,shortest,pre_action= env.reset()
File "/home/RL/scene_loader.py", line 175, in reset
cidx=random.randint(0,n_points-1)
File "/usr/lib/python3.6/random.py", line 221, in randint
return self.randrange(a, b+1)
File "/usr/lib/python3.6/random.py", line 199, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,0, 0)
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/RL/multiprocessing_env.py", line 19, in worker
pim1, pim2, pim3,ob,g,shortest,pre_action= env.reset()
File "/home/RL/scene_loader.py", line 175, in reset
cidx=random.randint(0,n_points-1)
File "/usr/lib/python3.6/random.py", line 221, in randint
return self.randrange(a, b+1)
File "/usr/lib/python3.6/random.py", line 199, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,0, 0)
['Pillow', 'Laptop', 'Television', 'GarbageCan', 'Box', 'Bowl', 'Book', 'FloorLamp', 'Painting', 'Sofa']
['Pillow', 'Laptop', 'Television', 'GarbageCan', 'Box', 'Bowl', 'Book', 'FloorLamp', 'Painting', 'Sofa']
I run another eva1_checkpoint.py, some problems arised.
Loggingto/home/RL/.config/unity3d/Allen Institute for Artificial Intelligence/AI2-Thor/Player.log
test path: 1000
=> loaded checkpoint './checkpoint/best.ckpt' (global_t 90011)
FloorPlan22
['Toaster', 'Microwave', 'Fridge', 'CoffeeMaker', 'GarbageCan', 'Box', 'Bowl', 'Apple', 'Chair', 'DiningTable', 'Plate', 'Sink', 'SinkBasin']
Traceback (most recent call last):
File "/home/RL/eva1_checkpoint.py", line 297, in
slen=len(tenv.bc.shortest_plan(tenv.graph, start_agent,tenv.mygoal))
File "/home/RL/.local/lib/python3.6/site-packages/ai2thor/controller.py", line 1164, in shortest_plan
path=nx.shortest_path(graph,self.key_for_point(agent['position']),self.key_for_point(target['position']))
File "/home/RL/.local/lib/python3.6/site-packages/networkx/algorithms/shortest_paths/generic.py", line 170, in shortest_path
paths = nx.bidirectional_shortest_path(G, source, target)
File "/home/RL/.local/lib/python3.6/site-packages/networkx/algorithms/shortest_paths/unweighted.py", line 223, in bidirectional_shortest_path
raise nx.NodeNotFound(msg.format(source, target))
networkx.exception.NodeNotFound: Either source 0.750|2.250 or target -1.500|0.250 is not in G

Process finished with exit code 1

How to migrate the navigation algorithm to the real world, the Turtlebot. Should we use ROS?

Visual Navigation Code Requested

Dear Author, I would like to express my appreciation for the valuable work you have done. I am currently working on visual navigation and I was wondering if you could kindly provide the code for visualizing the results of your paper or other related papers.
Thank you very much for your time and consideration.

NodeNotFound: Either source -2.500|0.000 or target -1.500|0.750 is not in G

作者您好，在运行eval_checkpoint.py时，出现了这个错误？请问这个问题您有出现过吗？需要怎么解决？