[S-00-1] Project Planning about project-sandwich-man

[CUN-bjy]

Goal

stacking problem

• multi-stage task 수행
• multi-task(pick and place) 수행
• multi-goal task 수행
• (옵션) task를 주는 방식
  • (1) description (빨강, 파랑, 노랑) -> 순서로 쌓기
  • (2) 이미지를 통해 전달(object detection x) --> deep visual reasoning
• (옵션) object위치, 색깔 랜덤

(meta-learning??)

problem survey -> #2

[CUN-bjy]

Survey

Approach from TAMP

• http://web.mit.edu/caelan/www/presentations/6.881_TAMP.pdf
• https://www.ics.uci.edu/~kkask/Fall-2014%20CS271/slides/09-planning.pdf

Approach from Hierarchical RL

• https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch
• https://bluediary8.tistory.com/4
• https://arxiv.org/abs/1712.00948
• http://khanrc.github.io/RL-Hierarchy.html

Good Examples

• https://github.com/IanYangChina/pybullet_multigoal_gym

[CUN-bjy]

~8.20 까지 Survey 기간

[CUN-bjy]

Stage1. Single-step Task
Stage2. UOF, HAC, Multi-step Task(Hierarchical)
Stage3. Relational RL based -> goal shaping(Intrinsic Reward(Curiosity))

[CUN-bjy]

Development Plan

0. Survey(#2)
1. Single-step Stacking
   • Multi-goal problem -> sparse reward and hindsight concept
     • (DDPG+HER, SAC+HER, TQC+HER, etc.)
   • experiments for various environment
2. Multi-step Stacking
   • Hierarchical Structure
     • Option Framework(high-level/low-level policy) (UOF etc.)
     • Multi-level Sub-goal(HAC etc.)
3. Goal Representation Embedding
   • GAT(Graph ATtention network)-based relational goal representation
   • Intrinsic Motivated Goal Modular(CURIOUS etc.)
4. Further More
   • Vision-based Goal Embedding
   • Curiosity-based Goal Discovery

[CUN-bjy]

@benthebear93
See the plan and feedback pls.