zju-vipa / odyssey

ODYSSEY: Empowering Agents with Open-World Skills

Odyssey

Directory Description

1. LLM-Backend

   Code to deploy LLM backend.

2. MC-Crawler

   Crawling Minecraft game information from Minecraft Wiki, and store data in markdown format.

3. MineMA-Model-Fine-Tuning

   Code to fine-tune the LLaMa model and genearte training and test datasets.

4. Odyssey

   Code for Minecaft agents based on a large language model and skill base.

Odyssey Installation

We use Python ≥ 3.9 and Node.js ≥ 16.13.0. We have tested on Ubuntu 20.04, Windows 10, and macOS.

Python Install

pip install -e .
pip install -r requirements.txt

Node.js Install

npm install -g yarn
cd odyssey/env/mineflayer
yarn install
cd odyssey/env/mineflayer/mineflayer-collectblock
npx tsc
cd odyssey/env/mineflayer
yarn install
cd odyssey/env/mineflayer/node_modules/mineflayer-collectblock
npx tsc

Minecraft Server

You can deploy a Minecraft server using docker. See here.

Embedding Model

1. Need to install git-lfs first.

2. Download mebedding model repository

   git lfs install
   git clone https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2.git

3. The directory where you clone the repository is then used to set embedding_dir.

Config

You need to create config.json according to the format of conf/config.json.keep.this in conf directory.

• server_host: LLaMa backend server ip.
• server_port: LLaMa backend server port.
• NODE_SERVER_PORT: Node service port.

Odyssey Tasks

Subgoal

def test_subgoal():
    odyssey_l3_8b = Odyssey(
        mc_port=mc_port,
        mc_host=mc_host,
        env_wait_ticks=env_wait_ticks,
        skill_library_dir="./skill_library",
        reload=True, # set to True if the skill_json updated
        embedding_dir=embedding_dir, # your model path
        environment='subgoal',
        resume=False,
        server_port=node_port,
        critic_agent_model_name = ModelType.LLAMA3_8B_V3,
        comment_agent_model_name = ModelType.LLAMA3_8B_V3,
        curriculum_agent_qa_model_name = ModelType.LLAMA3_8B_V3,
        curriculum_agent_model_name = ModelType.LLAMA3_8B_V3,
        action_agent_model_name = ModelType.LLAMA3_8B_V3,
    )
    # 5 classic MC tasks
    test_sub_goals = ["craft crafting table", "craft wooden pickaxe", "craft stone pickaxe", "craft iron pickaxe", "mine diamond"]
    try:
        odyssey_l3_8b.inference_sub_goal(task="subgoal_llama3_8b_v3", sub_goals=test_sub_goals)
    except Exception as e:
        print(e)

Model	For what
action_agent_model_name	Choose one of the k retrieved skills to execute
curriculum_agent_model_name	Propose tasks for farming and explore
curriculum_agent_qa_model_name	Schedule subtasks for combat, generate QA context, and rank the order to kill monsters
critic_agent_model_name	Action critic
comment_agent_model_name	Give the critic about the last combat result, in order to reschedule subtasks for combat

Long-term Planning Task

def test_combat():
    odyssey_l3_70b = Odyssey(
        mc_port=mc_port,
        mc_host=mc_host,
        env_wait_ticks=env_wait_ticks,
        skill_library_dir="./skill_library",
        reload=True, # set to True if the skill_json updated
        embedding_dir=embedding_dir, # your model path
        environment='combat',
        resume=False,
        server_port=node_port,
        critic_agent_model_name = ModelType.LLAMA3_70B_V1,
        comment_agent_model_name = ModelType.LLAMA3_70B_V1,
        curriculum_agent_qa_model_name = ModelType.LLAMA3_70B_V1,
        curriculum_agent_model_name = ModelType.LLAMA3_70B_V1,
        action_agent_model_name = ModelType.LLAMA3_70B_V1,
    )
    
    multi_rounds_tasks = ["1 enderman", "3 zombie"]
    l70_v1_combat_benchmark = [
                        # Single-mob tasks
                         "1 skeleton",  "1 spider", "1 zombified_piglin", "1 zombie",
                        # Multi-mob tasks
                        "1 zombie, 1 skeleton", "1 zombie, 1 spider", "1 zombie, 1 skeleton, 1 spider"
                        ]
    for task in l70_v1_combat_benchmark:
        odyssey_l3_70b.inference(task=task, reset_env=False, feedback_rounds=1)
    for task in multi_rounds_tasks:
        odyssey_l3_70b.inference(task=task, reset_env=False, feedback_rounds=3)

Dynamic-Immediate Planning Task

def test_farming():
    odyssey_l3_8b = Odyssey(
        mc_port=mc_port,
        mc_host=mc_host,
        env_wait_ticks=env_wait_ticks,
        skill_library_dir="./skill_library",
        reload=True, # set to True if the skill_json updated
        embedding_dir=embedding_dir, # your model path
        environment='farming',
        resume=False,
        server_port=node_port,
        critic_agent_model_name = ModelType.LLAMA3_8B_V3,
        comment_agent_model_name = ModelType.LLAMA3_8B_V3,
        curriculum_agent_qa_model_name = ModelType.LLAMA3_8B_V3,
        curriculum_agent_model_name = ModelType.LLAMA3_8B_V3,
        action_agent_model_name = ModelType.LLAMA3_8B_V3,
    )

    farming_benchmark = [
                    # Single-goal tasks
                    "collect 1 wool by shearing 1 sheep",
                    "collect 1 bucket of milk",
                    "cook 1 meat (beef or mutton or pork or chicken)",
                    # Multi-goal tasks
                    "collect and plant 1 seed (wheat or melon or pumpkin)"
                    ]
   	for goal in farming_benchmark:
	      odyssey_l3_8b.learn(goals=goal, reset_env=False)

Autonomous Exploration Task

def explore():
    odyssey_l3_8b = Odyssey(
        mc_port=mc_port,
        mc_host=mc_host,
        env_wait_ticks=env_wait_ticks,
        skill_library_dir="./skill_library",
        reload=True, # set to True if the skill_json updated
        embedding_dir=embedding_dir, # your model path
        environment='explore',
        resume=False,
        server_port=node_port,
        critic_agent_model_name = ModelType.LLAMA3_8B,
        comment_agent_model_name = ModelType.LLAMA3_8B,
        curriculum_agent_qa_model_name = ModelType.LLAMA3_8B,
        curriculum_agent_model_name = ModelType.LLAMA3_8B,
        action_agent_model_name = ModelType.LLAMA3_8B,
        username='bot1_8b'
    )
    odyssey_l3_8b.learn()

TODOs

FAQ

1. LLaMa api application LLaMa2大语言模型有哪些API接口_模型服务灵积(DashScope)-阿里云帮助中心 (aliyun.com)

Related Works

1. MineRL: A Large-Scale Dataset of Minecraft Demonstrations William H. Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, Ruslan Salakhutdinov IJCAI 2019. paper
2. Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, Jeff Clune arXiv 2022. paper
3. MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Anima Anandkumar NeurIPS 2022. paper
4. Creative Agents: Empowering Agents with Imagination for Creative Tasks Chi Zhang, Penglin Cai, Yuhui Fu, Haoqi Yuan, Zongqing Lu arXiv 2023. paper
5. GROOT: Learning to Follow Instructions by Watching Gameplay Videos Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang arXiv 2023. paper
6. Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, Jifeng Dai arXiv 2023. paper
7. JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, Xiaojian Ma, Yitao Liang arXiv 2023. paper
8. LLaMA Rider: Spurring Large Language Models to Explore the Open World Yicheng Feng, Yuxuan Wang, Jiazheng Liu, Sipeng Zheng, Zongqing Lu arXiv 2023. paper
9. MCU: A Task-centric Framework for Open-ended Agent Evaluation in Minecraft Haowei Lin, Zihao Wang, Jianzhu Ma, Yitao Liang arXiv 2023. paper
10. See and Think: Embodied Agent in Virtual Environment Zhonghan Zhao, Wenhao Chai, Xuan Wang, Li Boyi, Shengyu Hao, Shidong Cao, Tian Ye, Jenq-Neng Hwang, Gaoang Wang arXiv 2023. paper
11. Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar arXiv 2023. paper
12. Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang CVPR 2023. paper
13. Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, Yitao Liang NeurIPS 2023. paper
14. STEVE-1: A Generative Model for Text-to-Behavior in Minecraft Shalev Lifshitz, Keiran Paster, Harris Chan, Jimmy Ba, Sheila McIlraith NeurIPS 2023. paper
15. Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks Haoqi Yuan, Chi Zhang, Hongcheng Wang, Feiyang Xie, Penglin Cai, Hao Dong, Zongqing Lu NeurIPSW 2023. paper
16. A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges Xinrun Xu, Yuxin Wang, Chaoyi Xu, Ziluo Ding, Jiechuan Jiang, Zhiming Ding, Börje F. Karlsson arXiv 2024. paper
17. Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, Jifeng Dai arXiv 2024. paper
18. MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception Yiran Qin, Enshen Zhou, Qichang Liu, Zhenfei Yin, Lu Sheng, Ruimao Zhang, Yu Qiao, Jing Shao arXiv 2024. paper