Odyssey: Empowering Minecraft Agents with Open-World Skills
Overview
Odyssey is an innovative project designed to enhance Minecraft agents by equipping them with advanced open-world skills, allowing them to navigate and interact within the expansive virtual world of Minecraft. Unlike other approaches that focus mainly on basic tasks like material collection and crafting following a predetermined tech-tree, Odyssey extends the capabilities of agents to explore and leverage diverse gameplay strategies. This expansion is achieved through a robust framework that integrates Large Language Models (LLM) with a comprehensive skill library and refined benchmarking tasks.
Main Components of Odyssey
The Odyssey framework includes several primary components:
-
Interactive Agent with a Skill Library:
- Odyssey introduces a highly interactive agent powered by a skill library of 40 primitive skills and 183 compositional skills. This diverse library enables agents to perform a wide range of actions and combinations, enhancing their adaptability and strategic thinking in the Minecraft world.
-
Fine-Tuned LLaMA-3 Model:
- The project utilizes a refined version of the LLaMA-3 model, trained on an extensive dataset comprising more than 390,000 question-answer pairs sourced from the Minecraft Wiki. This model empowers agents with contextual knowledge and the ability to execute complex tasks based on textual instructions.
-
Benchmark for Agent Capabilities:
- Odyssey establishes a new benchmark to evaluate different aspects of agent capabilities. This includes assessing agents on their long-term planning, dynamic-immediate planning, and autonomous exploration tasks. These benchmarks ensure a comprehensive understanding of the agents’ performance in an open-world environment.
Recent Developments
Odyssey is continuously evolving, with recent updates including more comprehensive test scenarios and comparisons with other open-source models. The project has also released additional resources such as a web crawler program and a comprehensive skill library, aiding researchers in furthering their exploration and enhancement of Minecraft agents.
Demonstrations
Odyssey showcases various capabilities through demonstration videos. These videos highlight tasks like mining diamonds from scratch, crafting tools and weapons, farming activities like shearing sheep and milking cows, and autonomous exploration. These demonstrations provide a tangible representation of the agents' abilities when interacting with the Minecraft environment.
Installation and Configuration
Odyssey supports installation across various platforms using Python (≥3.9) and Node.js (≥16.13.0). The setup process involves installing the necessary dependencies, handling Minecraft server deployment, and configuring the embedding models required for the agents to function effectively.
Odyssey Tasks
The project highlights several tasks for agents:
- Subgoal Tasks: Focus on accomplishing specific goals within the game by leveraging the agent’s skill set.
- Long-term Planning Tasks: Involve strategic planning over extended periods to achieve complex objectives.
- Dynamic-Immediate Planning Tasks: Require quick decision-making based on changing conditions or immediate needs.
- Autonomous Exploration Tasks: Emphasize the agent's ability to independently explore and interact with the environment without direct guidance.
Related Works and Future Research
Odyssey builds upon a rich body of research in the field of game-based AI and agents in virtual environments. It connects with prior studies and frameworks, offering a foundation for future advancements and more sophisticated autonomous agent solutions.
Licensing and Contact
Odyssey is open-source, licensed under the MIT License, and actively encourages collaboration and further research. For more information or to connect with the development team, researchers are encouraged to reach out to the VIPA Lab at Zhejiang University. Their aim is to foster a community of innovation and exploration in the world of AI and virtual environments.
By providing a robust framework and showcasing the potential of LLMs in open-world scenarios, Odyssey promises to pave the way for more advanced and capable agents in virtual environments. Whether for academic research or practical applications, Odyssey offers an exciting frontier in the exploration of artificial intelligence within the realms of Minecraft.