Lemur: Open Foundation Models for Language Agents
Introduction
Lemur is an innovative open-source language model designed to seamlessly integrate natural language processing and coding abilities, aiming to enhance the capabilities of language agents. As language models evolve beyond simple conversational tasks to becoming functional agents in the real world, a strong combination of language understanding and action execution is required. Lemur is tailored to meet these needs by offering a balanced approach between text and code.
Key Features of Lemur
-
Open Accessibility: Lemur is openly accessible, allowing developers and researchers to utilize and build upon its capabilities. The models and related resources are available on platforms like HuggingFace, enabling easy integration and usage.
-
Balanced Capabilities: Unlike many existing models that focus solely on either language or coding, Lemur combines both strengths. The model has been pretrained on a massive dataset with a significant coding component, facilitating sophisticated instruction following and task reasoning.
-
Innovative Training Approach: Lemur employs a two-stage training process:
- Pretraining is performed on a 90 billion token dataset with a significant code-to-text ratio to create the Lemur-70B-v1 model.
- Further instruction tuning on a set of 300,000 examples results in Lemur-70B-Chat-v1, enhancing its conversational capabilities.
Latest Developments
The Lemur project has been actively evolving, with several updates including:
- Open-Source Code for OpenAgents: Launched in October 2023, this platform facilitates the creation and deployment of language agents.
- Research Papers and Codebase: Released in October 2023, these resources provide a detailed look into Lemur's workings and contributions.
- Model Versions: The release of the
lemur-70b-v1andlemur-70b-chat-v1models, available on the HuggingFace Hub for anyone to explore and use.
Models
Lemur offers two primary models:
- Lemur-70B: The foundational model supporting various text and code generation tasks.
- Lemur-70B-Chat: An enhanced version with refined conversational abilities, capable of interacting more naturally and effectively with users.
Why Choose Lemur?
Lemur's dual focus on language and coding provides an edge over other models, making it a top choice for researchers looking to develop versatile language agents. Its impressive performance in diverse benchmarks demonstrates its strength, bridging the gap between open-source and commercial solutions.
Quickstart Guide
To get started with Lemur, users need to set up their working environment by installing necessary libraries and the xchat package. The models can be utilized for both text and code generation, providing flexibility for different application scenarios.
Evaluation Strategy
Lemur's capabilities are thoroughly evaluated using language and code datasets, ensuring balanced performance across various tasks. Additionally, its interactive agent skills are tested in scenarios requiring tool usage, adaptability, and environmental interaction.
Training and Onward Collaboration
Lemur represents a collaborative effort between XLang Lab and Salesforce Research, with contributions from organizations like Google Research and Amazon AWS. This joint initiative not only enhances Lemur's development but also promotes further research and innovation in language models.
Learning and Contribution
For individuals interested in exploring Lemur further or contributing to its development, detailed documentation, training scripts, and evaluation tools are available. As an open project, Lemur welcomes community engagement and seeks to foster a collaborative environment for research advancements.
Embrace Lemur's pioneering capabilities and explore the future of language agents through this comprehensive and cutting-edge language model.