council

Council: AI Agent Platform with Scalable Oversight

Introduction

Council is an innovative open-source platform designed to streamline the creation and deployment of customized generative AI applications. It operates through teams of agents built primarily in Python, with plans to incorporate Rust. This platform expands the capabilities of the Large Language Model (LLM) ecosystem by offering enhanced control and scalability for AI agents. Council enables users to design advanced agents that operate predictably by using a system of Controllers, Filters, Evaluators, and Budgets. This setup ensures efficient handling of tasks by automatically routing, comparing, evaluating, and selecting the best results for given tasks. Importantly, Council provides connectivity to numerous LLMs and integrates effectively with popular libraries such as LangChain.

Key Features

• Support for Advanced Agents: Council enables the creation of dependable agents capable of handling complex tasks, exploring different solutions, and evaluating task quality within budget constraints.
• Designed for Data Scientists: The platform is built using Python, supporting a local development environment with integration into popular frameworks.
• Efficient Production Deployments: Council offers easy packaging and deployment processes on various platforms, including Kubernetes, alongside robust monitoring capabilities.
• Ecosystem Integration: The platform is well-connected with a growing AI agent ecosystem and integrates with tools like LangChain and LlamaIndex, along with leading AI models.
• Scalable Oversight: Council comes equipped with tools for managing, versioning, monitoring, evaluating, and controlling deployed agents.

Key Concepts

• Agent: An agent handles the entire application logic, from initial input to the final output, through a series of Controllers, Evaluations, and Skills. Agents can nest within other agents, forming AgentChains.

• Controller: Controllers assess user intent and direct the initial prompt to registered Chains. They evaluate the output to decide on any further action, potentially revisiting Chains based on performance or budget limits. Controllers also manage the state across user sessions and are programmable in Python or Rust.

• Skill: Skills are services that receive input and provide an output, covering a wide array of tasks. Skills can interact with language models like OpenAI's GPT-4 or provide interfaces to knowledge bases and application-specific code.

• Chain: A Chain is a directed graph of Skills that facilitates execution and serves as an entry point for Controllers. Chains can also function as recursively nested agents.

• Evaluator: Evaluators are tasked with assessing the quality of Skills/Chains at runtime, ensuring only high-quality responses are delivered.

• Filter: Filters act to sieve responses returned to controllers to ensure pertinence and quality.

• State Management: This feature maintains context such as message history and intermediate results, aiding the continuity of agent sessions.

Quick Start Guide

To begin using Council, follow these simple steps for installation and setup:

1. Installation:

   • Recommended: Use pip to install via PyPI: pip install council-ai
   • Alternative: Clone the repository and install from a local copy.

2. Setup: Establish necessary API keys (e.g., for OpenAI) in a .env file, and refer to the provided example.

3. Usage:

   • Import the necessary modules.
   • Set up API keys for LLM access.
   • Create a basic skill and encapsulate it in a Chain.
   • Develop a controller for routing tasks to Chains.
   • Configure an Evaluator to assess task quality.
   • Finalize by assembling an Agent that coordinates these components.

4. Invoke an Agent with a simple prompt to see the system in action.

Community and Support

For any assistance, users can raise issues on GitHub or connect with the development team and community through the Discord channel provided. Council encourages contributions and welcomes feedback, feature requests, and participation in the community.

Documentation

Comprehensive documentation can be found at council.dev. Users are encouraged to explore the documentation to fully leverage Council's capabilities.

Council represents a robust solution for developing scalable and controlled AI applications, offering data scientists and developers a user-friendly environment to innovate and deploy AI solutions efficiently.