Artificial Intelligence Controller Interface (AICI)
The Artificial Intelligence Controller Interface (AICI) is an innovative platform that allows developers to create Controllers to manage and direct the output of a Large Language Model (LLM) in real time. It acts like a guide for the AI, ensuring it generates responses under specific rules and constraints.
What is AICI?
AICI is designed to help build Controllers—computer programs that can limit and shape the AI's response. These programs can do things like constrained decoding, meaning they can control which words or phrases the AI can choose next. They can also dynamically edit the prompts given to the AI and coordinate multiple outputs at the same time.
How Does AICI Work?
Controllers use custom logic to decode and maintain a state during an LLM request. This means that it can handle various strategies, whether programming a specific response, managing dialogues with AI agents, or ensuring efficient communication with the AI itself.
Purpose and Advantages
The goal of AICI is to simplify and enhance the development of Controllers, making it easier to create fast and compatible ones across different LLM engines. By handling the complex details of AI processing, AICI enables developers and researchers to focus on experimenting and improving AI outputs.
Flexibility and Security
- Flexibility: Developers can write Controllers in any language that can compile to WebAssembly (Wasm), such as Rust, C, or C++, or even interpreted languages like Python and JavaScript.
- Security: AICI's Controllers run in a sandboxed environment, which means they cannot access the file system, network, or any other external resources, ensuring the system's security.
Performance
The system is designed to be fast, with Wasm modules running in parallel with the LLM inference engine. This means that the processing adds minimal delay to the generation process, making it efficient and suitable for real-time applications.
Practical Implementation
AICI can run locally or in the cloud and intends to support multi-tenant deployments. By using light-weight Wasm modules, it efficiently works alongside the LLM processing, utilizing CPU resources while the GPU focuses on token generation.
Integrations
Currently, AICI integrates with various LLM engines such as llama.cpp, HuggingFace Transformers, and rLLM, with plans for vLLM integration in the future.
Building and Using AICI
To start using AICI, developers need to:
- Set up a development environment, primarily using Rust and Python.
- Compile AICI components and build a Controller using provided examples.
- Deploy the Controller and utilize it to direct AI output, customizing AI responses with specific rules.
Conclusion
Developed by Microsoft Research, AICI is a forward-thinking tool that empowers developers to take control of AI outputs. It abstracts the complexities of AI processing, allowing for seamless experimentation and integration across various applications, ultimately enhancing the way we interact with AI technologies.