aici
AICI provides a framework for developers to create versatile and secure controllers to efficiently manage and refine the output of large language models (LLMs). It simplifies the development of controllers, offering compatibility with diverse inference engines like llama.cpp and HuggingFace Transformers. These controllers are implemented as WebAssembly modules that run concurrently with the LLM engine, ensuring minimal overhead and enhanced performance. Experience flexible experimentation and tailored solutions with upcoming multi-tenant deployments prioritizing security and efficiency.
