TurboPilot: An Overview
Introduction
TurboPilot was a self-hosted alternative to Copilot, inspired by the Fauxpilot project. This tool used the library behind Llama.cpp to operate the 6 Billion Parameter Salesforce Codegen model in just 4GiB of RAM. Although TurboPilot offered unique features and flexibility, it has been deprecated as of September 30, 2023, due to the availability of more mature solutions that better meet community needs.
Key Features
- StableCode Support: TurboPilot supported StableCode 3B Instruct models, allowing users to employ advanced quantized GGML models for improved performance.
- Model Variety and Flexibility: Users could leverage different models like Wizardcoder, Starcoder, and Santacoder for state-of-the-art code completion and support for multiple programming languages.
- Ease of Use and Access: TurboPilot was designed to be easy to extend and modify, allowing users to add new models quite seamlessly.
Contributing to TurboPilot
Contributions to TurboPilot were encouraged. The process was straightforward: users could fork the repository, make changes, and submit a pull request (PR) for review. The project maintained an open-source ethos, welcoming improvements and new ideas from the community.
Getting Started with TurboPilot
For those interested in exploring TurboPilot, it was recommended to obtain pre-processed models and use Docker to run the server easily.
Obtaining Models
There were two main options:
- Direct Download: Quick and straightforward, users could download pre-converted, pre-quantized models suitable for various system capabilities.
- Self-Conversion: A more flexible approach where users could follow a detailed guide to convert and quantize models themselves.
Running the TurboPilot Server
With the model in hand, users would download the latest binary or build their own and execute the server command. The application would initiate on a default port, ready to integrate with tools like VSCode for seamless code completion.
Docker Deployment
TurboPilot could also be executed using a Docker image. This method was particularly beneficial for setups involving CUDA, where the application could leverage GPU capabilities for faster processing.
Interacting with TurboPilot
The API designed for TurboPilot allowed for interoperability with various plugins, like the VSCode fauxpilot plugin. By configuring settings appropriately, users could integrate TurboPilot's powerful code completion features into their development environments.
Known Limitations
TurboPilot had its constraints, such as support for only one GPU at a time. Users weighing the pros and cons would have to consider these limitations against their specific needs.
Acknowledgements
The development of TurboPilot relied heavily on prior work and open-source contributions from various individuals and projects, including:
- Llama.cpp and GGML library by Georgi Gerganov
- The Fauxpilot project
- The Salesforce Codegen models
- Contributions by Moyix and the use of CrowCPP for the server infrastructure
TurboPilot was a testament to the power of community-driven development and collaborative innovation.
Despite its discontinuation, TurboPilot paved the way for better tools and exemplified the rapid evolution of AI-driven coding assistance technologies.