Introduction to Basaran
Overview
Basaran is an open-source alternative to the widely used OpenAI text completion API. You may ask, "Why would I need an alternative?" Well, Basaran is built to offer flexibility and compatibility with Hugging Face Transformers, a popular community-driven method for machine learning models, especially those dealing with generating text. With Basaran, users can replace OpenAI’s service seamlessly, using the latest open-source models without altering their existing codebase.
Key Features
Basaran comes packed with several features that make it a robust tool for developers focusing on text generation:
- Streaming Generation: Basaran supports a variety of decoding strategies to stream text, making it a powerful option for real-time applications.
- Model Support: It is compatible with both decoder-only and encoder-decoder models, widening its range of potential use cases.
- Efficient Detokenizer: Handles text complexities like surrogates and spaces efficiently, ensuring smooth text handling.
- Hardware Utilization: Supports multi-GPU usage and offers optional model quantization for improved performance.
- Real-time Updates: Implements real-time partial progress updates using server-sent events, giving users immediate feedback.
- OpenAI Compatibility: Its API and client libraries are designed to mimic the OpenAI API’s look and feel, assisting developers in easy integration.
- Interactive Playground: Offers a web-based interactive playground for testing and experimentation.
Getting Started
Quick Setup
To quickly get started with Basaran, replace user/repo with your selected text generation model from Hugging Face and X.Y.Z with the latest version, then execute a simple Docker command. This sets up both the API and an interactive playground for Basaran locally.
Installation Options
Basaran can be deployed using Docker, recommended for its simplicity and efficiency. Alternatively, it is possible to install Basaran using Python's package manager, pip, suitable for users who may prefer Python environments or wish to run it directly from the source for customization.
- With Docker: Besides quick setup, Docker supports GPU acceleration with the installation of NVIDIA’s driver and runtime, which are essential for large-scale model training.
- With pip: Install Basaran with pip and optionally install dependencies for GPU acceleration. This method allows the use of Python environments.
- From Source: Clone the repository from GitHub for access to the latest features or for development purposes.
Using Basaran
Users can interact with Basaran’s API in a manner consistent with the OpenAI API, making it easy for those familiar with OpenAI to transition. Input and response formats, like a typical text completion request using curl, are straightforward.
Developers can also use Basaran as a Python library, integrating it directly into Python applications by installing it via pip and utilizing the model loading functionalities.
Compatibility and Roadmap
The API maintains full consistency with OpenAI, with minor differences in parameter support. Basaran can handle only one model per process, ensuring the API always outputs using the same model.
Future endeavors include expanding chat functionalities and refining model architectures, decoding strategies, and other features to improve model interactions.
Contribution and Community
Basaran is an open-source project, inviting contributions from the community to develop and enhance its capabilities. Ideas, questions, or issues can be discussed via the project's issue tracker on GitHub. An MIT License covers its usage, allowing flexibility for both personal and commercial usage.
As the project continues to grow, its robust feature set and open-source nature position Basaran as an invaluable tool for developers seeking an alternative to proprietary text completion APIs.