Introduction to Self-Hosted AI
Self-Hosted AI is an innovative project designed for individuals and organizations seeking to utilize artificial intelligence (AI) technologies on their own terms. Developed with an API interface compatible with OpenAI, this project allows various open-source initiatives to seamlessly integrate with it. The project's architecture is designed to encourage everyone to develop compatible API interfaces, thus fostering the growth of related applications and enhancing the robustness of the AI ecosystem.
Getting Started
To dive into the Self-Hosted AI experience, users can download the necessary packaged files from the release page. Updating the system is straightforward using the update.bat script, which updates everything except the models. For those who need to make their API accessible over the internet, setting the ngrok_token environment variable will handle that aspect.
Usage Example
A practical way to explore Self-Hosted AI is by visiting ChatGPTBox and switching to the compatible API mode for a quick trial. This method enables users to experience the capabilities of the project instantly.
Chat Models
Self-Hosted AI supports several popular chat models, each serving different needs:
-
RWKV
- Hosted on GitHub, RWKV can be installed online with an 8MB installer. More information is available via the RWKV-Runner.
-
ChatGLM 6B Int4
- For offline usage, users can download a package that includes this model. The package is based on ChatGLM and open-sourced under the Apache-2.0 license, including a streamlined version of a model available on Huggingface.
- The self-hosted variant can be accessed on Huggingface for a more interactive experience. Users can easily copy the interface for their use.
- When using with ChatGPTBox, make sure to set the model name to
chatglm-6b-int4.
-
llama.cpp
- Another compelling model included is based on llama.cpp, supporting unicode input on Windows. Users are instructed to run
get_llama_model.shto obtain these models, particularly in a WSL or Linux environment. - Model name setting for ChatGPTBox is
llama-7b-int4.
- Another compelling model included is based on llama.cpp, supporting unicode input on Windows. Users are instructed to run
Painting Capabilities
Self-Hosted AI also comprises capabilities for artistic endeavors through the stable-diffusion-webui component:
- It offers a one-click installer for quick setup. The installer and installation instructions for other systems are available on its GitHub repository.
- After downloading, users need to execute
update.batfollowed byrun.baton Windows. For future use, simply runningrun.batsuffices. - Users can import models from sites like Civitai into specific directories within the system to leverage different creative models.
Integrated Applications
Incorporating Self-Hosted AI's diverse functionalities, several integrated applications are available:
-
text-generation-webui
- An accessible solution with numerous one-click installers, facilitating easy adoption and integration. Full instructions and download links are provided on their GitHub page.
-
Wenda
- Providing a user-friendly packaged program, Wenda can be downloaded and utilized effectively for various AI tasks. Detailed usage instructions and the download link are provided directly on the project's repository.
Self-Hosted AI represents a pivotal development in making powerful AI models more accessible and manageable at a personal or enterprise level, encouraging broader experimentation and application integration.