api-for-open-llm
Offers a unified API for open-source large language models based on OpenAI's standards, featuring real-time streaming responses, text embedding, and support for tools such as langchain and vLLM. Allows easy substitution of ChatGPT with open models through simple environment changes, supporting various applications. Compatible with custom-trained LoRA models and optimized for rapid processing with vLLM's acceleration. Integrates with popular models like MiniCPM-Llama3 and GLM-4V for seamless project compatibility.
