cortex
Cortex is a Local AI API Platform that allows for the running and customization of Language Learning Models (LLMs) using a user-friendly CLI. This platform, implemented in C++, integrates with Huggingface and Cortex's own models, and supports various engines like llama.cpp, ONNXRuntime, and TensorRT-LLM. Available with local and network installers, it provides cross-platform compatibility across Windows, MacOS, and Linux. Cortex enables flexible model management and can be deployed as a standalone API or integrated into other applications. The platform supports multiple model quantizations and is on track to include full OpenAI API features.
