Introducing llm-ls
llm-ls is a cutting-edge Language Server Protocol (LSP) server designed to enhance the development experience by leveraging the power of large language models (LLMs). While the project is still under active development, llm-ls aims to streamline coding by providing a common platform upon which Integrated Development Environment (IDE) extensions can be built. The server handles complex interactions with LLMs, allowing developers to create lightweight extensions.
Key Features
Intelligent Prompting
At the heart of llm-ls is its intelligent prompting feature. By using the current file as context, the server creates sophisticated prompts that suit your needs—whether simple or requiring more detailed, "fill in the middle" style completion. Additionally, llm-ls ensures you stay within the model's context window by tokenizing prompts for efficient processing.
Telemetry for Improvement
llm-ls collects vital telemetry data on requests and completions, which is valuable for model retraining and optimization. Importantly, this data remains private and is stored locally in a specified log file, ensuring user privacy.
Completion Control
The system intelligently parses the Abstract Syntax Tree (AST) of your code to decide the type of completion required. This could be a multi-line, single-line, or no completion at all, tailoring the development experience to your specific context.
Support for Multiple Backends
Compatibility is a key advantage of llm-ls, as it works seamlessly with several backends. These include the Hugging Face's Inference API, Hugging Face's text-generation-inference, ollama, and OpenAI-compatible APIs such as the Python llama.cpp server bindings. This flexibility allows developers to utilize their preferred tools.
Compatible IDE Extensions
llm-ls supports a variety of extensions designed to integrate smoothly with popular development environments:
- llm.nvim: A Neovim extension.
- llm-vscode: A Visual Studio Code extension.
- llm-intellij: An extension catering to the IntelliJ platform.
- jupytercoder: Although not yet fully integrated, this extension is intended for Jupyter environments.
Roadmap for the Future
As llm-ls continues to develop, several exciting enhancements are planned, including:
- Expanding support to obtain context from multiple files within a workspace.
- Introducing a
suffix_percentsetting for prompt structuring, allowing control over the token ratio between prefix and suffix. - Adjusting context handling to either optimize context window fill percentage or change to
max_tokens. - Implementing filters to exclude repetitive or redundant suggestions.
- Exploring methodologies for online transaction processing (OLTP) traces.
With these planned features, llm-ls aims to further refine and improve the ability of developers to leverage the capabilities of LLMs in their coding environment.