llm.ts

Introduction to llm.ts

llm.ts is a versatile library designed to enhance the utility of language models by offering a unified way to communicate with over 30 different large language models (LLMs) through a single API. This unique feature simplifies the process of sending multiple prompts to numerous models and receiving their responses in a cohesive, single response.

Key Features

• Single API for Multiple LLMs: llm.ts allows easy interaction with a wide range of language models via a unified API. This makes it convenient to experiment with different models without needing to switch between different interfaces.

• Zero Dependencies: This library is lightweight, with a minified version under 10kB, meaning it won’t bloat your project with unnecessary dependencies.

• Flexible API Key Management: With llm.ts, users can bring their API keys, offering flexibility and security tailored to individual needs.

• Cross-Platform Usability: Whether you're working in Node.js, Deno, or directly in the browser, llm.ts functions without a hitch.

Installation

You can easily add llm.ts to your project using npm or yarn:

npm

npm install llm.ts

yarn

yarn add llm.ts

Usage Example

Below is a simple example demonstrating how to use llm.ts:

import { LLM, MODEL } from 'llm.ts';

(async function () {
    await new LLM({
        apiKeys: {
            openAI: process.env.OPENAI_API_KEY ?? '',
            cohere: process.env.COHERE_API_KEY ?? '',
            huggingface: process.env.HF_API_TOKEN ?? '',
        }
    }).completion({
        prompt: [
            'Repeat the following sentence: "I am a robot."',
            'Repeat the following sentence: "I am a human."',
        ],
        model: [
            'text-ada-001',
            'cohere/command-nightly',
            MODEL.HF_GPT2,
        ],
    }).then(resp => {
        console.log(resp);
    })
})();

The above code snippet executes multiple prompts to various models and logs their respective text outputs to the console.

Supported Models

llm.ts supports a wide variety of models, including but not limited to:

• OpenAI models: text-ada-001, text-babbage-001, text-curie-001, etc.
• Cohere models: cohere-command, cohere-command-nightly
• HuggingFace models: gpt2, bloom-1b, llama-7b, gptj-6b
• Other specialized models such as distilgpt2, stablelm-tuned-3b

If you wish to add more models, the project is open for contributions via pull requests.

Supported Hosting Providers

llm.ts collaborates with renowned hosting providers, permitting integrations with:

• OpenAI
• Cohere
• HuggingFace

Licensing

The project is open-source and shared under the MIT License, allowing for open collaboration and widespread usage.

llm.ts presents itself as a powerful yet straightforward tool for developers and researchers needing a multi-model approach to language processing, enabling efficient and flexible interactions with numerous language models through a single, elegant interface.