cursive

Cursive: An Intuitive Framework for LLM Interactions

Cursive stands as a universal and remarkably intuitive framework designed for interacting with Large Language Models (LLMs). It offers seamless functionality across any JavaScript runtime, focusing heavily on extensibility and a positive experience for developers.

Highlights

• Compatible: Cursive works effortlessly in various environments such as the browser, Node.js, Deno, Bun, and Cloudflare Workers. Through a tool called WindowAI, users can securely utilize their own credentials and models for language tasks.

• Extensible: With cursive, developers can effortlessly integrate custom functionalities at any stage of a completion's lifecycle, whether it's for logging purposes, caching, or modifying output results.

• Functions: Developers can define functions that the LLMs can employ, working seamlessly with models like GPT-4, GPT-3.5, Claude 2, and Claude Instant.

• Universal: Cursive aims to bridge capabilities across different models, allowing users to choose their preferred model through a unified interface.

• Informative: The framework comes enriched with features that accurately calculate token usage and associated costs.

• Reliable: It includes built-in mechanisms for automatic retries and switching to models with greater context length if required, ensuring stability and performance.

Quickstart

To get started with Cursive, follow these simple steps:

1. Install:

   npm i cursive
   

2. Start using:

   import { useCursive } from 'cursive'
   
   const cursive = useCursive({
       openAI: {
           apiKey: 'sk-xxxx'
       }
   })
   
   const { answer } = await cursive.ask({
       prompt: 'What is the meaning of life?',
   })
   

Usage

Conversation

Creating conversational chains is straightforward with Cursive, allowing you to use options familiar from OpenAI's API.

const resA = await cursive.ask({
    prompt: 'Give me a good name for a gecko.',
    model: 'gpt-4',
    maxTokens: 16,
})

console.log(resA.answer) // Outputs: Zephyr

const resB = await resA.conversation.ask({
    prompt: 'How would you say it in Portuguese?'
})

console.log(resB.answer) // Outputs: Zéfiro

Streaming

Cursive supports streaming responses while keeping track of token usage.

const result = await cursive.ask({
    prompt: 'Count to 10',
    stream: true,
    onToken(partial) {
        console.log(partial.content)
    }
})

console.log(result.usage.totalTokens) // Outputs: 40

Functions

You can define functions using Type from the typebox library, then describe and execute them via Cursive.

import { Type, createFunction, useCursive } from 'cursive'

const cursive = useCursive({
    openAI: {
        apiKey: 'sk-xxxx'
    }
})

const sum = createFunction({
    name: 'sum',
    description: 'Sums two numbers',
    parameters: {
        a: Type.Number({ description: 'Number A' }),
        b: Type.Number({ description: 'Number B' }),
    },
    async execute({ a, b }) {
        return a + b
    },
})

const { answer } = await cursive.ask({
    prompt: 'What is the sum of 232 and 243?',
    functions: [sum],
})

console.log(answer) // Outputs: The sum of 232 and 243 is 475.

Paused functions can also be created to control execution flow as needed.

Hooks

Developers can hook into the lifecycle of a completion to handle events, such as capturing errors or calculating costs.

cursive.on('completion:after', (result) => {
    console.log(result.cost.total)
    console.log(result.usage.total_tokens)
})

cursive.on('completion:error', (error) => {
    console.log(error)
})

cursive.ask({
    prompt: 'Can androids dream of electric sheep?',
})

// Example output: Cost and token usage information

Embedding

Generate embeddings for documents and prepare for future integration possibilities.

const embedding = await cursive.embed('This should be a document.')

Reliability

Cursive automatically retries failed completions and can substitute models with greater context lengths when necessary. Settings for retries and expansions are easily configurable through the useCursive function.

const cursive = useCursive({
    maxRetries: 5,
    expand: {
        enable: true,
        defaultsTo: 'gpt-3.5-turbo-16k',
        modelMapping: {
            'gpt-3.5-turbo': 'gpt-3.5-turbo-16k',
            'gpt-4': 'claude-2',
        },
    },
    allowWindowAI: true,
    countUsage: false,
})

Examples

Cursive provides sample applications for quick integration, like a simple application with Nuxt and a simple edge API with Cloudflare Workers.

Roadmap

The team behind Cursive is continuously advancing the framework with plans to support more vendors like Anthropic, Cohere (via WindowAI), Azure, Huggingface, and Replicate.

Credits

Special thanks to @disjukr for transferring the cursive npm package name, enabling the advancement of Cursive as a reliable tool for developers interacting with LLMs.