TokenCost: Simplifying Cost Estimation for Large Language Model Apps
TokenCost is a valuable tool designed for developers and businesses utilizing Large Language Models (LLMs) in their applications. It provides a streamlined way to estimate the costs associated with using popular LLM APIs such as OpenAI, by calculating the potential expenses of both prompts and message completions.
Key Features
-
LLM Price Tracking: As LLM providers frequently update their model offerings and pricing structures, TokenCost helps users stay informed about the latest changes. This ensures accurate cost estimation based on current price rates.
-
Token Counting: The tool accurately counts the number of tokens in a prompt message before it is sent to an LLM, allowing for precise cost calculations.
-
Easy Integration: TokenCost simplifies the integration process by providing a single function call to obtain the costs associated with prompt and completion tokens. This is especially beneficial for developers looking to incorporate AI functionalities into their applications without worrying about hidden expenses.
Example Usage
Here's a simple example demonstrating how easy it is to use TokenCost for cost estimations:
from tokencost import calculate_prompt_cost, calculate_completion_cost
model = "gpt-3.5-turbo"
prompt = [{"role": "user", "content": "Hello world"}]
completion = "How may I assist you today?"
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000135 + 0.000014 = 0.0000275
Installation
The recommended installation method is via PyPI. Simply run:
pip install tokencost
This command will add TokenCost to your Python environment, making it ready for immediate use in projects requiring LLM price estimation.
Practical Applications
Calculating Prompt and Completion Costs: TokenCost can be used to calculate the cost of input prompts and AI-generated outputs efficiently. For example:
from openai import OpenAI
client = OpenAI()
model = "gpt-3.5-turbo"
prompt = [{"role": "user", "content": "Say this is a test"}]
chat_completion = client.chat.completions.create(messages=prompt, model=model)
completion = chat_completion.choices[0].message.content
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000180 + 0.000010 = 0.0000280
Token Counting: The tool provides functionality for counting tokens in both message-based and string-based prompts:
from tokencost import count_message_tokens, count_string_tokens
message_prompt = [{"role": "user", "content": "Hello world"}]
print(count_message_tokens(message_prompt, model="gpt-3.5-turbo")) # Output: 9
print(count_string_tokens(prompt="Hello world", model="gpt-3.5-turbo")) # Output: 2
Comprehensive Cost Table
TokenCost maintains an updated cost table for various LLM models, ensuring users have access to the latest pricing information. This table helps in comparing different models and choosing the most cost-effective solution for specific application needs.
By offering these features, TokenCost empowers developers to make informed decisions about the financial aspects of integrating AI models into their applications, thus optimizing expenditure and enhancing resource allocation.