semantic-router

Introducing Semantic Router

Semantic Router is a cutting-edge tool designed to enhance the decision-making process for Large Language Models (LLMs) and AI agents. This innovative technology is focused on accelerating decisions by leveraging the power of semantic vector spaces, thereby bypassing the sluggishness of generating responses purely through traditional LLMs. Semantic Router smartly routes requests using the inherent semantic meaning behind them, making interactions efficient and purposeful.

Getting Started with Semantic Router

For those eager to explore Semantic Router, getting started is a breeze. The package can be easily installed using pip:

pip install -qU semantic-router

Semantic Router offers flexibility for users who prefer running the tool locally by supporting different configurations through additional installations:

• Fully Local Execution:

  pip install -qU "semantic-router[local]"
  

• Hybrid Route Layer:

  pip install -qU "semantic-router[hybrid]"
  

Configuring Routes

One of the core functionalities of Semantic Router is defining Route objects. These objects are pathways that the router can choose based on semantic relevance. For example, you can create distinct routes for handling political discussions and general chit-chat.

from semantic_router import Route

politics = Route(
    name="politics",
    utterances=[
        "isn't politics the best thing ever",
        "why don't you tell me about your political opinions",
    ],
)

chitchat = Route(
    name="chitchat",
    utterances=[
        "how's the weather today?",
        "lovely weather today",
    ],
)

routes = [politics, chitchat]

Setting Up the Encoder

To enable rapid decision-making based on semantics, Semantic Router requires an encoder. Currently, it supports CohereEncoder and OpenAIEncoder.

import os
from semantic_router.encoders import CohereEncoder, OpenAIEncoder

# Using Cohere Encoder
os.environ["COHERE_API_KEY"] = "<YOUR_API_KEY>"
encoder = CohereEncoder()

# Or using OpenAI Encoder
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()

Implementing the Route Layer

With the routes and encoder in place, the next step is to implement the RouteLayer, which handles the semantic decision-making process.

from semantic_router.layer import RouteLayer

rl = RouteLayer(encoder=encoder, routes=routes)

The Route Layer can be tested with queries to verify its decision-making capabilities:

rl("don't you love politics?").name  # Output: 'politics'
rl("how's the weather today?").name  # Output: 'chitchat'

If a query doesn't match any defined route, the result will be None.

Integrations and Extensions

Semantic Router integrates seamlessly with various platforms and tools, providing extensive flexibility for users. It supports integrations with Cohere, OpenAI, Hugging Face, FastEmbed, and enables multi-modality as well.

Additionally, its vector space can integrate with vector databases like Pinecone and Qdrant.

Resources and Community

For more detailed documentation and to explore use cases, users can refer to the comprehensive documentation and online resources. The community around Semantic Router is active, featuring discussions and papers that highlight its application in enhancing network management and AI functionalities.

Semantic Router is shaping the future of AI interactions by providing faster, more intelligent routing decisions. Whether for chatbots or large-scale AI systems, this project offers a robust foundation for efficient decision-making in the digital age.