agent-search

AgentSearch: A Framework for Enhancing Search Capabilities

AgentSearch is an innovative framework designed to empower search agents by seamlessly integrating large language models (LLMs) with various search engines. This integration allows these search agents to perform a wide array of tasks using a technique called Retrieval-Augmented Generation (RAG). These tasks include summarizing search results, generating new search queries, and retrieving detailed results.

Features of AgentSearch

• Search Agent Integration: With AgentSearch, users can effortlessly build a search agent. They simply need to connect any search-specialized LLM, like Sensei-7B, to a compatible search engine.

• Customizable Search: By using the AgentSearch dataset alongside this framework, users can deploy a customizable local search engine that suits their specific needs.

• API Endpoint Integration: AgentSearch offers seamless integration with a variety of hosted provider APIs, providing flexibility and ease of use. Supported APIs include Bing, SERP API, and AgentSearch itself. Additionally, support is available for LLMs from various providers, including SciPhi, HuggingFace, OpenAI, and Anthropic.

Quickstart Guide

Installation

To get started with AgentSearch, install the package using pip:

pip install agent-search

Configuration

First, obtain your free API key from SciPhi and set it in your environment:

export SCIPHI_API_KEY=$MY_SCIPHI_API_KEY

Usage

To call a pre-configured search agent endpoint, follow these steps:

# Ensure SCIPHI_API_KEY is set in the environment
from agent_search import SciPhi

client = SciPhi()

# Search, summarize results, and generate related queries
agent_summary = client.get_search_rag_response(query='latest news', search_provider='bing', llm_model='SciPhi/Sensei-7B-V1')
print(agent_summary)
# { 'response': '...', 'other_queries': '...', 'search_results': '...' }

AgentSearch supports standalone searches and searches from its own search engine:

# Ensure SCIPHI_API_KEY is set in the environment
from agent_search import SciPhi

client = SciPhi()

# Perform a search
search_response = client.search(query='Quantum Field Theory', search_provider='agent-search')

print(search_response)
# [{ 'score': '.89', 'url': 'https://...', 'metadata': {...} }]

To create a custom search agent workflow, users can code their instructions as follows:

# Ensure SCIPHI_API_KEY is set in the environment
from agent_search import SciPhi

client = SciPhi()

# Define task instructions
instruction = "Your task is to perform retrieval augmented generation (RAG) over the given query and search results. Return your answer in a json format that includes a summary of the search results and a list of related queries."
query = "What is Fermat's Last Theorem?"

# Construct search context
search_response = client.search(query=query, search_provider='agent-search')
search_context = "\n\n".join(
      f"{idx + 1}. Title: {item['title']}\nURL: {item['url']}\nText: {item['text']}"
      for idx, item in enumerate(search_response)
).encode('utf-8')

# Enforce a JSON response
json_response_prefix = '{"summary":'

# Prepare a prompt
formatted_prompt = f"### Instruction:{instruction}\n\nQuery:\n{query}\n\nSearch Results:\n${search_context}\n\nQuery:\n{query}\n### Response:\n{json_response_prefix}"

# Generate a completion with Sensei-7B-V1
completion = json_response_prefix + client.completion(formatted_prompt, llm_model_name="SciPhi/Sensei-7B-V1")

print(completion)
# {
#   "summary":  "\nFermat's Last Theorem is a mathematical proposition first prop ... ",
#   "other_queries": ["The role of elliptic curves in the proof of Fermat's Last Theorem", ...]
# }

Community & Support

• Engage with Us: Users are encouraged to join the Discord community for discussions and updates.

• Feedback & Inquiries: Personalized support can be obtained by contacting the team via email.

Additional Notes

• Commands should be executed from the root directory of the AgentSearch project.
• A comprehensive User Guide is planned to be released soon!