Semantic Search App Template
The Semantic Search App Template is a helpful resource for developers looking to build applications that utilize semantic search. This template is powered by the Atlas Embedding Database and FastAPI, with optional integrations with the OpenAI Embedding API and Langchain.
Overview of Semantic Search
To understand what this project offers, it's important to grasp semantic search. Unlike traditional keyword searches, semantic search aims to understand the meaning behind the text, rather than just matching terms. This allows for more relevant search results, as the search engine interprets the intent and context of the query.
Getting Started
The journey to developing a semantic search application generally involves a few key steps:
-
Dataset Collection: Gather a collection of text, images, or any other content that you want your application to search through.
-
Generating Embeddings: Use an Embedding API, such as those provided by Cohere or OpenAI, to create embeddings from your data. Embeddings are vector representations that capture the semantic meanings of data.
-
Vector Similarity Search: Develop programming logic that can perform vector similarity searches on your embeddings and fetch the relevant content.
-
REST API Backend: Integrate the above functionalities into a backend service accessible through a REST API.
Simplifying with the App Template
This app template streamlines the process by offering a ready-to-use FastAPI and React application that interfaces efficiently with the Atlas Embedding Database. Atlas allows developers to upload content – such as text documents and their embeddings – and gain immediate access to semantic search capabilities. One of the key advantages of using Atlas is its built-in visual debugger that helps you understand the type of results your semantic search app will yield for different queries.
Step-by-Step Guide
Starting the App
Begin by building the FastAPI Docker image, a step necessary at the first build or when new dependencies are introduced:
DOCKER_BUILDKIT=1 docker build -t backend_api --progress plain -f backend/Dockerfile.buildkit .
Following the build, start the backend with:
docker compose up --build
Don't forget to insert your Nomic API Key—granting access to your Atlas Embedding Database—into the settings file or Docker compose environment variables. Obtain this API Key by creating an account on Atlas.
Uploading and Indexing Content in Atlas
Use the demo news dataset provided in the tutorial to upload content to Atlas. This data will be stored within an Atlas Embedding DB project named '10k News Articles'. Ensure the project name and your OpenAI API key are included in the settings file as environment variables.
Running the App
After setting up, use docker compose up to run the backend. It's crucial to review the logs for any errors, especially those related to missing environment variables.
Testing and Frontend
Once the FastAPI application is running, you can access documentation and test its search capabilities at localhost:80/docs.
As of this tutorial, the front-end is not yet implemented. Developers are encouraged to contribute by building a React App, with the possibility of having it merged into the main project.
In summary, the Semantic Search App Template provides a robust framework to expedite the development of semantic search applications, leveraging FastAPI and the Atlas Embedding Database to offer an efficient and intelligent search experience.