Feishu-Vector-Knowledge-Management - Improve Efficiency in Knowledge Management with Embeddings and Qdrant

Feishu-Vector-Knowledge-Management: A Comprehensive Introduction

Feishu-Vector-Knowledge-Management is a sophisticated tool designed to enhance knowledge management by leveraging the functionalities of Feishu-OpenAI and integrating specific knowledge base features.

👻 Bot Features

Beyond the inherent capabilities offered by Feishu-OpenAI, this project introduces several additional knowledge base functionalities:

Knowledge Base Q&A: Users can engage with the knowledge base by using the command /faq or directly querying it.
CSV Import for Knowledge Base: The system supports the importation of vector data in CSV format, making it easy to manage large datasets.
CSV Creation for Knowledge Base: It provides the ability to reconstruct vector files in CSV format, significantly reducing token costs.
Knowledge Base Management: Users can import data from URLs and files, allowing for streamlined management of data resources.
Record Management: It features robust capabilities to query, add, delete, and modify database records, ensuring comprehensive data handling.
Caching of Question Vectors: By caching question vectors, the system lowers token costs and minimizes redundant queries.

🌟 Project Highlights

Feishu-Vector-Knowledge-Management stands out due to its foundation on Embeddings combined with Qdrant for contextual querying. This unique combination allows for highly efficient and accurate data retrieval.

Project Deployment

Initial Setup

To begin utilizing Feishu-Vector-Knowledge-Management, users should follow these steps:

git clone https://github.com/ConnectAI-E/Feishu-Vector-Knowledge-Management
cd Feishu-Vector-Knowledge-Management

Data Importation

To import your data into the system, execute the following commands:

# Splitting QA data into a CSV file
go run ./cmd - prepare split -f ./data/demo/raw.txt -o ./data/demo/raw.csv

# Converting CSV to vector data using OpenAI's embedding API
go run ./cmd - prepare analyze -f ./data/demo/raw.csv -o ./data/demo/vector.csv

# Import vector data from a CSV file
go run ./cmd - prepare import -f ./data/demo/vector.csv

CSV File Structure

The CSV files should have the following headers:

id,url,title,content,title_vector,content_vector,vector_id

Sample data can be downloaded from the project repository.

Testing Qdrant Interface

For testing and debugging, users can access the online Swagger documentation at: https://ui.qdrant.tech/#/

Deployment

For deploying the project, users can follow the steps below, particularly if using Docker Compose:

Edit the docker-compose.yaml file, configure the necessary environment variables, or mount corresponding configuration files through volumes.
Run the following commands to build and manage the service:

# Build the docker image
docker compose build

# Start the service
docker compose up -d

# Stop the service
docker compose down

Webhook and card callback addresses can be set to:

Event callback address: http://IP:9000/webhook/event
Card callback address: http://IP:9000/webhook/card

Further Interaction

For more detailed engagement with the project, users are encouraged to visit the Feishu-OpenAI page.

Acknowledgements

Special thanks to 'Find My Ai' for providing partial financial support for the project.