Milvus-Backup Project Introduction
Milvus-Backup is a specialized tool designed to facilitate the backup and restoration of data in Milvus, an open-source vector database. The tool supports operations either through a command-line interface (CLI) or via an API server, making it accessible for a range of users from developers to system administrators.
Minimal Impact on Performance
One of the standout features of Milvus-Backup is its minimal impact on the ongoing operations of a Milvus cluster. Users can comfortably run backup and restoration tasks without experiencing any disruptions in the database’s performance, enabling continuous and seamless data operations.
Installation
Adding Milvus-Backup to your toolkit involves straightforward steps. For most users, downloading the latest version from the release page is recommended. Mac users have the added convenience of using homebrew to install the tool with a simple command:
brew install zilliztech/tap/milvus-backup
Configuration
The tool requires configuration to establish connections with a Milvus proxy and a Minio cluster, which are detailed in a file named backup.yaml. It’s crucial to ensure that settings for Minio are precise, as defaults can vary based on whether Milvus is deployed via docker-compose or Kubernetes. Notably, Milvus-Backup mandates the use of object storage like Minio for backups instead of local paths, reflecting its focus on scalability and ease of management.
Development
For developers wishing to extend or test the functionalities:
Building the Tool:
The tool can be built using Go commands, which generates an executable named milvus-backup.
go get
go build
Testing the Tool:
Testing can be accomplished via an IDE or through command-line tests, with dedicated test scripts provided in the repository.
API Server
Milvus-Backup features a REST API server, which can be started after the tool is built:
./milvus-backup server
The server typically listens on port 8080, but this can be customized using the -p parameter.
Users gain access to a comprehensive API interface available through Swagger UI for detailed interactions with Milvus-Backup’s features, including creating, listing, retrieving, deleting, and restoring backups.
Command Line Interface
The CLI is powered by Cobra, providing simple commands for various operations:
- Create a Backup: Forms a backup of selected collections, either specified by name or all if none is given.
- List Backups: Displays all existing backups stored in MinIO.
- Get a Backup: Retrieves specific backups using their names.
- Delete a Backup: Removes selected backups from storage.
- Restore a Backup: Recreates collections and restores data, with optional asynchronous execution and index restoration.
Demonstration
To showcase Milvus-Backup’s functionality, users can follow a four-step demo process:
-
Connection Check: Verifies connection status to Milvus and storage.
./milvus-backup check -
Data Preparation: Involves setting up a Milvus collection and populating it with data.
-
Backup Creation: Initiates a backup process for the prepared data.
-
Backup Restoration and Verification: Restores data and optionally indexes. Verifies the accuracy of restored data via a testing script.
By following these steps, users can effectively leverage Milvus-Backup to secure and manage their data environments.
License
Milvus-Backup is released under the Apache License, Version 2.0, ensuring it remains free and open for users to adapt and distribute, fostering a collaborative and innovative environment for database management.