Knowhere Project Overview
Introduction
Knowhere is a sophisticated C++ project that forms the internal core of Milvus, a powerful vector database system. Despite being integral to Milvus, Knowhere operates as an independent entity and is pivotal for enhancing performance and functionality related to vector data processing.
Building Knowhere Within Milvus
Users interested in utilizing Knowhere within the Milvus framework can seamlessly do so. Without altering the Knowhere source code, Milvus can be built directly from its main project repository, allowing Knowhere to be constructed automatically during Milvus's build process.
System Requirements
Knowhere is primarily developed on several Linux distributions, especially Ubuntu and CentOS. There is also support from contributors using Mac systems, including both x86_64 and Apple Silicon architectures. The project welcomes contributions to extend support to other operating systems. As of now, Knowhere runs effectively on:
- Ubuntu 20.04 (x86_64 and Aarch64)
- MacOS (x86_64 and Apple Silicon)
Building Knowhere From Source Code
Installing Dependencies
For those looking to build Knowhere from its source, it is essential to install certain dependencies. On Linux, this involves using the terminal to install components like build-essential, libopenblas-openmp-dev, and others. Additionally, Conan, a package manager, needs to be installed via Python's package installer to handle project configurations.
Building Process
Once the dependencies are in place, the building process varies slightly between Ubuntu and MacOS systems, primarily involving configuring build types such as Debug or Release and opting for different hardware settings (CPU or GPU). A Conan package manager is used extensively in this process.
Running Unit Tests
After building the project, it is critical to perform unit tests to ensure everything functions correctly. This involves executing specific commands within the build directories for Debug and Release versions.
Generating Python Wheel
To extend the functionality of Knowhere, users can generate a Python Wheel - a compiled package that can be easily installed. This requires additional dependencies such as SWIG (Simplified Wrapper and Interface Generator) and subsequent Python bindings to be prepared and installed following the build.
Clean-up Procedure
Post build and test, it may be necessary to clean up the project directory, especially if alteration or reconfiguration of the build is needed. This involves using git commands to remove extraneous files.
Contributing to Knowhere
Contributions to Knowhere are encouraged, with a pre-commit checking system set up to ensure code quality. Before submitting pull requests, contributors should use pre-commit hooks to automatically check code style and logical errors using tools like clang-format and clang-tidy.
This structured approach not only facilitates a smooth development process within Knowhere but also ensures that the project maintains high standards of code quality and performance. The project remains open for further enhancement and adaptation to more systems and environments.