Introduction to sqlite-vec
sqlite-vec is a highly efficient vector search extension for SQLite, renowned for its minimal size and satisfactory speed. This extension is capable of running on virtually any platform where SQLite is functional, including operating systems like Linux, MacOS, and Windows, as well as web browsers using WebAssembly (WASM), and devices such as Raspberry Pis. As the successor to sqlite-vss, sqlite-vec is still in its pre-version 1 stage, which means that users should anticipate potential breaking changes.
Core Features
- Data Storage and Querying:
sqlite-vecprovides the capability to store and query various types of vectors, including float, int8, and binary vectors, throughvec0virtual tables. - Multi-environment Compatibility: Written purely in C without any dependencies,
sqlite-veccan run anywhere SQLite does, ensuring broad compatibility and ease of integration. - Efficient Data Filtering: It allows pre-filtering of vectors using subqueries in the form of
rowid IN (...), making vector searches more efficient.
Project Sponsorship
sqlite-vec has garnered support from several reputable organizations, with Mozilla leading sponsorship under its Builders project initiative. Additional sponsors include:
- Fly.io: Known for its platform-as-a-service offerings
- Turso: Focused on providing optimized database services
- SQLite Cloud: Specializes in cloud database solutions
These sponsorships underline the community and industry confidence in sqlite-vec and its potential to enhance local AI applications.
Installing sqlite-vec
Installing sqlite-vec is straightforward and varies slightly based on the programming language or system you are using:
- Python: Install with
pip install sqlite-vec - Node.js: Use
npm install sqlite-vec - Ruby: Utilize
gem install sqlite-vec - Go: Incorporate using
go get -u github.com/asg017/sqlite-vec/bindings/go - Rust: Integrate with
cargo add sqlite-vec - Datasette, rqlite, sqlite-utils: Each has its own specific instructions, which can be easily found in the detailed documentation
For more comprehensive installation guidance, users are encouraged to visit the sqlite-vec installation page.
Sample Usage
A basic use-case scenario in SQL illustrates the ease with which users can set up and query vector data with sqlite-vec. After loading the vec0 module and creating a virtual table, users can insert vectors and perform K-Nearest Neighbors (KNN) style queries to find similar vectors based on distance calculations.
Community and Sponsorship Opportunities
The ongoing development of sqlite-vec is supported through community contributions and individual sponsors via platforms like Github. Companies or individuals interested in supporting the project can connect with the developer for more information, fostering a collaborative environment for advancing this powerful extension.
Related Projects
For those interested in exploring more SQLite enhancements, the following projects are recommended:
sqlite-ecosystem: A collection of third-party SQLite extensions.sqlite-rembed: Facilitates generating text embeddings from remote APIs for testing and SQL scripting purposes.sqlite-lembed: Allows for local generation of text embeddings using models in the.ggufformat.
By tapping into these resources, users can significantly leverage the capabilities of SQLite and sqlite-vec for innovative and practical applications.