tvhong/muopdb

MuopDB - A Vector Database

★ 0Forks 0RustGitHub ↗Compare

Project website ↗

README

MuopDB - A vector database for machine learning

Introduction

MuopDB is a vector database for machine learning. This project is done under TechCare Coaching. It plans to support the following features:

V0 (Done)

  • Query path
    • Vector similarity search
    • Hierarchical Navigable Small Worlds (HNSW)
    • Product Quantization (PQ)
  • Indexing path
    • Support periodic offline indexing
  • Database Management
    • Doc-sharding & query fan-out with aggregator-leaf architecture
    • In-memory & disk-based storage with mmap

V1

  • Query path
    • Inverted File (IVF)
    • Improve locality for HNSW
    • RabitQ quantization
  • Indexing path
    • Support realtime indexing

Why MuopDB?

This is an educational project for me to learn Rust & vector database.

Building

Install prerequisites:

# macos
brew install hdf5 protobuf

export HDF5_DIR="$(brew --prefix hdf5)"

Build:

# from top-level workspace
cargo build --release

Test:

cargo test --release

Contributors

hicdertyb0807tvhongpetrpan26

Issues