Skip to content
HUIAHUB

RyanCodrai/turbovec

turbovec

A vector index built on TurboQuant, written in Rust with Python bindings

Metrics updated Established
View on GitHub (opens in a new tab)Homepage / demo

GitHub metrics

Stars
17K
growth after 7 days of data
Forks
1.5K
30-day growth
—
Open issues
23

The chart appears once the repository has been checked on two or more days.

What it does

turbovec is an in-process vector index for Python and Rust that stores embeddings at 2 or 4 bits per dimension and searches the compressed vectors directly. It is written in Rust with Python bindings and uses TurboQuant, a quantization method from Google Research.

The README's headline figures: ten million 768-dimensional embeddings that need 31 GB as float32 fit in 4 GB, and search runs 4.4× faster than FAISS FastScan at 4-bit, at matching recall. Its benchmark table puts the 4-bit speedup at 3.8–5.1× across eight test cells on GCP ARM and x86 machines. These are the project's own benchmarks, and the scripts and raw JSON are published alongside them.

No training step is needed, so a vector is searchable as soon as it is added, and removal takes microseconds. sync() writes only what changed since the last call. IdMapIndex keys vectors by your own uint64 ids and accepts an allowlist filter at query time. Drop-in stores exist for LangChain, LlamaIndex, Haystack and Agno. It is a library rather than a server, and it scans every vector instead of using a graph index.

Why it matters

Float32 embeddings take a lot of RAM as a corpus grows. turbovec compresses them 7.8× at 4-bit and 15.5× at 2-bit, per the README, with a codebook that comes from the math rather than your data, so there is nothing to train. Updates stay cheap too: the README reports `remove(id)` at 1.4–5.8 µs against 0.19–1.03 s for FAISS, which repacks all codes on each call.

Main features

  • 2-bit and 4-bit quantization based on Google Research's TurboQuant
  • SIMD kernels for ARM and x86, with AVX-512 or AVX2 picked at runtime
  • Add vectors without a training step; remove them in microseconds
  • Incremental, crash-safe persistence through `sync()`
  • `IdMapIndex` with your own `uint64` ids and O(1) removal
  • Allowlist or bitmask filtering at search time
  • Drop-in stores for LangChain, LlamaIndex, Haystack and Agno

Who it's for

  • Python or Rust developers who need vector search inside their own process
  • RAG apps on LangChain, LlamaIndex, Haystack or Agno that use an in-memory store
  • Projects where RAM is tight or the index changes often

Strengths

  • Memory drops 7.8× at 4-bit and 15.5× at 2-bit versus float32, per the README
  • Adding or removing vectors needs no rebuild
  • Benchmarks ship with scripts and raw JSON for re-running
  • Nothing leaves the process, so it can run fully offline

Limitations

  • Every query scans all vectors with no graph index, so cost grows linearly with corpus size
  • Quantization is lossy; the README advises checking recall on your own data
  • No network API, replication or multi-tenant service
  • Scores are inner products, so cosine similarity requires normalized vectors

Getting started

Install the package from PyPI for Python, or add the crate for Rust. Building the Python wheel from source uses maturin.

pip install turbovec
cargo add turbovec

Requirements: Python or Rust; float32 input vectors with a dimension that is a multiple of 8 and at most 16,384.

Always follow the official instructions in the README on GitHub. This overview is our summary, not a copy.

Sources