Google's MUVERA: The One-Vector Trick Behind Fast Multi-Vector Search
Summary
Multi-vector models like ColBERT keep one embedding per token, which wins on accuracy but breaks fast indexes: Chamfer scoring forces custom pipelines like PLAID. MUVERA's Fixed Dimensional Encodings randomly partition the embedding space, aggregate each region's tokens (queries sum, documents average), and concatenate the blocks into one vector whose dot product provably approximates the multi-vector score, the first single-vector proxy with theoretical guarantees. On BEIR: about 10% higher recall at 90% lower latency than PLAID, 2 to 5x fewer candidates, 32x memory compression via product quantization. Paper: arxiv.org/abs/2405.19504
About this video
Multi-vector models like ColBERT keep one embedding per token, which wins on accuracy but breaks fast indexes: Chamfer scoring forces custom pipelines like PLAID. MUVERA's Fixed Dimensional Encodings randomly partition the embedding space, aggregate each region's tokens (queries sum, documents average), and concatenate the blocks into one vector whose dot product provably approximates the multi-vector score, the first single-vector proxy with theoretical guarantees. On BEIR: about 10% higher recall at 90% lower latency than PLAID, 2 to 5x fewer candidates, 32x memory compression via product quantization. Paper: arxiv.org/abs/2405.19504