The Short Answer
You need a dedicated vector database when exact search stops fitting your latency budget, and not before. For most corpora that threshold sits between 100,000 and 1,000,000 vectors, but the number that decides it is not the vector count: it is vectors x dimensions x query rate against the latency you owe a user. Below that line, brute-force cosine similarity in NumPy or pgvector is simpler, exact, and free.
The four things that actually force a dedicated store are filtered search at scale, sustained write throughput while serving reads, recall you can tune rather than accept, and more vectors than fit in the memory you are willing to buy. If none of those describe you, adding a vector database adds an operational dependency and buys latency you were not short of.
The Arithmetic, Because It Is Simpler Than It Looks
A brute-force search is one matrix multiply. Every vector is touched, so the cost is exactly
n x d multiply-adds per query, and it is the same cost every time regardless of what you search for.At 768 dimensions in float32, one million vectors is about 3 GB and a single query is roughly 768 million operations. On one modern CPU core that is in the region of a few hundred milliseconds; with BLAS across several cores and batched queries it drops well under 100ms. At 100,000 vectors it is 300 MB and single-digit milliseconds. At 10,000 it is not worth discussing.
Two things move that line more than the vector count does. Dimensionality is linear: cutting 1536 dimensions to 768 halves both memory and query cost, which is what makes Matryoshka-style nested embeddings useful before you have any scaling problem at all. And query rate is what turns a comfortable 80ms into a queue: one query per second at 80ms is idle hardware, fifty per second is a capacity plan.
What An ANN Index Buys, And What It Costs
Approximate nearest neighbour indexes trade exactness for sublinear search. HNSW builds a navigable graph and answers in roughly logarithmic time; IVF partitions the space and searches only the nearest partitions. Both stop touching every vector, which is the entire point.
What you pay is threefold and only the first is usually discussed. Recall drops below 1.0, so some true nearest neighbours are missed and you will not know which. Build time and memory go up: HNSW commonly adds 30 to 60% memory overhead on top of the vectors themselves. And the index has parameters (
M, efConstruction, efSearch, or nlist and nprobe) whose defaults are chosen for benchmark corpora rather than yours.Measure recall before and after. Take a few hundred queries, compute exact top-k with brute force, then compare against what the index returns. If recall@10 is 0.98 the approximation is invisible to a user; if it is 0.80 you have quietly made search worse to fix a latency problem you may not have had.
The Four Reasons That Genuinely Force A Dedicated Store
Filtered search at scale. This is the most common real breaking point and it arrives earlier than raw size does. "Nearest neighbours where tenant = X and created_at > Y" is hard for an ANN index: filter first and you may have too few candidates left for the graph to traverse well, filter after and you may retrieve 100 results and have 3 survive. Dedicated stores implement filtered ANN deliberately. Brute force, ironically, handles filters perfectly, because you were touching everything anyway.
Writes while serving reads. HNSW graphs are expensive to mutate and many implementations degrade or lock under sustained inserts. If your corpus changes continuously rather than being rebuilt nightly, that is a property to shop for rather than assume.
Recall as a dial. Once relevance matters commercially, you need to be able to trade latency for recall deliberately and measure the result. That is a product feature, not a config file.
Memory you are not willing to buy. Beyond roughly 10M vectors at typical dimensions, keeping everything resident stops being casually affordable, and the question becomes which parts live in memory and which on disk or object storage. That is architecture, not tuning.
How The Options Compare
| Option | Exact | Filters | Scale before it hurts | Main limitation |
| NumPy brute force | Yes | Perfect, you scan anyway | ~100k-1M vectors | Rebuilds and persistence are yours to write |
| SQLite + brute force | Yes | Perfect | ~100k | Single writer, no concurrency story |
| pgvector, exact | Yes | Perfect, real SQL | ~100k-500k | Query cost grows linearly with the table |
| pgvector, HNSW | No | Good, SQL predicates | ~1M-10M | Index build is slow and memory-hungry |
| Dedicated vector DB | No | Purpose-built filtered ANN | 10M+ | A service to run, and another consistency boundary |
| Object-storage-backed | No | Varies | 100M+ | Cold reads pay a load; residency becomes your problem |
Why This Is Different When The Vectors Describe Media
Text corpora tend to have one vector per chunk. Media does not. A video decomposed into shots produces tens to hundreds of vectors per file; an image library with region-level features multiplies similarly. So a "small" catalogue of 20,000 videos is not 20,000 vectors, it is several million, and the threshold arrives while the file count still sounds trivial.
Media vectors are also usually wider. A visual encoder like DINOv3 or a retrieval-first text model at 1024 or 4096 dimensions costs proportionally more per vector, and the raw media sits in object storage while the vectors sit somewhere else, which makes keeping the two in sync a real job rather than an afterthought.
And media search is rarely pure vector search. A parsed invoice is matched by its number as often as by its meaning, which is the argument for hybrid keyword and vector retrieval and a reason the store needs to hold more than vectors.
How To Decide In An Afternoon
Take your real corpus, or a representative tenth of it. Embed it. Run brute-force search over a few hundred real queries and record the p95 latency and the exact top-k. That gives you both your baseline latency and your ground truth in one pass.
If p95 fits your budget with headroom for growth, stop. You do not have a vector database problem yet, and the exact top-k you just computed is the recall benchmark you will need on the day you do.
If it does not fit, add an ANN index and re-run the same queries. Compare recall@10 against the exact results and latency against the same budget. Now you are choosing with two numbers instead of a vendor comparison.
Doing This On Mixpeek
If the measurement above says you need a store, MVS is the standalone vector store, designed for vectors that live over object storage rather than beside a database. If the measurement says you also do not want to own the embedding and sync half, managed Mixpeek indexes the files already in your buckets and keeps the vectors current as they change. Vector store docs cover the shape of both, and pricing covers the difference. If you are still comparing options, best vector databases is the vendor-neutral list.
Frequently Asked Questions
At exactly how many vectors do I need a vector database?
There is no single number, and any source that gives you one is guessing. The threshold is vectors x dimensions x query rate against your latency budget. As a rough orientation at 768 dimensions on ordinary hardware: under 100k, brute force is comfortable; 100k to 1M, it depends on your query rate and how much latency you can spend; past a few million it is usually time. Measure your own p95 before believing any of those.
Is pgvector a real vector database?
For exact search, yes, and for many workloads it is the right answer precisely because it is not a separate system. With HNSW it becomes a genuine ANN store as well. Where it stops being enough is sustained high write rates concurrent with reads, very large indexes where build time and memory become operational problems, and workloads needing fine-grained control over the recall-latency trade.
Does an approximate index make my search worse?
It makes it inexact, which is not the same thing. At recall@10 of 0.98 no user will notice. The danger is not the approximation, it is shipping it unmeasured: default index parameters are tuned for benchmark datasets, and the only way to know what yours cost you is to compare against exact results on your own queries.
What if my vectors do not fit in memory?
That is the point where the architecture question replaces the tuning question. Options are disk-backed indexes, quantization (which trades recall for a large memory saving), sharding across machines, or an object-storage-backed store that loads partitions on demand. Each moves the cost somewhere different; none makes it disappear.
Do I need a vector database if I am using hybrid search?
Hybrid needs a keyword index and a vector index, and they do not have to live in the same system. A common and perfectly reasonable setup is Postgres full-text plus pgvector in one database. You need a dedicated store when the vector half outgrows what sits beside your keyword half, not because you added BM25.