Filtered Vector Search: Why the Same Filter Is Free or Fatal

Adding a filter to a vector search looks like adding a WHERE clause to SQL. It isn't.

The same filter either costs you nothing or destroys your results, and the API call is identical either way. What decides it is selectivity: rows matching the filter divided by total rows. A tenant filter across 10,000 tenants sits near 0.0001. A filter on language equals English sits near 0.7.

Three strategies, each correct somewhere and silently wrong somewhere else.

Post-filter searches the whole index and drops failures afterward. Simple and fast when the filter is weak. At selectivity 0.0001 you'd need roughly 200,000 candidates to expect 20 survivors, which isn't approximate search anymore.

Pre-filter resolves the filter first and searches inside the matching set. Exact by construction. When most rows pass you've built a near-full scan and thrown away the index you paid for.

In-place predicate-aware traversal evaluates the filter during the graph walk, routing through failing nodes instead of stopping at them. Filtered DiskANN and ACORN-style expansion both work this way. It holds across the whole range, and it breaks quietly when the graph fragments: matching nodes end up in islands the walk never reaches.

So the right mental model is a planner, not a favourite. Estimate selectivity per query the way a SQL optimiser does.

The part that catches people is that none of this errors. Recall falls from 95% to 30% and everything looks fine. You only see it if you measure recall against an exhaustive answer, with the filter on.

And if a predicate is always present, like tenant, it belongs in the index layout rather than the query. Separate graphs per tenant turn the hardest case into no filtering at all.

mxp.co/d/filtered-vector-search

[image: filtered-vector-search.png]