The Short Answer
Hybrid search runs a keyword retriever and a vector retriever over the same corpus and merges their two result lists into one ranking. Keyword retrieval, almost always BM25, matches the literal terms in a query. Vector retrieval matches meaning, so it finds documents that never contain the query words. Each one fails on the queries the other handles well, which is why production systems run both.
The hard part is the merge. BM25 produces an unbounded relevance score whose scale shifts with the corpus and the query, while a vector index returns a cosine similarity bounded in a fixed range. Adding those two numbers together is meaningless, because a BM25 score of 18 and a cosine of 0.81 are not comparable quantities. The standard fix is Reciprocal Rank Fusion, which throws the scores away and combines the RANKS instead.
Why Neither Retriever Is Enough On Its Own
Lexical and semantic retrieval fail on opposite inputs, and the failures are structural rather than a matter of tuning.
BM25 cannot match a document that uses different words. Search for "car" and a document that only says "automobile" scores zero on that term. This is the vocabulary mismatch problem, and it is why pure keyword systems feel brittle to users who describe things in their own words.
Vector retrieval has the mirror weakness. Embeddings compress a passage into a few hundred floats, and rare tokens are exactly what gets compressed away. Part numbers, SKUs, error codes, surnames, ticket IDs and version strings are the tokens a user is most likely to paste in verbatim, and they are the ones a dense retriever is worst at. Ask for error code E4172 and a semantic index will happily return passages about error codes in general.
So the split is: BM25 owns exact tokens and rare terms, embeddings own paraphrase and intent. If you are choosing the embedding side, the multimodal embedding model comparison and the vector database comparison cover the two decisions that follow. A query log contains both kinds, usually mixed inside single queries.
What BM25 Actually Computes
BM25 is a bag-of-words scoring function. For a query Q against document D:
score(D, Q) = sum over terms qi in Q of:
IDF(qi) * ( f(qi, D) * (k1 + 1) )
---------------------------------------------
( f(qi, D) + k1 * (1 - b + b * |D| / avgdl) )Three pieces are doing the work.
IDF weights rare terms above common ones. A term appearing in almost every document carries almost no information, so it contributes almost nothing.
Term frequency saturation is the k1 parameter, and it is the part people miss. The numerator and denominator both grow with f(qi, D), so the ratio approaches a ceiling. A document mentioning your term forty times does not score twenty times higher than one mentioning it twice. Early TF-IDF variants had no ceiling, which made them easy to spam by repetition. Typical k1 sits around 1.2.
Length normalization is b, usually 0.75. Long documents accumulate term matches by being long, so the denominator scales with the document's length over the collection average. Setting b to 0 disables the correction and setting it to 1 applies it fully.
Note what is absent: word order, syntax, and any notion of a term's meaning. BM25 is a well-calibrated counter of rare words. The semantic half it pairs with is covered in late interaction retrieval, where per-token vectors recover some of the exactness BM25 gets for free.
Once both retrievers return their lists, you have two rankings with incompatible score scales. BM25 scores have no upper bound and their magnitude depends on corpus statistics and query length, so a "good" score for one query may be a poor one for another. Cosine similarity is bounded, and dense retrievers tend to squash their top results into a narrow band where the first and tenth hit differ by hundredths.
That mismatch breaks the obvious approaches:
Raw addition lets whichever score happens to be numerically larger dominate. Usually that is BM25, so the vector half quietly stops contributing.
Min-max normalization rescales each list into 0 to 1 using that query's own minimum and maximum. It makes the numbers comparable, but it is computed per query over the retrieved window, so the same document can normalize differently depending on what else came back. It also stretches noise: if every result is bad, min-max still promotes the least bad one to 1.0.
Reciprocal Rank Fusion discards the scores entirely and uses position:
IDF weights rare terms above common ones. A term appearing in almost every document carries almost no information, so it contributes almost nothing.
Term frequency saturation is the k1 parameter, and it is the part people miss. The numerator and denominator both grow with f(qi, D), so the ratio approaches a ceiling. A document mentioning your term forty times does not score twenty times higher than one mentioning it twice. Early TF-IDF variants had no ceiling, which made them easy to spam by repetition. Typical k1 sits around 1.2.
Length normalization is b, usually 0.75. Long documents accumulate term matches by being long, so the denominator scales with the document's length over the collection average. Setting b to 0 disables the correction and setting it to 1 applies it fully.
Note what is absent: word order, syntax, and any notion of a term's meaning. BM25 is a well-calibrated counter of rare words. The semantic half it pairs with is covered in late interaction retrieval, where per-token vectors recover some of the exactness BM25 gets for free.
The Fusion Problem
Once both retrievers return their lists, you have two rankings with incompatible score scales. BM25 scores have no upper bound and their magnitude depends on corpus statistics and query length, so a "good" score for one query may be a poor one for another. Cosine similarity is bounded, and dense retrievers tend to squash their top results into a narrow band where the first and tenth hit differ by hundredths.
That mismatch breaks the obvious approaches:
Raw addition lets whichever score happens to be numerically larger dominate. Usually that is BM25, so the vector half quietly stops contributing.
Min-max normalization rescales each list into 0 to 1 using that query's own minimum and maximum. It makes the numbers comparable, but it is computed per query over the retrieved window, so the same document can normalize differently depending on what else came back. It also stretches noise: if every result is bad, min-max still promotes the least bad one to 1.0.
Reciprocal Rank Fusion discards the scores entirely and uses position:
RRF_score(d) = sum over each retriever r of: 1 / (k + rank_r(d))
A document ranked first by BM25 and fiftieth by the vector index gets 1/(k+1) + 1/(k+50). The constant k, set to 60 in the original paper by Cormack, Clarke and Buettcher, damps the influence of the very top positions so a single retriever cannot unilaterally decide the final order.
RRF wins in practice for a specific reason: rank is the one thing both retrievers produce on the same scale. It needs no per-corpus calibration and no training data, and it degrades gently when one retriever returns garbage, because garbage lands at low ranks where the reciprocal is small.
The cost is that RRF is scale-blind in both directions. It cannot tell a run where the top hit is a perfect match from one where everything is mediocre, since only the ordering survives. When you have labeled relevance data, a tuned convex combination of normalized scores, alpha times dense plus one minus alpha times sparse, can beat RRF. Without labels, RRF is the safer default. Getting those labels is its own exercise, covered in evaluating multimodal retrieval, and the reason raw scores mislead is in calibrating similarity scores.
Hybrid is a default worth questioning, and two cases argue against it.
When your queries are overwhelmingly one type, the second retriever adds latency and cost for very little recall. A codebase search where every query is an identifier is a BM25 problem. Recommendation-style retrieval with no text query at all is a vector problem.
The subtler case is that fusion can demote a document both retrievers ranked moderately well beneath one that a single retriever loved. That is usually the desired behavior, and occasionally it is not, particularly for known-item search where the user is trying to re-find one specific document they have seen before.
Measure it. Run the same query set through BM25 alone, vectors alone, and the fusion, and compare recall at your actual cutoff. The result is corpus-specific and the intuition transfers poorly.
The BM25 half assumes a document is a bag of words. Video, images and audio have no words until something produces them, which makes hybrid search over multimodal content a different construction.
In practice the lexical side runs over generated text: transcripts from speech recognition, detected on-screen text from OCR, object and scene labels, and human-entered metadata like titles and rights fields. The semantic side runs over embeddings of the frames or segments themselves. A query for "the shot where someone says quarterly earnings" wants the transcript index; "a wide shot of a factory floor" wants the visual embedding, since nobody ever wrote those words down.
This is where the granularity question matters more than in text search. A one hour video is not a document. Matching at file level tells a user the clip is somewhere in the next hour, so the useful unit is the scene or the segment, and both halves of a hybrid index need to agree on that unit for their ranks to be fusable at all. Mixpeek indexes at token and segment granularity over object storage for this reason, so a lexical hit on a transcript line and a vector hit on a frame refer to the same addressable moment. You can run that yourself two ways: Mixpeek Vector Store if you already have vectors and want agent-native search over object storage, or the managed pipeline if you want the extraction, embedding and indexing handled. Pricing covers both, and the model catalog lists the encoders available on each side.
No. It is better on mixed query loads, which is most real applications, because it recovers exact-token queries that embeddings compress away. If your traffic is entirely paraphrase-style natural language, the lexical half contributes little and costs latency. The honest answer is that it depends on your query distribution, which you can measure from your own logs.
Start at 60, the value from the original paper, and change it only with evidence. Smaller k sharpens the influence of top-ranked results, larger k flattens the contribution across positions. It is a damping constant rather than a relevance parameter, so it rarely repays heavy tuning.
No. Most vector databases and search engines now run both sides in one system, including Elasticsearch, OpenSearch, Vespa, Weaviate and Qdrant. Running two stores is a legitimate choice when you already operate a mature lexical index, but it makes consistency between the two your problem.
They compose rather than compete. Hybrid fusion produces a candidate set from two cheap retrievers. A reranker then scores a small number of those candidates with an expensive cross-encoder that reads query and document together. Fusion is about recall, reranking is about precision on what recall produced. See cross-encoder reranking for the second stage and multi-stage retrieval for how the stages fit together.
Only through text that describes it. There are no terms to count in raw pixels, so the lexical half operates on transcripts, OCR output, labels and metadata. The visual content itself is reachable through embeddings.
RRF wins in practice for a specific reason: rank is the one thing both retrievers produce on the same scale. It needs no per-corpus calibration and no training data, and it degrades gently when one retriever returns garbage, because garbage lands at low ranks where the reciprocal is small.
The cost is that RRF is scale-blind in both directions. It cannot tell a run where the top hit is a perfect match from one where everything is mediocre, since only the ordering survives. When you have labeled relevance data, a tuned convex combination of normalized scores, alpha times dense plus one minus alpha times sparse, can beat RRF. Without labels, RRF is the safer default. Getting those labels is its own exercise, covered in evaluating multimodal retrieval, and the reason raw scores mislead is in calibrating similarity scores.
Choosing Between The Merge Strategies
| Strategy | What it needs | Where it holds up | Main limitation |
| Raw score addition | Nothing | Nowhere in practice | Scales are incomparable, so one retriever silently dominates |
| Min-max normalization | Per-query min and max | Small, uniform corpora | Rescales noise as confidently as signal; unstable across queries |
| Reciprocal Rank Fusion | Ranks only | General purpose, no labels | Blind to score magnitude, so it cannot express "nothing here is good" |
| Weighted convex combination | Labeled relevance data | Tuned single-domain search | Needs judgments, and drifts as the corpus changes |
When Hybrid Search Is Worse Than One Retriever
Hybrid is a default worth questioning, and two cases argue against it.
When your queries are overwhelmingly one type, the second retriever adds latency and cost for very little recall. A codebase search where every query is an identifier is a BM25 problem. Recommendation-style retrieval with no text query at all is a vector problem.
The subtler case is that fusion can demote a document both retrievers ranked moderately well beneath one that a single retriever loved. That is usually the desired behavior, and occasionally it is not, particularly for known-item search where the user is trying to re-find one specific document they have seen before.
Measure it. Run the same query set through BM25 alone, vectors alone, and the fusion, and compare recall at your actual cutoff. The result is corpus-specific and the intuition transfers poorly.
What Changes When The Corpus Is Not Text
The BM25 half assumes a document is a bag of words. Video, images and audio have no words until something produces them, which makes hybrid search over multimodal content a different construction.
In practice the lexical side runs over generated text: transcripts from speech recognition, detected on-screen text from OCR, object and scene labels, and human-entered metadata like titles and rights fields. The semantic side runs over embeddings of the frames or segments themselves. A query for "the shot where someone says quarterly earnings" wants the transcript index; "a wide shot of a factory floor" wants the visual embedding, since nobody ever wrote those words down.
This is where the granularity question matters more than in text search. A one hour video is not a document. Matching at file level tells a user the clip is somewhere in the next hour, so the useful unit is the scene or the segment, and both halves of a hybrid index need to agree on that unit for their ranks to be fusable at all. Mixpeek indexes at token and segment granularity over object storage for this reason, so a lexical hit on a transcript line and a vector hit on a frame refer to the same addressable moment. You can run that yourself two ways: Mixpeek Vector Store if you already have vectors and want agent-native search over object storage, or the managed pipeline if you want the extraction, embedding and indexing handled. Pricing covers both, and the model catalog lists the encoders available on each side.
Frequently Asked Questions
Is hybrid search always better than vector search alone?
No. It is better on mixed query loads, which is most real applications, because it recovers exact-token queries that embeddings compress away. If your traffic is entirely paraphrase-style natural language, the lexical half contributes little and costs latency. The honest answer is that it depends on your query distribution, which you can measure from your own logs.
What value of k should I use in RRF?
Start at 60, the value from the original paper, and change it only with evidence. Smaller k sharpens the influence of top-ranked results, larger k flattens the contribution across positions. It is a damping constant rather than a relevance parameter, so it rarely repays heavy tuning.
Do I need two separate databases for hybrid search?
No. Most vector databases and search engines now run both sides in one system, including Elasticsearch, OpenSearch, Vespa, Weaviate and Qdrant. Running two stores is a legitimate choice when you already operate a mature lexical index, but it makes consistency between the two your problem.
How is hybrid search different from reranking?
They compose rather than compete. Hybrid fusion produces a candidate set from two cheap retrievers. A reranker then scores a small number of those candidates with an expensive cross-encoder that reads query and document together. Fusion is about recall, reranking is about precision on what recall produced. See cross-encoder reranking for the second stage and multi-stage retrieval for how the stages fit together.
Can BM25 work on image or video content?
Only through text that describes it. There are no terms to count in raw pixels, so the lexical half operates on transcripts, OCR output, labels and metadata. The visual content itself is reachable through embeddings.