Why can't my search find exact part numbers, codes or names?
Because vector search ranks by meaning, and an identifier has very little of it. An embedding model turns "AB-4471-X" into the same kind of vector it makes for a sentence, built from the fragments a tokenizer cut the string into, so "AB-4417-X" and "AB-4471-Y" land almost on top of it. The record you wanted is usually in the index, ranked below its look-alikes. The fix is a keyword search that scores literal tokens and rewards rare ones, run beside the vector search and merged with it by rank. When the identifier sits in a structured field, an exact filter on that field is simpler still.
How do I know this is the problem?
Run one test before you change anything. Copy an identifier out of a record you know is indexed (a SKU, an order number, an error code, a customer surname) and search for exactly that string. If near-miss records fill the first page and the one you copied from is missing, this is your failure. If the record never shows up at any depth, check ingestion first, since no ranking change can surface a document that was never indexed.
Other signs that point the same way:
E1034 returns general troubleshooting pages instead of the one page that names it.Why does vector search miss exact matches?
The model reads a code as fragments
Embedding models never see whole identifiers. A tokenizer first splits the input into subword pieces, and the model builds one vector from those pieces. Two codes that differ by two digits share most of their pieces, so their vectors sit close together. The digits that make it a different part barely move the result, and nothing in training taught the model to treat them as decisive.
Names and new terms have little meaning to learn from
A model learns what a word means from the contexts it saw during training. A rare surname, an internal project name or a product launched last month appeared rarely or never. Its vector falls back on spelling, so the nearest neighbors are strings that look alike.
The identifier is one token in a long chunk
A chunk of a few hundred words gets one vector, and that vector mostly represents what the chunk is about. A part number mentioned once contributes a small share of it. A query made only of that part number matches chunks on the same topic, which may or may not include the chunk that contains it.
The query and the record are formatted differently
Records store "AB-4471-X" while people type "ab4471x" or "AB 4471 X". This one hurts keyword search as much as vector search, which is why normalization is part of the fix below.
How do I fix it?
Add a keyword search next to the vector search
BM25, the ranking function behind most keyword search, scores a document by the query terms it contains and weights each term by how rare it is across the collection. Rarity is what matters here. A part number appears in one or two records, so a match on it scores high, while a common word scores low. An embedding treats the same string the other way around. How BM25 and the inverted index work walks through the formula.
Run both searches over the same content. The vector search keeps handling descriptions and paraphrases, and the keyword search catches literal tokens.
Merge the two result lists by rank
The two searches score on different scales. Cosine similarity sits in a narrow band, and BM25 scores are unbounded and shift from query to query, so adding them lets one side drown the other. Reciprocal rank fusion (RRF) ignores the scores and uses positions: every document earns 1 / (k + rank) from each list it appears in, and the totals decide the final order. The original paper by Cormack, Clarke and Buettcher used k = 60. Each list gets an equal say, so an exact keyword hit can no longer be buried by the vector ordering, and documents both searches rank well rise to the top. Hybrid search fusion compares RRF with weighted score fusion in depth.
Use an exact filter when the identifier has its own field
If part numbers, order IDs or employee IDs are stored in a structured field, finding one needs no ranking at all. Filter that field for equality and return what matches. Many teams route on the shape of the query: a string that matches the pattern of an ID goes to the exact filter first, and everything else goes to hybrid search.
Normalize identifiers the same way on both sides
Strip spaces, hyphens and case from identifiers when you index them, keep the normalized copy in its own field, and run the same function on identifier-shaped queries. "AB-4471-X", "ab4471x" and "AB 4471 X" then become one value for the filter and for the keyword index.
Which approach finds which query?
| Query | Vector search only | Keyword (BM25) only | Exact filter on a field | Vector and keyword, merged by rank |
AB-4471-X (part number) | Returns look-alike codes | Finds it | Finds it | Finds it |
ab4471x (same part, typed differently) | Returns look-alike codes | Finds it only if identifiers are normalized | Finds it only if identifiers are normalized | Finds it only if identifiers are normalized |
E1034 timeout (error string) | Returns general troubleshooting pages | Finds the page that names it | Only if the code is stored as a field | Finds it, with related pages below |
| A customer surname | Returns similar-looking names | Finds exact spellings | Finds it if names are a field | Finds it |
shoe for wet trails (description) | Finds relevant products | Needs words the products share | Does not apply | Finds relevant products |
recieving dock schedule (typo) | Usually tolerates the typo | Misses the misspelled word | Does not apply | The vector side still finds it |
How do I check which fix I need?
1. Collect twenty or thirty real queries that failed, from search logs or support tickets. 2. Label each one as an identifier, a name, an error or quoted text, a description, or a misspelling. 3. For the identifiers, check whether the value is stored in a field of its own. Where it is, an exact filter covers those queries. 4. Run the remaining failures against a keyword index on its own. The queries it fixes are the ones hybrid search will fix. 5. Keep the descriptive queries in the test set and confirm they still return what they did before the keyword side was added.
How does this work in Mixpeek?
Keyword and vector search run inside one retriever stage. A
feature_search
stage takes a list of searches, and a search with lexical: true runs BM25
against the namespace's full-text index instead of querying vectors. List the same
feature twice, once as a vector search and once as a lexical search, and fuse them
with rrf:{
"stage_name": "hybrid_search",
"stage_type": "filter",
"config": {
"stage_id": "feature_search",
"parameters": {
"fusion": "rrf",
"final_top_k": 25,
"searches": [
{
"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
"query": { "input_mode": "text", "value": "{{INPUT.query}}" },
"top_k": 100
},
{
"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
"query": { "input_mode": "text", "value": "{{INPUT.query}}" },
"lexical": true,
"top_k": 100
}
]
}
}
}The keyword side reads a full-text index. Declare
text payload indexes on the
fields that hold your identifiers and names, then register them. Registering
backfills existing documents without re-extracting anything:import os
import requests
NAMESPACE = "product-search"
HEADERS = {"Authorization": f"Bearer {os.environ['MIXPEEK_API_KEY']}"}
requests.patch(
f"https://api.mixpeek.com/v1/namespaces/{NAMESPACE}",
headers=HEADERS,
json={"payload_indexes": [
{"field_name": "title", "type": "text"},
{"field_name": "description", "type": "text"},
]},
)
requests.post(
f"https://api.mixpeek.com/v1/namespaces/{NAMESPACE}/ensure-indexes",
headers=HEADERS,
)When the identifier is its own field, an
Two limits are worth knowing before you rely on this. BM25 matches across every
The full-text index lives in the same namespace as the vectors in Mixpeek Vector Store, so the keyword side needs no separate search engine. Pricing covers what search and storage cost, and the feature search reference lists every search option.
Rarely. A larger model represents meaning better, and an identifier still has little meaning to represent. It still builds its vector from subword fragments, so codes that share most of their characters stay close together at any model size. A keyword search beside the vector search fixes the problem whatever model you use.
No. Keyword search misses paraphrases, synonyms and misspellings, and those are the queries vector search handles well. Dropping the vector side swaps one set of failures for another. Run both and merge the results by rank.
It merges ranked lists by position. Each document collects points for where it appears in each list, more for a higher position, and the totals set the final order. It never compares raw scores, so the difference in scale between keyword and vector scores stops mattering.
Keyword search matches whole tokens, and a misspelled word is a different token that appears in no document. The subword fragments of a misspelling overlap heavily with those of the correct spelling, so the two vectors stay close and the vector side still finds the content.
Yes, when the part number is stored in its own field and the query is exactly that value. A filter is the fastest and most precise option for that case. It cannot find a part number mentioned inside a description, a manual or a transcript, and that is the case keyword search covers.
What Is Hybrid Search? BM25, Vector Retrieval, and How to Fuse Their Rankings -- the technique behind the fix, end to end
Hybrid Search Fusion: How to Combine Dense and Lexical Retrieval Without Breaking Ranking -- rank fusion against weighted scores
BM25 and the Inverted Index -- how the keyword side scores a document
Learned Sparse Retrieval and Dense-Sparse Hybrid -- when a learned model replaces plain BM25
Why Do My Search Results Get Worse as I Add More Data? -- another reason content you know is indexed stops surfacing
Why Does My Search Find the Right Document but the Wrong Part of It? -- when the document matches and the passage does not
Best Hybrid Search Engines -- products that run keyword and vector search together
Glossary: BM25 and hybrid search
Mixpeek docs: text indexes for BM25 and the attribute filter stage
attribute_filter stage with the eq
operator matches it exactly, for example metadata.part_number equal to
AB-4471-X.Two limits are worth knowing before you rely on this. BM25 matches across every
text-indexed field at once and cannot be scoped to a single field; for one
field on its own, use an attribute_filter with contains (a match, without
relevance ranking) or give that field its own text embedding. RRF also ignores
how strong each match was, so a weak keyword hit and a perfect one at the same
rank count the same. Keep each search's top_k deep enough that the fusion has
real candidates to choose from.The full-text index lives in the same namespace as the vectors in Mixpeek Vector Store, so the keyword side needs no separate search engine. Pricing covers what search and storage cost, and the feature search reference lists every search option.
Frequently asked questions
Will a bigger embedding model find exact part numbers?
Rarely. A larger model represents meaning better, and an identifier still has little meaning to represent. It still builds its vector from subword fragments, so codes that share most of their characters stay close together at any model size. A keyword search beside the vector search fixes the problem whatever model you use.
Should I replace vector search with keyword search?
No. Keyword search misses paraphrases, synonyms and misspellings, and those are the queries vector search handles well. Dropping the vector side swaps one set of failures for another. Run both and merge the results by rank.
What does reciprocal rank fusion do, in plain terms?
It merges ranked lists by position. Each document collects points for where it appears in each list, more for a higher position, and the totals set the final order. It never compares raw scores, so the difference in scale between keyword and vector scores stops mattering.
Why does keyword search fail on typos when vector search does not?
Keyword search matches whole tokens, and a misspelled word is a different token that appears in no document. The subword fragments of a misspelling overlap heavily with those of the correct spelling, so the two vectors stay close and the vector side still finds the content.
Can I filter on the part number instead of adding keyword search?
Yes, when the part number is stored in its own field and the query is exactly that value. A filter is the fastest and most precise option for that case. It cannot find a part number mentioned inside a description, a manual or a transcript, and that is the case keyword search covers.