Why does my AI agent's search ignore the filter it chose?
Usually the filter never reaches the vector search. The model writes the constraint into its tool call, for example brand equals acme, but the search tool's schema has no filters argument, or the code that turns the call into a database query drops it. The search then runs over every document and returns other brands' or other customers' results with no error. The second common cause is an agent that searches several times in one turn, where each result list replaces the one before. The fix: put filters in the tool's schema with the allowed fields listed, pass them to the database's own metadata filter, refuse a field the index does not have so the model can try again, enforce access filters in code the model cannot change, and merge results across searches.
Where does the agent's filter get lost?
There are four places between the model and the index, and none of them raises an error:
1. The tool schema has no filters argument. The model can only send what the schema describes. If the tool takes a query string alone, the model either puts "acme" into the query text, where the embedding treats it as one more word, or leaves it out. 2. The adapter drops or renames it. The model sends a filter and the glue code builds the search without it, or the model writes a field called client while the index stores brand. A filter on a field the index does not have matches nothing or is ignored, depending on the database. 3. The database filters after ranking. A post-filter takes the top k by similarity and then removes what fails the filter, so a selective filter leaves few or zero results. See why vector search returns fewer results when you add a filter. 4. A later search replaces an earlier one. An agent that runs one search per sub-question and keeps only the last list loses everything the earlier searches found, while its summary still says every sub-question was answered.
Reproduce it in plain Python
This script builds a small index of 400 ad descriptions from four brands, then runs the same tool call ("summer sale" with brand equals acme) through a tool that drops filters and one that applies them. It needs no packages and no model: the tool call is fixed, because the bug lives in the tool layer.
"""Where an agent's filter gets lost between the model and the vector search.
Stdlib only. The "model" is a fixed tool call, so the run is deterministic: the
point is what the tool layer does with the arguments the model sends.
"""
import json
import math
import random
random.seed(7)
BRANDS = ["acme", "globex", "initech", "umbrella"]
TOPICS = ["summer sale", "holiday gift", "back to school", "new product launch"]
def unit(v):
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
TOPIC_VECS = {t: unit([random.gauss(0, 1) for _ in range(32)]) for t in TOPICS}
def embed(topic):
base = TOPIC_VECS[topic]
return unit([x + random.gauss(0, 0.08) for x in base])
DOCS = [
{"id": f"{b}-{t.replace(' ', '-')}-{i}", "brand": b, "topic": t, "vec": embed(t)}
for b in BRANDS for t in TOPICS for i in range(25)
]
def cos(a, b):
return sum(x * y for x, y in zip(a, b))
def vector_search(query_vec, k=10, filters=None):
"""filters: list of {field, operator, value}; only 'eq' here."""
pool = DOCS
for f in filters or []:
if f["field"] not in ("brand", "topic") or f["operator"] != "eq":
raise ValueError(f"unsupported filter {f}")
pool = [d for d in pool if d[f["field"]] == f["value"]]
return sorted(pool, key=lambda d: -cos(query_vec, d["vec"]))[:k]
# The model read "acme's summer sale ads" and chose this tool call.
TOOL_CALL = {"query": "summer sale", "filters": [{"field": "brand", "operator": "eq", "value": "acme"}]}
def tool_without_filters(args):
# Schema had no filters argument: the adapter passes the query and drops the rest.
return vector_search(embed(args["query"]))
def tool_with_filters(args):
return vector_search(embed(args["query"]), filters=args.get("filters"))
def off_brand(results, brand="acme"):
return sum(d["brand"] != brand for d in results)
a = tool_without_filters(TOOL_CALL)
b = tool_with_filters(TOOL_CALL)
print(f"filters dropped: {off_brand(a)} of {len(a)} results are another brand's")
print(f"filters applied: {off_brand(b)} of {len(b)} results are another brand's")
# Second bug: two searches in one turn, the second overwrites the first.
calls = [
{"query": "summer sale", "filters": [{"field": "brand", "operator": "eq", "value": "acme"}]},
{"query": "holiday gift", "filters": [{"field": "brand", "operator": "eq", "value": "acme"}]},
]
kept = []
for c in calls:
kept = tool_with_filters(c) # overwrite
overwrite_topics = sorted({d["topic"] for d in kept})
runs = [tool_with_filters(c) for c in calls]
merged, seen = [], set()
for rank in range(max(len(r) for r in runs)): # rank 1 of each run, then rank 2, ...
for r in runs:
if rank < len(r) and r[rank]["id"] not in seen:
seen.add(r[rank]["id"])
merged.append(r[rank])
print(f"overwrite: topics kept {overwrite_topics}")
print(f"merge by rank: topics kept {sorted({d['topic'] for d in merged})}, {len(merged)} results")
# Third: a filter the model invents is refused, so the model can correct it.
try:
tool_with_filters({"query": "summer sale", "filters": [{"field": "client", "operator": "eq", "value": "acme"}]})
except ValueError as e:
print(f"bad field: refused -> {json.dumps(str(e))}")filters dropped: 8 of 10 results are another brand's
filters applied: 0 of 10 results are another brand's
overwrite: topics kept ['holiday gift']
merge by rank: topics kept ['holiday gift', 'summer sale'], 20 results
bad field: refused -> "unsupported filter {'field': 'client', 'operator': 'eq', 'value': 'acme'}"How do I give the search tool a filters argument?
Describe the filters in the tool's JSON schema and list the fields the index can filter on as an enum, so the model cannot invent one. With OpenAI function calling, set strict to true: the model's arguments then follow the schema, which requires additionalProperties false on every object and every property listed as required (a null type makes one optional).
{
"type": "function",
"name": "search_ads",
"description": "Search ad descriptions. Use filters for exact constraints such as brand or year.",
"strict": true,
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "What the ad is about, without the constraints"},
"filters": {
"type": ["array", "null"],
"items": {
"type": "object",
"properties": {
"field": {"type": "string", "enum": ["brand", "topic", "year"]},
"operator": {"type": "string", "enum": ["eq", "gte", "lte"]},
"value": {"type": "string"}
},
"required": ["field", "operator", "value"],
"additionalProperties": false
}
}
},
"required": ["query", "filters"],
"additionalProperties": false
}
}How do I turn the tool call into a database filter?
Translate each item into the database's native filter and pass it to the search call, so the index applies it during the search. Two common translations of brand equals acme:
# Pinecone: metadata filter on the query
index.query(vector=qvec, top_k=10, filter={"brand": {"$eq": "acme"}})
# Qdrant: a must condition on the payload
from qdrant_client import models
client.query_points(
collection_name="ads",
query=qvec,
limit=10,
query_filter=models.Filter(must=[
models.FieldCondition(key="brand", match=models.MatchValue(value="acme")),
]),
)Should the model choose the tenant or permission filter?
No. Treat the model's filters as relevance hints and access rules as code. Read the user's tenant, team or permissions from the session, add that filter in the tool itself, and combine it with whatever the model sends using AND. The model never sees it and cannot remove or widen it, so a prompt that talks the model out of its filter cannot reach another tenant's data.
What should happen when the agent searches more than once?
Keep every search's results and merge them. Two simple merges:
If a later step should narrow the set, make that an explicit filter step over the merged list, so results narrow only when the agent asks for it.
How does Mixpeek handle this?
Mixpeek is a multimodal data warehouse: it indexes video, images, audio and documents where they sit in object storage and searches them with retrievers built from stages. In a retriever's agent_search stage, the model's feature_search tool takes filters as a list of field, operator and value, and they are applied inside the vector search. A filter the index cannot apply fails that tool call, and the model sees the error. Filters set on the agent_search stage itself apply to every search the agent runs, under the model's own filters. Results from every search are merged by rank with each document once, and an attribute_filter step narrows the merged list when the agent needs to. This shipped on November 16, 2026; see the changelog entry and the agent search stage docs.
Frequently Asked Questions
Why does my RAG agent return documents from the wrong customer?
Because the customer constraint never reached the vector search. Either the search tool has no filters argument, the adapter drops it, or the model wrote a field name the index does not store. Access filters such as the customer or tenant should never depend on the model: add them in the tool code from the session. In the script on this page, dropping one brand filter returned 8 results of 10 from other brands.
What is a self-querying retriever?
A retriever that asks a language model to split a natural-language request into a search query and structured metadata filters, then runs the query with those filters. It needs a description of the metadata fields (name, type, meaning) in the prompt. LlamaIndex calls its version the auto-retriever; it returns a query string plus a list of filters, each a key, operator and value.
Can I put the filter in the query text instead?
It does not enforce anything. An embedding of "acme summer sale" leans toward documents that mention acme, and documents from other brands that are similar enough still rank above acme's. A hard constraint needs a metadata filter that the index applies.
How do I stop the model from inventing filter fields?
List the allowed fields as an enum in the tool's JSON schema, turn on strict schema adherence where your provider has it, and refuse any other field in the tool code with an error that names the allowed ones. The model reads the error and calls the tool again with a valid field.
Why does my agent get fewer results when it adds a filter?
The database is probably filtering after ranking: it takes the top k nearest vectors and then drops the ones that fail the filter. With a filter that matches 1% of documents, a top 10 can come back empty. Use the database's filtered search, or see why vector search returns fewer results when you add a filter.
Should each agent search replace the previous results?
No. Keep each search's results and merge them by rank or with reciprocal rank fusion. Replacing them keeps only the last search, and an agent that answered three sub-questions returns the evidence for one.
Key Takeaways
Sources
Where to go next
Filtered vector search strategies · agentic retrieval · multi-stage retrieval · best vector databases · MVS, vector search on object storage · pricing