Agentic Retrieval: What Changes When the Caller Is Not a Person

A person types "valve maintenance schedule".

An agent sends you this:

"the user asked about downtime. i already checked the ticket log and found nothing after march. tool output: 3 rows returned, status ok. i now need the valve maintenance schedule so i can compare it against the outage window."

The question is in there. So is prior context, tool output and status chatter. Nothing marks which span is the query, and the embedding has to represent all of it at once.

Every retrieval benchmark is built from queries that look like the first one. Which is why a system that scores well can still disappoint inside an agent loop.

A single fixed-length vector has to average the question together with everything around it, so the noise drags the representation away from what you actually wanted. Late interaction keeps one vector per token and scores each query token against its best match in the document, so noise tokens score low on their own and contribute almost nothing. If 40 percent of the query tokens are noise, that difference is the whole ballgame. The trade is storage and compute: a 512-token query becomes 512 vectors of 128 dimensions instead of a single 768-dim vector.

The other half is that the query is never alone. An agent researching a topic finds an overview, chases a specific claim with terms it didn't have before, looks for a second source that agrees or disagrees, and then decides it has enough. Nothing in a retrieval API tells it when that's true.

The AgentIR benchmark, published 2026, measures retrieval quality when queries come from reasoning traces instead of clean human queries. That's the workload worth measuring, because it's the one you have.

And budget the loop rather than the call. An agent will issue ten searches where a person issued one. Total steps to an answer matters more than latency per call, and no single-query benchmark measures it.

mxp.co/d/agentic-retrieval

[image: agentic-retrieval.png]