Three Ways to Score a Query Against a Document

Every relevance score in retrieval is produced by one of three architectures, and they differ on exactly one axis: when the query is allowed to see the document.

A bi-encoder never lets them meet until the very end. Query and document go through the encoder separately and come out as one vector each, and the score is the cosine between them. Because the document side is encoded before anyone knows what will be asked, the encoder has to guess what matters. Two sentences about EMEA inside a 500-word chunk get diluted by everything around them. What you buy for that loss is enormous: every document vector is computed once at index time, and search is an ANN lookup over the whole corpus.

A cross-encoder lets them meet immediately. Query and document are concatenated into one sequence and run through the transformer together, so every query token attends to every document token at every layer. That is where term weighting and negation handling come from, and it is worth roughly 5 to 15% on standard benchmarks. It also means there is nothing to precompute: the cost is quadratic in sequence length per pair, and scoring a million documents at 5ms each is 83 minutes per query. A cross-encoder can only ever be a reranker over a shortlist someone else produced.

Late interaction is the third option and the one people skip. Both sides are encoded independently, so the document side is still precomputable, but you keep a vector per token instead of collapsing to one. At query time each query token takes its best match among the document's token vectors and those maxima are summed. That is MaxSim. The interaction is real and it happens at query time, but it happens over cached vectors rather than a forward pass.

The bill is storage. One vector per document becomes hundreds, which is why the multi-vector literature is largely about making that affordable: pooling, projection down to smaller dimensions, quantization, and fixed-length encodings that let you use ordinary ANN infrastructure. That is the problem MUVERA and the ColBERT/ColPali line are solving, and this trichotomy is the thing they assume you already know.