Anatomy of a Retriever Pipeline

What is a retriever pipeline?

Most search APIs give you one thing: vector similarity. You pass a query, get back the top-K nearest embeddings. Done.

But real retrieval is never one step. After the similarity call you still need to filter by metadata, rerank with a cross-encoder for precision, deduplicate near-identical results, enrich with context, and reduce to the right shape. That logic lives in your application code — fragile, hard to measure, impossible to iterate without redeploying.

A retriever pipeline makes each step explicit and composable.

You define stages in JSON. Each stage processes the output of the previous one:

1. Search — vector similarity across dense, sparse, or multi-vector embeddings

2. Filter — metadata predicates, LLM-based semantic filtering, deduplication

3. Rank — cross-encoder reranking, MMR for diversity, score normalization

4. Enrich — LLM-generated fields, taxonomy classification, RAG preparation

5. Reduce — aggregate, group, cluster, limit, summarize

The pipeline is stored, versioned, and callable as a single endpoint. Swap stages without touching application code. Benchmark with NDCG, MRR, and Recall. Track interaction signals to learn optimal weights over time.

31 stages. Under 100ms. Callable by LLM agents via MCP.

mixpeek.com/retrievers