NEWVectors or files. Pick a path.Start →
    Back to DiagramsIR Foundations

    Pick the metric that matches user behavior

    Pick the metric that matches user behavior

    Choosing a retrieval eval metric is a product decision, not a math one: use Mean Reciprocal Rank (MRR) when users trust the first result (support bots, agents, voice assistants), NDCG@10 when users scan a page of results (product and media search, e-commerce), and recall@k when users need every match (legal discovery, brand safety, compliance). Optimizing the wrong metric makes the dashboard improve while the product degrades, and unmeasured retrieval is not working — it is just not failing loudly yet.
    Pick the metric that matches user behavior

    How do you know your search works? Most teams answer with a shrug and a demo query.

    Retrieval has real metrics, and picking one is a product decision, not a math one. If users trust the first result (support bots, agents), measure MRR: how high the first correct answer ranks. If they scan a page of results, NDCG@10: whether the good stuff sits near the top. If missing anything is expensive (legal discovery, brand safety), recall@k: what fraction of the true matches came back.

    Optimize the wrong one and things get worse invisibly. Chasing recall floods agents with context they never read. Chasing MRR hides the third relevant document a lawyer needed.

    We built eval directly into Mixpeek retrievers because the alternative is what most pipelines do today: ship, vibe-check, drift, rot. If your eval can fail silently, you don't have quality. You have luck.

    Unmeasured retrieval isn't working. It's just not failing loudly yet.

    Where this diagram appears

    Run this on your own data

    Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

    Search your own data, free

    Related diagrams

    The unstructured-data problem: the large majority of enterprise data is unstructured — video, images, audio, and documents — that keyword systems and relational databases cannot search by meaning until it is turned into embeddings and indexed. The raw files are stored but not answerable.

    99% of your data has no rows

    An embedding turns meaning into coordinates: a model reads 'dog', 'puppy', and 'invoice' and outputs vectors, so 'dog' and 'puppy' land near each other while 'invoice' lands far away. Text, images, audio clips, video frames, and faces all map into one shared coordinate space where distance measures difference in meaning.

    An embedding is meaning turned into coordinates

    Semantic search as nearest-neighbor lookup: embed the query into the same vector space as your content, measure distance to every indexed point, and return the closest top-K results with no keyword matching. Exact brute-force compares against everything and is accurate but slow, while approximate nearest-neighbor (ANN) indexes like HNSW and IVF check a smart fraction of the space and return 95–99% of true answers in milliseconds.

    Search by meaning = nearest neighbors