NEWVectors or files. Pick a path.Start →
    Back to DiagramsIR Foundations

    The model answers with what you feed it

    In RAG, retrieval sets the quality ceiling — the model answers from the passages you feed it.

    Retrieval-augmented generation (RAG): a user asks a question, search retrieves the few most relevant passages from your indexed data, and the model answers from those passages placed in its prompt. The quality ceiling is retrieval, not the model — most hallucinations are retrieval failures, because the model can only answer with what it was fed.
    In RAG, retrieval sets the quality ceiling — the model answers from the passages you feed it.

    RAG gets explained like it's an architecture. It's simpler than that: the model doesn't know your data, so you hand it the relevant bits at question time.

    Retrieve the three passages that matter, put them in the prompt, let the model answer from them. That's the whole pattern. And it means the quality ceiling isn't the model. It's whatever retrieval handed over.

    A benchmark this month made that concrete: on multimodal queries, generation accuracy tracked retrieval accuracy almost step for step. Better retriever, better answers. Nobody touched the model.

    This is why I keep saying most hallucination complaints are retrieval failures wearing a trench coat. The model did its job with what it got. What it got was garbage.

    When your data is video and images rather than text, 'what it got' depends on extraction, decomposition, chunking, hybrid search, reranking. Everything in this series feeds the same three lines of context.

    Fix retrieval and the model suddenly looks smart.

    Where this diagram appears

    Run this on your own data

    Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

    Search your own data, free

    Related diagrams

    The unstructured-data problem: the large majority of enterprise data is unstructured — video, images, audio, and documents — that keyword systems and relational databases cannot search by meaning until it is turned into embeddings and indexed. The raw files are stored but not answerable.

    99% of your data has no rows

    An embedding turns meaning into coordinates: a model reads 'dog', 'puppy', and 'invoice' and outputs vectors, so 'dog' and 'puppy' land near each other while 'invoice' lands far away. Text, images, audio clips, video frames, and faces all map into one shared coordinate space where distance measures difference in meaning.

    An embedding is meaning turned into coordinates

    Semantic search as nearest-neighbor lookup: embed the query into the same vector space as your content, measure distance to every indexed point, and return the closest top-K results with no keyword matching. Exact brute-force compares against everything and is accurate but slow, while approximate nearest-neighbor (ANN) indexes like HNSW and IVF check a smart fraction of the space and return 95–99% of true answers in milliseconds.

    Search by meaning = nearest neighbors