NEWVectors or files. Pick a path.Start →
    Back to DiagramsIR Foundations

    Search by meaning = nearest neighbors

    Search by meaning is a nearest-neighbor lookup — embed the query, return the closest points.

    Semantic search as nearest-neighbor lookup: embed the query into the same vector space as your content, measure distance to every indexed point, and return the closest top-K results with no keyword matching. Exact brute-force compares against everything and is accurate but slow, while approximate nearest-neighbor (ANN) indexes like HNSW and IVF check a smart fraction of the space and return 95–99% of true answers in milliseconds.
    Search by meaning is a nearest-neighbor lookup — embed the query, return the closest points.

    Once everything is a point in space, search gets almost embarrassingly simple: embed the query, find the nearest neighbors, return them.

    That's vector search. No keywords, no exact matching. Ask for 'the tense scene before the logo reveal' and the nearest points might be video frames, because those frames were embedded into the same space as your words.

    The engineering problem is scale. Comparing your query against 100 million vectors one at a time is too slow, so real systems use approximate indexes (HNSW, IVF) that check a smart fraction of the space and return 95 to 99% of the true answers in milliseconds. Approximate is a feature, not a compromise. Exact search at that scale would be slower than the person asking.

    We run this on object storage at Mixpeek, vectors living next to the media they describe, but the mechanics are the same everywhere.

    Search used to ask 'what contains these words.' Now it asks 'what is this near.'

    Where this diagram appears

    Run this on your own data

    Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

    Search your own data, free

    Related diagrams

    The unstructured-data problem: the large majority of enterprise data is unstructured — video, images, audio, and documents — that keyword systems and relational databases cannot search by meaning until it is turned into embeddings and indexed. The raw files are stored but not answerable.

    99% of your data has no rows

    An embedding turns meaning into coordinates: a model reads 'dog', 'puppy', and 'invoice' and outputs vectors, so 'dog' and 'puppy' land near each other while 'invoice' lands far away. Text, images, audio clips, video frames, and faces all map into one shared coordinate space where distance measures difference in meaning.

    An embedding is meaning turned into coordinates

    Object decomposition: you search inside the file, not for it. A one-hour recording.mp4 is split into scenes (visual embeddings), speech (transcript embeddings on silence and speaker turns), faces (who appeared and when), and on-screen text (OCR of slides and captions). A whole file being findable is not the same as usable — the thing you need is a moment inside it, like 23:41 where the renewal number is spoken while the pricing slide is on screen. The same move works on video, PDFs, and audio: one file becomes dozens of small, independently searchable documents.

    You search inside the file, not for it