NEWVectors or files. Pick a path.Start →
    Back to DiagramsDiagram Series

    The Embedding Model Landscape

    The Embedding Model Landscape

    The embedding model landscape: text-only models (E5, BGE, Cohere) for pure text, vision-language models (CLIP, SigLIP, OpenCLIP) for image plus text search, and multimodal models (Vertex 1408D, ImageBind, ONE-PEACE) when video, audio, images, and text must share one space.
    The Embedding Model Landscape

    There's no universal embedding model. What you should use comes down to your modalities and how much quality you need.

    Three families, three tradeoffs.

    Text-only models like E5, BGE, and Cohere embed v3 give you the best quality on language. If your data is documents, Q&A pairs, or plain text search, they win. They're also completely blind to anything that isn't text.

    Vision-language models like CLIP, SigLIP, and OpenCLIP put images and text in the same space, so you can query with text and get images back. That shared space costs you some text quality. On pure text search, a text-only model beats CLIP every time.

    Multimodal models like Vertex AI, ImageBind, and ONE-PEACE hold video, audio, images, and text in one space. You pay for it with high dimensionality (1408D and up), higher cost, and quality that varies by modality.

    So, roughly: text only, reach for E5 or BGE. Images plus text, CLIP or SigLIP. Video, audio, everything at once, Vertex AI. And if you need top text quality and images both, run both. Multi-vector search stores a text-optimized embedding and a vision embedding per document and searches them together.

    Start with your data, not the model.

    Where this diagram appears

    Run this on your own data

    Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

    Search your own data, free

    Related diagrams

    An embedding turns meaning into coordinates: a model reads 'dog', 'puppy', and 'invoice' and outputs vectors, so 'dog' and 'puppy' land near each other while 'invoice' lands far away. Text, images, audio clips, video frames, and faces all map into one shared coordinate space where distance measures difference in meaning.

    An embedding is meaning turned into coordinates

    A variable-size set of token vectors becomes one fixed-size vector via SimHash bucketing, projection, and repetition — no neural encoder required.

    MUVERA: How a Multi-Vector Set Becomes One Fixed Vector

    Recall@100 versus final FDE dimensionality on nfcorpus with ColBERTv2, log x-axis, one line per swept knob (k_sim, d_proj, r_reps); r_reps dominates at low dimensionality, d_proj floors near 16, and k_sim saturates past 5.

    MUVERA: Which Knob Actually Buys Recall (FDE Dimensionality Ablation)