Embeddings

20 min read

Updated 2026-06-24

# Matryoshka Representation Learning: Nested Embeddings for Adaptive Multimodal Retrieval

How a single embedding model can produce vectors that stay useful when truncated to fewer dimensions, and how AI agents exploit nested embeddings to run fast coarse shortlists then precise full-dimension reranks over huge unstructured corpora.

![Ethan Steininger](/people/ethan.webp)

By [Ethan Steininger](/blog/authors/ethan), Founder & CEO, Mixpeek

Matryoshka

Nested Embeddings

Adaptive Retrieval

Dimensionality

Multimodal Search

Agent Perception

[Run this on your own data. Try it as you read.Search your own archive →](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=guide_cta&utm_campaign=mvs_onramp)

[One-click templatesStart from a templateEach template deploys a working search app on your own files.Browse templates](/templates)

## The Problem: One Embedding Size Does Not Fit Every Query

  
  
An AI agent searching unstructured content faces a constant tension. To cover a corpus of millions of images, video frames, audio spans, or document chunks, it wants embeddings small enough to store cheaply and scan fast. To make a final, confident decision about which few results to act on, it wants embeddings expressive enough to separate near-duplicates. Those two needs pull in opposite directions.  
  
The naive fix is to pick one dimensionality and live with the compromise: a 1024-dimensional vector that is precise but expensive to store and search at scale, or a 256-dimensional vector that is cheap but blurry. Neither is right for every stage of a search.  
  
Matryoshka Representation Learning (MRL) removes the forced choice. It trains a single model so that the **first** \*k\* dimensions of every embedding are themselves a complete, usable embedding. Named after Russian nesting dolls, an MRL vector contains smaller vectors inside it. You can slice the first 64, 128, 256, or 512 dimensions off a 1024-dimensional vector and each prefix still works as a coordinate in a meaningful space. That single property is what lets an agent search coarse first and refine fine later, without re-embedding anything.  
  
This guide explains the real training objective, why truncation works at all, how it compares to alternatives, the cascade pattern agents use, and the practical gotchas that quietly break naive implementations.  
  

## The Core Idea: Coarse-to-Fine Information Packing

  
  
A normal embedding model spreads information across all dimensions with no ordering guarantee. Dimension 900 might carry as much signal as dimension 5. If you truncate that vector, you discard random pieces of meaning and the result is degraded in unpredictable ways.  
  
MRL changes the objective so the model is forced to pack the most important, most general information into the **front** of the vector and progressively finer detail toward the back. The result is a single vector you can read at multiple resolutions:  
  
1\. **First 64 dims**: coarse semantics. Enough to tell a cat photo from a car photo, or a refund email from an invoice. 2. **First 128-256 dims**: mid resolution. Enough to separate breeds of cat, or distinguish a refund request from a refund confirmation. 3. **Full 1024 dims**: fine resolution. Enough to rank near-duplicates and resolve subtle distinctions an agent's final answer depends on.  
  
The mental model: a normal embedding is a single photograph at one fixed resolution. An MRL embedding is a progressive JPEG. The first bytes already give you a recognizable thumbnail, and every additional byte sharpens it. You decide how much to load based on what the moment requires.  
  

![An ordinary embedding spreads information across all dimensions with no ordering guarantee, so truncating it discards random pieces of meaning. MRL packs the most general information into the front of the vector and finer detail toward the back, so every prefix is a complete usable coordinate. Five bars all starting at dimension 0 show the nesting: the first 64 dimensions carry coarse semantics, enough to tell a cat photo from a car photo; 128 is the prefix the cascade indexes the corpus at; 256 separates breeds of cat or a refund request from a refund confirmation; and the full 1024 ranks near-duplicates. The objective is a modified loss rather than a special architecture: one forward pass, the loss computed independently at each prefix in 64, 128, 256, 512 and 1024, then summed, so the front must work alone. The cascade it buys indexes 10,000,000 items at a 128-dimension truncation for 5.1 GB instead of 41 GB, shortlists the top 200, reranks those 200 at full 1024 dimensions, and returns a top 10.](/diagrams/matryoshka-representation-learning.webp)

An ordinary embedding spreads information across all dimensions with no ordering guarantee, so truncating it discards random pieces of meaning. MRL packs the most general information into the front of the vector and finer detail toward the back, so every prefix is a complete usable coordinate. Five bars all starting at dimension 0 show the nesting: the first 64 dimensions carry coarse semantics, enough to tell a cat photo from a car photo; 128 is the prefix the cascade indexes the corpus at; 256 separates breeds of cat or a refund request from a refund confirmation; and the full 1024 ranks near-duplicates. The objective is a modified loss rather than a special architecture: one forward pass, the loss computed independently at each prefix in 64, 128, 256, 512 and 1024, then summed, so the front must work alone. The cascade it buys indexes 10,000,000 items at a 128-dimension truncation for 5.1 GB instead of 41 GB, shortlists the top 200, reranks those 200 at full 1024 dimensions, and returns a top 10.

  
  
\*One vector read at five resolutions: the prefixes all start at dimension 0, which is what makes each one usable on its own.\*  
  

### The Training Objective

  
  
The mechanism is a modified loss function, not a special architecture. You take an ordinary encoder (a transformer, a vision encoder, a multimodal model) that outputs a \*d\*-dimensional vector. You then choose a set of nested prefix sizes: a common choice is powers of two such as `{64, 128, 256, 512, 1024}`.  
  
During training, for a single forward pass producing one \*d\*-dimensional output, you compute the training loss (for example, a contrastive or classification loss) **independently at each prefix size**, then sum them:

L\_matryoshka(x) =
  sum over m in {64, 128, 256, 512, 1024} of
    w\_m \* L\_task( truncate(f(x), m) )

Here `f(x)` is the full embedding, `truncate(v, m)` keeps the first \*m\* dimensions, `L_task` is the original loss (often computed with its own small classifier head or against the same contrastive targets), and `w_m` are per-prefix weights (frequently all equal to 1).  
  
The key consequences:  
  
*   The gradient from the 64-dim loss flows only through the first 64 dimensions, the 128-dim loss through the first 128, and so on. The front dimensions receive gradient from **every** prefix term, so they are optimized to be useful on their own. Back dimensions are optimized only by the larger prefixes, so they specialize in the residual detail the smaller prefixes could not capture.
*   You train **once**. There is no separate model per dimension. The full vector and all its prefixes come out of the same forward pass.
*   Inference is unchanged. The model emits the full vector; truncation happens downstream at index time or query time. There is no extra inference cost to support multiple resolutions.
  
  

### Why Truncation Works: Variance Lands at the Front

  
  
Because the smallest prefix must solve the task alone, the optimizer is pushed to concentrate the highest-variance, most discriminative directions into the earliest dimensions. The front of the vector ends up carrying the components that most separate items; the tail carries diminishing, more specialized refinements.  
  
This is conceptually similar to how PCA orders components by explained variance: except MRL bakes the ordering into the learned representation through the loss, so the prefixes are optimized end-to-end for the actual retrieval task rather than for reconstruction of the input distribution. The practical payoff is graceful degradation: a 128-dim prefix of an MRL model loses only a little accuracy versus the full 1024, while a 128-dim truncation of a non-MRL model can be near-useless.  
  

## How MRL Compares to the Alternatives

  
  
It is worth being precise about what MRL replaces and what it does not.  
  

Approach

What it does

Cost

Downside

**Train N models, one per dim**

Separate 128-d, 256-d, 1024-d models

N times the training and serving footprint; N inference passes if you want multiple sizes

Wasteful; vectors from different models are not comparable, so you cannot mix resolutions in one index

**PCA / post-hoc reduction**

Fit a projection after training, then reduce

Cheap to apply

Projection is fit to a distribution, not the retrieval task; quality drops faster than MRL and you must store/version the projection matrix

**Plain quantization**

Reduce bits per dimension (float32 to int8 or binary)

Very cheap

Orthogonal to dimensionality: it shrinks each number, not the count of numbers

**MRL truncation**

Slice the first \*m\* dims of one trained vector

One model, one inference pass, free truncation

Requires the model to be MRL-trained; truncating a non-MRL vector destroys it

  
  
Two clarifications matter for agents.  
  
**MRL versus PCA.** Both produce a lower-dimensional vector ordered by importance. But PCA optimizes for reconstructing the variance of the embedding distribution, while MRL optimizes the prefixes directly against the downstream retrieval objective. Empirically MRL prefixes retain retrieval accuracy better at aggressive truncation, and they need no separate projection step or stored matrix at query time: you just keep the first \*m\* numbers.  
  
**MRL versus quantization.** These solve different axes and **compose**. Dimensionality reduction (MRL) cuts the \*number\* of dimensions; quantization cuts the \*bits per\* dimension. You can take a 1024-dim float32 MRL vector, truncate to 256 dims, then quantize those 256 dims to int8, stacking both savings. For the bit-level half of this picture, see the companion guide on [embedding quantization and compression](/guides/embedding-quantization-compression). The two techniques are most powerful used together in a storage-tiering strategy.  
  

## The Adaptive Retrieval Cascade

  
  
The reason agents care about MRL is the **retrieve-coarse, rerank-fine** cascade. Instead of running one expensive search over full-dimension vectors, the agent runs a cheap first pass to build a shortlist, then spends precision only on that shortlist.  
  
1\. **Index the full corpus at a small prefix.** Store, say, a 128-dim truncation of every item in the ANN index used for the first-pass scan. This is the index the agent searches against the whole corpus. 2. **Coarse shortlist.** Embed the query, truncate the query vector to the same 128 dims, and run approximate nearest neighbor search to retrieve a candidate set: for example the top 200 of 10 million items. This pass is fast and memory-light because the vectors are small. 3. **Fine rerank.** Fetch the **full** 1024-dim vectors for only those 200 candidates and re-score them with the query's full-dim vector. Reorder, then return the final top 10.  
  
The agent pays full-dimension cost on 200 items, not 10 million. Recall stays high because the coarse pass is good enough to keep the truly relevant items inside the 200-candidate net, and the fine pass restores the precision needed to rank within it.  
  

### A Worked Numeric Example

  
  
Take a corpus of 10,000,000 items embedded at 1024 dimensions in float32 (4 bytes per dimension).  
  
**Storage.** Full vectors cost `10,000,000 × 1024 × 4 bytes ≈ 41 GB`. A 128-dim prefix index costs `10,000,000 × 128 × 4 bytes ≈ 5.1 GB`: an 8x reduction in the hot index that has to live in fast memory. You can keep the full 41 GB on a cheaper tier (see [vector storage tiering](/guides/vector-storage-tiering)) and only read the handful of full vectors you actually rerank.  
  
**Latency.** Brute-force scan cost scales with dimensionality. A single-pass full-dim scan does work proportional to `10,000,000 × 1024`. The cascade does `10,000,000 × 128` for the coarse pass plus `200 × 1024` for the rerank:

Single full-dim pass:  10,000,000 × 1024        = 10,240,000,000 units
Cascade coarse pass:   10,000,000 ×  128         =  1,280,000,000 units
Cascade rerank:               200 × 1024         =        204,800 units
Cascade total:                                   ≈  1,280,204,800 units
Speedup:               10,240,000,000 / 1,280,204,800  ≈ 8x

(Real ANN indexes are sublinear, so absolute numbers differ: but the \*ratio\* between coarse and fine work is what drives the savings, and that ratio is set by the prefix size you choose.) The cost/recall knob is the prefix size and the candidate count: a smaller prefix and larger shortlist favor recall and speed at some precision risk; a larger prefix narrows the shortlist needed to hit a recall target. Tune the prefix and `top_k` together against a labeled eval set rather than guessing.  
  
For the index internals that make the coarse pass sublinear, see [approximate nearest neighbor algorithms](/guides/approximate-nearest-neighbor-algorithms). For the broader pattern of staging cheap-then-expensive retrieval, see [multi-stage retrieval: how agents search unstructured data](/guides/multi-stage-retrieval-how-agents-search-unstructured-data).  
  

## Why This Matters for Multimodal and Agent Perception

  
  
The cascade is useful for text, but it is \*decisive\* for multimodal corpora, where vector counts and dimensions are both large.  
  
*   **Images and video.** A single hour of video segmented into frames or shots can produce thousands of vectors. A media library produces billions. Storing every frame at full dimension is the dominant cost in many perception systems. Truncating the first-pass index slashes that cost while the full vectors stay available for the few segments an agent reranks.
*   **Audio.** Long-form audio split into spans behaves like video, many vectors per asset, and benefits from the same coarse index.
*   **Agents that need a fast first look.** An agent answering a question rarely needs maximum precision on its first move. It needs a fast, cheap survey of what exists, then precision on the narrow set it decides to inspect. MRL maps directly onto that loop: coarse scan to orient, fine rerank to commit. This pairs naturally with budget-aware and adaptive strategies: see [budget-aware multi-vector retrieval](/guides/budget-aware-multi-vector-retrieval) and [adaptive indexing for agentic search](/guides/adaptive-indexing-for-agentic-search).
  
  

### Models That Ship Nested Embeddings (as of 2026)

  
  
MRL has moved from a research idea into mainstream production models. Verify the exact prefix sizes against each provider's current card before relying on a specific number, but these families are known to support truncatable / nested embeddings:  
  
*   **OpenAI text-embedding-3 (small and large).** Expose a `dimensions` parameter that lets you request shorter vectors trained to remain useful when truncated.
*   **Nomic Embed.** Released as an open model explicitly trained with Matryoshka so you can truncate the output.
*   **Jina embeddings (v3 and later).** Support Matryoshka-style truncation of the output dimensionality.
*   **Google EmbeddingGemma.** Ships nested-embedding support, letting you trade dimensionality for footprint.
*   **Alibaba GTE / Qwen3-Embedding family.** Models in this line support reduced output dimensions via MRL-style training.
  
  
If you are unsure whether a given checkpoint was MRL-trained, treat it as **not** truncatable until the model card says otherwise: and prefer phrasing like "models such as these" over asserting a specific model's internals you have not verified. For broader comparisons see the curated lists of [best multimodal embedding models](/curated-lists/best-multimodal-embedding-models) and [best self-hosted embedding models](/curated-lists/best-self-hosted-embedding-models).  
  

## Practical Gotchas

  
  
These are the failure modes that quietly break naive MRL implementations.  
  

### 1\. Normalize After Truncation, Not Before

  
  
If your similarity metric is cosine (or you rely on unit vectors for dot-product equivalence), you must **L2-normalize the truncated prefix**, not the full vector. A vector that is unit-length at 1024 dims is generally **not** unit-length after you slice it to 128 dims, because you dropped part of its magnitude. The correct order is: truncate first, then normalize the prefix. Normalizing the full vector and then truncating leaves you with mismatched magnitudes and degraded cosine scores. Apply the same truncate-then-normalize order to **both** the indexed items and the query so they live in the same space.  
  

### 2\. Not All Models Are MRL-Trained

  
  
Truncating a non-MRL embedding does not give you a smaller usable embedding: it gives you garbage, because the information was never ordered front-to-back. Before you truncate anything, confirm the model was trained with a Matryoshka objective (check the model card or the provider's `dimensions` parameter docs). If it was not, you must either keep the full vector or use a proper learned reduction; a raw slice will silently wreck recall.  
  

### 3\. Choosing the Prefix Size

  
  
There is no universal prefix. The right size depends on corpus size, the hardness of the queries, and your recall target. A practical procedure:  
  
1\. Build a labeled eval set of queries with known relevant items. 2. Sweep candidate prefix sizes (64, 128, 256, 512) for the coarse pass. 3. For each, measure recall@k of the coarse shortlist (does the relevant item survive into the candidate set?) and end-to-end precision after the full-dim rerank. 4. Pick the smallest prefix that keeps coarse recall above your threshold. Smaller prefix means cheaper index and faster scan; you recover precision in the rerank.  
  

### 4\. The ANN Index Is Built on the Truncated Dimension

  
  
This is the subtle one. Your approximate nearest neighbor index (its graph or partitions, its distance computations) is built over the vectors you actually insert. For the coarse pass, that means the index is built on the **truncated** dimension. You do not index full vectors and then ask the index for a 128-dim answer; you index 128-dim vectors. The full-dim vectors are stored separately (often on a cheaper tier) and fetched only for rerank. Keep the index dimension, the stored-vector dimension, and the rerank dimension explicitly distinct in your design so you never accidentally search the wrong resolution. The adaptive-indexing guide above covers how to manage this when the agent changes resolution per query.  
  

## Mapping This to Mixpeek

  
  
Mixpeek expresses the coarse-to-fine cascade as a **multi-stage retriever**. You upsert the truncated prefix as its own named vector, a feature\_search stage shortlists over it, and a rerank stage spends precision on the shortlist. No retriever stage re-scores a shortlist at a second vector dimension today, so the precision step is a cross-encoder over each candidate's text.

pip install mixpeek

from mixpeek import Mixpeek

NAMESPACE\_ID = "YOUR\_STANDALONE\_NAMESPACE\_ID"
mx = Mixpeek(api\_key="YOUR\_API\_KEY", namespace=NAMESPACE\_ID)

# Embed each asset with an MRL-capable model (nomic-ai/nomic-embed-vision) and
# upsert the first 128 dimensions as the coarse index. The caption travels as
# payload for the precision stage.
full = embedding
mx.namespaces.documents.upsert(
    namespace\_id=NAMESPACE\_ID,
    documents=\[
        {
            "document\_id": "asset\_8812:frame\_0040",
            "vectors": {"mrl\_128": full\[:128\]},
            "payload": {
                "asset\_id": "asset\_8812",
                "keyframe\_url": "s3://brand-media-library/asset\_8812/frame\_0040.jpg",
                "caption": "overhead shot of a blender on a marble counter",
            },
        }
    \],
)

# Stage 1 shortlists 200 over the 128-d prefix. Stage 2 is a cross-encoder
# that reads those 200 captions against the query.
retriever = mx.retrievers.create(
    retriever\_name="coarse-then-precise",
    input\_schema={
        "qv": {"type": "array", "required": True},
        "query": {"type": "text", "required": True},
    },
    stages=\[
        {
            "stage\_name": "coarse\_shortlist",
            "stage\_id": "feature\_search",
            "parameters": {
                "searches": \[
                    {
                        "feature\_uri": "mrl\_128",
                        "query": {"input\_mode": "vector", "value": "{{INPUT.qv}}"},
                        "top\_k": 200,
                    }
                \],
                "final\_top\_k": 200,
            },
        },
        {
            "stage\_name": "precise\_rerank",
            "stage\_id": "rerank",
            "parameters": {
                "inference\_name": "BAAI\_\_bge\_reranker\_v2\_m3",
                "query": "{{INPUT.query}}",
                "document\_field": "caption",
                "top\_k": 10,
            },
        },
    \],
)

results = mx.retrievers.execute(
    retriever\["retriever\_id"\],
    inputs={
        "qv": query\_embedding\[:128\],
        "query": "overhead shot of a product on a marble kitchen counter",
    },
)

The agent gets a fast first pass over millions of vectors at 128 dimensions, then spends cross-encoder precision on only the 200 it shortlisted. To store and query your own pre-computed MRL vectors with object-storage economics, see [MVS](/mvs). For cost planning across resolutions and tiers, see [pricing](/pricing) and the [docs](/docs/overview/introduction).  
  

## Production Checklist

  
  
*   Confirm the embedding model is genuinely MRL-trained before truncating anything.
*   Truncate first, then L2-normalize the prefix: apply the same order to items and queries.
*   Build the coarse ANN index on the truncated dimension; store full vectors separately for rerank.
*   Pick the prefix size from a labeled eval, not intuition: smallest prefix that holds coarse recall.
*   Tune prefix size and shortlist `top_k` together against your recall/latency target.
*   Compose MRL with quantization for stacked savings (fewer dims and fewer bits per dim).
*   Tier storage: cheap coarse index hot, full vectors on cheaper storage, fetched only on rerank.
*   Measure coarse recall@k separately from end-to-end precision so you know which stage to tune.
  
  

## Key Takeaways

  
  
*   MRL trains one model so the first \*k\* dimensions of every vector are an independently usable embedding: coarse semantics at the front, fine detail toward the back.
*   It works because the nested loss forces the highest-variance, most discriminative directions into the earliest dimensions, giving graceful degradation under truncation.
*   It beats training N models (one pass, comparable vectors) and beats PCA (task-optimized prefixes, no stored projection); it is orthogonal to and composable with quantization.
*   The payoff for agents is the retrieve-coarse, rerank-fine cascade: a cheap low-dim scan over the whole corpus, then full-dim precision on a small shortlist, roughly 8x storage and scan savings in the worked example.
*   It matters most for multimodal and agent perception, where vector counts are enormous and an agent wants a fast first look before committing precision.
*   Watch the gotchas: normalize after truncation, never truncate non-MRL models, choose the prefix empirically, and remember the ANN index is built on the truncated dimension.
  
  

## Related Resources

  
  
*   [Embedding Quantization and Compression](/guides/embedding-quantization-compression) -- the bit-level companion to dimensionality reduction
*   [Approximate Nearest Neighbor Algorithms](/guides/approximate-nearest-neighbor-algorithms) -- what makes the coarse pass sublinear
*   [Multi-Stage Retrieval: How Agents Search Unstructured Data](/guides/multi-stage-retrieval-how-agents-search-unstructured-data) -- the staging pattern in full
*   [Budget-Aware Multi-Vector Retrieval](/guides/budget-aware-multi-vector-retrieval) -- spending compute where it counts
*   [Adaptive Indexing for Agentic Search](/guides/adaptive-indexing-for-agentic-search) -- changing resolution per query
*   [Vector Storage Tiering](/guides/vector-storage-tiering) -- hot coarse index, cold full vectors
*   [Best Multimodal Embedding Models](/curated-lists/best-multimodal-embedding-models) -- compare nested-embedding support
*   [Best Self-Hosted Embedding Models](/curated-lists/best-self-hosted-embedding-models) -- open MRL-capable options
*   [Best Vector Databases](/curated-lists/best-vector-databases) -- where the truncated index lives

## Run this on your own data

Point Mixpeek at the storage you already have and search your video, images, audio, and documents the way this guide describes. Build starts at $25/mo for up to 1M vectors.

[Search your own archive](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=guide_cta&utm_campaign=mvs_onramp)[Read Docs](/docs)

[Explore MVS](/mvs)[See pricing](/pricing)

## Related guides

[

Embeddings

### Embedding Fine-Tuning and Distillation: Teaching an Agent to See and Hear Your Domain

A generic embedding model puts your two near-identical product SKUs, your defect classes, or your domain jargon almost on top of each other -- so an agent searching your content retrieves the wrong thing. This guide teaches the algorithms that fix that: contrastive fine-tuning, hard-negative mining for adaptation, cross-encoder to bi-encoder distillation, and parameter-efficient methods, plus how to deploy a new model without silently corrupting an existing index.

Read guide →](/guides/embedding-fine-tuning-distillation)[

Embeddings

### How Do You Convert Text Into Vector Embeddings?

Run the text through an embedding model: chunk it under the model's token limit, embed each chunk, normalize, and embed queries with the same model. Hosted APIs do it in one call (OpenAI's text-embedding-3-small: 1,536 dimensions, 8,192 tokens, $0.02 per million tokens). Open models run free on a CPU: 1,000 paragraphs took 19 s with all-MiniLM-L6-v2 in our test.

Read guide →](/guides/how-to-convert-text-into-vector-embeddings)[

Embeddings

### Why Are My SigLIP 2 Search Results So Bad? Text Padding and Lowercasing, Measured

SigLIP 2 needs query text padded to 64 tokens and lowercased, as it was trained. Unpadded queries scored 0.74% recall@1 on Flickr30k in our run; padded, 73.86%; padded and lowercased, 79.32% against the paper's 80.7%. Why padding silently fails on fixed-resolution checkpoints, and how to fix it.

Read guide →](/guides/why-siglip-2-text-to-image-search-results-are-bad)