MetaEncoder v1
by facebook
Meta's 28B multimodal embedding model: text, images, video and document pages into one 6,656-dimension vector, plus closed-set decisions scored as retrieval
facebook/MetaEncoder v1Deploy MetaEncoder v1
Single-tenantMixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.
Overview
MetaEncoder v1 is a multimodal embedding model from Meta, published on Hugging Face on 12 November 2026. It turns text, images, video frames and document page images into L2-normalised 6,656-dimension vectors, and every input can carry an instruction that says what the embedding is for, so a query and a document are embedded differently. It also answers closed-set questions about an input (yes or no, one of several options, or a level on a scale) by scoring each option as a retrieval candidate and returning probabilities. On Meta's own table it leads Qwen3-VL-Embedding 8B on MMEB V3 image (0.816 against 0.780) and video (0.743 against 0.672) retrieval. The model card does not state a license.
Architecture
A decoder transformer of about 28.4B parameters (the muse_glimmer architecture, which needs transformers 5.15 or later) with a separate embedding head. Images keep their native resolution, snapped to a 28-pixel grid, up to 4,096 merged tokens; video is encoded frame by frame and subsampled to 8 frames. The embedding is pooled from the last token of the user turn and projected by the head. Inputs are never truncated, and training sequences ran up to 16,384 tokens. One 80 GB GPU holds it in bf16.
Mixpeek SDK Integration
# Runs on your own GPU: one 80 GB card holds it in bf16, with transformers>=5.15.
# The wrapper ships in the repo; put it next to this script first:
# hf download facebook/meta-encoder-v1 modeling_metaencoder.py --local-dir .
import requests
from PIL import Image
from modeling_metaencoder import MetaEncoder
model = MetaEncoder.from_pretrained("facebook/meta-encoder-v1")
paths = ["pages/q3-report-04.png", "pages/q3-report-05.png"]
vectors = model.encode(
[{"image": Image.open(p)} for p in paths],
instruction="Understand the content of the provided document image.",
) # (2, 6656), L2-normalised
# Write the vectors to a Mixpeek collection as they are (up to 1,000 per call).
requests.post(
"https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
headers={"Authorization": "Bearer API_KEY"},
json={
"collection_id": "col_your_collection",
"documents": [
{"document_id": p.replace("/", "-"),
"vectors": {"meta_encoder_v1": v.tolist()},
"payload": {"source_key": p}}
for p, v in zip(paths, vectors)
],
},
)Capabilities
- Text, image, video and document-page inputs in one embedding space
- Instructions per input, so queries and documents embed for their role
- 6,656-dimension L2-normalised vectors
- Closed-set decisions (yes or no, choice, score) with probabilities and a calibration call
- Video encoded as up to 8 frames
Use Cases on Mixpeek
Benchmarks
| Dataset | Metric | Score | Source |
|---|---|---|---|
| MMEB V3 (image) | Image tasks | 0.816 | Model card: facebook/meta-encoder-v1 (self-reported); Qwen3-VL-Embedding 8B 0.780 |
| MMEB V3 (video) | Video tasks | 0.743 | Model card (self-reported); Qwen3-VL-Embedding 8B 0.672 |
| MMEB V3 (visdoc) | Visual document tasks | 0.827 | Model card (self-reported); WeMM-Embedding 9B 0.833, Qwen3-VL-Embedding 8B 0.824 |
| NanoBEIR | Text retrieval | 0.701 | Model card (self-reported); Qwen3-VL-Embedding 8B 0.606 |
| MMMU | Closed-set, answered as retrieval | 0.665 | Model card (self-reported); Qwen3-VL-Embedding 8B 0.506 |
| VideoMMMU | Closed-set, answered as retrieval | 0.646 | Model card (self-reported); Qwen3-VL-Embedding 8B 0.419 |
Performance
We have not measured it. The scores are Meta's own, from the table on the model card. At 6,656 dimensions a float32 vector is 26,624 bytes, about 6.5 times a 1,024-dimension vector, so storage and search cost per item are higher than for most embedding models. The card does not state a license; check it before production use.
Common Pipeline Companions
Frequently Asked Questions
What is MetaEncoder v1?
A multimodal embedding model from Meta, published on Hugging Face on 12 November 2026. It embeds text, images, video frames and document page images into one space as 6,656-dimension vectors, takes an instruction per input, and can answer closed-set questions by scoring each option as a retrieval candidate.
How does MetaEncoder v1 compare with Qwen3-VL-Embedding?
On Meta's own table it scores 0.816 on MMEB V3 image tasks against 0.780 for Qwen3-VL-Embedding 8B, 0.743 on video against 0.672, and 0.827 on visual documents against 0.824. It is about 28.4B parameters to Qwen's 8B, and its vectors have 6,656 dimensions, so it costs more to run and to store.
What hardware does MetaEncoder v1 need?
The card says one 80 GB GPU holds the model in bf16, and it needs transformers 5.15 or later for the muse_glimmer architecture. The repository ships a wrapper, modeling_metaencoder.py, with encode, retrieve and decide calls.
What license is MetaEncoder v1 released under?
The model card does not state one, as of 15 November 2026. Confirm the terms with Meta before you use it in a product.
Can I search MetaEncoder v1 embeddings with Mixpeek?
Yes. Run the model on your own GPU, write its vectors to a collection through the documents upsert endpoint, and query them with a retriever whose feature search takes a pre-computed vector. Embed queries with a query instruction such as "Find a document image that matches the given query." so they match how the documents were embedded.
Specification
Build a pipeline with MetaEncoder v1
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Run it on your own data, free