Ovis-Omni-Embedding-3B
by ATH-MaaS
One embedding space for text, images, documents, video and audio, from a 3B model
ATH-MaaS/Ovis-Omni-Embedding-3BDeploy Ovis-Omni-Embedding-3B
Single-tenantMixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.
Overview
Ovis-Omni-Embedding-3B maps text, images, visual documents, video, audio and mixed inputs into one vector space, so a text query can find a clip, a page or a recording with the same index. It is a bi-encoder built on Qwen2.5-Omni-3B: queries and candidates are encoded separately, each input becomes one 2048-dimension vector, and candidates are ranked by cosine similarity. ATH-MaaS released it on 20 September 2026 under Apache-2.0.
The card reports 58.46 overall on MMEB-v3, a 190-dataset benchmark spanning image, video, visual-document, text, audio and agent retrieval, ahead of the best baseline it compares against (53.27). The largest margins it reports are on audio (+6.91) and agent retrieval (+6.10).
Weigh the evidence before adopting it. The scores are self-reported on the card, some of the audio and video results are the authors' local evaluations, and the repository showed about 100 downloads in its first week. Test it on your own content and treat the numbers as the vendor's claim.
Architecture
Initialized from Qwen2.5-Omni-3B and kept whole: the native text tokenizer, vision encoder and audio encoder feed one interleaved token sequence through the shared Thinker, with no separate per-modality towers and no projection head. The embedding is the final-layer hidden state at the last non-padding token, L2-normalized, which gives a native width of 2048. Queries are formatted with a retrieval instruction through the model's own processor and chat template. The card describes a post-hoc elastic-dimension module for smaller widths and says queries and candidates must use the same width and transformation. Training ran in three stages: omni-modal contrastive pretraining, full-parameter finetuning on single-source batches, and embedding distillation from several expert models.
Mixpeek SDK Integration
# No Mixpeek extractor runs these weights. Bring your own vectors: encode each
# item with Ovis-Omni-Embedding-3B as its card describes (native processor and chat
# template, last non-padding token of the final layer, L2-normalized, 2048 floats),
# then upsert into a namespace whose vector index is declared at 2048 dimensions.
import requests
requests.post(
"https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
headers={"Authorization": "Bearer API_KEY"},
json={
"collection_id": "col_your_collection",
"documents": [{
"document_id": "clip-0042",
"vectors": {"ovis_omni": clip_vector}, # the name of your 2048-d index
"payload": {"source_key": "s3://media/clip-0042.mp4"},
}],
},
)Capabilities
- One vector space for text, images, visual documents, video, audio and interleaved inputs
- Bi-encoder retrieval: encode once, rank by cosine similarity
- 2048-dimension native output, with a post-hoc module for smaller widths
- Apache-2.0 license
Use Cases on Mixpeek
Benchmarks
| Dataset | Metric | Score | Source |
|---|---|---|---|
| MMEB-v3 (190 datasets) | Overall | 58.46 | Model card: ATH-MaaS/Ovis-Omni-Embedding-3B (self-reported) |
| MMEB-v3 video | Group score | 64.99 | Model card (self-reported) |
| MMEB-v3 visual document | Group score | 78.26 | Model card (self-reported) |
| MMEB-v3 audio | Group score | 50.08 | Model card (self-reported) |
| RTEB (15 English retrieval tasks) | Mean | 67.35 | Model card (self-reported) |
Performance
We have not measured encoding latency or memory. At 2048 float32 dimensions each vector is 8 KiB before any compression.
Common Pipeline Companions
Frequently Asked Questions
What is an omni-modal embedding model?
A single model that turns text, images, video, audio and documents into vectors in the same space, so any of them can be compared with any other. A text query can then retrieve a video clip or an audio recording directly, without a separate model and index per file type.
What embedding dimension does Ovis-Omni-Embedding-3B produce?
2048 natively, taken from the final hidden layer with no projection head. The card describes a post-hoc module for smaller widths and says queries and candidates must always use the same width and transformation, L2-normalized after projection.
Are its benchmark results independently verified?
No. The MMEB-v3 and RTEB figures are reported on the model card, and the card notes that some audio and video results are the authors' local evaluations. The benchmarks are public, so the numbers can be reproduced, and until they are it is worth testing on a sample of your own content.
Does Mixpeek run Ovis-Omni-Embedding-3B?
Not on the managed tier. You can encode with it yourself and store the vectors in a Mixpeek namespace, then search them with a retriever that takes a query vector. On a single-tenant Enterprise deployment the weights can be uploaded and run by a custom plugin.
Specification
Research Paper
Ovis-Omni-Embedding technical report (arXiv 2609.25165)
arxiv.orgBuild a pipeline with Ovis-Omni-Embedding-3B
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Run it on your own data, free