Trident-Qwen3VL-2B
by openbmb
OpenBMB's 2.1B mixed-modality embedder: one space for text, images and text plus image, trained so text documents stop outranking the relevant image
openbmb/Trident-Qwen3VL-2BDeploy Trident-Qwen3VL-2B
Single-tenantMixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.
Overview
Trident-Qwen3VL-2B is a multimodal embedding model from OpenBMB, published on Hugging Face on 12 October 2026 under the MIT license. It maps text, images and fused text-image inputs into one shared space, so a text query can rank text-only, image-only and fused documents in a single list. It is fine-tuned from Qwen3-VL-2B-Instruct.
It comes from the paper Chaos in the Text: Revealing the Modality Preference in Mixed-Modality Retrievers (arXiv 2610.11816, October 2026). The authors report that retrievers stay strong on single-modality corpora and degrade when text and images coexist, that irrelevant text hurts more than an equal number of irrelevant images, and that text representations get systematically higher similarity scores, which lets irrelevant text outrank relevant images. Trident trains the text, image and fused views of each document as equal positives to reduce that bias.
This is the failure a mixed corpus shows in practice: a search over PDFs, screenshots and their OCR text keeps returning text chunks when the answer is in a chart. The card publishes no scores, and we have not measured it.
Architecture
A Qwen3-VL-2B-Instruct backbone fine-tuned for embedding (Qwen3VLForEmbedding in config.json), about 2.1B parameters. Each input is a dict with an optional text string and a list of images; text only, image only and both together are all valid documents. Embeddings come back L2-normalized, so a dot product is cosine similarity and scores compare across document types. Queries and documents use different default instructions, which are part of the training format. Training used Multi-Positive View InfoNCE over text, image and fused views of each document, on the OpenBMB/Trident-Train dataset.
Mixpeek SDK Integration
# Embed page images, text chunks or both with Trident-Qwen3VL-2B (run from the
# root of github.com/OpenBMB/Trident, per the card), then upsert the vectors
# into a Mixpeek Vector Store namespace you query directly. Needs a CUDA GPU.
import sys
sys.path.insert(0, "src")
from embedding_backends.trident_qwen3vl import TridentQwen3VLBackend
from mixpeek import Mixpeek
backend = TridentQwen3VLBackend(
{"model_type": "trident_qwen3vl", "checkpoint": "checkpoints/Trident-Qwen3VL-2B", "dtype": "bfloat16"},
device="cuda:0",
)
backend.load()
def encode(items, role):
backend.args_dict["role"] = role
instruction = backend.default_instruction(role)
embeddings, _ = backend.process([{**i, "instruction": instruction} for i in items], role=role)
return embeddings # L2-normalized
client = Mixpeek(api_key="YOUR_KEY")
docs = [
{"id": "report-p14-chart", "text": None, "images": ["report-p14.png"]},
{"id": "report-p14-text", "text": "Stores by region, 2018 to 2020...", "images": []},
]
vectors = encode([{"text": d["text"], "images": d["images"]} for d in docs], role="doc")
client.namespaces.upsert(
namespace="reports",
documents=[{"id": d["id"], "dense_embedding": v.tolist(), "text": d["text"] or ""}
for d, v in zip(docs, vectors)],
)Capabilities
- One space for text-only, image-only and text plus image documents
- Text queries ranked against all three document types in one list
- Training aimed at the bias that lets irrelevant text outrank relevant images
- L2-normalized output, so scores are comparable across document types
- MIT license; code and evaluation data published by OpenBMB
Use Cases on Mixpeek
Performance
The card was tested with transformers 4.57.1, and Qwen3-VL needs 4.57 or later. The paper evaluates on visual document and natural image benchmarks, and the card publishes no scores; we have not measured it.
Common Pipeline Companions
Frequently Asked Questions
What is Trident-Qwen3VL-2B?
A 2.1B-parameter multimodal embedding model from OpenBMB, fine-tuned from Qwen3-VL-2B-Instruct and released under MIT on 12 October 2026. It puts text, images and text plus image documents in one space so a text query can rank all three in one list.
Why does my multimodal search return text when the answer is in an image?
The Trident paper (arXiv 2610.11816) reports that mixed-modality retrievers give text representations systematically higher similarity scores, so irrelevant text can outrank a relevant image. Trident trains each document's text, image and fused views as equal positives to reduce that preference. Measure it on your own corpus by checking how often an image document that answers the query ranks below text that does not.
Can I mix text-only and image-only documents in one index with it?
Yes. That is what it was trained for. Each document can be text only, images only or both, and the L2-normalized vectors are compared by dot product in the same space.
How do I use Trident-Qwen3VL-2B with Mixpeek?
Embed documents and queries with it on your own GPU using the Trident repository's backend, and upsert the vectors into a Mixpeek Vector Store namespace, as in the example on this page. On Enterprise, the weights can also be uploaded and run by a custom plugin.
Specification
Research Paper
Model paper or technical report
arxiv.orgBuild a pipeline with Trident-Qwen3VL-2B
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Run it on your own data, free