WeMM-Embedding-4B

Name: WeMM-Embedding-4B
Author: tencent

by tencent

Tencent's 4B multimodal embedding model: text, image, video and visual documents in one 2,560-dimension space

184dl/month

5.17B total (Qwen3.5-4B backbone plus vision tower)params

HuggingFace Run it on your own data, free

Identifiers

Model ID

tencent/WeMM-Embedding-4B

Feature URI

mixpeek://multimodal_extractor@v2/tencent_wemm_embedding_4b_v1

Overview

WeMM-Embedding-4B is a universal multimodal embedding model from the WeChat team at Tencent, released 26 August 2026 on a Qwen3.5-4B backbone. Give it text, an image, a video, a scanned page, or several of those interleaved in one call, and it returns a single L2-normalized 2,560-dimension vector. Audio is the one input it refuses.

It sits between the two other sizes on almost every published axis, at 79.2 MMEB-v2 average against 77.9 for the 2B and 80.6 for the 9B. One number breaks that ordering. On MMEB-v3 MCMR the 4B scores 41.9 while the 2B scores 42.5, so the smaller model wins that task family, and "buy the largest size you can afford" is the wrong rule if multi-candidate multi-reference retrieval is what you are building.

Matryoshka training lets you pick the output width when you build the index. The published truncation points are 64, 128, 256, 512, 1,024 and 2,560, and Tencent measured 256-dimension embeddings retaining 98.7% of full-dimension image and video performance on MMEB-v2. Storing 256 floats instead of 2,560 is roughly a 10x cut in bytes per vector, which is usually the line item that decides how much of an archive you can afford to index.

Architecture

Qwen3.5-4B language backbone with a 1,024-dimension vision tower, 5,173,725,696 parameters in BF16. The size in the name refers to the language model, so the safetensors total runs higher once the vision encoder is counted. Position embeddings extend to 262,144 tokens. Output is a 2,560-dimension L2-normalized vector with Matryoshka truncation at 64, 128, 256, 512, 1,024 and 2,560. Tencent publishes serving paths for Transformers, Sentence Transformers, vLLM 0.27.0 with the pooling runner, and SGLang 0.5.9.

Mixpeek SDK Integration

// WeMM-Embedding has no managed Mixpeek extractor yet, so encode with the
// model and upsert the vectors. Route verified against the live OpenAPI:
// POST /v1/namespaces/{namespace_id}/documents/upsert (document_id + vectors
// are the only required fields).
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection_id: "col_your_collection",
      documents: [
        {
          document_id: "reel-00412",
          // 2560 floats from model.embedding(...). Slice and renormalize first
          // if you are storing a truncated Matryoshka dimension.
          vectors: { "multimodal-embedding": wemmVector },
          payload: { title: "Q3 launch reel", duration_s: 94 },
          metadata: { source: "brand-archive" },
        },
      ],
    }),
  },
);

Capabilities

One vector space covering text, image, video and visual-document queries
Matryoshka truncation to 64, 128, 256, 512, 1,024 and 2,560 without re-encoding
Interleaved inputs, so several images and a video can share a single embedding call
262,144-token position range for long visual documents
Chinese and English
Published vLLM 0.27.0 and SGLang 0.5.9 serving recipes

Use Cases on Mixpeek

Text-to-video search over an archive, returning moments rather than filenames

Retrieving scanned pages and slide decks without an OCR stage in front of the index

Cross-modal dedup, where an image and its caption should land in the same neighbourhood

Shrinking an existing index to 256 dimensions once storage cost outgrows the accuracy it buys

Benchmarks

Dataset	Metric	Score	Source
MMEB-v2 (78 datasets)	Average	79.2	WeMM-Embedding technical report, Table 1
MMEB-v2 Image	Hit@1	80.8	Technical report, Table 1
MMEB-v2 Video	Hit@1	72.1	Technical report, Table 1
MMEB-v2 VisDoc	NDCG@5	82.0	Technical report, Table 1
MMEB-v3 (190 tasks)	V3-All	58.2	Technical report, Table 2
MMEB-v3 Text	NDCG@5	47.9	Technical report, Table 2
MMEB-v3 Agent	Hit@1	49.0	Technical report, Table 2
MMEB-v3 MCMR	Hit@1	41.9	Below the 2B's 42.5; Table 2
MMEB-v3 Audio	Hit@1	0.0	Audio is unsupported and scores zero by construction

Performance

Input SizeText, image, video, visual document, or interleaved; 262,144-token position range

Embedding Dim2560 (Matryoshka down to 64)

GPU LatencyInput dependent

GPU ThroughputBatch dependent

GPU Memory~10 GB in BF16 for weights alone

Tencent publishes no latency or throughput figures for this model. The memory number is computed from the 5,173,725,696 BF16 parameters in the safetensors index and covers weights only, so serving needs headroom for activations and for batched video frames.

Common Pipeline Companions

tencent/WeMM-Embedding-2B

Cheaper size that beats this one on MMEB-v3 MCMR

tencent/WeMM-Embedding-9B

Larger size, +1.4 MMEB-v2 average

Qwen/Qwen3-VL-8B-Instruct

Caption or answer over the retrieved candidates

Frequently Asked Questions

Can I run WeMM-Embedding on Mixpeek today?

Not as a managed extractor. GET /v1/discovery/extractors returns 13 shipping extractors and none of them accepts an arbitrary Hugging Face model id, so a collection cannot be pointed at these weights. Two paths work now. Run the model on your own hardware and upsert the vectors through POST /v1/namespaces/{namespace_id}/documents/upsert, which takes a named-vector map and stores it in Mixpeek Vector Store alongside everything else. Or package it as a custom extractor plugin, which is available on Enterprise namespaces.

Does WeMM-Embedding handle audio?

No. The model card states that audio input is not supported, and on the 11 audio tasks in MMEB-v3 it scores 0.0, because the benchmark assigns zero to tasks a model cannot attempt. If audio has to live in the same index, the published omni-modal alternatives are Omni-Embed-Nemotron at 36.5 and E5-Omni-7B at 43.0 on the same tasks.

What licence does WeMM-Embedding carry?

Apache 2.0. The Hugging Face card sets the license field to "other", which reads as restrictive, and the LICENSE file in the repo resolves it: Tencent states the model is "licensed under Apache-2.0" except for listed third-party components, which keep their original terms. No additional restrictions are imposed beyond those.

Why does the 4B lose to the 2B on MCMR?

Tencent reports 41.9 for the 4B and 42.5 for the 2B on MCMR in MMEB-v3 Table 2, and offers no explanation for the inversion. Every other reported axis has the 4B ahead. Treat it as a reason to score both on your own queries rather than a reason to prefer either: a 0.6 point gap on one task family inside a 190-task benchmark is well inside the range where a workload-specific evaluation can flip the answer.

Is the 4B actually 4 billion parameters?

The safetensors index reports 5,173,725,696 parameters in BF16. The 4B in the name refers to the Qwen3.5-4B language backbone; the vision tower accounts for the rest. Plan GPU memory against 5.17B, roughly 10 GB for weights alone before activations and batched video frames.