[Models](/models)/tencent/WeMM-Video 35B-A3B

Apache 2.0

# WeMM-Video 35B-A3B

by tencent

Tencent's open video model that watches the frames and listens to the soundtrack together: text, images, audio and video in, text out, with 3B of 35B parameters active

27likes

35B total (about 3B active)params

[HuggingFace](https://huggingface.co/tencent/WeMM-Video-35BA3B)[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)

Identifiers

Model ID

`tencent/WeMM-Video 35B-A3B`

## Deploy WeMM-Video 35B-A3B

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

[Read the customer-hosted guide](https://mixpeek.com/docs/customer-hosted/overview) [Talk to us about single-tenant](/demo)

## Overview

WeMM-Video-35B-A3B is the mixture-of-experts checkpoint in Tencent's WeMM-Video family, released on Hugging Face on November 17, 2026 under the Apache 2.0 license. It takes text, images, audio and video and writes text in English or Chinese. What sets it apart from most open vision-language models is the soundtrack: given a video file, it can read the frames and the original audio together, so it can describe an event, transcribe the speech and reason about what is seen and heard in one pass.

It has 35B parameters in total with about 3B active per token, and its BF16 weights are 67.39 GiB. Tencent also publishes FP8, AWQ 4-bit, GGUF Q4\_K\_M and NVFP4 versions, plus dense 9B and 2B siblings in the same family. The technical report (arXiv 2611.26409) describes pretraining on about 1.2 trillion multimodal tokens.

For a media pipeline, it can turn a clip into a description of what happens and what is said, which you can then index and search as text.

## Architecture

Separate vision and audio encoders turn frames and sound into tokens, which a mixture-of-experts language model (35B total, about 3B active per token) processes alongside text. Video frames carry timestamps. Post-training used multi-teacher on-policy distillation, an anti-repetition preference step (MrDPO) and specialist reinforcement learning.

## Mixpeek SDK Integration

IngestRetrieve

\# WeMM-Video is not a managed extractor. Serve it with Tencent's patched vLLM
# (see https://github.com/Tencent/WeMM-Video/blob/main/docs/serving.md), which
# the card runs across two GPUs:
#   vllm serve tencent/WeMM-Video-35BA3B --trust-remote-code --dtype bfloat16 \\
#     --max-model-len 8192 --tensor-parallel-size 2
# Then describe each clip, frames and sound together, and store the text.
import requests
from openai import OpenAI

llm = OpenAI(base\_url="http://localhost:8000/v1", api\_key="unused")

def describe(clip\_url):
    r = llm.chat.completions.create(
        model="tencent/WeMM-Video-35BA3B",
        messages=\[{"role": "user", "content": \[
            {"type": "video\_url", "video\_url": {"url": clip\_url, "use\_audio": True}},
            {"type": "text", "text": "Describe what happens and summarize what is said."},
        \]}\],
        max\_tokens=200,
    )
    return r.choices\[0\].message.content

requests.post(
    "https://api.mixpeek.com/v1/namespaces/ns\_your\_namespace/documents/upsert",
    headers={"Authorization": "Bearer API\_KEY"},
    json={"collection\_id": "col\_clip\_descriptions",
          "documents": \[{"document\_id": "clip-042",
                         "payload": {"description": describe("https://example.com/clip-042.mp4"),
                                     "video": "clip-042.mp4"}}\]},
)

## Capabilities

*   Video with its original soundtrack, silent video, audio, images and text as input
*   Text output in English or Chinese, with an instruct mode and a thinking mode
*   About 3B active parameters per token out of 35B
*   BF16 weights of 67.39 GiB, with FP8, AWQ, GGUF and NVFP4 versions
*   Apache 2.0 license

## Use Cases on Mixpeek

Describing what happens and what is said in each clip of a video library, so it can be searched as text

Transcribing speech in footage while also describing the scene it was spoken in

Answering questions about a clip that need both the picture and the sound

Writing summaries of recorded meetings, lectures or broadcasts from the video file itself

## Performance

Input SizeText, images, audio and video; the card's vLLM command sets an 8,192-token context and samples video at 2 fps, 4 to 16 frames per clip

GPU LatencyInput dependent

GPU ThroughputBatch dependent

GPU MemoryModel dependent

Tencent's serving recipes use tensor parallelism across two GPUs and say to size memory for the full expert weights, the media inputs and the generation cache. Tencent reports its benchmark results as charts on the model card. We have not measured it.

## Common Pipeline Companions

[

tencent/Hy4-flash

Tencent's image and text model, for frames and documents when you do not need the soundtrack

](/model/tencent/Hy4-flash)

## Frequently Asked Questions

### What is WeMM-Video?

A family of open video understanding models from Tencent, released November 17, 2026 under Apache 2.0. The models take text, images, audio and video, including a video's own soundtrack, and write text in English or Chinese. The 35B-A3B checkpoint is the mixture-of-experts model, with about 3B of its 35B parameters active per token; 9B and 2B versions are also published.

### Can WeMM-Video understand the audio in a video?

Yes. Given a video file with sound, it reads the frames and the original soundtrack together, so it can describe an event and transcribe or summarize the speech in one answer. In the serving API, set use\_audio to true on the video\_url input; set it to false to send the frames alone.

### What hardware does WeMM-Video-35B-A3B need?

The BF16 weights are 67.39 GiB, and Tencent's vLLM and SGLang commands split them across two GPUs with tensor parallelism. Only about 3B parameters are active per token, which keeps generation fast, but all 35B must fit in GPU memory along with the media inputs and cache. The FP8, AWQ 4-bit, GGUF and NVFP4 versions need less memory, and the dense 9B and 2B models less again.

### Can I use WeMM-Video commercially?

The model card releases it under the Apache License 2.0, which allows commercial use. The card also points to a collection license and third-party attribution file; read both before you ship it.

### How do I use WeMM-Video with Mixpeek?

Serve it with Tencent's patched vLLM, send each clip with its soundtrack, and upsert the description it writes into a Mixpeek collection. A retriever then searches those descriptions by meaning, next to any scene embeddings, faces or transcripts in the same namespace.

## Explore on Mixpeek

[

Browse all models

Search the full model catalog

](/models)[

Feature Extractors

See how Mixpeek runs models as extractors

](/extractors)[

Mixpeek Vector Store

Store & search embeddings at scale

](/mvs)[

Pricing

Usage-based pricing for pipelines

](/pricing)[

All Curated Lists

Compare models, APIs & infrastructure

](/curated-lists)

### Specification

Organizationtencent

Retriever\-

Parameters35B total (about 3B active)

LicenseApache 2.0

Downloads/moN/A

Likes27

[

### Research Paper

WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding

arxiv.org](https://arxiv.org/abs/2611.26409)

### Build a pipeline with WeMM-Video 35B-A3B

Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)