VideoJev-4B

Name: VideoJev-4B
Author: ZhuYuhan

by ZhuYuhan

A 4.47B video model from Nanjing University's MCG lab that scores candidates directly: finds when something happens in a video and picks the right answer to a multiple-choice question about it

25likes

4.47Bparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

ZhuYuhan/VideoJev-4B

Feature URI

Deploy VideoJev-4B

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

Read the customer-hosted guide Talk to us about single-tenant

Overview

VideoJev-4B is a video understanding model published on Hugging Face in October 2026 under the Apache-2.0 license, fine-tuned from Qwen/Qwen3.5-4B. It does two jobs. For temporal grounding, you give it a video and a sentence such as "The woman in green eats pasta" and it returns the start and end of that moment in seconds. For multiple-choice video question answering, you give it a question and options and it returns the option it picks with a probability for each.

It answers by scoring candidates. Grounding returns an independent probability per time step, and QA returns a softmax over the options, so every answer carries a score you can threshold. The authors call it a "System One" model in the paper VideoJev: System One Model for Video Understanding and Interaction (arXiv 2610.18615). The card shows benchmark charts for grounding and QA quality against latency; we have not measured it.

Architecture

A Qwen3.5-4B backbone (model_type qwen3_5 in config.json) with VideoJev's own temporal and video modules, loaded with trust_remote_code as VideoJevForVideoUnderstanding. 4,471,106,560 parameters in BF16. The model reads sampled frames: the quick start samples 2 frames a second with up to 2,048 frames. Grounding output is a per-step sigmoid probability that a decoder turns into an interval, with a default threshold of 0.25 and smoothing of 1. QA output is a softmax over the options you pass. Per the usage guide, QA reads frames only, without subtitles or audio.

Mixpeek SDK Integration

# VideoJev-4B scores a video against your questions at query time, so there is
# nothing to index for it. Index the library with a video embedder (for example
# in a Mixpeek collection), then run VideoJev on the clips a search returns.
# The card was tested with Python 3.12, PyTorch 2.8, CUDA 12.8 and an H200.
import torch
from transformers import AutoModel, AutoProcessor

repo = "ZhuYuhan/VideoJev-4B"
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16,
    device_map={"": "cuda:0"}, attn_implementation="flash_attention_2",
)
model.prepare_for_inference()

Capabilities

Temporal grounding: returns the start and end, in seconds, of the moment a sentence describes
Multiple-choice video QA with a probability for every option
Several grounding and QA questions about one video in a single call
A tunable grounding threshold, per call or as the loaded model's default
Apache-2.0 license; fine-tuned from Qwen3.5-4B

Use Cases on Mixpeek

Trimming a search hit to the seconds where the described action happens

Checking a retrieved clip before showing it: does it contain the moment at all

Answering yes or no and pick-one questions about footage, with a score to threshold

Labelling training clips with the interval of an action described in text

Performance

Input SizeOne video per call, sampled at 2 frames a second in the quick start (4 for grounding in the usage guide), up to 2,048 frames

GPU LatencyInput dependent

GPU ThroughputBatch dependent

GPU MemoryModel dependent

The usage guide sets a 147,456-token context for grounding at 4 frames a second and 262,144 for QA, and the card was tested on an H200 with Flash Attention 2, so plan for a large GPU on long videos. Run it on short clips that a first-stage search already found. QA reads frames only, so a question answered by dialogue needs a transcript from another model. The card shows grounding and QA charts; we have not measured it.

Common Pipeline Companions

google/videoprism-base-f16r288

Embed the library and find candidate clips before VideoJev scores them

Qwen/Qwen3-VL-Embedding-8B

A multimodal first stage when the query can be text or an image

Frequently Asked Questions

What is VideoJev-4B?

A 4.47B-parameter video model fine-tuned from Qwen3.5-4B and released under Apache-2.0. It finds when a described moment happens in a video (temporal grounding) and answers multiple-choice questions about a video, with a probability for each candidate.

What is temporal grounding?

Finding the start and end time of the part of a video that matches a sentence. For a 10-second clip and the sentence "The woman in green eats pasta", the card's example returns the interval 3.0 to 10.0 seconds.

Is VideoJev-4B an embedding model?

It is tagged feature-extraction on Hugging Face, and what it returns is a score per question: an interval with per-step probabilities, or a chosen option. To search a library, find the clips with a video embedder first, then use VideoJev to locate the moment inside each clip.

Can VideoJev-4B answer questions about what is said in a video?

Not from audio. The usage guide says QA uses frames without subtitles or audio. For spoken content, transcribe the clip and search the transcript, then use VideoJev for what is visible.

How do I use VideoJev-4B with Mixpeek?

Index your videos in a Mixpeek collection, search them with a retriever, and run VideoJev on your own GPU over the clips that come back, as in the example on this page, to trim each hit to the seconds that match.