[Models](/models)/[Captioning](/models?category=captioning)/nvidia/4D-RGPT-8B

HFScene CaptioningCC-BY-NC-4.0

# 4D-RGPT-8B

by nvidia

8B video model for region-grounded 3D and 4D reasoning

108dl/month

13likes

8Bparams

[HuggingFace](https://huggingface.co/nvidia/4D-RGPT-8B)[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)

Identifiers

Model ID

`nvidia/4D-RGPT-8B`

## Deploy 4D-RGPT-8B

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

[Read the customer-hosted guide](https://mixpeek.com/docs/customer-hosted/overview) [Talk to us about single-tenant](/demo)

## Overview

4D-RGPT-8B is an NVIDIA video-text model focused on region grounding, 3D reasoning, and 4D reasoning. Those capabilities are important when an agent needs more than a clip-level summary. The agent needs to know which region changed, where the object moved, and how the event evolved over time.

On Mixpeek, 4D-RGPT can enrich video indexes with region-grounded temporal evidence. It is a fit for robotics footage, surveillance review, sports clips, and operational video where the retrieval result must preserve spatial and temporal context.

## Architecture

NVILA-Lite-8B based video-text-to-text model. The Hugging Face metadata tags it for video understanding, region grounding, 3D reasoning, 4D reasoning, and perceptual distillation.

## Mixpeek SDK Integration

IngestRetrieve

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so 4D-RGPT-8B runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace\_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns\_your\_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API\_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection\_id: "col\_your\_collection",
      documents: \[
        {
          document\_id: "asset-00412",
          // The model produces text, so it lands in payload. Give the
          // collection a text vector index and embed that text to make it
          // searchable rather than only filterable.
          payload: { extracted\_text: modelOutput, source\_key: "archive/2026/asset-00412" },
          vectors: { "multimodal-embedding": embeddingOfModelOutput },
        },
      \],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// universal\_extractor@v1 runs google/gemini-embedding-2
// (3072-d) over a bucket, with no inference of your own.

## Capabilities

*   Region-grounded video understanding
*   3D and 4D reasoning over spatial-temporal evidence
*   Video-text-to-text analysis for agent perception loops
*   Designed for grounding objects and events through time

## Use Cases on Mixpeek

Retrieve clips where an object moves through a specific region

Audit robotics or physical-world agent observations

Build evidence bundles for security and operations review

Search sports or live-event video with spatial-temporal constraints

## Performance

Input SizeVariable

GPU LatencyInput dependent

GPU ThroughputVideo length dependent

GPU Memory~18 GB

Region-grounded video reasoning cost depends heavily on clip length and frame sampling.

## Common Pipeline Companions

[

depth-anything/DA3-LARGE-1.1

Depth cues for spatial reasoning pipelines

](/model/depth-anything/DA3-LARGE-1.1)[

facebook/sam2.1-hiera-large

Mask and region proposals before video grounding

](/model/facebook/sam2.1-hiera-large)[

google/videoprism-large-f8r288

Video embedding baseline for retrieval

](/model/google/videoprism-large-f8r288)

## Explore on Mixpeek

[

More Captioning models

Compare alternatives in this category

](/models?category=captioning)[

Best Document AI Platforms

Hand-picked tools & platforms compared

](/curated-lists/best-document-ai-platforms)[

OCR & Document AI Internals

Deep-dive technical guide

](/guides/ocr-document-ai-internals)[

Feature Extractors

See how Mixpeek runs models as extractors

](/extractors)[

Mixpeek Vector Store

Store & search embeddings at scale

](/mvs)[

Pricing

Usage-based pricing for pipelines

](/pricing)[

All Curated Lists

Compare models, APIs & infrastructure

](/curated-lists)

### Specification

FrameworkHF

Organizationnvidia

FeatureScene Captioning

Outputtext

Modalitiesvideo, image

RetrieverSemantic Search

Parameters8B

LicenseCC-BY-NC-4.0

Downloads/mo108

Likes13

[

### Research Paper

4D-RGPT

arxiv.org](https://arxiv.org/abs/2512.17012)

### Build a pipeline with 4D-RGPT-8B

Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)

### Alternative Models

[

Salesforce/blip2-opt-2.7b

Scene Captioning

](/model/Salesforce/blip2-opt-2.7b)[

microsoft/Florence-2-large

Scene Captioning

](/model/microsoft/Florence-2-large)[

google/paligemma2-3b-mix-448

Scene Captioning

](/model/google/paligemma2-3b-mix-448)[

Qwen/Qwen3-VL-8B-Instruct

Scene Captioning

](/model/Qwen/Qwen3-VL-8B-Instruct)