Mage-VL

Name: Mage-VL
Author: microsoft

by microsoft

Vision-language model built for streaming video rather than single frames

4.7Bparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

microsoft/Mage-VL

Feature URI

Overview

Most vision-language models take one image and some text. Mage-VL is tagged for video understanding and streaming, which is a different problem: the input keeps arriving, and the model has to stay useful without re-reading everything it has already seen.

That matters for anyone indexing video, because the naive approach is to sample frames, embed each one independently, and lose every relationship between them. A model that consumes a stream can describe what changed, not just what is present.

At 4.7B parameters under Apache 2.0 it sits in the range you can self-host. Treat it as a describer and a reranker rather than a first-pass indexer: generating text for every frame of a large archive is expensive, and a cheap embedding model is the right thing to run first.

Architecture

MageVLForConditionalGeneration, model type mage_vl, 4,741,793,792 parameters. Image-text-to-text pipeline with video-understanding and streaming support declared on the model card. Requires trust_remote_code.

Mixpeek SDK Integration

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so Mage-VL runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection_id: "col_your_collection",
      documents: [
        {
          document_id: "asset-00412",
          // The model produces text, so it lands in payload. Give the
          // collection a text vector index and embed that text to make it
          // searchable rather than only filterable.
          payload: { extracted_text: modelOutput, source_key: "archive/2026/asset-00412" },
          vectors: { "multimodal-embedding": embeddingOfModelOutput },
        },
      ],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// universal_extractor@v1 runs google/gemini-embedding-2
// (3072-d) over a bucket, with no inference of your own.