[Models](/models)/Alibaba-NLP/gme-Qwen2-VL-2B-Instruct

Apache-2.0

# gme-Qwen2-VL-2B-Instruct

by Alibaba-NLP

Unified multimodal embedding: text, images, and visual documents in one vector space

2.2Bparams

[HuggingFace](https://huggingface.co/Alibaba-NLP/gme-Qwen2-VL-2B-Instruct)[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)

Identifiers

Model ID

`Alibaba-NLP/gme-Qwen2-VL-2B-Instruct`

## Deploy gme-Qwen2-VL-2B-Instruct

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

[Read the customer-hosted guide](https://mixpeek.com/docs/customer-hosted/overview) [Talk to us about single-tenant](/demo)

## Overview

GME (General Multimodal Embedding) turns a vision-language model, Qwen2-VL-2B-Instruct, into a single-vector embedding model. Unlike a CLIP-style dual encoder that keeps separate towers for text and images, GME feeds text, an image, or an interleaved image+text pair through the same VLM and pools the final hidden states into one dense vector. Because every modality lands in the same space, you can retrieve any-to-any: text→image, image→text, text→text, and (image+text)→(image+text).

The practical payoff is visual-document retrieval. A slide, a scanned invoice, or a chart-heavy PDF page is encoded as an image, so layout, figures, and text are embedded together without a brittle OCR-then-embed pipeline. On Mixpeek, GME powers a single multimodal\_embedding index where an agent's text query reaches document pages, product photos, and screenshots side by side.

## Architecture

Built on Qwen2-VL-2B-Instruct (~2.2B params). The VLM encodes the input; the last-layer hidden states are mean-pooled into one dense vector (instruction-tunable via a task prompt). Inherits Qwen2-VL's dynamic-resolution vision encoder, so images are processed at native aspect ratio without forced 224×224 cropping: important for dense document pages. Trained with contrastive learning over text, image, and fused image-text pairs plus hard negatives.

## Mixpeek SDK Integration

IngestRetrieve

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so gme-Qwen2-VL-2B-Instruct runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace\_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns\_your\_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API\_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection\_id: "col\_your\_collection",
      documents: \[
        {
          document\_id: "asset-00412",
          // The vector name has to match a vector index on the collection.
          vectors: { "multimodal-embedding": yourVector },
          payload: { source\_key: "archive/2026/asset-00412" },
        },
      \],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// multimodal\_extractor@v2 runs google/gemini-embedding-2
// (3072-d) over a bucket, with no inference of your own.

## Capabilities

*   Single shared space for text, image, and interleaved image-text inputs
*   Any-to-any retrieval (text↔image, image↔image, fused queries)
*   Strong visual-document retrieval (ViDoRe): embeds page layout + figures, no OCR step
*   Dynamic image resolution inherited from Qwen2-VL (no fixed crop)
*   Instruction-conditioned embeddings via a task prompt

## Use Cases on Mixpeek

Multimodal RAG over corpora that mix text passages and images in one index

Visual-document search: slides, scanned PDFs, charts, and screenshots by text query

Product search where the query or catalog item is image + text together

Agent retrieval where queries and documents span modalities without per-modality pipelines

## Benchmarks

Dataset

Metric

Score

Source

UMRB (Universal Multimodal Retrieval Benchmark)

introduced + evaluated

\-

Introduced alongside the GME paper; the 2B model is a top unified-embedding performer for its size at release. See paper for full table.

ViDoRe (visual document retrieval)

nDCG@5

\-

Competitive screenshot/document retrieval without an OCR stage.

## Common Pipeline Companions

[

vidore/colqwen2.5-v0.2

Late-interaction reranker over the top visual-document candidates

](/model/vidore/colqwen2.5-v0.2)

## Explore on Mixpeek

[

Browse all models

Search the full model catalog

](/models)[

Feature Extractors

See how Mixpeek runs models as extractors

](/extractors)[

Mixpeek Vector Store

Store & search embeddings at scale

](/mvs)[

Pricing

Usage-based pricing for pipelines

](/pricing)[

All Curated Lists

Compare models, APIs & infrastructure

](/curated-lists)

### Specification

OrganizationAlibaba-NLP

Retriever\-

Parameters2.2B

LicenseApache-2.0

Downloads/moN/A

[

### Research Paper

GME: Improving Universal Multimodal Retrieval by Multimodal LLMs

arxiv.org](https://arxiv.org/abs/2412.16855)

### Build a pipeline with gme-Qwen2-VL-2B-Instruct

Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)