1.  [](/)
2.  [Curated Lists](/curated-lists)
3.  Best Vision-Language Models (VLMs)

[Back to All Lists](/curated-lists)

# Best Vision-Language Models (VLMs) in 2026

A vision-language model reads images (and often video) alongside text and answers in natural language. We compared the leading 2026 VLMs, proprietary frontier APIs and open-weight models you can self-host, on multimodal reasoning, OCR and document understanding, license, and cost.

![Ethan Steininger](/people/ethan.webp)

By [Ethan Steininger](/blog/authors/ethan) · [LinkedIn](https://www.linkedin.com/in/ethansteininger/)

Published July 20, 2026 · Updated November 11, 2026

Last tested: August 28, 2026

11 tools evaluated

A VLM analyzes one image or clip at a time; it is not a search system. Mixpeek runs VLM-grade extraction (captions, on-screen text, objects, embeddings) over your video, images, and documents in your own object storage, then makes every result searchable through one API. The VLM is the extractor; Mixpeek is the index and query engine. Build starts at $25/mo for up to 1M vectors.

[Search what a VLM sees, across your whole library](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=curated_list&utm_campaign=managed_onramp&utm_content=vlms)

No account needed: [Query a hosted retriever over images and video](/marketplace)

[One-click templatesStart from a templateEach template deploys a working search app on your own files.Browse templates](/templates)

## Key Takeaways

*   Google Gemini 3 Pro ranks first, best for teams that want the highest multimodal reasoning quality and native long-video understanding via API. Native video + audio in a 1M-token context, with a Flash tier in the same family when latency and cost matter more than the reasoning ceiling.
*   OpenAI GPT-4o / GPT-5 vision is second, best for teams already on OpenAI who want strong vision plus the richest tooling. Frontier reasoning with the deepest developer ecosystem and structured-output tooling.
*   Mixpeek ranks #11, best for teams that need to search what a VLM sees across a whole library, not analyze one image at a time.
*   We ranked 11 options on multimodal reasoning (30%), ocr & document understanding (25%), license & deployability (25%) and cost & efficiency (20%). Prices and claims were checked against each vendor's public pages in August 2026.

Skip the comparison? Mixpeek runs vision-language models (vlms) on your own data: extraction, indexing, and search in one platform.

[Get started](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=curated_list_quick_answer&utm_campaign=mvs_onramp)

## How We Evaluated

Evaluated by the Mixpeek engineering team, who build and operate multimodal retrieval infrastructure in production. Last tested August 2026; rankings re-checked when the market shifts, with pricing and claims verified against each vendor's public documentation.

### Multimodal Reasoning

30%

General image-and-text reasoning quality on public benchmarks like MMMU and ChartQA: can the model answer hard questions about what it sees?

### OCR & Document Understanding

25%

Reading text, tables, and structured fields from documents, screenshots, and photos (OCRBench, DocVQA): the most common production VLM job.

### License & Deployability

25%

Open weights vs API-only, license terms (Apache 2.0 / MIT vs restricted vs closed), context length, and whether you can self-host for data residency.

### Cost & Efficiency

20%

Price per image or token for APIs, parameter count and GPU footprint for open weights, and available model sizes to trade quality against latency.

## Quick answer

The short version, before the detail:

*   [Google Gemini 3 Pro](#item-1)best for teams that want the highest multimodal reasoning quality and native long-video understanding via apiNative video + audio in a 1M-token context, with a Flash tier in the same family when latency and cost matter more than the reasoning ceiling
*   [OpenAI GPT-4o / GPT-5 vision](#item-2)best for teams already on openai who want strong vision plus the richest toolingFrontier reasoning with the deepest developer ecosystem and structured-output tooling
*   [Anthropic Claude (vision)](#item-3)best for document-heavy and agentic workloads that value careful, reliable visual readingCareful, low-hallucination document and chart reading, tuned for agentic use
*   [Qwen3-VL (Alibaba)](#item-4)best for teams that want top open-weight quality with strong ocr and video, self-hosted or via a providerThe strongest all-round open-weight VLM, with long-video localization and strong OCR
*   [Meta Llama 4 (multimodal)](#item-5)best for teams standardizing on the llama ecosystem that want strong open-weight visionTop open-weight general-vision benchmarks inside the largest open model ecosystem
*   [InternVL3 (OpenGVLab)](#item-6)best for teams that need top open-weight quality under a genuinely permissive (mit) licenseFrontier open-weight quality with a clean MIT license and no use restrictions
*   [InternVideo3-8B-Instruct](#item-7)best for long-video reasoning where a question spans an entire recording rather than one frameA 256K context aimed at hour-scale video, in a size one GPU can hold
*   [Mistral Pixtral](#item-8)best for teams wanting permissively-licensed open weights with an easy hosted fallbackApache-2.0 open weights with strong instruction following and a clean managed API

## Overview

Short answer: for the highest ceiling on multimodal reasoning and the longest context, use a proprietary frontier model, Google Gemini 3 Pro (1M-token context, native video and audio), OpenAI's GPT-5 vision, or Anthropic Claude; for the strongest open weights you can self-host, Qwen3-VL and Meta's Llama 4 lead general vision benchmarks, InternVL3 is the best permissively (MIT) licensed option, Pixtral is a strong Apache-2.0 choice, Molmo2 is the one to reach for when you need to point at or track objects across video frames, and Phi-4 multimodal is the efficient small model. Model choice is task-shaped rather than a single ranking: Molmo2 at 8B open weights beats Gemini 3 Pro on video pointing (38.4 vs 20.0 F1) while trailing it badly on general reasoning. If your goal is to search what a VLM sees across a whole library rather than analyze one image at a time, a VLM is the extractor, not the retrieval system: pair it with a vector store or a platform like Mixpeek. A vision-language model (VLM) fuses a vision encoder with a language model so it can look at an image, chart, screenshot, document page, or video frame and reason about it in words, answering questions, extracting fields, describing scenes, reading text, or grounding a region. The 2026 field splits two ways. Proprietary frontier models (Gemini, GPT-4o/GPT-5, Claude) hold the top of the quality leaderboards and offer the longest context and native video, but you rent them per token and cannot self-host. Open-weight models closed most of the gap: Qwen3-VL, Llama 4, InternVL3, Pixtral, Molmo, and Phi-4 multimodal now score competitively on public benchmarks (MMMU for reasoning, OCRBench and DocVQA for document reading, ChartQA for charts) at a fraction of the deployment cost, and you can run them on your own GPUs with full data residency. The axes that actually decide it are reasoning and OCR quality on your own documents, license terms, context length, and cost per image or token at your volume. One thing no VLM does on its own: retrieval. Answering "find every clip where X happens" across thousands of videos means running the VLM once per item at ingest to produce searchable signals and embeddings, then indexing and searching those, which is where a vector store or a retrieval platform comes in. Learn the internals in our guide on [how vision-language models work](/guides/vision-language-model-architecture), and see the related lists for [multimodal embedding models](/curated-lists/best-multimodal-embedding-models), [document parsing tools](/curated-lists/best-document-parsing-tools), and [OCR APIs](/curated-lists/best-ocr-apis).

## Best Vision-Language Models (VLMs): comparison at a glance

#

Tool

Best for

Pricing

Key differentiator

Main limit

1

[Google Gemini 3 Pro](#item-1)

Teams that want the highest multimodal reasoning quality and native long-video understanding via API

API, usage-based per token (Google AI Studio / Vertex AI); free tier in AI Studio

Native video + audio in a 1M-token context, with a Flash tier in the same family when latency and cost matter more than the reasoning ceiling

Proprietary and API-only: no self-hosting or data residency

2

[OpenAI GPT-4o / GPT-5 vision](#item-2)

Teams already on OpenAI who want strong vision plus the richest tooling

API, usage-based per token (images billed as tokens); OpenAI platform pricing

Frontier reasoning with the deepest developer ecosystem and structured-output tooling

Proprietary and API-only; images priced as tokens can get costly

3

[Anthropic Claude (vision)](#item-3)

Document-heavy and agentic workloads that value careful, reliable visual reading

API, usage-based per token (Anthropic); tiered by model (Haiku / Sonnet / Opus)

Careful, low-hallucination document and chart reading, tuned for agentic use

Proprietary and API-only; no weight access

4

[Qwen3-VL (Alibaba)](#item-4)

Teams that want top open-weight quality with strong OCR and video, self-hosted or via a provider

Open weights (Apache 2.0 on most sizes); free to self-host, or hosted via Alibaba and third-party providers

The strongest all-round open-weight VLM, with long-video localization and strong OCR

Largest sizes need real GPU memory to serve at low latency

5

[Meta Llama 4 (multimodal)](#item-5)

Teams standardizing on the Llama ecosystem that want strong open-weight vision

Open weights under Meta's Llama Community License (some use restrictions); free to self-host

Top open-weight general-vision benchmarks inside the largest open model ecosystem

Llama Community License has use restrictions (not fully open)

6

[InternVL3 (OpenGVLab)](#item-6)

Teams that need top open-weight quality under a genuinely permissive (MIT) license

Open weights (MIT license); free to self-host

Frontier open-weight quality with a clean MIT license and no use restrictions

Smaller community than Qwen or Llama

7

[InternVideo3-8B-Instruct](#item-7)

Long-video reasoning where a question spans an entire recording rather than one frame

Open weights; free to self-host

A 256K context aimed at hour-scale video, in a size one GPU can hold

No published benchmark scores yet, so quality claims cannot be compared to InternVL3 or Qwen3-VL on equal footing

8

[Mistral Pixtral](#item-8)

Teams wanting permissively-licensed open weights with an easy hosted fallback

Open weights (Apache 2.0); free to self-host, or usage-based via the Mistral API

Apache-2.0 open weights with strong instruction following and a clean managed API

Tops out below the largest Qwen / Llama / InternVL models on raw benchmarks

9

[Molmo2 (Allen AI)](#item-9)

Teams that need to locate and track objects across video frames, and researchers who need a fully reproducible VLM

Open weights (Apache 2.0), open training data; free to self-host

Open weights AND open training data, with video grounding that beats frontier models at pointing and counting

Smaller ecosystem and fewer integrations than Qwen or Llama

10

[Microsoft Phi-4 multimodal](#item-10)

Edge, on-device, or cost-constrained deployments needing a capable small VLM

Open weights (MIT license); free to self-host

Frontier-family quality-per-parameter in a small, MIT-licensed, edge-capable model

Lower ceiling than the 70B-class open models on hard reasoning

11

[Mixpeek](#item-11)

Teams that need to search what a VLM sees across a whole library, not analyze one image at a time

Plans from $25/mo plus usage; Managed indexing from $1.485 per thousand images or pages, MVS Build covers up to 1M vectors

The retrieval layer VLMs lack: extract once at ingest, then search everything through one API

A retrieval platform, not a VLM you prompt directly

## The Mental Model That Makes Multimodal Search Click

![](https://i.ytimg.com/vi/6mQK21ivkSs/hqdefault.jpg)

### Jump to

[#1 Google Gemini 3 Pro](#item-1)[#2 OpenAI GPT-4o / GPT-5 vision](#item-2)[#3 Anthropic Claude (vision)](#item-3)[#4 Qwen3-VL (Alibaba)](#item-4)[#5 Meta Llama 4 (multimodal)](#item-5)[#6 InternVL3 (OpenGVLab)](#item-6)[#7 InternVideo3-8B-Instruct](#item-7)[#8 Mistral Pixtral](#item-8)[#9 Molmo2 (Allen AI)](#item-9)[#10 Microsoft Phi-4 multimodal](#item-10)[#11 Mixpeek](#item-11)

1

### Google Gemini 3 Pro

Google's frontier multimodal model, and the reasoning tier of the Gemini 3 family. Natively ingests text, images, video, and audio in a single very long context window (1M+ tokens), which makes it strong at reasoning over long documents and full-length video. The same family ships a cheaper, faster Flash tier (3.5 through 3.7 Flash, plus Flash Lite variants) that trades reasoning ceiling for latency and cost; we benchmarked seven of those generations on [metadata extraction](/curated-lists/best-ai-metadata-extraction-tools) in [Best Gemini Flash models for metadata extraction](/curated-lists/best-gemini-flash-models-for-metadata-extraction).

What Sets It Apart

Native video + audio in a 1M-token context, with a Flash tier in the same family when latency and cost matter more than the reasoning ceiling

#### Strengths

*   +Top-tier multimodal reasoning across public vision benchmarks
*   +Native video and audio, not just images
*   +1M+ token context: reason over long PDFs and hour-long video
*   +Free tier in Google AI Studio; scales on Vertex AI

#### Limitations

*   \-Proprietary and API-only: no self-hosting or data residency
*   \-Per-token cost adds up at high image or video volume
*   \-Closed weights; you cannot fine-tune on your own data
*   \-Not the leader on every visual task: Molmo2-8B, an open 8B model, scores higher on video pointing (38.4 vs 20.0 F1)

#### Real-World Use Cases

*   •Answering questions across an entire multi-hundred-page contract or an hour-long recording in a single prompt
*   •Prototyping a multimodal agent that needs the strongest available visual reasoning before optimizing cost

#### Choose This When

When you need the best reasoning ceiling, native long-video understanding, or the longest context, and API-only is acceptable.

#### Skip This If

When you must self-host for data residency, need to fine-tune, or your volume makes per-token pricing expensive.

API, usage-based per token (Google AI Studio / Vertex AI); free tier in AI Studio

Best for: Teams that want the highest multimodal reasoning quality and native long-video understanding via API

[Visit Website](https://deepmind.google/technologies/gemini/)

2

### OpenAI GPT-4o / GPT-5 vision

OpenAI's flagship multimodal models read images alongside text with strong general reasoning and the deepest tooling ecosystem (SDKs, function calling, structured outputs, assistants). The default choice for teams already building on OpenAI.

What Sets It Apart

Frontier reasoning with the deepest developer ecosystem and structured-output tooling

#### Strengths

*   +Excellent general multimodal reasoning and instruction following
*   +Deepest ecosystem: SDKs, structured outputs, function calling, tools
*   +Fast, well-documented, widely integrated
*   +Strong at chart, screenshot, and UI understanding

#### Limitations

*   \-Proprietary and API-only; images priced as tokens can get costly
*   \-No self-hosting or weight access
*   \-Long-video support lags native-video models like Gemini

#### Real-World Use Cases

*   •Adding image understanding to an existing OpenAI-based agent with function calling and JSON outputs
*   •Extracting structured fields from receipts, forms, and screenshots via a single API

#### Choose This When

When you are already building on OpenAI and want strong image reasoning with mature SDKs and structured outputs.

#### Skip This If

When you need native long-video, self-hosting, or the lowest cost at very high image volume.

API, usage-based per token (images billed as tokens); OpenAI platform pricing

Best for: Teams already on OpenAI who want strong vision plus the richest tooling

[Visit Website](https://platform.openai.com/docs/guides/vision)

3

### Anthropic Claude (vision)

Anthropic's Claude models read images alongside text with a strong reputation for careful document reading, chart and diagram comprehension, and reliable, low-hallucination outputs: a common pick for agentic and document-heavy workloads.

What Sets It Apart

Careful, low-hallucination document and chart reading, tuned for agentic use

#### Strengths

*   +Strong, careful document and chart understanding
*   +Reliable instruction following with fewer hallucinations
*   +Large context for long multi-page documents
*   +Well-suited to agentic and tool-use workflows

#### Limitations

*   \-Proprietary and API-only; no weight access
*   \-Per-token image cost at scale
*   \-Image-focused; video is not a native input

#### Real-World Use Cases

*   •Reading complex multi-page financial or legal documents where a wrong extraction is expensive
*   •Powering an agent that interprets screenshots and charts as part of a tool-use loop

#### Choose This When

When accuracy and reliability on documents and charts matter more than native video, and API-only is fine.

#### Skip This If

When you need native video input, self-hosting, or the absolute lowest per-image cost.

API, usage-based per token (Anthropic); tiered by model (Haiku / Sonnet / Opus)

Best for: Document-heavy and agentic workloads that value careful, reliable visual reading

[Visit Website](https://www.anthropic.com/claude)

4

### Qwen3-VL (Alibaba)

The leading open-weight VLM family in 2026. Qwen3-VL (and the widely-deployed Qwen2.5-VL before it) offers multiple sizes, strong OCR and document understanding, hour-long video with timestamp-level localization, and permissive licensing on most sizes: the default open-weight choice.

What Sets It Apart

The strongest all-round open-weight VLM, with long-video localization and strong OCR

#### Strengths

*   +Best-in-class open-weight reasoning and OCR (Qwen2.5-VL-72B ~70% MMMU, ~888 OCRBench on public benchmarks)
*   +Multiple sizes from a few billion to 72B+ parameters
*   +Long-video understanding with timestamp-level localization
*   +Mostly permissive licensing; fully self-hostable and fine-tunable

#### Limitations

*   \-Largest sizes need real GPU memory to serve at low latency
*   \-You own serving, scaling, and updates
*   \-License terms vary by size: check the specific checkpoint

#### Real-World Use Cases

*   •Self-hosting document extraction and video Q&A under a permissive license for data-residency-sensitive workloads
*   •Fine-tuning a VLM on in-domain documents to beat a generic API on your own forms

#### Choose This When

When you want the best open-weight quality, need strong OCR or video, and want to self-host or fine-tune.

#### Skip This If

When you have no GPU capacity or want a fully managed frontier model with zero ops.

Open weights (Apache 2.0 on most sizes); free to self-host, or hosted via Alibaba and third-party providers

Best for: Teams that want top open-weight quality with strong OCR and video, self-hosted or via a provider

[Visit Website](https://github.com/QwenLM/Qwen3-VL)

5

### Meta Llama 4 (multimodal)

Meta's multimodal Llama 4 models (Scout and Maverick) bring native image understanding into the Llama ecosystem, leading several public vision benchmarks among open-weight models and backed by the broadest tooling and community support of any open model family.

What Sets It Apart

Top open-weight general-vision benchmarks inside the largest open model ecosystem

#### Strengths

*   +Top open-weight MMMU scores (Llama 4 Maverick ~73% on public benchmarks)
*   +Huge ecosystem, tooling, and community
*   +Multiple sizes; strong general reasoning
*   +Self-hostable with wide inference-stack support

#### Limitations

*   \-Llama Community License has use restrictions (not fully open)
*   \-Large sizes are compute-heavy to serve
*   \-OCR and long-video localization can trail Qwen3-VL

#### Real-World Use Cases

*   •Adding vision to an existing Llama-based stack without switching model families
*   •Self-hosting a capable general-purpose VLM on existing GPU infrastructure

#### Choose This When

When you already build on Llama and want strong open-weight multimodal reasoning with broad tooling.

#### Skip This If

When the Llama license restrictions are a problem, or you need the strongest OCR / video localization.

Open weights under Meta's Llama Community License (some use restrictions); free to self-host

Best for: Teams standardizing on the Llama ecosystem that want strong open-weight vision

[Visit Website](https://www.llama.com/)

6

### InternVL3 (OpenGVLab)

The strongest MIT-licensed open-weight VLM family, InternVL3 scales to 78B parameters and scores near the top of open-weight leaderboards while carrying a genuinely permissive license: the pick when license terms are a hard requirement.

What Sets It Apart

Frontier open-weight quality with a clean MIT license and no use restrictions

#### Strengths

*   +Top-tier open-weight quality (InternVL3-78B ~72% MMMU on public benchmarks)
*   +MIT license: the most permissive of the frontier open models
*   +Multiple sizes; strong document and chart understanding
*   +Fully self-hostable and fine-tunable with no use restrictions

#### Limitations

*   \-Smaller community than Qwen or Llama
*   \-Largest sizes are GPU-heavy
*   \-Fewer turnkey hosted-API options than Qwen or Pixtral

#### Real-World Use Cases

*   •Shipping a VLM inside a commercial product where a permissive MIT license removes legal review friction
*   •Research or regulated deployments that require full weight access and unrestricted licensing

#### Choose This When

When license terms are non-negotiable (MIT) and you still want near-top open-weight quality.

#### Skip This If

When you want the largest community, the most hosted options, or the very best OCR/video.

Open weights (MIT license); free to self-host

Best for: Teams that need top open-weight quality under a genuinely permissive (MIT) license

[Visit Website](https://github.com/OpenGVLab/InternVL)

7

### InternVideo3-8B-Instruct

A video-specialist VLM rather than an image model that tolerates frames: a 256K-token context built for hour-scale footage and agentic tool use, at 8B parameters. Pick it when the unit of work is a long video and the question spans the whole thing.

What Sets It Apart

A 256K context aimed at hour-scale video, in a size one GPU can hold

#### Strengths

*   +256K-token context, sized for hour-scale footage instead of a handful of sampled frames
*   +8B parameters, so it runs on a single modern GPU rather than a multi-card host
*   +Built for agentic tool use, not just single-turn captioning
*   +Open weights on Hugging Face

#### Limitations

*   \-No published benchmark scores yet, so quality claims cannot be compared to InternVL3 or Qwen3-VL on equal footing
*   \-The model card's own quickstart points at a repository that does not publicly resolve; use the yanziang path
*   \-Newer and less battle-tested than the InternVL line it descends from
*   \-Frame budget, not token count, is what binds on very long video

#### Real-World Use Cases

*   •Asking a question about a two-hour recording where the answer depends on events far apart in time
*   •Driving an agent that inspects video and calls tools, rather than returning a single caption

#### Choose This When

When your unit of work is a long video and the answer depends on what happened across the whole thing, not a sampled frame.

#### Skip This If

When you need published benchmark numbers to justify the choice, or your content is images and documents rather than long video.

Open weights; free to self-host

Best for: Long-video reasoning where a question spans an entire recording rather than one frame

[Visit Website](https://huggingface.co/yanziang/InternVideo3-8B-Instruct)

8

### Mistral Pixtral

Mistral's open-weight VLM line (Pixtral 12B and Pixtral Large) pairs strong instruction following and document understanding with Apache-2.0 licensing and a clean hosted API: a balanced choice for teams that want open weights plus an easy managed path.

What Sets It Apart

Apache-2.0 open weights with strong instruction following and a clean managed API

#### Strengths

*   +Apache 2.0: fully permissive open weights
*   +Strong instruction following and document reading for its size
*   +Both a compact 12B and a larger flagship available
*   +Self-host or use the Mistral API with no license friction

#### Limitations

*   \-Tops out below the largest Qwen / Llama / InternVL models on raw benchmarks
*   \-Smaller ecosystem than Qwen or Llama
*   \-No native video input

#### Real-World Use Cases

*   •Self-hosting a permissively-licensed VLM for document field extraction with a managed API as overflow
*   •Adding image understanding to a Mistral-based stack under one vendor and one license

#### Choose This When

When you want Apache-2.0 open weights, good document understanding, and the option of a hosted API.

#### Skip This If

When you need the absolute top benchmark scores or native video.

Open weights (Apache 2.0); free to self-host, or usage-based via the Mistral API

Best for: Teams wanting permissively-licensed open weights with an easy hosted fallback

[Visit Website](https://mistral.ai/news/pixtral-large)

9

### Molmo2 (Allen AI)

Allen AI's Molmo2 is one of the few VLMs that publishes open weights AND open training data, so a result can be reproduced end to end rather than taken on trust. [Molmo2-8B](/model/allenai/Molmo2-8B) pairs a Qwen3-8B language backbone with a [SigLIP 2](/models/siglip2-so400m-patch14-384) vision encoder and handles image, multi-image, and video input. Its distinguishing skill is spatial grounding: it points at, tracks, and counts objects across frames, beating Qwen3-VL on video counting (35.5 vs 29.6) and Gemini 3 Pro on video pointing (38.4 vs 20.0 F1).

What Sets It Apart

Open weights AND open training data, with video grounding that beats frontier models at pointing and counting

#### Strengths

*   +Apache 2.0 with open training data: maximally transparent and reproducible
*   +Video understanding with object pointing, tracking, and counting
*   +Outperforms far larger proprietary models on video pointing and counting
*   +8B runs on a single GPU, so grounding is cheap to deploy

#### Limitations

*   \-Smaller ecosystem and fewer integrations than Qwen or Llama
*   \-Less tuned for long-document OCR than Qwen3-VL or InternVL3
*   \-Grounding strength does not carry over to general multimodal reasoning, where frontier APIs still lead

#### Real-World Use Cases

*   •Building a grounded visual agent that must point at the region it is referring to
*   •Temporal grounding for video RAG: locating which frames contain the object a query names
*   •Research requiring a fully reproducible, open-data VLM

#### Choose This When

When you need to point at, track, or count objects across video frames, or when full reproducibility (open data + weights) is a requirement.

#### Skip This If

When you need the strongest document OCR, the highest general reasoning ceiling, or a large integration ecosystem.

Open weights (Apache 2.0), open training data; free to self-host

Best for: Teams that need to locate and track objects across video frames, and researchers who need a fully reproducible VLM

[Visit Website](https://molmo.allenai.org/)

10

### Microsoft Phi-4 multimodal

Microsoft's small multimodal model brings vision (and speech) into the efficient Phi family, delivering strong quality-per-parameter under an MIT license: the pick when you need a capable VLM that runs on modest hardware or at the edge.

What Sets It Apart

Frontier-family quality-per-parameter in a small, MIT-licensed, edge-capable model

#### Strengths

*   +MIT license; small and efficient
*   +Strong quality for its size; runs on modest GPUs or the edge
*   +Handles image and speech input, not just text
*   +Cheap to serve at scale

#### Limitations

*   \-Lower ceiling than the 70B-class open models on hard reasoning
*   \-Best for focused tasks, not open-ended frontier reasoning
*   \-Smaller context than the frontier APIs

#### Real-World Use Cases

*   •Running visual field extraction on-device or at the edge where a 70B model is impractical
*   •High-volume, cost-sensitive image classification and captioning where a small model is enough

#### Choose This When

When you need a small, efficient, permissively-licensed VLM for focused tasks on modest hardware.

#### Skip This If

When you need top-end reasoning on hard, open-ended visual questions or the longest context.

Open weights (MIT license); free to self-host

Best for: Edge, on-device, or cost-constrained deployments needing a capable small VLM

[Visit Website](https://huggingface.co/microsoft/Phi-4-multimodal-instruct)

11

### Mixpeek

Our Pick

[Try MVS](/start)

Not a VLM itself, Mixpeek is where VLM output becomes searchable. It runs VLM-grade extraction (captions, on-screen text, objects, faces, embeddings) over your video, images, audio, and documents in your own object storage, then indexes and serves it through one retrieval API for humans and agents.

What Sets It Apart

The retrieval layer VLMs lack: extract once at ingest, then search everything through one API

Use with MVS

Use any VLM as the extractor, then upsert its embeddings and structured signals into an MVS retriever to search your whole video and document library in milliseconds.

#### Strengths

*   +Turns per-item VLM analysis into library-scale search
*   +Runs over your own object storage (S3/GCS/B2) with agent-native APIs
*   +Spans video, images, audio, and documents in one pipeline
*   +Model-agnostic: swap the underlying encoder without rewriting your app

#### Limitations

*   \-A retrieval platform, not a VLM you prompt directly
*   \-For single-image Q&A a VLM API alone is simpler
*   \-Most valuable when you need search across many items, not one

#### Real-World Use Cases

*   •Making an entire video archive searchable by what was shown, said, and written on screen
*   •Building an agent that retrieves the exact moments and pages relevant to a question across thousands of files

#### Choose This When

When the job is find-across-a-library rather than analyze-one-image, and you want extraction, indexing, and search in one system.

#### Skip This If

When you only need to prompt a model about a single image or clip: a VLM API alone is enough.

Plans from $25/mo plus usage; Managed indexing from $1.485 per thousand images or pages, MVS Build covers up to 1M vectors

Best for: Teams that need to search what a VLM sees across a whole library, not analyze one image at a time

[Get started](/start?from=curated_list)

## Which one fits your use case?

Find your situation in the first column. The last column says when the pick is the wrong call.

Choose it when

Pick

Look elsewhere when

You need the best reasoning ceiling, native long-video understanding, or the longest context, and API-only is acceptable.

[Google Gemini 3 Pro](#item-1)

You must self-host for data residency, need to fine-tune, or your volume makes per-token pricing expensive.

You are already building on OpenAI and want strong image reasoning with mature SDKs and structured outputs.

[OpenAI GPT-4o / GPT-5 vision](#item-2)

You need native long-video, self-hosting, or the lowest cost at very high image volume.

Accuracy and reliability on documents and charts matter more than native video, and API-only is fine.

[Anthropic Claude (vision)](#item-3)

You need native video input, self-hosting, or the absolute lowest per-image cost.

You want the best open-weight quality, need strong OCR or video, and want to self-host or fine-tune.

[Qwen3-VL (Alibaba)](#item-4)

You have no GPU capacity or want a fully managed frontier model with zero ops.

You already build on Llama and want strong open-weight multimodal reasoning with broad tooling.

[Meta Llama 4 (multimodal)](#item-5)

The Llama license restrictions are a problem, or you need the strongest OCR / video localization.

License terms are non-negotiable (MIT) and you still want near-top open-weight quality.

[InternVL3 (OpenGVLab)](#item-6)

You want the largest community, the most hosted options, or the very best OCR/video.

Your unit of work is a long video and the answer depends on what happened across the whole thing, not a sampled frame.

[InternVideo3-8B-Instruct](#item-7)

You need published benchmark numbers to justify the choice, or your content is images and documents rather than long video.

You want Apache-2.0 open weights, good document understanding, and the option of a hosted API.

[Mistral Pixtral](#item-8)

You need the absolute top benchmark scores or native video.

## Frequently Asked Questions

### What is Kakao's Kanana-2-v-decide, and is it a vision-language model?

Kanana-2-v-decide-3b-2611 is a 3.67B-parameter decision model that Kakao released on 6 November 2026, built on its Kanana-2-v-3b vision-language model. It does not generate text. Given text, JSON or a chat history with up to 10 images, it answers typed questions (pick one option, yes or no, or a point on a scale) and returns a probability for each answer, per its [model card](https://huggingface.co/kakaocorp/kanana-2-v-decide-3b-2611). Kakao reports the highest average (61.23) across the public Decision Index and Decision Index Vision benchmarks among the four models it compared. The weights are under the Kanana Open License: running it inside your own service needs no extra license, reselling access through an API, an on-premise product or a device needs a commercial license from Kakao, and products must display "Powered by Kanana". We have not benchmarked it. It suits labeling jobs where a generative VLM is slow and costly, such as routing, moderation flags and category picks. On Mixpeek, [taxonomies](https://mixpeek.com/docs/enrichment/taxonomies) classify each indexed file against a label set you define, and you can load this model from Hugging Face in a [custom extractor](https://mixpeek.com/docs/processing/model-registry) instead.

### What is Google's Gemini Decision 1, and how does it differ from a vision-language model?

Gemini Decision 1 (gemini-decision-1) is a scoring model Google announced on 11 November 2026 through its new Gemini Decisions API. It reads one shared input of text, images, audio, video or PDFs, up to 256K tokens, and answers up to 1,000 typed questions about it in one request: yes or no, a pick from 2 to 255 options, or a grade on a rubric. It returns a probability for each answer and generates no text. Google reports 150 to 250 ms end to end for inputs up to 16K tokens and prices it at $0.10 per million input tokens with output free, per the [Gemini Decisions API announcement](https://blog.google/innovation-and-ai/technology/developers-tools/gemini-decisions-api/). The accuracy and calibration claims are Google's own, and we have not benchmarked it. It suits policy checks, routing and moderation flags where a generative VLM is slower than you need. Mixpeek does not call Gemini Decision 1 today. Inside a Mixpeek retriever, the [LLM Filter stage](https://mixpeek.com/docs/retrieval/stages/llm-filter) keeps or drops each search result against criteria you write, and with include\_media on it sends a video segment to Gemini as video, so the judgment runs on what the clip shows.

### Is Liquid AI's LFM3-VL-2B a good small vision-language model?

LFM3-VL-2B is Liquid AI's on-device vision-language model, released on 27 October 2026 with open weights under Liquid AI's own LFM license. It has 1.92B parameters (a 1.73B LFM3 language model, a 164M vision encoder and a 25M projector), takes text, images and video (sampled at 2 frames a second), and supports 16 languages, per its [model card](https://huggingface.co/LiquidAI/LFM3-VL-2B). It is the first Liquid vision model that accepts video. We have not benchmarked it yet, so treat it as a candidate for edge and low-cost captioning or visual question answering and test it on your own images. On Mixpeek you can load it from Hugging Face inside a [custom extractor](https://mixpeek.com/docs/processing/model-registry) to describe images or video at indexing time.

### What is a vision-language model (VLM)?

A vision-language model fuses a vision encoder (which turns pixels into features) with a large language model (which reasons and generates text), so a single model can look at an image, chart, screenshot, document page, or video frame and answer questions about it in natural language. Modern VLMs like Gemini, GPT-4o, Claude, and Qwen3-VL can describe scenes, read and extract text, interpret charts and tables, ground objects to regions, and follow multi-step visual instructions. They are the general-purpose successor to task-specific computer-vision models: instead of a separate model per task, one VLM handles captioning, OCR, VQA, and reasoning through prompts.

### Should I use an open-weight VLM or a proprietary API?

Use a proprietary API (Gemini 3 Pro, GPT-5, Claude) when you want the highest reasoning ceiling, the longest context, native video, and zero infrastructure, and your volume keeps per-token costs reasonable. Use an open-weight model (Qwen3-VL, Llama 4, InternVL3, Pixtral, Molmo2, Phi-4 multimodal) when you need data residency, want to fine-tune on your own documents, must control cost at high volume, or require a specific license. The open-weight gap to the frontier has narrowed sharply (top open models are competitive on MMMU, OCRBench, and DocVQA) so for many production tasks (document extraction, classification, captioning) a self-hosted model is both cheaper and sufficient. Always benchmark on your own images rather than trusting a leaderboard.

### Which benchmarks matter for vision-language models?

The most-cited public VLM benchmarks are MMMU (college-level multi-discipline visual reasoning), OCRBench and DocVQA (reading text and answering questions over documents and scanned pages), ChartQA and AI2D (charts and diagrams), and MMBench/MME (broad multimodal capability). For production, weight the benchmark that matches your job: OCRBench and DocVQA for document extraction, ChartQA for analytics screenshots, MMMU for open-ended reasoning. Treat leaderboard scores as a shortlist, not a verdict: real documents differ from benchmark distributions, so run a sample of your own data through the top two or three candidates before committing.

### Can a VLM do OCR and document parsing?

Yes, and increasingly it is the preferred approach for hard layouts. A VLM reads a page image directly and emits markdown or structured JSON in one pass, handling dense tables, multi-column layouts, handwriting, and scanned pages that defeat rule-based OCR. Qwen3-VL, InternVL3, and the frontier APIs all score strongly on OCRBench and DocVQA. The tradeoff versus a dedicated parser is cost and latency (a model call per page) and consistency, so a common production pattern is to route clean, born-digital documents to a fast rule-based parser and send only the hard pages to a VLM. See our list of [document parsing tools](/curated-lists/best-document-parsing-tools) for the parser side of this decision.

### Can a vision-language model search my video or image library?

Not by itself. A VLM analyzes one image or clip per call, so asking it to "find every video where the logo appears" across a library would mean running it on every item for every query: far too slow and expensive at scale. The scalable pattern is to run a VLM (or a video foundation model) once per item at ingest to produce captions, extracted text, and embeddings, then index those in a vector store and answer queries with fast similarity search. That is exactly what Mixpeek does: it runs the extraction over your object storage and makes the whole library searchable through one API, so the VLM handles perception and the platform handles retrieval.

### Which vision-language model is best for video?

It depends on which video job you mean, and the answer flips between two of them. For reasoning over a full-length video in one pass, a frontier API with native video and a very long context wins: Gemini 3 Pro ingests video and audio directly inside a 1M+ token window, so you can ask questions about an hour of footage without building a pipeline. For locating things inside frames, pointing at an object, tracking it across time, or counting instances, an open 8B model currently leads: Molmo2-8B scores 38.4 F1 on video pointing against Gemini 3 Pro's 20.0, and beats Qwen3-VL on video counting (35.5 vs 29.6). Qwen3-VL is the strongest general-purpose open option when you want one model for both. A third shape arrived in August 2026: video-specialist models such as InternVideo3-8B-Instruct, which carries a 256K-token context aimed at hour-scale footage in a size a single GPU can hold. It has no published benchmark scores yet, so treat it as promising rather than proven, and compare it on your own footage before betting on it. Note that none of these searches a library; per-item extraction plus a vector index does that.

### How much does it cost to run a VLM over a large media library?

Cost is driven by how many items you process, not by how many queries you run, provided you extract once at ingest instead of calling the model per query. A proprietary API charges per token, and images and video frames consume tokens quickly, so a library in the millions of items usually pushes teams toward open weights. An 8B open model like Molmo2 or Phi-4 multimodal runs on a single GPU, which turns a per-token bill into a fixed hourly one. The pattern that keeps both bounded is the same: run the VLM once per asset to produce captions, text, and embeddings, store those, and serve every subsequent query from the index rather than from the model.

### Are open-weight VLMs good enough to replace GPT-5 or Gemini?

For a specific task, often yes; as a general replacement, not yet. Open models closed most of the gap on document reading and captioning, which is where the majority of production VLM work sits, and on some narrow tasks they now lead outright, as Molmo2 does on video pointing and counting. What frontier APIs still hold is the reasoning ceiling on open-ended questions, the longest context windows, and native audio alongside video. The practical test is unglamorous: take 50 real examples from your own data, run your two or three candidates, and score them on the exact output your pipeline consumes. Benchmark distributions rarely resemble a specific corpus, so a leaderboard should narrow the shortlist rather than decide it.

### What is the difference between a vision-language model and a multimodal embedding model?

A VLM generates text: you give it an image and a question, and it answers in words. A multimodal embedding model generates vectors: it maps images and text into one shared space so that similar things land near each other, with no language output at all. They solve different halves of a search system. The embedding model is what makes "find clips that look like this" work across millions of items, because comparing vectors is cheap. The VLM is what produces rich descriptions, extracted fields, and answers about a specific item, which is expensive per call. Most production pipelines use both: a VLM at ingest for captions and structured fields, an embedding model for the searchable vectors. See [multimodal embedding models](/curated-lists/best-multimodal-embedding-models) for the retrieval side.

## See how Mixpeek handles this

Purpose-built for vision-language models (vlms), not bolted on.

### Talk to a Mixpeek engineer: free

30 minutes. Bring your use case and we'll tell you exactly what would work and what wouldn't.

[Schedule a Free Call](https://calendly.com/mixpeek/30min?hide_gdpr_banner=1&background_color=ffffff&text_color=1a1a2e&primary_color=6366f1&utm_source=curated_list_modal)

## Explore Other Curated Lists

multimodal ai

### Best Multimodal AI APIs

A hands-on comparison of the top multimodal AI APIs for processing text, images, video, and audio through a single integration. We evaluated latency, modality coverage, retrieval quality, and developer experience.

11 tools ranked[View List](/curated-lists/best-multimodal-ai-apis)

multimodal ai

### Best Feature Extraction APIs

A technical evaluation of APIs for extracting features, embeddings, and structured data from unstructured content. Covers text, image, video, and audio feature extraction for AI applications.

10 tools ranked[View List](/curated-lists/best-feature-extraction-apis)

multimodal ai

### Best Multimodal Embedding Models

What is the best multimodal embedding model? For image and text search you can self-host, SigLIP 2. For PDFs, slides and screenshots, a document model such as GME or Cohere Embed v4. For video and audio in the same space as text, an API model such as TwelveLabs Marengo 3.0 or Amazon Nova Multimodal Embeddings, or EmbeddingGemma 2 when the model has to run on your own hardware or on the device. CLIP is still the best-supported baseline. The 14 options below compare quality, hosting, price and fit.

14 tools ranked[View List](/curated-lists/best-multimodal-embedding-models)