Models

One vector space for video segments, images, PDFs, audio and text queries; the default for object-level multimodal extractors (multimodal_extractor v2, universal_extractor).

Category-level text-to-video and text-to-image search in one 1408-d space (committed mAP 0.855 on a 10-video text eval).

Text-to-image and image-to-image search over photos, products and video frames in one shared space.

Face detection, 5-point alignment and 512-d face identity vectors for finding the same person across images and video frames.

The fastest face detector option (SCRFD-500MF) for high-volume frames where small faces matter less.

Image-to-image similarity on visual structure (layout, near-duplicates, the same object or scene) with no text in the loop.

The extractors' default for structured extraction and vision work: video description and OCR in multimodal_extractor v2, web page extraction, PDF layout (document_graph), scrolling-text OCR.

High-quality video QA and structured tagging.

Strongest SigLIP v1 image-text model; the vision tower behind many VLMs (PaliGemma).

Low-cost bulk tagging of scenes and frames.

Text search over document page images (PDF pages, slides, infographics, tables) by late interaction, with no OCR step.

Fast binary normal/nsfw screening of images, and of video by the max score over sparse sampled frames.

The platform's default LLM (DEFAULT_GEMINI_MODEL) for cheap per-document work in retriever stages (enrich, filter, summarize) and for video-segment description and OCR in multimodal_extractor v1.

Image-to-image similarity from a large web-trained ViT-L/14 image tower.

Logos, signage, brands and venue shots in concert footage (the visual arm of an internal eval); strongest SigLIP 2 below giant.

Multi-step tool use and reasoning; the default model of agent sessions (DEFAULT_AGENT_MODEL) and of agentic_enrich when no provider or model is given.

Real-time DETR with fine-grained box refinement; among the highest COCO AP per latency of the permissive detectors.

Cheapest Gemini video tagging and captioning at 1 fps.

Stronger video reasoning than 2.5 Flash at Flash latency.

Brand, artist, venue and moment tags per shot (the default tag model in a custom shot plugin).

Cheap short generations such as query expansion (query_expand's default expansion_model) and the fallback chat model of the universal inference endpoint.

Fast promptable masks; the size to try on CPU for image masks from detector boxes.

Product attributes in fashion catalogs: garment type, color, pattern, material, style words.

Fashion product search from text or image (ViT-B/16 CLIP fine-tuned with generalized contrastive learning on product titles, categories and attributes); Marqo reports gains over FashionCLIP 2.0 on its fashion benchmarks.