NEWVectors or files. Pick a path.Start →
    Models/Embeddings/nvidia/omni-embed-nemotron-3b
    HFVisual Embeddingsother

    omni-embed-nemotron-3b

    by nvidia

    Unified embedding model for text, image, audio, and video retrieval in a single vector space

    13Kdl/month
    128likes
    4.7Bparams
    Identifiers
    Model ID
    nvidia/omni-embed-nemotron-3b
    Feature URI
    mixpeek://image_extractor@v1/nvidia_omni_embed_nemotron_3b_v1

    Deploy omni-embed-nemotron-3b

    Single-tenant

    Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

    Overview

    Omni-Embed Nemotron is NVIDIA's omnimodal embedding model that encodes text, images, audio, and video into a shared 2048-dimensional vector space. Built on the Thinker component of Qwen2.5-Omni-3B, it processes each modality independently and projects into a single retrieval-ready embedding.

    On Mixpeek, Omni-Embed Nemotron enables true cross-modal search: query with text and retrieve matching video clips, audio segments, document pages, or images from a single index. One model replaces four separate embedding pipelines.

    Architecture

    Transformer-based encoder derived from Qwen2.5-Omni-3B (Thinker only, no Talker). 2048-dim output embeddings. 32K max context tokens. Modality-separated encoding with independent audio and video processing paths. 4.7B parameters.

    Mixpeek SDK Integration

    // No extractor parameter takes a Hugging Face model id (checked against
    // GET /v1/discovery/extractors, which returns 13), so omni-embed-nemotron-3b runs
    // on your side and the output is upserted through POST
    // /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
    // path is to upload the weights instead: POST /v1/namespaces/{id}/models
    // accepts the huggingface format and a custom plugin loads them.
    const res = await fetch(
      "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
      {
        method: "POST",
        headers: {
          Authorization: "Bearer API_KEY",
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          collection_id: "col_your_collection",
          documents: [
            {
              document_id: "asset-00412",
              // The vector name has to match a vector index on the collection.
              vectors: { "multimodal-embedding": yourVector },
              payload: { source_key: "archive/2026/asset-00412" },
            },
          ],
        }),
      },
    );
    
    // Managed alternative, if this exact model is not the requirement:
    // multimodal_extractor@v2 runs google/gemini-embedding-2
    // (3072-d) over a bucket, with no inference of your own.

    Capabilities

    • Unified text, image, audio, and video embeddings in one model
    • 2048-dimensional dense vectors for cross-modal retrieval
    • 32K token context window
    • State-of-the-art video retrieval among embedding models
    • Competitive visual document retrieval (85.7 nDCG@5 on ViDoRe V1)

    Use Cases on Mixpeek

    Cross-modal search: query with text, retrieve matching video clips or audio segments
    Unified media index: embed an entire multimedia library into one searchable vector space
    Podcast and meeting search: find audio moments matching visual or textual queries
    Video library retrieval: surface relevant clips by scene description or spoken content

    Benchmarks

    DatasetMetricScoreSource
    ViDoRe V1 (visual doc)nDCG@585.7%NVIDIA, 2025: Model Card
    MTEB text retrieval (10 tasks)nDCG@10 avg0.606NVIDIA, 2025: Model Card
    Video retrieval (LPM + FineVideo)nDCG@10 avg0.706NVIDIA, 2025: Model Card

    Performance

    Input SizeVariable (text/image/audio/video)
    Embedding Dim2048
    GPU Latency~18ms / item (A100)
    GPU Throughput~55 items/sec (A100)
    GPU Memory~9.5 GB

    Specification

    FrameworkHF
    Organizationnvidia
    FeatureVisual Embeddings
    Output768-dim vector
    Modalitiesvideo, image
    RetrieverVector Search
    Parameters4.7B
    Licenseother
    Downloads/mo13K
    Likes128

    Research Paper

    Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model

    arxiv.org

    Build a pipeline with omni-embed-nemotron-3b

    Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

    Run it on your own data, free