NEWVectors or files. Pick a path.Start →
    Models/Segmentation/facebook/sam3
    PyTorchSegmentationother

    sam3

    by facebook

    Concept-level segmentation with open-vocabulary detection and video tracking

    2.3Mdl/month
    2,740likes
    860Mparams
    Identifiers
    Model ID
    facebook/sam3
    Feature URI
    mixpeek://image_extractor@v1/facebook_sam3_v1

    Overview

    SAM 3 is Meta's unified foundation model for concept-level segmentation. It detects, segments, and tracks objects using open-vocabulary text prompts or visual exemplars, handling 270K+ unique concepts. It bridges the gap between detection and segmentation in a single model.

    On Mixpeek, SAM 3 enables concept-driven content analysis: specify any concept in text and SAM 3 will find, segment, and track every instance across images and video.

    Architecture

    Decoupled detector-tracker architecture sharing a vision encoder. 848M total parameters. Uses a presence token for discriminating closely related prompts. Trained on 4M+ automatically annotated concepts.

    Mixpeek SDK Integration

    // No extractor parameter takes a Hugging Face model id (checked against
    // GET /v1/discovery/extractors, which returns 13), so sam3 runs
    // on your side and the output is upserted through POST
    // /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
    // path is to upload the weights instead: POST /v1/namespaces/{id}/models
    // accepts the huggingface format and a custom plugin loads them.
    const res = await fetch(
      "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
      {
        method: "POST",
        headers: {
          Authorization: "Bearer API_KEY",
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          collection_id: "col_your_collection",
          documents: [
            {
              document_id: "asset-00412",
              // Boxes, masks, depth maps and anomaly scores are structured
              // results, not vectors. They go in payload and are reachable
              // through pre_filters on a retriever, not through similarity.
              payload: {
                detections: modelOutput,
                source_key: "archive/2026/asset-00412",
              },
            },
          ],
        }),
      },
    );
    
    // No managed alternative for an open label set. Two extractors do emit a
    // bbox, for the one thing each detects: document_graph_extractor@v1 per
    // layout block, face_identity_extractor@v1 per face. Nothing ships that
    // returns masks, depth maps or anomaly scores.

    Capabilities

    • Open-vocabulary detection + segmentation (270K+ concepts)
    • Video tracking with mask propagation
    • Text and visual exemplar prompts
    • Concept-level exhaustive segmentation
    • Outperforms OWLv2, DINO-X, Gemini 2.5 on benchmarks

    Use Cases on Mixpeek

    Exhaustive concept detection across large video libraries
    Brand and logo tracking in video content
    Open-vocabulary content moderation at scale
    Concept-driven video analytics and tagging

    Benchmarks

    DatasetMetricScoreSource
    SA-V (video seg.)J&F83.2SAM 3 model card

    Performance

    Input Size1024×1024 px
    GPU Latency~15ms / frame (A100)
    GPU Throughput~66 frames/sec (A100)
    GPU Memory~3.0 GB

    Specification

    FrameworkPyTorch
    Organizationfacebook
    FeatureSegmentation
    Outputmask + label
    Modalitiesvideo, image
    RetrieverMask Filter
    Parameters860M
    Licenseother
    Downloads/mo2.3M
    Likes2,740

    Research Paper

    SAM 3: Segment Anything with Concepts

    arxiv.org

    Build a pipeline with sam3

    Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

    Run it on your own data, free