Runs as a custom pluginSegmentationVerified 2026-11-05
SAM 2.1 Hiera Large
facebook/sam2.1-hiera-large
SAM 2.1 Hiera Large is a image and video segmentation model (Apache-2.0). Can run on Mixpeek as a custom plugin.
Mixpeek does not run it today. It can run as a custom plugin: the weights load without remote code.
No demo for SAM 2.1 Hiera Large yet
A video, image, slideshow or playground goes here once we record one on our own data.
What it is good and bad at
- Best at
- Highest-quality promptable masks in images and video, with memory-based tracking of an object across frames.
- Worst at
- GPU in practice for video; needs points or boxes (no text prompts).
- Task
- Segmentation
- Input
- Image, Video
- Output
- Masks, Scores, Tracks
- Use cases
- Segmentation, Object detection, Tracking
Templates that use it
No template uses this model yet. Browse templates.
Input and output
Input
- image
- resolution
- 1024x1024 (longest side resized)
- video
- frames
- streamed frame by frame with a memory bank
- prompts
- points (positive/negative), boxes, or a mask; no text
Output
- masks
- binary masks per prompt (3 candidates with IoU scores for ambiguous prompts)
- scores
- predicted IoU
- tracks
- object ids across video frames
How Mixpeek wires it
- Extractors
- None today
- Feature URIs
- None today
- Vector indexes
- None
- Retriever stages
- None
- Combines with
- detector boxes (RT-DETR, D-FINE) -> SAM 2 box prompts -> masks -> masked crops -> SigLIP vectors
- masks per frame -> tracks -> object documents spanning a time range
How it runs
- Status
- Runs as a custom plugin Not served. Would run as a custom extractor plugin with transformers Sam2Model (safetensors, no remote code), prompted by boxes from a permissive detector.
- Serve apps
- None
- Runtime
- transformers Sam2Model + Sam2Processor (Sam2VideoModel for video)
- Weights
- safetensors, no remote code
- Parameters
- 224 M
- Size on disk
- 898 MB
- Preprocessing
- Resize longest side to 1024, pad, normalize ImageNet mean/std; prompts rescaled to the same frame.
- Postprocessing
- Upsample low-res mask logits, threshold at 0, pick the best mask by predicted IoU.
- Serving
- both
- Hardware
- Needs a GPU
Scale to zero
- Minimum replicas
- 0
- Maximum replicas
- 1
- Idle timeout
- 600 s
- Memory
- 6 GB
- CPUs
- 2
- GPUs
- 1
- Cold start, node warm
- Not recorded
- Cold start, new node
- Not recorded
Measured cost
Unmeasured
No committed measurement; plan row 3 measures it in the standalone image.
Our evaluations
Unmeasured
We have not evaluated this model on our own data yet. Publisher results are in the sources below.
Questions
- What is SAM 2.1 Hiera Large best at?
- Highest-quality promptable masks in images and video, with memory-based tracking of an object across frames.
- What are the limits of SAM 2.1 Hiera Large?
- GPU in practice for video; needs points or boxes (no text prompts).
- Can SAM 2.1 Hiera Large be used commercially?
- Yes. Apache-2.0 weights and code; no use restrictions.
- How does SAM 2.1 Hiera Large run on Mixpeek?
- Not served. Would run as a custom extractor plugin with transformers Sam2Model (safetensors, no remote code), prompted by boxes from a permissive detector.