1.  [](/)
2.  [Diagrams](/diagrams)
3.  Temporal Grounding: Query In, Timestamp Out

[Back to Diagrams](/diagrams)Video Understanding

# Temporal Grounding: Query In, Timestamp Out

Frame search returns stills. Temporal grounding takes a text query and a video and returns the start and end timestamps of the moment you asked for.

![Temporal grounding takes a natural language query and a video and returns the start and end timestamps of the described event, turning the video from an opaque blob into a queryable timeline. Frame-level similarity search only returns stills: the best it can say is that one frame looks relevant, with no beginning or end. The core loop is per-segment scoring plus interval merging: sample frames at 1 fps, embed frames and query with the same model (CLIP or SigLIP), score every frame against the query, slide a window and average the scores, then merge the overlapping windows that clear a threshold into one segment with a start and an end (for example start 134.2, end 141.8, score 0.94). Fancier methods keep the same contract: proposal networks like 2D-TAN and Moment-DETR learn the boundaries, dense captioning turns grounding into text search, and production systems decompose the video into timestamped feature streams and intersect intervals with temporal joins. The primary metric is Recall at 1 with IoU 0.5.](/diagrams/temporal-grounding.webp)Full screen

Frame search returns stills. Temporal grounding takes a text query and a video and returns the start and end timestamps of the moment you asked for.

Frame-level similarity search on video returns stills. You embed the query, embed sampled frames, take nearest neighbors, and the best answer available is that frame 3,840 looks relevant: no start, no end, no context. A 90 minute match is 162,000 frames at 30 fps, and a still from the middle of it is not an answer. Temporal grounding returns intervals instead, which is what turns a video from an opaque blob into a queryable timeline.

The core loop is per-segment scoring plus interval merging. Sample frames at 1 fps, embed the frames and the query with the same model (CLIP or SigLIP), score every frame against the query embedding, slide a window and average the scores, then merge the overlapping windows that clear a threshold into one segment with a start and an end, like start 134.2 to end 141.8 at score 0.94. Everything more sophisticated keeps that same contract. Proposal networks such as 2D-TAN and Moment-DETR learn the boundaries directly, dense captioning turns grounding into a text search over timestamped captions, and production systems decompose the video into timestamped feature streams (frames, objects, faces, transcripts) and intersect intervals with temporal joins when one model cannot answer alone.

This is the deliberate opposite of reverse video search, which is clip in and trajectory match out. Temporal grounding is text in and interval out. Score it on Recall at 1 with IoU 0.5, where the best models clear 60% on ActivityNet Captions and 70% on the shorter, more precise Charades-STA.

## Where this diagram appears

*   [Guide: Video Temporal Grounding →](/guides/video-temporal-grounding)

Run this on your own data

Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

[Search your own data](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=diagram_detail&utm_campaign=mvs_onramp)

## Related diagrams

[

![Object decomposition: you search inside the file, not for it. A one-hour recording.mp4 is split into scenes (visual embeddings), speech (transcript embeddings on silence and speaker turns), faces (who appeared and when), and on-screen text (OCR of slides and captions). A whole file being findable is not the same as usable: the thing you need is a moment inside it, like 23,41 where the renewal number is spoken while the pricing slide is on screen. The same move works on video, PDFs, and audio: one file becomes dozens of small, independently searchable documents.](/diagrams/foundations/decomposition.webp)

You search inside the file, not for it



](/diagrams/decomposition)[

![Feature extraction turns raw pixels into searchable signals: one video frame becomes a structured row, face, scene, spoken words (transcript), on-screen text (OCR), and a visual embedding. Every question you later ask the video is answered from these extracted signals, not from the raw pixels; if a signal was never extracted, nothing downstream can search it.](/diagrams/foundations/feature-extraction.webp)

Pixels in, signals out



](/diagrams/feature-extraction)[

![Documents fill empty buckets with a mean; queries sum and leave empty buckets zero: the asymmetry is what makes the FDE dot product approximate Chamfer / MaxSim.](/diagrams/muvera-query-doc-asymmetry.webp)

MUVERA: Why Query and Document Are Encoded Asymmetrically



](/diagrams/muvera-query-doc-asymmetry)