Skip to main content
Moment Group stage merging frame-level intervals into consolidated video moments
The Moment Group stage takes frame-level or chunk-level documents with temporal metadata and merges contiguous intervals into consolidated video moments (time ranges), grouped by parent object. Instead of returning individual frames that matched a query, you get precise start/end time ranges pinpointing where in a video the match occurs.
Stage Category: REDUCE (Merges intervals into moments)Transformation: N frame/chunk documents → M consolidated moments per parent video

​
When to Use

Use CaseDescription
Sub-scene localizationPinpoint exactly where in a video a query matches
Moment extractionReturn time-range clips instead of individual frames
Video search resultsProvide start/end timestamps for player seek
Highlight reelsExtract the top-scoring moments across a video library

​
When NOT to Use

ScenarioRecommended Alternative
Grouping by non-temporal fieldsgroup_by
Time-window aggregations (hour/day/week)temporal
Results without temporal metadataaggregate
Semantic clustering of resultscluster

​
Requirements

  • Input documents must have temporal intervals — either query_chunks with start_ms/end_ms (attached by feature_search preprocessing) or document-level time fields.
  • The parent_field must exist on the documents so results can be grouped by source video/object.

​
Parameters

ParameterTypeDefaultDescription
parent_fieldstring"source_object_id"Field to group results by parent video/object. Common values: source_object_id (standard ingest lineage), root_object_id (top-level ancestor), or any custom field.
time_fieldstring"query_chunks"Source of temporal intervals. "query_chunks" reads the array feature_search attaches (each entry has start_ms, end_ms, score). For document-level timestamps, use a dot-path like "metadata.start_ms" — the stage derives the end field by replacing "start" with "end".
merge_tolerance_msinteger2000Maximum gap in milliseconds between intervals before they are split into separate moments. 0 = exact-overlap only. Range: 0–60000.
max_moments_per_parentinteger10Maximum moments returned per parent, sorted by score (highest first). Range: 1–100.
score_strategystring"max"How to aggregate scores across merged intervals: "max", "avg", or "sum".
min_scorefloatnullDrop moments scoring below this threshold. Range: 0.0–1.0.
output_modestring"annotated""annotated" keeps the best-scoring document per parent and attaches a moments array (preserves all original document fields). "moments_only" emits standalone moment documents with IDs like moment_{parent}_{start_ms}.

​
Configuration Examples

{
  "stage_type": "reduce",
  "stage_id": "moment_group",
  "parameters": {
    "parent_field": "source_object_id",
    "time_field": "query_chunks",
    "merge_tolerance_ms": 2000,
    "max_moments_per_parent": 5,
    "score_strategy": "max",
    "output_mode": "annotated"
  }
}

​
Output Schema

​
Annotated Mode (default)

In annotated mode, the best-scoring document per parent is preserved with a moments array attached: 
{
  "document_id": "doc_abc123",
  "source_object_id": "video_001",
  "score": 0.92,
  "content": "...",
  "metadata": { "..." : "..." },
  "moments": [
    {
      "start_ms": 12000,
      "end_ms": 18500,
      "score": 0.92,
      "match_count": 4,
      "document_ids": ["doc_abc123", "doc_abc124", "doc_abc125", "doc_abc126"]
    },
    {
      "start_ms": 45000,
      "end_ms": 51000,
      "score": 0.78,
      "match_count": 2,
      "document_ids": ["doc_abc130", "doc_abc131"]
    }
  ]
}

​
Moments-Only Mode

In moments_only mode, standalone moment documents are emitted: 
{
  "document_id": "moment_video_001_12000",
  "start_ms": 12000,
  "end_ms": 18500,
  "score": 0.92,
  "match_count": 4,
  "document_ids": ["doc_abc123", "doc_abc124", "doc_abc125", "doc_abc126"]
}

​
Moment Object Fields

FieldTypeDescription
start_msintegerStart of the moment in milliseconds
end_msintegerEnd of the moment in milliseconds
scorefloatAggregated score (per score_strategy)
match_countintegerNumber of source intervals merged into this moment
document_idsarrayIDs of the original documents that contributed to this moment

​
Performance

MetricValue
Latency5-50ms
MemoryO(N) where N = input documents
CostFree
ScalabilityEfficient — runs in the API layer, no engine call

​
Common Pipeline Patterns

​
Video Search with Moment Localization

The canonical use case: search across video frames, then consolidate matches into seekable moments. 
[
  {
    "stage_type": "filter",
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 200
        }
      ],
      "final_top_k": 200
    }
  },
  {
    "stage_type": "reduce",
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 1500,
      "max_moments_per_parent": 3,
      "score_strategy": "max",
      "min_score": 0.6,
      "output_mode": "annotated"
    }
  }
]

​
Moment Extraction with Reranking

Search, rerank for precision, then group into moments: 
[
  {
    "stage_type": "filter",
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 500
        }
      ],
      "final_top_k": 500
    }
  },
  {
    "stage_type": "sort",
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "baai_bge_reranker_v2_m3",
      "query": "{{INPUT.query}}",
      "top_k": 100
    }
  },
  {
    "stage_type": "reduce",
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 2000,
      "max_moments_per_parent": 5,
      "score_strategy": "max",
      "output_mode": "annotated"
    }
  }
]

​
Error Handling

ErrorBehavior
Missing parent_field on documentDocument skipped
Missing temporal metadataDocument skipped
No intervals pass min_scoreParent omitted from output
Empty inputEmpty results returned
  • Temporal - Time-window aggregations with drift detection
  • Group By - Group documents by any field value
  • Aggregate - Statistical aggregations
  • Deduplicate - Remove duplicate documents