Skip to main content
Retrieval cookbook: ready-to-copy pipeline recipes for RAG, hybrid search, video moments, face search, and dedup
Every recipe below is a complete, copy-paste retriever config. Create the retriever once, then execute it with runtime inputs.
Stage shape. Every stage is { "stage_name", "stage_type", "config": { "stage_id", "parameters" } }. stage_name is your label; stage_id is the implementation (feature_search, rerank, attribute_filter, …); stage_type is the category (filter, sort, reduce, group, enrich, apply). See Retrievers and Stages.
Create + execute (every recipe uses this)
# Create the retriever (body = one of the recipes below)
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" -d @recipe.json
# -> returns { "retriever_id": "ret_..." }

# Execute it with runtime inputs
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "inputs": { "query": "people discussing electric vehicles" } }'

​
Multimodal RAG

Retrieve context for an LLM across a video’s visual content and transcript, fuse with RRF, rerank, and format into a single prompt-ready context block. 
{
  "retriever_name": "multimodal-rag",
  "collection_identifiers": ["col_videos"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    {
      "stage_name": "search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "fusion": "rrf",
          "final_top_k": 50,
          "searches": [
            { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 },
            { "feature_uri": "mixpeek://multimodal_extractor@v1/transcription_embedding",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
          ]
        }
      }
    },
    { "stage_name": "rerank", "stage_type": "sort",
      "config": { "stage_id": "rerank", "parameters": { "model": "bge-reranker-v2-m3", "query": "{{INPUT.query}}", "top_n": 10 } } },
    { "stage_name": "prepare", "stage_type": "apply",
      "config": { "stage_id": "rag_prepare", "parameters": { "max_tokens": 8000, "output_mode": "single_context", "citation": { "style": "numbered" } } } }
  ]
}
Result: the 10 most relevant moments, reranked, formatted into a cited context block ready to paste into an LLM prompt.

​
Hybrid Search (dense + keyword/BM25)

Fuse semantic recall (dense vectors) with exact-keyword precision (BM25) under RRF — so brand names, SKUs, and prices like $9.99 still match. Requires a text payload index (see Text Indexes). 
{
  "retriever_name": "hybrid-search",
  "collection_identifiers": ["col_docs"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    {
      "stage_name": "hybrid",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "fusion": "rrf",
          "final_top_k": 25,
          "searches": [
            { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 },
            { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "lexical": true, "top_k": 100 }
          ]
        }
      }
    }
  ]
}
Result: 25 results ranked by fused semantic + keyword relevance. Use rrf here — it ranks by position, immune to the cosine-vs-BM25 score-scale mismatch.

​
Video Moment Localization

Search a video and collapse matching segments into a handful of seekable moments with start/end timestamps. 
{
  "retriever_name": "video-moments",
  "collection_identifiers": ["col_videos"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "search", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 200,
        "searches": [ { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 200 } ] } } },
    { "stage_name": "moments", "stage_type": "reduce",
      "config": { "stage_id": "moment_group", "parameters": {
        "parent_field": "source_object_id",
        "time_field": "start_time",
        "merge_tolerance_ms": 2000,
        "max_moments_per_parent": 5,
        "score_strategy": "max",
        "output_mode": "annotated" } } }
  ]
}
Result: per video, up to 5 merged moments with timestamps you can seek to.
time_field must point at the field holding each segment’s timestamp. For a plain text query use the segment’s own start_time. query_chunks is only present when you search with a file input via query preprocessing.

​
Face Search (1:N identification)

Find every clip a person appears in by passing a reference face image as a content query against the face embedding. 
{
  "retriever_name": "face-search",
  "collection_identifiers": ["col_videos"],
  "input_schema": { "reference_face_url": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "face", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 20,
        "searches": [ { "feature_uri": "mixpeek://face_identity_extractor@v1/embedding",
          "query": { "input_mode": "content", "value": "{{INPUT.reference_face_url}}" }, "top_k": 50 } ] } } }
  ]
}
Result: the 20 closest face matches. A cosine score above ~0.30 typically indicates the same person (see Face Identity).

​
Reverse Image Search + Dedup

Find visually similar items from other sources, deduplicated. 
{
  "retriever_name": "reverse-image",
  "collection_identifiers": ["col_catalog"],
  "input_schema": {
    "image_url": { "type": "text", "required": true },
    "exclude_source": { "type": "text" }
  },
  "stages": [
    { "stage_name": "similar", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 100,
        "searches": [ { "feature_uri": "mixpeek://image_extractor@v1/embedding",
          "query": { "input_mode": "content", "value": "{{INPUT.image_url}}" }, "top_k": 100, "min_score": 0.7 } ] } } },
    { "stage_name": "exclude_self", "stage_type": "filter",
      "config": { "stage_id": "attribute_filter", "parameters": {
        "field": "metadata.source", "operator": "neq", "value": "{{INPUT.exclude_source}}" } } },
    { "stage_name": "dedupe", "stage_type": "reduce",
      "config": { "stage_id": "deduplicate", "parameters": {
        "strategy": "field", "fields": ["metadata.source_url"], "keep": "first" } } }
  ]
}
Result: unique visually-similar items, excluding the original source.

​
Search + Classify

Search, then attach an LLM/taxonomy label to each result in one pipeline. 
{
  "retriever_name": "search-classify",
  "collection_identifiers": ["col_content"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "search", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 25,
        "searches": [ { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 } ] } } },
    { "stage_name": "classify", "stage_type": "enrich",
      "config": { "stage_id": "classify", "parameters": {
        "model": "gpt-4o-mini",
        "output_schema": { "category": "string", "confidence": "float" } } } }
  ]
}
Result: 25 results, each annotated with a classification. See Classify and Taxonomies for taxonomy-backed labeling.

​
Scan a batch of inputs

Any retriever can run many queries at once — ideal for moderating an upload queue or scanning a catalog: 
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute/batch" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "queries": [ {"inputs": {"image_url": "https://…/a.jpg"}}, {"inputs": {"image_url": "https://…/b.jpg"}} ] }'
The retriever is planned once and reused across the batch (query optimization).

​
Composing recipes

Stages are independent — add, remove, or reorder them. Common extensions: append a rerank for precision, an attribute_filter for metadata scoping (the optimizer pushes it down for you), or a rag_prepare to format for an LLM.