> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Contextual video matching

> Match videos to the article a reader is on: S3 video ingest, IAB, entity, sentiment and brand-safety labels, and a ranked playlist retriever.

A publisher's video player sits inside an article page. For each article, it should play the videos that fit the story and are safe for the advertisers on the page. This guide builds that on Mixpeek:

1. Videos sync from your S3 bucket and are embedded and described.
2. Articles arrive through the API with their text.
3. A labeling retriever writes IAB categories, entities, sentiment and a brand-safety rating onto every article and video at ingest.
4. A playlist retriever takes an article and returns ranked, brand-safe videos.
5. Your player calls it behind a feature flag, keeps its own playlist as the fallback, and sends viewer events back.

Every resource below can live in one [manifest](https://docs.mixpeek.com/docs/operations/manifests.md), so dev, staging and production run the same configuration ([environment branching](https://docs.mixpeek.com/docs/operations/environment-branching.md)).

## Prerequisites

* An S3 bucket of video files and an IAM role Mixpeek can assume, set up as in [AWS S3](https://docs.mixpeek.com/docs/integrations/object-storage/s3.md).
* A Mixpeek API key and a namespace. Requests below use `$MP_API_KEY` and `$MP_NAMESPACE`.

## 1. Bring in the video library

Create a `videos` bucket whose schema holds the file and the metadata your player filters on:

```bash theme={null}
curl -sS -X POST "https://api.mixpeek.com/v1/buckets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "videos",
    "bucket_schema": {
      "properties": {
        "video":        { "type": "video" },
        "video_id":     { "type": "string", "required": true },
        "title":        { "type": "string" },
        "publisher_id": { "type": "string" },
        "channel_id":   { "type": "string" },
        "is_shared":    { "type": "boolean" },
        "breaking":     { "type": "boolean" },
        "published_at": { "type": "datetime" }
      }
    }
  }'
```

Then [sync your S3 prefix](https://docs.mixpeek.com/docs/integrations/object-storage/s3.md#configuration-steps) into it. Upload each video with its metadata as `x-amz-meta-*` headers, and map those keys to the bucket fields with `metadata` sources, as shown in [Object metadata](https://docs.mixpeek.com/docs/integrations/object-storage/s3.md#object-metadata). A `continuous` sync with a 60-second poll picks up a new upload within a minute.

For the existing catalog, run the backfill at a lower priority than new uploads. See [Backfill and one-at-a-time uploads](https://docs.mixpeek.com/docs/operations/batch-ingestion-at-scale.md#backfill-and-one-at-a-time-uploads).

## 2. Embed and describe the videos

A collection with `multimodal_extractor` v2 embeds each video with Gemini Embedding 2. With `run_video_description`, Gemini also watches and listens to the video and writes a text description, which the labeling step reads:

```bash theme={null}
curl -sS -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "video-signals",
    "source": { "type": "bucket", "bucket_ids": ["bkt_videos"] },
    "feature_extractor": {
      "feature_extractor_name": "multimodal_extractor",
      "version": "v2",
      "input_mappings": { "video": "video" },
      "field_passthrough": [
        { "source_path": "video_id" }, { "source_path": "title" },
        { "source_path": "publisher_id" }, { "source_path": "channel_id" },
        { "source_path": "is_shared" }, { "source_path": "published_at" }
      ],
      "parameters": {
        "run_multimodal_embedding": true,
        "run_video_description": true,
        "description_prompt": "Describe what this news video shows and says. Name every person, organisation, place, event and product seen or heard, and copy any on-screen text.",
        "split_method": "time",
        "time_split_interval": 120
      }
    }
  }'
```

`field_passthrough` copies the bucket fields onto each document, so the playlist can filter on them. See [Multimodal extractor](https://docs.mixpeek.com/docs/processing/extractors/multimodal.md) for every parameter.

## 3. Send articles

Create an `articles` bucket with `article_id`, `title`, `body` and `publisher_id`, and an `article-signals` collection on it with the same extractor and `"input_mappings": { "text": "body" }`. Articles and videos are then embedded in the same vector space, so an article can search the videos directly.

Send each article as an object, one blob per field, with its text in `body`. Strip the page markup before you send it: the text extractor reads `body` as plain text.

```bash theme={null}
curl -sS -X POST "https://api.mixpeek.com/v1/buckets/bkt_articles/objects" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "blobs": [
      { "property": "article_id",   "type": "text", "data": "a_5521" },
      { "property": "title",        "type": "text", "data": "City council approves new stadium" },
      { "property": "publisher_id", "type": "text", "data": "pub_123" },
      { "property": "body",         "type": "text", "data": "The council voted 7 to 2 on Tuesday to ..." }
    ]
  }'
```

Submit the new object in a batch, and pass a `webhook_url` or subscribe to `collection.documents.batch_completed` to learn when its document is ready ([Webhooks](https://docs.mixpeek.com/docs/operations/webhooks.md)).

## 4. Label articles and videos at ingest

The signals come from a retriever that runs once on every new document, as a [retriever enrichment](https://docs.mixpeek.com/docs/enrichment/retriever-enrichments.md). Its first stage selects the document being labeled. Its second is an [`llm_enrich`](https://docs.mixpeek.com/docs/retrieval/stages/llm-enrich) stage that returns the labels as structured output:

```json theme={null}
{
  "retriever_name": "label-content",
  "collection_identifiers": ["article-signals", "video-signals"],
  "input_schema": { "document_id": { "type": "string", "required": true } },
  "stages": [
    {
      "stage_name": "this_document",
      "stage_type": "filter",
      "config": {
        "stage_id": "attribute_filter",
        "parameters": { "field": "document_id", "operator": "eq", "value": "{{INPUT.document_id}}" }
      }
    },
    {
      "stage_name": "label",
      "stage_type": "enrich",
      "config": {
        "stage_id": "llm_enrich",
        "parameters": {
          "provider": "google",
          "model_name": "gemini-2.5-flash-lite",
          "prompt": "Label this news content for contextual video advertising. Title: {{DOC.title}}. Text: {{DOC.body}} {{DOC.description}}",
          "output_field": "labels",
          "output_schema": {
            "type": "object",
            "properties": {
              "iab_categories": { "type": "array", "items": { "type": "string" },
                                  "description": "1 to 3 IAB Content Taxonomy 3.1 paths, best fit first" },
              "entities":       { "type": "array", "items": { "type": "string" },
                                  "description": "Named people, organisations, places, events and products" },
              "sentiment":      { "type": "string", "enum": ["positive", "neutral", "negative"] },
              "brand_safety":   { "type": "string", "enum": ["safe", "sensitive", "unsafe"],
                                  "description": "GARM brand safety floor and suitability" }
            }
          }
        }
      }
    }
  ]
}
```

Attach it to both collections. Each run writes the label fields back onto the document it labeled:

```json theme={null}
{
  "retriever_enrichments": [
    {
      "retriever_id": "ret_label_content",
      "input_mappings": [
        { "input_key": "document_id", "source": { "source_type": "document_field", "path": "document_id" } }
      ],
      "write_back_fields": [
        { "source_field": "labels.iab_categories", "target_field": "iab_categories", "mode": "first" },
        { "source_field": "labels.entities",       "target_field": "entities",       "mode": "first" },
        { "source_field": "labels.sentiment",      "target_field": "sentiment",      "mode": "first" },
        { "source_field": "labels.brand_safety",   "target_field": "brand_safety",   "mode": "first" }
      ]
    }
  ]
}
```

Put the full IAB 3.1 list in the prompt, so the model copies a category rather than inventing one. Articles and videos get the same field names, so the playlist can compare them.

## 5. Build the playlist retriever

The playlist retriever takes an article's `document_id` and searches the videos with that article's own embedding, using `input_mode: "document"`. No new inference runs at query time. The search filter keeps brand-unsafe videos out and applies the publisher's rights: its own videos or the shared library, minus channels it has excluded ([filter example](https://docs.mixpeek.com/docs/retrieval/filters.md#example-own-content-or-a-shared-library-minus-excluded-channels)). Pass `excluded_channels` as `[]` when a publisher excludes none.

```json theme={null}
{
  "retriever_name": "playlist",
  "collection_identifiers": ["video-signals"],
  "input_schema": {
    "article_document_id": { "type": "string", "required": true },
    "publisher_id":        { "type": "string", "required": true },
    "excluded_channels":   { "type": "array", "required": true }
  },
  "stages": [
    {
      "stage_name": "candidates",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://multimodal_extractor@v2/gemini-embedding-2",
              "query": {
                "input_mode": "document",
                "document_ref": { "collection_id": "article-signals", "document_id": "{{INPUT.article_document_id}}" }
              },
              "top_k": 100,
              "filters": {
                "AND": [
                  { "field": "brand_safety", "operator": "ne", "value": "unsafe" },
                  { "OR": [
                    { "field": "publisher_id", "operator": "eq", "value": "{{INPUT.publisher_id}}" },
                    { "field": "is_shared", "operator": "eq", "value": true }
                  ] },
                  { "NOT": { "field": "channel_id", "operator": "in", "value": "{{INPUT.excluded_channels}}" } }
                ]
              }
            }
          ],
          "final_top_k": 100
        }
      }
    },
    {
      "stage_name": "dedupe_titles",
      "stage_type": "reduce",
      "config": { "stage_id": "deduplicate", "parameters": { "strategy": "field", "fields": ["title"] } }
    },
    {
      "stage_name": "top5",
      "stage_type": "reduce",
      "config": { "stage_id": "limit", "parameters": { "limit": 5 } }
    }
  ]
}
```

To rank on more than vector similarity, add stages between search and limit. A [`document_enrich`](https://docs.mixpeek.com/docs/retrieval/stages/document-enrich) stage can attach the article's entities and IAB categories to each candidate. A [`code_execution`](https://docs.mixpeek.com/docs/retrieval/stages/code-execution) stage can then score shared entities and IAB depth, and [`sort_attribute`](https://docs.mixpeek.com/docs/retrieval/stages/sort-attribute) can order by that score.

## 6. Call it from the player, behind a flag

Execute the retriever with the article's document id. Keep your current playlist as the fallback: use it when the flag is off, the call fails or times out, the run is `degraded`, or fewer videos come back than the player needs.

```javascript theme={null}
async function playlistFor(article, fallback) {
  if (!flags.mixpeekPlaylist) return fallback;
  try {
    const res = await fetch(`https://api.mixpeek.com/v1/retrievers/${RETRIEVER_ID}/execute`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${MP_API_KEY}`,
        "X-Namespace": MP_NAMESPACE,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        inputs: {
          article_document_id: article.documentId,
          publisher_id: article.publisherId,
          excluded_channels: article.excludedChannels,
        },
      }),
      signal: AbortSignal.timeout(800),
    });
    if (!res.ok) return fallback;
    const run = await res.json();
    if (run.status !== "completed" || run.documents.length < 3) return fallback;
    return { videos: run.documents, executionId: run.execution_id };
  } catch {
    return fallback;
  }
}
```

Read `documents`, check `status`, and log `warnings`. A run that skips a stage answers `200` with `status: "degraded"`, so a short list is not always a lack of matches. Keep `execution_id`: the next step needs it.

## 7. Send viewer and editor signals back

Record what happened to each video as an [interaction](https://docs.mixpeek.com/docs/retrieval/interactions.md#player-events-and-editor-review), with the `execution_id` and `retriever_id` of the run that chose it: `impression` when it is shown, `play`, `long_view` when watched to the end, and `skip`. During a pilot, record editors' verdicts on a match as `positive_feedback` or `negative_feedback`.

These rows land in your namespace's [system collections](https://docs.mixpeek.com/docs/vector-store/system-collections.md), joined to the query that produced them. Use them to build an [evaluation set](https://docs.mixpeek.com/docs/retrieval/evaluations.md), compare ranking configurations, and roll a new one out to a share of traffic with [auto-tune rollout](https://docs.mixpeek.com/docs/retrieval/auto-tune-rollout.md).

## Next steps

> [card] [Manifests](https://docs.mixpeek.com/docs/operations/manifests.md)
>
> Declare the buckets, collections and retrievers above in one file.

> [card] [Batch ingestion at scale](https://docs.mixpeek.com/docs/operations/batch-ingestion-at-scale.md)
>
> Size the backfill and keep new uploads ahead of it.

> [card] [Filters](https://docs.mixpeek.com/docs/retrieval/filters.md)
>
> Every operator for rights, recency and brand-safety rules.

> [card] [Reward signals](https://docs.mixpeek.com/docs/retrieval/reward-signals.md)
>
> How plays and completions feed ranking.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.