Video Understanding

8 min read

Updated 2026-11-19

# How Do I Search Video From Snowflake?

Keep the campaign rows in Snowflake and index the video files they point to scene by scene in Mixpeek, with the warehouse key on every scene. Then load the scene matches back into a Snowflake table and answer questions like "which ads show the product in the first three seconds" with a SQL join.

![Ethan Steininger](/people/ethan.webp)

By [Ethan Steininger](/blog/authors/ethan), Founder & CEO, Mixpeek

Snowflake

Video Search

Data Warehouse

Scene Detection

SQL

[Index the videos your warehouse points to. Try it as you read.Mixpeek + Snowflake docs →](https://mixpeek.com/docs/integrations/snowflake-warehouse)

[One-click templateFootage Intelligence at Archive ScaleMillions of scenes you can move through instead of search. Footage, ads and editor assets land in three clean collections, and scenes are clustered one partition at a time so every run stays under the sample-size caps.Deploy it](/templates/footage-intelligence)

## How do I search video from Snowflake?

  
  
Keep the facts in Snowflake and put the video in a search index keyed the same way. A typical warehouse already has a table with one row per video: an ID, a URL to the file in S3 or GCS, and the campaign, spend and performance columns. Read the IDs and URLs out of that table, index each file scene by scene in Mixpeek with the warehouse ID attached to every scene, run your questions as searches, and write the matching scenes back into a Snowflake table. From there it is an ordinary join: scenes that show a product, next to the click-through rate of the ad they came from.  
  

## What does searching video next to warehouse data need?

  
  
A row with a URL tells you nothing about what happens on screen. To answer "which scenes show someone holding the product", something has to decode each file, cut it where the shot changes, and turn every scene into a vector that a sentence can be compared against. That index lives outside the warehouse. What makes it usable from SQL is the key: if every scene carries the same `ad_id` as the row it came from, every search result can be joined to anything else you know about that ad.  
  

Step

Where it runs

What happens

1\. Read the video list

Snowflake

`SELECT ad_id, video_url` from your ads table

2\. Register each file

Mixpeek bucket

One object per video, with `ad_id` on it

3\. Index scenes

Mixpeek collection

The video is cut at scene changes and each scene is embedded, transcribed and given a thumbnail

4\. Ask questions

Mixpeek retriever

A sentence returns matching scenes with `ad_id`, start and end time, and a score

5\. Join

Snowflake

Matches land in a `SCENE_MATCHES` table and join to campaign data on `ad_id`

  
  
Mixpeek runs as a separate API. It reads each file from the URL you give it and returns results over HTTP. In this pattern Snowflake makes no outbound calls and nothing is installed in your account.  
  

## Step 1: read the video list from Snowflake

\# pip install snowflake-connector-python pandas requests
import os
import snowflake.connector

sf = snowflake.connector.connect(
    account=os.environ\["SNOWFLAKE\_ACCOUNT"\], user=os.environ\["SNOWFLAKE\_USER"\],
    password=os.environ\["SNOWFLAKE\_PASSWORD"\], warehouse="ANALYTICS\_WH",
    database="MARKETING", schema="PUBLIC",
)
ads = sf.cursor().execute(
    "SELECT ad\_id, video\_url FROM ads WHERE video\_url IS NOT NULL"
).fetchall()
print(len(ads), "videos")

Mixpeek has to be able to fetch each URL. Public CDN links work as they are. For private buckets, either select presigned URLs (for S3, generate them in the script with `boto3`) or skip the URL column and connect the bucket itself as a sync source, so Mixpeek reads the files with its own credentials (see [object storage integrations](/docs/integrations/object-storage/overview)). If you sync the bucket, put the ad ID in the file path or in object tags so it reaches every scene.  
  

## Step 2: index every video scene by scene

  
  
The script creates a namespace, a bucket whose objects hold a video and an `ad_id`, and a collection that runs the multimodal extractor with scene splitting and copies `ad_id` onto every scene. Then it registers one object per row. `idempotency_key` makes a rerun return the existing object instead of indexing the same ad twice, so you can run it again on new rows.

import json, urllib.request

API = "https://api.mixpeek.com"

def call(method, path, body=None, ns=None):
    headers = {"Authorization": f"Bearer {os.environ\['MIXPEEK\_API\_KEY'\]}",
               "Content-Type": "application/json"}
    if ns:
        headers\["X-Namespace"\] = ns
    req = urllib.request.Request(API + path, method=method, headers=headers,
                                 data=json.dumps(body).encode() if body is not None else None)
    return json.load(urllib.request.urlopen(req, timeout=300))

ns = call("POST", "/v1/namespaces", {"namespace\_name": "ad-scenes",
          "feature\_extractors": \[{"feature\_extractor\_name": "multimodal\_extractor", "version": "v1"}\]})\["namespace\_id"\]
# A new namespace has a daily budget of $0.00, so set one before indexing.
call("PATCH", f"/v1/namespaces/{ns}", {"spend\_budget": {"daily\_usd": 50, "monthly\_usd": 100}})

bucket = call("POST", "/v1/buckets", {"bucket\_name": "ads", "bucket\_schema": {"properties": {
    "video": {"type": "video", "required": True},
    "ad\_id": {"type": "string", "required": True}}}}, ns)\["bucket\_id"\]

col = call("POST", "/v1/collections", {"collection\_name": "ad\_scenes",
    "source": {"type": "bucket", "bucket\_ids": \[bucket\]},
    "feature\_extractor": {"feature\_extractor\_name": "multimodal\_extractor", "version": "v1",
        "input\_mappings": {"video": "video"},
        "field\_passthrough": \[{"source\_path": "ad\_id"}\],
        "parameters": {"split\_method": "scene", "run\_multimodal\_embedding": True,
                       "run\_transcription": True, "enable\_thumbnails": True}}}, ns)\["collection\_id"\]

for ad\_id, url in ads:
    call("POST", f"/v1/buckets/{bucket}/objects", {
        "blobs": \[{"property": "video", "type": "video", "data": url}\],
        "ad\_id": str(ad\_id),
        "idempotency\_key": f"ad-{ad\_id}",
    }, ns)
call("POST", f"/v1/collections/{col}/trigger", {}, ns)

Scene splitting means a 30-second ad with eight cuts becomes eight documents, each with its own `start_time`, `end_time`, thumbnail and transcript slice. The [multimodal extractor docs](/docs/processing/extractors/multimodal) list the other parameters, such as fixed-length segments instead of scene cuts and on-screen text. The extractor is one choice of model: Mixpeek runs any model as an extractor, so a logo detector or a brand-safety classifier can label the same scenes in the same pipeline.  
  

## Step 3: search the scenes and write matches to Snowflake

  
  
Create a retriever over the collection, run each question you care about, and keep the scenes above a score you have checked by eye.

import pandas as pd
from snowflake.connector.pandas\_tools import write\_pandas

ret = call("POST", "/v1/retrievers", {"retriever\_name": "ad\_scene\_search",
    "collection\_identifiers": \[col\],
    "input\_schema": {"query": {"type": "text", "required": True}},
    "stages": \[{"stage\_name": "scene\_search", "stage\_type": "filter", "config": {
        "stage\_id": "feature\_search", "parameters": {
            "searches": \[{"feature\_uri": "mixpeek://multimodal\_extractor@v1/vertex\_multimodal\_embedding",
                          "query": {"input\_mode": "text", "text": "{{INPUT.query}}"}, "top\_k": 200}\],
            "final\_top\_k": 200}}}\]}, ns)
ret = ret.get("retriever", ret)\["retriever\_id"\]

questions = \["a person holding the product up to the camera",
             "the product on a plain background",
             "a person talking directly to the camera"\]
rows = \[\]
for q in questions:
    out = call("POST", f"/v1/retrievers/{ret}/execute", {"inputs": {"query": q}}, ns)
    for m in out\["results"\]:
        rows.append({"QUESTION": q, "AD\_ID": m\["ad\_id"\], "DOCUMENT\_ID": m\["document\_id"\],
                     "START\_S": m\["start\_time"\], "END\_S": m\["end\_time"\], "SCORE": m\["score"\]})

write\_pandas(sf, pd.DataFrame(rows), "SCENE\_MATCHES", auto\_create\_table=True, overwrite=True)

The scores are cosine similarity between the sentence and the scene. They rank scenes within one question and do not compare across questions, so pick a cutoff per question by looking at the thumbnails around it. In a public sample of stock footage indexed the same way, a correct top answer scored 0.13 on one question and 0.065 on another.  
  

## Step 4: join the scenes to campaign data

\-- Do ads that show the product in the first 3 seconds get more clicks?
WITH early\_product AS (
  SELECT DISTINCT ad\_id
  FROM scene\_matches
  WHERE question = 'a person holding the product up to the camera'
    AND start\_s < 3
    AND score >= 0.10          -- the cutoff you checked for this question
)
SELECT
  IFF(e.ad\_id IS NULL, 'no early product shot', 'product in first 3s') AS opening,
  COUNT(\*)                                   AS ads,
  SUM(p.clicks) / NULLIF(SUM(p.impressions), 0) AS ctr
FROM ad\_performance p
LEFT JOIN early\_product e ON e.ad\_id = p.ad\_id
GROUP BY 1;

`DOCUMENT_ID` is kept so an analyst can go from a row back to the exact scene: fetch the document from Mixpeek to get its thumbnail and a link that plays only that segment.  
  

## What does it cost?

  
  
Indexing is the cost that scales with the library, and it is paid once per video. Video indexing is priced per minute on the [pricing page](/pricing): at $0.066 a minute, a library of 1,000 thirty-second ads is 500 minutes, or about $33, plus $0.012 a minute (about $6) for the transcripts. The Snowflake side is a small query to read the list and a small table of matches. New ads only cost their own minutes when you rerun step 2 on rows added since the last run.  
  

## Where do the Snowflake connector and sync fit?

  
  
There are two other ways to connect the two systems, and they solve different problems.  
  
*   **Snowflake as a sync source.** A bucket sync with `provider_type: "snowflake"` reads a table
and stores each row as a JSON object, incrementally by an `updated_at`\-style column. Use it to make text and metadata rows searchable. The video files themselves come from storage, as in step 2.*   **The [Snowflake connector](/connectors/snowflake).** An open-source package
([github.com/mixpeek/connectors](https://github.com/mixpeek/connectors/tree/main/snowflake)) with external function and stream modules for calling Mixpeek from SQL to enrich rows.  
  

## FAQ

  
  

### Can I run the search from inside a SQL query?

  
  
Snowflake external functions call an HTTPS endpoint through an API integration, and the connector's external function module wraps the Mixpeek call for that. Most teams start with the batch pattern above because a table of matches is cheaper to query a thousand times than a thousand API calls, and it keeps a record of what each question returned on a given day.  
  

### Do I need to move my videos?

  
  
Your files stay where they are. Mixpeek fetches each one from the URL or bucket you point it at to process it, and stores what it derives (scene boundaries, embeddings, transcripts, thumbnails) in its own index.  
  

### How do I keep the index current as new ads launch?

  
  
Select only rows added since the last run (`WHERE created_at > :last_run`) in step 1 and run step 2 again. Rows already indexed are skipped by their `idempotency_key`. Then rerun step 3 and replace `SCENE_MATCHES`.  
  

### Can I use my own model for the scenes?

  
  
Yes. Mixpeek runs any model as an extractor, so a model trained on your products or your brand guidelines can score every scene at ingest, and its output lands on the same documents as the embeddings and joins to Snowflake the same way.

## Index the videos your warehouse points to

Mixpeek cuts each video into scenes where it sits in your storage, embeds every scene, and keeps your warehouse key on each one, so search results join back to the tables you already report from.

[Mixpeek + Snowflake docs](https://mixpeek.com/docs/integrations/snowflake-warehouse)[See pricing](/pricing)

[Explore MVS](/mvs)[See pricing](/pricing)

## Related guides

[

Video Understanding

### How Do I Pick a Good Thumbnail From a Video Automatically?

Score sampled frames for brightness, contrast and sharpness, drop the ones near a cut, and keep the sharpest per shot, then rank by whether the subject is in frame. A runnable OpenCV script on the Sintel trailer shows the black first frame, the shots with no usable frame, and why sharpness alone picks texture over faces.

Read guide →](/guides/how-to-pick-a-good-thumbnail-from-a-video)[

Video Understanding

### How Do I Find Failed Grasps and Handoffs Across Thousands of Robot Episodes?

Cut each episode's camera video into 2 to 5 second windows, search them in plain language, confirm every match by sending the clip to a vision model with a yes-or-no question, and merge what survives into timestamped moments per episode. Then export the segments, embeddings and lineage as Parquet and count failures by task, robot and camera in DuckDB.

Read guide →](/guides/how-to-find-failed-grasps-and-handoffs-in-robot-and-egocentric-video)[

Video Understanding

### How Do I Find Specific Moments Across Thousands of Hours of Robot or Dashcam Recordings?

Narrow by telemetry first, skip idle footage, then search what is left by meaning. Cut continuous recordings into fixed 10 to 30 second segments, embed each segment, store its timestamps next to the vehicle or robot's sensor data, and filter before you rank. For 10,000 hours, that is 1.2 million vectors and about 15 GB of index, next to 36 TB of footage.

Read guide →](/guides/how-to-find-moments-in-thousands-of-hours-of-dashcam-or-robot-footage)