Video Understanding

9 min read

Updated 2026-11-19

# How Do I Pick a Good Thumbnail From a Video Automatically?

Score sampled frames for brightness, contrast and sharpness, drop the ones near a cut, and keep the sharpest per shot, then rank by whether the subject is in frame. A runnable OpenCV script on the Sintel trailer shows the black first frame, the shots with no usable frame, and why sharpness alone picks texture over faces.

![Ethan Steininger](/people/ethan.webp)

By [Ethan Steininger](/blog/authors/ethan), Founder & CEO, Mixpeek

Video

Thumbnails

Keyframes

Scene Detection

OpenCV

[A thumbnail for every scene you index. Try it as you read.Multimodal extractor docs →](https://mixpeek.com/docs/processing/extractors/multimodal)

[One-click templateFootage Intelligence at Archive ScaleMillions of scenes you can move through instead of search. Footage, ads and editor assets land in three clean collections, and scenes are clustered one partition at a time so every run stays under the sample-size caps.Deploy it](/templates/footage-intelligence)

## How do I pick a good thumbnail from a video automatically?

  
  
Sample frames, throw out the ones no person would choose, then rank what is left. Score every sampled frame for brightness, contrast and sharpness (the variance of its Laplacian, a standard blur measure), drop frames that are too dark, too flat or too close to a cut, and keep the sharpest frame of each shot. That reliably avoids the classic failures: the black first frame, the fade, the title card and the frame caught mid-motion. Choosing the best of the good frames needs one more signal, such as a face or subject detector or a text query with an image-text model, because sharpness rewards busy texture rather than the subject.  
  

## Why are auto-generated thumbnails so often bad?

  
  
The two common defaults both pick by position. The first frame of a film or an edited clip is usually black or a fade-in, and the frame at a fixed point (the midpoint, or 10 seconds in) lands wherever it lands: a cut, a motion blur, a dark transition. Neither looks at the image. On the 52-second Sintel trailer used below, the first frame measures brightness 0.0 and sharpness 0.0, a fully black image.  
  

## What makes a frame a usable thumbnail?

  
  

Check

How to measure it

Threshold used below

Not too dark or blown out

Mean gray level, 0 to 255

40 to 215

Not flat (fades, title cards, sky)

Standard deviation of gray levels

30 or more

In focus, not motion-blurred

Variance of the Laplacian (edge strength)

Highest in the shot

Not at a cut

Distance from the nearest shot boundary

Skip the first and last frames of each shot

Shows the subject

Face or object detector, or an image-text match to "the main character"

Not in the script; see below

  
  
The first four are cheap image statistics that run at hundreds of frames a second on a CPU. The fifth needs a model, and it is the one that separates a usable frame from a good one.  
  

## Reproduce it: pick thumbnails from a trailer

  
  
The script decodes a video with ffmpeg at 4 frames a second, finds the cuts by comparing colour histograms of neighbouring frames, filters each shot's frames with the checks above and keeps the sharpest. It needs OpenCV, NumPy and ffmpeg.

"""Pick a thumbnail per shot: skip dark and blurry frames, stay away from cuts.

    pip install opencv-python numpy      (and ffmpeg on the PATH)
    python pick\_thumbnails.py video.webm
"""
import subprocess
import sys

import cv2
import numpy as np

FPS, W, H = 4, 426, 240          # sample 4 frames a second at a small size

raw = subprocess.run(
    \["ffmpeg", "-v", "error", "-i", sys.argv\[1\], "-vf", f"fps={FPS},scale={W}:{H}",
     "-f", "rawvideo", "-pix\_fmt", "bgr24", "-"\],
    capture\_output=True, check=True).stdout
frames = np.frombuffer(raw, np.uint8).reshape(-1, H, W, 3)


def stats(f):
    g = cv2.cvtColor(f, cv2.COLOR\_BGR2GRAY)
    sharp = cv2.Laplacian(g, cv2.CV\_64F).var()       # edges: low on blur and flat frames
    hist = cv2.calcHist(\[f\], \[0, 1, 2\], None, \[8, 8, 8\], \[0, 256\] \* 3)
    return g.mean(), g.std(), sharp, cv2.normalize(hist, None).flatten()


s = \[stats(f) for f in frames\]

# 1. Shots: a cut is a big jump in the colour histogram between neighbouring frames.
cuts = \[0\] + \[i for i in range(1, len(s))
              if cv2.compareHist(s\[i - 1\]\[3\], s\[i\]\[3\], cv2.HISTCMP\_BHATTACHARYYA) > 0.5\] + \[len(s)\]
shots = \[(a, b) for a, b in zip(cuts, cuts\[1:\]) if b - a >= 2\]

# 2. In each shot, drop dark, flat and near-cut frames, then keep the sharpest.
def usable(i, a, b):
    mean, std, \_, \_ = s\[i\]
    return 40 <= mean <= 215 and std >= 30 and a + 1 <= i <= b - 2


def describe(i):
    mean, std, sharp, \_ = s\[i\]
    return f"{i / FPS:5.2f}s  brightness {mean:5.1f}  contrast {std:4.1f}  sharpness {sharp:7.1f}"


print(f"{len(frames)} frames sampled, {len(shots)} shots")
print("first frame:   " + describe(0))
print("middle frame:  " + describe(len(frames) // 2))
picks = \[\]
for a, b in shots:
    cand = \[i for i in range(a, b) if usable(i, a, b)\]
    if cand:
        picks.append(max(cand, key=lambda i: s\[i\]\[2\]))
best = max(picks, key=lambda i: s\[i\]\[2\])
print("best overall:  " + describe(best))
print(f"{len(picks)} shots have a usable frame, {len(shots) - len(picks)} are too dark, flat or short")
for i in picks\[:8\]:
    print("  shot pick   " + describe(i))
cv2.imwrite("thumbnail.jpg", frames\[best\])

Output, from running it on the 480p transcode of the [Sintel trailer](https://commons.wikimedia.org/wiki/File:Sintel_trailer-1080p.ogv) (Blender Foundation, CC BY 3.0):

206 frames sampled, 11 shots
first frame:    0.00s  brightness   0.0  contrast  0.0  sharpness     0.0
middle frame:  25.75s  brightness  55.6  contrast 58.6  sharpness   357.4
best overall:  20.00s  brightness  42.6  contrast 42.8  sharpness   617.6
6 shots have a usable frame, 5 are too dark, flat or short
  shot pick    4.50s  brightness  79.1  contrast 51.6  sharpness   130.8
  shot pick    5.50s  brightness 122.9  contrast 69.7  sharpness   249.8
  shot pick   20.00s  brightness  42.6  contrast 42.8  sharpness   617.6
  shot pick   26.25s  brightness 128.8  contrast 90.9  sharpness   473.4
  shot pick   29.75s  brightness 108.8  contrast 78.8  sharpness   542.6
  shot pick   33.25s  brightness 104.5  contrast 80.5  sharpness   540.6

What the numbers say:  
  
*   **The first frame is unusable.** Brightness 0.0 and sharpness 0.0: a black frame, which is
what "use frame 0" ships.*   **Five of the eleven shots have no usable frame.** They are fades, dark transitions and title
cards, which the brightness and contrast checks remove before sharpness is compared.*   **The run is cheap.** Decoding and scoring 206 frames took 0.94 seconds of wall time with a
peak of 190 MB of memory (ffmpeg 8.1.2, OpenCV 4.11.0, NumPy 1.26.4).  
  
What they do not say: the sharpest frame overall (20.00 s) is a detailed street with the character small in the frame, while the midpoint frame (25.75 s) shows the dragon close up with some motion blur. Sharpness measures texture, and a busy background has more of it than a face. The filters above are good at rejecting bad frames; to pick the frame a person would pick, add a subject signal. Common choices are a face detector (prefer frames with a large, frontal, open-eyed face), an object detector for the product or subject, or an image-text model such as CLIP scored against a description like "a close-up of the main character".  
  

## How do I make the picker more accurate?

  
  
*   **Crop letterbox bars first.** Widescreen films carry black bars, as the Sintel trailer does,
and they lower every brightness and contrast reading. Detect rows that are black across the whole width and crop them before scoring.*   **Sample more often for fast cuts.** At 4 frames a second a shot shorter than half a second has
one or two frames and gets skipped. Sample 8 to 12 frames a second for trailers and ads.*   **Tune the cut threshold.** The script calls a cut when the Bhattacharyya distance between
neighbouring histograms passes 0.5. Lower it to catch soft cuts, raise it if camera moves are splitting shots.*   **Score at the size you display.** A thumbnail shown at 320 px wide does not need 4K sharpness;
scoring a downscaled frame matches what viewers see and runs faster.  
  

## How does Mixpeek handle video thumbnails?

  
  
Mixpeek is a multimodal data warehouse: it indexes video, images, audio and documents where they sit in object storage and searches them with retrievers. Its [multimodal extractor](https://mixpeek.com/docs/processing/extractors/multimodal) can cut each video at its scene changes (`split_method: "scene"`) instead of at fixed intervals, and with `enable_thumbnails` it stores a thumbnail for every segment, 640 pixels wide, next to the segment's embeddings, transcript and timestamps. Search results then come back with a picture of the matching scene. A `custom_thumbnail` URL replaces the generated one when you have picked a frame yourself, for example with the script above.  
  

## Frequently Asked Questions

  
  

### How do I choose the best frame from a video for a thumbnail?

  
  
Sample frames (4 a second is enough for most video), drop the ones that are too dark, too flat or next to a cut, and keep the sharpest frame in each shot. Then rank those candidates by whether they show the subject, with a face detector or an image-text model. Sharpness alone favours busy backgrounds over faces.  
  

### How do I detect if a video frame is blurry?

  
  
Compute the variance of the Laplacian of the grayscale frame: `cv2.Laplacian(gray, cv2.CV_64F).var()`. Sharp frames have strong edges and a high variance; blurred and flat frames score low. In the Sintel trailer the black first frame scored 0.0 and the sharpest frame 617.6. Compare scores within one video, since the number depends on resolution and content.  
  

### Why is my video thumbnail black?

  
  
Because the thumbnail was taken from the first frame, and edited video almost always opens on black or a fade-in. Take the thumbnail from a scored frame instead, or at least skip frames whose mean brightness is below about 40 on a 0 to 255 scale.  
  

### How do I detect scene changes in a video?

  
  
Compare each frame's colour histogram with the previous frame's and mark a cut when the distance jumps. With 8 bins per colour channel and the Bhattacharyya distance, a threshold of 0.5 found 11 shots in the 52-second Sintel trailer. ffmpeg's `select='gt(scene,0.3)'` filter and the PySceneDetect library do the same with their own defaults.  
  

### Can AI pick a better thumbnail than sharpness alone?

  
  
Yes, for the last step. Image statistics remove bad frames well but cannot tell a face from a textured wall. A face detector, an object detector, or an image-text model such as CLIP scored against "a close-up of the main character" ranks the remaining frames by what a viewer cares about.  
  

### How many frames should I sample to find a thumbnail?

  
  
Four a second covers most video and scored the 52-second Sintel trailer's 206 frames in under a second on a CPU. Use 8 to 12 a second for trailers and ads with very short shots, and 1 a second for long, slow footage such as lectures or security video.  
  

## Key Takeaways

  
  
*   Position-based thumbnails fail on black openings, fades and cuts; the Sintel trailer's first
frame is fully black.*   Brightness (40 to 215), contrast (standard deviation 30 or more) and distance from a cut remove
unusable frames; five of eleven shots in the trailer had none left.*   Variance of the Laplacian finds sharp frames, but it rewards texture, so the sharpest frame is
not always the best thumbnail.*   Add a subject signal (faces, objects or an image-text match) to choose among the good frames.
*   Scoring 206 frames took 0.94 seconds and 190 MB on a CPU.
  
  

## Sources

  
  
*   [Sintel trailer on Wikimedia Commons (Blender Foundation, CC BY 3.0)](https://commons.wikimedia.org/wiki/File:Sintel_trailer-1080p.ogv)
*   [FFmpeg filters: select and scene detection](https://ffmpeg.org/ffmpeg-filters.html#select_002c-aselect)
*   [PySceneDetect](https://www.scenedetect.com/)
*   [Mixpeek multimodal extractor](https://mixpeek.com/docs/processing/extractors/multimodal)
  
  

## Where to go next

  
  
[Video scene segmentation](/guides/video-scene-segmentation) · [best AI video analysis tools](/curated-lists/best-ai-video-analysis-tools) · [MVS, vector search on object storage](/mvs) · [pricing](/pricing)

## A thumbnail for every scene you index

Mixpeek cuts video at its scene changes where it sits in your bucket and stores a thumbnail with each segment's embeddings and timestamps, so search results come back with a picture of the moment.

[Multimodal extractor docs](https://mixpeek.com/docs/processing/extractors/multimodal)[See pricing](/pricing)

[Explore MVS](/mvs)[See pricing](/pricing)

## Related guides

[

Video Understanding

### Video Scene Segmentation: How AI Decomposes Continuous Video into Searchable Segments

A technical guide to video scene segmentation -- the preprocessing step that transforms continuous video into discrete, searchable segments. Covers shot boundary detection, content-based adaptive thresholds, audio-visual alignment, keyframe selection, and how scene decomposition enables downstream search and retrieval.

Read guide →](/guides/video-scene-segmentation)[

Video Understanding

### How Do I Search Video From Snowflake?

Keep the campaign rows in Snowflake and index the video files they point to scene by scene in Mixpeek, with the warehouse key on every scene. Then load the scene matches back into a Snowflake table and answer questions like "which ads show the product in the first three seconds" with a SQL join.

Read guide →](/guides/how-to-search-video-from-snowflake)[

Video Understanding

### Agentic Audiences contextual embeddings from video: building the ext.aa segment

How to send a type 2 contextual Agentic Audiences embedding for video in an OpenRTB bid request: read per-scene vectors from a Mixpeek collection, pool them into one vector per video or pick the scene at the ad break, encode little-endian float32 as Base64, build user.data\[\].segment\[\].ext.aa, and validate dimension and model on the buyer side. Tested against the extension's own example and a real video, with payload sizes by dimension.

Read guide →](/guides/agentic-audiences-contextual-embeddings-video)