How do I pick a good thumbnail from a video automatically?
Sample frames, throw out the ones no person would choose, then rank what is left. Score every sampled frame for brightness, contrast and sharpness (the variance of its Laplacian, a standard blur measure), drop frames that are too dark, too flat or too close to a cut, and keep the sharpest frame of each shot. That reliably avoids the classic failures: the black first frame, the fade, the title card and the frame caught mid-motion. Choosing the best of the good frames needs one more signal, such as a face or subject detector or a text query with an image-text model, because sharpness rewards busy texture rather than the subject.
Why are auto-generated thumbnails so often bad?
The two common defaults both pick by position. The first frame of a film or an edited clip is usually black or a fade-in, and the frame at a fixed point (the midpoint, or 10 seconds in) lands wherever it lands: a cut, a motion blur, a dark transition. Neither looks at the image. On the 52-second Sintel trailer used below, the first frame measures brightness 0.0 and sharpness 0.0, a fully black image.
What makes a frame a usable thumbnail?
| Check | How to measure it | Threshold used below |
|---|---|---|
| Not too dark or blown out | Mean gray level, 0 to 255 | 40 to 215 |
| Not flat (fades, title cards, sky) | Standard deviation of gray levels | 30 or more |
| In focus, not motion-blurred | Variance of the Laplacian (edge strength) | Highest in the shot |
| Not at a cut | Distance from the nearest shot boundary | Skip the first and last frames of each shot |
| Shows the subject | Face or object detector, or an image-text match to "the main character" | Not in the script; see below |
The first four are cheap image statistics that run at hundreds of frames a second on a CPU. The fifth needs a model, and it is the one that separates a usable frame from a good one.
Reproduce it: pick thumbnails from a trailer
The script decodes a video with ffmpeg at 4 frames a second, finds the cuts by comparing colour histograms of neighbouring frames, filters each shot's frames with the checks above and keeps the sharpest. It needs OpenCV, NumPy and ffmpeg.
"""Pick a thumbnail per shot: skip dark and blurry frames, stay away from cuts.
pip install opencv-python numpy (and ffmpeg on the PATH)
python pick_thumbnails.py video.webm
"""
import subprocess
import sys
import cv2
import numpy as np
FPS, W, H = 4, 426, 240 # sample 4 frames a second at a small size
raw = subprocess.run(
["ffmpeg", "-v", "error", "-i", sys.argv[1], "-vf", f"fps={FPS},scale={W}:{H}",
"-f", "rawvideo", "-pix_fmt", "bgr24", "-"],
capture_output=True, check=True).stdout
frames = np.frombuffer(raw, np.uint8).reshape(-1, H, W, 3)
def stats(f):
g = cv2.cvtColor(f, cv2.COLOR_BGR2GRAY)
sharp = cv2.Laplacian(g, cv2.CV_64F).var() # edges: low on blur and flat frames
hist = cv2.calcHist([f], [0, 1, 2], None, [8, 8, 8], [0, 256] * 3)
return g.mean(), g.std(), sharp, cv2.normalize(hist, None).flatten()
s = [stats(f) for f in frames]
# 1. Shots: a cut is a big jump in the colour histogram between neighbouring frames.
cuts = [0] + [i for i in range(1, len(s))
if cv2.compareHist(s[i - 1][3], s[i][3], cv2.HISTCMP_BHATTACHARYYA) > 0.5] + [len(s)]
shots = [(a, b) for a, b in zip(cuts, cuts[1:]) if b - a >= 2]
# 2. In each shot, drop dark, flat and near-cut frames, then keep the sharpest.
def usable(i, a, b):
mean, std, _, _ = s[i]
return 40 <= mean <= 215 and std >= 30 and a + 1 <= i <= b - 2
def describe(i):
mean, std, sharp, _ = s[i]
return f"{i / FPS:5.2f}s brightness {mean:5.1f} contrast {std:4.1f} sharpness {sharp:7.1f}"
print(f"{len(frames)} frames sampled, {len(shots)} shots")
print("first frame: " + describe(0))
print("middle frame: " + describe(len(frames) // 2))
picks = []
for a, b in shots:
cand = [i for i in range(a, b) if usable(i, a, b)]
if cand:
picks.append(max(cand, key=lambda i: s[i][2]))
best = max(picks, key=lambda i: s[i][2])
print("best overall: " + describe(best))
print(f"{len(picks)} shots have a usable frame, {len(shots) - len(picks)} are too dark, flat or short")
for i in picks[:8]:
print(" shot pick " + describe(i))
cv2.imwrite("thumbnail.jpg", frames[best])206 frames sampled, 11 shots first frame: 0.00s brightness 0.0 contrast 0.0 sharpness 0.0 middle frame: 25.75s brightness 55.6 contrast 58.6 sharpness 357.4 best overall: 20.00s brightness 42.6 contrast 42.8 sharpness 617.6 6 shots have a usable frame, 5 are too dark, flat or short shot pick 4.50s brightness 79.1 contrast 51.6 sharpness 130.8 shot pick 5.50s brightness 122.9 contrast 69.7 sharpness 249.8 shot pick 20.00s brightness 42.6 contrast 42.8 sharpness 617.6 shot pick 26.25s brightness 128.8 contrast 90.9 sharpness 473.4 shot pick 29.75s brightness 108.8 contrast 78.8 sharpness 542.6 shot pick 33.25s brightness 104.5 contrast 80.5 sharpness 540.6
What they do not say: the sharpest frame overall (20.00 s) is a detailed street with the character small in the frame, while the midpoint frame (25.75 s) shows the dragon close up with some motion blur. Sharpness measures texture, and a busy background has more of it than a face. The filters above are good at rejecting bad frames; to pick the frame a person would pick, add a subject signal. Common choices are a face detector (prefer frames with a large, frontal, open-eyed face), an object detector for the product or subject, or an image-text model such as CLIP scored against a description like "a close-up of the main character".
How do I make the picker more accurate?
How does Mixpeek handle video thumbnails?
Mixpeek is a multimodal data warehouse: it indexes video, images, audio and documents where they sit in object storage and searches them with retrievers. Its multimodal extractor can cut each video at its scene changes (
split_method: "scene") instead of at fixed intervals, and with
enable_thumbnails it stores a thumbnail for every segment, 640 pixels wide, next to the
segment's embeddings, transcript and timestamps. Search results then come back with a picture of
the matching scene. A custom_thumbnail URL replaces the generated one when you have picked a
frame yourself, for example with the script above.Frequently Asked Questions
How do I choose the best frame from a video for a thumbnail?
Sample frames (4 a second is enough for most video), drop the ones that are too dark, too flat or next to a cut, and keep the sharpest frame in each shot. Then rank those candidates by whether they show the subject, with a face detector or an image-text model. Sharpness alone favours busy backgrounds over faces.
How do I detect if a video frame is blurry?
Compute the variance of the Laplacian of the grayscale frame:
cv2.Laplacian(gray, cv2.CV_64F).var().
Sharp frames have strong edges and a high variance; blurred and flat frames score low. In the
Sintel trailer the black first frame scored 0.0 and the sharpest frame 617.6. Compare scores
within one video, since the number depends on resolution and content.Why is my video thumbnail black?
Because the thumbnail was taken from the first frame, and edited video almost always opens on black or a fade-in. Take the thumbnail from a scored frame instead, or at least skip frames whose mean brightness is below about 40 on a 0 to 255 scale.
How do I detect scene changes in a video?
Compare each frame's colour histogram with the previous frame's and mark a cut when the distance jumps. With 8 bins per colour channel and the Bhattacharyya distance, a threshold of 0.5 found 11 shots in the 52-second Sintel trailer. ffmpeg's
select='gt(scene,0.3)' filter and the
PySceneDetect library do the same with their own defaults.Can AI pick a better thumbnail than sharpness alone?
Yes, for the last step. Image statistics remove bad frames well but cannot tell a face from a textured wall. A face detector, an object detector, or an image-text model such as CLIP scored against "a close-up of the main character" ranks the remaining frames by what a viewer cares about.
How many frames should I sample to find a thumbnail?
Four a second covers most video and scored the 52-second Sintel trailer's 206 frames in under a second on a CPU. Use 8 to 12 a second for trailers and ads with very short shots, and 1 a second for long, slow footage such as lectures or security video.
Key Takeaways
Sources
Where to go next
Video scene segmentation · best AI video analysis tools · MVS, vector search on object storage · pricing