The multimodal warehouse · Video
Find any moment in any video with one video search API.
Describe the moment and get the scene and the timestamp back. Every video is cut into scenes with what was said, shown and written on screen, stored next to the rest of your files so a scene can be joined to the contract, the product photo or the database row it belongs with.
From $25/mo. Index once. Query every way.
| scene | Presenter holds the phone up to camera | 00:12 to 00:19 |
| transcript | “Ships to every store on Friday” | 00:14 to 00:17 |
| on-screen text | “Pre-order now, $799” | 00:16 |
| face | One recurring speaker, matched across clips | 00:12 |
| keyframe | An image embedding of the frame | 00:15 |
One warehouse
Your videos sit beside every other file you own.
Each file is broken into rows that point back to the file they came from. Video scenes, images, audio segments and document blocks live in one namespace, so a question can start in one kind of file and finish in another.
Joins your videos make
Which scenes show a product from our catalog?
video scenesproduct photos
Runs on the Cross compare stageAttach each ad's spend from our database to its scenes.
video scenesyour SQL table
Runs on the SQL lookup stageShow the talent contract next to every clip a person appears in.
video scenescontract PDFs
Runs on the Document enrich stageWhat you can ask
Ask in plain words, or with a file.
› the presenter holding the phone to camera
The scene, its video and the second it starts
› every clip where someone says “free shipping”
Transcript matches with their timestamps
› this photo of our product
Scenes that look like it, ranked
› price shown on screen, under 30 seconds long
On-screen text matches filtered by length
Start in one click
Templates that index videos
Millions of scenes you can move through instead of search. Footage, ads and editor assets land in three clean collections, and scenes are clustered one partition at a time so every run stays under the sample-size caps..
Deploy in one click →Search every scene of every ad you have run, and find the ones that keep earning spend.
Deploy in one click →Segment a video corpus, extract a keyframe record for every segment, and match brand marks against a reference set you upload. Face and public-figure matching run on a dedicated or customer-hosted deployment. This deploy is the extraction layer a moderation policy sits on, and the page carries a state on every part so you can see which policy pieces are yours to add..
Deploy in one click →An article goes in, a ranked playlist of video comes out, and every match carries the entities, keywords and categories that produced it. Both sides are embedded twice, once by a text model and once by a multimodal one, so nothing is embedded at query time and a clip in another language still matches..
Deploy in one click →An empty namespace
Connect the bucket your videos already live in and pick the extractors. Studio indexes it and your first query runs against your own files.
Create a namespace →In code
One call to search what was indexed.
The extractors that turn videos into rows:
- Multimodal →Scenes, transcripts, OCR and descriptions
- Scrolling text →Every word that appears on screen
- Face identity →The same person across clips
- Audio fingerprint →Jingles and sound marks in the audio track
from mixpeek import Mixpeek
client = Mixpeek(api_key="YOUR_API_KEY")
# Describe the moment; get back the scenes that match, with timestamps
results = client.retrievers.execute(
"ret_YOUR_RETRIEVER_ID",
inputs={"query": "person demonstrating the product features"},
filters={"AND": [
{"field": "metadata.duration_seconds", "operator": "gte", "value": 30},
]},
pagination={"method": "cursor", "limit": 10},
)
for doc in results["documents"]:
print(doc["document_id"], doc.get("start_time"), doc.get("end_time"), doc["score"])Questions
What people ask about video search
How do you search inside a video for a specific moment?
Split the video into scenes, turn what each scene shows, says and displays into searchable features (image embeddings of its frames, a transcript of its speech, the text on screen), and match your query against those features instead of the file's title or tags. The answer is a list of timestamps, not a list of files. With Mixpeek you describe the moment in plain English, or give an image or a clip, and get back the matching scenes with their start and end times.
Why does my video search return the wrong scene?
Usually for one of four reasons. The index holds one embedding per video, so a short moment is averaged away by everything else in the file. Keyframes are sampled too sparsely, so the moment falls between two frames. The search only reads one channel: a line that is spoken but never shown is invisible to a frame-only index, and a sign on screen is invisible to a transcript-only one. Or the query is embedded differently from the content, which lowers every score and reorders results. Index per scene, sample at scene boundaries, search frames, speech and on-screen text together, and embed queries the same way the content was embedded.
How do I find a clip in hours of footage that nobody tagged?
Search the content rather than the tags. Once each scene is indexed by what it shows and says, a description such as 'a forklift reversing near a loading dock' finds the scene directly, and a reference image finds visually similar shots. Tags are still useful as filters on top, for example limiting the search to one shoot or one date range.
What is video search?
Video search is the ability to find specific moments, objects, scenes, or spoken content within video files using queries. Unlike traditional video search that relies on titles and metadata, semantic video search understands the actual content of the video -- visual elements, audio, text overlays, and context -- enabling natural language queries like 'find the scene where someone demonstrates the product'.
How does semantic video search work?
Semantic video search works by extracting multi-modal features from video content: visual embeddings from frames, transcriptions from audio, OCR from text overlays, and object detections from scenes. These features are indexed as multi-dimensional vectors. When you search, your query is converted to the same vector space, and the system finds the closest matching moments using approximate nearest neighbor algorithms.
What video formats does Mixpeek support?
Mixpeek supports all major video formats including MP4, MOV, AVI, WebM, MKV, FLV, and WMV. Videos are automatically transcoded during ingestion, so you can upload in any format without preprocessing. We support resolutions up to 4K and videos of any duration.
Can I search for specific moments in a video?
Yes. Mixpeek returns timestamped results with precise start and end times for every match. You can search for specific visual moments ('the red car turning left'), spoken phrases ('when they mention pricing'), or combinations of both. Results include confidence scores and frame-level previews.
How does video search handle long-form content?
Long-form content is processed using configurable frame extraction intervals and intelligent scene detection. Rather than analyzing every frame, Mixpeek identifies keyframes and scene transitions to create an efficient index. This means a 2-hour video can be fully indexed and searchable without processing millions of redundant frames.
What is the difference between video search and video intelligence?
Video search focuses on finding and retrieving specific content within videos based on queries. Video intelligence is broader -- it includes search but also encompasses content understanding, automated tagging, anomaly detection, content moderation, and analytics. Mixpeek provides both capabilities through its feature extraction and retrieval pipeline.
Can I use custom models for video feature extraction?
Yes. Mixpeek supports bring-your-own-model (BYOM) for feature extraction. You can deploy custom visual models, audio models, or embedding models alongside Mixpeek's default extractors. This is useful for domain-specific recognition tasks like medical imaging, manufacturing inspection, or branded content detection.
Is video search available for on-premise deployment?
Yes. Mixpeek offers BYO Cloud deployment where the entire video search pipeline runs within your own infrastructure (AWS, GCP, or Azure VPC). This ensures your video data never leaves your environment, meeting strict compliance and data residency requirements. See our deployment options page for details.
Put your videos in the warehouse.
Index them once, then ask them anything, alone or joined to the rest of your files.
Other file types:ImagesAudioDocumentsHow the warehouse works