Guides
Vendor-neutral, engineer-written guides to the concepts behind multimodal AI: perception, retrieval, embeddings, and the infrastructure agents use to see, hear, and search unstructured data. Learn the idea first; then see how Mixpeek applies it.
195 guides across 25 topics
Why Does My Logo Search Return the Wrong Brand on Transparent Images?
A bare image.convert("RGB") throws away the alpha channel and keeps whatever color the file stored under it, so the model embeds a logo on a background you never chose. We flattened nine transparent logos onto black and onto white and queried a production SigLIP index: on black, Adidas returned Louis Vuitton at 0.941. Here is the measurement, the fix and how to check your own index.
How to Search Inside Videos in iconik with Natural Language
iconik search matches metadata, so finding a moment by what is on screen needs an index built from the footage. Compare iconik's own AI search, the iconik and TwelveLabs connector, and syncing iconik to Mixpeek, with setup steps, costs and FAQs.
What Should I Use Instead of the Google Video Intelligence API?
Google deprecated the Video Intelligence API on September 14, 2026 and shuts it down on September 14, 2027. Google's own recommendation is the Gemini API, which handles labels, on-screen text, speech and summaries from a prompt for a fraction of the per-minute price. Shot boundaries, faces and object tracks map better to Amazon Rekognition or open-source models. Here is the feature-by-feature replacement, the cost at 10,000 minutes a month and a migration plan.
What Is AI Video Analysis? How It Works, and How to Find a Moment in a Video
AI video analysis turns video into data you can query: what is on screen, what is said, what text appears and when each happens. How it works in five steps, a request you can run against a public footage library with no key, and how Mixpeek, Twelve Labs, Google Video Intelligence, Amazon Rekognition and an open-source CLIP pipeline compare on price and on what they return.
How Do I Generate Alt Text for Thousands of Images?
To generate alt text for thousands of images, sort the images by role first, send only the ones that carry meaning to a vision model along with the page they sit on, check the drafts with simple rules, and have a person review the risky ones. Here is the workflow, the prompt and where it goes wrong.
IAB Tech Lab Agent Registry: every agent and MCP server, by category
All 51 agents in IAB Tech Lab's Agent & MCP Server Registry on 2026-11-01, grouped by category, with vendor, protocol, deployment type, tools found and the registry's endpoint check. Counts by category, protocol and verification, how to query the registry's API with an anonymous token, and how to list an agent. Refreshed monthly.
Why Does My Document Search Miss the Answer That Is in a Chart or Table?
To search a folder of PDFs when the answer is inside a chart, index what each page shows as well as what it says: extract charts and tables into text, index the page images, or have a vision model describe each figure. This guide explains why ordinary PDF search misses chart and table answers, how to test your own folder, and which of the three fixes to use.
How Do I Find the Images My Computer Vision Model Gets Wrong?
Find the images a vision model gets wrong by storing each prediction next to its label, sorting the misses by confidence, and searching the dataset for more images that look like each failure. Confident misses show you label errors and systematic gaps, similarity search turns one failure into a set, and a deduplicated, label-balanced pick becomes the next training or evaluation set. Measured on a 40-image production run: 16 misses ranked in 0.7 s for 0.88 USD.
AdCP, AAMP, ARTF and Agentic Audiences: what each one does
AdCP and AAMP are two separate efforts to standardize AI agents in advertising. AdCP is one protocol from AgenticAdvertising.org, carried over MCP or A2A, with tasks like get_signals and create_media_buy. AAMP is IAB Tech Lab's umbrella, which includes ARTF (agent containers inside an exchange that return bid request mutations) and Agentic Audiences (embeddings carried in OpenRTB). A dated comparison table, spec links, and what to do when a contextual embedding model changes.
How Does Scene-Level Contextual Targeting Work for CTV?
Scene-level contextual targeting picks a CTV ad by the scene that plays just before the ad break, read from the video's frames, audio and captions, with no data about the viewer. Each scene is classified when the stream is ingested, the categories are stored with timestamps, and at the break the signal is a lookup. How it differs from program-level targeting, what it reads, how it reaches the bid, and who sells it.
Visual search that handles negation, attribute bleed and unseen classes without retraining
Why CLIP-style image search fails on "not a dog" and "white dog", and how to fix it without a full fine-tune: recall with embeddings, check attributes on the candidates, onboard new classes from exemplars, collect click and label feedback, and learn a query correction from the queries that fail.
Can You Do a Reverse Video Search on Google?
Not directly: Google has no upload-a-video search. Screenshot a distinctive frame and search it with Google Lens, or use Circle to Search on Android, then open the earliest upload among the matches. Why frame search misses some videos, what Lens does with a recorded clip, and how to search a library you own.
All guides
Every guide in one place, 195 in total.
- Why Does My Logo Search Return the Wrong Brand on Transparent Images?
- What Should I Use Instead of the Google Video Intelligence API?
- How Do I Generate Alt Text for Thousands of Images?
- How Do I Find the Images My Computer Vision Model Gets Wrong?
- How Does Scene-Level Contextual Targeting Work for CTV?
- Visual search that handles negation, attribute bleed and unseen classes without retraining
- Can You Do a Reverse Video Search on Google?
- Cyberdefense as a search problem: narrow billions of audit logs to the few an AI investigator should read
- Will My Stored IAB Category IDs Still Work After a Taxonomy Upgrade?
- How Do You Convert Text Into Vector Embeddings?
- How Do I Find Specific Moments Across Thousands of Hours of Robot or Dashcam Recordings?
- How Much Does It Cost to Transcribe an Hour of Audio?
- How Many Tokens Is an Hour of Video, and What Does It Cost to Send to an LLM?
- How Much Does It Cost to Scan Every Video Ad in Our Archive for Logos and Products?
- Why Is My AI Agent Using Thousands of Tokens Before I Ask It Anything?
- Why Does My Docs Chatbot Return Half a Code Example?
- Why Does My Vector Search Return Fewer Results When I Add a Filter?
- Video Embedding Models in 2026: Limits, Prices, and a Search You Can Run
- How to Find the Full Video From a Clip or a Screenshot
- What Is AI Video Analysis? How It Works, and How to Find a Moment in a Video
- Why Is the First Vector Search Query on S3 Slow? Cold, Warm and Cached Latency, Measured
- Our Brand Safety Tool Flags Too Many False Positives on Video. How Do We Fix It?
- We Can't Tell What Content Our CTV Ads Ran Next To. How Do We Find Out?
- Can you run vector search on NFS, Lustre or a local NVMe drive?
- Did my search get better? How to test a retrieval change before you ship it
- AdCP, AAMP, ARTF and Agentic Audiences: what each one does
- IAB Content Taxonomy 3.1 classification benchmark: Mixpeek on IAB Tech Lab's test set
- Using Mixpeek in the IAB Tech Lab buyer agent: classification, brand safety, contextual search
- IAB Tech Lab Agent Registry: every agent and MCP server, by category
- Agentic Audiences contextual embeddings from video: building the ext.aa segment
- OpenProposal (AAMP 3.0) explained: the object, its 71 fields and what buyer agents read
- Build an ARTF agent container that adds IAB content categories
- Why CTV bid requests lack content categories, and how to fill them
- Why Is My Image Embedding Model So Slow on CPU?
- Why Does My Content Moderation Flag So Many False Positives?
- Why Does My Image Search Miss Small Objects in a Photo?
- Why Does AI Miss Things When I Ask It About a Long Video?
- How Do You Run Agentic Search Over a Hierarchy of Clusters?
- Should a RAG Pipeline Use a Classification Model or an LLM to Filter and Rerank Results?
- Why Do I Get Different Search Results Every Time I Run the Same Query?
- Why Does My Video Search Return the Wrong Scene?
- How Do I Stop Duplicate Videos Showing Up in My Search Results?
- How Do I Let an AI Agent Look Through My Own Video Library?
- How Do I Find Where Users Got Stuck Across Dozens of Usability Test Recordings?
- How Do I Search Photos, Video and Audio on the Device, Without Uploading Them?
- How Do I Match the Right Video to Each Article on My Site?
- How Do I Keep Footage of Minors Out of a Video Training Dataset?
- How Do I Get an Alert When My Security Camera Sees Something Specific?
- How Do I Find One Specific Moment in Hours of Footage?
- How Do I Find Where Someone Said Something in Our Meeting Recordings?
- How Do I Find Duplicate Photos, Including Edited and Resized Copies?
- How Do I Make Hundreds of Scanned PDFs Searchable?
- How Do I Find Every Ad We've Run That Shows a Specific Product?
- What Does 'Multimodal' Mean in Google Search Console?
- Why Does My Image Search Return Similar-Looking Items Instead of the Exact One?
- How Do I Find Out Where a Video Came From?
- Why Do My Photos Have No Date or Location, and Can I Get It Back?
- Why Can't My Search Find Exact Part Numbers, Codes or Names?
- Why Do My Search Results Get Worse as I Add More Data?
- Why Your Video Search Can't Find Words That Appear On Screen
- How to Prove Who Changed a Configuration, When, and Why
- Why Does My Search Find the Right Document but the Wrong Part of It?
- How Do You Search a 3D World You Generated?
- Search Every World You Generate with Atlas
- Search Every World You Generate with Runway
- Search Everything You Generate on fal
- Search Every Image You Generate with ChatGPT
- Why Does My Podcast Search Find the Right Episode but Not the Moment?
- Why Does My Search Only Work in One Language?
- Search Every Conversation You Have with GPT Live
- What Is a Semantic Contract? The Interface Between Extraction and Retrieval
- What Does a Semantic Layer Compile To When the Target Is Not SQL?
- How Should an AI Agent Choose Between Retrieval Capabilities?
- Should Semantic Features Be Materialized or Computed at Query Time?
- How Do You Model Unstructured Data From Raw Objects to Serving Contracts?
- What a Semantic Layer Looks Like in Production
- Compute Pushdown: Why Your Vector Search Over Object Storage Is Slow
- How to Merge Data Sources With Different Schemas Into One Searchable Index
- Write-Ahead Logs on Object Storage: How S3 Becomes a Database's Durability Layer
- Query Planning for Multimodal Retrieval: Stage Order, Filter Pushdown, and Candidate Budgets
- Multi-Stage Retrieval: Retrieve, Filter, Rerank
- Change Data Capture for a Search Index: Keeping Downstream Systems in Sync
- Query Rewriting, Expansion and HyDE: Fixing Retrieval Before It Searches
- Cross-Lingual Retrieval: Searching Content in Languages You Did Not Query In
- C2PA Content Credentials: How Media Provenance Actually Works
- How Do You Detect and Redact PII in Images, Video, and Audio?
- How Do You Delete Data From a Vector Index? Embeddings, Tombstones, and the Right to Be Forgotten
- Do You Need a Vector Database? When Brute Force, pgvector, and a Dedicated Store Each Win
- How to Build a Taxonomy From Unlabeled Data: Clustering, Labeling, and Promotion
- What Is Hybrid Search? BM25, Vector Retrieval, and How to Fuse Their Rankings
- What Is MUVERA? Turning Multi-Vector Retrieval Into a Single-Vector Search
- How Do You Run Your Own Model Inside a Managed Search Pipeline?
- How Do You Isolate Tenants in a Vector Index?
- What Does It Cost to Make a Video Library Searchable?
- How to Search, Deduplicate, and Moderate AI-Generated Video and Images (FLUX 3, Kling, Veo, Runway)
- How Does LoRA Fine-Tuning Work? (Adapters, QLoRA, DoRA, and Fine-Tuning Retrieval Models)
- How Do I Evaluate a RAG Pipeline? (Faithfulness, Answer Relevance, Hallucination, and Context Metrics)
- How Do I Ingest Millions of Files into a Search Index? (Images, Video, Documents at Scale)
- How Do AI Agents Search Big Datasets by Navigating Clusters? (Hierarchical Cluster Search)
- What Is Composite Clustering? Clustering Across Multiple Feature Spaces (and Clusters of Clusters)
- How Do I Filter Vector Search Results by Location? (Radius, Bounding Box, Polygon)
- How Do I Automatically Classify Content Against a Taxonomy?
- Brand Safety vs Brand Suitability: How AI Classifies Video for Advertisers
- How Do I Debug Bad Retrieval Results in RAG and Vector Search?
- Can You Run a Vector Database on S3? Object-Storage-Backed Vector Search, Explained
- How Do I Build a Deep Research Agent Over My Own Data?
- What Concepts Exist in My Data That Nobody Has Labeled Yet?
- Marengo 3.0 on Amazon Bedrock: Architecture, Benchmarks and Vector Storage
- How to Store TwelveLabs Marengo Embeddings in Your Own Vector Store
- Video Highlight Detection: How AI Finds the Best Moments
- How Much Does a Vector Database Cost? A 2026 Pricing Comparison
- How to Switch Embedding Models Without Re-Embedding Everything
- Reverse Video Search: How It Works and How to Build One
- Semantic Caching: How Agents Skip Work They Have Already Done
- Efficient Attention: How Models Read Hour-Long Video and Book-Length Documents
- Optical Context Compression: Reading Documents as Images, Not Text
- Multi-Object Tracking: How Agents Follow Objects Across Video Frames
- Monocular Depth Estimation: How Models Infer 3D From a Single Image
- Instance-Level Visual Matching: Finding the Same Object, Not Just Similar Ones
- Face Recognition and Identity Clustering: How Agents Recognize and Group People in Video
- Reasoning Rerankers: How Listwise LLM Rerankers Reorder Retrieval Results
- Retrieval Feedback Loops: Learning to Rank from Clicks, Outcomes, and Agent Interactions
- Matryoshka Representation Learning: Nested Embeddings for Adaptive Multimodal Retrieval
- Creative Ad Analysis for AI Agents: JEPA, Multi-Vector Retrieval, and Signal Fusion
- How to Check if an Image Is Copyrighted
- How Do I Find Out If Someone Reposted My Video?
- Video Copyright Checker: How to Check if a Video Is Copyrighted
- How to Check if a Song Is Copyrighted on YouTube
- How to Check if a Picture Is Copyrighted Before Using It
- The Complete Guide to Pre-Publication IP Clearance
- What Is a Multimodal Data Warehouse?
- How to Build a Multimodal Data Warehouse
- Multimodal Data Warehouse Architecture Deep Dive
- Vector Storage Tiering: The Architecture Guide to Hot, Warm, and Cold Vector Data
- How to Build MCP Tools for Multimodal AI Agents
- Embedding Portability and Versioning: Why Your Vectors Are Not as Portable as You Think
- How to Build a Multimodal RAG Pipeline
- Context Engineering for AI Agents
- Video Temporal Grounding: How AI Agents Find Specific Moments in Video
- Multimodal Perception for AI Agents: How to Give Your Agent Eyes, Ears, and Memory
- Open-Vocabulary Object Detection: Teaching AI to Find Anything You Describe
- Audio Feature Extraction: How AI Agents Learn to Hear
- Contrastive Learning: How CLIP, SigLIP, and CLAP Actually Work
- Multi-Stage Retrieval: How AI Agents Search Unstructured Data at Scale
- Visual Document Retrieval: How AI Agents Search Documents Without OCR
- Video Scene Segmentation: How AI Decomposes Continuous Video into Searchable Segments
- Omnimodal Embeddings: One Model for Text, Image, Audio, and Video Retrieval
- Speaker Diarization: How AI Agents Know Who Said What in Audio and Video
- Cross-Encoder Reranking: Why Two-Stage Retrieval Beats One-Stage Search
- MCP Tool Design for Multimodal Search
- Long-Context Video Understanding for Agent Perception
- Structured Extraction from Unstructured Documents: How Vision-Language Models Replace OCR Pipelines
- Late Interaction Retrieval: How ColBERT, ColPali, and ColQwen Search Without Losing Token-Level Detail
- Agentic Retrieval: How AI Agents Search Differently Than Humans
- Embedding Quantization & Compression: Making Vector Search Practical at Billion Scale
- Video RAG: Building Retrieval-Augmented Generation Over Video Corpora
- Evaluating Multimodal Retrieval: Metrics, Benchmarks, and Ground Truth
- Multimodal Chunking Strategies: How to Decompose Video, Audio, Images, and Documents for Search
- How to Build a Video Perception Layer for AI Agents
- Approximate Nearest Neighbor Search: The Algorithms Behind Fast Vector Retrieval
- Instruction-Tuned Embeddings: How Task Prompts Transform Retrieval Quality
- Multi-Index Search Architecture: How to Combine Visual, Audio, and Text Embeddings for Rich Media
- Embedding Space Geometry: Why Cosine Similarity Doesn't Always Mean What You Think
- Object Decomposition and Layered Indexing for AI Agent Perception
- Retrieval Control Planes for AI Agents: Streaming, Cancellation, and Budgets
- Video Anomaly Detection for AI Agents: From Signals to Searchable Events
- Agent Perception Evals: Testing Whether AI Agents Can See, Hear, and Search
- Audio-Visual Retrieval for AI Agents: How to Search What Happened, Not Just What Was Said
- Computer-Use Agent Memory: How to Search Screens, Tools, and UI State
- Production Ingestion Reliability for Agent Perception: Ledgers, Backfills, and Recall Checks
- Mask-Aware Retrieval for AI Agents: Segment First, Search Crops, Then Reason
- Forced Alignment for AI Agents: Word Timestamps, Diarization, and Audio Evidence Search
- Payload Projection for Agentic Vector Search: Field Selection, Evidence Handles, and Context Budgets
- Adaptive Indexing for Agentic Search: Query Logs, Payload Indexes, and Retrieval Routing
- Video Frame Sampling: How Many Frames to Embed and Which Ones to Keep
- Calibrating Similarity Scores: What Cosine Similarity Actually Means for Retrieval
- Chunk Contextualization: How Late Chunking and Contextual Retrieval Fix Broken RAG
- Budget-Aware Multi-Vector Retrieval: How to Make Late Interaction Affordable at Scale
- Diversity-Aware Retrieval: How Agents Avoid Twenty Near-Duplicate Results
- Query Transformation Pipelines: Turning a Raw Agent Question Into Searchable Intent
- Learned Sparse Retrieval and Dense-Sparse Hybrid: Why Agents Need Both Vocabularies
- The Modality Gap: Why Text-to-Image Search Underperforms and How Agents Fix It
- How OCR Actually Works: Detection, Recognition, Reading Order, and Tables
- How ASR Decoding Actually Works: Beam Search, LM Fusion, and Confidence
- Index Freshness and Incremental Updates: How Just-Ingested Content Becomes Searchable
- Filtered Vector Search: How Agents Combine Similarity with Hard Constraints
- How Vision-Language Models Fuse Image and Text Tokens
- Hybrid Search Fusion: How to Combine Dense and Lexical Retrieval Without Breaking Ranking
- Audio Fingerprinting: How Agents Recognize a Specific Recording in Noise
- Embedding Fine-Tuning and Distillation: Teaching an Agent to See and Hear Your Domain
- BM25 and the Inverted Index: The Lexical Retriever Every Hybrid Search Treats as a Black Box
- Perceptual Image Hashing: How Agents Recognize the Same Picture After It Has Been Re-Encoded, Cropped, and Recolored
- Streaming Video Understanding: How Agents Watch an Unbounded Live Feed in Real Time
- Why Does My Document Search Miss the Answer That Is in a Chart or Table?
- How to Search Inside Videos in iconik with Natural Language
From concept to production
These guides explain how multimodal perception and retrieval actually work. Mixpeek is the platform that runs them: point it at your storage and get back relevant, timestamped results.