How do I classify video, images and audio into IAB Content Taxonomy 3.1?
Classify video, images and audio into IAB Content Taxonomy 3.1. Each video scene returns tier 1 to tier 4 category IDs and names, a confidence score and start and end timestamps, ready to store at ingest and read at bid time as OpenRTB cat with cattax.
Engineers at SSPs, DSPs, publishers, CTV platforms and podcast networks who need IAB Content Taxonomy 3.1 IDs on video, image and audio inventory, and who want to call an API and check the output themselves.
Most contextual classification reads the text around a piece of media: a page, a title, a description. Video, CTV and podcast inventory often arrives with little or none of that text, so it goes unlabeled or gets a label from its metadata alone. Buyers then either skip the inventory or bid on it blind.
Ready to implement?
The short answer
Mixpeek classifies video, images and audio against the official IAB Content Taxonomy 3.1. Video is split into scenes, and each scene is read from its frames, its speech transcript and its on-screen text. The response gives tier 1 to tier 4 category IDs and names, a confidence score per category, and start and end timestamps for every scene. Classify once at ingest, store the IDs, and read them at bid time as OpenRTB cat with cattax set to the AdCOM value for Content Taxonomy 3.1.
See It in Action
Try the free IAB contextual classifier: paste a video URL and see the IAB categories it returns.
Before & After Mixpeek
Before
Input
Page text, titles and descriptions only
Granularity
One category per asset
Taxonomy IDs
Hand-maintained list, unchecked against the official file
Bid time
Classify in the auction, or skip unlabeled inventory
After
Input
Video frames, speech transcripts, on-screen text and images
Granularity
Tier 1 to tier 4 categories per scene, with timestamps and confidence scores
Taxonomy IDs
Generated from IAB Tech Lab's 3.1 file and diffed against it by a unit test
Bid time
Look up stored IDs and send them as OpenRTB cat with cattax
Why Mixpeek
The taxonomy is IAB Tech Lab's own 3.1 file, generated into the platform and checked by a unit test that compares every ID, name, parent and tier against it, so the IDs you pass downstream are the official ones. The same API reads frames, speech and on-screen text, and classification runs once, at ingest, so the auction only reads stored IDs.
Overview
To classify video, images or audio into IAB Content Taxonomy 3.1, send the media to Mixpeek and classify it against the official 3.1 tree. Video is split into scenes, and each scene is read from its frames, its speech transcript and its on-screen text. Each scene comes back with tier 1 to tier 4 category IDs and names, a confidence score per category, and start and end timestamps. Images return the same fields without timestamps. Podcast episodes are classified from their transcripts, scene by scene or as a whole episode. The taxonomy holds all 704 categories in IAB Tech Lab's published file, 37 at tier 1, 323 at tier 2, 275 at tier 3 and 69 at tier 4 (Content Taxonomy 3.1 TSV). Mixpeek generates its copy from that file, and a unit test diffs every category ID, name, parent ID and tier against it, so a renamed category or a moved parent fails that test. IAB Tech Lab released 3.1 in December 2024, and its content taxonomy page describes the change from 3.0 as an update to the genre category that adds Thriller (IAB Tech Lab). In the 3.1 file Thriller is ID 700 under Genres (Content Taxonomy 3.1 TSV). The pattern for programmatic use is to classify at ingest, store the categories and embeddings with the media, and look them up at bid time. The stored IDs go into the OpenRTB content object as cat, with cattax set to 9, the AdCOM value for IAB Tech Lab Content Taxonomy 3.1 (AdCOM category taxonomies list). You can try the free IAB contextual classifier demo before writing any code. An open IAB 3.1 benchmark for this classification is not yet published.
Challenges This Solves
Video and audio arrive without usable text
Text classifiers need a page, a title or a description. CTV episodes, short-form video and podcast episodes often carry a generic title and nothing else.
Impact: Inventory with the most watch time gets the thinnest labels, so buyers exclude it or price it as unknown.
One label per asset hides what each scene is about
A long episode can move from cooking to travel to a sports segment. A single asset-level category averages those away.
Impact: Ads land beside a scene that has nothing to do with the category the asset was sold under.
Taxonomy IDs drift from the official file
Hand-maintained category lists pick up renamed nodes, wrong parents and IDs from older versions. The IDs still look valid, so nothing errors.
Impact: Downstream systems read the wrong category for the right ID, and the mismatch is hard to see until a buyer audits it.
Classification inside the auction is too slow
Running a model on media while a bid request waits does not fit a real-time bidding timeout.
Impact: Teams either skip classification for new inventory or classify only the text they can process quickly.
Recipe Composition
This use case is composed of the following recipes, connected as a pipeline.
Feature Extractors Used
multimodal extractor
Scene Classification
Categorize images based on scene type (indoor, outdoor, etc.)
transcription
ocr text extraction
Retriever Stages Used
taxonomy-enrich
attribute-filter
feature-search
Expected Outcomes
Official IAB Content Taxonomy 3.1 IDs, checked by a unit test
Taxonomy
Tier paths per scene with timestamps
Video and CTV
Categories from the spoken transcript
Podcasts
Stored IDs read by lookup
Bid time
Build this in the docs
The exact stages and extractors this use case runs on, with API reference and worked examples.
Classify your inventory into IAB 3.1
Load the IAB Content Taxonomy 3.1 tree, point a collection at your video, image or audio bucket, and read tier paths with confidence scores for every scene.
Frequently Asked Questions
Related Use Cases
Contextual Page Signals for Ad Placement
One article URL in, five structured signals out: IAB category, entities with salience scores, sentiment, brand safety, keywords.
Brand Safety Verification
AI-powered brand safety scoring for ad placements and content partnerships
AdTech Creative Intelligence
Understand what makes ad creatives perform before they run
Ready to Implement This Use Case?
Our team can help you get started with IAB Content Taxonomy 3.1 classification API for video, images and audio in your organization.