What we measured
IAB Tech Lab publishes a small, open benchmark for classifying content into IAB Content Taxonomy 3.1: iab-taxonomy-classifier (Apache-2.0). It has 704 synthetic web pages, one written for each 3.1 category, and scores a classifier by whether the page's own category comes back first (top-1) or in the first five (top-5). IAB Tech Lab ran several open embedding models on Cloudflare Workers AI and published the results in the same repository.
On 2026-10-19 we ran the same 704 pages, scored the same way, through a Mixpeek retriever built for IAB 3.1 text classification. This page reports what came back, including where it lost.
Results
| Classifier | Embedding size | Top-1 | Top-5 | Source |
|---|---|---|---|---|
| EmbeddingGemma-300M (IAB Tech Lab run) | 768 | 75.0% | 94.5% | IAB Tech Lab data |
| EmbeddingGemma-300M (IAB Tech Lab run) | 512 | 74.0% | 94.9% | IAB Tech Lab data |
| Mixpeek IAB 3.1 retriever (multilingual-e5-large-instruct) | 1024 | 72.7% | 93.8% | method |
| EmbeddingGemma-300M (IAB Tech Lab run) | 256 | 72.0% | 92.8% | IAB Tech Lab data |
| Qwen3-Embedding-0.6B (IAB Tech Lab run) | 1024 | 70.3% | 91.1% | IAB Tech Lab data |
| BGE-M3 (IAB Tech Lab run) | 1024 | 65.9% | 88.8% | IAB Tech Lab data |
| EmbeddingGemma-300M (IAB Tech Lab run) | 128 | 64.8% | 85.4% | IAB Tech Lab data |
The IAB Tech Lab rows are read from the summary files in the repository's data folder. Mixpeek: 512 of 704 correct at top-1, 660 of 704 within the top five.
In 627 of the 704 pages (89.1%), Mixpeek's first result was at least in the right tier 1 category, so most misses land on a neighbouring category in the same branch (method).
Where it misses: tier 1 pages
Accuracy depends heavily on how deep the page's category sits:
| Category tier of the page | Pages | Top-1 correct | Source |
|---|---|---|---|
| Tier 1 (for example "Attractions") | 37 | 5 (13.5%) | method |
| Tier 2 | 323 | 233 (72.1%) | method |
| Tier 3 | 275 | 218 (79.3%) | method |
| Tier 4 | 69 | 56 (81.2%) | method |
A page written to illustrate a broad tier 1 category almost always describes something specific. The test page for "Attractions" is about a suspension bridge, a botanical garden and an arts district, and the classifier returned "Parks & Nature" and "Historic Site and Landmark Tours". Flat matching across all 704 categories will usually prefer the specific child. If your system needs tier 1 labels, read them from the path of the top result (89.1% correct here) rather than asking for an exact tier 1 match (method).
Threshold or top-N?
A classifier has to decide when to return a label at all. Raising the minimum score on the top result trades coverage for precision:
| Minimum score on the top result | Pages that get a label | Of those, top-1 correct | Source |
|---|---|---|---|
| 0.76 | 99.1% | 73.1% | method |
| 0.78 | 91.8% | 74.9% | method |
| 0.80 | 69.7% | 77.2% | method |
| 0.82 | 39.6% | 81.7% | method |
| 0.84 | 14.5% | 90.2% | method |
These scores are on this model's scale. Other embedding models score on very different ranges (IAB Tech Lab's README notes EmbeddingGemma's scores ran roughly 0.24 to 0.66), so a threshold never transfers between models. Pick it per model, on a labelled sample of your own content, for the precision your buyers need.
Method
data/synthetic-content.ndjson from IAB Tech Lab's repository, all 704 records. Each page is classified from title + "\n\n" + body, as IAB Tech Lab's script does.feature_search stage. The same setup is in Using Mixpeek in the IAB Tech Lab buyer agent, with a script to build it.Limits, stated plainly:
What comes next
This run leaves three questions open: how accuracy holds on real labelled pages instead of synthetic ones, on video and audio, and as embeddings are shortened to fit a bid request. The next phase of this benchmark covers video and audio with human labels and accuracy against embedding size. This page will be updated with a dated changelog entry when it does.
Changelog
Frequently Asked Questions
How accurate is Mixpeek at IAB Content Taxonomy 3.1 classification?
On IAB Tech Lab's open 704-page synthetic text benchmark, 72.7% top-1 and 93.8% top-5 on 2026-10-19, with 89.1% of first results in the correct tier 1 category. That is below IAB Tech Lab's best published run (EmbeddingGemma-300M at 768 dimensions, 75.0% and 94.5%) and above its Qwen3-Embedding-0.6B and BGE-M3 runs (IAB Tech Lab data, method).
What is the IAB Tech Lab taxonomy classifier benchmark?
An open repository from IAB Tech Lab with 704 synthetic pages, one per Content Taxonomy 3.1 category, and a script that classifies each page by embedding similarity and reports top-1 and top-5 accuracy. It is licensed Apache-2.0 and includes the results of several embedding models.
Why is tier 1 accuracy so low?
Pages written for a broad tier 1 category describe specific things, so a classifier matching against all 704 categories usually returns a more specific child. Read tier 1 from the top result's path instead; that was correct for 89.1% of pages (method).
What threshold should I use for IAB classification?
It depends on the model and the precision you need. With this model, a minimum top score of 0.80 labelled 69.7% of pages at 77.2% precision, and 0.84 labelled 14.5% at 90.2% (method). Scores are not comparable across models, so calibrate on a labelled sample of your own content.