What is Taxonomy
Taxonomy - A declared label hierarchy that content is classified against, flat or nested
A taxonomy is the set of categories you want your content sorted into, together with the rules for assigning them. It can be flat, a single list of labels, or hierarchical, where a label has parents and children and assigning a leaf implies its ancestors. The point is that the vocabulary is declared rather than emergent: everyone filtering, reporting or buying against those labels means the same thing by them.
How It Works
You define the labels and, in a hierarchy, their relationships. Classification then assigns content to them, either by embedding similarity against label descriptions, by a model prompted with the label set, or by explicit rules. Hierarchies make the assignment cheaper as well as more legible, since a classifier can decide at the top level and only then descend, rather than scoring every leaf.
Technical Details
Two decisions shape everything downstream. Whether a document may hold several labels at once, because single-label classification forces a choice that a piece of content with three subjects will get wrong. And what happens below the confidence threshold, because a taxonomy that always assigns something produces confident nonsense on out-of-vocabulary content, while one that can decline leaves a reviewable gap. A taxonomy is also a versioned artifact: re-labelling a corpus after the vocabulary changes is a re-run, and comparing metrics across a vocabulary change without saying so produces a trend that is really a definition change.
Common Pitfalls
- Forcing a single label onto content that genuinely spans several categories
- Having no threshold, so out-of-vocabulary content is assigned the least-wrong label with high confidence
- Changing the vocabulary and comparing before-and-after metrics as though they measured the same thing
- Building the hierarchy around internal team structure rather than around how people search
- Treating classifier output as ground truth when it was never checked against a labelled sample
Relevance to Multimodal Systems
Taxonomies are how multimodal libraries become navigable rather than merely searchable. Free-text search answers a question someone already knows how to ask; a taxonomy answers browsing, reporting, brand-safety screening and ad-adjacency decisions, all of which need a stable vocabulary rather than a similarity score. See hierarchical taxonomy, the taxonomy documentation, classifying content against a taxonomy, and building one from unlabeled data.
Put multimodal search to work
Connect a bucket and Mixpeek runs the whole multimodal search pipeline for you: extraction, indexing, and search over your own objects. No models to wire up, nothing to host.
Start with ManagedAlready have vectors?
Keep your embeddings on your own cloud and run dense, sparse, and BM25 search directly on object storage. From $25/mo.
Start with MVS