What Is Composite Clustering? Clustering Across Multiple Feature Spaces (and Clusters of Clusters)

The Short Answer

There are two different things people mean by "composite clustering," and mixing them up leads to the wrong pipeline. Multi-feature clustering groups each document once using *several* embedding spaces together — text + image + audio + faces — so the groups reflect the whole item, not one channel. Composite (cluster-of-clusters) clustering goes a level up: it clusters the *centroids of prior clusterings* to reveal how your groupings relate to each other — "how do my visual-style groups line up with my messaging-theme groups?" The first produces one set of multimodal groups; the second produces a higher-order map of patterns across dimensions you already discovered. Most real analysis uses both: multi-feature to build the per-dimension groupings, composite to see how they interact.

Multi-feature clustering: one grouping from many signals

A video is not one vector — it is a transcript, on-screen text, visual scenes, objects, actions, faces, and metadata, each with its own embedding space. Clustering on a single one of those throws away the rest. Multi-feature clustering combines them, and the interesting decision is *how* to combine, because there are three honest strategies with different meanings:

Strategy

What it does

Use when

Concatenate (default)	Fuses all embeddings into one vector, then clusters once — with optional per-feature weights	The signals are complementary and you want a single set of groups
Independent	Runs a separate clustering per feature, one output per modality	You want to *compare* how the text groups differ from the image groups
Weighted	Auto-learns the optimal blend of features via Bayesian optimization	You do not know which modality should dominate — let the data decide

The trap to avoid is naive concatenation without thought: stacking a 1024-dim text vector next to a 512-dim image vector lets the larger, higher-variance space silently dominate the distance metric. Per-feature weights (or the weighted strategy that learns them) exist precisely to stop one modality from drowning out the others. And because embeddings from different models live on different scales and in different regions of space, normalization before fusing is not optional — it is the difference between clusters that mean something and clusters that just track vector magnitude.

Composite clustering: clustering the clusters

Once you have several clusterings of the same corpus — one over visual style, one over messaging theme, one over audio mood — a natural question appears that no single clustering answers: *how do these groupings relate to each other?* Composite clustering answers it by clustering the centroids of those prior runs together. Each prior clustering contributes one vector per group (a 10,000-document run with 50 groups contributes just 50 vectors), so it operates on a tiny, information-dense set and runs fast.

What surfaces is higher-order structure: when a "street-interview format" centroid lands next to a "risk-reversal claim" centroid in the composite, that co-location *is* the finding — a recurring combination worth acting on, discovered without anyone tagging a single document. This is the concrete mechanism behind "what concepts exist in my data that nobody labeled?": discovered structure, then discovered structure *about* that structure.

Two honest limits to state up front. First, composite works on centroids, so it tells you how groups relate at the *pattern* level — it is a map, not a per-document cross-tab. If you need the exact count of documents in "group A AND group B," that per-document intersection is a separate query, not what the composite returns; each input clustering keeps its own per-document memberships. Second, the quality of a composite is capped by the quality of its inputs: cluster the wrong dimensions and the composite faithfully maps a relationship nobody cares about.

How is this different from multi-vector or late-interaction retrieval?

Because the words overlap, it is worth drawing the line. Multi-feature and composite clustering are unsupervised discovery — they find structure with no query. Late-interaction / multi-vector retrieval is ranking at query time — it scores documents against a specific query using many token- or region-level vectors. One organizes your corpus into groups you can browse and label; the other answers a search. They compose well (cluster to understand the corpus, retrieve to answer questions over it) but they solve different problems, and a "cluster of clusters" is not a reranker.

How does this work on Mixpeek?

Both live in the same clustering resource (clusters docs). For multi-feature, you pass multiple feature_uris and a combine strategy (concatenate / independent / weighted) in the vector_config; preprocessing steps like whitening and UMAP run before the algorithm, and any of the supported density- or centroid-based algorithms (HDBSCAN and friends) do the grouping. For composite, you first run several clusterings on the same collection (one per dimension), then execute with mode: "composite" and the prior runs as source_execution_ids — the run groups their centroids into the higher-order map. You read the result on the cluster visualization in Studio: lasso a region to search inside it, or spin up a retriever scoped to a group straight from the map. From there, a group that matters can be promoted into a taxonomy and used to classify new content automatically — discovery becoming governance. Rates for the underlying indexing are on pricing; to browse the visualization tooling landscape, see the best embedding visualization tools.