The Short Answer
Cluster your collection into a hierarchy, give each cluster a label, and search by walking that hierarchy from the top. At each level a model reads the query and the labels of the child clusters and picks the one most likely to contain the answer. At the bottom level, you search or return the documents inside the clusters it chose.
The model making those choices picks one option from a short list and reports how sure it is. An evaluation model such as TypeSafe's Jev is built for that job: each branch decision comes back as a probability for each child cluster. With probabilities you can keep the best two or three paths open at each level and rank complete paths at the end, which is a beam search over your corpus.
In Mixpeek this is the
cluster_navigation strategy of the agent_search
retriever stage, with model_name: "jev-latest".Cases a single vector search misses
A single vector search compares the query against the indexed documents and returns the nearest ones. That works when the words in the query sit close to the words in the answer. It struggles in three situations:
Walking a labeled tree decides the category first and searches second.
Step 1: build and label the tree
Run hierarchical clustering over the collection's embeddings. KMeans or HDBSCAN at the top level, then the same algorithm inside each top-level cluster, gives you a two-level tree. Each cluster gets a centroid document that records its label, summary and parent.
Label the clusters from a vocabulary you control. Generated labels drift between runs and between levels. A fixed list keeps "Pharmaceutical Ads" spelled the same way on each run, and it lets you line the tree up with a taxonomy you already use. Jev picks each cluster's label from the list and returns the probability, so a cluster labeled with low confidence is easy to find and review.
Step 2: walk it with probabilities
At the top level the model sees the query and the top-level labels, with each cluster's summary as the description of that option. It returns a probability for each. Keep the best few. At the next level, ask about the children of each path you kept, all in one request. Score each complete path by the geometric mean of the probabilities along it, so a shallow leaf and a deep leaf compare on equal terms.
Greedy descent, keeping one path, is a beam width of one. It makes one request per level and cannot recover from an early mistake. A beam of two or three lets a later level correct an ambiguous first choice at almost no extra latency, because the paths share a request.
On an eleven-cluster, two-level tree we tested, the walk routed "Nike commercial with a basketball player" to Product Ads > Sneaker Ads at 0.99, and kept Sports > Basketball as the runner-up at 0.14. Each two-level walk took 0.8 to 1.3 seconds.
Step 3: return the documents
Return the members of the best leaf clusters, or run a vector search first and keep only the hits that fall inside the chosen clusters. The second form keeps the vector search's ordering and uses the tree as a filter.
Use a flat search for short, specific queries
Use a single vector search when queries name the thing itself ("the Q3 earnings call"), when the corpus is small enough that a query reads most of it anyway, or when latency under 100 ms matters more than precision. Tree walking earns its extra second when queries name categories, when topics overlap, or when the path is part of the answer.