The Short Answer
Use a decision model for any pipeline step whose output is a choice from a set you can list: keep or discard a document, pick a category, rate relevance on a scale, choose which branch of a hierarchy to search. Summaries, fresh cluster names and answers are new text, and they stay with a generative LLM.
A decision model such as TypeSafe's Jev takes the text and a typed question and returns a probability for each option. That makes it cheaper and more predictable for the decision steps, and the probability is useful on its own: it tells you which answers to trust and which to send for review.
Spotting a decision step
Look at the output schema of each step. If each field is a boolean, an enum, a number from 0 to 1, or an integer with a small range, the step is a decision.
| Step | Output | Decision model fits |
| Relevance filter | keep: boolean | Yes |
| Classification at ingestion | category: enum, is_ad: boolean | Yes |
| Reranking | relevance probability per document | Yes |
| Cluster labeling from a vocabulary | label: enum | Yes |
| Routing a query through a cluster tree | branch: enum at each level | Yes |
| Cluster naming from scratch | label: free text | No |
| Summaries, answers, descriptions | free text | No |
Use the probability
A decision model answers
keep with a number such as 0.98 or 0.51. That number
lets you set a threshold per use case, send
the uncertain middle to a person or to a larger model, and notice when a batch of
documents is more ambiguous than usual. In a reranker the probability is the score itself.In our own tests, a filter for "product complaints" kept a complaint at 0.98 and dropped a recipe at 0.01. In production, a reranker asked for "food or a food gift" put four food products at the top of a 30-document candidate list, each at 0.98, in 377 ms, and a classifier over eight product categories got six of six right.
Phrasing the question
Decision models read the question word for word. Write the exact condition:
Cost
TypeSafe prices Jev at $0.042 per million input tokens, with output free. A filter decision over a short document is a few hundred tokens, so a million decisions costs a few dollars. Jev reads text only, so for video, images and audio it judges the transcript, captions and metadata.
In Mixpeek
Mixpeek offers Jev at five decision points:
llm_provider: "typesafe"
in the text extractor, model_name: "jev-latest" in the LLM Filter and LLM Enrich
stages, inference_name: "typesafe__jev" in the Rerank stage, fixed-vocabulary
cluster labeling through candidate_labels, and the cluster_navigation strategy
for walking a cluster hierarchy.