> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Filter

> Filter documents using LLM-based content evaluation and criteria matching

<Frame>
  <img src="https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/llm-filter.svg?fit=max&auto=format&n=TwtTrae3Fi3EFJ72&q=85&s=a25e99aac2e2ead150a2ab912d8f6109" alt="LLM Filter stage showing content-based filtering with language models" width="1000" height="400" data-path="assets/retrievers/llm-filter.svg" />
</Frame>

The LLM Filter stage uses language models to evaluate document content against specified criteria, filtering based on semantic understanding rather than metadata fields.

<Note>
  **Stage Category**: FILTER (Reduces document set)

  **Transformation**: N documents → M documents (where M ≤ N, based on LLM evaluation)
</Note>

## When to Use

| Use Case                      | Description                              |
| ----------------------------- | ---------------------------------------- |
| **Content quality filtering** | Remove low-quality or irrelevant content |
| **Semantic criteria**         | Filter by meaning, not just keywords     |
| **Complex requirements**      | "Only technical documentation"           |
| **Subjective evaluation**     | Tone, style, or sentiment filtering      |

## When NOT to Use

| Scenario                  | Recommended Alternative     |
| ------------------------- | --------------------------- |
| Simple metadata filtering | `attribute_filter` (faster) |
| Large result sets (100+)  | Too slow, pre-filter first  |
| Deterministic rules       | `attribute_filter`          |
| Low latency requirements  | Use metadata filters        |

## Parameters

| Parameter       | Type    | Default    | Description                      |
| --------------- | ------- | ---------- | -------------------------------- |
| `model`         | string  | *Required* | LLM model to use                 |
| `criteria`      | string  | *Required* | Natural language filter criteria |
| `content_field` | string  | `content`  | Field to evaluate                |
| `explanation`   | boolean | `false`    | Include filtering explanation    |
| `batch_size`    | integer | `10`       | Documents per LLM call           |

## Available Models

| Model             | Speed  | Quality   | Cost   |
| ----------------- | ------ | --------- | ------ |
| `gpt-4o-mini`     | Fast   | Good      | Low    |
| `gpt-4o`          | Medium | Excellent | Medium |
| `claude-3-haiku`  | Fast   | Good      | Low    |
| `claude-3-sonnet` | Medium | Excellent | Medium |

## Configuration Examples

<CodeGroup>
  ```json Basic Content Filter theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Keep only documents that contain technical information about software development"
      }
    }
  }
  ```

  ```json Quality Filter theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Filter out documents that are: incomplete, contain mostly ads/spam, or are not in English",
        "explanation": true
      }
    }
  }
  ```

  ```json Topic Relevance theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "anthropic",
        "model_name": "claude-3-haiku",
        "criteria": "Keep documents specifically about {{INPUT.topic}}. Exclude tangentially related or off-topic content."
      }
    }
  }
  ```

  ```json Professional Tone Filter theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Include only documents with professional, formal tone suitable for business communication. Exclude casual, informal, or inappropriate content.",
        "content_field": "content"
      }
    }
  }
  ```

  ```json Factual Content Only theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "criteria": "Keep only factual, informative content. Remove: opinions without evidence, speculation, promotional content, and entertainment-focused material.",
        "explanation": true
      }
    }
  }
  ```
</CodeGroup>

## Writing Effective Criteria

### Good Criteria Examples

```
✓ "Keep documents about machine learning algorithms and their implementations"
✓ "Filter out content that is primarily promotional or marketing material"
✓ "Include only documents written in the last 5 years about cloud computing"
✓ "Keep technical documentation; remove blog posts and news articles"
```

### Poor Criteria Examples

```
✗ "Good documents" (too vague)
✗ "Relevant content" (not specific)
✗ "High quality" (subjective without definition)
```

<Tip>
  Be specific about what to include AND exclude. The LLM makes a binary keep/discard decision for each document.
</Tip>

## Output Schema

### Without Explanation

Documents that pass the filter are returned unchanged:

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Technical documentation about...",
  "metadata": {...}
}
```

### With Explanation

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Technical documentation about...",
  "metadata": {...},
  "llm_filter": {
    "passed": true,
    "explanation": "Document contains detailed technical information about API implementation."
  }
}
```

### Filtered Out (not in results)

Documents that don't match criteria are removed from the result set.

## Performance

| Metric          | Value                          |
| --------------- | ------------------------------ |
| **Latency**     | 200-500ms per batch            |
| **Batch size**  | 10 documents default           |
| **Token usage** | \~100 tokens per document      |
| **Parallel**    | Batches processed concurrently |

<Warning>
  LLM filtering is expensive and slow. Always apply `attribute_filter` or use search `top_k` limits to reduce the document set before LLM filtering.
</Warning>

## Common Pipeline Patterns

### Search + Metadata Filter + LLM Filter

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "field": "metadata.type",
          "operator": "eq",
          "value": "documentation"
        }
      }
    }
  },
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Keep only documents that provide actionable, step-by-step instructions"
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 5
      }
    }
  }
]
```

### Quality + Relevance Pipeline

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 30 }
        ],
        "final_top_k": 30
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 15
      }
    }
  },
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Keep only high-quality, authoritative sources. Remove: user-generated content without verification, outdated information (pre-2020), and incomplete documents."
      }
    }
  }
]
```

## Cost Optimization

| Strategy                 | Impact                      |
| ------------------------ | --------------------------- |
| Pre-filter with metadata | Reduce documents before LLM |
| Use cheaper models       | `gpt-4o-mini` vs `gpt-4o`   |
| Increase batch size      | Fewer API calls             |
| Limit input documents    | Use `top_k` in search       |

## Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys. This gives you control over costs, rate limits, and API usage.

### Why Use BYOK?

| Benefit          | Description                                   |
| ---------------- | --------------------------------------------- |
| **Cost Control** | Use your own LLM provider account and billing |
| **Rate Limits**  | Use your own rate limits instead of shared    |
| **Compliance**   | Keep API calls under your own account         |
| **Key Rotation** | Rotate keys without changing retrievers       |

### Setup

<Steps>
  <Step title="Store your API key as a secret">
    Store your LLM provider API key in the organization secrets vault:

    ```bash theme={null}
    curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
      -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "secret_name": "openai_api_key",
        "secret_value": "sk-proj-abc123..."
      }'
    ```
  </Step>

  <Step title="Reference the secret in your stage">
    Use the `api_key` parameter with template syntax:

    ```json theme={null}
    {
      "stage_name": "llm_filter",
      "stage_type": "filter",
      "config": {
        "stage_id": "llm_filter",
        "parameters": {
          "provider": "openai",
          "model_name": "gpt-4o-mini",
          "criteria": "Keep only technical documentation",
          "api_key": "{{secrets.openai_api_key}}"
        }
      }
    }
    ```
  </Step>
</Steps>

### BYOK Configuration Example

<CodeGroup>
  ```json OpenAI BYOK theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Keep only high-quality, professional content",
        "api_key": "{{secrets.openai_api_key}}"
      }
    }
  }
  ```

  ```json Anthropic BYOK theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "anthropic",
        "provider": "anthropic",
        "model_name": "claude-3-haiku-20240307",
        "criteria": "Filter out promotional or marketing content",
        "api_key": "{{secrets.anthropic_api_key}}"
      }
    }
  }
  ```

  ```json Google BYOK theme={null}
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "google",
        "provider": "google",
        "model_name": "gemini-3.1-flash-lite",
        "criteria": "Keep only factual, informative documents",
        "api_key": "{{secrets.google_api_key}}"
      }
    }
  }
  ```
</CodeGroup>

### Supported Providers

| Provider  | Secret Name Example | Models                                         |
| --------- | ------------------- | ---------------------------------------------- |
| OpenAI    | `openai_api_key`    | gpt-4o, gpt-4o-mini                            |
| Anthropic | `anthropic_api_key` | claude-3-haiku, claude-3-sonnet, claude-3-opus |
| Google    | `google_api_key`    | gemini-3.1-flash-lite, gemini-2.5-pro          |

<Note>
  When `api_key` is not specified, the stage uses Mixpeek's default API keys and usage is charged to your Mixpeek account.
</Note>

## Custom Filter Model (BYO Plugin)

Use your own filter model deployed as a [custom extractor](/docs/processing/custom-extractors) instead of a hosted LLM provider. Set `feature_uri` to route filtering through your extractor's inference endpoint.

### Parameters

| Parameter     | Type   | Default | Description                                                                   |
| ------------- | ------ | ------- | ----------------------------------------------------------------------------- |
| `feature_uri` | string | `null`  | Feature URI of a custom filter plugin. Overrides `provider`/`model` when set. |

Your plugin must accept `{prompt: str, document: dict}` and return `{keep: bool, reason: str}`.

### Configuration Example

```json theme={null}
{
  "stage_name": "my_filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "feature_uri": "mixpeek://my_filter_model@1.0.0/filter",
      "criteria": "Keep only documents relevant to the query"
    }
  }
}
```

<Tip>
  Set `inference_type: "generate"` in your plugin's manifest to declare compatibility with LLM stages.
</Tip>

## Error Handling

| Error           | Behavior                         |
| --------------- | -------------------------------- |
| LLM timeout     | Retry once, then fail            |
| Rate limit      | Automatic backoff                |
| Invalid model   | Stage fails                      |
| Empty criteria  | All documents pass               |
| Invalid API key | Error returned with auth failure |

## Related

* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Metadata-based filtering
* [Rerank](/docs/retrieval/stages/rerank) - Relevance-based ordering
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - Extract data with LLM