# PostgreSQL
Source: https://docs.mixpeek.com/docs/integrations/databases/postgresql

Sync PostgreSQL table rows into a Mixpeek bucket and run SQL lookups from a retriever.

> [!NOTE]
>
> Each row becomes one bucket object. The same connection also backs the `sql_lookup` retriever stage.

## Overview

A PostgreSQL connection serves two paths:

* **Sync.** Rows from a table land in a bucket as JSON objects, with incremental sync via a watermark column.
* **Lookup.** The [`sql_lookup`](/docs/retrieval/stages/sql-lookup) retriever stage enriches documents mid-pipeline.

## Prerequisites

* PostgreSQL 12 or later.
* A role with `CONNECT` on the database and `USAGE` on the schema.
* `SELECT` on every table you sync or query.
* Network access from Mixpeek to the server.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `credentials.username` | Yes | PostgreSQL username |
| `credentials.password` | Yes | Encrypted at rest, never returned in responses |
| `host` | Yes | Hostname or IP |
| `database` | Yes | Database name |
| `port` | No | Defaults to `5432` |
| `schema` | No | Defaults to `public` |
| `ssl_mode` | No | `disable`, `allow`, `prefer` (default), `require`, `verify-ca`, `verify-full` |
| `incremental_column` | No | `TIMESTAMP` or `DATE` column used as the sync watermark |
| `primary_key_columns` | No | Columns that form the key, used to build stable object IDs |
| `query_timeout_seconds` | No | Defaults to `300` |
| `fetch_size` | No | Rows per batch, defaults to `1000` |

### Sync-level fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | `table`, or `schema.table` to override the connection schema |
| `polling_interval_seconds` | No | Defaults to `300` |

> [!WARNING]
>
> `source_path` accepts one or two dot-separated parts. Three or more returns a validation error.

## Setup

**1. Create a read-only role**

```sql theme={null}
CREATE ROLE mixpeek_sync LOGIN PASSWORD 'choose-a-strong-password';
GRANT CONNECT ON DATABASE production TO mixpeek_sync;
GRANT USAGE ON SCHEMA public TO mixpeek_sync;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO mixpeek_sync;
```

**2. Create the storage connection**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Production Postgres",
    provider_type="postgresql",
    provider_config={
        "credentials": {
            "type": "username_password",
            "username": "mixpeek_sync",
            "password": "choose-a-strong-password",
        },
        "host": "db.example.com",
        "port": 5432,
        "database": "production",
        "schema": "public",
        "ssl_mode": "require",
        "incremental_column": "updated_at",
        "primary_key_columns": ["id"],
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Production Postgres',
  provider_type: 'postgresql',
  provider_config: {
    credentials: {
      type: 'username_password',
      username: 'mixpeek_sync',
      password: 'choose-a-strong-password',
    },
    host: 'db.example.com',
    port: 5432,
    database: 'production',
    schema: 'public',
    ssl_mode: 'require',
    incremental_column: 'updated_at',
    primary_key_columns: ['id'],
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Postgres",
    "provider_type": "postgresql",
    "provider_config": {
      "credentials": {
        "type": "username_password",
        "username": "mixpeek_sync",
        "password": "choose-a-strong-password"
      },
      "host": "db.example.com",
      "port": 5432,
      "database": "production",
      "schema": "public",
      "ssl_mode": "require",
      "incremental_column": "updated_at",
      "primary_key_columns": ["id"]
    }
  }'
```

**3. Sync a table**

```python Python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="customers",
    polling_interval_seconds=1800,
)
print(f"Sync created: {sync['sync_config_id']}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "analytics.customers",
    "polling_interval_seconds": 1800
  }'
```

## Incremental sync

Set `incremental_column` to a `TIMESTAMP` or `DATE` column. Mixpeek records the highest value it has read and filters the next run to rows above it.

Set `primary_key_columns` so object IDs stay stable across runs. Without it, a re-read can create a second object for the same row.

## Bulk import and change capture

For a large table, or one with an embedding column, set `provider_filters.import` on the sync. The table is read in primary-key slices, each slice lands in the bucket as one Parquet object, and every row becomes one document in the collection. Each document's ID derives from the source table and primary key, so a re-run, a resumed import or a later change updates the same document.

```json theme={null}
{
  "connection_id": "conn_your_connection_id",
  "source_path": "public.scenes",
  "sync_mode": "continuous",
  "polling_interval_seconds": 60,
  "batch_collection_ids": ["col_your_collection_id"],
  "provider_filters": {
    "import": {
      "mode": "slices",
      "slice_rows": 20000,
      "objects_per_batch": 4,
      "text_column": "caption",
      "vector_columns": {
        "embedding": {"model": "text-embedding-3-large", "dimensions": 3072, "mode": "carry"}
      },
      "replication": "logical",
      "publication": "mixpeek_pub"
    }
  }
}
```

| Field | Description |
| - | - |
| `mode` | `slices` for the bulk import. `rows` keeps one JSON object per row. |
| `slice_rows` | Rows per slice. Defaults to `20000`. |
| `objects_per_batch` | Slices per processing batch. Defaults to `4`. |
| `text_column` | The column the collection's extractor reads as text. |
| `vector_columns` | One entry per `vector` or `halfvec` column: the model that produced it, its dimension, and `carry` (store the vector as the document's vector, with no embedding call) or `reextract` (drop it and embed `text_column`). |
| `replication` | `logical`, `polling` or `none`. See below. |
| `publication` | The publication the logical slot reads. Required for `logical`. |
| `slot_lag_alert_bytes` | Retained WAL that raises the slot-lag alert. Defaults to 1 GiB. |
| `slot_drop_after_pause_seconds` | A sync paused this long has its slot dropped. Defaults to 24 hours. |

Customer columns keep their names in the document. Mixpeek's own fields, including the source row's lineage (schema, table, primary key and vector origin), live under `_internal`.

A carried vector is refused at sync create when its model or dimension differs from the collection's feature, or when the model has no registered query-time model, because its documents could not be searched by text. Tables without a vector column can import into a passthrough collection.

### Change capture

With `replication: logical`, the import creates a logical replication slot first and copies every slice inside the slot's exported snapshot. After the copy, each sync run streams the changes committed since, writes them as one change object, and confirms the slot position only after that object's documents are written. Inserts, updates and deletes all reach the collection.

Logical replication needs:

* `wal_level = logical` on the server.
* A role with the `REPLICATION` attribute, connecting directly to Postgres. Transaction pooling (PgBouncer in transaction mode) cannot carry a replication connection.
* A publication that includes the table, created by the table owner:

```sql theme={null}
ALTER ROLE mixpeek_sync REPLICATION;
CREATE PUBLICATION mixpeek_pub FOR TABLE public.scenes;
```

A slot keeps WAL on your server until Mixpeek confirms it. Set `max_slot_wal_keep_size` so a stalled consumer cannot fill the disk, and watch the slot-lag alert on the sync record. Deleting the sync drops its slot.

`replication: polling` reads rows whose `updated_at` (or `updated_at_column`) moved past the last value seen. It needs no replication role, and it misses hard deletes: a row deleted at the source stays in the collection.

Every session that reads rows runs with `default_transaction_read_only = on`, so the connection cannot write to your database. The replication slot is the only object the import creates on the source.

## Related

> **[SQL Lookup stage](/docs/retrieval/stages/sql-lookup)**
>
> Enrich documents from a SQL source inside a retriever.

> **[Snowflake](/docs/integrations/snowflake-warehouse)**
>
> Warehouse connection with the same two paths.


# Mixpeek + Databricks
Source: https://docs.mixpeek.com/docs/integrations/databricks-warehouse

Multimodal data warehouse meets data lakehouse -- complementary layers for the modern data stack

## Overview

Mixpeek and Databricks occupy different layers of the data stack. Mixpeek ingests unstructured multimodal files and extracts structured features, embeddings, and classifications. Databricks provides the lakehouse platform -- Delta Lake for storage, Unity Catalog for governance, and integrated ML for training and serving models. Together, they give you a complete path from raw media files to governed, analytics-ready data.

> **Mixpeek**
>
> Ingests unstructured files, extracts features (embeddings, transcripts, classifications, metadata), and powers multimodal retrieval.

> **Databricks**
>
> Stores structured outputs in Delta tables, enforces governance via Unity Catalog, and runs ML training and serving at scale.

## Architecture

```
                        Mixpeek                              Databricks
               +-----------------------+            +------------------------+
               |                       |            |                        |
  Files -----> |  Buckets & Collections|            |   Delta Lake Tables    |
  (images,     |                       |            |                        |
   video,      |  Decompose files into |  export    |  - classifications     |
   audio,      |  features:            | ---------> |  - extracted metadata  |
   PDFs)       |   - embeddings        |            |  - taxonomy labels     |
               |   - transcripts       |            |  - document payloads   |
               |   - classifications   |            |                        |
               |   - metadata          |  enrich    |  Unity Catalog governs |
               |                       | <--------- |  all tables. Databricks|
               |  Retrieval & Search   |            |  ML retrains models.   |
               +-----------------------+            +------------------------+
```

## Use Cases

### Write extracted features as Delta tables

After Mixpeek processes your files, export the structured outputs -- transcripts, object detections, taxonomy labels, metadata -- as Delta tables. This makes them queryable with Spark SQL, joinable with your existing business data, and available to any tool in the Databricks ecosystem.

### Use Unity Catalog for governance

Unity Catalog provides fine-grained access control, lineage tracking, and audit logging for all data assets. Once Mixpeek outputs land in Delta tables, Unity Catalog governs who can access them and how they flow through your organization.

### Combine Mixpeek retrieval with Databricks ML

Use Mixpeek to power real-time multimodal search and retrieval. Feed the same structured features into Databricks ML for batch training -- fine-tune classifiers, build recommendation models, or run large-scale analytics on extracted content.

## Quick Start

Export Mixpeek document metadata to a Delta table using the Mixpeek Python SDK and the Databricks SQL Connector.

**1. Install dependencies**

```bash theme={null}
pip install mixpeek databricks-sql-connector
```

**2. List documents from Mixpeek**

```python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# List documents from a collection
documents = client.collections.documents.list(
    collection_id="your-collection-id",
    page_size=100
)
```

**3. Write to a Delta table via Databricks SQL**

```python theme={null}
from databricks import sql
import json

connection = sql.connect(
    server_hostname="YOUR_WORKSPACE.cloud.databricks.com",
    http_path="/sql/1.0/warehouses/YOUR_WAREHOUSE_ID",
    access_token="YOUR_ACCESS_TOKEN"
)

cursor = connection.cursor()

# Create table if it does not exist
cursor.execute("""
    CREATE TABLE IF NOT EXISTS mixpeek_catalog.default.documents (
        document_id STRING,
        source_url STRING,
        content_type STRING,
        metadata STRING,
        created_at TIMESTAMP
    )
""")

# Insert each document
for doc in documents:
    cursor.execute(
        """
        INSERT INTO mixpeek_catalog.default.documents
            (document_id, source_url, content_type, metadata, created_at)
        VALUES (%s, %s, %s, %s, %s)
        """,
        (
            doc.get("document_id"),
            doc.get("source", {}).get("url"),
            doc.get("content_type"),
            json.dumps(doc.get("metadata", {})),
            doc.get("created_at"),
        )
    )

connection.commit()
cursor.close()
connection.close()
```

> [!TIP]
>
> For production workloads, write Mixpeek outputs to cloud storage (S3 or ADLS) and use Databricks Auto Loader to incrementally ingest new files into Delta tables.

## When to Use Each

| Capability | Mixpeek | Databricks |
| - | - | - |
| Ingest unstructured files (video, images, audio, PDFs) | Yes | No |
| Extract features (embeddings, transcripts, classifications) | Yes | No |
| Multimodal semantic search | Yes | No |
| Structured SQL analytics | No | Yes (Spark SQL) |
| Data governance and lineage | Document-level ACL | Unity Catalog |
| ML model training and serving | No | Yes (MLflow, Model Serving) |
| Streaming ingestion | Webhooks + batch triggers | Structured Streaming, Auto Loader |

> [!NOTE]
>
> Mixpeek handles everything before the data is structured. Databricks handles everything after. Use both to bridge the gap between raw multimodal files and governed, ML-ready data.

## Related

* [Taxonomies](/docs/enrichment/taxonomies) -- classify content and export labels
* [SQL Lookup Stage](/docs/retrieval/stages/sql-lookup) -- query external databases from retriever pipelines
* [API Call Stage](/docs/retrieval/stages/api-call) -- call external APIs during retrieval
* [Webhooks](/docs/operations/webhooks) -- trigger Databricks jobs when Mixpeek processing completes


# Email
Source: https://docs.mixpeek.com/docs/integrations/email

Receive emails as bucket objects with a dedicated inbound address for document intake

## Overview

The Email connector lets you ingest documents by forwarding emails to a dedicated address. Each email becomes a bucket object with the body as a text blob, each attachment as a typed blob, and the original `.eml` preserved for chain of custody.

This is built for compliance-oriented workflows — legal document intake, healthcare record forwarding, secure support inboxes — where email is the transport and the documents (attachments) are the payload.

## Prerequisites

* A Mixpeek account with an active namespace
* A bucket and sync configured for the email connection

## How It Works

1. **Create an email connection** — Mixpeek assigns a unique inbound address (e.g., `conn_abc123@inbound.mixpeek.com`)
2. **Cloudflare receives the email** — MX records point to Cloudflare Email Routing, which routes to a Worker
3. **Worker POSTs raw .eml** — The Cloudflare Email Worker reads the raw RFC 2822 bytes and POSTs them to the Mixpeek webhook
4. **Mixpeek parses and stores** — MIME parsing extracts headers → metadata, body → text blob, attachments → S3-backed blobs, raw .eml → S3

```
Customer Email Client
        │
        ▼
  Cloudflare MX (inbound.mixpeek.com)
        │
        ▼
  Cloudflare Email Routing (catch-all)
        │
        ▼
  mixpeek-email-ingest Worker
        │  POST raw .eml bytes
        ▼
  api.mixpeek.com/v1/webhooks/email/{connection_id}
        │  MIME parse → S3 upload → object + blobs
        ▼
  Bucket Objects
        │  blobs in object storage, metadata in Mongo
        ▼
  mvs_bucket_<bucket_id>
     one document per object, searchable and alertable
```

The last step happens the first time anything names the bucket as a collection.
See [Bucket-backed collections](/docs/vector-store/system-collections#bucket-backed-collections).

## Configuration

### Connection-level fields

| Field | Required | Default | Description |
| - | - | - | - |
| `allowed_senders` | No | `[]` (all) | Sender allowlist. Exact addresses or domain wildcards (`*@company.com`). Empty = accept all. |
| `store_raw_eml` | No | `true` | Store the original `.eml` file as an additional blob for chain of custody. |

### Auto-provisioned fields (read-only)

| Field | Description |
| - | - |
| `inbound_address` | System-assigned email address for this connection (e.g., `conn_abc123@inbound.mixpeek.com`) |
| `webhook_secret` | Auto-generated HMAC-SHA256 signing secret for webhook verification |

## Setup

**1. Create an email connection**

```python Python theme={null}
from mixpeek import Mixpeek

mp = Mixpeek("your_api_key")

connection = mp.connections.create(
    name="Legal Intake Inbox",
    provider_type="email",
    provider_config={
        "credentials": {"type": "webhook_secret"},
        "allowed_senders": ["*@lawfirm.com", "paralegal@partner.com"],
        "store_raw_eml": True,
    },
)

print(f"Inbound address: {connection.provider_config['inbound_address']}")
print(f"Webhook URL: https://api.mixpeek.com/v1/webhooks/email/{connection.connection_id}")
```

```javascript JavaScript theme={null}
const { Mixpeek } = require("mixpeek");
const mp = new Mixpeek("your_api_key");

const connection = await mp.connections.create({
  name: "Legal Intake Inbox",
  provider_type: "email",
  provider_config: {
    credentials: { type: "webhook_secret" },
    allowed_senders: ["*@lawfirm.com", "paralegal@partner.com"],
    store_raw_eml: true,
  },
});

console.log(`Inbound address: ${connection.provider_config.inbound_address}`);
console.log(`Webhook URL: https://api.mixpeek.com/v1/webhooks/email/${connection.connection_id}`);
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Legal Intake Inbox",
    "provider_type": "email",
    "provider_config": {
      "credentials": {"type": "webhook_secret"},
      "allowed_senders": ["*@lawfirm.com", "paralegal@partner.com"],
      "store_raw_eml": true
    }
  }'
```

**2. Create a bucket sync**

Link the email connection to a bucket so incoming emails become objects:

```python Python theme={null}
sync = mp.buckets.syncs.create(
    bucket_id="your_bucket_id",
    connection_id=connection.connection_id,
    source_path="inbox",
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/your_bucket_id/syncs \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "source_path": "inbox"
  }'
```

**3. Send a test email**

Send an email with an attachment to the inbound address and verify the object appears in your bucket.

## Object Structure

Each email becomes **one bucket object** with multiple blobs:

| Blob Property | Type | Content |
| - | - | - |
| `email_body` | `text` | Email body (plain text preferred, HTML fallback) |
| `attachment_0`, `attachment_1`, ... | varies | Each attachment, typed by MIME (image, pdf, video, etc.) |
| `raw_eml` | `text` | Original `.eml` file stored in S3 (if `store_raw_eml` is enabled) |

### Email metadata fields

These are set as **root-level fields** on the object and can be mapped to your bucket schema:

| Field | Type | Description |
| - | - | - |
| `email_from` | string | Sender address |
| `email_to` | list\[string] | Recipient addresses |
| `email_cc` | list\[string] | CC addresses |
| `email_subject` | string | Subject line |
| `email_date` | string (ISO 8601) | Date the email was sent |
| `email_message_id` | string | RFC 2822 Message-ID (used for deduplication) |
| `email_in_reply_to` | string | Parent message ID (for threading) |
| `email_references` | list\[string] | Thread reference IDs |
| `email_attachment_count` | integer | Number of attachments |

## Schema Mapping

Map email fields to your collection schema to make them searchable:

```json theme={null}
{
  "schema": {
    "fields": [
      {"name": "email_from", "type": "text"},
      {"name": "email_subject", "type": "text"},
      {"name": "email_date", "type": "text"},
      {"name": "email_attachment_count", "type": "text"}
    ]
  }
}
```

Use `attribute_filter` in your retriever to query by sender, date, or subject:

```json theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "AND": [
          {"field": "email_from", "operator": "contains", "value": "@lawfirm.com"},
          {"field": "email_date", "operator": "gte", "value": "2026-01-01"}
        ]
      }
    }
  }
}
```

## Security

| Feature | Description |
| - | - |
| **Sender allowlist** | Only accept emails from specified addresses or domains |
| **Webhook signature** | HMAC-SHA256 verification of inbound payloads |
| **Deduplication** | Duplicate emails (same Message-ID) are skipped |
| **Chain of custody** | Raw `.eml` uploaded to S3 with SHA-256 hash for forensic integrity |
| **Credential encryption** | Webhook secret encrypted at rest (Fernet / CSFLE) |
| **Audit logging** | All connection events written to the audit log (400-day retention) |

> [!WARNING]
>
> Email headers and bodies may contain PII (names, email addresses, phone numbers). Consider enabling PII redaction in your collection pipeline or restricting access to the namespace containing email data.

## Compliance Notes

| Requirement | How Mixpeek addresses it |
| - | - |
| **HIPAA — encryption in transit** | Cloudflare enforces TLS on MX; webhook endpoint requires HTTPS (TLS 1.2+) |
| **HIPAA — encryption at rest** | Credentials encrypted via CSFLE; all blobs (body, attachments, raw .eml) stored in encrypted S3 |
| **HIPAA — audit trail** | All access written to the audit log |
| **eDiscovery — immutability** | Raw `.eml` in S3 with SHA-256 hash, stored alongside parsed content |
| **eDiscovery — chain of custody** | Source tracking: `source_provider=email`, `source_object_id=email://{message_id}` |
| **SOC 2 — access control** | Per-namespace RBAC (ADMIN, MEMBER, VIEWER) with granular operations |

> [!NOTE]
>
> Mixpeek does not currently hold a HIPAA BAA. If you need a BAA for PHI handling, contact us at [sales@mixpeek.com](mailto:sales@mixpeek.com) to discuss your requirements.

## Troubleshooting

| Issue | Solution |
| - | - |
| **403 — sender not in allowlist** | Add the sender's address or domain to `allowed_senders` |
| **404 — connection not found** | Verify the `connection_id` in the webhook URL matches an active email connection |
| **400 — no active bucket sync** | Create a bucket sync linked to this email connection |
| **Duplicate emails skipped** | Expected behavior — emails with the same Message-ID are deduplicated |
| **Attachments not appearing** | Check that the email service is sending the full raw RFC 2822 message, not a stripped-down version |

## Related

* [Buckets](/docs/ingestion/buckets) — Bucket schemas and objects
* [Create Sync Configuration](/docs/api-reference/bucket-syncs/create-sync-configuration) — Link connections to buckets
* [Attribute Filter](/docs/retrieval/stages/attribute-filter) — Filter by email metadata


# Azure Blob Storage
Source: https://docs.mixpeek.com/docs/integrations/object-storage/azure-blob

Connect Mixpeek to your Azure Blob Storage containers to ingest and process your data.

> [!NOTE]
>
> This guide explains how to connect your Azure Blob Storage containers to Mixpeek, enabling automated data ingestion and processing.

## Prerequisites

* An active Azure subscription.
* A Storage Account with Blob Storage containers containing the data you want Mixpeek to process.
* Permissions to create and manage storage account access keys or managed identities.

## Configuration Steps

Connecting Mixpeek to Azure Blob Storage requires granting Mixpeek read access to your containers. We recommend using a managed identity or storage account access key for authentication.

**1. Configure Storage Account Access**

First, you need to configure access to your Azure Blob Storage containers. Choose one of the following methods:

**Option A: Storage Account Access Key (Simpler Setup)**

1. Navigate to your Storage Account in the Azure Portal.
2. Go to **Access keys** under **Security + networking**.
3. Copy either the **key1** or **key2** value. You'll need this to configure the connection in Mixpeek.
4. Store the key securely.

**Option B: Managed Identity (Recommended for Security)**

1. Navigate to your Storage Account in the Azure Portal.
2. Go to **Access Control (IAM)**.
3. Click **Add** > **Add role assignment**.
4. Select the **Storage Blob Data Reader** role.
5. Assign access to the managed identity that Mixpeek will use.

**2. Configure Container Access**

Ensure your container has the appropriate access level:

1. Navigate to your container in the Azure Portal.
2. Go to **Change access level**.
3. Set the access level to either:
   * **Private (no anonymous access)** - Use this if you're using access keys or managed identity
   * **Blob (anonymous read access for blobs only)** - Use this if you want to allow public read access
4. Click **OK** to save the changes.

**3. Add Azure Blob Storage Connection in Mixpeek**

In Studio, go to **Settings > Connections > New Connection** and pick **Azure Blob Storage**.

1. Choose your authentication method, account key or managed identity.
2. Enter the **storage account name** and the **container** name.
3. Optionally set a **prefix** to scope the sync to one folder.
4. Save. Mixpeek tests the credentials before storing them.

To create the same connection over the API, `POST /v1/organizations/connections`. The
`credentials.type` field selects the method:

```bash Account key theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer mxp_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "azure-media",
    "provider_type": "azure_blob",
    "provider_config": {
      "provider_type": "azure_blob",
      "account_name": "YOUR_STORAGE_ACCOUNT",
      "container": "YOUR_CONTAINER",
      "prefix": "media/",
      "credentials": {
        "type": "account_key",
        "account_name": "YOUR_STORAGE_ACCOUNT",
        "account_key": "YOUR_ACCESS_KEY"
      }
    },
    "test_before_save": true
  }'
```

```bash Managed identity theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer mxp_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "azure-media",
    "provider_type": "azure_blob",
    "provider_config": {
      "provider_type": "azure_blob",
      "account_name": "YOUR_STORAGE_ACCOUNT",
      "container": "YOUR_CONTAINER",
      "credentials": {
        "type": "managed_identity",
        "client_id": "OPTIONAL_USER_ASSIGNED_IDENTITY_CLIENT_ID"
      }
    },
    "test_before_save": true
  }'
```

```python Python theme={null}
import requests

requests.post(
    "https://api.mixpeek.com/v1/organizations/connections",
    headers={"Authorization": "Bearer mxp_sk_..."},
    json={
        "name": "azure-media",
        "provider_type": "azure_blob",
        "provider_config": {
            "provider_type": "azure_blob",
            "account_name": "YOUR_STORAGE_ACCOUNT",
            "container": "YOUR_CONTAINER",
            "prefix": "media/",
            "credentials": {
                "type": "account_key",
                "account_name": "YOUR_STORAGE_ACCOUNT",
                "account_key": "YOUR_ACCESS_KEY",
            },
        },
        "test_before_save": True,
    },
)
```

> [!NOTE]
>
> The provider type is `azure_blob`. A bare `azure` returns HTTP 422.
>
> Managed identity uses `DefaultAzureCredential`. Set `client_id` only when you target a
> specific user-assigned identity; leave it out for the system-assigned one. `account_key`
> is redacted when you read the connection back.

## Verification

Once connected, Mixpeek should start discovering files in your specified Azure Blob container (and prefix, if provided). You can monitor the ingestion status within the Mixpeek Studio. Depending on your pipeline configuration, feature extraction and indexing will begin automatically for supported file types.

If you encounter issues, double-check the access permissions and the credentials provided in Mixpeek. Ensure the storage account name, container name, and authentication details are correct.


# Backblaze B2
Source: https://docs.mixpeek.com/docs/integrations/object-storage/backblaze

Sync files from Backblaze B2 Cloud Storage into Mixpeek buckets using the S3-compatible API.

> [!NOTE]
>
> Backblaze B2 is an S3-compatible object storage service. Mixpeek auto-discovers your regional endpoint — you only need your application key ID and application key.

## Overview

The Backblaze B2 integration lets Mixpeek sync objects from any B2 bucket into your Mixpeek buckets for processing. Each file becomes a bucket object with its metadata, ready for feature extraction in your collections.

Mixpeek uses Backblaze's S3-compatible API via the standard AWS SDK. On connection creation, it calls the B2 authorization API to discover your account's correct regional endpoint (e.g., `s3.us-east-005.backblazeb2.com`) automatically.

## Prerequisites

* An active [Backblaze](https://www.backblaze.com/b2/cloud-storage.html) account with B2 Cloud Storage enabled.
* An application key with at minimum: `listBuckets`, `listFiles`, and `readFiles` capabilities.
* The key ID and application key (shown once at creation — copy it immediately).

## Configuration

### Connection-Level Fields

| Field | Required | Description |
| - | - | - |
| `key_id` | Yes | Backblaze B2 application key ID |
| `application_key` | Yes | Backblaze B2 application key (secret) — encrypted at rest |

### Sync-Level Fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | `b2://bucket-name/optional/prefix` or `bucket-name/prefix` |
| `sync_mode` | No | `continuous` (default) or `initial_only` |
| `polling_interval_seconds` | No | Seconds between polls in `continuous` mode |
| `include_patterns` | No | Glob patterns to include (e.g., `["*.mp4", "*.jpg"]`) |
| `exclude_patterns` | No | Glob patterns to exclude (e.g., `["*.tmp"]`) |
| `modified_since` | No | Only sync files modified after this ISO 8601 timestamp |

## Setup

**1. Create a Backblaze application key**

1. Log in to the [Backblaze Console](https://secure.backblaze.com/b2_buckets.htm).
2. Go to **App Keys** in the left sidebar.
3. Click **Add a New Application Key**.
4. Set a name (e.g., `mixpeek-sync`) and choose the buckets to grant access to.
5. Enable: **Read and Write Files**, **List Buckets**, **List Files**, **Read Files**.
6. Click **Create New Key** and **copy both the keyID and applicationKey immediately** — the key is shown only once.

> [!WARNING]
>
> The application key is shown only once at creation. If you lose it, you must create a new key.

**2. Create the storage connection in Mixpeek**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Backblaze B2 Production",
    provider_type="backblaze",
    provider_config={
        "credentials": {
            "type": "application_key",
            "key_id": "005870eff85b6c60000000001",
            "application_key": "K005...",
        },
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Backblaze B2 Production',
  provider_type: 'backblaze',
  provider_config: {
    credentials: {
      type: 'application_key',
      key_id: '005870eff85b6c60000000001',
      application_key: 'K005...',
    },
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Backblaze B2 Production",
    "provider_type": "backblaze",
    "provider_config": {
      "credentials": {
        "type": "application_key",
        "key_id": "005870eff85b6c60000000001",
        "application_key": "K005..."
      }
    }
  }'
```

**3. Create a sync configuration on your bucket**

```python Python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="b2://my-backblaze-bucket/videos/",
    sync_mode="continuous",
    polling_interval_seconds=3600,  # Hourly
)
print(f"Sync created: {sync['sync_config_id']}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "b2://my-backblaze-bucket/videos/",
    "sync_mode": "continuous",
    "polling_interval_seconds": 3600
  }'
```

**4. Trigger your first sync**

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs/SYNC_CONFIG_ID/trigger \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id"
```

## Advanced Configuration

### Scoped Key (Recommended)

For production, create a key scoped to a specific bucket:

1. In Backblaze Console → App Keys → Add New Key
2. Under **Allow access to Bucket(s)**, select a single bucket
3. This limits the key's blast radius if it's ever compromised

### File Filtering

Filter which files are synced using glob patterns:

```python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="b2://my-bucket/",
    sync_mode="continuous",
    polling_interval_seconds=86400,
    include_patterns=["*.mp4", "*.mov", "*.jpg"],
    exclude_patterns=["*_thumbnail.*", "*.tmp"],
)
```

### Incremental Sync

Only sync files added or modified after a specific date:

```python theme={null}
from datetime import datetime, timezone

sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="b2://my-bucket/",
    sync_mode="continuous",
    polling_interval_seconds=300,
    modified_since="2024-01-01T00:00:00Z",
)
```

## Source Path Format

The `source_path` supports multiple formats:

| Format | Example | Description |
| - | - | - |
| `b2://bucket/prefix` | `b2://my-videos/2024/` | Preferred — explicit B2 scheme |
| `s3://bucket/prefix` | `s3://my-videos/2024/` | S3-style URI also accepted |
| `bucket/prefix` | `my-videos/2024/` | Bare bucket + prefix |
| `bucket` | `my-videos` | Entire bucket |

## Sync Modes

| Mode | Description | When to Use |
| - | - | - |
| `continuous` | Polls every `polling_interval_seconds` | Real-time monitoring, active uploads. Set a longer interval for regular batch processing |
| `initial_only` | Single import, then completes | Historical backfills, migrations |

## Troubleshooting

> **Connection test returns 401 Unauthorized**
>
> Your key ID or application key is incorrect:
>
> 1. In the Backblaze Console → App Keys, verify the keyID column matches what you entered.
> 2. The application key is only shown once — if you lost it, create a new key.
> 3. Ensure the key has not expired or been deleted.

> **No files synced from a bucket that has objects**
>
> * Verify the `source_path` matches the correct bucket name (case-sensitive).
> * Check that your application key has **listFiles** and **readFiles** capabilities.
> * If the key is scoped to a specific bucket, ensure the bucket name in `source_path` matches exactly.
> * Check `include_patterns` — make sure they match your file extensions.

> **Connection test succeeds but sync returns Access Denied**
>
> Your application key may have **listBuckets** but not **listFiles** on a specific bucket:
>
> 1. Create a new key with **List Files** and **Read Files** enabled for the target bucket.
> 2. Update the connection with the new key.

> **Files in sub-folders are not being synced**
>
> Backblaze B2 uses flat key namespacing (like S3). Folders are just key prefixes.
>
> * Use `b2://bucket/` (trailing slash) to sync all files recursively.
> * Use `b2://bucket/subfolder/` to scope to a specific prefix.

## Related

* [Bucket Syncs](/docs/api-reference/bucket-syncs/create-sync-configuration)
* [Storage Connections](/docs/api-reference/organization-connections/create-storage-connection)
* [Amazon S3 Integration](/docs/integrations/object-storage/s3)
* [Tigris Integration](/docs/integrations/object-storage/tigris)


# Box
Source: https://docs.mixpeek.com/docs/integrations/object-storage/box

Connect Mixpeek to your Box account to sync and process files from Box folders.

> [!NOTE]
>
> This guide explains how to connect your Box storage to Mixpeek, enabling automated ingestion and processing of files from Box folders.

## Prerequisites

* An active Box account (Business or Enterprise plan recommended for CCG auth)
* A Box application created at [developer.box.com](https://developer.box.com)
* Client ID and Client Secret from your Box app

## Authentication Methods

Box supports two authentication methods:

| Method | Best For | Token Expiry |
| - | - | - |
| **OAuth (Developer Token)** | Development and testing | 60 minutes |
| **CCG (Client Credentials Grant)** | Production deployments | Auto-refreshes |

> [!WARNING]
>
> Developer tokens expire after 60 minutes and **cannot be refreshed**. Use CCG authentication for production workloads.

## Configuration Steps

**1. Create a Box Application**

1. Go to [developer.box.com](https://developer.box.com) and sign in
2. Click **My Apps** → **Create New App**
3. Select **Custom App** and choose your authentication method:
   * For testing: **User Authentication (OAuth 2.0)**
   * For production: **Server Authentication (Client Credentials Grant)**
4. Name your app and click **Create App**
5. In the app's **Configuration** tab, note down your **Client ID** and **Client Secret**

**2. Get a Developer Token (OAuth only)**

For quick testing with OAuth:

1. Open your app in the Box Developer Console
2. Go to the **Configuration** tab
3. Scroll down to **Developer Token**
4. Click **Generate Developer Token**
5. Copy the token — it's valid for 60 minutes

> [!WARNING]
>
> Generating a new developer token invalidates the previous one. Store it immediately.

**3. Configure CCG (Production)**

For production use with Client Credentials Grant:

1. In your Box app configuration, select **App Access Only** under **App Access Level**
2. Enable **Generate User Access Tokens** under **Advanced Features** (if needed)
3. In the Box Admin Console, approve the app:
   * Go to **Admin Console** → **Apps** → **Custom Apps Manager**
   * Authorize your CCG app
4. Note your **Enterprise ID** from **Admin Console** → **Enterprise Settings** → **Account Info**

**4. Add Box Connection in Mixpeek**

1. Navigate to **Settings** → **Connections** in Mixpeek Studio
2. Click **Add Connection** and select **Box**
3. Enter your connection details:
   * **Connection Name:** A descriptive name (e.g., "Production Box")
   * **Auth Method:** Choose OAuth or CCG
   * **Client ID:** From your Box app configuration
   * **Client Secret:** From your Box app configuration
   * For OAuth: **Access Token** (developer token)
   * For CCG: **Enterprise ID** (from Box Admin Console)
   * **Folder ID:** Box folder to sync (use `0` for root folder)
4. Click **Test Connection** to verify credentials
5. Click **Create Connection**

## API Configuration

You can also create a Box connection directly via the API:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# OAuth (developer token)
connection = client.organizations.connections.create(
    name="My Box Storage",
    provider_type="box",
    provider_config={
        "provider_type": "box",
        "credentials": {
            "type": "oauth",
            "client_id": "your-client-id",
            "client_secret": "your-client-secret",
            "access_token": "your-developer-token"
        },
        "folder_id": "0"  # Root folder
    }
)

# CCG (production)
connection = client.organizations.connections.create(
    name="My Box Storage (CCG)",
    provider_type="box",
    provider_config={
        "provider_type": "box",
        "credentials": {
            "type": "ccg",
            "client_id": "your-client-id",
            "client_secret": "your-client-secret",
            "enterprise_id": "your-enterprise-id"
        },
        "folder_id": "0"
    }
)
```

```bash cURL theme={null}
# OAuth (developer token)
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Box Storage",
    "provider_type": "box",
    "provider_config": {
      "provider_type": "box",
      "credentials": {
        "type": "oauth",
        "client_id": "your-client-id",
        "client_secret": "your-client-secret",
        "access_token": "your-developer-token"
      },
      "folder_id": "0"
    }
  }'

# CCG (production)
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Box Storage (CCG)",
    "provider_type": "box",
    "provider_config": {
      "provider_type": "box",
      "credentials": {
        "type": "ccg",
        "client_id": "your-client-id",
        "client_secret": "your-client-secret",
        "enterprise_id": "your-enterprise-id"
      },
      "folder_id": "0"
    }
  }'
```

## Setting Up a Sync

Once your connection is created, configure a sync to start ingesting files:

```python theme={null}
# Create a bucket sync from Box
sync = client.buckets.syncs.create(
    bucket_id="your-bucket-id",
    connection_id="your-connection-id",
    source_path="/",          # Folder path within Box (relative to folder_id)
    sync_mode="continuous",   # Keep syncing new/changed files (vs initial_only)
)
```

## Finding Your Folder ID

To sync a specific Box folder (instead of root):

1. Open Box and navigate to the folder
2. Look at the URL: `https://app.box.com/folder/123456789`
3. The number at the end is your **Folder ID**

Use `0` for the root folder.

## Supported File Types

Mixpeek processes all file types from Box. Common supported formats include:

| Type | Formats |
| - | - |
| Documents | PDF, DOCX, XLSX, PPTX, TXT |
| Images | PNG, JPEG, GIF, WEBP |
| Video | MP4, MOV, AVI |
| Audio | MP3, WAV, M4A |
| Data | JSON, CSV, XML |

## Troubleshooting

> **Developer token expired (0 files discovered)**
>
> Developer tokens expire after 60 minutes. Generate a new token from the Box Developer Console and update your connection credentials.
>
> For production use, switch to **CCG authentication** to avoid expiry issues.

> **No files discovered on re-run**
>
> In `continuous` sync mode, only new or modified files since the last sync are processed. If you want to re-process all files, delete and recreate the sync configuration to reset the `last_sync_at` timestamp.

> **Authentication failed with CCG**
>
> Ensure your CCG app has been authorized in the Box Admin Console:
>
> 1. Go to **Admin Console** → **Apps** → **Custom Apps Manager**
> 2. Find your app and click **Authorize**
>
> Also verify that the **Enterprise ID** matches your Box enterprise account.

> **Access denied to specific folders**
>
> For OAuth connections, the developer token grants access based on the Box user's permissions. Ensure the user has access to the target folder.
>
> For CCG connections, the app acts as a service account. Use the Box Admin Console to grant the app access to specific folders if needed.

## Related

* [S3 Integration](/docs/integrations/object-storage/s3)
* [S3 Integration](/docs/integrations/object-storage/s3)
* [Bucket Syncs](/docs/ingestion/buckets)


# Google Cloud Storage
Source: https://docs.mixpeek.com/docs/integrations/object-storage/gcs

Connect Mixpeek to your Google Cloud Storage buckets to ingest and process your data.

> [!NOTE]
>
> This guide explains how to connect your Google Cloud Storage (GCS) buckets to Mixpeek, enabling automated data ingestion and processing.

## Prerequisites

* An active Google Cloud project.
* A GCS bucket containing the data you want Mixpeek to process.
* Permissions to create service accounts and manage IAM policies in your Google Cloud project.

## Configuration Steps

Connecting Mixpeek to GCS requires granting Mixpeek read access to your bucket. We recommend using a service account with appropriate IAM roles for enhanced security.

**1. Create Service Account**

First, create a service account that Mixpeek will use to access your GCS bucket.

1. Navigate to the IAM & Admin > Service Accounts section in your Google Cloud Console.
2. Click **Create Service Account**.
3. Enter a name for the service account (e.g., `mixpeek-gcs-access`).
4. Click **Create and Continue**.
5. Grant the service account the following roles:
   * `Storage Object Viewer` (roles/storage.objectViewer)
   * `Storage Legacy Bucket Reader` (roles/storage.legacyBucketReader)
6. Click **Done**.

**2. Create and Download Service Account Key**

To authenticate with GCS, you'll need to create and download a service account key.

1. Find your newly created service account in the list and click on it.
2. Go to the **Keys** tab.
3. Click **Add Key** > **Create new key**.
4. Choose **JSON** as the key type.
5. Click **Create**.
6. **Important:** The JSON key file will be downloaded to your computer. Store this file securely as you'll need it to configure the connection in Mixpeek.

**3. Add GCS Connection in Mixpeek**

In Studio, go to **Settings > Connections > New Connection** and pick **Google Cloud Storage**.

1. Paste the service account JSON key you downloaded.
2. Enter the **bucket** name. Omit the `gs://` prefix.
3. Optionally set a **prefix** to scope the sync to one folder.
4. Save. Mixpeek tests the credentials before storing them.

To create the same connection over the API, `POST /v1/organizations/connections`:

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer mxp_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "gcs-media",
    "provider_type": "gcs",
    "provider_config": {
      "provider_type": "gcs",
      "bucket": "YOUR_BUCKET_NAME",
      "prefix": "media/",
      "credentials": {
        "type": "service_account",
        "service_account_json": { "type": "service_account", "project_id": "..." }
      }
    },
    "test_before_save": true
  }'
```

```python Python theme={null}
import requests

requests.post(
    "https://api.mixpeek.com/v1/organizations/connections",
    headers={"Authorization": "Bearer mxp_sk_..."},
    json={
        "name": "gcs-media",
        "provider_type": "gcs",
        "provider_config": {
            "provider_type": "gcs",
            "bucket": "YOUR_BUCKET_NAME",
            "prefix": "media/",
            "credentials": {
                "type": "service_account",
                "service_account_json": service_account_key_dict,
            },
        },
        "test_before_save": True,
    },
)
```

```javascript JavaScript theme={null}
await fetch("https://api.mixpeek.com/v1/organizations/connections", {
  method: "POST",
  headers: {
    Authorization: "Bearer mxp_sk_...",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    name: "gcs-media",
    provider_type: "gcs",
    provider_config: {
      provider_type: "gcs",
      bucket: "YOUR_BUCKET_NAME",
      prefix: "media/",
      credentials: {
        type: "service_account",
        service_account_json: serviceAccountKey,
      },
    },
    test_before_save: true,
  }),
});
```

> [!NOTE]
>
> Mixpeek uses the native `google-cloud-storage` SDK, not the S3-compatible XML API, so
> you do not need to mint HMAC interoperability keys. `service_account_json` takes the
> whole key object. `private_key` is redacted when you read the connection back.

## Optional: allow writes for export destinations

Read access is all Mixpeek needs to ingest from your bucket. Add write access only if you
want to use this connection as an **export destination**, where Mixpeek writes a dataset
export and its `manifest.json` into your bucket.

Grant the service account one more permission on the bucket, `storage.objects.create`.
The predefined role **Storage Object Creator** (`roles/storage.objectCreator`) carries it,
and **Storage Object Admin** carries it along with delete:

```bash theme={null}
gsutil iam ch \
  serviceAccount:YOUR_SERVICE_ACCOUNT@YOUR_PROJECT.iam.gserviceaccount.com:objectCreator \
  gs://YOUR_BUCKET_NAME
```

Then turn the capability on for that connection, which is off by default:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/organizations/connections/conn_xxx \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"write_enabled": true}'
```

Both are required, and they are independent: `write_enabled` is your permission for
Mixpeek to try, and the IAM role is Google letting it succeed. An export naming a
connection without the flag is refused before anything is written, and the error names the
permission to add. Connections you never enable stay read-only.

## Verification

Once connected, Mixpeek should start discovering files in your specified GCS bucket (and prefix, if provided). You can monitor the ingestion status within the Mixpeek Studio. Depending on your pipeline configuration, feature extraction and indexing will begin automatically for supported file types.

If you encounter issues, double-check the service account permissions and the credentials provided in Mixpeek. Ensure the bucket name and project ID are correct.


# Google Drive
Source: https://docs.mixpeek.com/docs/integrations/object-storage/google-drive

Sync files from a Google Drive folder or shared drive into Mixpeek

Mixpeek reads files directly from Google Drive — personal folders, shared drives, and team collaboration folders. You create a storage **connection** (how to authenticate), then a bucket **sync** that points at a folder. New and changed files are picked up automatically.

> [!NOTE]
>
> Mixpeek only **reads** from your Drive. It never writes or deletes files in your Drive.

## 1. Authenticate

Two auth methods are supported. **Service account** is recommended for unattended, server-to-server sync.

**Service account (recommended)**

1. In Google Cloud Console, create a **service account** and download its JSON key.
2. **Share the target Drive folder (or shared drive) with the service account's `client_email`** — this is what grants Mixpeek access. Viewer permission is enough.
3. Create the connection with the fields from the JSON key file:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/organizations/connections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Marketing Google Drive",
    "provider_type": "google_drive",
    "provider_config": {
      "provider_type": "google_drive",
      "credentials": {
        "type": "service_account",
        "project_id": "my-project",
        "private_key_id": "<from JSON key file>",
        "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
        "client_email": "sync@my-project.iam.gserviceaccount.com",
        "client_id": "<from JSON key file>"
      },
      "shared_drive_id": "0AH-Xabc123"
    }
  }'
```

`shared_drive_id` is optional — include it only when syncing a **shared drive** (find it in the drive's URL). For a folder in a regular My Drive, omit it and point the sync at the folder ID instead (step 2). `private_key` and the other secrets are encrypted at rest.

**OAuth**

Use OAuth when syncing a specific user's Drive. Supply a client ID/secret and a long-lived refresh token:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/organizations/connections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Drive",
    "provider_type": "google_drive",
    "provider_config": {
      "provider_type": "google_drive",
      "credentials": {
        "type": "oauth",
        "client_id": "<oauth client id>",
        "client_secret": "<oauth client secret>",
        "refresh_token": "<long-lived refresh token>"
      }
    }
  }'
```

The response includes a `connection_id` (`conn_...`).

> [!TIP]
>
> List folders and files the connection can see with `GET /v1/organizations/connections/{connection_id}/folders` and `…/files` — handy for finding the folder ID to sync.

## 2. Sync a folder into a bucket

Create a [bucket](/docs/platform/data-model) first, then attach a sync. `source_path` is the **Drive folder ID** (the trailing segment of the folder's URL, e.g. `1A2b3C...`):

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "source_path": "1A2b3C4d5E6f7G8h9I",
    "sync_mode": "continuous",
    "polling_interval_seconds": 3600
  }'
```

Then trigger the first sync:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID/trigger" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

After the initial sync, new and changed files in the folder are picked up automatically at the polling interval. See [Syncs](/docs/platform/syncs) for file/metadata filters, `on_delete` cascade behavior, and monitoring.

## Next steps

* [Syncs reference](/docs/platform/syncs) — filters, reconciliation, lifecycle, and robustness
* [Object Storage overview](/docs/integrations/object-storage/overview) — all supported providers
* [Ingest Data](/docs/platform/data-model) — buckets, objects, and batches


# Iconik
Source: https://docs.mixpeek.com/docs/integrations/object-storage/iconik

Connect Iconik DAM to Mixpeek for automated media asset processing and search

## Overview

[Iconik](https://iconik.io) is a cloud-native media asset management platform. By connecting Iconik to Mixpeek, you can automatically sync your media library — enabling search, classification, and analysis across video, image, and audio assets.

Iconik uses the same **connection sync** pattern as all other Mixpeek storage integrations. Mixpeek polls Iconik for assets, downloads proxy files, and creates bucket objects with full metadata. It can also parse editorial project files into **footage ↔ ad relationships** (see [Project-file linkage](#project-file-linkage)). Optionally, you can configure webhooks for real-time delete and update events.

## Prerequisites

* An Iconik account with assets
* An Iconik **App ID** and **Auth Token** with read access to assets, proxies, and files
* A Mixpeek account with an active namespace

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `auth_token` | Yes | Iconik authentication token (JWT or API token) |
| `app_id` | Yes | Iconik application ID (UUID format) |
| `webhook_secret` | No | Shared signing secret for inbound Iconik webhooks. Required only if you want Mixpeek to cascade-delete objects when assets are deleted in Iconik (see [Webhooks: delete and update](#webhooks-delete-and-update)). |

### Sync-level fields

| Field | Required | Default | Description |
| - | - | - | - |
| `source_path` | No | `iconik://` | Sync all assets, or filter by collection |
| `sync_mode` | No | `continuous` | `initial_only` or `continuous` |
| `polling_interval_seconds` | No | 300 | How often to check for new/modified assets (continuous mode) |
| `skip_duplicates` | No | `true` | Skip assets already synced with the same modification time |
| `reconcile` | No | all on | Object of `on_delete`, `on_update`, `on_filter_drift`, `re_extract_on_update` (each defaults `true`). Set a key to `false` to opt out. `on_update: true` repeats the per-asset calls for every asset on every walk (see [Modification detection](#modification-detection)) |
| `provider_filters` | No | — | Filter by collection IDs, asset status, or media type |

## Setup

**1. Create Iconik API credentials**

1. In the Iconik admin panel, go to **Admin > Integrations > Applications**
2. Click **Create Application**
3. Copy the **App ID** and **Auth Token** — you'll need both for the Mixpeek connection

> [!TIP]
>
> Use a dedicated application for the Mixpeek integration rather than a personal token. Application tokens don't expire when a user leaves the organization.

**2. Create the connection in Mixpeek**

In the Mixpeek Studio, go to **Settings > Storage Connections > Add Connection** and select **Iconik**.

Or via the API:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="YOUR_API_KEY")

connection = client.organizations.connections.create(
    name="Iconik Production",
    provider_type="iconik",
    provider_config={
        "credentials": {
            "auth_token": "YOUR_ICONIK_AUTH_TOKEN",
            "app_id": "YOUR_ICONIK_APP_ID"
        }
    },
    test_before_save=True
)

print(connection.connection_id)  # e.g. conn_3010b0af974c8d83
```

```javascript JavaScript theme={null}
import Mixpeek from "mixpeek";

const client = new Mixpeek({ apiKey: "YOUR_API_KEY" });

const connection = await client.organizations.connections.create({
  name: "Iconik Production",
  provider_type: "iconik",
  provider_config: {
    credentials: {
      auth_token: "YOUR_ICONIK_AUTH_TOKEN",
      app_id: "YOUR_ICONIK_APP_ID",
    },
  },
  test_before_save: true,
});
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Iconik Production",
    "provider_type": "iconik",
    "provider_config": {
      "credentials": {
        "auth_token": "YOUR_ICONIK_AUTH_TOKEN",
        "app_id": "YOUR_ICONIK_APP_ID"
      }
    },
    "test_before_save": true
  }'
```

Save the returned `connection_id`.

**3. Create a bucket with Iconik sync**

Create a bucket and configure it to sync from your Iconik connection:

```python Python theme={null}
bucket = client.buckets.create(
    namespace_id="your-namespace-id",
    name="iconik-assets",
    description="Media assets from Iconik",
    blob_type="video"
)

sync = client.buckets.syncs.create(
    namespace_id="your-namespace-id",
    bucket_id=bucket["bucket_id"],
    connection_id=connection.connection_id,
    source_path="iconik://",
    sync_mode="continuous",
    polling_interval_seconds=300,
    skip_duplicates=True
)
```

```bash cURL theme={null}
# Create bucket
curl -X POST https://api.mixpeek.com/v1/buckets \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "iconik-assets",
    "description": "Media assets from Iconik",
    "bucket_schema": {
      "properties": {
        "asset_url": {"type": "video"},
        "title": {"type": "string"}
      }
    }
  }'

# Create sync config
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_YOUR_CONNECTION_ID",
    "source_path": "iconik://",
    "sync_mode": "continuous",
    "polling_interval_seconds": 300,
    "skip_duplicates": true
  }'
```

**4. Verify ingestion**

Monitor the sync status in Studio or via the API. Assets will be automatically downloaded (proxy files) and processed through your collection's feature extractors.

```bash theme={null}
curl "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE"
```

## Collection traversal and directory mapping

When you specify `collection_ids` in `provider_filters`, the connector recursively traverses the Iconik collection hierarchy (up to 10 levels deep) and maps the collection tree to a `directory_path` on each synced object.

For example, an Iconik collection structure like:

```
Brand Campaign (root collection)
├── 01 Product Shots
│   ├── 01 Ingredients
│   └── 08 Outros
├── 02 UGC Footage
│   ├── 01 Talent (Before)
│   │   ├── 00 Young
│   │   └── 01 Mature
│   └── 02 Talent (After)
└── 03 High-End Footage
```

produces objects with `directory_path` values like:

* `Brand Campaign/01 Product Shots/08 Outros`
* `Brand Campaign/02 UGC Footage/01 Talent (Before)/01 Mature`
* `Brand Campaign/03 High-End Footage`

This path is available in `source_metadata.directory_path` and can be used for filtering in retrievers.

### Glob filtering with `path_patterns`

Use `path_patterns` in `provider_filters` to include only assets whose `directory_path` matches one or more glob patterns. Patterns use standard `fnmatch` syntax and are case-insensitive.

```json theme={null}
{
  "provider_filters": {
    "collection_ids": ["15202dd4-9b00-11ef-8481-fedcde789d87"],
    "media_type": "video",
    "path_patterns": ["*UGC*", "*High-End*"]
  }
}
```

| Pattern | Matches |
| - | - |
| `*UGC*` | Any path containing "UGC" |
| `Brand Campaign/01*` | All paths under "01 Product Shots" |
| `*Talent*` | Before and After talent folders |
| `Brand Campaign/*/01*` | First subfolder in each top-level category |
| `*Outros*` | Only the Outros folder |

## Iconik metadata on objects and retriever results

When the connector syncs an Iconik asset into a bucket, it captures metadata from two sources and exposes them as **`source_metadata`** on the resulting bucket object.

### Asset-level fields

These come from the Iconik asset record directly:

| Field | Description |
| - | - |
| `iconik_asset_id` | Iconik asset UUID |
| `title` | Asset title |
| `status` | Asset status (e.g., `ACTIVE`) |
| `media_type` | Media type (`video`, `image`, `audio`) |
| `date_created` | Asset creation timestamp |
| `date_modified` | Last modification timestamp |
| `date_imported` | When the asset was imported into Iconik |
| `is_online` | Whether the asset is currently online |
| `archive_status` | Archive state of the asset |
| `analyze_status` | Iconik analysis status |
| `category` | Asset category |
| `type` | Iconik object type (e.g., `ASSET`) |
| `external_id` | External reference ID |
| `created_by_user` | Iconik user ID who created the asset |
| `updated_by_user` | Iconik user ID who last modified the asset |
| `directory_path` | Collection hierarchy path (see above) |
| `thumbnail_url` | Keyframe/thumbnail URL (if available) |
| `proxy_url` | Pre-signed proxy download URL |

### Custom metadata from views

The connector automatically discovers all metadata views configured in your Iconik workspace and fetches custom metadata for each asset across all views. Fields are flattened from Iconik's nested `field_values` format into simple key-value pairs.

Example fields (depending on your Iconik configuration):

| Field | Description |
| - | - |
| `Keywords` | Tag cloud / keywords |
| `ProjectName` | Project or campaign name |
| `Category` | Asset category |
| `Description` | Asset description |
| `FolderNamePaths` | Iconik folder path metadata |
| `CreatorName` | Creator / uploader name |

All populated custom metadata fields are included automatically — no configuration needed. If you want to limit metadata to a specific view, set `metadata_view_id` in the connection's `provider_config`:

```json theme={null}
{
  "provider_config": {
    "credentials": { "auth_token": "...", "app_id": "..." },
    "metadata_view_id": "97324f86-55ba-11ef-877b-c2a940a89d9c"
  }
}
```

## Project-file linkage

Beyond per-asset metadata, the Iconik connector captures the **relationships between assets** that come from editorial (NLE) projects. For Iconik **`NLE_PROJECT` assets** (e.g. an Adobe Premiere project — one per produced ad), the connector reads Iconik's **relations** — the media assets the project uses, already parsed by Iconik — and attaches typed, directed **[edges](/docs/retrieval/stages/traverse-edge)** to the object — Mixpeek's first-class model for customer-owned relationships between objects.

When the project file itself is downloadable (native `.prproj`, `.fcpxml`, or an `xmeml` `.xml` export), the connector additionally parses it to enrich each edge with **clip order and timeline ticks** — matched to the relations by footage file name. This is how an assembled ad is linked to every piece of footage it uses, with its position in the cut — captured as a **saved value** on the object at ingestion, not recomputed at query time.

### What gets captured

For an NLE project asset, the connector emits one edge per related media asset, stored at the **root** of the object under `edges`:

| Edge field | Description |
| - | - |
| `type` | `uses_footage` for related video assets; `uses_media` for audio/image/other (with `attributes.media_type`) |
| `target_object_id` | The footage object this relation references, resolved from the source asset id. Until that footage object exists in Mixpeek, the edge preserves the source asset reference and is flagged `attributes.target_unresolved` so a later reconcile can rewrite it. |
| `direction` | `out` (ad → footage) |
| `attributes.iconik_asset_id` | The related Iconik asset's id (the durable foreign key) |
| `attributes.clip_order` | The clip's order within the ad sequence (1-based; from the project-file parse) |
| `attributes.start_ticks_in` / `start_ticks_out` | Clip position in Adobe ticks (`254,016,000,000` per second). `attributes.tick_semantics` says what they measure: `"timeline"` (position in the ad — native `.prproj`) or `"source"` (in/out within the source footage — `xmeml` exports) |

The connector also sets two root-level signals on the ad object:

| Field | Description |
| - | - |
| `is_final_ad` | `true` when the asset is an assembled/produced ad (a project file with clips) — distinguishing it from raw pre-production footage |
| `ad_clip_count` | Number of footage clips composited into the ad |

Plain media assets (raw footage) get no edges and pay no extra API calls — linkage capture is gated to NLE project assets. Everything is fully **fail-open**: a relations error or a missing/malformed project file never blocks asset ingestion (edges without a parseable project file simply carry no tick attributes).

> [!NOTE]
>
> Edges and these signals are **customer-owned data** and live at the object **root**, never under `_internal`. They flow automatically from the object through your collections into the document's search payload, so they are available to retrievers.

### Traversing the relationships

Once footage↔ad edges are on your documents, the [`traverse_edge`](/docs/retrieval/stages/traverse-edge) retriever stage follows them at query time — e.g. starting from an ad, fetch every piece of footage it uses, carrying the clip order and start-ticks of each:

```json theme={null}
{
  "stage_id": "traverse_edge",
  "parameters": { "edge_type": "uses_footage", "direction": "out" }
}
```

> [!NOTE]
>
> This linkage is a **generalizable** capability of Mixpeek's source-adapter framework — any connector whose source carries a companion/sidecar file (an NLE project, an XMP sidecar, a per-object `.json`) can populate edges the same way. The Adobe Premiere `xmeml` parser is the Iconik-specific piece.

## File resolution

The Iconik connector resolves downloadable files in priority order:

1. **Proxy URL** — pre-signed download URL from `GET /files/v1/assets/{id}/proxies/`
2. **Original file URL** — download URL from `GET /files/v1/assets/{id}/formats/`

Per asset, the connector makes: one asset metadata call, one proxy URL resolution call, and one metadata view call per discovered view (views are cached after the first asset).

## Webhooks: delete and update

Mixpeek handles Iconik webhook events to keep your index in sync in real time, without waiting for the next poll cycle.

### How it works

1. Iconik POSTs events to a Mixpeek endpoint that includes your `connection_id`.
2. Mixpeek verifies the `X-Iconik-Signature` header against the `webhook_secret` stored on that connection.
3. **`assets.asset_deleted`** — Mixpeek finds every object synced from that Iconik asset and deletes them, cascading through `OBJECT_DELETED` to remove derived documents from your collections.
4. **`assets.asset_updated`** — Mixpeek re-evaluates the asset against each sync config's `metadata_filters`. If the asset no longer matches and `reconcile.on_filter_drift` is `true`, which is the default, the corresponding objects are unindexed.
5. **`assets.asset_created`** — Acknowledged; the asset will be picked up on the next sync poll.

### Setup

**1. Add a webhook secret to the connection**

Pick any high-entropy string (32+ chars) and store it as `webhook_secret` in the Iconik connection's credentials. You can set it at create time or patch an existing connection.

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/organizations/connections/{connection_id} \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider_config": {
      "credentials": {
        "auth_token": "...",
        "app_id": "...",
        "webhook_secret": "your-32-char-random-string"
      }
    }
  }'
```

**2. Register the webhook in Iconik**

In the Iconik admin panel:

1. Go to **Admin > Integrations > Webhooks**
2. Click **Create Webhook**
3. Set the **URL** to `https://api.mixpeek.com/v1/webhooks/iconik/{connection_id}` — replace `{connection_id}` with the Mixpeek connection ID
4. Subscribe to events: **Asset Created**, **Asset Updated**, **Asset Deleted**
5. Save the webhook

**3. Verify with a test delete**

Delete a synced asset in Iconik and confirm the corresponding object disappears from your bucket. The webhook returns a JSON body documenting what it did:

```json theme={null}
{
  "received": true,
  "event_type": "assets.asset_deleted",
  "handled": true,
  "asset_id": "e0f0ec84-4d63-11f1-99fa-b221c0ff30c8",
  "deleted_objects": 1
}
```

### Signature format

Mixpeek verifies inbound Iconik webhooks using HMAC-SHA256:

* Header: `X-Iconik-Signature: <hex_hmac_sha256>`
* HMAC computed as `HMAC_SHA256(webhook_secret, raw_body)` over the **exact raw bytes** of the request body.
* Signatures are compared with `hmac.compare_digest` to avoid timing attacks.

### Response codes

| Status | Meaning |
| - | - |
| `200` | Event received. `handled` is `true` for delete/update/create, `false` for unrecognized event types. |
| `400` | Connection has no `webhook_secret` configured, or payload is not valid JSON. |
| `401` | `X-Iconik-Signature` is missing, malformed, or the HMAC doesn't match. |
| `404` | `connection_id` in the URL is not an Iconik connection. |

## Modification detection

When `skip_duplicates` is `true` (default), re-syncing the same Iconik account skips assets whose `date_modified` hasn't changed since the last sync. Only new or modified assets are re-processed.

When `skip_duplicates` is `false`, every asset is re-downloaded and re-processed on each sync cycle, replacing existing objects with fresh ones.

### What `reconcile.on_update` costs per walk

`reconcile.on_update` defaults to `true`. In that mode the sync does not skip an unchanged asset at the listing. It makes the per-asset Iconik calls for every asset the walk lists, on every walk, and drops the unchanged assets afterwards. A metadata edit that leaves `date_modified` untouched is caught this way, and the API cost scales with the size of the library, not with the number of assets that changed.

Set `on_update` to `false` and the sync compares each asset's stored `date_modified` with the listing before it makes any per-asset call. It fetches only the assets that are new or whose value moved. A metadata edit that leaves `date_modified` untouched is not picked up in this mode.

```json theme={null}
{
  "reconcile": { "on_update": false }
}
```

> [!TIP]
>
> For libraries of 10k+ assets, set `reconcile.on_update` to `false` unless you need to catch metadata edits that leave `date_modified` untouched. Iconik rate-limits requests, so a walk that repeats every per-asset call spends most of its budget on assets that did not change.

## Reconciliation

Each sync cycle checks whether previously synced assets still exist in Iconik. Assets deleted from Iconik are removed from your bucket. `reconcile.on_delete` controls this and defaults to `true`, so you get this behaviour without configuring anything. Set it to `false` to keep objects after the source asset disappears.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_YOUR_CONNECTION_ID",
    "source_path": "iconik://",
    "reconcile": { "on_delete": true, "on_filter_drift": true }
  }'
```

> [!WARNING]
>
> Reconciliation checks each indexed object against the Iconik API on every sync cycle. For large libraries (10k+ assets), this adds API calls proportional to the number of indexed objects. Use it when stale objects are unacceptable.

## Provider filters

Filter which Iconik assets get synced using `provider_filters`:

```json theme={null}
{
  "provider_filters": {
    "collection_ids": ["col_abc", "col_def"],
    "status": "ACTIVE",
    "media_type": "video"
  }
}
```

| Filter | Description |
| - | - |
| `collection_ids` | Only sync assets from specific Iconik collections |
| `status` | Filter by asset status (e.g., `ACTIVE`) |
| `media_type` | Filter by media type (`video`, `image`, `audio`) |

## Sync modes

| Mode | Description |
| - | - |
| `initial_only` | Sync all current assets once |
| `continuous` | Poll for new/modified assets at the configured interval |

## Troubleshooting

> **401 Unauthorized on connection test**
>
> Verify your App ID and Auth Token are correct. The App ID is a UUID (e.g., `e0f0ec84-4d63-...`). Tokens can be regenerated in the Iconik admin panel under **Admin > Integrations > Applications**.

> **Assets not appearing in sync**
>
> Only assets with proxy files or original files available are synced. Check that your Iconik credentials have read access to the files API (`GET /files/v1/assets/{id}/proxies/`).

> **Duplicate objects appearing**
>
> Ensure `skip_duplicates` is set to `true` on your sync config. This deduplicates assets by their Iconik asset ID and modification timestamp.

> **Deleted assets still in bucket**
>
> Leave `reconcile.on_delete` at its default of `true` on your sync config, or configure webhooks to handle `assets.asset_deleted` events for real-time cleanup.

> **Webhook 401 errors**
>
> The `X-Iconik-Signature` header doesn't match. Verify the `webhook_secret` on your Mixpeek connection matches what Iconik is using to sign payloads.

## Related

* [Buckets](/docs/ingestion/buckets) — How bucket sync works
* [Webhooks](/docs/operations/webhooks) — How outbound Mixpeek webhooks work
* [Storage Connections API](/docs/api-reference/organization-connections/create-storage-connection) — Full API reference


# Mux
Source: https://docs.mixpeek.com/docs/integrations/object-storage/mux

Connect Mux video infrastructure to Mixpeek for automated video asset processing

## Overview

[Mux](https://mux.com) is a video infrastructure platform for building video experiences. By connecting Mux to Mixpeek, you can automatically process your video assets through feature extraction pipelines — enabling search, classification, and analysis across your entire video library.

## Prerequisites

* A Mux account with video assets
* A Mux access token with **Mux Video** read permissions
* A Mixpeek account with an active namespace
* **Static renditions or MP4 support enabled on your Mux assets** (see below)

> [!WARNING]
>
> **Static renditions are required.** By default, Mux assets only have HLS streaming — no downloadable MP4 files. Mixpeek needs MP4 renditions to download and process your videos. Assets without static renditions will be **skipped** during sync.
>
> Enable static renditions when creating assets:
>
> ```json theme={null}
> {
>   "input": [{"url": "https://example.com/video.mp4"}],
>   "playback_policy": ["public"],
>   "static_renditions": [{"resolution": "720p"}]
> }
> ```
>
> Or enable them on existing assets in the [Mux Dashboard](https://dashboard.mux.com) under each asset's settings.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `access_token_id` | Yes | Mux access token ID (UUID format) |
| `access_token_secret` | Yes | Mux access token secret |
| `webhook_secret` | No | Shared signing secret for inbound Mux webhooks. Required only if you want Mixpeek to cascade-delete objects when assets are deleted in Mux (see [Cascade delete via webhooks](#cascade-delete-via-webhooks)). |

### Sync-level fields

| Field | Required | Default | Description |
| - | - | - | - |
| `source_path` | No | `mux://` | Sync all assets, or `mux://{asset_id}` for a specific asset |
| `sync_mode` | No | `continuous` | `initial_only` or `continuous` |
| `polling_interval_seconds` | No | 300 | How often to check for new assets (continuous mode) |
| `file_filters.metadata_filters` | No | — | Array of metadata filter rules applied to Mux asset fields (see [Metadata filtering](#metadata-filtering)) |
| `reconcile` | No | all on | Object of `on_delete`, `on_update`, `on_filter_drift`, `re_extract_on_update` (each defaults `true`). `on_filter_drift` unindexes previously indexed assets that no longer match metadata filters (see [Reconciliation](#reconciliation)) |

## Setup

**1. Create a Mux access token**

1. Go to [Mux Dashboard → Settings → Access Tokens](https://dashboard.mux.com/settings/access-tokens)
2. Click **Generate new token**
3. Select **Mux Video** with **Read** permissions
4. Copy the **Token ID** and **Token Secret** immediately — the secret is only shown once

**2. Create the connection in Mixpeek**

In the Mixpeek Studio, go to **Settings → Storage Connections → Add Connection** and select **Mux**.

Or via the API:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="YOUR_API_KEY")

connection = client.organizations.connections.create(
    name="Mux Production",
    provider_type="mux",
    provider_config={
        "credentials": {
            "type": "access_token",
            "access_token_id": "a3b02074-dbca-47b6-...",
            "access_token_secret": "Am9+RGn2mhmz..."
        }
    },
    test_before_save=True
)
```

```javascript JavaScript theme={null}
import Mixpeek from "mixpeek";

const client = new Mixpeek({ apiKey: "YOUR_API_KEY" });

const connection = await client.organizations.connections.create({
  name: "Mux Production",
  provider_type: "mux",
  provider_config: {
    credentials: {
      type: "access_token",
      access_token_id: "a3b02074-dbca-47b6-...",
      access_token_secret: "Am9+RGn2mhmz...",
    },
  },
  test_before_save: true,
});
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Mux Production",
    "provider_type": "mux",
    "provider_config": {
      "credentials": {
        "type": "access_token",
        "access_token_id": "a3b02074-dbca-47b6-...",
        "access_token_secret": "Am9+RGn2mhmz..."
      }
    },
    "test_before_save": true
  }'
```

**3. Create a bucket with Mux sync**

Create a bucket and configure it to sync from your Mux connection:

```python Python theme={null}
bucket = client.buckets.create(
    namespace_id="your-namespace-id",
    name="mux-videos",
    description="Video assets from Mux",
    blob_type="video"
)

sync = client.buckets.syncs.create(
    namespace_id="your-namespace-id",
    bucket_id=bucket["bucket_id"],
    connection_id=connection.connection_id,
    source_path="mux://",
    sync_mode="continuous",
    polling_interval_seconds=300
)
```

**4. Verify ingestion**

Monitor the sync status in Studio or via the API. Video assets in `ready` status will be automatically downloaded and processed through your collection's feature extractors.

## Mux metadata on objects and retriever results

When the connector syncs a Mux asset into a bucket, it captures the Mux identifiers and exposes them as **`source_metadata`** on the resulting bucket object. This metadata flows through unchanged to downstream documents and is returned alongside every retriever match — so your application can render results directly from Mux without falling back to the S3 mirror.

| Field | Description |
| - | - |
| `asset_id` | Mux asset ID — use to look the asset up via the Mux API |
| `playback_id` | Primary public/signed playback ID |
| `playback_ids` | Full list of playback IDs on the asset |
| `playback_url` | HLS manifest URL (`https://stream.mux.com/{playback_id}.m3u8`) ready for `<mux-player>` or any HLS client |
| `thumbnail_url` | Default Mux thumbnail (`https://stream.mux.com/{playback_id}/thumbnail.jpg`) |
| `duration` | Asset duration in seconds |
| `resolution_tier` | Source quality tier (e.g. `1080p`, `1440p`) |
| `status` | Mux asset status at sync time |
| `aspect_ratio` | Asset aspect ratio (e.g. `16:9`) |

Example object payload:

```json theme={null}
{
  "object_id": "obj_7fa70fd60a0547152df57cff",
  "bucket_id": "bkt_648b4be8",
  "source_metadata": {
    "asset_id": "5sCF1Yk3rC0102zMVEh74jJE9pT6KFeZ1N8gcQZ3l53FI",
    "playback_id": "YGDmuLw7AOiQKwvCZbHAGjdZRZwxMqqg1iBELyJig5E",
    "playback_url": "https://stream.mux.com/YGDmuLw7AOiQKwvCZbHAGjdZRZwxMqqg1iBELyJig5E.m3u8",
    "thumbnail_url": "https://stream.mux.com/YGDmuLw7AOiQKwvCZbHAGjdZRZwxMqqg1iBELyJig5E/thumbnail.jpg",
    "duration": 23.86,
    "resolution_tier": "1080p",
    "status": "ready",
    "aspect_ratio": "16:9"
  }
}
```

In retriever results, the same `source_metadata` is attached to each match. A typical UI flow is:

```javascript theme={null}
// Each search result carries the original Mux IDs — no S3 hop needed
results.forEach((match) => {
  const { playback_id, playback_url, thumbnail_url } = match.source_metadata;
  renderMuxPlayer({ playbackId: playback_id, poster: thumbnail_url });
});
```

## Webhooks: delete and update

Mixpeek handles two Mux webhook event types to keep your index in sync with Mux automatically.

### How it works

1. Mux POSTs events to a Mixpeek endpoint that includes your `connection_id`.
2. Mixpeek verifies the `Mux-Signature` header against the `webhook_secret` stored on that connection.
3. **`video.asset.deleted`** — Mixpeek finds every object synced from that Mux asset and deletes them, cascading through `OBJECT_DELETED` to remove derived documents from your collections.
4. **`video.asset.updated`** — Mixpeek re-fetches the asset's current metadata from Mux and re-evaluates it against each sync config's `metadata_filters`. If a previously non-matching asset now matches, it becomes eligible for indexing on the next sync. If the asset no longer matches and `reconcile.on_filter_drift` is `true`, which is the default, the corresponding objects are unindexed. With it set to `false`, the event is acknowledged but no objects are removed.

### Setup

**1. Generate a signing secret and attach it to the connection**

Pick any high-entropy string (32+ chars) and store it as `webhook_secret` in the Mux connection's credentials. You can set it at create time or patch an existing connection.

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/organizations/connections/{connection_id} \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider_config": {
      "credentials": {
        "type": "access_token",
        "access_token_id": "...",
        "access_token_secret": "...",
        "webhook_secret": "your-32-char-random-string"
      }
    }
  }'
```

**2. Register the webhook in the Mux dashboard**

In the [Mux Dashboard → Settings → Webhooks](https://dashboard.mux.com/settings/webhooks), add a new endpoint:

* **URL:** `https://api.mixpeek.com/v1/webhooks/mux/{connection_id}` — replace `{connection_id}` with the Mixpeek connection ID returned when you created the connection.
* **Signing secret:** the **same** value you put in `webhook_secret`.
* **Events:** at minimum `video.asset.deleted` and `video.asset.updated`. You can subscribe to additional events; Mixpeek will acknowledge them and ignore any it doesn't act on.

**3. Verify with a test delete**

Delete a synced asset in Mux (or use the dashboard's "Send test event" button) and confirm the corresponding object disappears from your bucket. The webhook returns a JSON body documenting what it did:

```json theme={null}
{
  "received": true,
  "event_type": "video.asset.deleted",
  "handled": true,
  "asset_id": "5sCF1Yk3rC0102zMVEh74jJE9pT6KFeZ1N8gcQZ3l53FI",
  "deleted_objects": 1
}
```

### Signature format

Mixpeek follows the same scheme Mux documents for outgoing webhooks:

* Header: `Mux-Signature: t=<unix_ts>,v1=<hex_hmac_sha256>`
* HMAC computed as `HMAC_SHA256(webhook_secret, f"{t}.{raw_body}")` over the **exact raw bytes** of the request body.
* Signatures are compared with `hmac.compare_digest` to avoid timing attacks.

### Response codes

| Status | Meaning |
| - | - |
| `200` | Event received. `handled` is `true` for `video.asset.deleted`, `false` for ignored event types. |
| `400` | Connection has no `webhook_secret` configured, or payload is not valid JSON. |
| `401` | `Mux-Signature` is missing, malformed, or the HMAC doesn't match. |
| `404` | `connection_id` in the URL is not a Mux connection. |

## Source path format

| Path | Behavior |
| - | - |
| `mux://` | Sync all video assets in ready status |
| `mux://{asset_id}` | Sync a specific asset by ID |

## Asset filtering

Only assets in `ready` status are synced. Assets that are `preparing` or `errored` are skipped automatically. You can also use file filters to narrow sync scope:

```python theme={null}
sync = client.buckets.syncs.create(
    namespace_id="your-namespace-id",
    bucket_id=bucket["bucket_id"],
    connection_id=connection.connection_id,
    source_path="mux://",
    file_filters={
        "min_size": 1000000,  # Skip assets under ~1MB (estimated)
    }
)
```

## Metadata filtering

You can filter which Mux assets get synced based on **asset-level metadata** — `passthrough`, `meta.external_id`, or any field captured from the Mux asset. This is useful when you want a single Mux environment connected to Mixpeek but only certain assets indexed.

Metadata filters are evaluated against the asset's metadata dict (which includes `passthrough`, `duration`, `resolution_tier`, `status`, etc.). Each filter specifies a `field`, `operator`, and `value`.

### Supported operators

| Operator | Description | Example |
| - | - | - |
| `contains` | Field value contains substring | `passthrough` contains `vi:1` |
| `equals` | Exact match | `status` equals `ready` |
| `not_equals` | Does not match | `resolution_tier` not\_equals `audio_only` |
| `not_contains` | Does not contain substring | `passthrough` not\_contains `skip` |
| `gt`, `lt`, `gte`, `lte` | Numeric comparison | `duration` gt `10` |
| `exists` | Field is present and non-empty | `passthrough` exists `true` |

### Example: sync only flagged assets

A common pattern is to use the Mux `passthrough` field as a sync selector. Your application sets a flag on assets that should be indexed, and Mixpeek only syncs matching assets.

```python Python theme={null}
sync = client.buckets.syncs.create(
    namespace_id="your-namespace-id",
    bucket_id=bucket["bucket_id"],
    connection_id=connection.connection_id,
    source_path="mux://",
    file_filters={
        "metadata_filters": [
            {
                "field": "passthrough",
                "operator": "contains",
                "value": "vi:1"
            }
        ]
    }
)
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "source_path": "mux://",
    "file_filters": {
      "metadata_filters": [
        {
          "field": "passthrough",
          "operator": "contains",
          "value": "vi:1"
        }
      ]
    }
  }'
```

With this configuration, an asset with `passthrough: "myapp|vi:1|v:abc"` would be synced, while `passthrough: "myapp|vi:0|v:abc"` would be skipped.

> [!TIP]
>
> **Metadata filters work with any sync provider**, not just Mux. The same `metadata_filters` syntax applies to S3, GCS, and all other storage connections — the filters are evaluated against whatever metadata the provider exposes for each file.

## Reconciliation

Each sync cycle re-checks all previously indexed assets against the current metadata filters, and assets that no longer match are unindexed. `reconcile.on_filter_drift` controls this and defaults to `true`.

This is useful when your application changes asset metadata after initial sync — for example, removing the visual-index flag from a Mux asset's `passthrough`. Without reconciliation, the already-indexed object would remain in the bucket. With reconciliation, it gets cleaned up on the next sync.

```bash theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{"reconcile": {"on_filter_drift": true}}'
```

> [!WARNING]
>
> Reconciliation queries the source provider for each indexed object on every sync cycle. For large libraries (10k+ assets), this adds API calls proportional to the number of indexed objects. Use it when metadata changes are common and stale objects are unacceptable.

## Sync modes

| Mode | Description |
| - | - |
| `initial_only` | Sync all current assets once |
| `continuous` | Poll for new assets at the configured interval |

## Troubleshooting

> **401 Unauthorized on connection test**
>
> Verify your access token ID and secret are correct. The token ID is a UUID (e.g., `a3b02074-dbca-47b6-...`), not the environment ID. Tokens can be regenerated in the [Mux Dashboard](https://dashboard.mux.com/settings/access-tokens).

> **Assets not appearing in sync**
>
> Only assets with `ready` status and at least one playback ID are synced. Check your asset status in the Mux Dashboard. Assets that are still encoding (`preparing`) will be picked up on the next sync cycle.

> **Download fails for specific assets**
>
> Assets need a public or signed playback ID **and** either static renditions or MP4 support enabled. By default, Mux assets only have HLS streaming — MP4 downloads require explicit configuration.
>
> To enable static renditions when creating an asset via the Mux API:
>
> ```json theme={null}
> {
>   "input": [{"url": "https://example.com/video.mp4"}],
>   "playback_policy": ["public"],
>   "static_renditions": [{"resolution": "720p"}]
> }
> ```
>
> You can also enable static renditions on existing assets in the Mux Dashboard under the asset's settings. Mixpeek will try multiple rendition qualities (low, medium, high, highest) and use the first available one.

## Related

* [Buckets](/docs/ingestion/buckets) — How bucket sync works
* [Webhooks](/docs/operations/webhooks) — How outbound Mixpeek webhooks work
* [Storage Connections API](/docs/api-reference/organization-connections/create-storage-connection) — Full API reference


# Overview
Source: https://docs.mixpeek.com/docs/integrations/object-storage/overview

Connect Mixpeek to your cloud object storage to ingest multimodal data in place.

> [!NOTE]
>
> Integrate Mixpeek directly with your existing object storage solutions like AWS S3, Google Cloud Storage (GCS), and Azure Blob Storage to process and analyze your multimodal data where it lives.

Object storage is the warehouse's canonical store, where all extracted features persist across tiers and serve as the durable source of truth regardless of which serving layer is active. Object storage systems are a fundamental component for storing large amounts of unstructured data, making them ideal upstream sources for Mixpeek. By connecting your buckets, you enable Mixpeek to automatically discover, process, and index your files, making images, videos, audio, and PDFs searchable in place.

## Why Connect Object Storage?

> **Centralized Data Processing**
>
> Process diverse file types stored in your buckets without needing to move data. Mixpeek accesses files directly from your provider.

> **Scalable Ingestion**
>
> Use the scalability of cloud object storage. Mixpeek can handle growing volumes of data as your needs evolve.

> **Automated Workflows**
>
> Set up automated pipelines. New files added to connected buckets can be automatically indexed and enriched by Mixpeek.

> **Secure Access**
>
> Utilize secure authentication methods (like IAM roles or access keys) to grant Mixpeek the necessary permissions to access your data.

## Supported Providers

Mixpeek supports direct integration with the major cloud object storage providers:

> **[AWS S3](/docs/integrations/object-storage/s3)**
>
> Connect your Amazon Simple Storage Service (S3) buckets.

> **[Google Drive](/docs/integrations/object-storage/google-drive)**
>
> Google Drive and Workspace shared drives, via service account or OAuth.

> **[Supabase Storage](/docs/integrations/object-storage/supabase)**
>
> S3-compatible storage on Supabase, via project S3 access keys.

> **[Cloudflare R2](/docs/integrations/object-storage/r2)**
>
> S3-compatible egress-free storage on Cloudflare's edge.

> **[Backblaze B2](/docs/integrations/object-storage/backblaze)**
>
> Low-cost S3-compatible object storage.

> **[Wasabi](/docs/integrations/object-storage/wasabi)**
>
> S3-compatible hot object storage.

> **[Tigris](/docs/integrations/object-storage/tigris)**
>
> Globally distributed S3-compatible storage.

> **[Box](/docs/integrations/object-storage/box)**
>
> Connect Box workspaces with OAuth or CCG.

> **[Mux](/docs/integrations/object-storage/mux)**
>
> Sync video assets from the Mux video infrastructure platform — playback metadata flows through to retriever results.

> **[Iconik](/docs/integrations/object-storage/iconik)**
>
> Sync media assets from Iconik DAM — proxy files, metadata, and real-time webhook sync.

Choose your provider above to find specific setup instructions.

> [!NOTE]
>
> **Google Cloud Storage** and **Azure Blob Storage** use the provider's native SDK, so
> neither needs S3-compatible interop keys. The provider types are `gcs` and `azure_blob`.
> A bare `azure` returns HTTP 422.

The full connector list is the provider picker in Studio under **Settings > Connections > New Connection**. It covers the object stores above plus [Snowflake](/docs/integrations/snowflake-warehouse), [PostgreSQL](/docs/integrations/databases/postgresql), [SharePoint / OneDrive](/docs/integrations/object-storage/sharepoint), [RSS](/docs/integrations/web-data/rss), [HTTP API](/docs/integrations/web-data/http-api), [BrightData](/docs/integrations/web-data/brightdata), [Instagram](/docs/integrations/social-media/instagram), [TikTok](/docs/integrations/social-media/tiktok), and [inbound email](/docs/integrations/email).

## Best Practices for Data Structuring

While Mixpeek can process complex, nested data structures within a single bucket connection, a more scalable strategy often involves structuring your data upfront in your object storage, then relying on taxonomy joins to put them together.

We rely on the post processing joins to intelligently combine related content after ingestion. This approach offers several advantages over trying to group files during the initial upload:

> **Explicit Structure**
>
> By organizing content into separate buckets or prefixes, you make the relationships between files clear and programmatically accessible. This is more reliable than trying to infer relationships from file names or metadata.

> **Flexible Joining**
>
> Mixpeek's taxonomy and clustering features allow you to join related content based on multiple criteria (IDs, metadata, semantic similarity) after ingestion, giving you more control over how content is combined.

> **Scalable Processing**
>
> Processing pipelines become simpler and more focused when handling one type of content structure at a time, making it easier to scale and maintain.

> **Reliable Updates**
>
> When new content is added to a structured bucket, Mixpeek can process it independently and then join it with related content, avoiding issues with partial uploads or timing dependencies.

**Recommended Approach: Pre-Structured Pipelines**

1. **Separate Buckets or Prefixes:** Organize related but distinct types of content into separate S3 buckets or dedicated prefixes (folders) within a single bucket.
   * *Example:* For analyzing video content, you might store raw videos in `s3://my-videos/raw/`, extracted transcripts in `s3://my-videos/transcripts/`, and associated metadata JSON files in `s3://my-videos/metadata/`.
2. **Multiple Mixpeek Connections:** Set up distinct Mixpeek buckets & collections pointing to each specific bucket or prefix.
3. **Join in Mixpeek:** After ingestion, use Mixpeek's enrichment features like **Clustering** or **Taxonomies** (joining based on matching IDs or other rules) to link the related pieces of content (e.g., connecting a transcript to its corresponding video and metadata).

This approach is more reliable than alternatives like:

* Relying on file naming conventions or metadata (which can be inconsistent)
* Using "agent mode" with LLMs to infer relationships (which is experimental and less predictable)
* Trying to group files during upload (which can be fragile due to timing issues)

Consider this approach if you are dealing with complex multimodal data where different components (like video, audio, text transcripts, metadata) need to be linked and analyzed together.

## Turn On Versioning in Your Bucket

Mixpeek reads your objects by reference: the files stay in your bucket, and Mixpeek stores the features it extracts from them. A namespace snapshot records which version of each object it saw (on Google Cloud Storage, the highest object generation at the time of the snapshot). A restore can point back at those versions only if your bucket still keeps them. If an object is overwritten or deleted after the snapshot and the bucket keeps no old versions, the restored documents point at the new file, or at nothing.

We recommend turning on object versioning, with old versions kept at least as long as your snapshots. Namespace snapshots are kept for 30 days by default.

* **Google Cloud Storage:** enable [Object Versioning](https://cloud.google.com/storage/docs/object-versioning), then add a lifecycle rule that deletes noncurrent versions after 30 days (`daysSinceNoncurrentTime: 30`).
* **AWS S3 and S3-compatible stores:** enable [bucket versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html), then add a lifecycle rule with `NoncurrentVersionExpiration` set to 30 days.
* **Azure Blob Storage:** enable [blob versioning](https://learn.microsoft.com/azure/storage/blobs/versioning-overview), then add a lifecycle policy that deletes previous versions after 30 days.

Versioning stores a copy of every overwritten object for the retention you choose, so its cost is your overwrite volume multiplied by the retention days. Buckets whose objects are written once and never replaced pay almost nothing for it.

## Getting Started

1. **Choose your Provider:** Select the object storage provider you use (AWS S3, GCS, Azure Blob Storage).
2. **Configure Access:** Follow the provider-specific guide to grant Mixpeek secure access to your desired bucket(s). This typically involves setting up appropriate permissions (e.g., read access).
3. **Add Connection in Mixpeek:** Use the Mixpeek dashboard or API to add the connection details for your object storage bucket.

   > [!WARNING]
   >
   > Creating, updating, or deleting a storage connection requires an API key with the **`admin`** permission. Connections hold credentials, so they sit behind organization administration alongside members, billing, secrets, and audit.
   >
   > The presets in **Settings → API Keys → New key** top out at read + write + delete, which is enough to create buckets, collections, and syncs but **not** connections. To grant `admin`, open the key's **Edit** dialog and tick **Admin**. A key without it gets:
   >
   > ```json theme={null}
   > {"error": {"type": "UnauthorizedError",
   >            "details": {"required_permissions": ["admin"],
   >                        "your_permissions": ["read", "write", "delete"]}}}
   > ```
   >
   > Reading and listing connections only needs `read`, so a scripted sync that references an existing `connection_id` does not need an admin key.
4. **Start Processing:** Once connected, Mixpeek can begin discovering and processing files according to your configured pipelines.

Ready to connect your data? Select a provider guide above to begin.


# Cloudflare R2
Source: https://docs.mixpeek.com/docs/integrations/object-storage/r2

Connect Mixpeek to your Cloudflare R2 buckets to ingest and process your data.

> [!NOTE]
>
> This guide explains how to connect your Cloudflare R2 storage buckets to Mixpeek, enabling automated data ingestion and processing. R2 is S3-compatible, so the connection process is very similar.

## Prerequisites

* An active Cloudflare account.
* An R2 bucket containing the data you want Mixpeek to process.
* Permissions to create API Tokens in your Cloudflare account with R2 access.

## Configuration Steps

Connecting Mixpeek to R2 requires granting Mixpeek read access to your bucket.

**1. Create R2 API Token**

First, create an API Token that grants the necessary permissions for Mixpeek to list and read objects from your R2 bucket.

1. Navigate to your Cloudflare Dashboard.
2. Go to **My Profile** > **API Tokens**.
3. Click **Create Token**.
4. You can use the "Read all R2 Buckets" template or create a custom token.
5. For a custom token, ensure you grant at least `Object Read` and `Bucket Read` permissions for the specific bucket(s) Mixpeek will access.
   * **Permissions**:
     * Account > R2 > Read
   * **Account Resources**:
     * Include > Specific account > Your Account
   * **Bucket Resources** (if you want to limit access to specific buckets):
     * Include > Specific bucket(s) > Your R2 Bucket Name(s)
6. Continue to summary and click **Create Token**.
7. **Important:** Copy the **Access Key ID** and **Secret Access Key** displayed. You will also need your **Account ID**. You will need these to configure the connection in Mixpeek. Store them securely. Cloudflare will only show you the Secret Access Key once.

**2. Add R2 Connection in Mixpeek**

1. Navigate to the **Integrations** or **Data Sources** section in your Mixpeek dashboard (or Mixpeek Studio).
2. Click **Add Connection** or **New Source** and select **Cloudflare R2** (or **S3-Compatible Storage** if a dedicated R2 option is not available).
3. Enter the required details:
   * **Bucket Name:** The name of your R2 bucket.
   * **Endpoint URL:** Your R2 S3 API endpoint. This looks like `https://<ACCOUNT_ID>.r2.cloudflarestorage.com`. Replace `<ACCOUNT_ID>` with your Cloudflare Account ID.
   * **Access Key ID:** The Access Key ID from the API Token you created.
   * **Secret Access Key:** The Secret Access Key from the API Token you created.
   * Optionally, specify a **Prefix** if you only want Mixpeek to process files within a specific folder in your bucket.
4. Click **Test Connection** (if available) to verify the credentials and permissions.
5. Click **Save** or **Connect**.

## Verification

Once connected, Mixpeek should start discovering files in your specified R2 bucket (and prefix, if provided). You can monitor the ingestion status within the Mixpeek Studio. Depending on your pipeline configuration, feature extraction and indexing will begin automatically for supported file types.

If you encounter issues, double-check the API Token permissions and the credentials provided in Mixpeek. Ensure the bucket name and S3 endpoint URL (including your Account ID) are correct.

## Conceptual Diagram: Mixpeek and R2 Integration

```mermaid theme={null}
graph TD
    R2Bucket[R2 Bucket] -- Contains Objects (Files) --> R2Bucket;
    MixpeekPlatform[Mixpeek Platform] -- "Connects via S3 API using API Token credentials" --> R2Bucket;
    MixpeekPlatform -- "Retrieves & Processes Each Object" --> ProcessedData[Processed & Indexed Data];
    MixpeekPlatform -- "Exposes" --> SearchAPI[Mixpeek Search API];
    YourApp[Your Application / Users] -- "Sends Search Queries" --> SearchAPI;
    SearchAPI -- "Returns Results" --> YourApp;

    subgraph "Mixpeek Infrastructure"
        direction LR
        MixpeekPlatform
        ProcessedData
        SearchAPI
    end

    style R2Bucket fill:#F6821F,stroke:#333,stroke-width:2px
    style MixpeekPlatform fill:#4CAF50,stroke:#333,stroke-width:2px
    style ProcessedData fill:#2196F3,stroke:#333,stroke-width:2px
    style SearchAPI fill:#7E57C2,stroke:#333,stroke-width:2px
    style YourApp fill:#00BCD4,stroke:#333,stroke-width:2px
```

This diagram illustrates the workflow:

1. Your data (objects/files) resides in your **R2 Bucket**.
2. The **Mixpeek Platform** securely connects to your R2 Bucket using the S3 API, authenticating with the API Token credentials you provide (as detailed in the configuration steps).
3. Mixpeek retrieves each object from the bucket, performs its processing and indexing routines, creating **Processed & Indexed Data** within its infrastructure.
4. Mixpeek then exposes a **Mixpeek Search API**.
5. **Your Application or Users** interact with this API by sending search queries.
6. The API processes these queries against the indexed data and **Returns Results** to your application or users.


# AWS S3
Source: https://docs.mixpeek.com/docs/integrations/object-storage/s3

Connect Mixpeek to your Amazon S3 buckets to ingest and process your data.

> [!NOTE]
>
> This guide explains how to connect your AWS Simple Storage Service (S3) buckets to Mixpeek, enabling automated data ingestion and processing.

## Prerequisites

* An active AWS account.
* An S3 bucket containing the data you want Mixpeek to process.
* Permissions to create IAM policies and users/roles in your AWS account.

## Configuration Steps

Connecting Mixpeek to S3 requires granting Mixpeek read access to your bucket. We recommend using an IAM Role for enhanced security, but you can also use an IAM User with access keys.

**1. Create IAM Policy**

First, create an IAM policy that grants the necessary permissions for Mixpeek to list and read objects from your S3 bucket.

1. Navigate to the IAM service in your AWS Console.
2. Go to **Policies** and click **Create policy**.
3. Switch to the **JSON** tab and paste the following policy document. Remember to replace `YOUR_BUCKET_NAME` with the actual name of your S3 bucket.

   ```json theme={null}
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket"
               ],
               "Resource": [
                   "arn:aws:s3:::YOUR_BUCKET_NAME"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:GetObject"
               ],
               "Resource": [
                   "arn:aws:s3:::YOUR_BUCKET_NAME/*"
               ]
           }
       ]
   }
   ```
4. Click **Next: Tags**, then **Next: Review**.
5. Give the policy a descriptive name (e.g., `MixpeekS3ReadAccessPolicy`) and click **Create policy**.

**2. Optional: allow writes for export destinations**

Read access is all Mixpeek needs to ingest from your bucket. Add write access only if
you want to use this connection as an **export destination**, where Mixpeek writes a
dataset export and its `manifest.json` into your bucket.

Add one statement to the policy above:

```json theme={null}
{
    "Effect": "Allow",
    "Action": [
        "s3:PutObject"
    ],
    "Resource": [
        "arn:aws:s3:::YOUR_BUCKET_NAME/*"
    ]
}
```

Then turn the capability on for that connection, which is off by default:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/organizations/connections/conn_xxx \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"write_enabled": true}'
```

Both are required, and they are independent: `write_enabled` is your permission for
Mixpeek to try, and `s3:PutObject` is AWS letting it succeed. An export naming a
connection without the flag is refused before anything is written, and the error names
the permission to add. Connections you never enable stay read-only.

**3. Create IAM User or Role**

Choose one of the following authentication methods:

**Option A: IAM User (Simpler Setup)**

1. In the IAM service, go to **Users** and click **Add users**.
2. Enter a user name (e.g., `mixpeek-s3-user`).
3. Select **Access key - Programmatic access** as the AWS credential type.
4. Click **Next: Permissions**.
5. Choose **Attach existing policies directly**.
6. Search for and select the policy you created in the previous step (e.g., `MixpeekS3ReadAccessPolicy`).
7. Click **Next: Tags**, then **Next: Review**, then **Create user**.
8. **Important:** Copy the **Access key ID** and **Secret access key**. You will need these to configure the connection in Mixpeek. Store them securely.

**Option B: IAM Role (Recommended for Security)**

* Follow AWS best practices to create an IAM role that Mixpeek can assume. This typically involves setting up a trust policy allowing Mixpeek's AWS account (or a specific identifier provided by Mixpeek) to assume the role.
* Attach the `MixpeekS3ReadAccessPolicy` created in the previous step to this role.
* You will use the **Role ARN** when configuring the connection in Mixpeek instead of access keys. (Consult Mixpeek's specific requirements for Role ARN setup if available).

**4. Add S3 Connection in Mixpeek**

1. Navigate to the **Integrations** or **Data Sources** section in your Mixpeek dashboard (or Mixpeek Studio).
2. Click **Add Connection** or **New Source** and select **AWS S3**.
3. Enter the required details:
   * **Bucket Name:** The name of your S3 bucket (e.g., `YOUR_BUCKET_NAME`).
   * **Region:** The AWS region where your bucket is located (e.g., `us-east-1`).
   * **Authentication:**
     * If using an IAM User: Provide the **Access Key ID** and **Secret Access Key** obtained previously.
     * If using an IAM Role: Provide the **Role ARN** obtained previously.
   * Optionally, specify a **Prefix** if you only want Mixpeek to process files within a specific folder in your bucket.
4. Click **Test Connection** (if available) to verify the credentials and permissions.
5. Click **Save** or **Connect**.

## Verification

Once connected, Mixpeek should start discovering files in your specified S3 bucket (and prefix, if provided). You can monitor the ingestion status within the Mixpeek Studio. Depending on your pipeline configuration, feature extraction and indexing will begin automatically for supported file types.

If you encounter issues, double-check the IAM policy permissions and the credentials provided in Mixpeek. Ensure the bucket name and region are correct.


# SharePoint
Source: https://docs.mixpeek.com/docs/integrations/object-storage/sharepoint

Connect SharePoint sites and OneDrive for Business document libraries to a Mixpeek bucket.

> [!NOTE]
>
> Mixpeek reads document libraries through the Microsoft Graph API. Each file becomes a bucket object.

## Overview

SharePoint nests four levels: tenant, site, drive (document library), item. A connection pins the first three, and the sync path selects the folder.

| Field | Selects |
| - | - |
| `site_id` | The SharePoint site |
| `drive_id` | A document library inside that site |
| `folder_path` | A folder inside that library |

## Prerequisites

* An Azure AD application registration.
* Microsoft Graph permissions `Sites.Read.All` and `Files.Read.All`.
* Admin consent, for the client-credentials flow.

## Authentication

Two flows, chosen by the `type` field on `credentials`.

**`client_credentials`** — app-only access with no user in the loop. Use this in production.

| Field | Required |
| - | - |
| `tenant_id` | Yes |
| `client_id` | Yes |
| `client_secret` | Yes |

**`delegated`** — acts as a user, limited to that user's permissions. Needs refresh-token upkeep.

| Field | Required |
| - | - |
| `tenant_id` | Yes, or `common` for a multi-tenant app |
| `client_id` | Yes |
| `client_secret` | Yes |
| `refresh_token` | Yes |

> [!WARNING]
>
> `client_secret` and `refresh_token` are encrypted at rest and never returned in a response.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `credentials` | Yes | One of the two flows above |
| `site_id` | No | Omit for personal OneDrive |
| `drive_id` | No | Omit to use the site's default document library |
| `folder_path` | No | Omit to sync from the drive root |

### Sync-level fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | For example `/sites/SiteName/Shared Documents/folder` |
| `polling_interval_seconds` | No | Defaults to `300` |

## Setup

**1. Register the Azure AD application**

1. Open **Azure Portal → App registrations → New registration**.
2. Copy the **Application (client) ID** and **Directory (tenant) ID**.
3. Under **Certificates & secrets**, create a client secret and copy its value.
4. Under **API permissions**, add `Sites.Read.All` and `Files.Read.All`, then grant admin consent.

**2. Find your site and drive IDs**

```bash cURL theme={null}
curl -H "Authorization: Bearer YOUR_GRAPH_TOKEN" \
  "https://graph.microsoft.com/v1.0/sites/contoso.sharepoint.com:/sites/Marketing"

curl -H "Authorization: Bearer YOUR_GRAPH_TOKEN" \
  "https://graph.microsoft.com/v1.0/sites/SITE_ID/drives"
```

A site ID looks like `contoso.sharepoint.com,guid1,guid2`. A drive ID starts with `b!`.

**3. Create the storage connection**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Marketing SharePoint",
    provider_type="sharepoint",
    provider_config={
        "credentials": {
            "type": "client_credentials",
            "tenant_id": "your-tenant-id",
            "client_id": "your-client-id",
            "client_secret": "your-client-secret",
        },
        "site_id": "contoso.sharepoint.com,guid1,guid2",
        "drive_id": "b!your-drive-id",
        "folder_path": "/Shared Documents/Marketing",
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Marketing SharePoint',
  provider_type: 'sharepoint',
  provider_config: {
    credentials: {
      type: 'client_credentials',
      tenant_id: 'your-tenant-id',
      client_id: 'your-client-id',
      client_secret: 'your-client-secret',
    },
    site_id: 'contoso.sharepoint.com,guid1,guid2',
    drive_id: 'b!your-drive-id',
    folder_path: '/Shared Documents/Marketing',
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Marketing SharePoint",
    "provider_type": "sharepoint",
    "provider_config": {
      "credentials": {
        "type": "client_credentials",
        "tenant_id": "your-tenant-id",
        "client_id": "your-client-id",
        "client_secret": "your-client-secret"
      },
      "site_id": "contoso.sharepoint.com,guid1,guid2",
      "drive_id": "b!your-drive-id",
      "folder_path": "/Shared Documents/Marketing"
    }
  }'
```

**4. Browse folders before creating a sync**

```bash cURL theme={null}
curl "https://api.mixpeek.com/v1/organizations/connections/conn_your_connection_id/folders" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY"
```

**5. Create the sync**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "/sites/Marketing/Shared Documents/Campaigns",
    "polling_interval_seconds": 1800
  }'
```

## Related

> **[Google Drive](/docs/integrations/object-storage/google-drive)**
>
> The same file-library pattern on Google Workspace.

> **[Box](/docs/integrations/object-storage/box)**
>
> Enterprise content management connector.


# Supabase Storage
Source: https://docs.mixpeek.com/docs/integrations/object-storage/supabase

Sync files from Supabase Storage buckets into Mixpeek using the S3-compatible API.

> [!NOTE]
>
> Supabase Storage exposes an S3-compatible API that Mixpeek talks to directly. You can authenticate with dedicated S3 access keys (recommended) or with a project's `anon` + `service_role` JWTs in session-token mode.

## Overview

The Supabase integration lets Mixpeek pull objects from any Supabase Storage bucket into your Mixpeek buckets for processing. Each file becomes a Mixpeek bucket object, automatically batched into the collections that source the bucket.

Mixpeek derives the endpoint from your project reference — the short URL hash in your project's Dashboard URL (`https://<ref>.supabase.co`). You do not need to manage endpoints by hand.

## Prerequisites

* A Supabase project with at least one Storage bucket.
* Either:
  * **S3 access keys** (recommended): a dedicated key pair generated in **Project Settings → Storage → S3 Access Keys**, or
  * **Session-token credentials**: the project's `anon` key and `service_role` key from **Project Settings → API**.

## Configuration

### Connection-Level Fields

| Field | Required | Description |
| - | - | - |
| `project_ref` | Yes | Supabase project reference (the short hash in `https://<ref>.supabase.co`) |
| `region` | Yes | Region that hosts the project (e.g., `us-east-2`, `eu-west-1`) |
| `credentials.type` | Yes | `access_key` or `session_token` |
| `credentials.access_key_id` | for `access_key` | S3 access key ID generated in Supabase |
| `credentials.secret_access_key` | for `access_key` | S3 secret access key — encrypted at rest |
| `credentials.anon_key` | for `session_token` | Project `anon` JWT |
| `credentials.service_role_key` | for `session_token` | Project `service_role` JWT — encrypted at rest |

### Sync-Level Fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | `<supabase-bucket>/<optional-prefix>` — e.g. `user-uploads/photos/` |
| `sync_mode` | No | `continuous` (default) or `initial_only` |
| `polling_interval_seconds` | No | Seconds between polls in `continuous` mode |
| `include_patterns` | No | Glob patterns to include (e.g. `["*.mp4", "*.jpg"]`) |
| `exclude_patterns` | No | Glob patterns to exclude |
| `modified_since` | No | ISO 8601 timestamp; only sync files modified after this date |

## Setup

**1. Generate credentials in Supabase**

**Option A — Dedicated S3 keys (recommended):**

1. Open your project in the [Supabase Dashboard](https://supabase.com/dashboard).
2. Go to **Project Settings → Storage → S3 Access Keys**.
3. Click **New access key**, give it a name (e.g. `mixpeek-sync`) and copy the `access_key_id` + `secret_access_key` immediately — the secret is shown only once.

**Option B — Session-token credentials:**

1. Go to **Project Settings → API**.
2. Copy the `anon` key and the `service_role` key. Treat `service_role` like a password — it has full access to Storage.

> [!WARNING]
>
> Session-token mode is convenient when you don't have permission to mint S3 keys, but the `service_role` JWT is a superuser credential. Prefer dedicated S3 keys scoped to the buckets you actually need.

**2. Create the storage connection in Mixpeek**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Supabase Uploads",
    provider_type="supabase",
    provider_config={
        "project_ref": "abcdefghijklmnopqrst",
        "region": "us-east-2",
        "credentials": {
            "type": "access_key",
            "access_key_id": "supabase_ak_...",
            "secret_access_key": "supabase_sk_...",
        },
    },
)
print(f"Connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Supabase Uploads',
  provider_type: 'supabase',
  provider_config: {
    project_ref: 'abcdefghijklmnopqrst',
    region: 'us-east-2',
    credentials: {
      type: 'access_key',
      access_key_id: 'supabase_ak_...',
      secret_access_key: 'supabase_sk_...',
    },
  },
})
console.log('Connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Supabase Uploads",
    "provider_type": "supabase",
    "provider_config": {
      "project_ref": "abcdefghijklmnopqrst",
      "region": "us-east-2",
      "credentials": {
        "type": "access_key",
        "access_key_id": "supabase_ak_...",
        "secret_access_key": "supabase_sk_..."
      }
    }
  }'
```

To use session-token mode instead, swap the `credentials` block:

```json theme={null}
{
  "type": "session_token",
  "anon_key": "eyJ...anon",
  "service_role_key": "eyJ...service_role"
}
```

**3. Create a sync configuration on your bucket**

```python Python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="user-uploads/photos/",
    sync_mode="continuous",
    polling_interval_seconds=3600,
)
print(f"Sync: {sync['sync_config_id']}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "user-uploads/photos/",
    "sync_mode": "continuous",
    "polling_interval_seconds": 3600
  }'
```

**4. Trigger your first sync**

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs/SYNC_CONFIG_ID/trigger \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id"
```

Mixpeek lists objects under your Supabase prefix, downloads each one, stores it in the Mixpeek-side S3 bucket, and creates a bucket object. Any collection that sources this bucket picks the new objects up on the next batch.

## Advanced Configuration

### File Filtering

```python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="user-uploads/",
    sync_mode="continuous",
    polling_interval_seconds=3600,
    include_patterns=["*.mp4", "*.mov", "*.jpg"],
    exclude_patterns=["*_thumbnail.*", "*.tmp"],
)
```

### Incremental Sync

Only sync files added or modified after a specific date:

```python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="user-uploads/",
    sync_mode="continuous",
    polling_interval_seconds=300,
    modified_since="2024-06-01T00:00:00Z",
)
```

## Source Path Format

| Format | Example | Description |
| - | - | - |
| `bucket/prefix` | `user-uploads/photos/` | Preferred — Supabase bucket + prefix |
| `s3://bucket/prefix` | `s3://user-uploads/photos/` | S3-style URI also accepted |
| `bucket` | `user-uploads` | Entire Supabase bucket |

## Sync Modes

| Mode | Description | When to Use |
| - | - | - |
| `continuous` | Polls every `polling_interval_seconds` | Active uploads, near-real-time ingestion. Set a longer interval for a predictable batch cadence |
| `initial_only` | Single import, then completes | Historical backfills, migrations |

## Troubleshooting

> **Connection test returns 403 / SignatureDoesNotMatch**
>
> * With **access\_key** mode: the keys must come from **Project Settings → Storage → S3 Access Keys**, not the general Supabase API keys.
> * With **session\_token** mode: verify `anon_key` and `service_role_key` are both JWTs from the same project and that `project_ref` matches the project they belong to.
> * Confirm `region` matches the region shown in the Supabase Dashboard.

> **NoSuchBucket when triggering sync**
>
> The Supabase bucket in `source_path` does not exist or your credentials cannot see it:
>
> * Double-check the bucket name (case-sensitive).
> * For access keys with bucket scope, confirm the key grants access to the bucket you're syncing.

> **No files synced from a populated bucket**
>
> * The `source_path` prefix is too narrow — widen it, or drop the trailing prefix.
> * `include_patterns` may exclude everything — remove them temporarily to see all files.
> * Check the sync run's `matched` counter in the job log to confirm discovery.

> **Session-token auth works locally but fails in Mixpeek**
>
> Some Supabase projects rotate `anon`/`service_role` keys during upgrades. Refresh both JWTs from **Project Settings → API** and update the connection.

## Related

* [Bucket Syncs](/docs/api-reference/bucket-syncs/create-sync-configuration)
* [Storage Connections](/docs/api-reference/organization-connections/create-storage-connection)
* [Backblaze B2 Integration](/docs/integrations/object-storage/backblaze)
* [Tigris Integration](/docs/integrations/object-storage/tigris)


# Tigris
Source: https://docs.mixpeek.com/docs/integrations/object-storage/tigris

Connect Mixpeek to your Tigris buckets to ingest and process your data.

> [!NOTE]
>
> This guide explains how to connect your Tigris buckets to Mixpeek, enabling automated data ingestion and processing. Tigris is a globally distributed, multi-cloud object storage service with built-in support for the S3 API.

## Prerequisites

* An active Tigris account
* A Tigris bucket containing the data you want Mixpeek to process
* Access key and secret key from your Tigris account

## Configuration Steps

Connecting Mixpeek to Tigris requires granting Mixpeek read access to your bucket. Tigris uses S3-compatible authentication, making it straightforward to set up.

**1. Get Tigris Credentials**

To access your Tigris buckets, you'll need your access key and secret key:

1. Log in to your Tigris account
2. Navigate to your account settings
3. Generate or copy your access key and secret key
4. Store these credentials securely - you'll need them to configure the connection in Mixpeek

**2. Configure AWS CLI (Optional)**

If you want to test your Tigris connection locally first, you can configure the AWS CLI:

```bash theme={null}
# Configure AWS CLI
aws configure set aws_access_key_id <access-key>
aws configure set aws_secret_access_key <secret-key>
aws configure set region auto

# List buckets
aws s3 ls --endpoint-url https://t3.storage.dev

# Create a bucket
aws s3api create-bucket --bucket <bucket-name> --endpoint-url https://t3.storage.dev
```

This step is optional but can help verify your credentials are working correctly.

**3. Add Tigris Connection in Mixpeek**

1. Navigate to the **Integrations** or **Data Sources** section in your Mixpeek dashboard
2. Click **Add Connection** or **New Source** and select **Tigris**
3. Enter the required details:
   * **Bucket Name:** The name of your Tigris bucket
   * **Endpoint URL:** `https://t3.storage.dev`
   * **Access Key ID:** Your Tigris access key
   * **Secret Access Key:** Your Tigris secret key
   * Optionally, specify a **Prefix** if you only want Mixpeek to process files within a specific folder in your bucket
4. Click **Test Connection** to verify the credentials and permissions
5. Click **Save** or **Connect**

## Verification

Once connected, Mixpeek should start discovering files in your specified Tigris bucket (and prefix, if provided). You can monitor the ingestion status within the Mixpeek Studio. Depending on your pipeline configuration, feature extraction and indexing will begin automatically for supported file types.

If you encounter issues, double-check your Tigris credentials and ensure the bucket name is correct. Since Tigris is S3-compatible, you can use standard S3 tools and libraries to verify your bucket access.

## Features

* **Global Distribution:** Tigris automatically places your data close to users for low-latency access worldwide
* **Zero Egress Fees:** No per-GB charge for reading your data out
* **Strong Consistency:** Ensures you always get the correct version of your data
* **S3 Compatibility:** Works with all your familiar S3 tools and libraries


# Wasabi Hot Cloud Storage
Source: https://docs.mixpeek.com/docs/integrations/object-storage/wasabi

Connect Mixpeek to your Wasabi Hot Cloud Storage buckets to ingest and process your data.

> [!NOTE]
>
> This guide explains how to connect your Wasabi Hot Cloud Storage buckets to Mixpeek, enabling automated data ingestion and processing. Wasabi is S3-compatible, making the integration process straightforward.

## Prerequisites

* An active Wasabi account.
* A Wasabi bucket containing the data you want Mixpeek to process.
* Permissions to create IAM users and policies within your Wasabi account.

## Configuration Steps

Connecting Mixpeek to Wasabi requires granting Mixpeek read access to your bucket. This is done by creating an IAM user with appropriate permissions.

**1. Create IAM Policy in Wasabi**

First, create an IAM policy in your Wasabi console that grants Mixpeek the necessary permissions to list and read objects from your bucket.

1. Log in to the Wasabi Console.
2. Navigate to **IAM** from the left-hand menu.
3. Select **Policies** and click **Create Policy**.
4. Switch to the **JSON** tab and paste the following policy document. Remember to replace `YOUR_BUCKET_NAME` with the actual name of your Wasabi bucket.

   ```json theme={null}
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket"
               ],
               "Resource": [
                   "arn:aws:s3:::YOUR_BUCKET_NAME"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:GetObject"
               ],
               "Resource": [
                   "arn:aws:s3:::YOUR_BUCKET_NAME/*"
               ]
           }
       ]
   }
   ```
5. Click **Next: Tags** (optional), then **Next: Review**.
6. Give the policy a descriptive name (e.g., `MixpeekWasabiReadAccessPolicy`) and an optional description. Click **Create policy**.

**2. Create IAM User in Wasabi**

Next, create an IAM user and attach the policy you just created.

1. In the Wasabi IAM console, go to **Users** and click **Add User**.
2. Enter a user name (e.g., `mixpeek-wasabi-user`).
3. Select **Programmatic access (generates an access key ID and secret access key)** for the Access type.
4. Click **Next: Permissions**.
5. Choose **Attach existing policies directly**.
6. Search for and select the policy you created in the previous step (e.g., `MixpeekWasabiReadAccessPolicy`).
7. Click **Next: Tags** (optional), then **Next: Review**, then **Create user**.
8. **Important:** Copy the **Access key ID** and **Secret access key**. You will need these to configure the connection in Mixpeek. Store them securely, as the Secret Access Key will not be shown again.

**3. Add Wasabi Connection in Mixpeek**

1. Navigate to the **Integrations** or **Data Sources** section in your Mixpeek dashboard (or Mixpeek Studio).
2. Click **Add Connection** or **New Source** and select **Wasabi** or **S3-Compatible Storage**.
3. Enter the required details:
   * **Bucket Name:** The name of your Wasabi bucket (e.g., `YOUR_BUCKET_NAME`).
   * **Region:** The Wasabi service region where your bucket is located (e.g., `us-east-1`, `eu-central-1`). Refer to Wasabi documentation for the correct service URL endpoint to determine your region or use the specific endpoint.
   * **Endpoint URL (Optional but Recommended):** The S3 endpoint for your Wasabi region (e.g., `s3.wasabisys.com` or `s3.us-east-2.wasabisys.com`). Using the specific regional endpoint is best practice.
   * **Access Key ID:** The Access Key ID obtained from the Wasabi IAM user.
   * **Secret Access Key:** The Secret Access Key obtained from the Wasabi IAM user.
   * Optionally, specify a **Prefix** if you only want Mixpeek to process files within a specific folder in your bucket.
4. Click **Test Connection** (if available) to verify the credentials and permissions.
5. Click **Save** or **Connect**.

## Verification

Once connected, Mixpeek should start discovering files in your specified Wasabi bucket (and prefix, if provided). You can monitor the ingestion status within the Mixpeek Studio. Depending on your pipeline configuration, feature extraction and indexing will begin automatically for supported file types.

If you encounter issues, double-check the IAM policy permissions, the credentials provided in Mixpeek, and ensure the bucket name and region/endpoint are correct.

## Conceptual Diagram: Mixpeek and Wasabi Integration

```mermaid theme={null}
graph TD
    WasabiBucket[Wasabi Bucket] -- Contains Objects (Files) --> WasabiBucket;
    MixpeekPlatform[Mixpeek Platform] -- "Connects via S3 API using IAM User credentials" --> WasabiBucket;
    MixpeekPlatform -- "Retrieves & Processes Each Object" --> ProcessedData[Processed & Indexed Data];
    MixpeekPlatform -- "Exposes" --> SearchAPI[Mixpeek Search API];
    YourApp[Your Application / Users] -- "Sends Search Queries" --> SearchAPI;
    SearchAPI -- "Returns Results" --> YourApp;

    subgraph "Mixpeek Infrastructure"
        direction LR
        MixpeekPlatform
        ProcessedData
        SearchAPI
    end

    style WasabiBucket fill:#FF9900,stroke:#333,stroke-width:2px
    style MixpeekPlatform fill:#4CAF50,stroke:#333,stroke-width:2px
    style ProcessedData fill:#2196F3,stroke:#333,stroke-width:2px
    style SearchAPI fill:#7E57C2,stroke:#333,stroke-width:2px
    style YourApp fill:#00BCD4,stroke:#333,stroke-width:2px
```

This diagram illustrates the workflow:

1. Your data (objects/files) resides in your **Wasabi Bucket**.
2. The **Mixpeek Platform** securely connects to your Wasabi Bucket using the S3 API, authenticating with the IAM user credentials you provide (as detailed in the configuration steps).
3. Mixpeek retrieves each object from the bucket, performs its processing and indexing routines, creating **Processed & Indexed Data** within its infrastructure.
4. Mixpeek then exposes a **Mixpeek Search API**.
5. **Your Application or Users** interact with this API by sending search queries.
6. The API processes these queries against the indexed data and **Returns Results** to your application or users.


# Mixpeek + Snowflake
Source: https://docs.mixpeek.com/docs/integrations/snowflake-warehouse

Use Mixpeek for unstructured multimodal data, Snowflake for structured analytics

## Overview

Mixpeek and Snowflake serve complementary roles in the modern data stack. Mixpeek decomposes unstructured files (images, video, audio, PDFs) into structured features and searchable documents. Snowflake stores, governs, and analyzes structured data at scale. Together, they close the gap between raw multimodal content and business-ready analytics.

> **Mixpeek**
>
> Ingests unstructured files, extracts features (embeddings, transcripts, classifications, metadata), and powers multimodal retrieval.

> **Snowflake**
>
> Stores structured outputs, enforces governance, and drives dashboards, ML pipelines, and cross-functional analytics.

## Architecture

```
                        Mixpeek                              Snowflake
               +-----------------------+            +------------------------+
               |                       |            |                        |
  Files -----> |  Buckets & Collections|            |   Structured Tables    |
  (images,     |                       |            |                        |
   video,      |  Decompose files into |  export    |  - classifications     |
   audio,      |  features:            | ---------> |  - extracted metadata  |
   PDFs)       |   - embeddings        |            |  - taxonomy labels     |
               |   - transcripts       |            |  - document payloads   |
               |   - classifications   |            |                        |
               |   - metadata          |  enrich    |  Dashboards, BI, ML    |
               |                       | <--------- |  (feed back into       |
               |  Retrieval & Search   |            |   Mixpeek retrievers)  |
               +-----------------------+            +------------------------+
```

## Use Cases

### Export taxonomy classifications to Snowflake tables

After Mixpeek classifies your content with [taxonomies](/docs/enrichment/taxonomies), export the labels into Snowflake for reporting and governance.

### Feed extracted metadata into Snowflake dashboards

Mixpeek extracts rich metadata from every file it processes -- transcripts, detected objects, face identities, brand logos, audio fingerprints. Load these structured outputs into Snowflake and build dashboards in Tableau, Sigma, or Snowsight.

### Use Snowflake data to enrich Mixpeek retrievers

Pull structured attributes from Snowflake (pricing, inventory, customer segments) and attach them to Mixpeek documents with the [api-call](/docs/retrieval/stages/api-call) retriever stage, pointed at an endpoint that queries Snowflake. The [sql-lookup](/docs/retrieval/stages/sql-lookup) stage reads PostgreSQL connections only. See [Join Warehouse Metrics onto Results](/docs/retrieval/stages/sql-lookup#join-warehouse-metrics-onto-results).

## Quick Start

Export Mixpeek document metadata to a Snowflake table using the Mixpeek Python SDK and the Snowflake Connector.

**1. Install dependencies**

```bash theme={null}
pip install mixpeek snowflake-connector-python
```

**2. List documents from Mixpeek**

```python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# List documents from a collection
documents = client.collections.documents.list(
    collection_id="your-collection-id",
    page_size=100
)
```

**3. Write to Snowflake**

```python theme={null}
import snowflake.connector
import json

conn = snowflake.connector.connect(
    user="YOUR_USER",
    password="YOUR_PASSWORD",
    account="YOUR_ACCOUNT",
    warehouse="YOUR_WAREHOUSE",
    database="MIXPEEK_DATA",
    schema="PUBLIC"
)

cursor = conn.cursor()

# Create table if it does not exist
cursor.execute("""
    CREATE TABLE IF NOT EXISTS mixpeek_documents (
        document_id VARCHAR,
        source_url VARCHAR,
        content_type VARCHAR,
        metadata VARIANT,
        created_at TIMESTAMP_NTZ
    )
""")

# Insert each document
for doc in documents:
    cursor.execute(
        """
        INSERT INTO mixpeek_documents
            (document_id, source_url, content_type, metadata, created_at)
        VALUES (%s, %s, %s, PARSE_JSON(%s), %s)
        """,
        (
            doc.get("document_id"),
            doc.get("source", {}).get("url"),
            doc.get("content_type"),
            json.dumps(doc.get("metadata", {})),
            doc.get("created_at"),
        )
    )

conn.commit()
cursor.close()
conn.close()
```

> [!TIP]
>
> For production workloads, use Snowflake's `COPY INTO` with staged files or Snowpipe for continuous loading instead of row-by-row inserts.

## When to Use Each

| Capability | Mixpeek | Snowflake |
| - | - | - |
| Ingest unstructured files (video, images, audio, PDFs) | Yes | No |
| Extract features (embeddings, transcripts, classifications) | Yes | No |
| Multimodal semantic search | Yes | No |
| Structured SQL analytics | No | Yes |
| Data governance and access control | Document-level ACL | Role-based, column-level |
| Dashboard and BI integration | No | Yes (Snowsight, Tableau, etc.) |
| ML feature store | Embedding vectors | Tabular features |

> [!NOTE]
>
> Mixpeek handles everything before the data is structured. Snowflake handles everything after. Use both to get a complete pipeline from raw files to business insights.

## Related

* [Taxonomies](/docs/enrichment/taxonomies) -- classify content and export labels
* [SQL Lookup Stage](/docs/retrieval/stages/sql-lookup) -- query external databases from retriever pipelines
* [API Call Stage](/docs/retrieval/stages/api-call) -- call external APIs during retrieval
* [Webhooks](/docs/operations/webhooks) -- trigger Snowflake loads when Mixpeek processing completes


# Instagram
Source: https://docs.mixpeek.com/docs/integrations/social-media/instagram

Monitor public Instagram Business and Creator accounts using Mixpeek's Instagram integration.

> [!NOTE]
>
> The Instagram integration uses the **Business Discovery API** to fetch public media from other Instagram Business or Creator accounts. You authenticate with your own account, then monitor any number of public accounts.

## Prerequisites

* A **Facebook account** with a **Facebook Page** that has a linked **Instagram Business or Creator account**. This is your "viewer" account — you don't need to own the accounts you're monitoring.
* The Facebook/Instagram app must have the `instagram_basic`, `pages_show_list`, and `business_management` permissions.

> [!NOTE]
>
> You don't need any relationship with the accounts you monitor. Business Discovery works with any public Instagram Business or Creator account.

## How It Works

The Instagram integration uses Meta's Business Discovery API, which allows an authenticated Instagram Business Account to query public profile data and media from other Business or Creator accounts.

**Key concepts:**

* **Viewer Account** — Your authenticated Instagram Business Account. This is the account that makes API calls on your behalf.
* **Target Accounts** — The public Instagram accounts you want to monitor. Each target is configured as a separate sync config with the account's username as the `source_path`.
* **One Connection, Many Targets** — A single OAuth connection provides the viewer account. You can create unlimited sync configs to monitor different target accounts.

### Data Flow

```
Instagram OAuth → Connection (viewer: @your_business_account)
    │
    ├── Sync Config (source_path: "nike")
    │       → Business Discovery API → Media items
    │       → Download to S3 → Bucket Objects
    │       → Collection Pipeline → MVS Documents
    │
    ├── Sync Config (source_path: "adidas")
    │       → ...
    │
    └── Sync Config (source_path: "spotify")
            → ...
```

### What Gets Synced

For each media item on the target account, Mixpeek captures:

| Field | Description |
| - | - |
| **Media Content** | The image or video file, downloaded and stored in your bucket |
| **Media Type** | `IMAGE`, `VIDEO`, `CAROUSEL_ALBUM`, or `REEL` |
| **Caption** | The post's caption text |
| **Timestamp** | When the post was published |
| **Like Count** | Number of likes at sync time |
| **Comments Count** | Number of comments at sync time |
| **Permalink** | Direct link to the post on Instagram |

## Setup

**1. Create an Instagram Connection**

Navigate to **Connections** in Mixpeek Studio and click **Add Connection**. Select **Instagram** and complete the OAuth flow.

During OAuth, you'll be asked to grant permissions and select which Facebook Pages to share. Make sure to select a Page that has a linked Instagram Business Account.

> [!WARNING]
>
> If none of your Facebook Pages have a linked Instagram Business Account, the connection will fall back to your Facebook user profile and Business Discovery will not work. Ensure at least one Page has a linked IG Business Account in **Page Settings → Instagram**.

**2. Create a Bucket**

Create a bucket to store the synced Instagram media.

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

bucket = client.buckets.create(
    bucket_name="instagram-brand-monitor"
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "instagram-brand-monitor",
    "bucket_schema": {
      "properties": {
        "media_url": {"type": "image"},
        "caption": {"type": "text"}
      }
    }
  }'
```

**3. Add Sync Configs for Target Accounts**

Create a sync config for each Instagram account you want to monitor. The `source_path` is the target account's username.

```python Python theme={null}
# Monitor multiple brands
brands = ["nike", "adidas", "apple", "spotify", "netflix"]

for brand in brands:
    sync = client.buckets.syncs.create(
        bucket_id=bucket.bucket_id,
        connection_id="your-connection-id",
        source_path=brand,
        sync_mode="initial_only",
        batch_size=100
    )
    print(f"Created sync for @{brand}: {sync.sync_config_id}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/BUCKET_ID/syncs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "your-connection-id",
    "source_path": "nike",
    "sync_mode": "initial_only",
    "batch_size": 100
  }'
```

**Sync modes:**

* `initial_only` — Fetches all available media once.
* `continuous` — Periodically checks for new posts and syncs them incrementally.

**4. Trigger the Sync**

Trigger each sync config to start fetching media.

```python Python theme={null}
for sync_config_id in sync_config_ids:
    result = client.buckets.syncs.trigger(
        bucket_id=bucket.bucket_id,
        sync_config_id=sync_config_id
    )
    print(f"Triggered: {result.sync_job_id}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/BUCKET_ID/syncs/SYNC_CONFIG_ID/trigger \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE"
```

**5. Process with a Collection**

Create a collection with a feature extractor to process the synced media. The `multimodal_extractor` generates 1408-dimensional embeddings for both images and videos.

```python Python theme={null}
collection = client.collections.create(
    collection_name="instagram-multimodal",
    source={
        "type": "bucket",
        "bucket_ids": [bucket.bucket_id]
    },
    feature_extractor={
        "feature_extractor_name": "multimodal_extractor",
        "version": "v1",
        "parameters": {
            "run_multimodal_embedding": True
        }
    }
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/collections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: YOUR_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "instagram-multimodal",
    "source": {
      "type": "bucket",
      "bucket_ids": ["BUCKET_ID"]
    },
    "feature_extractor": {
      "feature_extractor_name": "multimodal_extractor",
      "version": "v1",
      "parameters": {
        "run_multimodal_embedding": true
      }
    }
  }'
```

Batch processing runs automatically when syncs complete. Documents are created in [MVS](https://mixpeek.com/mvs) with multimodal embeddings, video segments, thumbnails, and full lineage back to the original bucket object.

## Resilience

The Instagram sync provider includes built-in resilience for handling the Facebook Graph API at scale:

| Feature | Behavior |
| - | - |
| **Adaptive Page Size** | Starts at 25 items per request. Automatically halves on server errors (minimum 5), preventing failures on large accounts. |
| **Exponential Backoff** | Retries up to 3 times with exponential backoff and jitter on 5xx errors and network timeouts. |
| **Rate Limit Handling** | Respects `429 Retry-After` headers from the Graph API. The Instagram API allows \~200 calls per hour per account. |
| **Graph API Error Parsing** | Parses 400 error responses from the Graph API. Retries "reduce data" errors with smaller page sizes. Fails fast on non-retryable errors (invalid username, permission denied). |
| **Graceful Degradation** | If a page of results fails after all retries, items already synced from previous pages are preserved. |
| **CDN Download Retries** | Media downloads from Instagram's CDN retry independently with their own backoff logic. |

## Token Lifecycle

Instagram access tokens have a limited lifespan. The integration handles token management automatically:

1. **Short-lived token** (1 hour) — Obtained during the OAuth callback.
2. **Long-lived token** (60 days) — Exchanged automatically during the OAuth flow.
3. **Auto-refresh** — When a token is within 7 days of expiry, it's refreshed automatically before each sync execution.

> [!WARNING]
>
> If a token expires without being refreshed (e.g., no syncs run for 60+ days), you'll need to re-authenticate by creating a new connection.

## Limitations

* **Business/Creator accounts only** — Business Discovery only works with public Instagram Business or Creator accounts. Personal accounts cannot be discovered.
* **Public media only** — Only publicly visible posts are accessible. Stories, DMs, and private account content are not available.
* **Rate limits** — The Graph API allows approximately 200 calls per hour per authenticated account. With a default page size of 25, this supports syncing \~5,000 media items per hour.
* **No real-time updates** — Media is fetched on-demand when a sync is triggered. Use `continuous` sync mode for periodic polling.

## Use Cases

> **Brand Monitoring**
>
> Track competitor visual strategies across Instagram. Analyze creative trends, posting frequency, and content themes using multimodal search.

> **Influencer Analysis**
>
> Build a searchable database of influencer content. Find visually similar posts, track engagement patterns, and identify content themes.

> **Content Intelligence**
>
> Process Instagram media through feature extractors to detect objects, extract text from images, transcribe video audio, and generate semantic embeddings for search.

> **Creative Benchmarking**
>
> Compare visual content across brands. Use multimodal retrieval to find similar creative executions and track how visual trends evolve over time.


# Overview
Source: https://docs.mixpeek.com/docs/integrations/social-media/overview

Connect Mixpeek to social media platforms to ingest and analyze public media at scale.

> [!NOTE]
>
> Social media integrations allow you to monitor and analyze public media from other accounts using platform APIs. Mixpeek handles authentication, media download, and processing automatically.

Social media platforms are rich sources of multimodal content — images, videos, carousels, and metadata like captions, engagement metrics, and timestamps. By connecting Mixpeek to social media APIs, you can build brand monitoring, competitive analysis, and content intelligence pipelines without manual data collection.

## Why Connect Social Media?

> **Brand Monitoring**
>
> Track visual content from competitor and partner accounts. Analyze trends in imagery, messaging, and creative strategy across your industry.

> **Multimodal Analysis**
>
> Process images and videos through Mixpeek's feature extractors to generate embeddings, detect scenes, extract text, and classify content automatically.

> **Scalable Collection**
>
> Sync media from multiple accounts with built-in pagination, rate limit handling, and adaptive resilience for large accounts.

> **Unified Pipeline**
>
> Social media content flows through the same bucket, collection, and retriever pipeline as your other data sources — no special handling required.

## How It Works

Social media integrations use a **connection + sync config** architecture:

1. **Connection** — Authenticates with the platform via OAuth. One connection provides API access through your authenticated account.
2. **Sync Configs** — Each sync config targets a specific account to monitor. One connection can have many sync configs, each pointing at a different account.
3. **Sync Execution** — When triggered, Mixpeek fetches media from the target account, downloads content to your bucket (S3/LocalStack), and creates bucket objects with metadata.
4. **Processing** — Bucket objects flow through your collection pipeline for feature extraction, embedding generation, and indexing — just like any other data source.

```
Connection (OAuth)
    ├── Sync Config: @nike       → Bucket Objects → Collection → Documents
    ├── Sync Config: @adidas     → Bucket Objects → Collection → Documents
    └── Sync Config: @spotify    → Bucket Objects → Collection → Documents
```

## Supported Platforms

> **[Instagram](/docs/integrations/social-media/instagram)**
>
> Monitor public Instagram Business and Creator accounts via the Business Discovery API.

## Getting Started

1. **Authenticate** — Create a connection by completing the OAuth flow for your chosen platform.
2. **Add Accounts** — Create sync configs for each account you want to monitor.
3. **Trigger Sync** — Run an initial sync to pull media into your bucket.
4. **Process** — Set up a collection with a feature extractor to generate embeddings and index the content.
5. **Retrieve** — Build retrievers to search across your synced social media content.


# TikTok
Source: https://docs.mixpeek.com/docs/integrations/social-media/tiktok

Sync videos from a TikTok account into a Mixpeek bucket through the TikTok Content API.

> [!NOTE]
>
> Each video becomes a bucket object carrying its description, likes, views, shares, and comment count.

## Overview

Mixpeek calls the official TikTok Content API as the authorized account. It syncs that account's videos, not arbitrary public ones.

## Prerequisites

* A TikTok Developer account with an approved app.
* The `user.info.basic` and `video.list` scopes granted during authorization.
* An OAuth authorization completed for the account you want to sync.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `credentials.client_key` | Yes | App Client Key from the TikTok Developer Portal |
| `credentials.client_secret` | Yes | App Client Secret, encrypted at rest |
| `credentials.access_token` | Yes | Access token, valid for 24 hours |
| `credentials.refresh_token` | Yes | Used to mint a new access token, encrypted at rest |
| `credentials.open_id` | Yes | The account's `open_id` from the authorization response |
| `credentials.token_expires_at` | No | Access token expiry |
| `scopes` | No | Defaults to `["user.info.basic", "video.list"]` |

### Sync-level fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes by the API | Ignored by this provider, which uses the authorized `open_id` |
| `polling_interval_seconds` | No | Defaults to `300` |

> [!WARNING]
>
> The sync API requires `source_path`, and the TikTok provider does not read it. Pass `/` and select the account through `open_id` on the connection.

## Setup

**1. Create and approve a TikTok app**

1. Open the [TikTok Developer Portal](https://developers.tiktok.com).
2. Create an app and request Login Kit plus Content Posting API access.
3. Copy the **Client Key** and **Client Secret**.
4. Add `user.info.basic` and `video.list` to the app's scopes.

**2. Authorize the account**

Run the TikTok Login Kit OAuth flow. The callback returns an access token, a refresh token, and the account's `open_id`.

> [!WARNING]
>
> Access tokens last 24 hours. Store the refresh token; Mixpeek uses it to renew.

**3. Create the storage connection**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Brand TikTok Account",
    provider_type="tiktok",
    provider_config={
        "credentials": {
            "type": "oauth",
            "client_key": "your-client-key",
            "client_secret": "your-client-secret",
            "access_token": "your-access-token",
            "refresh_token": "your-refresh-token",
            "open_id": "your-open-id",
        },
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Brand TikTok Account',
  provider_type: 'tiktok',
  provider_config: {
    credentials: {
      type: 'oauth',
      client_key: 'your-client-key',
      client_secret: 'your-client-secret',
      access_token: 'your-access-token',
      refresh_token: 'your-refresh-token',
      open_id: 'your-open-id',
    },
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Brand TikTok Account",
    "provider_type": "tiktok",
    "provider_config": {
      "credentials": {
        "type": "oauth",
        "client_key": "your-client-key",
        "client_secret": "your-client-secret",
        "access_token": "your-access-token",
        "refresh_token": "your-refresh-token",
        "open_id": "your-open-id"
      }
    }
  }'
```

**4. Create the sync**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "/",
    "polling_interval_seconds": 3600
  }'
```

## Related

> **[Instagram](/docs/integrations/social-media/instagram)**
>
> The same account-scoped pattern on Instagram.

> **[Social media overview](/docs/integrations/social-media/overview)**
>
> How social connectors map to buckets.


# RTSP
Source: https://docs.mixpeek.com/docs/integrations/streaming/rtsp

Capture segments from a live RTSP camera or encoder into a Mixpeek bucket.

> [!NOTE]
>
> One connection is one camera or encoder. Mixpeek cuts the stream into fixed-length segments, and each completed segment becomes one bucket object.

## Overview

Each sync run opens the stream, captures `segment_seconds * segments_per_run` seconds, and closes it.

> [!WARNING]
>
> **Capture is windowed, not gapless.** Whatever the camera shows between runs is not captured. A sync polling every 300 seconds with a 30 second window keeps 10% of the wall clock.
>
> An RTSP connection samples the stream. It does not archive it.

Coverage is `segment_seconds * segments_per_run` divided by `polling_interval_seconds`. Raise either of the first two to widen it, or shorten the poll interval. A longer poll interval narrows coverage, because it is the divisor.

| Poll interval | `segment_seconds` | `segments_per_run` | Captured | Run holds the stream |
| - | - | - | - | - |
| 300s | 30 | 1 | 10% | 39s |
| 300s | 30 | 5 | 50% | 159s |
| 300s | 30 | 9 | 90% | 279s |

> [!WARNING]
>
> **Keep the run shorter than the interval.** A run holds the stream open for the whole capture plus about 9 seconds of setup. Runs overlap when that total exceeds `polling_interval_seconds`.
>
> This caps coverage below 100%. On a 300 second interval the ceiling is 97%; on a 60 second interval it is 84%.

## Prerequisites

* A camera or encoder reachable over `rtsp://` or `rtsps://`.
* Network access from Mixpeek to that host and port.
* The stream's username and password, if it is protected.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `url` | Yes | `rtsp://` or `rtsps://` endpoint, for example `rtsp://camera.example.com:554/stream1` |
| `credentials` | No | `{"type": "basic", "username": ..., "password": ...}` for a protected stream |
| `transport` | No | `tcp` (default) or `udp` |
| `segment_seconds` | No | 5 to 300, defaults to `30` |
| `segments_per_run` | No | 1 to 20, defaults to `1` |
| `connect_timeout_seconds` | No | 5 to 120, defaults to `15` |
| `scene_change_threshold` | No | 0 to 64. Omit to publish every segment |
| `gate_retriever_id` | No | Saved retriever used as the gate. Mutually exclusive with `gate_inference_name` |
| `gate_retriever_min_score` | No | Score the retriever's top result must reach. Unset keeps on any result |
| `gate_inference_name` | No | Real-time inference plugin that judges each segment |
| `gate_on_error` | No | `publish` (default) or `drop` |
| `gate_timeout_seconds` | No | 1 to 300, defaults to `30` |

### Sync-level fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | The stream path |
| `polling_interval_seconds` | No | Defaults to `300` |

## Credentials go in `credentials`, never in the URL

Most camera vendors document an RTSP URL as `rtsp://user:pass@host:554/stream`. Mixpeek rejects that form at validation time.

```json theme={null}
{
  "url": "rtsp://admin:hunter2@camera.example.com:554/stream1"
}
```

```
url must not embed credentials. Move the username and password into the
`credentials` field so the endpoint stays safe to log, then use the bare
host form, for example rtsp://camera.example.com:554/stream1.
```

A URL carrying `user:pass@` travels through every log line, error message, and connection listing that prints the endpoint. Splitting the fields keeps the endpoint safe to display.

Write it this way instead:

```json theme={null}
{
  "url": "rtsp://camera.example.com:554/stream1",
  "credentials": {
    "type": "basic",
    "username": "admin",
    "password": "hunter2"
  }
}
```

> [!WARNING]
>
> The scheme check rejects anything other than `rtsp://` and `rtsps://`. Many public camera feeds published as "streams" are HLS, which needs a different connector.

## `segment_seconds` is a target, not a guarantee

Segment cuts land on keyframes. A camera sending a keyframe every 3 seconds cannot produce a cut at an arbitrary second.

Measured against such a camera, a request for 10 second segments produced segments of 11.8, 9.0, and 12.0 seconds.

> [!TIP]
>
> Put the camera's keyframe interval in your budget before you size a downstream limit on segment length or file size.

## Choosing `segment_seconds`

Start at the default of 30.

**Capture cost does not depend on segment length.** Setup costs about 9.4 seconds per run whatever the configuration, and the marginal cost per segment is zero. Capture uses stream copy, so cutting more segments does not re-encode anything.

Measured against a live camera, four ways of capturing the same 24 seconds:

| Configuration | Wall clock | Overhead |
| - | - | - |
| 1 x 24s | 33.5s | 9.5s |
| 2 x 12s | 32.6s | 8.6s |
| 3 x 8s | 34.0s | 10.0s |
| 4 x 6s | 32.9s | 8.9s |

**The cost that scales is downstream.** Each segment is one object, one storage write, and one pipeline invocation. Halving segment length doubles the invocation count for the same footage. Size the segment against what your pipeline charges per invocation, not against capture.

**Shorten a segment to get a result sooner.** Latency to a first result is roughly `segment_seconds` plus 9.4 seconds plus your pipeline's time. A 30 second segment means you learn about a moment about 40 seconds after it happens.

**A static scene argues for longer segments.** Deduplication granularity is the segment, so a segment holding 5 seconds of activity still counts as changed.

> [!TIP]
>
> Ask the camera for its keyframe interval before picking a number below 30.
>
> ```bash theme={null}
> ffprobe -v error -select_streams v:0 -show_entries frame=key_frame \
>         -read_intervals '%+20' -of csv=p=0 rtsp://your-camera/stream
> ```
>
> A camera sending a keyframe every 10 seconds cannot produce 5 second segments.

## Dropping segments that do not matter

An overnight camera captures the same empty room every run. Two filters drop those before they cost extraction, and they run in that order.

**`scene_change_threshold` needs no model.** It is a Hamming distance over a 64-bit perceptual hash of each segment's first frame. A segment is dropped when its distance from the previous kept segment is at or below the number.

| Value | Drops |
| - | - |
| `0` | Only pixel-identical scenes |
| `5` | The same frame allowing for compression noise |
| Higher | Collapses more aggressively |

It costs a few milliseconds of CPU per segment, so it runs first and anything it drops never reaches the gate.

**A gate judges each segment**, and there are two ways to write one. They are mutually exclusive.

| Field | The gate is | Change it by |
| - | - | - |
| `gate_retriever_id` | A saved retriever. The segment becomes its query content, so the question is whether the segment resembles anything in the collections that retriever searches | Editing the retriever |
| `gate_inference_name` | A real-time inference plugin. Takes a segment, returns `keep` plus any fields you want on the object | Deploying the plugin |

Reach for the retriever first. The policy is then an object you can read, edit, and version per camera, and changing what a camera keeps does not need a deploy. A plugin is the right answer when the decision is not a similarity question.

`gate_retriever_min_score` keeps the segment when the retriever's top result scores at or above it. Leave it unset to keep on any result, which is right when the retriever's own stages already filter. It is a second filter on top of the retriever, not a replacement for one.

Omit all of these and every captured segment is published.

> [!WARNING]
>
> `gate_on_error` decides what happens when the gate itself fails, times out, or returns something unreadable. It defaults to `publish`, so a broken gate costs extraction rather than footage.
>
> Choose `drop` only when storing an unjudged segment is worse than losing it. A dropped segment of a live stream is gone, and the moment it covered cannot be recaptured.

Keep `gate_timeout_seconds` well below `segment_seconds * segments_per_run`, or the gate becomes the bottleneck and runs start overlapping.

## Setup

**1. Create the storage connection**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Loading Dock Camera",
    provider_type="rtsp",
    provider_config={
        "url": "rtsp://camera.example.com:554/stream1",
        "credentials": {
            "type": "basic",
            "username": "admin",
            "password": "your-camera-password",
        },
        "transport": "tcp",
        "segment_seconds": 30,
        "segments_per_run": 2,
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Loading Dock Camera',
  provider_type: 'rtsp',
  provider_config: {
    url: 'rtsp://camera.example.com:554/stream1',
    credentials: {
      type: 'basic',
      username: 'admin',
      password: 'your-camera-password',
    },
    transport: 'tcp',
    segment_seconds: 30,
    segments_per_run: 2,
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Loading Dock Camera",
    "provider_type": "rtsp",
    "provider_config": {
      "url": "rtsp://camera.example.com:554/stream1",
      "credentials": {
        "type": "basic",
        "username": "admin",
        "password": "your-camera-password"
      },
      "transport": "tcp",
      "segment_seconds": 30,
      "segments_per_run": 2
    }
  }'
```

**2. Test the connection**

Confirm Mixpeek can open the stream before you create a sync.

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections/conn_your_connection_id/test \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY"
```

A camera that does not start delivering within `connect_timeout_seconds` fails here rather than on the first sync run.

**3. Create a bucket whose schema declares the segment property**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "dock_camera",
    "bucket_schema": {
      "properties": {
        "segment": {"type": "video"},
        "captured_at": {"type": "datetime"},
        "segment_index": {"type": "integer"},
        "gate_applied": {"type": "boolean"},
        "motion_score": {"type": "number"}
      }
    }
  }'
```

The response carries `bucket_id`. The name `segment` is yours to choose, and the next three steps all have to use the same one.

**4. Create the sync, with a schema mapping**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "/stream1",
    "polling_interval_seconds": 60,
    "schema_mapping": {
      "mappings": {
        "segment": {
          "target_type": "blob",
          "blob_property": "segment",
          "blob_type": "video",
          "source": {"type": "file"}
        },
        "captured_at": {
          "target_type": "field",
          "source": {"type": "rtsp_field", "field": "captured_at"}
        },
        "segment_index": {
          "target_type": "field",
          "source": {"type": "rtsp_field", "field": "segment_index"}
        },
        "gate_applied": {
          "target_type": "field",
          "source": {"type": "rtsp_field", "field": "applied"}
        },
        "gate_top_score": {
          "target_type": "field",
          "source": {"type": "rtsp_field", "field": "top_score"}
        },
        "gate_match_count": {
          "target_type": "field",
          "source": {"type": "rtsp_field", "field": "match_count"}
        }
      }
    }
  }'
```

> [!WARNING]
>
> **Name the blob property, even though `schema_mapping` is optional.** Without it each segment lands on `content`, the default blob property, and not on the property your bucket schema declares.
>
> A bucket declaring a `segment` video property with `input_mappings` of `{"video": "segment"}` then passes collection validation and the batch matches nothing. You get a clean run over zero documents rather than an error.

## Getting capture and gate values onto the object

A segment carries more than video. Capture records where it came from and when; a gate records what it decided. None of that reaches the object unless a `schema_mapping` names it, and the bucket schema declares a property to hold it.

Use `rtsp_field` as the source type and name the field without a prefix.

| `field` | Comes from |
| - | - |
| `source_url` | Capture |
| `segment_index` | Capture |
| `segment_duration_seconds` | Capture |
| `captured_at` | Capture |
| `transport` | Capture |
| `applied` | The gate. `true` when it ran and kept the segment, `false` when it errored and `gate_on_error` published it |
| `top_score` | A retriever gate. The top match score, for example `0.62` |
| `match_count` | A retriever gate. How many results the retriever returned |
| `error` | The gate, when it failed |
| anything else the plugin returned besides `keep` | The gate |

```json theme={null}
"motion_score": {
  "target_type": "field",
  "source": {"type": "rtsp_field", "field": "motion_score"}
}
```

> [!TIP]
>
> **`gate_applied: false` is the useful one.** It records that the gate could not be reached and the segment was published unjudged.
>
> Without it an unjudged segment and a segment the gate passed look identical, so a run where the gate was down reads as a run where nothing moved.

> [!WARNING]
>
> `target_type` takes `field` or `blob`. There is no `metadata` value: a sync created with one saves, then returns 400 on every read and disappears from `syncs/list`.

## Searching what you captured

Segments in a bucket are not searchable yet. A collection processes them into documents, and a saved retriever searches those documents. That second half is why the first half exists.

```
camera -> connection -> sync -> bucket objects -> collection -> retriever
```

**1. Create a collection over the bucket**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/collections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "dock_camera_segments",
    "source": {
      "type": "bucket",
      "bucket_id": "bkt_your_bucket_id",
      "input_mappings": {"video": "segment"}
    },
    "features": ["video_search"]
  }'
```

`input_mappings` points the extractor at the blob property your sync writes. It has to match the `blob_property` you set in `schema_mapping`, or the collection processes nothing.

**2. Save a retriever over the collection**

A retriever needs `retriever_name` and `stages`, and each stage is `{"stage_name": ..., "config": {"stage_id": ..., "parameters": {...}}}`.

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "dock_camera_search",
    "collection_identifiers": ["dock_camera_segments"],
    "stages": [ ... ],
    "input_schema": {"query": {"type": "string", "required": true}}
  }'
```

See [Retrievers](/docs/retrieval/retrievers) for a complete `feature_search` stage, including the `feature_uri` to search against. `input_schema` is a flat dict of field names, not JSON Schema.

**3. Execute it**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/ret_your_retriever_id/execute \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{"inputs": {"query": "forklift near the loading bay"}}'
```

The response carries `documents`, each one a segment the retriever scored against your query.

**4. Record what the user did with the results**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/interactions \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "feature_id": "doc_the_segment_they_opened",
    "interaction_type": ["click"],
    "position": 0,
    "retriever_id": "ret_your_retriever_id",
    "execution_id": "exec_from_the_response",
    "query_snapshot": {"query": "forklift near the loading bay"}
  }'
```

Interactions are the signal a saved retriever learns from, which is the reason to save one rather than run an ad-hoc query. `retriever_id` and `query_snapshot` are optional and both are worth sending: without them an interaction cannot be tied back to what produced it.

> [!TIP]
>
> A saved retriever can also run at the other end of this pipeline. Set `gate_retriever_id` on the connection and it decides which segments are worth keeping in the first place, before extraction is paid for.

> [!WARNING]
>
> **The `schema_mapping` on your sync decides whether any of this finds anything.** With no mapping each segment lands on `content`, the default blob property, and the fields your mapping would have written are never written at all.
>
> A collection whose `input_mappings` name `segment` then processes nothing, and a retriever over that collection returns zero documents. Every step reports success.

## Alerting on what the gate kept

Once the verdict is a field on the document, an alert watches it. Create one with
`POST /v1/alerts`, pointed at a retriever whose filter names the verdict:

```bash theme={null}
curl -X POST "$MP_API_URL/v1/alerts" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "High-confidence camera matches",
    "source": "retriever",
    "retriever_id": "ret_...",
    "trigger_on": "results",
    "notification_config": {
      "channels": [{"channel_type": "webhook"}],
      "include_matches": true
    }
  }'
```

`name` and `notification_config` are required, `channels` needs at least one entry, and
`channel_type` is `webhook`, `slack` or `email`. `retriever_id` and `trigger_on` apply on
`source: "retriever"`; `trigger_on` is `results` or `no_results`.

### Filter the verdict, not the source

The retriever behind the alert should pre-filter on `gate_top_score` or `gate_match_count`.
A retriever filtered on `gate_top_score >= 0.5` returns the segments the gate kept with a
confident match, so the alert fires on those and stays quiet the rest of the time. At
`>= 0.99` it returns nothing, which is the same retriever telling you the threshold is the
thing you are tuning.

Filtering on `provider = rtsp` instead matches every segment the camera ever produced. The
alert then fires continuously, and a firing alert stops carrying information: you cannot tell
a real match from an alert that would fire on anything.

> [!NOTE]
>
> This is the second half of the gate's value. The gate decides what enters the bucket, and
> the verdict fields it writes are what lets a downstream alert be selective without
> re-running any inference. Dropping the segment and keeping the reason are the same
> mechanism used twice.

### Watch the objects, not a copy of them

The retriever above searches a collection you built over the bucket. You can point it at
the bucket itself:

```bash theme={null}
curl -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "bucket-object-match",
    "collection_identifiers": ["bkt_7b33c05c"],
    "input_schema": {"captured_at": {"type": "text", "required": true}},
    "stages": [{
      "stage_name": "verdict_filter",
      "config": {"stage_id": "attribute_filter", "parameters": {"conditions": {"AND": [
        {"field": "captured_at", "operator": "eq", "value": "{{INPUT.captured_at}}"},
        {"field": "gate_applied", "operator": "eq", "value": true},
        {"field": "gate_top_score", "operator": "gte", "value": 0.5}
      ]}}}
    }]
  }'
```

The response comes back with the bucket id resolved:

```json theme={null}
{"retriever_id": "ret_97bc40f1ff2379", "collection_identifiers": ["col_4422c585fd"]}
```

`col_4422c585fd` is `mvs_bucket_bkt_7b33c05c`, the collection a bucket's objects
materialise into. It carries the object's fields at the same paths the object uses, so
`captured_at`, `segment_index`, `gate_applied`, `gate_top_score` and `gate_match_count`
all filter there. Nothing about the API changes: you name the bucket, and the collection
is provisioned on the first reference.

Attach the alert to that collection, mapping the arriving object's `captured_at` into the
retriever's input so each run judges what just landed:

```bash theme={null}
curl -X PATCH "$MP_API_URL/v1/collections/col_4422c585fd" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{"alert_applications": [{
    "alert_id": "alt_87be1502f714",
    "execution_mode": "on_ingest",
    "input_mappings": [
      {"input_key": "captured_at",
       "source": {"source_type": "document_field", "path": "captured_at"}}
    ]
  }]}'
```

## From the alert to the footage an analyst reads

An alert that only emails leaves the analyst to go find the segment. A `batch` channel
processes the objects behind the matching documents through a collection, so the footage
that matched is the footage that gets embedded:

```bash theme={null}
curl -X POST "$MP_API_URL/v1/alerts" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "matched-footage",
    "source": "retriever",
    "retriever_id": "ret_97bc40f1ff2379",
    "trigger_on": "results",
    "notification_config": {
      "channels": [
        {"channel_type": "email"},
        {"channel_type": "batch", "config": {"collection_id": "col_ff02418ebf"}}
      ],
      "include_matches": true,
      "include_scores": true
    }
  }'
```

The target must be a user-defined collection sourced from the same bucket. A system
collection or a collection sourced from another collection is refused with a message
saying which.

> [!WARNING]
>
> **A sync feeds every collection on its bucket unless you say otherwise.** Batch
> creation discovers them, so a collection you meant to fill from an alert also receives
> every synced object, and the alert's selection stops meaning anything.
>
> Name what the sync feeds on the sync itself:
>
> ```json theme={null}
> {"batch_collection_ids": ["segments-index"]}
> ```
>
> Omit the field for the old behaviour, which is every collection on the bucket. The
> bucket's own `mvs_bucket_*` payload view stays in scope either way.

Then a second alert watches the collection the first one fills, which is where the
analyst is looking:

```bash theme={null}
curl -X PATCH "$MP_API_URL/v1/collections/col_ff02418ebf" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{"alert_applications": [{
    "alert_id": "alt_analysis",
    "execution_mode": "on_ingest",
    "input_mappings": [
      {"input_key": "marker", "source": {"source_type": "constant", "value": "ingest"}}
    ]
  }]}'
```

### Carry the camera and the capture time onto the analysis documents

A document in the target collection holds extractor output and lineage. It holds the object's own fields only when you list them in `field_passthrough` or set `include_all_source_fields`. Otherwise a document cannot say which camera or moment it came from, and you join back to the object through its lineage to find out.

Every synced object carries `provider` and `connection_id` at the top level. It also carries each field your `schema_mapping` wrote, such as `captured_at`. Name the ones you want in the collection's `feature_extractor`:

```json theme={null}
"field_passthrough": [
  {"source_path": "connection_id"},
  {"source_path": "captured_at"}
]
```

Each entry copies the value to a field of the same name on every document the collection writes. Add `target_path` to rename it. An object that does not carry the field leaves it off the document, unless you set `default` or `required`.

The entries apply to documents the collection writes after you add them. Documents already in the collection keep their old shape.

The whole chain, in the order a segment travels it:

| Step | Where it happens | What it leaves behind |
| - | - | - |
| Capture | the connection's ffmpeg window | a segment file |
| Gate | the saved retriever named in `gate_retriever_id` | `gate_applied`, `gate_top_score`, `gate_match_count` |
| Admission | the bucket | an object carrying those fields |
| Alert 1 | `mvs_bucket_<bucket_id>` | an execution row, an email, a batch |
| Analysis | the target collection | documents with embeddings and lineage to the object |
| Alert 2 | that collection | an execution row and a notification |

> [!NOTE]
>
> `GET /v1/alerts/{id}/executions` is the record of every firing, and a control alert is
> how you know a firing means something. Run the same filter at a bar nothing clears, and
> check that it stays at `triggered: false` in the same window the real one fires.

## Transport

`tcp` interleaves RTP over the RTSP control connection. It survives NAT and firewalls that drop the separate UDP ports, which is why it is the default.

Use `udp` when the camera or the network path does not handle interleaved TCP well.

## Related

> **[Sync configuration](/docs/platform/syncs)**
>
> Polling intervals, reconciliation, and scheduling.

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Stages, input schemas, and what execution returns.

> **[Interactions](/docs/retrieval/interactions)**
>
> The feedback signal a saved retriever learns from.

> **[Mux](/docs/integrations/object-storage/mux)**
>
> Video assets from a hosted platform rather than a live camera.


# BrightData
Source: https://docs.mixpeek.com/docs/integrations/web-data/brightdata

Collect web datasets and scraper results from BrightData and ingest them into Mixpeek.

> [!NOTE]
>
> BrightData is a web data platform with pre-built datasets (LinkedIn, Amazon, Google Maps, etc.) and custom web scrapers. Each sync triggers a new dataset snapshot, waits for it to be ready, and ingests every row as a bucket object.

## Overview

The BrightData integration connects Mixpeek to BrightData's Datasets API. When a sync runs, Mixpeek:

1. Triggers a new dataset snapshot for the configured dataset ID.
2. Polls the snapshot until status is `ready`.
3. Downloads the JSONL results.
4. Creates one bucket object per row, with the full JSON record as the object blob.

Each row's fields are stored as bucket object metadata, making them filterable and searchable alongside the extracted features from your collection pipeline.

## Prerequisites

* An active [BrightData](https://brightdata.com) account.
* A BrightData API token (found in Dashboard → Account → API Token).
* Access to the dataset(s) you want to sync (subscription required for most datasets).

## Configuration

### Connection-Level Fields

| Field | Required | Description |
| - | - | - |
| `api_token` | Yes | BrightData API token — encrypted at rest |
| `customer_id` | No | BrightData customer ID for zone-level auth |
| `default_output_format` | No | `jsonl` (default) or `json` |
| `country` | No | ISO 3166-1 alpha-2 code for geo-targeting (e.g., `us`) |

### Sync-Level Fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | BrightData dataset ID (e.g., `gd_l1vikfnt1wgvvqz95w`) |
| `sync_mode` | No | `continuous` (default) or `initial_only` |
| `polling_interval_seconds` | No | Seconds between polls in `continuous` mode |

> [!TIP]
>
> You can find a dataset's ID in the BrightData Marketplace under the dataset detail page. It starts with `gd_`.

## Setup

**1. Get your BrightData API token**

1. Log in to your [BrightData Dashboard](https://brightdata.com/cp/setting).
2. Go to **Account Settings → API Tokens**.
3. Create a new token or copy an existing one.

> [!WARNING]
>
> Keep your API token secret — it has full access to your BrightData account.

**2. Find your dataset ID**

1. Open the [BrightData Marketplace](https://brightdata.com/products/datasets).
2. Select the dataset you want to sync (e.g., LinkedIn Company Profiles, Amazon Products).
3. Copy the dataset ID from the URL or dataset detail page.

Common dataset IDs:

* LinkedIn Company Profiles: `gd_l1vikfnt1wgvvqz95w`
* Amazon Product Data: `gd_l7q7dkf244hwjntr0`
* Google Maps Business Data: `gd_l7q7dkf244hwjntr1`

**3. Create the storage connection in Mixpeek**

In [Studio](https://studio.mixpeek.com), open **Connections → Add connection** and pick **BrightData** from the storage provider list.

![Mixpeek Studio 'Select Storage Provider' modal with BrightData highlighted](https://mintcdn.com/mixpeek/Ldfu6RDMpHUxuaCg/assets/integrations/brightdata-select-provider.jpg?fit=max\&auto=format\&n=Ldfu6RDMpHUxuaCg\&q=85\&s=0aa2f07985fb58171d1dbda7de311771)

Or create it programmatically:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="BrightData Production",
    provider_type="brightdata",
    provider_config={
        "credentials": {
            "type": "api_token",
            "api_token": "your-brightdata-api-token",
        },
        "default_output_format": "jsonl",
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'BrightData Production',
  provider_type: 'brightdata',
  provider_config: {
    credentials: {
      type: 'api_token',
      api_token: 'your-brightdata-api-token',
    },
    default_output_format: 'jsonl',
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "BrightData Production",
    "provider_type": "brightdata",
    "provider_config": {
      "credentials": {
        "type": "api_token",
        "api_token": "your-brightdata-api-token"
      },
      "default_output_format": "jsonl"
    }
  }'
```

**4. Create a sync configuration on your bucket**

```python Python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    # source_path is the BrightData dataset ID
    source_path="gd_l1vikfnt1wgvvqz95w",
    sync_mode="continuous",
    polling_interval_seconds=86400,  # Daily
    batch_size=500,
)
print(f"Sync created: {sync['sync_config_id']}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "gd_l1vikfnt1wgvvqz95w",
    "sync_mode": "continuous",
    "polling_interval_seconds": 86400,
    "batch_size": 500
  }'
```

**5. Trigger your first sync**

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs/SYNC_CONFIG_ID/trigger \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id"
```

> [!NOTE]
>
> BrightData dataset snapshots can take from a few minutes to an hour depending on dataset size and your subscription tier. Mixpeek polls the snapshot status automatically and processes records as soon as they're ready.

## Advanced Configuration

### Geo-Targeting

Restrict data collection to a specific country:

```python theme={null}
connection = client.organizations.connections.create(
    name="BrightData US Only",
    provider_type="brightdata",
    provider_config={
        "credentials": {"type": "api_token", "api_token": "your-token"},
        "default_output_format": "jsonl",
        "country": "us",  # ISO 3166-1 alpha-2
    },
)
```

### Schema Mapping

Map BrightData record fields to Mixpeek document fields using the sync's `schema_mapping`:

```json theme={null}
{
  "mappings": {
    "content": {
      "target_type": "blob",
      "source": {"type": "file"},
      "blob_type": "auto"
    },
    "company_name": {
      "target_type": "field",
      "source": {"type": "tag", "key": "name"}
    },
    "industry": {
      "target_type": "field",
      "source": {"type": "tag", "key": "industry"}
    }
  }
}
```

### File Filters

Filter which records are ingested using standard Mixpeek file filter fields:

```json theme={null}
{
  "include_patterns": ["*.json"],
  "modified_after": "2024-01-01T00:00:00Z"
}
```

## Data Model

Each BrightData record becomes a Mixpeek bucket object with:

* **Blob:** Full JSON record body (stored as `application/json`)
* **Metadata:** Top-level string, integer, float, and boolean fields from the record
* **`source_provider`:** `brightdata`
* **`source_object_id`:** `<snapshot_id>:<record_id>` (deduplicated across syncs)

## Sync Modes

| Mode | Description | When to Use |
| - | - | - |
| `continuous` | Polls every `polling_interval_seconds` | Real-time monitoring, frequently updated datasets. Set a daily or weekly interval for periodic refreshes |
| `initial_only` | Single import, then completes | One-off data migrations, historical backfills |

> [!TIP]
>
> For most BrightData datasets (LinkedIn, Amazon, etc.), use `continuous` with a daily or weekly `polling_interval_seconds`, since the underlying data changes at that cadence.

## Troubleshooting

> **Snapshot timeout error**
>
> BrightData snapshots expire after 1 hour by default. If your dataset is large:
>
> * Reduce the number of records by adding geo-targeting (`country` field)
> * Use a higher-tier BrightData subscription with faster processing
> * Contact BrightData support to increase your snapshot limits

> **401 Unauthorized on connection test**
>
> Your API token may be invalid or revoked:
>
> 1. Go to BrightData Dashboard → Account → API Tokens
> 2. Verify the token is active
> 3. Create a new token if needed and update the connection

> **No records ingested after sync completes**
>
> * Verify the dataset ID in `source_path` is correct
> * Check that your BrightData subscription includes this dataset
> * Inspect the sync job logs in Mixpeek Studio for detailed error messages

> **Sync shows 'rate\_limit\_hits' in metrics**
>
> BrightData API has rate limits per subscription tier. Increase `polling_interval_seconds`
> or upgrade your BrightData plan for higher throughput.

## Related

* [Bucket Syncs](/docs/api-reference/bucket-syncs/create-sync-configuration)
* [Storage Connections](/docs/api-reference/organization-connections/create-storage-connection)
* [Bucket Uploads](/docs/ingestion/uploads)
* [Schema Mapping](/docs/ingestion/buckets)


# HTTP API
Source: https://docs.mixpeek.com/docs/integrations/web-data/http-api

Sync items from any JSON REST endpoint into a Mixpeek bucket, one object per item.

> [!NOTE]
>
> Each item in the API response becomes a bucket object. The item's JSON body is stored as the object blob.

## Overview

Mixpeek polls a JSON endpoint on a schedule. On each run it:

1. Calls the URL in `source_path` with the configured method and headers.
2. Reads the array of items at `items_path`.
3. Creates one bucket object per item, keyed on `item_id_field`.
4. Skips items whose ID already exists, unless you set `skip_duplicates` to `false`.

## Prerequisites

* An endpoint that returns JSON or newline-delimited JSON.
* An array of items, at the root or at a nested path.
* A field on each item that is unique and stable.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `item_id_field` | Yes | Field on each item used as the deduplication ID |
| `credentials.headers` | No | HTTP headers, for example `Authorization` or `X-API-Key` |
| `http_method` | No | Defaults to `GET` |
| `request_body` | No | JSON body for `POST` requests |
| `items_path` | No | Dot-notation path to the array, for example `data.results` |
| `item_modified_field` | No | Timestamp field for incremental sync (ISO 8601 or Unix epoch) |
| `response_content_type` | No | `json` (default) or `jsonl` |
| `user_agent` | No | Defaults to `Mixpeek HTTP API Sync/1.0` |
| `request_timeout` | No | Seconds, defaults to `30` |

### Sync-level fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | The full API URL to call |
| `polling_interval_seconds` | No | Defaults to `300` |
| `batch_size` | No | Defaults to `50` |

> [!TIP]
>
> Leave `items_path` unset when the response is a bare array. Set it to `hits` for `{"hits": [...]}`.

## Setup

**1. Create the storage connection**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Internal Orders API",
    provider_type="http_api",
    provider_config={
        "credentials": {
            "type": "http_header",
            "headers": {"Authorization": "Bearer your-api-token"},
        },
        "items_path": "data.orders",
        "item_id_field": "order_id",
        "item_modified_field": "updated_at",
    },
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Internal Orders API',
  provider_type: 'http_api',
  provider_config: {
    credentials: {
      type: 'http_header',
      headers: { Authorization: 'Bearer your-api-token' },
    },
    items_path: 'data.orders',
    item_id_field: 'order_id',
    item_modified_field: 'updated_at',
  },
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Internal Orders API",
    "provider_type": "http_api",
    "provider_config": {
      "credentials": {
        "type": "http_header",
        "headers": {"Authorization": "Bearer your-api-token"}
      },
      "items_path": "data.orders",
      "item_id_field": "order_id",
      "item_modified_field": "updated_at"
    }
  }'
```

**2. Create a sync on your bucket**

```python Python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="https://api.example.com/v2/orders",
    polling_interval_seconds=900,
    batch_size=100,
)
print(f"Sync created: {sync['sync_config_id']}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "https://api.example.com/v2/orders",
    "polling_interval_seconds": 900,
    "batch_size": 100
  }'
```

**3. Verify the connection**

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections/conn_your_connection_id/test \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY"
```

## Incremental sync

Set `item_modified_field` to a timestamp field. Mixpeek stores the highest value it has seen and requests only newer items on the next run.

Without it, every run reads the full response. Deduplication still applies, so repeated items do not create repeated objects.

## Related

> **[RSS](/docs/integrations/web-data/rss)**
>
> Sync RSS and Atom feeds.

> **[Sync configuration](/docs/platform/syncs)**
>
> Sync modes, reconciliation, and scheduling.


# RSS
Source: https://docs.mixpeek.com/docs/integrations/web-data/rss

Sync RSS and Atom feed entries into a Mixpeek bucket, one object per entry.

> [!NOTE]
>
> Each feed entry becomes a bucket object. Mixpeek polls the feed URL on the interval you set.

## Overview

Point a sync at a feed URL. Mixpeek reads the feed, creates one object per entry, and skips entries it has already seen.

Both RSS and Atom are supported.

## Prerequisites

* A reachable feed URL.
* HTTP headers for the feed, if it is private.

## Configuration

### Connection-level fields

| Field | Required | Description |
| - | - | - |
| `credentials.headers` | No | HTTP headers for private feeds, for example `Authorization` |
| `user_agent` | No | Defaults to `Mixpeek RSS Sync/1.0` |
| `request_timeout` | No | Seconds, defaults to `30` |

### Sync-level fields

| Field | Required | Description |
| - | - | - |
| `source_path` | Yes | The feed URL |
| `polling_interval_seconds` | No | Defaults to `300` |
| `batch_size` | No | Defaults to `50` |

> [!TIP]
>
> A public feed needs no credentials. Create the connection with an empty `provider_config` and put the URL on the sync.

## Setup

**1. Create the storage connection**

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-mixpeek-api-key")

connection = client.organizations.connections.create(
    name="Industry News Feeds",
    provider_type="rss",
    provider_config={},
)
print(f"Created connection: {connection['connection_id']}")
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

const connection = await client.organizations.connections.create({
  name: 'Industry News Feeds',
  provider_type: 'rss',
  provider_config: {},
})
console.log('Created connection:', connection.connection_id)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/connections \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Industry News Feeds",
    "provider_type": "rss",
    "provider_config": {}
  }'
```

For a private feed, pass the headers instead:

```json theme={null}
{
  "credentials": {
    "type": "http_header",
    "headers": {"Authorization": "Bearer your-feed-token"}
  }
}
```

**2. Create a sync per feed**

One connection can serve many feeds. Create one sync per feed URL.

```python Python theme={null}
sync = client.buckets.syncs.create(
    bucket_id="bkt_your_bucket_id",
    connection_id=connection["connection_id"],
    source_path="https://example.com/blog/feed.xml",
    polling_interval_seconds=3600,
)
print(f"Sync created: {sync['sync_config_id']}")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_your_connection_id",
    "source_path": "https://example.com/blog/feed.xml",
    "polling_interval_seconds": 3600
  }'
```

## Related

> **[HTTP API](/docs/integrations/web-data/http-api)**
>
> Sync items from any JSON REST endpoint.

> **[BrightData](/docs/integrations/web-data/brightdata)**
>
> Collect web datasets and scraper results.


# Agents
Source: https://docs.mixpeek.com/docs/overview/agents

Give AI agents tools to search, ingest, classify, and monitor multimodal content

![Agent integration: AI agents connect to Mixpeek via MCP, SDK, or REST to access ingest, search, classify, and monitor capabilities](https://mintcdn.com/mixpeek/gSJNHVjd4cCiZrjP/assets/mixpeek-agents.svg?fit=max\&auto=format\&n=gSJNHVjd4cCiZrjP\&q=85\&s=5ffca67f56b36f0f3a97c12acfe9bc67)

Mixpeek exposes its entire platform as agent-callable tools. Connect via MCP for zero-code setup, use the built-in Agent Runtime for stateful conversations, or wire retrievers into LangChain, OpenAI, or any framework via REST.

> **[Connect your agent to your own data](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_agents\&utm_campaign=managed_onramp)**
>
> Create a namespace to get your API key and MCP endpoint, then give your agent search, ingest, and classify tools over your own content.

## MCP (Model Context Protocol)

The fastest way to connect an AI agent to Mixpeek. Four hosted servers expose different tool scopes:

| Scope | URL | Use it for |
| - | - | - |
| **Full** | `https://mcp.mixpeek.com/mcp` | Everything below in one server |
| **Ingestion** | `https://mcp.mixpeek.com/ingestion/mcp` | Buckets, uploads, collections, batches |
| **Retrieval** | `https://mcp.mixpeek.com/retrieval/mcp` | Retrievers, search, agent sessions |
| **Admin** | `https://mcp.mixpeek.com/admin/mcp` | Namespaces, keys, taxonomies, clusters |

**Claude Desktop**

```json theme={null}
{
  "mcpServers": {
    "mixpeek": {
      "url": "https://mcp.mixpeek.com/mcp",
      "headers": { "Authorization": "Bearer YOUR_API_KEY" }
    }
  }
}
```

**Claude Code**

```bash theme={null}
claude mcp add mixpeek \
  --transport streamable-http \
  --url https://mcp.mixpeek.com/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"
```

**Cursor / Windsurf**

```json theme={null}
{
  "mcpServers": {
    "mixpeek": {
      "url": "https://mcp.mixpeek.com/mcp",
      "headers": { "Authorization": "Bearer YOUR_API_KEY" }
    }
  }
}
```

### Per-Retriever Server

For a focused search agent, scope the MCP server to a single retriever. It reads your retriever's `input_schema` and generates a typed `search` tool:

```bash theme={null}
pip install mixpeek-mcp-retriever

mixpeek-mcp-retriever \
  --retriever-id ret_xxx \
  --namespace-id ns_xxx \
  --api-key YOUR_API_KEY
```

Exposes three tools: `search` (typed to your schema), `describe` (retriever metadata), and `explain` (pipeline walkthrough).

[Full MCP reference →](/docs/agent-integrations/mcp)

## Agent Sessions

Mixpeek's built-in agent runtime gives you stateful, multi-turn conversations backed by your data. Each session runs as a dedicated process with tool access, conversation memory, and SSE streaming.

```bash theme={null}
# Create a session
curl -X POST "https://api.mixpeek.com/v1/agents/sessions" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_config": {
      "system_prompt": "You help users search and analyze video content.",
      "available_tools": ["execute_retriever", "list_collections", "get_taxonomy"]
    }
  }'
```

```bash theme={null}
# Send a message (SSE streaming)
curl -N -X POST "https://api.mixpeek.com/v1/agents/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{ "content": "Find videos about machine learning", "stream": true }'
```

The agent reasons through a `analyze → plan → execute → synthesize` workflow, calling tools as needed and streaming events back:

| Event | Description |
| - | - |
| `thinking` | Agent is analyzing or planning |
| `tool_call` | Agent is calling a tool |
| `tool_result` | Tool execution result |
| `message` | Response text chunk |
| `done` | Processing complete |

### Available Tools

| Tool | Description |
| - | - |
| `execute_retriever` | Search documents via a retriever pipeline |
| `search_retrievers` | Find available retrievers |
| `get_retriever` | Get retriever configuration |
| `list_collections` | List collections in the namespace |
| `get_collection` | Get collection details |
| `list_taxonomies` | List taxonomies |
| `get_taxonomy` | Get taxonomy details |
| `list_clusters` | List cluster configurations |
| `get_object` | Get object metadata |

Sessions persist for 7 days and automatically rehydrate after idle periods.

[Agent Sessions API →](/docs/api-reference/agent-sessions/create-session)

## LangChain

The `langchain-mixpeek` package provides a retriever, individual tools, and a full toolkit:

```bash theme={null}
pip install langchain-mixpeek
```

```python theme={null}
from langchain_mixpeek import MixpeekToolkit
from langgraph.prebuilt import create_react_agent
from langchain_anthropic import ChatAnthropic

toolkit = MixpeekToolkit(
    api_key="mxp_...",
    namespace="my-namespace",
    bucket_id="bkt_...",
    collection_id="col_...",
    retriever_id="ret_...",
)

agent = create_react_agent(
    ChatAnthropic(model="claude-sonnet-4-20250514"),
    toolkit.get_tools(),
)

result = agent.invoke({
    "messages": [("user", "Find product demos and summarize what's shown")]
})
```

### Toolkit Tools

| Tool | What it does |
| - | - |
| `mixpeek_search` | Search video, images, audio, documents |
| `mixpeek_ingest` | Upload content (text, images, video, audio, PDFs) |
| `mixpeek_process` | Trigger feature extraction |
| `mixpeek_classify` | Run taxonomy classification |
| `mixpeek_cluster` | Group similar documents |
| `mixpeek_alert` | Set up monitoring (webhook, Slack, email) |

Scope tools to what your agent needs with `toolkit.get_tools(actions=["search", "ingest"])`.

[Full LangChain guide →](/docs/agent-integrations/langchain)

## OpenAI Function Calling

Define a Mixpeek retriever as an OpenAI function schema:

```python theme={null}
from openai import OpenAI
from mixpeek import Mixpeek

openai_client = OpenAI()
mixpeek_client = Mixpeek(api_key="mxp_...")

tools = [{
    "type": "function",
    "function": {
        "name": "search_mixpeek",
        "description": "Search video, image, and audio content",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query"},
                "limit": {"type": "integer", "description": "Max results"}
            },
            "required": ["query"]
        }
    }
}]

response = openai_client.chat.completions.create(
    model="gpt-4o", messages=messages, tools=tools
)

# Handle tool_calls by calling mixpeek_client.retrievers.execute()
```

Works with both the Chat Completions API and Assistants API.

[Full OpenAI guide →](/docs/agent-integrations/openai-function-calling)

## Any Framework (REST)

The same pattern works with CrewAI, LlamaIndex, Haystack, Autogen, or plain HTTP — wrap the retriever execute endpoint as a tool:

```python theme={null}
import requests

def search(query: str, limit: int = 10) -> list:
    resp = requests.post(
        f"https://api.mixpeek.com/v1/retrievers/{RETRIEVER_ID}/execute",
        headers={
            "Authorization": f"Bearer {API_KEY}",
            "X-Namespace": NAMESPACE_ID,
            "Content-Type": "application/json",
        },
        json={"inputs": {"query_text": query}, "limit": limit},
    )
    return resp.json()["documents"]
```

[Retriever Execute API →](/docs/api-reference/retrievers/execute-retriever-auto-optimized)

## Choosing an Integration

| I want to... | Use |
| - | - |
| Connect Claude or Cursor with no code | [MCP](#mcp-model-context-protocol) |
| Build a stateful conversational agent | [Agent Sessions](#agent-sessions) |
| Build a LangChain/LangGraph agent | [LangChain](#langchain) |
| Add tools to GPT models | [OpenAI Function Calling](#openai-function-calling) |
| Use any other framework | [REST](#any-framework-rest) |


# Core Concepts
Source: https://docs.mixpeek.com/docs/overview/concepts

Understand the building blocks that power Mixpeek

Mixpeek is the **multimodal data warehouse** — a system that decomposes unstructured objects into queryable features, stores them across cost tiers, and reassembles them through multi-stage retrieval pipelines. This page introduces the warehouse primitives: the resources that make this work.

> **[Build with these primitives on your own data](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_concepts\&utm_campaign=managed_onramp)**
>
> Create a managed namespace to get a bucket, collection, and retriever of your own, then run the flow above on your files.

## The six objects, in one sentence each

If you remember nothing else, remember these:

| Object | In plain English |
| - | - |
| **Namespace** | Your project/environment boundary — everything lives inside one (think "database"). |
| **Bucket** | Where your raw files land before processing (think "inbox" / S3 folder). |
| **Object** | One raw item in a bucket — a video, image, PDF, or JSON record. |
| **Collection** | A recipe that turns objects into searchable **documents** by running the pipeline for the features you pick. |
| **Feature** | What you want extracted per modality (e.g. `video_search`, `faces`) — the plain-language capability you ask for. See [Features](/docs/processing/features). |
| **Extractor** | The model pipeline a feature runs under the hood (a Gemini embedder, Whisper, CLIP, …). You rarely name one directly — features resolve to extractors — but it's what a `feature_uri` points at when you search. See [Extractors](/docs/processing/feature-extractors). |
| **Retriever** | Your query pipeline — composable stages (search → filter → rerank → …) that return results. |

The flow: you put **objects** in a **bucket**, a **collection** extracts the **features** you picked to produce searchable **documents**, and a **retriever** queries those documents — all inside a **namespace**.

![Mixpeek flow: objects in a bucket are processed by a collection's feature pipeline into documents, queried by a retriever, all within a namespace](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-flow.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=ae6474d56d2179ffd66e0576c18e64a6)

## Entities & Relationships

The full resource model, including the operational and enrichment layers:

| Layer | Entity | What it Represents | Related APIs |
| - | - | - | - |
| Isolation | **Organization / API Key** | Authentication boundary (`Authorization: Bearer …`) | [API Keys](/docs/api-reference/organization-api-keys/list-api-keys) |
| Isolation | **Namespace** | Tenant or environment boundary (`X-Namespace`) | [Namespaces](/docs/vector-store/namespaces) |
| Storage | **Bucket** | Schema-validated container for objects | [Ingest Data](/docs/platform/data-model) |
| Storage | **Object** | Logical record referencing blobs (files/JSON) | [Ingest Data](/docs/platform/data-model) |
| Processing | **Batch** | Submission that feeds objects into collection processing | [Ingest Data](/docs/platform/data-model) |
| Processing | **Collection** | Document store + feature extraction recipe | [Extract Features](/docs/platform/processing) |
| Processing | **Feature** | What you want extracted per modality — resolved to a versioned pipeline internally | [Features](/docs/processing/features) (advanced pipeline config: [Feature Extractors](/docs/processing/feature-extractors)) |
| Retrieval | **Retriever** | Stage-based search pipeline | [Retrievers](/docs/retrieval/retrievers) |
| Enrichment | **Taxonomy** | Retrieval-backed enrichment recipe (flat or hierarchical) | [Taxonomies](/docs/enrichment/taxonomies) |
| Enrichment | **Cluster** | Vector-based grouping and enrichment artifacts | [Clusters](/docs/enrichment/clusters) |
| Operations | **Task** | Status wrapper for asynchronous jobs | [Tasks](/docs/processing/tasks) |
| Operations | **Webhook** | Event notification subscription | [Webhooks](/docs/platform/operations#webhooks) |

## Dual-ID Multi-Tenancy

Mixpeek separates authentication from authorization by using two IDs per organization:

* **`organization_id`** – Short, user-facing identifier returned in API responses
* **`internal_id`** – 24-character key used inside services, task payloads, and database documents

Namespaces are the primary isolation boundary. Every API request must include `X-Namespace` unless your organization has a single shared namespace. Enforced rules:

* All MongoDB collections index on `namespace_id`
* Each namespace maps to a dedicated [MVS](https://mixpeek.com/mvs) namespace (`ns_<namespace_id>`)
* Redis keys and Ray jobs include namespace prefixes
* Cross-namespace queries are not permitted by design

## Object → Document Lineage

Ingestion separates raw objects from processed documents so you can run multiple extraction tiers without duplicating data. Every document tracks:

```json theme={null}
{
  "root_object_id": "obj_video_123",
  "root_bucket_id": "bkt_marketing",
  "source_type": "collection",
  "source_collection_id": "col_frames",
  "source_document_id": "doc_frame_050",
  "lineage_path": "bkt_marketing/col_frames/col_scenes/col_highlights",
  "processing_tier": 3
}
```

* **Tier 0** – Raw object in the bucket
* **Tier N** – Document produced by another collection (`source_type = "collection"`)
* The `lineage_path` is a denormalized materialized path for fast queries
* Collections respect dependency tiers during extraction so downstream collections only execute when inputs are ready

Use the [Object Decomposition Tree](/docs/api-reference/document-lineage/get-decomposition-tree-visualization) endpoint to inspect the entire lineage for a given object.

## Feature URIs

Every feature a collection produces is addressed with a version-pinned URI that references the internal pipeline that produced it:

```
mixpeek://{extractor_name}@{version}/{output_name}
```

Examples:

* `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1`
* `mixpeek://image_extractor@v1/google_siglip_base_v1`
* `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding`

You don't construct these names yourself — discover the URIs a collection emits with `GET /v1/collections/{collection_id}/features`. Feature URIs are referenced by retriever stages (`feature_uri`), taxonomies, clustering jobs, and analytics. They guarantee query-time model compatibility with the ingestion pipeline.

## TaskStatusEnum Standard

All asynchronous operations—batches, clustering jobs, taxonomy materialization, namespace migrations—report status using the shared `TaskStatusEnum`:

```
PENDING → PROCESSING → COMPLETED
            ↘           ↘
            FAILED      COMPLETED_WITH_ERRORS
```

Terminal statuses are `COMPLETED`, `COMPLETED_WITH_ERRORS`, `FAILED`, and `CANCELED` — poll until any of these (see [Tasks](/docs/processing/tasks)). Additional lifecycle values include `IN_PROGRESS`, `SKIPPED`, `UNKNOWN`, `DRAFT`, `ACTIVE`, `ARCHIVED`, and `SUSPENDED`. Use the [Tasks API](/docs/processing/tasks) for short-term polling and fall back to the resource (e.g., batch or cluster) for long-running workflows.

## Caching Signatures

Mixpeek uses deterministic signatures to avoid stale results:

* Collection index signatures hash document count, vector dimensions, and schema state
* Retriever caches incorporate the collection signature to invalidate automatically
* Stage-level caches speed up pipelines that reuse expensive stages (KNN → rerank)
* Inference cache shortcuts repeated embedding requests for identical inputs

Learn more in [Caching](/docs/overview/caching).

## Putting It Together

```
Namespace
 └── Bucket
      ├── Object (Tier 0)
      └── Batch → Collection (Tier 1)
               └── Collection (Tier 2)
                    └── ...
```

* Documents retain lineage to the original object (`root_object_id`)
* Enrichment layers (taxonomies, clustering) augment documents in place
* Retrievers run on namespace-scoped data, returning results with presigned URLs, metrics, and cache hints

With these concepts in mind you can navigate deeper sections of the docs—whether you’re planning ingestion schemas, designing retriever pipelines, or wiring observability for production deployments.


# Data Model & Lineage
Source: https://docs.mixpeek.com/docs/overview/data-model

Understand how objects transform into documents and how lineage tracks provenance

Mixpeek's data model is built around a clear transformation pipeline: **Objects → Documents → Features**. Each entity has a specific purpose, and lineage metadata ensures you can always trace results back to their source.

## Core Entities

> **Objects**
>
> Raw inputs registered in buckets. Objects hold blobs (video, image, text, audio) and metadata but are **not processed** until added to a batch.

> **Documents**
>
> Processed representations produced by collection pipelines (the [features](/docs/processing/features) you enabled). Documents live in collections and include vectors, metadata, and lineage references.

> **Features**
>
> Extracted representations (embeddings, classifications, segments) stored as [MVS](https://mixpeek.com/mvs) vectors and referenced by feature URIs.

## Transformation Flow

![Mixpeek Transformation Flow](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-flow.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=ae6474d56d2179ffd66e0576c18e64a6)

### 1. Object Registration

Objects are created via the Buckets API and validated against the bucket's JSON schema:

```bash theme={null}
POST /v1/buckets/{bucket_id}/objects
```

**Object Structure:**

* `object_id` – unique identifier
* `bucket_id` – parent bucket reference
* `key_prefix` – logical path/grouping
* `blobs[]` – array of file references or inline data
* `metadata` – custom JSON validated against bucket schema
* `created_at` / `updated_at` – audit timestamps

Every bucket has a managed system collection, `mvs_bucket_<bucket_id>`, that
holds one document per object. Writing an object writes its document. Collections you define with
feature extractors are separate: an object becomes documents in them when a
batch processes it.

A bucket is addressable as a collection. Name a `bkt_...` id where a
`collection_id` goes and it resolves to `mvs_bucket_<bucket_id>`. See
[Bucket-backed collections](/docs/vector-store/system-collections#bucket-backed-collections).

### 2. Batch Processing

Batches group objects for efficient parallel processing:

```bash theme={null}
POST /v1/buckets/{bucket_id}/batches
POST /v1/buckets/{bucket_id}/batches/{batch_id}/submit
```

When submitted, the API:

1. Resolves all collections that consume the bucket
2. Generates per-extractor artifacts (manifests) and uploads to S3
3. Dispatches Ray tasks to the Engine with S3 artifact URIs

### 3. Document Creation

The Engine runs each collection's processing pipeline in parallel. For each object and each collection:

1. **Download manifest** – fetch pipeline config and input mappings
2. **Run the pipeline** – run model inference (embeddings, classifications, etc.)
3. **Write to MVS** – upsert vectors and payload with `internal_id` tenant filter
4. **Update metadata** – set `__fully_enriched`, `__pipeline_version`, `source_object_id`

**Document Structure:**

* `document_id` – MVS point ID
* `collection_id` – parent collection
* `source_object_id` – lineage back to originating object
* `root_object_id` – if object was derived, traces to original input
* `feature_refs[]` – array of feature URIs (e.g., `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1`)
* `metadata` – passthrough fields + enrichments from taxonomies/clusters
* `__fully_enriched` – boolean flag indicating the full pipeline succeeded
* `__missing_features` – array of feature addresses that failed
* `__pipeline_version` – integer tracking collection schema version

### 4. Feature Storage

Features are stored as [MVS](https://mixpeek.com/mvs) vectors with named indexes:

**Feature URI Format:**

```
mixpeek://<extractor_name>@<version>/<output_name>
```

Example:

```
mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1
mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding
mixpeek://image_extractor@v1/google_siglip_base_v1
```

These URIs are used in:

* Retriever stage configurations (`feature_uri`)
* Taxonomy input mappings
* Join enrichment strategies

## Lineage Tracking

Every document maintains lineage metadata for auditability and debugging:

Every field below lives under `_internal.lineage` on the document.

| Field | Purpose |
| - | - |
| `root_object_id` | The bucket object at the root of the tree |
| `root_bucket_id` | The bucket that object belongs to |
| `source_type` | `bucket` for a tier-0 document, `collection` for a derived one |
| `source_object_id` | The object this document came from. On a derived document it inherits the root |
| `source_document_id` | The upstream document, `null` on a tier-0 document |
| `source_collection_id` | The upstream collection, `null` on a tier-0 document |
| `chain` | One entry per processing step, each naming the collection, document and feature extractor |

### Querying Lineage

Use the Document Lineage API to trace provenance:

```bash theme={null}
# Full lineage for one document
GET /v1/collections/{collection_id}/documents/{document_id}/lineage

# Every collection and document derived from one object
GET /v1/objects/{object_id}/decomposition-tree

# Every document derived from one object, as a normal document list
POST /v1/collections/{collection_id}/documents/list
{"filters": {"AND": [
  {"field": "_internal.lineage.root_object_id", "operator": "eq", "value": "obj_..."}
]}}
```

The same filter works on any collection in the chain, so one object id reaches
its backing document and every document derived from it.

## Multi-Level Decomposition

Some pipelines produce **multiple documents per object** (e.g., video → scenes, PDF → pages):

```
Object: video_file.mp4
  └─ Document: full_video_summary
       ├─ Document: scene_001 (0:00-0:15)
       ├─ Document: scene_002 (0:15-0:32)
       └─ Document: scene_003 (0:32-1:00)
```

Each scene document includes:

* `_internal.lineage.root_object_id` → the `video_file.mp4` object
* `_internal.lineage.source_object_id` → the same object, inherited from the root
* `_internal.lineage.source_document_id` → the `full_video_summary` document
* `parent_document_id` → the same document, at the top level
* `segment_metadata` → `{ start_time: 0.0, end_time: 15.0 }`

## Tenant Isolation

All entities are scoped by:

* **`internal_id`** (organization) – injected at API layer from `Authorization` header
* **`namespace_id`** – resolved from `X-Namespace` header

MVS namespaces map 1:1 to namespaces, and all queries/writes automatically include `internal_id` filters. This ensures hard multi-tenancy without application-level filtering.

## Schema Evolution

Collections track `__pipeline_version` to handle schema changes:

1. Update collection definition (add/remove features, change mappings)
2. Increment `pipeline_version` automatically
3. New batches write documents with updated version
4. Old documents remain queryable with legacy schema
5. Optional: Trigger reprocessing to backfill with the new pipeline

## Feature Reuse Across Collections

Processing pipelines are **reusable**. Collections that enable the same feature emit the same feature URI, even when wired to different input fields (advanced per-pipeline config shown; the extractor names are deprecated aliases):

```json theme={null}
// Collection A: Product descriptions
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "input_mappings": { "text": "product_description" }
  }
}

// Collection B: Customer reviews
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "input_mappings": { "text": "review_text" }
  }
}
```

Both produce `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1` features, enabling **cross-collection search** via retrievers that span multiple collection IDs.

## Document Lifecycle States

| State | Meaning |
| - | - |
| `pending` | Batch submitted but Engine hasn't started processing |
| `processing` | Feature extraction in progress |
| `completed` | The full pipeline succeeded (`__fully_enriched: true`) |
| `partial` | Some features failed (`__missing_features` populated) |
| `enriching` | Materialized taxonomy/cluster enrichment running |

Query `__fully_enriched` in retriever filters to exclude incomplete documents.

## Best Practices

> **Design bucket schemas for validation, not storage**
>
> Bucket schemas enforce input shape but don't constrain downstream processing. Keep them minimal and use collection `field_passthrough` to propagate only what's needed.

> **Use key\_prefix for logical grouping**
>
> Organize objects with `key_prefix` (e.g., `/catalog/electronics`, `/users/avatars`) to enable bulk operations and filtering without custom metadata.

> **Batch efficiently**
>
> Group 100-1000 objects per batch for optimal throughput. Smaller batches add orchestration overhead; larger batches delay feedback and complicate retries.

> **Leverage lineage for debugging**
>
> When retrieval results are unexpected, use lineage APIs to trace documents back to source objects and inspect raw input data and processing history.

> **Version collections explicitly**
>
> When changing features or mappings, create a new collection rather than mutating the existing one. This preserves reproducibility and simplifies rollback.

## Example: E-Commerce Product Pipeline

```json theme={null}
1. Register object in "products" bucket:
   - blobs: [{ property: "image", type: "image", url: "s3://..." }]
   - metadata: { sku: "HD-9000", category: "headphones", price: 299 }

2. Process through "product-embeddings" collection (features: ["image_search", "text_search"]):
   - image_search → image embeddings
   - text_search → description embeddings
   - Passthrough: sku, category, price

3. Resulting document in MVS:
   - document_id: "doc_abc123"
   - source_object_id: "obj_prod_001"
   - feature_refs: [
       "mixpeek://image_extractor@v1/google_siglip_base_v1",
       "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
     ]
   - metadata: { sku: "HD-9000", category: "headphones", price: 299 }
   - __fully_enriched: true

4. Query with hybrid retriever:
   - Stage 1: feature_search on mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1
   - Stage 2: attribute_filter where category = "headphones"
   - Stage 3: sort_attribute by price ascending
```

## Next Steps

* Dive into advanced pipeline configuration with [Feature Extractors](/docs/processing/feature-extractors)
* Explore [Collections](/docs/ingestion/collections) configuration and output schemas
* Review [Document Lineage API](/docs/api-reference/document-lineage/get-document-lineage) for tracing provenance
* Understand [Namespaces](/docs/ingestion/namespaces) for tenant isolation


# Migrate from Elasticsearch
Source: https://docs.mixpeek.com/docs/overview/from-elasticsearch

From keyword search to multimodal warehouse retrieval

Elasticsearch is a search engine built on keyword matching and BM25. Over time, you may have bolted on vector search (`dense_vector`), external embedding generation, and custom pipelines to handle multimodal content. Mixpeek unifies all of this: feature extraction, tiered storage, and multi-stage retrieval in a single API.

This guide walks you through migrating your search workload from Elasticsearch to Mixpeek.

## Why Migrate

> [!NOTE]
>
> Elasticsearch started as a keyword search engine. Vector search, embedding generation, and multimodal processing are additions you configure and maintain yourself. Mixpeek was built from the ground up as a multimodal data warehouse where feature extraction, storage tiering, and multi-stage retrieval are native primitives, not plugins.

| Elasticsearch | Mixpeek |
| - | - |
| Keyword search (BM25) with bolted-on vector search | Native semantic, keyword, and hybrid search in one pipeline |
| You generate and manage embeddings externally | Feature extractors handle embedding generation automatically |
| Single storage tier (hot-warm-cold requires manual ILM) | Automatic tiered storage: active, cold, archive, up to 90% savings |
| Complex DSL for combining query types | Multi-stage retriever pipelines: chain stages declaratively |
| Ingest pipelines for basic transforms | Collections with ML-powered feature extractors (CLIP, Whisper, LayoutLM) |

## Concept Mapping

| Elasticsearch | Mixpeek | Notes |
| - | - | - |
| Index | Namespace | Top-level container for your data |
| Document | Document | Mixpeek documents contain extracted features, metadata, and source lineage |
| Mapping / Schema | Collection + Feature Extractor | Collections define what features to extract; schema is derived automatically |
| DSL query | Retriever (with stages) | Stages are like query clauses, composable and ordered |
| `bool` query (must/should/filter) | Multi-stage pipeline | Each clause becomes a stage: search, filter, boost, rerank |
| Aggregation | Reduce stages / Taxonomies | Group, classify, and summarize results |
| Ingest pipeline | Collection + Feature Extractors | Mixpeek pipelines extract ML features, not just field transforms |
| Analyzer (tokenizer + filters) | Feature Extractor configuration | Extractors handle tokenization, embedding, and structured extraction |
| ILM (Index Lifecycle Management) | Automatic storage tiering | Hot, cold, archive managed by the platform |

## Migration Steps

**1. Create a Namespace**

Replace your Elasticsearch index with a Mixpeek namespace.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/namespaces \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_name": "knowledge-base"
  }'
```

**2. Replace Index Mappings with Collections**

Instead of defining field types and analyzers, create a collection with a feature extractor that matches your content.

```json Elasticsearch (before) theme={null}
PUT /knowledge-base
{
  "mappings": {
    "properties": {
      "title": { "type": "text", "analyzer": "english" },
      "body": { "type": "text", "analyzer": "english" },
      "embedding": { "type": "dense_vector", "dims": 768 },
      "category": { "type": "keyword" },
      "published_at": { "type": "date" }
    }
  }
}
```

```python Mixpeek (after) theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="YOUR_API_KEY")

# Collection handles schema, embedding, and indexing
collection = client.collections.create(
    collection_name="articles",
    feature_extractor={
        "feature_extractor_name": "multimodal",
        "version": "v1"
    },
    namespace="knowledge-base"
)
```

> [!TIP]
>
> You do not need to define field types or manage embedding dimensions. The feature extractor handles all of this based on your content.

**3. Replace Ingest Pipelines with Feature Extraction**

Elasticsearch ingest pipelines handle basic field transforms. Mixpeek collections run ML models on your content: generating embeddings, extracting entities, transcribing audio, and more.

```python theme={null}
# Upload source files - the collection handles everything
client.objects.create(
    bucket_id="your-bucket-id",
    key_prefix="/articles",
    blobs=[
        {"property": "content", "data": "s3://your-bucket/article-001.pdf"}
    ],
    namespace="knowledge-base"
)
```

> [!WARNING]
>
> Do not try to bulk-import your Elasticsearch documents or vectors. Re-ingest your source files so the pipeline can extract multi-layered features and build proper lineage.

**4. Translate DSL Queries to Retriever Stages**

Elasticsearch's query DSL maps naturally to Mixpeek retriever stages. Each DSL clause becomes a stage in the pipeline.

```json Elasticsearch (before) theme={null}
POST /knowledge-base/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "knn": {
            "field": "embedding",
            "query_vector": [0.1, 0.2, ...],
            "num_candidates": 50,
            "k": 20
          }
        }
      ],
      "filter": [
        { "term": { "category": "engineering" } },
        { "range": { "published_at": { "gte": "2025-01-01" } } }
      ]
    }
  },
  "size": 10
}
```

```python Mixpeek (after) theme={null}
# One call - embedding, search, and filtering handled
results = client.retrievers.execute(
    retriever_id="article-search",
    inputs={
        "query": "distributed systems architecture",
        "category": "engineering"
    },
    limit=10,
    namespace="knowledge-base"
)
```

**5. Build Multi-Stage Retriever Pipelines**

Define a retriever that chains stages together. This replaces complex DSL queries with a declarative pipeline.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: knowledge-base" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "article-search",
    "stages": [
      {
        "stage_name": "hybrid_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "fusion": "rrf",
            "final_top_k": 50,
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 50
              },
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "lexical": true,
                "top_k": 50
              }
            ]
          }
        }
      },
      {
        "stage_name": "filter_category",
        "stage_type": "filter",
        "config": {
          "stage_id": "attribute_filter",
          "parameters": {
            "field": "category",
            "operator": "eq",
            "value": "{{INPUT.category}}"
          }
        }
      },
      {
        "stage_name": "rerank",
        "stage_type": "sort",
        "config": {
          "stage_id": "rerank",
          "parameters": {
            "top_k": 10
          }
        }
      }
    ]
  }'
```

> [!TIP]
>
> Notice the hybrid approach: semantic search and keyword search run as separate stages, then results are combined through reranking. No need to manually tune BM25 weights against vector scores.

**6. Test and Verify**

Execute your retriever and compare results against your Elasticsearch baseline.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: knowledge-base" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "query": "distributed systems architecture",
      "category": "engineering"
    },
    "limit": 10
  }'
```

## What You Gain

| Capability | Elasticsearch | Mixpeek |
| - | - | - |
| **Hybrid search** | Manual BM25 + kNN score tuning | Semantic and keyword stages with automatic reranking |
| **Feature extraction** | External embedding generation, custom ingest pipelines | Built-in ML extractors: CLIP, Whisper, LayoutLM, and more |
| **Multimodal** | Text-native; images/video require custom plugins | Native support for video, audio, images, and documents in the same namespace |
| **Storage tiering** | ILM policies you configure and maintain | Automatic tiering: active, cold, archive, managed by the platform |
| **No infrastructure** | Clusters, shards, replicas, JVM tuning | Fully managed API, no cluster operations |
| **Lineage** | Documents disconnected from source files | Trace any result back through document, object, and source file |
| **Multi-stage pipelines** | Complex nested DSL | Declarative stage pipelines: search, filter, rerank, enrich |

## Next Steps

> **[Quickstart](/docs/overview/quickstart)**
>
> Get Mixpeek running in 10 minutes

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Learn about automatic feature extraction

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Build multi-stage retrieval pipelines

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand the data model


# Migrate from Pinecone
Source: https://docs.mixpeek.com/docs/overview/from-pinecone

Move from single-vector search to multi-stage warehouse retrieval

Pinecone is a vector database built for single-embedding KNN search. Mixpeek is a multimodal data warehouse that decomposes files into searchable features, stores them across cost tiers, and reassembles answers through multi-stage retrieval pipelines.

This guide walks you through migrating your search workload from Pinecone to Mixpeek.

## Why Migrate

> [!NOTE]
>
> Pinecone stores and queries individual vectors. Mixpeek processes raw files end-to-end: extracting features, storing documents across tiered storage, and executing multi-stage retrieval pipelines. You stop managing embeddings and start working with content.

| Pinecone | Mixpeek |
| - | - |
| You generate embeddings externally | Feature extractors generate embeddings automatically |
| Single-vector KNN per query | Multi-stage pipelines: search, filter, rerank, enrich |
| Flat storage pricing | Tiered storage: hot (active), warm (cold), archive, up to 90% savings |
| Metadata filtering on vector results | Attribute filters, boolean logic, and cross-modal joins as pipeline stages |
| One index per embedding model | One namespace handles multiple modalities and models simultaneously |

## Concept Mapping

| Pinecone | Mixpeek | Notes |
| - | - | - |
| Index | Namespace | Top-level container for your data |
| Namespace | Namespace (via `X-Namespace` header) | Tenant or environment isolation within a namespace |
| Vector | Document (with features) | Documents contain extracted features, metadata, and lineage back to the source file |
| Upsert | Object upload + Collection processing | Data flows through the pipeline: upload to bucket, collection triggers extraction |
| Query (top-k KNN) | Retriever execution (multi-stage) | Retrievers chain stages: semantic search, filters, reranking, enrichment |
| Metadata filter | Attribute filter stage | Filters are composable stages in a retrieval pipeline |

## Migration Steps

**1. Create a Namespace**

Set up a namespace to hold your data. This replaces your Pinecone index.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/namespaces \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_name": "product-catalog"
  }'
```

**2. Create a Collection with Feature Extractors**

Define what features to extract from your data. This replaces the external embedding step you had with Pinecone.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/collections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: product-catalog" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "products",
    "source": { "type": "bucket", "bucket_ids": ["'$BUCKET_ID'"] },
    "feature_extractor": {
      "feature_extractor_name": "multimodal",
      "version": "v1"
    }
  }'
```

**3. Re-ingest Your Data Through the Pipeline**

Upload your source files to a bucket and let the collection process them. Do not try to import your existing Pinecone vectors directly. Mixpeek extracts richer, multi-modal features from your raw content.

> [!WARNING]
>
> Never insert vectors directly into the storage layer. All data must flow through the ingestion pipeline: bucket upload, collection trigger, feature extraction. This ensures proper lineage, validation, and multi-modal indexing.

```bash theme={null}
# Upload objects to a bucket
curl -X POST https://api.mixpeek.com/v1/buckets/{bucket_id}/objects \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: product-catalog" \
  -H "Content-Type: application/json" \
  -d '{
    "key_prefix": "/products",
    "blobs": [
      { "property": "image", "type": "image", "data": "s3://your-bucket/product-001.jpg" },
      { "property": "description", "type": "text", "data": "s3://your-bucket/product-001.txt" }
    ]
  }'
```

**4. Create a Retriever with Multi-Stage Pipelines**

Build a retriever that goes beyond single-vector KNN. Chain semantic search with filters, reranking, and enrichment.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: product-catalog" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product-search",
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "final_top_k": 50,
            "searches": [
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 50
              }
            ]
          }
        }
      },
      {
        "stage_name": "filter_category",
        "stage_type": "filter",
        "config": {
          "stage_id": "attribute_filter",
          "parameters": {
            "field": "category",
            "operator": "eq",
            "value": "{{INPUT.category}}"
          }
        }
      },
      {
        "stage_name": "rerank",
        "stage_type": "sort",
        "config": {
          "stage_id": "rerank",
          "parameters": {
            "top_k": 10
          }
        }
      }
    ]
  }'
```

**5. Test and Verify**

Execute your retriever and compare results against your Pinecone baseline.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: product-catalog" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "query": "red running shoes",
      "category": "footwear"
    },
    "limit": 10
  }'
```

## Side-by-Side Comparison

> [!TIP]
>
> The Mixpeek retriever does in one API call what requires multiple steps with Pinecone: embedding generation, vector search, and post-processing.

```python Pinecone theme={null}
import pinecone
from sentence_transformers import SentenceTransformer

# You manage the embedding model
model = SentenceTransformer("all-MiniLM-L6-v2")

# You generate the embedding
query_embedding = model.encode("red running shoes").tolist()

# Single-vector KNN search
pinecone.init(api_key="PINECONE_KEY", environment="us-east-1")
index = pinecone.Index("product-catalog")

results = index.query(
    vector=query_embedding,
    top_k=10,
    filter={"category": "footwear"},
    include_metadata=True
)

for match in results["matches"]:
    print(match["id"], match["score"], match["metadata"])
```

```python Mixpeek theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="YOUR_API_KEY")

# One call: embedding, search, filter, rerank - all handled
results = client.retrievers.execute(
    retriever_id="product-search",
    inputs={
        "query": "red running shoes",
        "category": "footwear"
    },
    limit=10,
    namespace="product-catalog"
)

for doc in results:
    print(doc["document_id"], doc["score"], doc["metadata"])
```

## What You Gain

| Capability | Pinecone | Mixpeek |
| - | - | - |
| **Multi-stage retrieval** | Single KNN query; post-processing is your problem | Chain search, filter, rerank, and enrich stages in one pipeline |
| **Automatic feature extraction** | You build and maintain embedding pipelines | Feature extractors handle it: CLIP, Whisper, LayoutLM, and more |
| **Tiered storage** | All vectors at one price tier | Hot, cold, and archive tiers, up to 90% savings on infrequently accessed data |
| **Multimodal search** | One embedding model per index | Search across text, images, video, and audio in the same namespace |
| **No per-query vector fees** | Per-read pricing on every query | Flat API pricing, no per-vector-read charges |
| **Complete lineage** | Vectors disconnected from source files | Trace any result back through document, object, and source file |

## Next Steps

> **[Quickstart](/docs/overview/quickstart)**
>
> Get Mixpeek running in 10 minutes

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Learn about automatic feature extraction

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Build multi-stage retrieval pipelines

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand the data model


# Migrate from Weaviate
Source: https://docs.mixpeek.com/docs/overview/from-weaviate

Upgrade from vector database to multimodal data warehouse

Weaviate is a vector database with built-in vectorization modules. Mixpeek is a multimodal data warehouse that goes further: decomposing files into layered features, storing them across cost tiers, and reassembling answers through multi-stage retrieval pipelines.

This guide walks you through migrating your search workload from Weaviate to Mixpeek.

## Why Migrate

Weaviate introduced multimodal capabilities with modules like `multi2vec-clip` and `img2vec-neural`. Mixpeek builds on this direction but takes a fundamentally different approach.

> [!NOTE]
>
> Where Weaviate adds vector search to a database, Mixpeek starts from the file itself. A single video becomes transcripts, visual embeddings, scene descriptions, and detected entities, each independently searchable, all stored across cost tiers, and reassembled through configurable pipelines.

| Weaviate | Mixpeek |
| - | - |
| Vectorization modules bolt onto storage | Feature extraction is the core of the pipeline |
| GraphQL queries with vector search | Multi-stage retrieval pipelines: search, filter, rerank, enrich |
| Single storage tier | Tiered storage: hot, cold, archive, up to 90% savings |
| Schema-per-class design | Namespace-level organization with collections per processing pipeline |
| Module-based multimodal support | Native decomposition: one file becomes many searchable layers |

## Concept Mapping

| Weaviate | Mixpeek | Notes |
| - | - | - |
| Class | Collection | Defines processing pipeline and feature extraction for a data type |
| Property | Document field | Documents contain extracted features, metadata, and source lineage |
| Object (with vector) | Document (with features) | Documents hold multiple feature types, not just one vector |
| Module (e.g., `text2vec-openai`) | Feature Extractor | Built-in extractors: multimodal, image, text, face-identity, document, and more |
| GraphQL query | Retriever execution | Retrievers are multi-stage pipelines, not single queries |
| `nearText` / `nearVector` | Semantic search stage | One stage in a larger pipeline |
| `where` filter | Attribute filter stage | Filters compose as pipeline stages alongside search and ranking |
| Cross-reference | Semantic JOIN / Taxonomy | Connect documents across collections using vector similarity |

## Migration Steps

**1. Create a Namespace**

Replace your Weaviate instance with a Mixpeek namespace.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/namespaces \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_name": "media-library"
  }'
```

**2. Map Classes to Collections**

Each Weaviate class becomes a Mixpeek collection with a feature extractor. Instead of choosing a vectorization module, you choose an extractor that matches your content type.

```python Weaviate (before) theme={null}
# Weaviate class definition
client.schema.create_class({
    "class": "Article",
    "vectorizer": "text2vec-openai",
    "properties": [
        {"name": "title", "dataType": ["text"]},
        {"name": "body", "dataType": ["text"]},
        {"name": "category", "dataType": ["text"]}
    ]
})
```

```python Mixpeek (after) theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="YOUR_API_KEY")

# Collection with automatic feature extraction
collection = client.collections.create(
    collection_name="articles",
    feature_extractor={
        "feature_extractor_name": "multimodal",
        "version": "v1"
    },
    namespace="media-library"
)
```

**3. Re-ingest Your Data**

Upload source files through the Mixpeek pipeline instead of importing Weaviate objects. The pipeline extracts richer features than a single vectorization module.

> [!WARNING]
>
> Do not export vectors from Weaviate and import them into Mixpeek. Re-ingest your source files so the pipeline can extract multi-layered features, build lineage, and index across modalities.

```python theme={null}
# Upload objects to a bucket for processing
client.objects.create(
    bucket_id="your-bucket-id",
    key_prefix="/articles",
    blobs=[
        {"property": "content", "data": "s3://your-bucket/article-001.pdf"}
    ],
    namespace="media-library"
)
```

**4. Replace GraphQL Queries with Retrievers**

Weaviate's GraphQL queries map to Mixpeek retrievers. The difference: retrievers chain multiple stages together.

```graphql Weaviate (before) theme={null}
{
  Get {
    Article(
      nearText: { concepts: ["machine learning trends"] }
      where: {
        path: ["category"]
        operator: Equal
        valueText: "technology"
      }
      limit: 10
    ) {
      title
      body
      _additional { score }
    }
  }
}
```

```python Mixpeek (after) theme={null}
results = client.retrievers.execute(
    retriever_id="article-search",
    inputs={
        "query": "machine learning trends",
        "category": "technology"
    },
    limit=10,
    namespace="media-library"
)
```

**5. Build Multi-Stage Pipelines**

Go beyond what Weaviate's query language supports. Chain semantic search with attribute filters, reranking, and enrichment in a single retriever.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: media-library" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "article-search",
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "final_top_k": 50,
            "searches": [
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 50
              }
            ]
          }
        }
      },
      {
        "stage_name": "filter_category",
        "stage_type": "filter",
        "config": {
          "stage_id": "attribute_filter",
          "parameters": {
            "field": "category",
            "operator": "eq",
            "value": "{{INPUT.category}}"
          }
        }
      },
      {
        "stage_name": "rerank",
        "stage_type": "sort",
        "config": {
          "stage_id": "rerank",
          "parameters": {
            "top_k": 10
          }
        }
      }
    ]
  }'
```

**6. Test and Verify**

Execute retrievers and validate results against your Weaviate baseline.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: media-library" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "query": "machine learning trends",
      "category": "technology"
    },
    "limit": 10
  }'
```

## What You Gain

| Capability | Weaviate | Mixpeek |
| - | - | - |
| **File decomposition** | Vectorize one property at a time | Decompose a file into multiple searchable layers automatically |
| **Multi-stage retrieval** | Single query with optional filters | Chain search, filter, rerank, and enrich stages in one pipeline |
| **Tiered storage** | All data at one storage tier | Hot, cold, and archive tiers, up to 90% savings |
| **Cross-modal search** | Modules per class, limited cross-modal | Native cross-modal: text query finds video moments, audio segments, image regions |
| **No infrastructure** | Self-hosted or managed cluster | Fully managed API, no clusters, no module configuration |
| **Complete lineage** | Objects in a class | Trace results back through document, object, and source file |

> [!TIP]
>
> If you are using Weaviate's `multi2vec-clip` for image-text search, Mixpeek's multimodal extractor handles the same use case and adds video, audio, and document support in the same namespace.

## Next Steps

> **[Quickstart](/docs/overview/quickstart)**
>
> Get Mixpeek running in 10 minutes

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Learn about automatic feature extraction

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Build multi-stage retrieval pipelines

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand the data model


# Introduction
Source: https://docs.mixpeek.com/docs/overview/introduction

Mixpeek is the semantic retrieval layer for unstructured and multimodal data

Your AI agent can read text. It cannot watch a video, scan a photo for faces, or search audio by what was said. Mixpeek is the semantic retrieval layer for unstructured and multimodal data: it turns video, images, audio and documents into named semantic features, then serves them through retrievers your agent calls by name.

> **[Start building on Mixpeek](https://studio.mixpeek.com/?utm_source=docs\&utm_medium=docs_intro\&utm_campaign=mvs_onramp)**
>
> Create a workspace and run your first multimodal search — index your own video, images, audio, or documents in minutes.

## How It Works

**1. Index**

Upload video, images, audio, and documents to [Buckets](/docs/platform/data-model#create-a-bucket). Mixpeek runs feature extraction automatically — faces, objects, transcripts, embeddings, and structured metadata all get indexed into searchable [Collections](/docs/overview/concepts).

![Indexing multimodal content into semantic layers](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-decomposition.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=4bfa40801fe7d78a68db0bd3f48b49c4)

See this running on real content — 7 PDFs that fan out into 59 searchable knowledge-graph nodes — in the [sample data](/docs/overview/sample-data#diagram-1-how-content-decomposes).

**2. Search**

Build retrieval pipelines that your agent calls. Semantic search, face search, object search, transcript search — chain them together into multi-stage [Retrievers](/docs/retrieval/retrievers) and expose them as a single endpoint.

![Multi-stage retrieval pipeline](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-retrieval.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=4089a289e16b0cb17b6decd075b46f13)

Run this exact pipeline against the [sample data](/docs/overview/sample-data#diagram-2-how-retrieval-reassembles) right now — no account or API key needed.

**3. Integrate**

Wire Mixpeek into your agent as a [LangChain tool](/docs/integrations/agents#langchain), an [MCP server](/docs/integrations/agents#mcp-model-context-protocol), or a direct REST call. Your agent sends a query, gets structured results back, and acts on them.

> [!TIP]
>
> **Already have embeddings?** Skip extraction entirely — bring your own vectors and search instantly with the [Mixpeek Vector Store](/docs/vector-store/overview). Sixty seconds to first query.

## Quickstart

> **[Get Started](/docs/overview/quickstart)**
>
> Index multimodal content and search it in under 10 minutes — LangChain, MCP, or REST

## What Can You Build?

Pick an outcome and follow the guide — each one is end-to-end and copy-pasteable.

> **[Search video](/docs/tutorials/video-understanding)**
>
> Find moments by what's shown or said — visual + speech embeddings

> **[Reverse image & video search](/docs/tutorials/reverse-search)**
>
> Use an image or clip as the query to find visually similar media

> **[Hybrid search](/docs/retrieval/stages/feature-search#lexical-bm25-search)**
>
> Combine dense vectors with keyword/BM25 and attribute filters

> **[Rerank for precision](/docs/retrieval/stages/rerank)**
>
> Push the most relevant results to the top with a cross-encoder

> **[Cluster content](/docs/enrichment/clusters)**
>
> Group, label, and visualize a collection automatically

> **[Build a taxonomy](/docs/enrichment/taxonomies)**
>
> Classify content against your own hierarchy with a multimodal join

> **[Secure per-user retrieval](/docs/platform/permissions)**
>
> Enforce access control with built-in ACLs or external OpenFGA

> **[Bring your own vectors](/docs/vector-store/overview)**
>
> Use Mixpeek as a standalone vector store — upsert and query instantly

> **[Auto-tune relevance](/docs/retrieval/auto-tune)**
>
> Improve ranking automatically from click and reward signals

> **[Migrate embedding models](/docs/processing/model-migration)**
>
> Re-extract, validate, and cut over to a new model with no downtime

## What Gets Extracted

| File Type | Extracted Features |
| - | - |
| **[Video](/docs/processing/extractors/multimodal)** | Face embeddings (ArcFace 512D), scene descriptions (Gemini), visual embeddings (Vertex AI 1408D), transcripts (Whisper), transcript embeddings (E5-Large 1024D), keyframes |
| **[Images](/docs/processing/extractors/image)** | Visual embeddings (SigLIP 768D or Vertex AI 1408D), face embeddings (ArcFace 512D), OCR text, descriptions, structured extraction |
| **[Audio](/docs/processing/extractors/multimodal)** | Transcripts (Whisper), transcript embeddings (E5-Large 1024D), multimodal audio embeddings (Vertex AI 1408D) |
| **[Documents](/docs/processing/extractors/document)** | Text chunks, text embeddings (E5-Large 1024D), OCR for scanned PDFs, structured extraction |

Each extracted feature becomes an independently searchable document. A single video can produce hundreds of documents — one per face, one per transcript segment, one per scene.

## Key Concepts

* **Namespaces** isolate data between tenants, environments, or projects. Every API request includes a namespace header.
* **Buckets** hold your raw files. Upload once, process many ways.
* **Collections** define what gets extracted. Each collection runs a feature extractor (CLIP, Whisper, LayoutLM, etc.) against objects in a bucket.
* **Retrievers** are search pipelines you configure in JSON. Chain stages together — vector search, face matching, filters, re-ranking — and expose the result as one endpoint your agent calls.

## Next Steps

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand namespaces, buckets, collections, and retrievers in depth

> **[Data Model](/docs/platform/data-model)**
>
> See how namespaces, buckets, collections, objects, and features fit together

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Learn what each extractor does and how to configure it

> **[Tutorials](/docs/tutorials)**
>
> Step-by-step guides for common use cases


# Quickstart
Source: https://docs.mixpeek.com/docs/overview/quickstart

Start searching in under 60 seconds with BYO vectors, or index multimodal content with the managed platform

Get a Mixpeek API key from [mixpeek.com/start](https://mixpeek.com/start?utm_source=docs\&utm_medium=docs_quickstart\&utm_campaign=mvs_onramp). The key is minted at signup, and a plan (Build, from \$25/mo) is needed before the API accepts data or searches; until then calls return `PlanRequiredError` with a link to choose one. Then pick your path — BYO vectors for instant search, or the managed platform: point it at your files, pick what you want to search by, and extraction is handled for you.

> [!TIP]
>
> **API-first?** Your first key is minted at signup, but you don't need the dashboard after that — create, scope, and rotate additional keys entirely from the API: [Create](/docs/api-reference/organization-api-keys/create-api-key) · [Rotate](/docs/api-reference/organization-api-keys/rotate-api-key) · [List](/docs/api-reference/organization-api-keys/list-api-keys).

> [!TIP]
>
> **New to Mixpeek?** Follow the first tab — **MVS Standalone** — end to end. It's the shortest path to a working search (under 60 seconds once your plan is active) and needs nothing but an API key. The other tabs wire that same search into an agent (LangChain, MCP) or call it over REST once you've created a retriever.

**MVS Standalone (fastest)**

Already have embeddings? Skip extraction — search in under 60 seconds.

```bash theme={null}
pip install mixpeek
export MIXPEEK_API_KEY="mxp_sk_replace_me"
```

**1. Create a standalone namespace**

```python theme={null}
import os
from mixpeek import Mixpeek

client = Mixpeek(api_key=os.environ["MIXPEEK_API_KEY"])

NAMESPACE_ID = "my-search"

client.namespaces.create(
    namespace_id=NAMESPACE_ID,   # the id you choose; every later call sends it
    mode="standalone",           # bring your own vectors, no extractors
)
```

**2. Upsert your vectors**

```python theme={null}
client.namespaces.documents.upsert(
    namespace_id=NAMESPACE_ID,
    documents=[{
        "document_id": "doc-001",
        "vectors": {"embedding": [0.12, -0.34, 0.56]},  # your embedding
        "payload": {"title": "First document", "category": "demo"},
    }],
)
```

**3. Search**

Vectors are searchable within **10–30 seconds** of upsert (the index flushes in the background). For immediate verification, add a short poll:

```python theme={null}
import time

# Wait for indexing
time.sleep(15)

results = client.search(
    namespace_id=NAMESPACE_ID,
    queries=[{
        "feature_uri": "embedding",  # the vector name you upserted under
        "query": {"input_mode": "vector", "value": [0.15, -0.28, 0.44]},  # query embedding
        "top_k": 10,
    }],
)
for hit in results["documents"]:
    print(f"{hit['score']:.3f} | {hit['document_id']}")
```

> [!TIP]
>
> MVS infers vector dimensions on first write — no schema needed. When you're ready for automatic extraction, [promote your namespace to Managed](/docs/vector-store/promote).

**LangChain Agent**

Build a LangChain agent with a `video_search` tool backed by Mixpeek.

```bash theme={null}
pip install mixpeek langchain langchain-openai
export MIXPEEK_API_KEY="mxp_sk_replace_me"
export OPENAI_API_KEY="sk-replace_me"
```

**1. Create namespace, bucket, and collection**

```python theme={null}
import os
from mixpeek import Mixpeek

client = Mixpeek(api_key=os.environ["MIXPEEK_API_KEY"])

# Managed namespace — `namespace_name` is the only required field. You don't
# register extractors up front; the collection's `features` (below) auto-provision
# whatever pipeline the namespace needs.
ns = client.namespaces.create(namespace_name="agent-video-demo")

# Scope the client to the namespace you just created — every call below sends it
# as the X-Namespace header, which bucket/collection create and search require.
client.namespace = ns["namespace_id"]

bucket = client.buckets.create(
    bucket_name="demo-videos",
    # "video" is the standard source property name per modality —
    # collections created with `features` wire to it automatically.
    bucket_schema={"properties": {"video": {"type": "video"}}},
)

# Create the collection by picking a feature — Mixpeek resolves the
# pipeline and default input wiring (see /processing/features).
col = client.collections.create(
    collection_name="video-scenes",
    source={"type": "bucket", "bucket_ids": [bucket["bucket_id"]]},
    features=["video_search"],
)
```

**2. Upload and process**

```python theme={null}
# Upload the video to the bucket, then trigger the collection to process it.
client.buckets.upload(
    bucket["bucket_id"],
    blobs=[{
        "property": "video",
        "type": "video",
        "data": "https://mixpeek-public-demo.s3.us-east-2.amazonaws.com/gtm/05-s3-files-reel.mp4",
    }],
)
result = client.collections.trigger(col["collection_id"])
```

Poll `client.tasks.get(task_id=result["task_id"])` until the status is terminal — `COMPLETED` or `COMPLETED_WITH_ERRORS` (1-5 min). Don't wait only for `COMPLETED`, or a batch that finished with some failed items will hang your loop.

**3. Create a retriever and wire it as a tool**

```python theme={null}
from langchain.tools import Tool
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

ret = client.retrievers.create(
    retriever_name="agent-video-search",
    input_schema={"query_text": {"type": "text", "required": True}},
    collection_identifiers=[col["collection_id"]],
    stages=[{
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [{
                    "feature_uri": "mixpeek://universal_extractor@v1/gemini-embedding-2",
                    "query": {"input_mode": "text", "value": "{{INPUT.query_text}}"},
                    "top_k": 50
                }],
                "final_top_k": 20
            }
        }
    }]
)

def search_video(query: str) -> str:
    results = client.retrievers.execute(
        retriever_id=ret["retriever_id"],
        inputs={"query_text": query},
    )
    return "\n".join(
        f"[{r.get('start_time', r.get('start_time_s', '?'))}s] (score: {r['score']:.3f}) {r.get('description','')}"
        for r in results["documents"]
    ) or "No results found."

video_tool = Tool(name="video_search", description="Search indexed video by natural language", func=search_video)
llm = ChatOpenAI(model="gpt-4o", temperature=0)
prompt = ChatPromptTemplate.from_messages([
    ("system", "You answer questions about video content. Use video_search to find relevant moments."),
    ("human", "{input}"), MessagesPlaceholder(variable_name="agent_scratchpad"),
])
agent = create_openai_tools_agent(llm, [video_tool], prompt)
executor = AgentExecutor(agent=agent, tools=[video_tool], verbose=True)
print(executor.invoke({"input": "What product features are shown?"})["output"])
```

**MCP (Claude)**

Add Mixpeek as a tool in Claude Desktop or Claude Code — no code required.

**Claude Desktop**

Open `claude_desktop_config.json` and add:

```json theme={null}
{
  "mcpServers": {
    "mixpeek-retrieval": {
      "url": "https://mcp.mixpeek.com/retrieval/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

**Claude Code**

Add `.mcp.json` to your project root:

```json theme={null}
{
  "mcpServers": {
    "mixpeek-retrieval": {
      "url": "https://mcp.mixpeek.com/retrieval/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

Restart Claude and ask: *"What Mixpeek tools do you have access to?"*

| Scope | URL | Use it for |
| - | - | - |
| **Full** | `https://mcp.mixpeek.com/mcp` | Everything below in one server |
| **Ingestion** | `https://mcp.mixpeek.com/ingestion/mcp` | Buckets, uploads, collections, batches |
| **Retrieval** | `https://mcp.mixpeek.com/retrieval/mcp` | Retrievers, search, agent sessions |
| **Admin** | `https://mcp.mixpeek.com/admin/mcp` | Namespaces, keys, taxonomies, clusters |

See the [full MCP reference](/docs/agent-integrations/mcp) for per-retriever servers and troubleshooting.

**REST API**

Call `POST /v1/retrievers/{id}/execute` from any language. `X-Namespace` takes the
namespace **id** (`ns_...`), not the namespace name.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/$MIXPEEK_RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"inputs": {"query_text": "safety regulations"}}'
```

```python Python theme={null}
import os
import requests

MIXPEEK_API_KEY = os.environ["MIXPEEK_API_KEY"]
MIXPEEK_NAMESPACE = os.environ["MIXPEEK_NAMESPACE"]  # the namespace_id (ns_...) returned at creation
RETRIEVER_ID = os.environ["MIXPEEK_RETRIEVER_ID"]    # the retriever_id (ret_...) from retrievers.create

HEADERS = {
    "Authorization": f"Bearer {MIXPEEK_API_KEY}",
    "X-Namespace": MIXPEEK_NAMESPACE,
    "Content-Type": "application/json",
}

resp = requests.post(
    f"https://api.mixpeek.com/v1/retrievers/{RETRIEVER_ID}/execute",
    headers=HEADERS,
    json={"inputs": {"query_text": "safety regulations"}},
)
results = resp.json()["documents"]
```

```javascript JavaScript theme={null}
const res = await fetch(
  `https://api.mixpeek.com/v1/retrievers/${process.env.MIXPEEK_RETRIEVER_ID}/execute`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.MIXPEEK_API_KEY}`,
      "X-Namespace": process.env.MIXPEEK_NAMESPACE,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ inputs: { query_text: "safety regulations" } }),
  },
);
const { documents } = await res.json();
```

Read `documents`, and check `status` and `warnings` before treating an empty list
as a real no-match. See [the execution lifecycle](/docs/retrieval/retrievers#execution-lifecycle).

Each result is a document with a top-level `score` plus the document's own fields (e.g. `content`, `start_time`) flattened to the top level. Wrap this call as a tool in any agent framework — OpenAI function calling, CrewAI, LlamaIndex, or plain HTTP.

See the [API reference](/docs/api-reference/retrievers/execute-retriever-auto-optimized) for full request and response details.


# Explore the sample data
Source: https://docs.mixpeek.com/docs/overview/sample-data

Run real multimodal queries against a live sample namespace — no account, no API key, no setup

The fastest way to understand Mixpeek is to run it. **`sample-data`** is a live
namespace loaded with real content — video, ad creatives, PDFs, and a product
catalog, already extracted and searchable — and the retrievers below are published
publicly, so **every example on this page runs with no account and no API key**.

Copy any `curl` here into a terminal right now. It will return real results.

It is also the working version of the two diagrams on the
[Introduction](/docs/overview/introduction) page: how content decomposes into layers,
and how a retriever reassembles those layers at query time.

## Run your first query

No `Authorization` header. No `X-Namespace`. Just a query.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/public/retrievers/sample-document-knowledge/execute" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": { "query": "chevron quarterly revenue" },
    "pagination": { "method": "offset", "page_size": 10, "page_number": 1 }
  }'
```

```python Python theme={null}
import requests

resp = requests.post(
    "https://api.mixpeek.com/v1/public/retrievers/sample-document-knowledge/execute",
    json={
        "inputs": {"query": "chevron quarterly revenue"},
        "pagination": {"method": "offset", "page_size": 10, "page_number": 1},
    },
)
for hit in resp.json()["results"]:
    print(hit["score"], hit["title"], "—", hit["description"][:60])
```

```javascript JavaScript theme={null}
const resp = await fetch(
  'https://api.mixpeek.com/v1/public/retrievers/sample-document-knowledge/execute',
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      inputs: { query: 'chevron quarterly revenue' },
      pagination: { method: 'offset', page_size: 10, page_number: 1 },
    }),
  }
);
const { results } = await resp.json();
```

> [!WARNING]
>
> On this **public** path, hits come back under **`results`**. This differs from the
> authenticated path — `POST /v1/retrievers/{retriever_id}/execute` returns hits
> under `documents`, where `results` is a deprecated field that is no longer
> populated. Read the field that matches the path you called, or you will parse an
> empty list from a working query.

## Diagram 1 — how content decomposes

The first diagram on the Introduction page (source → temporal segments → detected
entities) is a real two-tier pipeline here. Each bucket is extracted into a
**tier-0** collection, and tier-0 is extracted *again* into a **tier-1**
collection — the layer you actually search.

| Bucket | Tier 0 — segments | Tier 1 — entities |
| - | - | - |
| `video-library` | `video-scenes` — scene segments with transcript + on-screen text | `scene-entities` — faces per scene (512-d ArcFace, with crops and bounding boxes) |
| `ad-creatives` | `ad-visual` — creative-level embeddings | `ad-entities` — faces per creative |
| `document-archive` | `document-text` — PDF text + embeddings | `document-graph` — a knowledge graph: **7 PDFs fan out into 59 nodes** |
| `product-catalog` | `product-images` (SigLIP) and `product-text` (E5) | `product-attributes` — structured attributes per product |

The `document-archive` row is the clearest illustration: decomposition is a real
**one-to-many fan-out**, not a reformat. Seven PDFs become 59 independently
searchable nodes — and each node still knows which PDF it came from, which is what
makes the next section work.

## Diagram 2 — how retrieval reassembles

The second diagram (query → vector search → entity filter → similarity rank →
enrich) is exactly what **Scene Moment Reassembly** does. The other three apply the
same reassembly idea to different content. Every one is runnable as-is.

> **Scene Moment Reassembly — the diagram, literally**
>
> `sample-scene-reassembly` — all four stages in the diagram's order:
> `feature_search` (vector) → `attribute_filter` (genre) → `rerank`
> (cross-encoder) → `document_enrich` (join the tier-1 faces back in).
>
> > [!NOTE]
> >
> > The `rerank` stage runs a cross-encoder over every candidate the search hands
> > it, so this example takes a few seconds. The other examples on this page skip
> > that stage and return in about a second. See
> > [rerank](/docs/retrieval/stages/rerank) for what the stage costs and how to size
> > the candidate set feeding it.
>
> ```json theme={null}
> { "inputs": { "query": "functions in C programming" } }
> ```
>
> Returns 8 scene moments — `title`, `description`, `start_time`/`end_time`,
> `genre` — each carrying a `face_crop_url` for the person detected **in that
> scene**. That join is the whole point: the tier-0 scene and its tier-1 entities
> come back reassembled into one result, so you get "who is on screen, and when."

> **Person Across Media — one face, across two ads**
>
> `sample-person-across-media` — `feature_search` over ArcFace embeddings spanning
> **two buckets' tier-1 collections** (`scene-entities` + `ad-entities`) →
> `document_enrich` to join back to the source.
>
> ```json theme={null}
> {
>   "inputs": {
>     "query_image": "https://mixpeek-public-demo.s3.us-east-2.amazonaws.com/demo-faces/mixpeek-brand-reel-face.jpg"
>   }
> }
> ```
>
> Returns **2 hits with different `title`s** — *Pipeline is product reel* and
> *Vector DB scam reel*. That is the lesson: the same person found across two
> **different** documents, not a single self-match.
>
> Swap in any face image URL. A face that does not appear in the sample content
> correctly returns nothing — that is reverse-face search working, not failing.
> This retriever needs an image; a text query has no face to embed.

> **Document Knowledge Explorer — search nodes, get the source**
>
> `sample-document-knowledge` — `feature_search` over the graph **nodes** →
> `document_enrich` to join back to the parent PDF.
>
> ```json theme={null}
> { "inputs": { "query": "chevron quarterly revenue" } }
> ```
>
> Each hit's `description` is the matched **node** text and `object_type` tells you
> what it is (e.g. `paragraph`) — but `title` is the **parent document**
> (*Chevron Financial Filing*). You search fine-grained pieces and still get the
> source back. Several hits often share one parent: that is the 7-PDFs-to-59-nodes
> fan-out from Diagram 1, seen from the query side.

> **Hybrid Catalog Search — one query, two modalities**
>
> `sample-hybrid-catalog` — a single `feature_search` running **two modalities at
> once** (SigLIP text-to-image over `product-images` + E5 text over
> `product-text`), fused with [RRF](/docs/relevance/fusion-strategies).
>
> ```json theme={null}
> { "inputs": { "query": "adjustable LED lamp" } }
> ```
>
> Returns products with `product_name`, `price`, `category`, and `description`.
> A product can appear from **both** its imagery and its text — the same item
> reached down two independent paths and merged into one ranking.
>
> > [!NOTE]
> >
> > RRF scores are rank-based (all hits land near `0.017`), so read the **order**,
> > not the magnitude — the score is not a similarity. The sample catalog is small
> > (8 products), so a broad query returns most of it; ranking is what changes.

> [!NOTE]
>
> Media fields (`thumbnail_url`, `video_segment_url`, `face_crop_url`) come back
> from a retriever as signed URLs minted fresh on every request, and they expire
> after 24 hours. Do not expect a URL you copied yesterday to still resolve.
> Browsing the same documents through `POST /v1/documents/list` returns raw
> `gs://` values instead, unless you ask for signing. See
> [Media URLs](/docs/vector-store/documents#media-urls).

## Make it yours

These pipelines use the same stages you would configure yourself. Fork the
namespace in Studio for an editable copy, or read how the pieces work:

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Compose search stages into a pipeline

> **[Feature extraction](/docs/processing/features)**
>
> How content is decomposed into searchable layers


# Ingest Data
Source: https://docs.mixpeek.com/docs/platform/data-model

Get your files into Mixpeek — namespaces, buckets, objects, uploads, and batching

> **[Ingest your own files](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_ingest\&utm_campaign=managed_onramp)**
>
> Create a namespace, connect a bucket, and run your first batch in minutes — no API key setup required.

## Set Up a Namespace

Every project starts with a namespace — the isolation boundary for all your resources. Use one per environment (dev, staging, prod) or per tenant.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "namespace_name": "production" }'
```

> [!NOTE]
>
> That's the whole managed create — `namespace_name` is the only required field. You don't register extractors up front: creating a collection with `features: [...]` auto-provisions the pipelines the namespace needs (see [Features](/docs/processing/features)). The old `feature_extractors` field is deprecated.

### Standalone (bring your own vectors)

To upsert your own vectors instead of Mixpeek-managed extraction, declare the index shapes with `vector_configs`:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_name": "byo-vectors",
    "vector_configs": [
      { "name": "clip", "dimension": 768, "metric": "cosine" }
    ]
  }'
```

> [!NOTE]
>
> `vector_configs` is a **list** of `{ name, dimension, metric }` objects (not a single object). `name` and `dimension` are required; `metric` defaults to `cosine` (also `euclidean`, `dot_product`). Passing `vector_configs` with no `features`/`feature_extractors` **infers standalone mode** — no `mode` field needed. Omit `vector_configs` entirely and indexes auto-create on first upsert.

Every subsequent request needs two headers: `Authorization: Bearer mxp_sk_...` and `X-Namespace: ns_...`.

[Namespace API →](/docs/api-reference/namespaces/create-namespace)

## Create a Bucket

Buckets are schema-validated containers for raw files. Define what blob types you accept (text, image, audio, video, json, binary).

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "product-catalog",
    "bucket_schema": {
      "properties": {
        "product_text": { "type": "text", "required": true },
        "hero_image": { "type": "image" }
      }
    }
  }'
```

[Bucket API →](/docs/api-reference/buckets/create-bucket)

### Storage class

Pass an optional `storage_class` on create/update to pick a cost tier for a bucket's objects. It's provider-agnostic — mapped to your object store:

| `storage_class` | GCS | S3 / MinIO | Best for |
| - | - | - | - |
| `standard` (default) | STANDARD | STANDARD | Hot, frequently-read buckets |
| `nearline` | NEARLINE | STANDARD\_IA | Warm / occasional access |
| `coldline` | COLDLINE | GLACIER\_IR | Cold / rare access |
| `archive` | ARCHIVE | GLACIER | Long-term retention |

> [!NOTE]
>
> **Applied on write for sync-based ingestion.** For buckets fed by a storage **sync** (S3, GCS, Drive, RSS, and other sources, which is the primary media path), the tier is set on each object at write time. Direct uploads (`POST /objects`) and presigned client uploads do not set it, and existing objects keep the tier they were written with. Keep hot, retriever-source buckets on `standard`, and reserve cheaper tiers for large write-once/read-occasionally media.

## Connect External Storage

Sync files directly from your existing cloud storage instead of uploading manually. Mixpeek reads from your provider — no migration needed. This is a **two-step** flow: create a reusable **connection** (holds the credentials, lives at the organization level), then attach a **sync** to a bucket that references it.

**Step 1 — Create the connection** (once per provider account):

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/connections" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production S3",
    "provider_type": "s3",
    "provider_config": {
      "provider_type": "s3",
      "bucket_name": "my-source-bucket",
      "region": "us-east-1",
      "credentials": {
        "access_key_id": "AKIA...",
        "secret_access_key": "..."
      }
    }
  }'
```

The response includes a `connection_id` (`conn_...`). Credentials are encrypted at rest and reusable across buckets.

**Step 2 — Attach a sync** to your bucket (flat body — no wrapper objects):

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "source_path": "/videos/",
    "sync_mode": "continuous",
    "polling_interval_seconds": 3600
  }'
```

Then trigger the first sync:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID/trigger" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

After the initial sync, new files are picked up automatically at the configured polling interval. Use `continuous` mode (vs `initial_only`) to keep picking up new and changed files — only new or modified files since the last sync are processed, so existing files aren't reprocessed.

| Provider | Auth Method | S3-Compatible |
| - | - | - |
| AWS S3 | IAM User / Role | Native |
| Google Cloud Storage | Service Account Key | No |
| Azure Blob Storage | Access Key / Managed Identity | No |
| Cloudflare R2 | R2 API Token | Yes |
| Backblaze B2 | Application Key | Yes |
| Wasabi | Access Key | Yes |
| Tigris | Access Key | Yes |
| Box | OAuth | No |
| Mux | API Token | No |
| Supabase | Service Key | Yes |

See [Object Storage providers](/docs/integrations/object-storage/overview) for provider-specific setup guides.

[Sync API →](/docs/api-reference/bucket-syncs/create-sync-configuration)

## Register Objects

Objects are raw multimodal assets within a bucket. Two paths:

**URL references** — point to files in your existing storage:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/objects" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "key_prefix": "/products",
    "blobs": [
      { "property": "hero_image", "type": "image", "data": "https://example.com/photo.jpg" },
      { "property": "product_text", "type": "text", "data": "Wireless headphones" }
    ]
  }'
```

**Inline text.** A `text` blob takes the text itself in `data`:
`"data": "Wireless headphones"`, or `"data": {"text": "Wireless headphones"}`. A string
that starts with `http://`, `https://`, `s3://` or `file://` is read as a URL. A bare
base64 string that decodes to readable text is rejected with a 422. To send encoded text, use
`"data": "data:text/plain;base64,<encoded_data>"` or
`"data": {"base64": "<encoded_data>", "mime_type": "text/plain"}`.

**Direct uploads** — upload to Mixpeek-managed storage via presigned URLs:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/uploads" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{ "filename": "photo.jpg", "content_type": "image/jpeg" }'
```

Then PUT the file to the returned `presigned_url` and confirm with `POST /uploads/{id}/confirm`.

For bulk imports, use [batch uploads](/docs/api-reference/bucket-uploads/batch-create-uploads) or connect your object storage via [sync configurations](/docs/integrations/object-storage/overview).

### Content addressing

Every blob carries a `details.hash` in `sha256:<hex>` form. You can supply it, or
let the platform compute it.

Send `checksum` on a blob to pin the value yourself. It propagates verbatim to
the stored blob's `details.hash`:

```json theme={null}
{
  "property": "hero_image",
  "type": "image",
  "data": "https://example.com/photo.jpg",
  "checksum": "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
}
```

A bare 64-character hex string is accepted and normalized to `sha256:<hex>`.

> [!NOTE]
>
> **A malformed `checksum` is rejected with a 422 that tells you how to fix it**,
> so a typo cannot leave you with a blob you believe is pinned and is not.
>
> That guardrail arrived with the field itself. A deployment that predates it
> ignores `checksum` instead of rejecting it, and you get `details.hash: null`
> with no error — which is the failure the 422 exists to prevent. Confirm your
> deployment has it before relying on it:
>
> ```bash theme={null}
> for i in $(seq 1 12); do
>   curl -sS -H 'Connection: close' "$MIXPEEK_API_URL/docs/openapi.json" \
>     | python3 -c "import json,sys; print('checksum' in json.load(sys.stdin)['components']['schemas']['CreateBlobRequest']['properties'])"
> done | sort | uniq -c
> ```
>
> Twelve `True` means the guardrail is live everywhere. Any `False` means part of
> the fleet still drops `checksum` silently, so verify `details.hash` on a written
> blob rather than assuming it took.
>
> Sample rather than call once, and keep `Connection: close`. A single request
> answers *which server replied*, not *whether every server agrees*, and
> keepalive pins you to one of them — both give a clean unanimous answer from a
> mid-rollout fleet. Measured on Mixpeek Cloud during one rollout: 9 of 12
> fetches carried the field and 3 did not.
>
> Point it at **your own** host. A dedicated tenant deployment rolls on its own
> schedule, so it can be several releases behind the shared API or ahead of it.
> Measured the same minute: the shared API was mid-rollout on a newer build while
> a tenant host was converged on an older one, and the two gave opposite answers
> about the same field.

Omit `checksum` and the platform fills `details.hash` itself wherever it touches
the bytes: inline-text blobs on write, and every object arriving through
[storage sync](/docs/integrations/object-storage/overview).

Inline text works in both dialects, `"data": "some text"` and
`"data": {"text": "some text"}`, and both now persist. The dict form previously
skipped persistence entirely; blobs written that way also gain a
`properties.url` they did not have before.

> [!WARNING]
>
> **`details.hash` can be `null` on older objects.** Content addressing applies to blobs
> the platform has touched since the feature landed, so objects synced before then carry
> no hash.
>
> A null hash means the hash was never computed, and it says nothing about the content.
> Do not write code that treats `details.hash` as reliably present on older objects.

[Object API →](/docs/api-reference/bucket-objects/create-object) · [Upload API →](/docs/api-reference/bucket-uploads/create-upload)

## Process with Batches

Batches group objects for extraction. Create a batch, then submit it:

```bash theme={null}
# Create batch
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/batches" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{ "object_ids": ["obj_abc", "obj_def"] }'

# Submit for processing
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/batches/$BATCH_ID/submit" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

### Batch Lifecycle

```
DRAFT → QUEUED → PROCESSING → COMPLETED
                     ↘        ↘
                    FAILED    COMPLETED_WITH_ERRORS
```

Poll `GET /v1/buckets/{id}/batches/{id}` until the status is terminal — `COMPLETED`, `COMPLETED_WITH_ERRORS`, `FAILED`, or `CANCELED` (a poller that waits only for `COMPLETED` hangs on partial success) — or use [webhooks](/docs/platform/operations#webhooks) to get notified on `batch.completed`.

[Batch API →](/docs/api-reference/bucket-batches/create-batch)


# Extract Features
Source: https://docs.mixpeek.com/docs/platform/processing

Turn raw files into searchable documents by picking what you want to search by per file type — collections run the pipeline for you

## Collections

Collections bind a bucket to a processing pipeline. You choose **[features](/docs/processing/features)** — what you want to search by (visual similarity, faces, on-screen text, ...) — and Mixpeek resolves the pipeline internally. When you submit a batch, the engine processes each object and produces searchable documents.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "product-images",
    "source": { "type": "bucket", "bucket_ids": ["'$BUCKET_ID'"] },
    "features": ["image_search"]
  }'
```

Discover the menu with `GET /v1/collections/features` — every modality's unit, base feature, and add-ons (faces, on-screen text, layout, ...), with live rates. Add-on features like `faces` create a companion collection over the same source, so each pipeline's outputs stay independently versioned and queryable.

A single object can feed multiple collections — each extracting different features. Documents retain lineage to the source object via `root_object_id`.

[Features guide →](/docs/processing/features) · [Collection API →](/docs/api-reference/collections/create-collection)

> [!NOTE]
>
> Existing configs using explicit `feature_extractor` objects keep working as a deprecated alias — see the [migration guide](/docs/processing/extractor-migration). For advanced wiring (custom input mappings, field passthrough, parameters), see [Pipeline Configuration](/docs/processing/feature-extractors).

### Know the cost before you run

`POST /v1/organizations/billing/estimate` quotes planned ingestion in dollars — same rating engine that bills you:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/billing/estimate" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "items": [{ "mime_type": "video/mp4", "minutes": 42 }],
    "features": ["faces", "onscreen_text"]
  }'
```

See [Billing & Pricing](/docs/platform/billing) for units, rates, and tier usage pools.

### Embedding task

Instruction-aware embedding models use a **task hint** to optimize the embedding for a specific downstream use case. Set `embedding_task` at the collection level so it applies to every task-aware model in the pipeline.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "product-clusters",
    "embedding_task": "clustering",
    "source": { "type": "bucket", "bucket_ids": ["'$BUCKET_ID'"] },
    "features": ["text_search"]
  }'
```

| Task | Use Case | Default |
| - | - | - |
| `retrieval_document` | Search: find documents from queries | **Yes** |
| `retrieval_query` | Rare at index time — query-side is automatic | No |
| `semantic_similarity` | Symmetric comparison (dedup, matching) | No |
| `classification` | Document categorization pipelines | No |
| `clustering` | Grouping documents into clusters | No |

> [!NOTE]
>
> You almost never need to set this. The default `retrieval_document` is correct for search, and at query time Mixpeek automatically uses `retrieval_query`. Only override for clustering, classification, or symmetric similarity. Non-instruction-aware models ignore this setting.

## Feature URIs

Every extracted feature is addressed by a URI that pins it to a specific pipeline version:

```
mixpeek://{pipeline_name}@{version}/{output_name}
```

Feature URIs are referenced by retriever stages, taxonomies, and clustering jobs. They guarantee query-time compatibility with the extraction pipeline — swap the URI, re-embed, everything downstream stays consistent. Discover the URIs a collection produces with `GET /v1/collections/{id}/features` rather than constructing them by hand; see [Pipeline Configuration](/docs/processing/feature-extractors#feature-uris-the-stable-contract).

## Tiered pipelines

When a batch is submitted, the engine runs a DAG of pipelines:

1. **Tier 1** collections process raw objects from the bucket
2. **Tier 2** collections consume Tier 1 documents as input
3. Each tier waits for dependencies before executing

```
video → scenes (Tier 1) → faces per scene (Tier 2) → expressions per face (Tier 3)
```

Collections define the pipeline through their `source` and feature configuration. Dependencies are resolved automatically. See [Multi-Tier Feature Extraction](/docs/processing/multi-tier-extractors).

## What Mixpeek extracts

| Modality | Unit | Base feature | Add-on features |
| - | - | - | - |
| Image | per image | `image_search` — visual embeddings | `faces`, `multimodal_understanding` |
| Video | per minute | `video_search` — scene embeddings | `faces`, `onscreen_text`, `audio_fingerprint`, `multimodal_understanding` |
| Audio | per minute | `audio_search` — acoustic fingerprint + embeddings | `multimodal_understanding` |
| Document | per page | `document_search` — page embeddings | `faces`, `document_layout`, `multimodal_understanding` |
| Text | per token | `text_search` — semantic embeddings | `multimodal_understanding` |
| Web | per crawled page | `web_crawl` — crawl + extract | — |

Clustering and taxonomy enrichment are included on every modality at no additional charge. The live menu (with rates) is always `GET /v1/collections/features` — see the [Features guide](/docs/processing/features).

## Custom extractors

For extraction logic beyond the built-in features, build custom extractors:

```bash theme={null}
pip install mixpeek
mixpeek plugin init my-extractor     # Scaffold from template
mixpeek plugin test my-extractor     # Validate locally
mixpeek plugin publish my-extractor  # Upload and deploy
```

Custom extractors run on managed infrastructure with access to GPU/CPU resources, HuggingFace models, and LLM services. Once published, select yours in any collection as `features: ["custom:my-extractor"]` — it prices per unit from its declared compute profile, exactly like native features.

See the [full custom extractors guide](/docs/processing/custom-extractors) for manifest format, pipeline hooks, security constraints, and deployment lifecycle.

[Custom Extractors →](/docs/processing/custom-extractors)


# Syncs
Source: https://docs.mixpeek.com/docs/platform/syncs

Automatically ingest files from external storage into Mixpeek buckets on a schedule

Syncs pull files from your existing storage providers into Mixpeek buckets on a schedule. No migration required — connect your cloud storage, configure what to sync, and Mixpeek keeps your bucket up to date.

## How Syncs Work

```
┌──────────────┐     poll      ┌──────────────┐    create     ┌──────────────┐
│   External   │◄──────────────│    Sync      │───objects────►│    Bucket     │
│   Storage    │───file list──►│   Worker     │               │              │
│  (S3, GCS,   │               │              │───submit─────►│  Collection  │
│   Mux, etc.) │               │              │   batches     │   Pipeline   │
└──────────────┘               └──────────────┘               └──────────────┘
                                     │
                                     ▼
                              Resume cursor,
                              metrics, DLQ
```

1. **Poll** — At each interval, Mixpeek lists files from your storage provider using the configured `source_path` and filters.
2. **Filter** — Files are matched against glob patterns, size limits, MIME types, and provider-specific metadata filters.
3. **Create objects** — Matching files are registered in the target bucket. Duplicates are skipped by default via source tracking.
4. **Submit batches** — Objects are grouped into batches and submitted to the bucket's collection pipeline for processing.
5. **Checkpoint** — A resume cursor is saved so the next poll picks up where the last one left off.

## Sync Modes

| Mode | Behavior | Use Case |
| - | - | - |
| `continuous` | Polls repeatedly at `polling_interval_seconds` | Ongoing ingestion — new files are picked up automatically |
| `initial_only` | Runs once, then stops | One-time backfill or migration |

## Create a Sync

**1. Connect your storage**

Create a [storage connection](/docs/integrations/object-storage/overview) with credentials for your provider.

This step needs an API key with the **`admin`** permission, because connections hold credentials and sit behind organization administration. The key presets in Studio stop at read + write + delete; tick **Admin** in the key's Edit dialog to grant it. Everything below (buckets, syncs, triggers) works with a `write` key.

**2. Configure the sync**

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "source_path": "/videos/2025/",
    "sync_mode": "continuous",
    "polling_interval_seconds": 300,
    "skip_duplicates": true,
    "file_filters": {
      "include_patterns": ["*.mp4", "*.mov"],
      "max_size_bytes": 5368709120
    }
  }'
```

**3. Trigger the first sync**

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID/trigger" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

After the initial run, continuous syncs poll automatically at the configured interval.

## Configuration Reference

### Core Settings

| Field | Type | Default | Description |
| - | - | - | - |
| `connection_id` | string | required | Storage connection to pull from |
| `source_path` | string | required | Path in external storage (format varies by provider) |
| `sync_mode` | string | `continuous` | `continuous` or `initial_only` |
| `polling_interval_seconds` | int | `300` | Poll frequency (30–900 seconds) |
| `batch_size` | int | `50` | Files per batch (1–100) |
| `skip_duplicates` | bool | `true` | Skip files already in the bucket |
| `max_objects_per_run` | int | `100000` | Hard cap on objects processed in one sync run. Minimum `1` |
| `description` | string | — | Free-text description stored with the sync configuration |
| `reconcile` | object | — | Reconcile on source change: `on_delete` (cascade-delete objects when the source asset is removed, default `true`), `on_update` (propagate metadata changes + re-extract, default `true`; on Iconik, `true` repeats the per-asset calls for every asset on every walk, see [Iconik](/docs/integrations/object-storage/iconik#modification-detection)), `on_filter_drift` (drop objects that no longer match filters, default `true`) |

> [!NOTE]
>
> `max_objects_per_run` and `description` are accepted at create as well as on update. Both
> were previously settable only by `PATCH`ing a sync after creating it.

### File Filters

Narrow which files get synced. All filters combine with AND logic.

`include_patterns` and `exclude_patterns` are glob patterns, matched the same way for every provider. A pattern without `/` matches the file name in any folder, so `*.mp4` and `qa-*.mp4` match files at any depth. A pattern with `/` matches the path from the sync's source root, so `videos/*.mp4` matches only files in the `videos` folder.

When the filters exclude every file a run discovers, the run's warning says so, and the sync job's `metadata.files_excluded_by_filter` lists how many files each filter excluded with a few example paths.

```json theme={null}
{
  "file_filters": {
    "include_patterns": ["*.mp4", "*.mov", "*.webm"],
    "exclude_patterns": ["*/drafts/*", "*_temp.*"],
    "min_size_bytes": 1024,
    "max_size_bytes": 5368709120,
    "modified_after": "2025-01-01T00:00:00Z",
    "mime_types": ["video/mp4", "video/quicktime"]
  }
}
```

### Metadata Filters

Filter on provider-specific metadata fields. Useful for syncing only assets that match certain tags, statuses, or custom fields in your storage system.

```json theme={null}
{
  "file_filters": {
    "metadata_filters": [
      { "field": "status", "operator": "equals", "value": "approved" },
      { "field": "tags", "operator": "contains", "value": "hero" }
    ]
  }
}
```

Supported operators: `equals`, `not_equals`, `contains`, `not_contains`, `gt`, `lt`, `gte`, `lte`, `exists`.

### Schema Mapping

Map provider metadata to bucket schema fields during sync, so structured data arrives alongside your files.

```json theme={null}
{
  "schema_mapping": {
    "mappings": {
      "product_name": { "target_type": "field", "source": { "type": "metadata", "key": "title" } },
      "category": { "target_type": "field", "source": { "type": "tag", "key": "category" } }
    }
  }
}
```

## Lifecycle Management

### Pause and Resume

Temporarily stop a sync without losing progress:

```bash Pause theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID/pause" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

```bash Resume theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID/resume" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

### Manual Trigger

Force a sync to run immediately, outside the polling schedule:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID/trigger" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

### Monitoring

Check sync status and metrics:

```bash theme={null}
curl "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/syncs/$SYNC_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

Response includes:

| Metric | Description |
| - | - |
| `total_files_discovered` | Cumulative files found in source |
| `total_files_synced` | Successfully synced files |
| `total_files_failed` | Files that failed after retries (sent to DLQ) |
| `total_bytes_synced` | Total data transferred |
| `last_sync_at` | When the last sync completed |
| `next_sync_at` | When the next poll is scheduled |
| `consecutive_failures` | Sequential failure count (auto-suspends after threshold) |

### Job timing

Each run is a sync job. List them at `GET /v1/buckets/{bucket_id}/syncs/{sync_config_id}/jobs`.

| Field | Meaning |
| - | - |
| `started_at` | When the current attempt started executing. `null` while the job waits to run. |
| `completed_at` | When the job finished. |
| `metadata.started_at_source` | Set to `running_transition` when `started_at` is a measured execution start. |
| `metadata.prior_attempts` | Earlier attempts of a job that was re-run, each with its own start and elapsed time. |
| `resumed_via` | `acks_late_redelivery` when the job was re-run after its worker died mid-run. |

A job has no pending status. It is created as `RUNNING` before it executes, so to find jobs that have not started yet, filter on `started_at` being `null` rather than on `status`.

`completed_at` minus `started_at` is the execution time of the current attempt. `started_at` minus the job's creation time covers dispatch, admission and setup, so it measures more than queue time.

> [!WARNING]
>
> Check `metadata.started_at_source` before you compute a duration. Jobs without it were recorded before September 14, 2026, and their `started_at` holds the time the record was created, so their execution time is unknown. Averaging durations across both kinds of job mixes measured times with creation times, and the result looks plausible.

`metadata.prior_attempts` only records attempts that carried `started_at_source`, so an older job without it may still have been retried. On those jobs `resumed_via` is the only retry signal, and its absence does not show that a job never retried.

## Robustness

Syncs are designed for unattended, long-running operation:

* **Distributed locking** prevents concurrent runs of the same sync
* **Resume cursors** checkpoint progress so interrupted syncs pick up where they left off
* **Dead letter queue** retries failed files up to 3 times before marking them as failed
* **Auto-suspend** pauses syncs after consecutive failures to prevent runaway errors
* **Idempotent ingestion** uses source tracking to never duplicate objects on retries
* **Reconciliation** (the `reconcile` object) cascades source deletes (`on_delete`), propagates metadata updates (`on_update`), and drops objects that no longer match your filters (`on_filter_drift`) — all default `true`

> [!WARNING]
>
> If a sync gets stuck (e.g., a worker crashed mid-run), use the [force unlock endpoint](/docs/api-reference/bucket-syncs/force-unlock-sync-configuration) to release the distributed lock.

[Sync API reference →](/docs/api-reference/bucket-syncs/create-sync-configuration)


# Custom Extractor API (Dedicated Infrastructure)
Source: https://docs.mixpeek.com/docs/processing/custom-extractor-api

HTTP reference for the custom-extractor upload, deploy, real-time, and version-management endpoints — available on dedicated Enterprise deployments

> [!WARNING]
>
> **Dedicated infrastructure only.** These endpoints are served by dedicated Enterprise deployments — they are **not** exposed on the shared public API at `api.mixpeek.com`, where they return `404`/`405`. See [Custom Extractors → Availability](/docs/processing/custom-extractors#availability). [Contact your account team](https://mixpeek.com/contact) to provision a dedicated deployment.
>
> Because this surface isn't in the shared OpenAPI, it isn't part of the auto-generated API Reference. This page is the hand-maintained contract; the reference client is `plugins.py`.

> [!NOTE]
>
> **Naming.** On this surface, custom extractors are addressed as **plugins**: the path segment is `/plugins`, and a deployed extractor's `plugin_id` is `{name}_{version}` with dots replaced by underscores — e.g. `text_embed@1.0.0` → `text_embed_1_0_0`. All endpoints are namespace-scoped and take `Authorization: Bearer $MIXPEEK_API_KEY`.

## Lifecycle: upload → deploy → search

The CLI (`python plugins.py push` / `deploy`) wraps these calls. The raw HTTP flow:

```bash theme={null}
NS=ns_...                 # your namespace id
API=https://<your-dedicated-host>/v1
AUTH=(-H "Authorization: Bearer $MIXPEEK_API_KEY" -H "Content-Type: application/json")

# 1. Request a presigned upload URL (auto-bumps the patch version if you omit it)
UP=$(curl -s -X POST "$API/namespaces/$NS/plugins/uploads" "${AUTH[@]}" \
  -d '{"name":"text_embed","version":"1.0.0","file_size_bytes":5000}')
UPLOAD_ID=$(echo "$UP" | jq -r '.upload_id')
PRESIGNED=$(echo "$UP" | jq -r '.presigned_url')

# 2. PUT the zipped extractor to S3
curl -s -X PUT "$PRESIGNED" -H "Content-Type: application/zip" --data-binary @text_embed.zip

# 3. Confirm — runs the security scan + manifest validation
curl -s -X POST "$API/namespaces/$NS/plugins/uploads/$UPLOAD_ID/confirm" "${AUTH[@]}" \
  -d '{"message":"initial release"}'
# → { "success": true, "plugin_id": "text_embed_1_0_0", "validation_errors": [] }

# 4. Deploy (omit deployment_type for full deploy incl. realtime; batch_only skips the HTTP endpoint)
curl -s -X POST "$API/namespaces/$NS/plugins/text_embed_1_0_0/deploy" "${AUTH[@]}"

# 5. Poll deployment status until DEPLOYED
curl -s "$API/namespaces/$NS/plugins/text_embed_1_0_0/status" "${AUTH[@]}"
```

Once `DEPLOYED`, reference the extractor's `feature_uri` in a collection's `feature_extractor` and in retriever `feature_search` stages exactly as you would a built-in — see the [end-to-end walkthrough](/docs/processing/custom-extractors#end-to-end-extractor-collection-retriever).

| Deployment status | Meaning |
| - | - |
| `QUEUED` / `PENDING` | Waiting in / triggered the deployment queue |
| `IN_PROGRESS` | Blue-green rollout in progress |
| `DEPLOYED` | Ready for real-time inference |
| `NOT_DEPLOYED` | Batch-only mode |
| `FAILED` | Check the `error` field |

## Endpoint reference

All paths are prefixed with `/v1/namespaces/{namespace_id}`.

### Upload & deploy

| Method | Path | Purpose |
| - | - | - |
| `POST` | `/plugins/uploads` | Request a presigned upload URL. Body: `{name, version?, file_size_bytes}` → `{upload_id, presigned_url, version}` |
| `POST` | `/plugins/uploads/{upload_id}/confirm` | Confirm the S3 upload; runs security scan + manifest validation. Body: `{message?}` → `{success, plugin_id, validation_errors}` |
| `POST` | `/plugins` | Direct multipart upload (alternative to the presigned flow) |
| `POST` | `/plugins/{plugin_id}/deploy` | Deploy or redeploy. Query: `deployment_type=batch_only` to skip the realtime endpoint |
| `POST` | `/plugins/{plugin_id}/undeploy` | Tear down the deployment (keeps the version) |
| `GET` | `/plugins/{plugin_id}/status` | Deployment status |
| `POST` | `/plugins/{plugin_id}/realtime/test` | Invoke the deployed `realtime.py` endpoint with a test payload |

### Discover & inspect

| Method | Path | Purpose |
| - | - | - |
| `GET` | `/plugins` | List custom plugins in the namespace |
| `GET` | `/plugins/{plugin_id}` | Plugin details (schemas, deployment + validation status) |
| `GET` | `/plugins/{plugin_id}/source` | Download source files of the active version |
| `GET` | `/plugins/{plugin_id}/download` | Presigned archive download URL |
| `GET` | `/plugins/available` | Org-level plugins available to enable in this namespace |

### Version management

| Method | Path | Purpose |
| - | - | - |
| `GET` | `/plugins/by-name/{slug}/versions` | Version history with deploy timestamps + messages |
| `POST` | `/plugins/by-name/{slug}/rollback` | Restore a previous version as active |
| `GET` | `/plugins/by-name/{slug}/diff?v1=1.0.0&v2=1.0.1` | Diff source files between two versions |

### Org plugins & namespace config

| Method | Path | Purpose |
| - | - | - |
| `POST` | `/plugins/org/{plugin_id}/enable` | Enable an org-level plugin in this namespace |
| `POST` | `/plugins/org/{plugin_id}/disable` | Disable an org-level plugin in this namespace |
| `POST` | `/plugins/reconfigure` | Reconfigure the namespace vector-store schema for a plugin's features |
| `DELETE` | `/plugins/{plugin_id}` | Delete a plugin version |

## CLI equivalents

`plugins.py` is the maintained client for this surface (reads `MIXPEEK_API_KEY`, `MIXPEEK_NAMESPACE`, `MIXPEEK_API_URL`):

| CLI | Endpoint(s) |
| - | - |
| `push` | `POST /plugins/uploads` → `PUT` → `POST /plugins/uploads/{id}/confirm` |
| `pull` | `GET /plugins/by-name/{slug}/versions` → `GET /plugins/{id}/source` |
| `status` | `GET /plugins/by-name/{slug}/versions` |
| `log` | `GET /plugins/by-name/{slug}/versions` |
| `rollback` | `POST /plugins/by-name/{slug}/rollback` |
| `diff` | `GET /plugins/by-name/{slug}/diff` |
| `lint` / `test` | offline — no API call (work on any plan) |

See [Custom Extractors](/docs/processing/custom-extractors) for the full authoring guide and [Local Development](/docs/processing/custom-extractors#local-development) for `lint`/`test`.


# Custom Extractors
Source: https://docs.mixpeek.com/docs/processing/custom-extractors

Build, test, and deploy your own feature extractors and real-time inference endpoints on Mixpeek infrastructure

> **[Deploy your own extractor on Mixpeek](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_custom_extractors\&utm_campaign=managed_onramp)**
>
> Create a managed namespace to get your API key. Uploading and running your own extractor needs a dedicated deployment; see [Availability](#availability) for what the shared API supports.

> **[Browse the extractor catalog on GitHub](https://github.com/mixpeek/mixpeek-extractors)**
>
> Runnable reference for every built-in Mixpeek extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry, so it always matches production.

![Custom extractor flow: a single extractor archive powers both the ingestion pipeline (batch Ray Data) and the retrieval realtime endpoint (Ray Serve HTTP), used by retriever stages like feature\_search, rerank, and agentic\_enrich](https://mintcdn.com/mixpeek/JZ8e-UKMKuc4FtNv/assets/mixpeek-custom-plugins.svg?fit=max\&auto=format\&n=JZ8e-UKMKuc4FtNv\&q=85\&s=1f1d4f9fe0d081825609873010e514fc)

Custom extractors let you run your own code on Mixpeek infrastructure — inside the same Ray cluster that powers the built-in extractors and retriever stages. You keep full control of the logic, model, and I/O; Mixpeek handles packaging, scheduling, GPU allocation, caching, and observability.

## Availability

> [!WARNING]
>
> **The upload → deploy → real-time lifecycle runs on dedicated Enterprise infrastructure — it is not exposed on the shared public API at `api.mixpeek.com`.** What works where:
>
> | Capability | Shared API (`api.mixpeek.com`) | Dedicated deployment |
> | - | - | - |
> | Discover extractors (`GET /v1/namespaces/{ns}/extractors`, `…/{id}`) | ✅ | ✅ |
> | Author + `lint` + `test` locally (no API key) | ✅ | ✅ |
> | Contribute via the [Submission workflow](/docs/processing/extractor-marketplace) (reviewed → merged as built-in) | ✅ | ✅ |
> | Upload / deploy / undeploy / real-time inference | ❌ | ✅ |
> | Version management (`push` / `pull` / `rollback` / `diff`) | ❌ | ✅ |
>
> On the shared API the upload/deploy/realtime endpoints refuse by design, with error type `DedicatedInfrastructureRequired` and the alternatives below. [Contact your account team](https://mixpeek.com/contact) to provision a dedicated deployment for self-service custom-extractor uploads, or use Submissions to ship an extractor into the built-in catalog. The dedicated upload/deploy/realtime HTTP contract is documented in the [Custom Extractor API (Dedicated Infrastructure)](/docs/processing/custom-extractor-api) reference.

> [!TIP]
>
> Want your extractor available to everyone without dedicated infra? Submit it for review to be merged into the built-in catalog — see [Extractor Submissions](/docs/processing/extractor-marketplace).

## What You Can Build

Custom extractors plug into two places in the warehouse:

### 1. Feature Extractors (Decomposition)

Attach custom logic to a collection so every ingested object flows through your pipeline during decomposition. Your extractor is **your vocabulary** — unlike built-in pipelines (which are selected via [features](/docs/processing/features)), custom extractors keep their explicit names and are selected as a custom feature:

```json theme={null}
{
  "collection_name": "filings",
  "source": { "type": "bucket", "bucket_ids": ["bkt_123"] },
  "features": ["custom:my_extractor"]
}
```

The explicit `feature_extractor` config (shown later on this page) also works — use it when you need custom `input_mappings` or parameters. Use custom extractors to:

* Embed domain-specific content with your own model (fine-tuned CLIP, proprietary audio encoder, etc.)
* Extract structured attributes via a VLM you manage (brand compliance, regulated content classification)
* Transcribe, OCR, or segment media with a custom pre/post-processing chain
* Produce multiple named vector indexes from a single pass

Outputs land in MVS and MongoDB with the same feature URI scheme as built-in pipelines (`mixpeek://my_extractor@1.0.0/my_embedding`), so retrievers, taxonomies, and clusters can reference them.

> [!NOTE]
>
> **Pricing**: a `custom:<name>` feature derives its per-unit rate from the compute profile your extractor declares — the same machinery that prices native features. Quote it before running via `POST /v1/organizations/billing/estimate`; see [Billing & Pricing](/docs/platform/billing).

### 2. Retriever Operations (Query Time)

An extractor's `realtime.py` exposes a Ray Serve HTTP endpoint that retriever stages can call during execution. Use this to power:

* **`feature_search`** — embed queries at search time with the same model you used during ingestion, so the query vector lives in the same space as the indexed vectors
* **Inference operations** — on-the-fly classification, scoring, or re-ranking against your model
* **LLM calls** — wrap a hosted or private LLM behind a stable contract, with platform-managed secrets and cost tracking
* **Classifiers** — apply your own classifier to candidate results mid-pipeline

This is what lets a single custom extractor own both halves of a retrieval flow: it encodes documents on the way in, and encodes the query on the way out.

## Delivery Formats

Ship your extractor in either of two formats:

| Format | When to Use |
| - | - |
| **Zip archive** (`.zip`) | Pure-Python extractors. Mixpeek resolves dependencies against the managed runtime. Scanned by the security linter before deploy. Limit: 500 MB / 1,000 files. |
| **Container image** (OCI) | Extractors that need system packages, custom CUDA builds, compiled binaries, or non-Python runtimes. Base your image on the Mixpeek engine image, push to your org-scoped Artifact Registry repo, and set `container_image` in `manifest.py`. See [BYO Container Image](#byo-container-image). |

Both formats expose the same runtime APIs (batch `__call__`, real-time `run_inference`, platform LLM/secret accessors).

***

## Extractor Structure

Every extractor has the same layout:

```
my_extractor/
├── manifest.py      # Schemas, metadata, vector indexes
├── pipeline.py      # Batch processing pipeline
├── realtime.py      # Real-time HTTP endpoint (optional)
└── processors/
    └── core.py      # Your processing logic
```

* **`manifest.py`** declares what your extractor accepts, produces, and which vector indexes to create
* **`pipeline.py`** wires your processor into the Ray Data batch pipeline
* **`realtime.py`** exposes a Ray Serve endpoint for query-time inference (e.g., embedding queries for `feature_search`)
* **`processors/`** contains your actual logic — model loading, embedding, classification, etc.

***

## Manifest

The manifest is your extractor's contract with the platform.

```python theme={null}
# manifest.py
from pydantic import BaseModel, Field
from typing import List

class MyInput(BaseModel):
    text: str = Field(..., description="Input text to process")

class MyOutput(BaseModel):
    embedding: List[float] = Field(..., description="384-dim embedding vector")

class MyParams(BaseModel):
    model_size: str = Field(default="base", description="base or large")

metadata = {
    "feature_extractor_name": "my_extractor",
    "version": "1.0.0",
    "description": "Custom text embedding extractor",
    "category": "text",
    "inference_type": "embedding",  # declares real-time inference capability
}

input_schema = MyInput
output_schema = MyOutput
parameter_schema = MyParams
supported_input_types = ["text"]

features = [
    {
        "feature_type": "embedding",
        "feature_name": "my_embedding",
        "embedding_dim": 384,
        "distance_metric": "cosine",
    },
]
```

### Vector Index Keys

> [!WARNING]
>
> Use the **exact key names** below. Wrong keys silently create a collection with no vector indexes — your batch will show `COMPLETED` but produce 0 documents.

| Key | Required | Description |
| - | - | - |
| `feature_type` | Yes | Must be `"embedding"` |
| `feature_name` | Yes | Name of the vector index |
| `embedding_dim` | Yes | Vector dimensionality |
| `distance_metric` | Yes | `"cosine"`, `"euclid"`, or `"dot"` |

Multiple vectors are supported — add one entry per embedding your extractor produces.

### Inference Type

Declare what kind of real-time inference your extractor provides by setting `inference_type` in `metadata`. This lets retriever stages validate that an extractor is compatible with the stage slot.

| Value | Contract | Compatible Stages |
| - | - | - |
| `embedding` | Returns `{vector: [float]}` | `feature_search` |
| `rerank` | Accepts `{pairs: [[q, d]]}`, returns `{scores: [float]}` | `rerank` |
| `classify` | Accepts `{text: str}`, returns `{labels: [{label, confidence}]}` | `classify` |
| `generate` | Accepts `{prompt: str}`, returns `{text: str}` | `llm_filter`, `llm_enrich` |
| `general` | No specific contract | Raw `/inference` endpoint only |

If `inference_type` is omitted, the extractor can only be called via the raw inference endpoint.

### Compute Profile

Control resource allocation by adding `compute_profile` to your manifest:

```python theme={null}
compute_profile = {
    "resource_type": "cpu",      # "cpu", "gpu", or "api"
    "batch_size": 32,            # Rows per __call__ (default: 64)
    "max_concurrency": 4,        # Parallel Ray actors (default: 2)
}
```

> [!TIP]
>
> For API-based or hash-based extractors that don't need GPU, set `resource_type: "cpu"` to skip GPU allocation — saves \~3 minutes startup time and costs \~6x less.

### Serve Block

An extractor that answers real-time calls declares how it is served in `compute_profile.serve`, one Ray Serve app. It defaults to zero replicas when idle.

| Field | Type | Default | What it does |
| - | - | - | - |
| `enabled` | boolean | `true` | Serve this extractor in real time. `false` is batch only |
| `min_replicas` | integer | `0` | Replicas kept when idle. `0` scales to zero; the first call after that waits for a replica to start |
| `max_replicas` | integer | `1` | Most replicas under load |
| `min_reason` | string | none | Why replicas stay warm. Required when `min_replicas` is above 0 |
| `idle_timeout_s` | number, 300 to 1800 | `600` | Seconds of low load before a replica is removed, including the last one |
| `target_ongoing_requests` | integer | `5` | In-flight requests per replica the autoscaler aims for |
| `max_queued_requests` | integer | derived | Queued requests before new ones are refused with 503 |
| `placement` | `worker` or `head` | `worker` | Where replicas run. `head` needs a `min_reason` like any floor |

```python theme={null}
compute_profile = {
    "resource_type": "cpu",
    "serve": {
        "min_replicas": 0,
        "max_replicas": 2,
        "idle_timeout_s": 900,
    },
}
```

An extractor that uses a built-in app references it through `serve_app` and carries no `serve` block for a service it does not own. In a [manifest](/docs/operations/manifests#extractors), the block is validated and a floor above zero is refused with its monthly cost named; real-time calls still run under the extractor's built-in serve settings until the custom block is wired to deployment, and apply says so in `warnings`.

### BYO Container Image

If your extractor needs native binaries or system packages, specify a custom container image:

```python theme={null}
# manifest.py
container_image = "us-east1-docker.pkg.dev/mixpeek-inference-463103/extractors-<your-org-id>/my-image:1.0.0"
```

Base your image on the Mixpeek engine image to get Ray, FFmpeg, and all SDK helpers:

```dockerfile theme={null}
FROM us-east1-docker.pkg.dev/mixpeek-inference-463103/mixpeek-engine/engine-base:latest
RUN apt-get update && apt-get install -y libcustom-sdk ...
```

Images must be pushed to your org-scoped Artifact Registry repo. GKE Workload Identity handles pull auth. [Contact your account team](https://mixpeek.com/contact) to provision access.

***

## Batch Processor

Your processor receives a pandas DataFrame and returns it with new columns added.

```python theme={null}
# processors/core.py
import pandas as pd

class MyProcessor:
    def __init__(self, config, **kwargs):
        self.config = config
        self._model = None

    def _ensure_model_loaded(self):
        if self._model is None:
            from sentence_transformers import SentenceTransformer
            self._model = SentenceTransformer("all-MiniLM-L6-v2")

    def __call__(self, batch: pd.DataFrame) -> pd.DataFrame:
        self._ensure_model_loaded()
        texts = batch["data"].fillna("").tolist()
        embeddings = self._model.encode(texts).tolist()
        batch["my_embedding"] = embeddings
        return batch
```

### DataFrame Columns

Your `__call__` receives these columns:

| Column | Description |
| - | - |
| `data` | For text blobs: the raw string. For binary blobs: an S3 URL (not raw bytes). |
| `document_id` | Unique document ID |
| `object_id` | Source object in the bucket |
| `blob_type` | `"image"`, `"video"`, `"audio"`, `"text"` |
| `blob_property` | Property name from your bucket schema |
| `mime_type` | MIME type (e.g. `image/jpeg`) |

> [!WARNING]
>
> Always read from `batch["data"]` — not a column named after your blob property. If your bucket has a `text` property, the content is still in `batch["data"]`. (The local `test` harness feeds the same `data` column, so an extractor that passes `test` behaves the same in production.)

### Batched Processing

Process all rows together — never call a model or API inside a per-row loop:

```python theme={null}
# WRONG — one GPU call per row
for idx, row in batch.iterrows():
    embedding = self._model.encode(row["data"])
    batch.at[idx, "embedding"] = embedding

# RIGHT — single batched call
texts = batch["data"].fillna("").tolist()
batch["embedding"] = self._model.encode(texts).tolist()
```

### Loading Assets from S3

For binary blobs (images, video, audio), the `data` column contains S3 URLs. Use the Extractor SDK to download them:

```python theme={null}
from shared.extractors import open_asset
from shared.extractors.sdk import parallel_io, download_asset

class ImageProcessor:
    def __call__(self, batch: pd.DataFrame) -> pd.DataFrame:
        def load_image(url):
            from PIL import Image
            path, is_temp = download_asset(url, suffix=".jpg")
            img = Image.open(path).convert("RGB")
            if is_temp:
                import os; os.unlink(path)
            return img

        images = parallel_io(batch["data"].tolist(), load_image, max_workers=8)
        # batch-process all images on GPU...
        return batch
```

***

## Pipeline

Wire your processor into the Ray Data pipeline:

```python theme={null}
# pipeline.py
from engine.extractors.my_extractor.pipeline import (
    PipelineDefinition, ResourceType, StepDefinition, build_pipeline_steps
)
from .manifest import MyParams, metadata
from .processors.core import MyProcessor

def build_steps(extractor_request, container=None, **kwargs):
    params = MyParams(**(extractor_request.extractor_config.parameters or {}))

    pipeline = PipelineDefinition(
        name=metadata["feature_extractor_name"],
        version=metadata["version"],
        steps=[
            StepDefinition(
                service_class=MyProcessor,
                resource_type=ResourceType.CPU,
                config={"model_size": params.model_size},
            ),
        ]
    )
    return {"steps": build_pipeline_steps(pipeline)}
```

### Row Conditions

Filter which rows a step processes:

| Condition | Matches |
| - | - |
| `RowCondition.IS_TEXT` | text/\* MIME types |
| `RowCondition.IS_IMAGE` | image/\* MIME types |
| `RowCondition.IS_VIDEO` | video/\* MIME types |
| `RowCondition.IS_AUDIO` | audio/\* MIME types |
| `RowCondition.IS_PDF` | application/pdf |
| `RowCondition.ALWAYS` | All rows (default) |

### Using Built-in Models

Compose existing Mixpeek services instead of loading models yourself:

```python theme={null}
from shared.inference.registry import get_batch_service

WhisperBatch = get_batch_service("openai/whisper-large-v3-turbo")
E5Batch = get_batch_service("intfloat/multilingual-e5-large-instruct")

# Use in your pipeline steps
StepDefinition(service_class=E5Batch, resource_type=ResourceType.CPU, config=e5_config)
```

| Service | Type | Dimensions |
| - | - | - |
| `intfloat/multilingual-e5-large-instruct` | Embedding | 1024 |
| `google/siglip-base-patch16-224` | Embedding | 512 |
| `jinaai/jina-embeddings-v2-base-code` | Embedding | 768 |
| `BAAI/bge-reranker-v2-m3` | Reranker | — |
| `openai/whisper-large-v3-turbo` | Transcription | — |

For HuggingFace and your own fine-tuned weights, see the [Model Registry](/docs/processing/model-registry).

***

## Real-time Endpoint

Add `realtime.py` to expose an HTTP endpoint for query-time inference. This is what lets [retriever `feature_search` stages](/docs/retrieval/stages/feature-search) embed queries with your model.

```python theme={null}
# realtime.py
from shared.extractors.inference.serve import BaseInferenceService

class InferenceService(BaseInferenceService):
    def __init__(self):
        super().__init__()
        self._model = None

    async def run_inference(self, inputs: dict, parameters: dict) -> dict:
        if self._model is None:
            from sentence_transformers import SentenceTransformer
            self._model = SentenceTransformer("all-MiniLM-L6-v2")

        text = inputs.get("text", "")
        embedding = self._model.encode([text])[0].tolist()
        return {"embedding": embedding}
```

> [!WARNING]
>
> The return dict must include an `"embedding"` key — this is what `feature_search` uses as the query vector. For multi-vector extractors, include additional keys matching your `feature_name` values.

> [!NOTE]
>
> `realtime.py` handles embedding only, not retrieval. If your pipeline needs retrieval context (comparing against stored references), configure [retriever stages](/docs/retrieval/stages/overview) to handle that logic. The real-time endpoint serves on dedicated infrastructure (see [Availability](#availability)).

***

## Platform Services

### LLM Access

Use `container.llm` to call platform-managed LLMs with built-in cost tracking and caching:

```python theme={null}
# pipeline.py — pass to your processor
config = {"llm_service": container.llm}

# processors/core.py — call concurrently
from shared.extractors.sdk import concurrent_api_calls

async def analyze(text):
    return await self._llm.generate(
        instruction="Extract entities from this text",
        text=text,
        provider="google",
        model="gemini-2.5-flash",
        schema={"type": "object", "properties": {"entities": {"type": "array"}}}
    )

results = concurrent_api_calls(texts, analyze, max_concurrent=10)
```

| Provider | Models |
| - | - |
| `google` | `gemini-2.5-flash`, `gemini-2.5-pro` |
| `openai` | `gpt-4o`, `gpt-4o-mini` |
| `anthropic` | `claude-sonnet-4-20250514` |

### Secrets

Access encrypted org secrets at runtime via `container.secrets`:

```python theme={null}
api_key = await container.secrets.get("EXTERNAL_API_KEY")
```

For platform LLMs, use `container.llm` instead — it handles API keys automatically.

### CLI Tools

Custom extractors can't import `subprocess` directly. Use `run_tool` for whitelisted CLI tools:

```python theme={null}
from shared.extractors.sdk import run_tool

result = run_tool("ffmpeg", ["-y", "-i", input_path, "-c:v", "libx264", output_path], timeout=600)
```

Available tools: `ffmpeg`, `ffprobe`, `convert`, `identify`, `magick`, `exiftool`, `mediainfo`, `sox`, `soxi`, `REDline`, `art-cmd`

### Pre-installed Tools

The engine runtime includes these media tools, available via `run_tool`:

| Tool | Format | Description |
| - | - | - |
| `ffmpeg` / `ffprobe` | Standard video/audio | Transcode, extract frames, probe metadata |
| `REDline` | RED R3D | Decode RED cinema camera raw files to ProRes/DPX/EXR |
| `art-cmd` | ARRI RAW | Decode ARRI raw (.ari/.arriraw/.arx) to ProRes |
| `exiftool` | All media | Read/write EXIF and XMP metadata |
| `mediainfo` | All media | Detailed format and codec inspection |
| `convert` / `identify` | Images | ImageMagick image processing |
| `sox` / `soxi` | Audio | Audio processing and info |

**RED R3D** and **ARRI RAW** decode examples:

```python theme={null}
run_tool("REDline", ["--i", input_path, "--o", output_path, "--format", "201", "--resize", "2"], timeout=600)
run_tool("art-cmd", ["--input", input_path, "--output", output_dir, "--format", "prores"], timeout=600)
# Chain with FFmpeg for web playback:
run_tool("ffmpeg", ["-y", "-i", prores_path, "-c:v", "libx264", "-crf", "23", output_mp4], timeout=600)
```

***

## Typed SDK

The Extractor SDK provides typed base classes that replace bare Python variables with validated, IDE-friendly types — autocompletion, validation at upload time, and output column checking in the test harness. Import from `shared.extractors.sdk` or `shared.extractors`.

```python theme={null}
# manifest.py
from shared.extractors.sdk import ExtractorManifest, Feature

manifest = ExtractorManifest(
    feature_extractor_name="my_embedder",
    version="v1",
    description="Custom text embedder",
    dependencies=["sentence-transformers>=2.2"],
    system_packages=["libmagic1"],  # apt packages installed in the container
    features=[Feature.embedding("my_embedder_v1", dim=384)],
    inference_type="embedding",
)

# Backward-compat: AST parser reads these module-level variables
feature_extractor_name = manifest.feature_extractor_name
version = manifest.version
description = manifest.description
dependencies = manifest.dependencies
features = manifest.features_as_dicts()
feature_uri = manifest.feature_uri
```

```python theme={null}
# pipeline.py
from shared.extractors.sdk import BatchProcessor, prefetch_hf_model

class MyProcessor(BatchProcessor):
    def setup(self):
        prefetch_hf_model("sentence-transformers/all-MiniLM-L6-v2")
        from sentence_transformers import SentenceTransformer
        self.model = SentenceTransformer("all-MiniLM-L6-v2")

    def process(self, batch):
        texts = batch["data"].fillna("").tolist()
        batch["my_embedder_v1_embedding"] = self.model.encode(texts).tolist()
        return batch
```

`setup()` runs once on first batch (lazy model loading); `process()` runs on each batch. `prefetch_hf_model()` pre-downloads the model to the HF cache to reduce cold start.

```python theme={null}
# realtime.py
from shared.extractors.sdk import InferenceService

class MyInference(InferenceService):
    def setup(self):
        from sentence_transformers import SentenceTransformer
        self.model = SentenceTransformer("all-MiniLM-L6-v2")

    async def infer(self, inputs, parameters):
        text = inputs.get("text", "")
        return {"embedding": self.model.encode([text])[0].tolist()}
```

### SDK Reference

| Function / Class | Purpose |
| - | - |
| `ExtractorManifest` | Typed manifest with validation |
| `Feature.embedding(name, dim)` | Create an embedding feature definition |
| `Feature.classification(name, labels)` | Create a classification feature definition |
| `BatchProcessor` | Base class with `setup()` / `process()` lifecycle |
| `InferenceService` | Base class with `setup()` / `infer()` lifecycle |
| `prefetch_hf_model(model_id)` | Pre-download HF model to cache (cold start mitigation) |
| `parallel_io(items, fn, max_workers)` | Parallel file downloads and I/O |
| `concurrent_api_calls(items, async_fn, max_concurrent)` | Concurrent LLM/API calls |
| `open_asset(url, suffix)` | Context manager for S3 downloads |
| `download_asset(url, suffix)` | Manual S3 download with cleanup flag |
| `run_tool(tool, args, timeout)` | Execute whitelisted CLI tools |
| `upload_asset(path, namespace_id, internal_id, resource_id)` | Upload processed files back to S3 |

***

## Security Rules

Custom extractors are scanned before deployment. Code violating these rules is rejected.

* **Allowed:** `numpy`, `pandas`, `torch`, `transformers`, `sentence_transformers`, `onnxruntime`, `PIL`, `cv2`, `requests`, `httpx`, `os` (safe functions only), `json`, `re`, `pydantic`, `logging`, `getattr`, `hasattr`
* **Blocked:** `subprocess`, `os.system`, `os.popen`, `os.exec*`, `eval`, `exec`, `ctypes`, `socket`, `multiprocessing`, `open`, `setattr`, `delattr`

> [!NOTE]
>
> `import os` is allowed — only dangerous functions are blocked. Library-internal file I/O (`torch.load`, `transformers.from_pretrained`, `pd.read_csv`) is fine since the scanner only inspects your extractor's source code.

***

## Local Development

Validate and test your extractor locally with the CLI before uploading. **No API key needed** — these run fully offline and work on any plan.

```bash theme={null}
# Validate manifest and run the security scanner
python plugins.py lint path/to/my_extractor

# Run the pipeline through the Ray Data test harness (real map_batches + Arrow)
python plugins.py test path/to/my_extractor
```

`lint` catches common mistakes before upload:

* Wrong feature key names (`name` instead of `feature_name`)
* Missing required fields
* Security scanner violations

`test` runs your processor through real Ray Data `map_batches` with Arrow serialization — the same path used in production. Sample rows are fed in the `data` column (matching production), and the harness validates that your output columns match the manifest features.

### Version Management

On a [dedicated deployment](#availability), the same CLI manages deployed versions with a git-like workflow:

| Command | Description |
| - | - |
| `pull` | Download the active version's source files to a local directory |
| `push` | Zip, upload, and confirm a new version (auto-bumps patch version if omitted) |
| `log` | Show version history with deploy timestamps and commit messages |
| `status` | Show active version, extractor ID, and deployment status |
| `rollback` | Restore a previous version as active |
| `diff` | Compare source files between two versions |

The CLI reads `MIXPEEK_API_KEY`, `MIXPEEK_NAMESPACE`, and `MIXPEEK_API_URL` from the environment (or `--api-key` / `--namespace`).

### Archive Limits

| Limit | Value |
| - | - |
| Upload size | 500 MB |
| Max files | 1,000 |

Don't bundle model weights — download from HuggingFace Hub at init time, or use the [Model Registry](/docs/processing/model-registry) for custom weights.

***

## End-to-End: Extractor → Collection → Retriever

This walkthrough connects all the pieces. Set `MIXPEEK_API_KEY`, `MIXPEEK_NAMESPACE`, and `MIXPEEK_API_URL` first — the same variables the `plugins.py` CLI reads.

> [!NOTE]
>
> Steps 1 and 5 (the extractor upload/deploy) require a [dedicated deployment](#availability). On the shared API, use a built-in extractor (e.g. `text_extractor`) for the `feature_extractor` in step 3 and skip steps 1 and 5.

### 1. Deploy the Extractor (dedicated infra)

The simplest path is the CLI — `python plugins.py push` then `deploy`. The equivalent raw HTTP (custom extractors are addressed as `/plugins` on this surface; see the [Custom Extractor API](/docs/processing/custom-extractor-api) reference):

```bash theme={null}
zip -r my_extractor.zip my_extractor/

# Presigned upload → confirm → deploy
UPLOAD=$(curl -s -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/plugins/uploads" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "Content-Type: application/json" \
  -d '{"name":"my_extractor","version":"1.0.0","file_size_bytes":5000}')
UPLOAD_ID=$(echo "$UPLOAD" | jq -r '.upload_id')
PRESIGNED_URL=$(echo "$UPLOAD" | jq -r '.presigned_url')

curl -s -X PUT "$PRESIGNED_URL" -H "Content-Type: application/zip" --data-binary @my_extractor.zip

curl -s -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/plugins/uploads/$UPLOAD_ID/confirm" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "Content-Type: application/json" -d '{}'

curl -s -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/plugins/my_extractor_1_0_0/deploy" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

### 2. Create a Bucket and Upload Data

> [!NOTE]
>
> Buckets, collections, retrievers, and batches are **top-level** resources addressed by the **`X-Namespace` header** — not nested under `/namespaces/{ns}/`. (Only `extractors` and `models` are path-scoped.)

```bash theme={null}
BUCKET=$(curl -s -X POST "https://api.mixpeek.com/v1/buckets" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"bucket_name":"sec-filings","bucket_schema":{"properties":{"filing_text":{"type":"text"},"ticker":{"type":"text"},"form":{"type":"text"}}}}')
BUCKET_ID=$(echo "$BUCKET" | jq -r '.bucket_id')

curl -s -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/objects" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"blobs":[{"property":"filing_text","type":"text","data":"Revenue increased 22% year-over-year..."}],"ticker":"AAPL","form":"10-K"}'
```

### 3. Create a Collection with the Extractor

Every object uploaded to the bucket flows through this extractor's batch processor.

```bash theme={null}
COLLECTION=$(curl -s -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d "{
    \"collection_name\":\"sec-filing-chunks\",
    \"source\":{\"type\":\"bucket\",\"bucket_ids\":[\"$BUCKET_ID\"]},
    \"feature_extractor\":{\"feature_extractor_name\":\"my_extractor\",\"version\":\"1.0.0\",\"input_mappings\":{\"text\":\"filing_text\"}}
  }")
COLLECTION_ID=$(echo "$COLLECTION" | jq -r '.collection_id')
```

### 4. Process the Data (two-step batch)

A batch is **created** (with the objects + target collections) and then **submitted**:

```bash theme={null}
# List object IDs, then create the batch
OBJIDS=$(curl -s "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/objects" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  | jq -c '[.results[].object_id]')

BATCH=$(curl -s -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/batches" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d "{\"batch_name\":\"run-1\",\"object_ids\":$OBJIDS,\"collection_ids\":[\"$COLLECTION_ID\"]}")
BATCH_ID=$(echo "$BATCH" | jq -r '.batch_id')

# collection_ids are fixed at batch creation (above) — submit takes no scope
curl -s -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/batches/$BATCH_ID/submit" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d "{}"

# Poll GET /v1/buckets/$BUCKET_ID/batches/$BATCH_ID until status is COMPLETED
```

### 5. Build a Retriever and Search

At query time the retriever calls your extractor's `realtime.py` (dedicated infra) to embed the query, then searches the vectors your batch processor produced.

```bash theme={null}
RETRIEVER=$(curl -s -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d "{
    \"retriever_name\":\"filing-search\",
    \"collection_identifiers\":[\"$COLLECTION_ID\"],
    \"input_schema\":{\"query\":{\"type\":\"text\",\"required\":true}},
    \"stages\":[{\"stage_name\":\"semantic\",\"stage_type\":\"filter\",\"config\":{\"stage_id\":\"feature_search\",\"parameters\":{
      \"searches\":[{\"feature_uri\":\"mixpeek://my_extractor@1.0.0/my_embedding\",\"query\":{\"input_mode\":\"text\",\"value\":\"{{INPUT.query}}\"},\"top_k\":20}],
      \"final_top_k\":20,\"fusion\":\"rrf\"}}}]
  }")
RETRIEVER_ID=$(echo "$RETRIEVER" | jq -r '.retriever_id')

curl -s -X POST "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"inputs":{"query":"revenue growth year over year"}}'
```

***

## Troubleshooting

> **Upload/deploy endpoints return 404 or 405**
>
> The upload/deploy/realtime lifecycle is only available on a [dedicated deployment](#availability) — the shared `api.mixpeek.com` exposes only `GET` list/details for extractors. Develop + `lint` + `test` locally, then either provision a dedicated deployment or ship via [Submissions](/docs/processing/extractor-marketplace).

> **Task COMPLETED but 0 documents**
>
> Usually wrong `features` key names in `manifest.py`. Run `python plugins.py lint path/to/my_extractor` to validate. Check that your collection has non-empty `vector_indexes` via `GET /v1/collections/{id}`. See [Vector Index Keys](#vector-index-keys).

> **Batch produces 0 documents**
>
> Most common cause: reading from the wrong column. Always use `batch["data"]`, not `batch["text"]` or other property names. Check Ray logs for `[FailureAggregator]` entries.

> **Extractor validation failed**
>
> Check `validation_errors`. Common issues: using `subprocess` (use `run_tool`), using `open()` directly (use library I/O), using `eval`/`exec` (use `json.loads`).

> **Model loading is slow**
>
> Use `prefetch_hf_model()` in your `setup()` method to pre-download models to the HF cache. On GKE, `HF_HOME` points to a shared PVC so models persist across pod restarts. First cold start downloads (\~1-2 min); subsequent starts use cache.

> **First run is slow / first search returns 0 (cold start)**
>
> On a cold engine, the embedding model loads on demand — the first batch can take **several minutes** to leave `PROCESSING`, and the **first retriever `execute` afterward may return 0 results** (`status` `completed` or `degraded`) while the **query-side** model warms. This is a cold-start artifact, not a real no-match: **retry after a few seconds** and results appear. A warm namespace responds immediately. Keep a namespace warm by issuing a periodic lightweight query, or contact your account team about a warm replica floor for latency-sensitive workloads.

> **feature\_uri without realtime.py**
>
> If your manifest includes a `feature_uri`, the system expects a corresponding `realtime.py`. Without it, `feature_search` queries against that URI will fail. Omit both if you only need batch processing.

## Next Steps

> **[Quickstart](/docs/tutorials/custom-extractor-quickstart)**
>
> Build, test, and query a minimal text embedding extractor end-to-end.

> **[Model Registry](/docs/processing/model-registry)**
>
> Load HuggingFace models or your own fine-tuned weights inside an extractor.

> **[Extractor Submissions](/docs/processing/extractor-marketplace)**
>
> Submit your extractor for review to be merged into the built-in catalog.

> **[Multi-Tier Extraction](/docs/processing/multi-tier-extractors)**
>
> Chain collections into a DAG — transcribe, then embed, then classify.

> **[Reprocess Existing Content](/docs/tutorials/reprocess-existing-content)**
>
> Run a new extractor over an already-ingested corpus — scoped, cost-safe, priced before you run.


# Decomposition
Source: https://docs.mixpeek.com/docs/processing/decomposition

How Mixpeek turns raw objects into searchable documents and queryable features

![Decomposition: raw objects are split into documents, each with extracted features like embeddings, transcripts, and metadata](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-decomposition.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=4bfa40801fe7d78a68db0bd3f48b49c4)

Decomposition is the core transformation in Mixpeek. A raw file (video, PDF, image, audio) goes in as one **object**. It comes out as many **documents**, each with its own **features**. This is what makes sub-file search possible — you search *within* a video at the segment level, not *for* the video as a whole.

## Three Primitives

| Primitive | What It Is | Role |
| - | - | - |
| **Object** | Raw file or record in a bucket (video, PDF, JSON row, image). | The input boundary. You upload objects. |
| **Document** | One row of output in a collection, produced by decomposition. | The query boundary. You search documents. |
| **Feature** | A named output attached to a document (embedding, transcript, OCR text, label, score). | The composition boundary. Retrievers reference features by URI. |

The pipeline is always:

```
Object (bucket) → Decomposition → Document (collection) → Features (MVS + MongoDB)
```

## What Decomposition Decides

The feature extractor controls *how* an object is decomposed into documents. The strategy depends on the content type:

| Content Type | Decomposition Strategy | Result |
| - | - | - |
| **Video** | Time intervals, scene boundaries, or silence gaps | Each segment = 1 document with visual embedding + transcript + scene description |
| **Audio** | Silence boundaries or fixed intervals | Each segment = 1 document with transcript + transcript embedding |
| **PDF / Document** | Page, paragraph, or sentence boundaries | Each chunk = 1 document with text content + text embedding |
| **Image** | No split (1:1) | 1 image = 1 document with visual embedding + OCR + description |
| **Structured data** | Row-level (1:1) | 1 row = 1 document with field-level features |

## Why It Matters

**Without decomposition**, a 30-minute video is one record. Searching for "the moment the CEO mentions revenue" means scanning the entire video. There's no way to return a specific timestamp.

**With decomposition**, that video becomes \~180 ten-second segments, each with its own transcript embedding, visual embedding, and scene description. A search returns the exact segment at 14:30 where the CEO says "revenue grew 22%."

The same applies to documents: a 200-page PDF becomes 200 searchable chunks instead of one monolithic record.

## Feature URIs

Every feature produced by decomposition gets a URI that uniquely identifies it:

```
mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding
mixpeek://face_identity_extractor@v1/insightface__arcface
mixpeek://my_custom_plugin@1.0.0/domain_embedding
```

Retrievers, taxonomies, and clusters reference features by URI. This is the composition boundary — you can build a retriever that searches `multimodal_embedding` in one stage and `face_embedding` in another, even though they were produced by different extractors.

## Configuring Decomposition

Decomposition is configured via the `feature_extractor` field on a collection:

```json theme={null}
{
  "collection_name": "video-library",
  "source": { "type": "bucket", "bucket_ids": ["bkt_videos"] },
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "settings": {
      "video_segmentation": {
        "type": "time",
        "interval_sec": 10
      },
      "run_transcription": true,
      "run_scene_description": true
    }
  }
}
```

The `settings` object controls the decomposition strategy. Each extractor has its own settings — see the extractor-specific pages for details:

> **[From Video](/docs/processing/extractors/multimodal)**
>
> Time, scene, and silence segmentation strategies

> **[From Images](/docs/processing/extractors/image)**
>
> Visual embeddings, OCR, and structured extraction

> **[From Audio](/docs/processing/extractors/multimodal)**
>
> Silence-boundary segmentation and transcription

> **[From Documents](/docs/processing/extractors/document)**
>
> Page, paragraph, and sentence chunking

## Multi-Tier Decomposition

When a single extraction pass isn't enough — e.g., you need to transcribe audio *then* embed the transcription *then* classify each chunk — you chain collections. Each tier reads the output of the previous one, forming a DAG:

```
Tier 1: raw video → segments with transcripts
Tier 2: tier-1 docs → text chunks with embeddings
Tier 3: tier-2 docs → classifications per chunk
```

The engine resolves tiers automatically and executes them in dependency order. See [Multi-Tier Feature Extraction](/docs/processing/multi-tier-extractors) for the full guide.

## Lineage

Every document tracks its lineage back to the original object:

```json theme={null}
{
  "root_object_id": "obj_video_123",
  "root_bucket_id": "bkt_marketing",
  "source_collection_id": "col_segments",
  "lineage_path": "bkt_marketing/col_segments/col_chunks"
}
```

This lets you trace any search result back through tiers to the original file. Use the [Lineage API](/docs/api-reference/document-lineage/get-document-lineage) to visualize the decomposition tree.


# Deduplication & Re-processing
Source: https://docs.mixpeek.com/docs/processing/deduplication

Content-hash dedup decides what work is skipped, replaced, or forced when objects are processed into collections — and lineage decides what re-runs when a source changes

Every object ingested into Mixpeek is fingerprinted with a **SHA-256 content hash**, and every document produced from it carries that hash in its system envelope alongside a full [lineage record](/docs/retrieval/lineage-traversal). Together they answer the two questions every pipeline eventually asks:

1. **Have I already processed this?** — the content hash.
2. **What downstream work is stale now that this changed?** — the lineage chain.

## Content hashing

The hash is computed from the source content at ingestion and stored:

* on the **object** (in the bucket), and
* on every **document** derived from it (`content_hash` in the document's system envelope).

Two objects with identical bytes produce the same hash — regardless of filename, upload time, or path. A changed file produces a new hash, which is what marks its derived documents as stale.

"Latest version" of an asset is therefore a content-hash lookup, not a version counter you maintain yourself.

## Dedup strategy

When you submit a batch, `dedup_strategy` controls how objects that were **already processed in a prior batch** are handled. Dedup is scoped to the *(bucket, collection)* pair — an object is a duplicate if the target collection already has documents produced from that same source object.

| Strategy | Behavior | Cost profile |
| - | - | - |
| `skip` *(default)* | An object is reprocessed only if its content or the collection's extractor configuration changed since it was last extracted; everything else is skipped | Free for unchanged content — no extraction, no inference |
| `replace` | Every object is reprocessed. Once its new documents are written, its earlier documents in the collection are deleted. If the batch fails, or fails on that object, its earlier documents are kept | Full extraction cost; use for a clean rebuild after the source content or the extractor configuration changed |
| `force` | Every object is reprocessed and its documents are overwritten in place. Documents a re-run no longer produces are not removed | Full extraction cost; prefer `replace`, which also removes those |

```json theme={null}
{
  "bucket_id": "bkt_abc123",
  "collection_id": "col_xyz789",
  "dedup_strategy": "skip"
}
```

> [!WARNING]
>
> `skip` reprocesses an object when its content or the extractor configuration changed since it was last extracted, and skips everything else. It does not delete documents a re-run no longer produces, for example the extra chunks left behind when a new chunking setting yields fewer of them. For a clean rebuild that removes those, use `replace`.

## The re-processing cascade

Derived assets stay in sync with their source through the combination of both mechanisms:

1. A source object's content changes → its hash changes.
2. Re-submitting with `replace` (or re-syncing the source) reprocesses it — and every derived document downstream in its [lineage](/docs/retrieval/lineage-traversal) (transcode → embed → tag) is rebuilt from the new source.
3. Sources whose hash **didn't** change are skipped — unchanged content is never re-paid for.

This is what keeps N derived artifacts consistent with one upstream asset without a separate file system or job-tracking database: the document *is* the record, the hash *is* the version, and lineage *is* the dependency graph.

## Related

* [Lineage traversal](/docs/retrieval/lineage-traversal) — querying and walking derivation chains
* [Processing pipeline](/docs/platform/processing) — how objects become documents
* [Batch ingestion at scale](/docs/operations/batch-ingestion-at-scale) — batch mechanics, tiers, and monitoring


# Extractor Submissions
Source: https://docs.mixpeek.com/docs/processing/extractor-marketplace

Submit custom extractors for review and inclusion in the Mixpeek extractor catalog

> **[Browse the extractor catalog on GitHub](https://github.com/mixpeek/mixpeek-extractors)**
>
> Runnable reference for every built-in Mixpeek extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry, so it always matches production.

> [!WARNING]
>
> **Marketplace Deprecated.** The self-service marketplace (publish/install/browse via `/v1/plugins/...` and `/v1/public/plugins/...`) has been removed. The SDK methods `client.plugins.publish()`, `client.plugins.marketplace.list()`, and `client.plugins.install()` no longer exist.
>
> Extractor sharing now uses the **submission workflow** described below.

## How It Works

Instead of a self-service marketplace, community extractors follow a submission and review process:

1. **Develop** — Build your extractor following the [Custom Extractors guide](/docs/processing/custom-extractors)
2. **Submit** — Upload your extractor archive via `POST /v1/extractors/submissions`
3. **Review** — The Mixpeek team reviews the submission for quality, security, and compatibility
4. **Merge** — Approved extractors join the built-in catalog

```mermaid theme={null}
graph LR
    A[Develop Extractor] --> B[Submit Archive]
    B --> C[Mixpeek Reviews]
    C --> D[Approved & Merged]
    D --> E[Available as built-in extractor]
```

## Submitting an Extractor

Package your extractor as a zip archive and submit it for review:

```bash theme={null}
# Package your extractor
zip -r my_text_extractor.zip my_text_extractor/

# Submit for review
curl -X POST "https://api.mixpeek.com/v1/extractors/submissions" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "archive=@my_text_extractor.zip" \
  -F "display_name=My Text Extractor" \
  -F "description=Advanced text extraction with custom NLP models" \
  -F "category=text-processing"
```

### Submission Requirements

* Extractor must pass validation and security scan (same rules as custom extractor uploads)
* Include a complete `manifest.py` with correct `features` definitions
* Include working `pipeline.py` and optionally `realtime.py`
* Archive must be under 500 MB

### One-file YAML extractors

An archive that holds a single YAML manifest configures a built-in extractor instead of shipping code. It names the base with `extends` and can point the vector index at another embedding model, including one your organization uploaded (`org:<model_id>`). Set `output.vector_index.dimensions` to state the vector length the model returns. Validation runs one sample inference and rejects the extractor, with the traceback, when the vectors have another length or when the uploaded model's recorded dimension differs. The index is always sized from the model.

```yaml theme={null}
name: brand_text
version: v1
extends: text_extractor/v1
output:
  vector_index:
    model: org:mdl_7f3a9c
    dimensions: 384
```

## Review Process

After submission:

1. The archive is validated and scanned automatically
2. The Mixpeek team reviews the extractor code for quality and security
3. If approved, the extractor joins the built-in catalog
4. The submitting organization is notified and the extractor becomes available to all users

## Trust Tiers

| Tier | Description |
| - | - |
| `community` | Extractors submitted by community members and approved by Mixpeek |
| `verified` | Extractors with additional performance and reliability validation |
| `official` | Extractors developed and maintained by Mixpeek |

## Using Approved Extractors

Once an extractor is approved and merged, it works like any built-in extractor. Reference it by name and version in your collection configuration:

```python theme={null}
# Create a collection using the approved extractor
client.collections.create(
    namespace_id="ns_abc123",
    collection_name="my-collection",
    source={"type": "bucket", "bucket_ids": ["bkt_xyz"]},
    feature_extractor={
        "feature_extractor_name": "text_extractor",
        "version": "v1",
        "input_mappings": {"text": "description"},
    }
)
```


# Migrating from Extractor Names
Source: https://docs.mixpeek.com/docs/processing/extractor-migration

Built-in extractor names are deprecated aliases — map your existing feature_extractor configs to features keys

Mixpeek's public vocabulary is **modality + features**: you configure collections by picking [features](/docs/processing/features) (`image_search`, `faces`, `onscreen_text`, ...), and the platform resolves the implementation internally. Built-in extractor names — `image_extractor`, `universal_extractor`, `face_identity_extractor`, and friends — are **internal implementation details** and no longer appear in new API payloads, errors, or docs.

**Nothing breaks.** Existing configs and SDK clients that pass `feature_extractor: {...}` keep working: known built-in extractor names are accepted as deprecated aliases and resolved to their feature through the same catalog that powers the features path. Responses carry a deprecation notice pointing at `features: [...]`.

> [!NOTE]
>
> This page is the one place old names are documented, for migration. Custom extractors you publish yourself are **not** deprecated — your plugin is your vocabulary, selected as [`custom:<plugin_name>`](/docs/processing/features#custom-features-bring-your-own).

## Alias map

| Deprecated extractor name | Use instead | Modality |
| - | - | - |
| `image_extractor` | `features: ["image_search"]` | image |
| `image_extractor` (PDF input) | `features: ["document_search"]` | document |
| `universal_extractor` | `features: ["video_search"]` | video |
| `text_extractor` | `features: ["text_search"]` | text |
| `audio_fingerprint_extractor` | `features: ["audio_search"]` (audio) or `features: ["audio_fingerprint"]` (in-video adder) | audio / video |
| `face_identity_extractor` | `features: ["faces"]` | image, video, document |
| `scrolling_text_extractor` | `features: ["onscreen_text"]` | video |
| `document_graph_extractor` | `features: ["document_layout"]` | document |
| `web_scraper` | `features: ["web_crawl"]` | web |
| `multimodal_extractor` | `features: ["multimodal_understanding"]` | any |
| `gemini_multifile_extractor` | `features: ["multimodal_understanding"]` | any |
| `passthrough_extractor` | `features: ["storage_only"]` | any |

The live menu — including keys, display names, and rates — is always `GET /v1/collections/features` ([discovery](/docs/processing/features#discover-features)).

## Before / after

**Before** (still works, deprecated):

```json theme={null}
{
  "collection_name": "press-footage",
  "source": { "type": "bucket", "bucket_ids": ["bkt_123"] },
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": { "video": "video_url" }
  }
}
```

**After**:

```json theme={null}
{
  "collection_name": "press-footage",
  "source": { "type": "bucket", "bucket_ids": ["bkt_123"] },
  "features": ["faces"]
}
```

The features path also defaults `input_mappings` for the standard `uploads` bucket properties (`image`, `video`, `audio`, `pdf`, `content`, `url`) — see [default input wiring](/docs/processing/features#default-input-wiring). If you need explicit input mappings, field passthrough, or extractor parameters, the `feature_extractor` config object remains the [advanced path](/docs/processing/feature-extractors).

## What you don't need to migrate

* **Existing collections** — they keep running unchanged. Aliases have no sunset date.
* **Feature URIs** — query-side references like `mixpeek://text_extractor@v1/...` in existing retrievers remain valid contracts; they pin the pipeline version that produced your vectors.
* **Custom extractors** — your own plugins keep their explicit names.

## Why this changed

Pricing and configuration now share one vocabulary: features applied to modality units (images, video minutes, pages, tokens). When Mixpeek swaps or upgrades the models behind a feature, nothing about your config or your rates moves. See [Billing & Pricing](/docs/platform/billing) for how features are priced.


# Audio Fingerprint Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/audio-fingerprint

Audio fingerprinting with CLAP — 512-d embeddings from audio files or video audio tracks for sound-mark matching and audio similarity

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["audio_search"]` for audio files, or `features: ["audio_fingerprint"]` as an in-video add-on. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[Fingerprint your own audio](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_audio_fingerprint\&utm_campaign=managed_onramp)**
>
> Create a managed namespace, index your audio and video files, then match them by acoustic fingerprint.

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/audio_fingerprint_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

The audio fingerprint extractor produces **512-dimensional CLAP embeddings** (Contrastive Language-Audio Pretraining, `laion/clap-htsat-tiny`) from audio files or the audio track of a video. It segments audio into overlapping windows, embeds each segment, and L2-normalizes the vectors for cosine similarity. Use it for sound-mark matching, audio similarity, and retrieving audio by acoustic fingerprint.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/audio\_fingerprint\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/audio_fingerprint_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Resolve input** — apply `input_mappings` to get the audio or video URL.
2. **Audio extraction** — if the source is video, extract the audio track.
3. **Resample** — resample audio to `sample_rate` (48000 Hz CLAP default).
4. **Segment** — split into `segment_duration_sec` windows hopping by `segment_hop_sec` (default 50% overlap); truncate beyond `max_audio_length_sec`.
5. **CLAP embedding** — embed each segment to a 512-d vector.
6. **Normalize** (if `normalize_embeddings`) — L2-normalize to unit vectors.
7. **Output** — one document per segment with timing metadata.

## When to Use

| Use Case | Description |
| - | - |
| **Sound-mark matching** | Detect a known jingle, sound logo, or audio cue across a corpus |
| **Audio similarity** | Find acoustically similar clips (music, ambience, effects) |
| **Ad/asset detection** | Match the audio fingerprint of an ad or asset within longer media |
| **Video audio search** | Search the audio track of video assets without separate extraction |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Speech-to-text / transcription | A transcription extractor (e.g. Whisper-based) |
| Text semantic search over spoken words | Transcribe, then `text_extractor` |
| Whole-file multimodal embedding | `universal_extractor` / `multimodal_extractor` |
| Music metadata/tagging only | A classification taxonomy over fingerprints |

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `audio` | string | one of | URL or path to an audio file. Populated from `input_mappings`. |
| `video` | string | one of | URL or path to a video file; the audio track is extracted. |

```json theme={null}
{
  "audio": "s3://my-bucket/spots/jingle.wav"
}
```

Supported input types: **AUDIO, VIDEO** (max 1 each per object).

## Output Schema

One document per audio segment:

| Field | Type | Description |
| - | - | - |
| `audio_fingerprint_extractor_v1_embedding` | float\[512] | CLAP embedding (L2-normalized when enabled) |
| `segment_index` | integer | Segment index (0-based) |
| `start_time_sec` / `end_time_sec` | float | Segment start/end time in seconds |
| `duration_sec` | float | Duration of this segment (seconds) |
| `total_duration_sec` | float \| null | Source audio duration |
| `sample_rate` | integer | Sample rate used for processing |
| `audio_source_type` | string | Source type: `audio` or `video` |
| `embedding_model` | string | Embedding model used (default `laion/clap-htsat-tiny`) |
| `processing_time_ms` | float | Per-segment processing time |

```json theme={null}
{
  "audio_fingerprint_extractor_v1_embedding": [0.041, -0.018, 0.092, ...],
  "segment_index": 0,
  "start_time_sec": 0.0,
  "end_time_sec": 5.0,
  "duration_sec": 5.0,
  "sample_rate": 48000,
  "audio_source_type": "audio",
  "embedding_model": "laion/clap-htsat-tiny"
}
```

## Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `segment_duration_sec` | float | `5.0` | 1.0–30.0 | Duration of each audio segment (seconds). 5.0 recommended for sound-mark matching |
| `segment_hop_sec` | float | `2.5` | 0.5–15.0 | Hop between segments (seconds). 2.5 = 50% overlap. Set equal to `segment_duration_sec` for no overlap |
| `sample_rate` | integer | `48000` | — | Target sample rate (Hz). 48000 is the CLAP default; audio is resampled before embedding |
| `normalize_embeddings` | boolean | `true` | — | L2-normalize embeddings to unit vectors (recommended for cosine similarity) |
| `max_audio_length_sec` | float | `120.0` | 1.0–600.0 | Maximum audio length to process (seconds). Audio beyond this is truncated |

## Configuration Examples

```json Default (5s windows, 50% overlap) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "audio_fingerprint_extractor",
    "version": "v1",
    "input_mappings": {
      "audio": "audio_url"
    },
    "parameters": {}
  }
}
```

```json Video Audio Track theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "audio_fingerprint_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {}
  }
}
```

```json Non-Overlapping Segments, Longer Audio theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "audio_fingerprint_extractor",
    "version": "v1",
    "input_mappings": {
      "audio": "audio_url"
    },
    "parameters": {
      "segment_duration_sec": 10.0,
      "segment_hop_sec": 10.0,
      "max_audio_length_sec": 300.0
    }
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Cost** | See [Billing & Pricing](/docs/platform/billing) — rates come from `GET /v1/billing/pricing` |
| **Model** | `laion/clap-htsat-tiny` (CLAP) |
| **Default coverage** | First 120 s of audio (configurable to 600 s) |

## Vector Index

| Property | Value |
| - | - |
| **Index name** | `audio_fingerprint_extractor_v1_embedding` |
| **Dimensions** | 512 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `laion/clap-htsat-tiny` |
| **Normalization** | L2 normalized (when `normalize_embeddings`) |

## Limitations

* **Length cap**: Audio beyond `max_audio_length_sec` is truncated (default 120 s).
* **Not for transcription**: Produces acoustic fingerprints, not text — pair with a transcription extractor for spoken-word search.
* **Segment fan-out**: Overlapping windows multiply the document count per source; tune `segment_hop_sec` to control density.

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Universal Extractor](/docs/processing/extractors/universal)


# Document Graph Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/document

Extract spatial blocks from PDFs with layout classification, confidence scoring, optional VLM correction, and 1024-d E5 text embeddings

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["document_layout"]` (layout/structure extraction on top of the `document_search` base). Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/document_graph_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Document graph extractor pipeline showing PDF parsing, layout detection, and block extraction](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/document.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=0984be250546b00316f53538ab0a9d33)

The document graph extractor decomposes PDFs into **spatial blocks** — paragraphs, tables, forms, lists, headers, footers, figures, and handwritten content — each with a bounding box, a layout class, and a confidence score. Low-confidence blocks can be corrected by a vision language model (Gemini, GPT-4V, or Claude). Block text is optionally embedded with E5-Large (1024-d) for semantic search. It is the right tool for archival documents, scanned files, and anything that needs spatial understanding rather than a flat text dump.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/document\_graph\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/document_graph_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Layout detection** (if `use_layout_detection`, the default) — find all document elements with `layout_detector` (`pymupdf` fast rule-based, or `docling` SOTA ML/DiT). Detects text regions **and** non-text elements (scanned images, figures, charts) as separate blocks.
2. **Block grouping** — when layout detection is off, group text spans into blocks using `vertical_threshold` / `horizontal_threshold`; drop blocks shorter than `min_text_length`.
3. **Confidence scoring** — assign `base_confidence` to native text (penalties for OCR artifacts / encoding issues), then tag each block A/B/C/D.
4. **VLM correction** (if `use_vlm_correction` and confidence \< `min_confidence_for_vlm`) — re-read low-confidence blocks with `vlm_provider`/`vlm_model`. Skipped entirely in `fast_mode`.
5. **Text embedding** (if `run_text_embedding`) — embed block text with E5-Large (1024-d).
6. **Thumbnails** (if `generate_thumbnails`) — render full-page and/or per-block thumbnails per `thumbnail_mode`.
7. **Output** — one document per block with layout class, bbox, text, confidence, and optional embedding/thumbnails.

## When to Use

| Use Case | Description |
| - | - |
| **Archival / scanned PDFs** | OCR + layout recovery with VLM correction for noisy scans and historical records |
| **Forms & tables** | Classify and isolate form fields, tables, and structured regions |
| **Spatial search** | Retrieve blocks by location and type, not just text |
| **Confidence-gated pipelines** | Route low-confidence blocks (tags C/D) to review |
| **Multi-layout documents** | Reports, contracts, and multi-column layouts that need block-level granularity |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Plain text you already have, or born-digital PDFs with perfect text | [`text_extractor`](/docs/processing/extractors/text) (faster, simpler) |
| Whole-document multimodal embedding | [`multimodal_extractor`](/docs/processing/extractors/multimodal) / [`universal_extractor`](/docs/processing/extractors/universal) |
| Maximum throughput, no spatial detail | `text_extractor`, or this extractor with `fast_mode: true` |
| Images only | [`image_extractor`](/docs/processing/extractors/image) |
| Non-PDF inputs | This extractor is PDF-only |

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `pdf` | string | **Yes** | URL or path to the PDF file. Supports multi-page PDFs. Populated from `input_mappings`. |

```json theme={null}
{
  "pdf": "s3://my-bucket/contracts/lease.pdf"
}
```

**Input Examples:**

| Type | Example |
| - | - |
| Invoice | `s3://documents/invoices/inv-001.pdf` |
| Contract | `https://cdn.example.com/contracts/lease.pdf` |
| Scanned document | `s3://archive/scanned/1985-report.pdf` |
| Form | `s3://forms/application-form.pdf` |

Supported input types: **PDF only** (max 1 PDF per object). Max file size 100MB. For scanned documents, 150–300 DPI originals give the best OCR.

## Output Schema

One document per detected block:

| Field | Type | Description |
| - | - | - |
| `page_number` | integer | Page number in the original PDF (1-indexed) |
| `object_type` | enum | `paragraph`, `table`, `form`, `list`, `header`, `footer`, `figure`, or `handwritten` |
| `block_index` | integer | Block index within the page (0-indexed) |
| `bbox` | object | Bounding box: `{ x0, y0, x1, y1 }` |
| `text_raw` | string | Original extracted text |
| `text_corrected` | string \| null | Cleaned / VLM-corrected text |
| `overall_confidence` | float | Confidence score 0.0–1.0 |
| `confidence_tag` | enum | `A` (`≥0.85`), `B` (`≥0.70`), `C` (`≥0.50`), `D` (`<0.50`) |
| `document_graph_extractor_v1_text_embedding` | float\[1024] \| null | E5-Large block embedding (when `run_text_embedding`) |
| `thumbnail_url` / `segment_thumbnail_url` | string \| null | Full-page / per-block thumbnail URLs |
| `total_pages` | integer \| null | Total pages in the source PDF |
| `source_file` | string \| null | Original source file name |

```json theme={null}
{
  "page_number": 1,
  "object_type": "table",
  "block_index": 3,
  "bbox": { "x0": 72.0, "y0": 220.4, "x1": 540.0, "y1": 410.9 },
  "text_raw": "Item  Qty  Price\nWidget  10  $4.00",
  "text_corrected": "Item | Qty | Price\nWidget | 10 | $4.00",
  "overall_confidence": 0.91,
  "confidence_tag": "A"
}
```

## Parameters

### Layout Detection

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `use_layout_detection` | boolean | `true` | — | Enable ML-based layout detection to find all elements (text, images, tables, figures). When disabled, falls back to text-only extraction (faster, misses images). |
| `layout_detector` | string | `"pymupdf"` | `pymupdf`, `docling` | Engine used when `use_layout_detection=true`. `pymupdf`: fast rule-based (\~15 pages/sec). `docling`: SOTA ML/DiT — better semantic type detection and true table structure (\~3–8 sec/doc). |
| `render_dpi` | integer | `150` | 72–300 | DPI for page rendering (used for VLM correction). 72 fast/lower quality, 150 balanced, 300 high quality/slower. |

### Spatial Clustering (text-only fallback)

Only used when `use_layout_detection=false`.

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `vertical_threshold` | float | `15.0` | 1.0–100.0 | Max vertical gap (points) between lines grouped into the same block |
| `horizontal_threshold` | float | `50.0` | 1.0–200.0 | Max horizontal distance (points) for overlap/column detection |
| `min_text_length` | integer | `20` | 1–500 | Minimum block text length (chars); filters noise/fragments |

### Confidence & VLM Correction

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `base_confidence` | float | `0.85` | 0.0–1.0 | Base confidence score for embedded (native) text |
| `min_confidence_for_vlm` | float | `0.6` | 0.0–1.0 | Confidence threshold below which VLM correction is triggered (only when `use_vlm_correction=true`) |
| `use_vlm_correction` | boolean | `true` | — | Enable VLM correction for low-confidence blocks |
| `fast_mode` | boolean | `false` | — | Skip VLM correction entirely for max throughput (\~15 pages/sec). Overrides `use_vlm_correction` |
| `vlm_provider` | string | `"google"` | `google`, `openai`, `anthropic` | LLM provider for VLM correction |
| `vlm_model` | string | `"gemini-2.5-flash"` | — | Correction model, e.g. `gemini-2.5-flash`, `gpt-4o`, `claude-3-5-sonnet` |
| `llm_api_key` | string \| null | `null` | — | BYOK key for VLM correction. Supports secret references, e.g. `{{SECRET.openai_api_key}}`. Falls back to Mixpeek's default keys if unset. |

### Embedding & Thumbnails

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `run_text_embedding` | boolean | `true` | — | Generate E5-Large (1024-d) text embeddings for block content |
| `generate_thumbnails` | boolean | `true` | — | Generate thumbnail images for blocks |
| `thumbnail_mode` | string | `"both"` | `full_page`, `segment`, `both` | Which thumbnails to render (`segment` = cropped to the block's bbox) |
| `thumbnail_dpi` | integer | `72` | 36–150 | DPI for thumbnail generation. Lower DPI = smaller files. |

## Configuration Examples

```json Default (layout + VLM + embeddings) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "document_graph_extractor",
    "version": "v1",
    "input_mappings": {
      "pdf": "file_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.invoice_id" },
      { "source_path": "metadata.vendor" }
    ],
    "parameters": {}
  }
}
```

```json Fast Mode (max throughput, no VLM) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "document_graph_extractor",
    "version": "v1",
    "input_mappings": {
      "pdf": "file_url"
    },
    "parameters": {
      "fast_mode": true,
      "generate_thumbnails": false
    }
  }
}
```

```json High-Accuracy Scanned Docs (docling + VLM) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "document_graph_extractor",
    "version": "v1",
    "input_mappings": {
      "pdf": "scanned_doc"
    },
    "parameters": {
      "layout_detector": "docling",
      "min_confidence_for_vlm": 0.75,
      "vlm_provider": "anthropic",
      "vlm_model": "claude-3-5-sonnet",
      "render_dpi": 200
    }
  }
}
```

```json Text-Only Fallback (no layout detection) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "document_graph_extractor",
    "version": "v1",
    "input_mappings": {
      "pdf": "pdf_url"
    },
    "parameters": {
      "use_layout_detection": false,
      "vertical_threshold": 15.0,
      "horizontal_threshold": 50.0,
      "run_text_embedding": true
    }
  }
}
```

## Layout Types

The extractor classifies blocks into these `object_type` values:

| Type | Description | Example |
| - | - | - |
| `paragraph` | Body text blocks | Article content, descriptions |
| `table` | Tabular data | Financial tables, data grids |
| `form` | Form fields and labels | Application forms, surveys |
| `list` | Bulleted or numbered lists | Requirements, instructions |
| `header` | Page headers | Document titles, section headers |
| `footer` | Page footers | Page numbers, disclaimers |
| `figure` | Images and captions | Charts, diagrams, photos |
| `handwritten` | Handwritten text | Signatures, annotations |

## Confidence Tags

Extraction quality is graded with confidence tags (thresholds on `overall_confidence`):

| Tag | Confidence | Description | Action |
| - | - | - | - |
| **A** | ≥ 0.85 | Excellent | Use directly |
| **B** | ≥ 0.70 | Good | Reliable, minor issues |
| **C** | ≥ 0.50 | Fair | Verify or trigger VLM correction |
| **D** | \< 0.50 | Poor | Needs VLM correction |

## Performance & Costs

| Metric | Value |
| - | - |
| **Cost** | Billed per page — see [Billing & Pricing](/docs/platform/billing); rates come from `GET /v1/billing/pricing` |
| **`pymupdf` throughput** | \~15 pages/sec (rule-based) |
| **`docling` throughput** | \~3–8 sec/doc (ML/DiT) |
| **Fast mode** | \~15 pages/sec (skips VLM correction) |

## Vector Index

| Property | Value |
| - | - |
| **Index name** | `document_graph_extractor_v1_text_embedding` |
| **Dimensions** | 1024 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `multilingual_e5_large_instruct_v1` (`intfloat/multilingual-e5-large-instruct`) |
| **Normalization** | L2 normalized |

> [!NOTE]
>
> The embedding is optional. Set `run_text_embedding: false` for a layout-only extraction with no vector index.

## Limitations

* **PDF only**: Accepts a single PDF per object; does not process Word docs, images, or other formats.
* **VLM cost**: Each correction adds cost (see [Billing & Pricing](/docs/platform/billing)); gate it with `min_confidence_for_vlm` or disable via `fast_mode`.
* **Layout-detector tradeoff**: `docling` is more accurate but markedly slower than `pymupdf`.
* **Memory**: Large PDFs (100+ pages) may require increased memory.
* **Language / handwriting**: OCR works best with Latin scripts; handwriting detection is experimental and less reliable.
* **External dependency**: VLM correction depends on the selected provider's availability.

## Search the Extracted Text

Extracted block text (native or OCR/VLM-corrected) is embedded into the `document_graph_extractor_v1_text_embedding` index, so you search it with a [`feature_search`](/docs/retrieval/stages/feature-search) stage against `mixpeek://document_graph_extractor@v1/intfloat__multilingual_e5_large_instruct` (`input_mode: "text"`). For a ready-to-copy retriever — including confidence and layout-type filtering — see [Cookbook → Search OCR Text from Scanned PDFs](/docs/retrieval/cookbook#search-ocr-text-from-scanned-pdfs).

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Text Extractor](/docs/processing/extractors/text)
* [Image Extractor](/docs/processing/extractors/image)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)
* [Universal Extractor](/docs/processing/extractors/universal)
* [Retrieval Cookbook — OCR / scanned-document search](/docs/retrieval/cookbook#search-ocr-text-from-scanned-pdfs)


# Face Identity Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/face-identity

Production-grade face recognition (SCRFD + ArcFace, 99.8%+ accuracy) — track a specific person or person-of-interest (POI) across images and video

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["faces"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/face_identity_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Face identity extractor pipeline showing detection, alignment, and embedding generation](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/face.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=a9c4becfd879ba038997d057747d3f72)

The face identity extractor provides production-grade face recognition using state-of-the-art models (SCRFD for detection + ArcFace for embeddings). Detects faces, aligns to canonical template, and generates 512-dimensional embeddings with 99.8%+ verification accuracy (LFW benchmark).

> [!NOTE]
>
> **Tracking a specific person (person-of-interest / POI).** This is the extractor to reach for when the goal is "find every clip a **person / individual / face** appears in" or "trace a person-of-interest across a video library." Pass a reference face image as a `content` query against the face embedding to run 1:N identification — see the [Face Search recipe](/docs/retrieval/cookbook#face-search-1n-identification). (For *what a person said*, pair this with cross-lingual transcript search via the [text extractor](/docs/processing/extractors/text).)

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/face\_identity\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/face_identity_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Filter Dataset** (if collection\_id provided)
   * Filter to specified collection
2. **Content Type Routing**
   * **Images:** Direct to Step 3
   * **Videos:** Frame extraction (sampling at `video_sampling_fps`) → Step 3
   * **PDFs:** Page rendering → Step 3
   * **Mixed:** Branch by type, process separately, union results
3. **Face Detection** (SCRFD)
   * Detect all faces per image/frame/page
   * Extract 5-point facial landmarks (eyes, nose, mouth)
   * Filter by `min_face_size` and `detection_threshold`
4. **5-Point Affine Face Alignment**
   * Warp face to canonical 112×112 template
   * Ensures consistent embeddings
5. **ArcFace Embedding Generation**
   * arcface\_r100 model
   * 512D L2-normalized embeddings
   * Cosine similarity for matching
6. **Quality Scoring** (conditional: if `enable_quality_scoring=true`)
   * Assess blur, size, landmark confidence
   * Filter by `quality_threshold` if specified
7. **Video Deduplication** (conditional: if `video_deduplication=true` AND video content)
   * Remove duplicate faces across frames
   * Threshold-based similarity matching
   * Track face timelines in video
8. **Output Validation**
9. **Output**
   * Per-face documents with embeddings, bbox, landmarks, quality scores

## When to Use

| Use Case | Description |
| - | - |
| **Face verification** | 1:1 matching to verify identity |
| **Face identification** | 1:N search to identify a person in a database |
| **Face clustering** | Group photos by person automatically |
| **Employee verification** | Workplace identity systems |
| **Photo organization** | Organize photo libraries by people |
| **Surveillance** | Security and monitoring applications |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| General image search | `image_extractor` |
| Object/scene detection | `multimodal_extractor` |
| Video content analysis | `multimodal_extractor` |
| Non-face biometrics | Specialized extractors |

## Supported Input Types

| Input | Type | Description | Processing |
| - | - | - | - |
| `image` | string | URL or S3 path | Detect and embed all faces |
| `video` | string | URL or S3 path | Sample frames, detect faces, deduplicate |
| `video_frame` | string | URL or S3 path | Treated as image |

**Supported formats:**

* **Image**: JPEG, PNG, WebP, BMP
* **Video**: MP4, MOV, AVI, MKV, WebM

**Recommended resolution**: 640px+ for optimal face detection

## Input Schema

Provide **one** of the following inputs:

```json theme={null}
{
  "image": "s3://photos/john-doe-portrait.jpg"
}
```

```json theme={null}
{
  "video": "s3://segments/interview-clip.mp4"
}
```

| Field | Type | Description |
| - | - | - |
| `image` | string | Image URL or S3 path containing faces |
| `video` | string | Video URL or S3 path. Subject to `max_video_length` limit |
| `video_frame` | string | Single video frame URL or S3 path (treated as image) |

## Output Schema

Each detected face produces one document with the following fields:

| Field | Type | Description |
| - | - | - |
| `face_identity_extractor_v1_embedding` | float\[512] | ArcFace embedding, L2 normalized |
| `face_index` | integer | Index of this face in source image (0-based) |
| `bbox` | object | Bounding box `{x1, y1, x2, y2, width, height}` |
| `detection_score` | number | SCRFD detection confidence (0.0-1.0) |
| `landmarks` | object | 5 facial landmarks for alignment |
| `quality_score` | number | Face quality score (0.0-1.0) |
| `quality_components` | object | Quality component scores (blur, size, etc.) |
| `aligned_face_crop` | string | Base64 aligned 112x112 face crop (optional) |
| `frame_number` | integer | Frame number in source video |
| `timestamp` | number | Timestamp in source video (seconds) |
| `embedding_model` | string | Embedding model used |
| `detection_model` | string | Detection model used |
| `processing_time_ms` | number | Processing time (milliseconds) |

```json theme={null}
{
  "face_identity_extractor_v1_embedding": [0.023, -0.041, 0.018, ...],
  "face_index": 0,
  "bbox": {"x1": 120, "y1": 80, "x2": 280, "y2": 300, "width": 160, "height": 220},
  "detection_score": 0.98,
  "landmarks": {"left_eye": [150, 140], "right_eye": [230, 142], ...},
  "quality_score": 0.85,
  "embedding_model": "arcface_r100",
  "detection_model": "scrfd_2.5g",
  "processing_time_ms": 45.2
}
```

## Parameters

### Detection Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `detection_model` | string | `"scrfd_2.5g"` | SCRFD model variant |
| `min_face_size` | integer | `20` | Minimum face size in pixels to detect |
| `detection_threshold` | float | `0.5` | Confidence threshold (0.0-1.0) |
| `max_faces_per_image` | integer | `null` | Maximum faces to process per image |

#### Detection Models

| Model | Speed | Accuracy | Best For |
| - | - | - | - |
| `scrfd_500m` | 2-3ms | Good | Real-time applications |
| `scrfd_2.5g` | 5-7ms | Better | **Recommended** - balanced |
| `scrfd_10g` | 10-15ms | Best | Maximum accuracy |

### Embedding Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `normalize_embeddings` | boolean | `true` | L2-normalize to unit vectors |

The embedding model is fixed to `arcface_r100` for `v1`, so there is no parameter to choose another one. A different model would ship as a new extractor version.

### Quality Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `enable_quality_scoring` | boolean | `true` | Compute quality scores (adds \~5ms per face) |
| `quality_threshold` | float | `null` | Minimum quality to index (null = index all) |

**Quality threshold guide:**

* `null` - Index all detected faces
* `0.5` - Moderate filtering (removes low quality)
* `0.7` - High quality only

### Video Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `max_video_length` | integer | `60` | Maximum video length in seconds |
| `video_sampling_fps` | float | `1.0` | Frames per second to sample |
| `video_deduplication` | boolean | `true` | Remove duplicate faces across frames |
| `video_deduplication_threshold` | float | `0.8` | Cosine similarity for deduplication |

### Output Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `output_mode` | string | `"per_face"` | `per_face` or `per_image` |
| `include_face_crops` | boolean | `false` | Include aligned 112x112 face crops as base64 |
| `store_detection_metadata` | boolean | `true` | Store bbox, landmarks, detection scores |

## Configuration Examples

```json Employee Verification (High Quality) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "photo_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.employee_id" }
    ],
    "parameters": {
      "detection_model": "scrfd_2.5g",
      "detection_threshold": 0.7,
      "enable_quality_scoring": true,
      "quality_threshold": 0.5,
      "max_faces_per_image": 1,
      "min_face_size": 40
    }
  }
}
```

```json Group Photo Processing theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "image_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.photo_id" },
      { "source_path": "metadata.event_name" }
    ],
    "parameters": {
      "detection_model": "scrfd_10g",
      "detection_threshold": 0.5,
      "max_faces_per_image": null,
      "enable_quality_scoring": true
    }
  }
}
```

```json Surveillance Video theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.camera_id" },
      { "source_path": "metadata.location" }
    ],
    "parameters": {
      "detection_model": "scrfd_10g",
      "detection_threshold": 0.6,
      "max_video_length": 300,
      "video_sampling_fps": 1.0,
      "video_deduplication": true,
      "video_deduplication_threshold": 0.8,
      "min_face_size": 30,
      "quality_threshold": 0.4
    }
  }
}
```

```json Photo Library Organization theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "photo_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.album" },
      { "source_path": "metadata.date_taken" }
    ],
    "parameters": {
      "detection_model": "scrfd_2.5g",
      "enable_quality_scoring": true,
      "include_face_crops": true,
      "store_detection_metadata": true
    }
  }
}
```

```json Real-time Access Control theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "capture_url"
    },
    "parameters": {
      "detection_model": "scrfd_500m",
      "detection_threshold": 0.8,
      "max_faces_per_image": 1,
      "min_face_size": 60,
      "enable_quality_scoring": false
    }
  }
}
```

## Face Matching

Use cosine similarity to match faces:

| Similarity Score | Interpretation |
| - | - |
| > 0.30 | Very likely same person |
| 0.25 - 0.30 | Likely same person |
| 0.20 - 0.25 | Possibly same person |
| \< 0.20 | Different people |

**Recommended threshold**: 0.25-0.30 for same person verification

## Performance & Costs

| Metric | Value |
| - | - |
| **Detection accuracy** | 99%+ (WIDER FACE benchmark) |
| **Verification accuracy** | 99.8%+ (LFW benchmark) |
| **Processing speed** | Detection: 5-7ms, Embedding: 10-15ms per face |
| **Cost** | See [Billing & Pricing](/docs/platform/billing) — rates come from `GET /v1/billing/pricing` |

### Video Processing

* **Deduplication**: Reduces 90-95% redundancy in video
* **Sampling**: 1 FPS recommended for most use cases
* **Max length**: 300 seconds (extraction only)

## Vector Index

| Property | Value |
| - | - |
| **Index name** | `face_identity_extractor_v1_embedding` |
| **Dimensions** | 512 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Datatype** | float32 |
| **Inference model** | `face_identity_arcface_r100_v1` |

## Pipeline Overview

1. **SCRFD Detection** - Bounding boxes + 5 landmarks
2. **5-Point Affine Alignment** - 112x112 canonical face
3. **ArcFace Embedding** - 512-d L2-normalized vector
4. **Quality Scoring** (optional) - Filter low-quality faces

## Limitations

* **Face only**: Does not identify age, gender, or expressions
* **Pose sensitivity**: Extreme angles may reduce accuracy
* **Occlusion**: Masks, glasses, hair may affect detection
* **Resolution**: Minimum 20px face size, 40px+ recommended
* **Lighting**: Poor lighting reduces quality scores
* **Video length**: Maximum 300 seconds per video

## Search by face

Once faces are indexed, search them with a reference image (1:N identification) using a [`feature_search`](/docs/retrieval/stages/feature-search) stage — pass the reference face as a `content` query. The feature URI is `mixpeek://face_identity_extractor@v1/insightface__arcface` — the output name is the model (`insightface__arcface`), not the internal vector-index name (`face_identity_extractor_v1_embedding`). If unsure, `GET /v1/collections/features/extractors/face_identity_extractor_v1` returns the exact `feature_uri`:

```json theme={null}
{
  "stage_name": "face_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://face_identity_extractor@v1/insightface__arcface",
          "query": { "input_mode": "content", "value": "{{INPUT.reference_face_url}}" },
          "top_k": 50
        }
      ],
      "final_top_k": 20
    }
  }
}
```

Define `reference_face_url` in the retriever's `input_schema`, then execute with `{"inputs": {"reference_face_url": "https://.../person.jpg"}}` to find every document featuring that person.

## Build & maintain a named identity list

The 1:N search above matches **one** reference face at a time. To track a
**roster of known people** — a watchlist that puts a *name* on faces found in
your videos, that you add to and refine over time — build a **reference
collection** of labeled faces and let new footage auto-identify against it.

That whole lifecycle — enroll reference faces, label them with names,
auto-label people in incoming video, review the "unknown" faces, and promote a
newly-confirmed face back into the reference set so the list **self-improves** —
is the [**Bootstrap a Labeled Dataset**](/docs/tutorials/bootstrap-labeled-dataset)
tutorial (see its *People Identification* / *Face Recognition System* path).
Adding, correcting, or removing a person is just editing a document in the
reference collection; the next video ingest identifies against the updated list.

> [!NOTE]
>
> Rule of thumb: use **1:N search** (above) when you already have the one photo
> you're chasing; use the **reference-collection roster** when you're maintaining
> an ongoing list of named identities to match every new video against.

## Related

* [Feature Search stage](/docs/retrieval/stages/feature-search) — query face embeddings
* [Bootstrap a Labeled Dataset](/docs/tutorials/bootstrap-labeled-dataset) — build & maintain a named identity roster that auto-labels new video
* [Video Understanding](/docs/tutorials/video-understanding) — faces alongside visual + transcript search
* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Image Extractor](/docs/processing/extractors/image)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)


# Gemini Multifile Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/gemini-multifile

Embed multiple files from one object into a single 3072-d vector using Gemini Embedding 2 — one embedding per object, not per file

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["multimodal_understanding"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/gemini_multifile_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

The Gemini Multifile Extractor uses **Gemini Embedding 2** (`gemini-embedding-exp-03-07`, 3072-d) to embed all files of an object — images, PDFs, video, audio, and text — into **one unified vector per object** in a single API call.

This is fundamentally different from other extractors: instead of producing one document per file, it collapses all of an object's blobs into a single embedding representing the object as a whole.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/gemini\_multifile\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/gemini_multifile_extractor_v1)

## When to Use

| Use Case | Example |
| - | - |
| **Product catalogs** | Embed product photo + spec sheet + description together |
| **Medical records** | Embed scan + report + clinical notes as one object |
| **Legal documents** | Embed contract + exhibits + summary together |
| **E-commerce** | Embed product image + manual + label as one searchable unit |
| **Object-level search** | Find objects similar to a combination of inputs |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Per-file granularity needed (search within a PDF) | `document_graph_extractor` or `multimodal_extractor` |
| Single-file objects | `image_extractor` or `text_extractor` |
| Video frame-level search | `multimodal_extractor` |

## How It Works

Standard extractors produce **one document per blob** (file). The Gemini Multifile Extractor uses **array `input_mappings`** to collect multiple blob fields from one object into a single list, then sends all of them to Gemini Embedding 2 in one API call:

```
Object (hero_image + spec_sheet + description)
         ↓  array input_mappings
Single Gemini API call with all 3 parts
         ↓
One 3072-d embedding → One document per object
```

## Array `input_mappings`

The key difference from other extractors is the `input_mappings` value: instead of mapping one field, you map a **list of fields**. All listed fields are collected from each object and embedded together.

```json theme={null}
{
  "input_mappings": {
    "files": ["hero_image", "spec_sheet", "description"]
  }
}
```

The key (`"files"`) is the extractor's input name. The value is a list of blob field names from your bucket schema. Each field can be an image, PDF, video, audio, or text blob.

## Output

| Field | Type | Description |
| - | - | - |
| `gemini_multifile_extractor_v1_embedding` | `float[]` (3072-d) | Unified embedding for all input files |
| `source_blob_count` | `int` | Number of blobs embedded together |
| `source_blob_properties` | `string[]` | Names of the blob fields that contributed |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `output_dimensionality` | integer | `3072` | Embedding dimensions: `3072`, `768`, or `256` |
| `task_type` | string | `RETRIEVAL_DOCUMENT` | Embedding task hint for Gemini Embedding 2. Values: `RETRIEVAL_DOCUMENT`, `RETRIEVAL_QUERY`, `SEMANTIC_SIMILARITY`, `CLASSIFICATION`, `CLUSTERING` |
| `input_key` | string | `files` | Must match the key in `input_mappings` |

> [!NOTE]
>
> At query time, Mixpeek automatically uses `RETRIEVAL_QUERY` — you only need to set `task_type` at index time. The default `RETRIEVAL_DOCUMENT` is correct for most use cases. See [Text Extractor embedding task docs](/docs/processing/extractors/text#embedding-task) for details on each value.

## Complete Collection Setup

**1. Define your bucket schema with multiple blob fields**

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "product-catalog",
    "bucket_schema": {
      "properties": {
        "product_id":   {"type": "string"},
        "hero_image":   {"type": "image"},
        "spec_sheet":   {"type": "pdf"},
        "description":  {"type": "text"}
      }
    }
  }'
```

**2. Create a collection with array input\_mappings**

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "product-embeddings",
    "source": { "type": "bucket", "bucket_ids": ["bkt_abc123"] },
    "feature_extractor": {
      "feature_extractor_name": "gemini_multifile_extractor",
      "version": "v1",
      "input_mappings": {
        "files": ["hero_image", "spec_sheet", "description"]
      }
    }
  }'
```

> [!NOTE]
>
> The value of `"files"` is a **list** of field names, not a single string. This is what triggers the multi-file embedding behavior. All three fields are embedded together into one vector.

**3. Upload objects to the bucket**

Each object must have all the mapped fields populated:

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/objects" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "blobs": [
      { "property": "hero_image", "type": "image", "data": "s3://my-bucket/products/sku-001/hero.jpg" },
      { "property": "spec_sheet", "type": "pdf", "data": "s3://my-bucket/products/sku-001/spec.pdf" },
      { "property": "description", "type": "text", "data": "Lightweight carbon-fiber trail running shoe with Vibram outsole" }
    ],
    "product_id": "SKU-001"
  }'
```

**4. Process the batch**

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/batches" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_id": "bkt_abc123",
    "collection_ids": ["col_xyz789"]
  }'
```

Each object in the bucket produces **one document** containing one 3072-d embedding representing all its files combined.

## Creating a Retriever

After indexing, create a retriever to search the embedded objects. The `feature_search` stage supports two query input modes for this extractor:

### Single-item query (`text` or `content` mode)

Query with a single URL or text string — Gemini embeds it as-is and searches:

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product-object-search",
    "collection_identifiers": ["col_xyz789"],
    "input_schema": {
      "query": {"type": "text", "description": "Search query", "required": true}
    },
    "stages": [
      {
        "stage_name": "Object Search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://gemini_multifile_extractor@v1/gemini-embedding-exp-03-07",
                "query": {
                  "input_mode": "text",
                  "value": "{{INPUT.query}}"
                },
                "top_k": 20
              }
            ],
            "final_top_k": 10
          }
        }
      }
    ]
  }'
```

### Multi-file query (`multi_content` mode)

Query with **multiple files at once** — Gemini embeds all of them together, matching how objects were indexed. This produces the most accurate similarity scores because the query vector is built the same way as the index vectors.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product-multifile-search",
    "collection_identifiers": ["col_xyz789"],
    "input_schema": {
      "image_url":   {"type": "text", "description": "Product image URL"},
      "description": {"type": "text", "description": "Product description text"}
    },
    "stages": [
      {
        "stage_name": "Multi-file Object Search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://gemini_multifile_extractor@v1/gemini-embedding-exp-03-07",
                "query": {
                  "input_mode": "multi_content",
                  "values": [
                    "{{INPUT.image_url}}",
                    "{{INPUT.description}}"
                  ]
                },
                "top_k": 20
              }
            ],
            "final_top_k": 10
          }
        }
      }
    ]
  }'
```

Execute it by passing multiple files in `inputs`:

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "image_url": "https://example.com/query-shoe.jpg",
      "description": "trail running shoe lightweight"
    }
  }'
```

> [!NOTE]
>
> `multi_content` is only valid for feature URIs backed by `gemini_multifile_extractor`. Using it with any other extractor returns a `400` error.

## Dimensionality Reduction

Gemini Embedding 2 supports truncated dimensions for storage cost reduction:

```json theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "gemini_multifile_extractor",
    "version": "v1",
    "input_mappings": {
      "files": ["hero_image", "spec_sheet", "description"]
    },
    "params": {
      "output_dimensionality": 768
    }
  }
}
```

| Dimensions | Storage per vector | Quality |
| - | - | - |
| `3072` (default) | 12 KB | Best |
| `768` | 3 KB | Near-identical |
| `256` | 1 KB | Good — \~2% quality loss |

## Pricing

See [Billing & Pricing](/docs/platform/billing) — rates come from `GET /v1/billing/pricing`. One charge covers all N files within an object — not per file.

## Related

* [Feature Search Stage](/docs/retrieval/stages/feature-search) — Search with `multi_content` query mode
* [Image Extractor](/docs/processing/extractors/image) — Per-image embeddings (one document per image)
* [Multimodal Extractor](/docs/processing/extractors/multimodal) — Per-segment video/audio embeddings
* [Text Extractor](/docs/processing/extractors/text) — Per-chunk text embeddings


# Image Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/image

Dense vector embeddings for images using Google SigLIP (768D) for visual similarity search

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["image_search"]` (for PDF input, use `document_search`). Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/image_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Image extractor pipeline showing SigLIP processing and embedding generation](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/image.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=f6bf9fe32cfa5a823a28cb86321fee06)

The image extractor generates dense vector embeddings from images using Google's SigLIP model (768D). Optimized for visual similarity search, product matching, and cross-modal search with text queries. Fast (\~50-100ms per image) and cost-effective.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/image\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/image_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Filter Dataset** (if collection\_id provided)
   * Filter to specified collection
2. **Detect Content Types**
   * Sample 100 rows to identify images vs PDFs
3. **PDF Page Expansion** (conditional: if PDF content detected)
   * Render each PDF page at 72 DPI using PyMuPDF
   * Create separate image for each page
4. **SigLIP Image Embedding Generation**
   * Resize to 224×224 internally
   * GPU-accelerated inference
   * Generate 768D visual embeddings
5. **Thumbnail Generation** (conditional: if `enable_thumbnails=true`)
   * Resize to 640px width at 85% quality
   * Upload to S3 with optional CDN
6. **Output**
   * Image/page documents with embeddings
   * Optional thumbnail URLs

## When to Use

| Use Case | Description |
| - | - |
| **Image search** | Find visually similar images in large collections |
| **Visual similarity** | Match products, artwork, or content by appearance |
| **Content discovery** | Recommend similar visual content |
| **Cross-modal search** | Find images using text queries (via SigLIP text encoder) |
| **E-commerce** | Product image search and visual recommendations |
| **Stock photo search** | Media library search by visual content |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Face recognition | `face_identity_extractor` |
| Video content | `multimodal_extractor` |
| Text-heavy images requiring OCR | `multimodal_extractor` with OCR enabled |
| Audio content | `audio_extractor` |

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `image` | string | **Yes** | URL or S3 path to image file. Formats: JPEG, PNG, WebP, BMP. Any resolution (resized to 224x224 internally). |

```json theme={null}
{
  "image": "s3://my-bucket/products/laptop-pro.jpg"
}
```

**Input Examples:**

| Type | Example |
| - | - |
| Product image | `s3://my-bucket/products/laptop-pro.jpg` |
| Stock photo | `https://cdn.example.com/photos/sunset-beach.jpg` |
| Catalog image | `s3://catalog/items/SKU-12345.png` |

**Supported Formats**: JPEG, PNG, WebP, BMP, GIF (static)
**Recommended Resolution**: 224x224 or larger (automatically resized)
**Max File Size**: 10MB recommended

## Output Schema

| Field | Type | Description |
| - | - | - |
| `image_extractor_v1_embedding` | float\[768] | SigLIP image embedding, L2 normalized |
| `processing_time_ms` | number | Processing time in milliseconds |
| `thumbnail_url` | string | S3 URL of the thumbnail image (if generated) |

```json theme={null}
{
  "image_extractor_v1_embedding": [0.023, -0.041, 0.018, ...],
  "processing_time_ms": 85.2,
  "thumbnail_url": "s3://mixpeek-storage/ns_123/thumbnails/thumb_001.jpg"
}
```

## Parameters

The image extractor uses sensible defaults and requires no additional parameters for basic usage.

| Parameter | Type | Default | Description |
| - | - | - | - |
| *None required* | - | - | All parameters use optimized defaults |

## Configuration Examples

```json Basic Image Embedding theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "image_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.product_id" }
    ],
    "parameters": {}
  }
}
```

```json E-commerce Product Images theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "product_image"
    },
    "field_passthrough": [
      { "source_path": "metadata.sku" },
      { "source_path": "metadata.category" },
      { "source_path": "metadata.brand" }
    ],
    "parameters": {}
  }
}
```

```json Stock Photo Library theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "photo_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.photographer" },
      { "source_path": "metadata.tags" },
      { "source_path": "metadata.license" }
    ],
    "parameters": {}
  }
}
```

```json Art Collection theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "artwork_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.artist" },
      { "source_path": "metadata.title" },
      { "source_path": "metadata.year" },
      { "source_path": "metadata.medium" }
    ],
    "parameters": {}
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Processing speed** | \~50-100ms per image |
| **Batch processing** | Up to 16 images per batch |
| **GPU acceleration** | Supported for faster inference |
| **Cost** | Billed per image — see [Billing & Pricing](/docs/platform/billing); rates come from `GET /v1/billing/pricing` |

## Vector Index

| Property | Value |
| - | - |
| **Feature URI** | `mixpeek://image_extractor@v1/google_siglip_base_v1` |
| **Index name** | `image_extractor_v1_embedding` |
| **Dimensions** | 768 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Datatype** | float32 |
| **Inference model** | `google_siglip_base_v1` |

> [!NOTE]
>
> In retrievers, reference this feature by its **Feature URI** above (the output name is `google_siglip_base_v1`, **not** the index name `image_extractor_v1_embedding`).

### Cross-Modal Search

The SigLIP embeddings are compatible with SigLIP text embeddings, enabling cross-modal search where you can:

* Find images using natural language text queries
* Match images to text descriptions
* Build hybrid search combining visual and textual similarity

## Comparison with Other Image Extractors

| Feature | image\_extractor | multimodal\_extractor |
| - | - | - |
| **Dimensions** | 768 | 1408 |
| **Model** | SigLIP | Vertex AI Multimodal |
| **Processing** | Image only | Video, Image, Text, GIF |
| **Cross-modal** | SigLIP text encoder | Vertex text encoder |
| **Best For** | Fast image search | Unified multimodal search |
| **Cost** | See [Billing & Pricing](/docs/platform/billing) | Higher (includes more features) |

## Limitations

* **Image only**: Does not process video, audio, or text content
* **No OCR**: Cannot extract text from images; use `multimodal_extractor` with OCR
* **No face recognition**: For face matching, use `face_identity_extractor`
* **Single image**: Processes one image at a time (batch via API)
* **Resolution**: Input is resized to 224x224 internally

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)
* [Face Identity Extractor](/docs/processing/extractors/face-identity)
* [Text Extractor](/docs/processing/extractors/text)


# Multimodal Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/multimodal

Unified embeddings for video, image, audio, text, and GIF with transcription, OCR, thumbnails, and structured extraction

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["multimodal_understanding"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[Search your own video, images, and audio](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=extractor_page\&utm_campaign=managed_onramp\&utm_content=multimodal_extractor)**
>
> Create a managed namespace and run this pipeline on your own files — transcripts, faces, on-screen text, and unified embeddings, searchable in minutes.

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/multimodal_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Multimodal extractor pipeline showing video splitting, parallel processing with Whisper and embedding models, and output features](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/multimodal.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=8f068e849242497def22baeb67314d46)

The multimodal extractor processes **video, audio, image, text, and GIF content** through a unified pipeline. Videos and audio are decomposed into segments with transcription (Whisper), visual embeddings, OCR, and descriptions. Images and text are embedded directly without decomposition.

Two versions are available:

| Version | Embedding Model | Dimensions | Key Difference |
| - | - | - | - |
| **v1** | Vertex Multimodal Embedding | 1408 | Established, lower dimensionality |
| **v2** | Gemini Embedding 2 | 3072 (configurable: 1536, 768) | Higher dimensionality, Matryoshka support, native multimodal |

Both versions share the same pipeline (FFmpeg chunking, Whisper, thumbnails, Gemini vision) and differ only in the multimodal embedding step.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/multimodal\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/multimodal_extractor_v1) or [multimodal\_extractor\_v2](https://api.mixpeek.com/v1/collections/features/extractors/multimodal_extractor_v2). You can also fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Filter Dataset** (if collection\_id provided)
   * Filter to specified collection
2. **Apply Input Mappings**
3. **Detect Content Types** (sample 100 rows)
   * Identify: video, audio, image, text, or mixed
4. **Content Routing**
   * **Video:** FFmpeg chunking (time/scene/silence) → Steps 5-10
   * **Audio:** FFmpeg audio chunking (time/silence) → Steps 5-8
   * **Image:** Skip to Step 8
   * **Text:** Skip to Step 8
   * **Mixed:** Branch by type, process separately, union results
5. **Transcription** (conditional: if `run_transcription=true`, video/audio only)
   * Whisper API or Local GPU speech-to-text
6. **Transcription Embeddings** (conditional: if `run_transcription_embedding=true`)
   * E5-Large text embeddings (1024D) from transcribed audio
7. **Multimodal Embeddings** (conditional: if `run_multimodal_embedding=true`)
   * **v1:** Vertex AI embeddings (1408D)
   * **v2:** Gemini Embedding 2 (3072D, configurable)
   * Unified embedding space enables cross-modal search
8. **Thumbnail Generation** (conditional: if `enable_thumbnails=true`, visual content only)
   * 640px width at 85% quality, S3 upload with optional CDN
9. **Visual Analysis** (conditional: if `run_video_description` OR `run_ocr=true`, visual content only)
   * Gemini-based descriptions and/or OCR text extraction
10. **Output**
    * Segment/document records with embeddings, transcriptions, descriptions, OCR, thumbnails

## When to Use

| Use Case | Description |
| - | - |
| **Video content libraries** | Search and navigate video segments by content |
| **Media platforms** | Search across spoken and visual content |
| **Educational content** | Find moments in lectures and tutorials |
| **Surveillance/security** | Event detection in footage |
| **Social media** | Process user-generated video content |
| **Broadcasting/streaming** | Large video catalog management |
| **Marketing analytics** | Analyze video campaigns |
| **Cross-modal search** | Find videos/images using text queries |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Static image collections only | `image_extractor` |
| Audio-only content | `audio_extractor` |
| Very short videos (\< 5 seconds) | Processing overhead not worth it |
| Real-time live streams | Specialized streaming extractors |
| 8K+ resolution video | Consider downsampling first |
| Embed all files in one object as one vector | `gemini_multifile_extractor` |

## Supported Input Types

| Input | Type | Description | Processing |
| - | - | - | - |
| `video` | string | URL or S3 path | Decomposed into segments |
| `image` | string | URL or S3 path | Direct embedding (no decomposition) |
| `text` | string | Plain text content | Direct embedding |
| `gif` | string | URL or S3 path | Treated as video, frame-by-frame |

**Supported formats:**

* **Video**: MP4, MOV, AVI, MKV, WebM, FLV
* **Image**: JPG, PNG, WebP, BMP
* **GIF**: Animated GIF

## Input Schema

Provide **one** of the following inputs:

```json theme={null}
{
  "video": "s3://bucket/videos/lecture.mp4"
}
```

```json theme={null}
{
  "image": "https://cdn.example.com/products/laptop.jpg"
}
```

```json theme={null}
{
  "text": "High-performance laptop with M3 chip, perfect for developers"
}
```

| Field | Type | Description |
| - | - | - |
| `video` | string | URL/S3 path to video file. Recommended: 720p-1080p, \< 2 hours |
| `image` | string | URL/S3 path to image file. Recommended: \< 10MB |
| `text` | string | Plain text for cross-modal embedding |
| `gif` | string | URL/S3 path to GIF file |
| `custom_thumbnail` | string | Optional custom thumbnail URL instead of auto-generated |

## Output Schema

Each video segment produces one document. Images and text produce one document each without segmentation.

### Segment & Timing Fields

| Field | Type | Description |
| - | - | - |
| `start_time` | number \| null | Segment start time in seconds |
| `end_time` | number \| null | Segment end time in seconds |
| `start_frame` | integer \| null | Start frame number (`start_time × fps`) |
| `end_frame` | integer \| null | End frame number (`end_time × fps`) |
| `fps` | number \| null | Frame rate of the preprocessed video used for chunking |
| `source_fps` | number \| null | Original source video frame rate before any preprocessing (e.g. 29.97, 30, 23.976). Use this for precise frame-level calculations against the source video |
| `duration` | number \| null | Total duration of the entire source video in seconds (not the segment duration) |

> [!WARNING]
>
> Every field above is `null` when the source does not provide it. Handle the
> null rather than treating a missing value as `0`.
>
> These fields previously serialised as `0.0` when absent, which was
> indistinguishable from a real zero. A segment genuinely starting at
> `start_time: 0` and one whose start was never determined looked the same.
> Newly ingested documents carry `null`; existing corpora reflect it after
> re-ingest.
>
> **Re-ingest a whole collection at a time.** Until a collection is fully
> re-ingested it can hold both encodings of absent: `0.0` on documents written
> before the change and `null` on documents written after. A filter or
> aggregation over "missing" then matches one group and not the other, and the
> count looks plausible either way. A partial re-ingest leaves that split in
> place, so finish a collection before you rely on either encoding.

### Content Fields

| Field | Type | Description |
| - | - | - |
| `transcription` | string | Transcribed audio content (requires `run_transcription`) |
| `description` | string | AI-generated segment description (requires `run_video_description`) |
| `ocr_text` | string | Text extracted from video frames (requires `run_ocr`) |

### URL Fields

| Field | Type | Description |
| - | - | - |
| `thumbnail_url` | string | S3/CDN URL of the thumbnail image |
| `source_video_url` | string | URL of the original source video |
| `video_segment_url` | string | S3 URL of this specific segment file. Enables [collection-to-collection decomposition](#collection-to-collection-pipelines) |

### Embedding Fields

**v1**

| Field | Type | Description |
| - | - | - |
| `multimodal_extractor_v1_multimodal_embedding` | float\[1408] | Vertex AI multimodal embedding |
| `multimodal_extractor_v1_transcription_embedding` | float\[1024] | E5-Large transcription embedding |

**v2**

| Field | Type | Description |
| - | - | - |
| `multimodal_extractor_v2_multimodal_embedding` | float\[3072] | Gemini Embedding 2 multimodal embedding (configurable: 1536, 768) |
| `multimodal_extractor_v2_transcription_embedding` | float\[1024] | E5-Large transcription embedding |

### Example Output

**v1**

```json theme={null}
{
  "start_time": 10.0,
  "end_time": 20.0,
  "start_frame": 20,
  "end_frame": 40,
  "fps": 2.0,
  "source_fps": 29.97,
  "duration": 120.5,
  "transcription": "Welcome to today's lecture on machine learning fundamentals...",
  "description": "Instructor standing at whiteboard, introducing ML concepts",
  "ocr_text": "Machine Learning 101",
  "thumbnail_url": "s3://mixpeek-storage/ns_123/thumbnails/thumb_1.jpg",
  "source_video_url": "s3://mixpeek-storage/ns_123/obj_456/original.mp4",
  "video_segment_url": "s3://mixpeek-storage/ns_123/obj_456/segments/segment_001.mp4",
  "multimodal_extractor_v1_multimodal_embedding": [0.023, -0.041, "...1408 floats"],
  "multimodal_extractor_v1_transcription_embedding": [0.018, -0.032, "...1024 floats"]
}
```

**v2**

```json theme={null}
{
  "start_time": 10.0,
  "end_time": 20.0,
  "start_frame": 20,
  "end_frame": 40,
  "fps": 2.0,
  "source_fps": 29.97,
  "duration": 120.5,
  "transcription": "Welcome to today's lecture on machine learning fundamentals...",
  "description": "Instructor standing at whiteboard, introducing ML concepts",
  "ocr_text": "Machine Learning 101",
  "thumbnail_url": "s3://mixpeek-storage/ns_123/thumbnails/thumb_1.jpg",
  "source_video_url": "s3://mixpeek-storage/ns_123/obj_456/original.mp4",
  "video_segment_url": "s3://mixpeek-storage/ns_123/obj_456/segments/segment_001.mp4",
  "multimodal_extractor_v2_multimodal_embedding": [0.015, -0.038, "...3072 floats"],
  "multimodal_extractor_v2_transcription_embedding": [0.018, -0.032, "...1024 floats"]
}
```

> [!NOTE]
>
> `fps` reflects the preprocessed video frame rate (e.g. 2.0 fps after downsampling). `source_fps` is the original video's native frame rate (e.g. 29.97). Use `source_fps` when you need to map timestamps back to exact frame numbers in the original source file.

## Parameters

### Video Splitting

| Parameter | Type | Default | Description |
| - | - | - | - |
| `split_method` | string | `"time"` | Primary video splitting strategy: `time`, `scene`, or `silence` |
| `max_segment_duration` | float | `30.0` | Maximum seconds per segment. Scene/silence segments longer than this are subdivided. Set to `null` to disable |

#### Split Methods

**time**

**Fixed interval splitting** - Splits video into segments of equal duration.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `time_split_interval` | integer | `10` | Interval in seconds for each segment |

**Characteristics:**

* Predictable segment count: `video_duration / interval`
* Consistent chunk sizes for uniform processing
* May cut mid-sentence or mid-scene

**Best for:** General purpose, consistent chunking, when you need predictable segment counts

```json theme={null}
{
  "split_method": "time",
  "time_split_interval": 10
}
```

**scene**

**Visual change detection** - Splits video when significant visual changes occur (shot changes, transitions).

| Parameter | Type | Default | Description |
| - | - | - | - |
| `scene_detection_threshold` | float | `0.5` | Sensitivity threshold (0.0-1.0) |

**Threshold guide:**

* `0.3` - High sensitivity, detects subtle changes (more segments)
* `0.5` - Balanced (default)
* `0.7` - Low sensitivity, only major scene changes (fewer segments)

**Characteristics:**

* Variable segment count (typically 2-20 per minute)
* Segments align with visual content boundaries
* Better for content with distinct shots/scenes

**Best for:** Movies, dynamic content, shot changes, music videos, advertisements

```json theme={null}
{
  "split_method": "scene",
  "scene_detection_threshold": 0.5
}
```

**silence**

**Audio pause detection** - Splits video at moments of silence or low audio.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `silence_db_threshold` | integer | `-40` | Decibel level below which audio is considered silent |

**Threshold guide:**

* `-50` dB - Detects very quiet moments (more segments)
* `-40` dB - Balanced (default)
* `-30` dB - Only detects near-silence (fewer segments)

**Characteristics:**

* Variable segment count (typically 5-30 per minute)
* Segments align with natural speech pauses
* Preserves complete sentences/thoughts

**Best for:** Lectures, presentations, conversations, podcasts, interviews

```json theme={null}
{
  "split_method": "silence",
  "silence_db_threshold": -40
}
```

#### Split Methods Comparison

| Method | Segments/Min | Predictability | Best For |
| - | - | - | - |
| `time` | 60 / interval\_sec | High | General purpose, batch processing |
| `scene` | Variable (2-20) | Low | Movies, ads, dynamic visual content |
| `silence` | Variable (5-30) | Medium | Lectures, podcasts, spoken content |

### Feature Extraction Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `run_transcription` | boolean | `true` (v1) / `false` (v2) | Run Whisper transcription on audio |
| `transcription_language` | string \| null | `null` | Force a transcription language by ISO-639-1 code (`ru`, `fa`, `zh`). Leave unset and Whisper detects the language itself |
| `run_transcription_embedding` | boolean | `true` (v1) / `false` (v2) | Generate E5 embeddings for transcriptions |
| `run_multimodal_embedding` | boolean | `true` | Generate multimodal embeddings |
| `run_video_description` | boolean | `false` | Generate AI descriptions (adds 1-2s per segment) |
| `run_ocr` | boolean | `false` | Extract text from video frames |

### Languages and translation

Whisper detects the language of each segment on its own, so a corpus that mixes
Russian, Farsi and Mandarin needs no configuration. Set `transcription_language`
only to override that detection, which is worth doing when broadcast audio with
music beds or heavy code-switching makes Whisper guess wrong.

`transcription` comes back in the **language that was spoken**. There is no
translation output field, and no extractor produces one. Search still reaches
across languages, because the E5 and Gemini embeddings put all 100+ languages in
a single vector space, so an English query matches Russian speech directly.

When you need the English text itself alongside the original, translate at
retrieval time with an `enrich` stage. The
[cross-lingual recipe](/docs/retrieval/cookbook) has the stage config; each result
then carries both `content` and `translation`.

### Thumbnail Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `enable_thumbnails` | boolean | `true` | Generate thumbnail images |
| `use_cdn` | boolean | `false` | Use CloudFront CDN for thumbnails |

**CDN benefits**: Faster global delivery, permanent URLs, reduced bandwidth costs.

### v2-Only Parameters

These parameters are only available on `multimodal_extractor` v2:

| Parameter | Type | Default | Description |
| - | - | - | - |
| `output_dimensionality` | integer | `3072` | Embedding dimensions. Gemini Embedding 2 supports Matryoshka reduction: `3072` (full), `1536`, or `768` |
| `task_type` | string | `"RETRIEVAL_DOCUMENT"` | Embedding task hint: `RETRIEVAL_DOCUMENT`, `RETRIEVAL_QUERY`, `SEMANTIC_SIMILARITY`, `CLASSIFICATION` |

> [!NOTE]
>
> At query time, Mixpeek automatically uses `RETRIEVAL_QUERY` — you only need to set `task_type` at index time. The default `RETRIEVAL_DOCUMENT` is correct for most use cases.

### Embedding Task

When `run_transcription_embedding` is enabled, the E5 model generates text embeddings from transcribed audio. By default, these use `retrieval_document` for asymmetric search.

> [!NOTE]
>
> Set `embedding_task` at the **collection level**, not on the extractor. See [Collection Embedding Task](/docs/platform/processing#embedding-task) for full details and examples.

> [!NOTE]
>
> This only affects the E5 transcription embeddings. Vertex AI multimodal embeddings (v1) and Gemini Embedding 2 (v2) are not instruction-aware and ignore this parameter.

### Description Generation Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `description_prompt` | string | `"Describe the video segment in detail."` | Prompt for Gemini |
| `generation_config.temperature` | float | `0.7` | Randomness (higher = more creative) |
| `generation_config.max_output_tokens` | integer | `1024` | Maximum description length |
| `generation_config.top_p` | float | `0.8` | Nucleus sampling |

### LLM Structured Extraction

| Parameter | Type | Default | Description |
| - | - | - | - |
| `response_shape` | string \| object | `null` | Custom structured output schema |

**Natural Language Mode:**

```json theme={null}
{
  "response_shape": "Extract product names, colors, materials, and aesthetic style labels from this fashion segment"
}
```

**JSON Schema Mode:**

```json theme={null}
{
  "response_shape": {
    "type": "object",
    "properties": {
      "products": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "category": { "type": "string" },
            "visibility_percentage": { "type": "integer", "minimum": 0, "maximum": 100 }
          },
          "required": ["name"]
        }
      },
      "aesthetic": { "type": "string" }
    },
    "required": ["products"]
  }
}
```

Standard JSON Schema `required` lists work at any level. Each listed field is stored as required in the collection's output schema.

## Configuration Examples

```json v1 — Video with Time-Based Splitting theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.video_id" }
    ],
    "parameters": {
      "split_method": "time",
      "time_split_interval": 10,
      "run_transcription": true,
      "run_multimodal_embedding": true,
      "enable_thumbnails": true
    }
  }
}
```

```json v1 — Video with Scene Detection theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {
      "split_method": "scene",
      "scene_detection_threshold": 0.5,
      "run_transcription": true,
      "run_video_description": true,
      "enable_thumbnails": true
    }
  }
}
```

```json v1 — Lecture Video with Silence Splitting theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "lecture_url"
    },
    "parameters": {
      "split_method": "silence",
      "silence_db_threshold": -40,
      "run_transcription": true,
      "run_ocr": true,
      "enable_thumbnails": true
    }
  }
}
```

```json v2 — Video with Gemini Embedding 2 theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v2",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {
      "split_method": "time",
      "time_split_interval": 10,
      "run_multimodal_embedding": true,
      "output_dimensionality": 3072,
      "enable_thumbnails": true
    }
  }
}
```

```json v2 — Compact Embeddings (768D) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v2",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {
      "split_method": "scene",
      "scene_detection_threshold": 0.5,
      "run_multimodal_embedding": true,
      "output_dimensionality": 768,
      "run_transcription": true,
      "run_transcription_embedding": true,
      "enable_thumbnails": true
    }
  }
}
```

```json Image Embedding (v1 or v2) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "image": "image_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.product_id" }
    ],
    "parameters": {
      "run_multimodal_embedding": true,
      "enable_thumbnails": true
    }
  }
}
```

```json Text Embedding (Cross-Modal Search) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "product_description"
    },
    "parameters": {
      "run_multimodal_embedding": true
    }
  }
}
```

```json v1 — Full Extraction with All Features theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {
      "split_method": "scene",
      "scene_detection_threshold": 0.5,
      "run_transcription": true,
      "run_transcription_embedding": true,
      "run_multimodal_embedding": true,
      "run_video_description": true,
      "run_ocr": true,
      "enable_thumbnails": true,
      "use_cdn": true,
      "description_prompt": "Describe what is happening in this video segment, including any visible products, people, and actions.",
      "generation_config": {
        "temperature": 0.7,
        "max_output_tokens": 1024
      }
    }
  }
}
```

```json Fashion/E-commerce with Structured Extraction theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "fashion_video_url"
    },
    "parameters": {
      "split_method": "scene",
      "run_multimodal_embedding": true,
      "run_video_description": true,
      "response_shape": {
        "type": "object",
        "properties": {
          "products": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "name": { "type": "string" },
                "category": { "type": "string" },
                "color": { "type": "string" },
                "visibility_percentage": { "type": "integer" }
              }
            }
          },
          "aesthetic": { "type": "string" },
          "setting": { "type": "string" }
        }
      }
    }
  }
}
```

## Performance & Costs

### Processing Speed

| Content Type | Speed |
| - | - |
| Video | 0.5-2x realtime (depends on features enabled) |
| Image | \< 1 second |
| Text | \< 100ms |

**Example**: 10-minute video → 5-20 minutes processing time

| Feature | Latency per Segment |
| - | - |
| Transcription | \~200ms per second of audio |
| Visual embedding | \~50ms |
| OCR | \~300ms |
| Description | \~2s |

### Cost Estimates (per minute of video)

| Configuration | Cost |
| - | - |
| **Minimal** (transcription + embeddings) | \$0.01 |
| **Standard** (+ OCR) | \$0.05 |
| **Full** (+ descriptions) | \$0.15 |

**Images**: $0.001 per image **Text**: $0.0001 per query

## Vector Indexes

**v1**

### Multimodal Embedding

| Property | Value |
| - | - |
| **Feature URI** | `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding` |
| **Index name** | `multimodal_extractor_v1_multimodal_embedding` |
| **Dimensions** | 1408 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `vertex_multimodal_embedding` |
| **Supported inputs** | video, text, image |

### Transcription Embedding

| Property | Value |
| - | - |
| **Feature URI** | `mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1` |
| **Index name** | `multimodal_extractor_v1_transcription_embedding` |
| **Dimensions** | 1024 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `multilingual_e5_large_instruct_v1` |
| **Supported inputs** | text, string |

> [!NOTE]
>
> In retrievers, reference these by their **Feature URI** — the output name is the model name, **not** the `multimodal_extractor_v1_*` index name.

**v2**

### Multimodal Embedding

| Property | Value |
| - | - |
| **Index name** | `multimodal_extractor_v2_multimodal_embedding` |
| **Dimensions** | 3072 (configurable: 1536, 768) |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `google/gemini-embedding-2` |
| **Supported inputs** | video, text, image, audio |

### Transcription Embedding

| Property | Value |
| - | - |
| **Index name** | `multimodal_extractor_v2_transcription_embedding` |
| **Dimensions** | 1024 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `intfloat/multilingual-e5-large-instruct` |
| **Supported inputs** | text, string |

## Choosing v1 vs v2

| Consideration | v1 | v2 |
| - | - | - |
| **Embedding quality** | Good | Better (natively multimodal) |
| **Dimensions** | 1408 (fixed) | 3072, 1536, or 768 (configurable) |
| **Storage per vector** | 5.5 KB | 12 KB (3072D), 6 KB (1536D), 3 KB (768D) |
| **Audio input support** | Via transcription only | Native audio embedding |
| **Matryoshka support** | No | Yes — reduce dimensions without reindexing |
| **Stability** | Production-proven | Newer |

**Recommendation:** Use **v2** for new projects. Use **v1** if you have existing collections and don't need higher dimensions or native audio embedding.

## Limitations

* **Video duration**: Recommend \< 2 hours for optimal processing
* **Resolution**: 8K+ videos should be downsampled
* **Real-time**: Not suitable for live streaming
* **Short videos**: \< 5 second videos have disproportionate overhead
* **Audio quality**: Transcription accuracy depends on audio clarity
* **OCR/Description**: Add significant processing time, enable only when needed

## Collection-to-Collection Pipelines

The `video_segment_url` output enables decomposition chains:

1. **Initial collection**: Time-based segments (5s intervals)
2. **Downstream collection**: Scene detection within each segment
3. **Final collection**: Enhanced processing with different models

```json theme={null}
{
  "input_mappings": {
    "video": "video_segment_url"
  }
}
```

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Gemini Multifile Extractor](/docs/processing/extractors/gemini-multifile) — Embed multiple files per object into one vector
* [Passthrough Extractor](/docs/processing/extractors/passthrough)
* [Text Extractor](/docs/processing/extractors/text)


# Passthrough Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/passthrough

Copy source fields without processing for metadata propagation and vector passthrough

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["storage_only"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/passthrough_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Passthrough extractor showing direct field copy from source to output](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/passthrough.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=7eb05fc4ed420817552c55d0a59439a5)

The passthrough extractor copies source fields without any ML processing. By default, all source fields are included. Use `field_passthrough` to specify specific fields or `include_all_source_fields` to control behavior. Supports vector passthrough from collection to collection.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/passthrough\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/passthrough_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## When to Use

| Use Case | Description |
| - | - |
| **Metadata propagation** | Pass metadata fields from bucket objects to collection documents without transformation |
| **Vector passthrough** | Copy pre-computed embeddings from one collection to another |
| **Schema normalization** | Select specific fields to include in the output schema |
| **Collection-to-collection pipelines** | Route data through multi-tier processing without re-embedding |

## When NOT to Use

* When you need to generate embeddings → Use [text\_extractor](/docs/processing/extractors/text) or [multimodal\_extractor](/docs/processing/extractors/multimodal)
* When you need to transform or enrich data → Use extractors with ML models
* When you need to decompose content (chunking, video splitting) → Use appropriate extractors
* When you have embeddings computed outside Mixpeek → Upsert them into a [Vector Store](/docs/vector-store/overview) namespace. Vector passthrough copies vectors between Mixpeek collections, and this extractor declares no vector index of its own.

## Input Schema

The passthrough extractor accepts **any input type** and copies fields as-is.

```json theme={null}
{
  "type": "object",
  "properties": {},
  "description": "Accepts any fields - no required inputs"
}
```

## Output Schema

The output mirrors the input based on configuration:

| Configuration | Behavior |
| - | - |
| Default (`include_all_source_fields: true`) | All source fields copied to output |
| `field_passthrough` specified | Only listed fields copied |
| `include_all_source_fields: false` | Must specify `field_passthrough` |

## Parameters

The passthrough extractor has **no required parameters**. Configuration is handled through `field_passthrough` and `input_mappings` at the collection level.

## Configuration Examples

```json Copy All Fields theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "passthrough_extractor",
    "version": "v1",
    "input_mappings": {},
    "field_passthrough": [],
    "parameters": {}
  }
}
```

```json Select Specific Fields theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "passthrough_extractor",
    "version": "v1",
    "input_mappings": {},
    "field_passthrough": [
      { "source_path": "metadata.category" },
      { "source_path": "metadata.brand" },
      { "source_path": "metadata.price" }
    ],
    "parameters": {}
  }
}
```

```json Vector Passthrough theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "passthrough_extractor",
    "version": "v1",
    "input_mappings": {},
    "field_passthrough": [
      { "source_path": "text_extractor_v1_embedding" },
      { "source_path": "metadata.product_id" }
    ],
    "parameters": {}
  }
}
```

```json Rename Fields During Passthrough theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "passthrough_extractor",
    "version": "v1",
    "input_mappings": {},
    "field_passthrough": [
      { "source_path": "metadata.old_name", "target_path": "new_name" },
      { "source_path": "payload.description", "target_path": "content" }
    ],
    "parameters": {}
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Latency** | \< 1ms |
| **Cost** | Free |
| **GPU Required** | No |
| **Max Throughput** | Unlimited (no ML processing) |

## Vector Indexes

The passthrough extractor creates **no vector indexes**. If you pass through existing embeddings, they retain their original index configuration from the source collection.

## Best Practices

1. **Use for multi-tier pipelines** – When downstream collections need upstream data without reprocessing
2. **Minimize field selection** – Only pass through fields you need to reduce storage and query overhead
3. **Preserve lineage** – The passthrough extractor maintains `root_object_id` and `source_collection_id` for data lineage tracking
4. **Combine with other extractors** – Use passthrough fields alongside ML extractors in the same collection to include metadata with generated features

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Text Extractor](/docs/processing/extractors/text)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)


# Scrolling Text Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/scrolling-text

Extract scrolling/marquee text from video using phase-correlation band detection, panoramic stitching, and VLM OCR

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["onscreen_text"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[Extract on-screen text from your own video](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_scrolling_text\&utm_campaign=managed_onramp)**
>
> Create a managed namespace and run this pipeline on your own files — recover scrolling and static on-screen text, OCR it, and make it searchable in minutes.

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/scrolling_text_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

The scrolling text extractor recovers **scrolling or marquee text** from video — tickers, lower-third banners, end credits, and legal disclaimers — that no single frame ever shows in full. It detects scrolling bands via phase correlation, stitches frames panorama-style to reconstruct the complete text, then OCRs the panorama with a vision language model (Gemini). Output is payload-only (no vector); pair it with `text_extractor` if you need semantic search over the recovered text.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/scrolling\_text\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/scrolling_text_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Sample frames** — extract frames at `fps` frames per second.
2. **Phase correlation** — scan `strip_height`-pixel strips to measure per-frame pixel shift and detect motion.
3. **Classify bands** — a band counts as scrolling when shift exceeds `min_shift_px` and at least `consistency_ratio` of frame pairs agree.
4. **Crop** — crop each detected band with `pad` pixels of padding above and below.
5. **Stitch** — reconstruct the full scrolling content as a panorama image per band.
6. **VLM OCR** — read the panorama with a vision language model (Gemini).
7. **Output** — combined text plus per-band metadata (axis, direction, shift).

## When to Use

| Use Case | Description |
| - | - |
| **News tickers** | Recover the full crawl text from a horizontally scrolling ticker |
| **End credits** | Capture vertically scrolling credit rolls |
| **Compliance disclaimers** | Extract fast-scrolling legal/disclaimer banners for audit |
| **Sports/finance banners** | Read scrolling score or price strips |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Static on-screen text / captions | `text_extractor` on transcription, or a frame OCR extractor |
| Spoken-word transcription | A transcription/audio extractor |
| Semantic search over recovered text | Chain `text_extractor` on the `scrolling_text` field |
| Non-video inputs | This extractor is video-only |

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `video` | string | **Yes** | URL or path to the video file. Populated from `input_mappings`. |

```json theme={null}
{
  "video": "s3://my-bucket/clips/newscast.mp4"
}
```

Supported input types: **VIDEO**.

## Output Schema

| Field | Type | Description |
| - | - | - |
| `scrolling_text` | string \| null | Combined, deduplicated text from all detected scrolling bands |
| `scroll_bands` | object\[] \| null | Per-band details: `axis`, `direction`, `shift_per_frame`, `text` |
| `bands_detected` | integer \| null | Number of scrolling text bands detected in the video |

```json theme={null}
{
  "scrolling_text": "BREAKING: Markets rally as inflation cools ...",
  "bands_detected": 1,
  "scroll_bands": [
    {
      "axis": "horizontal",
      "direction": "right_to_left",
      "shift_per_frame": 6.4,
      "text": "BREAKING: Markets rally as inflation cools ..."
    }
  ]
}
```

## Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `fps` | float | `5.0` | 1.0–30.0 | Frame sampling rate. Higher values improve detection for fast-scrolling text but increase processing time |
| `strip_height` | integer | `40` | 10–200 | Height (px) of each scanning strip used for phase correlation. Should roughly match the scrolling text band height |
| `min_shift_px` | float | `2.0` | 0.5–20.0 | Minimum per-frame pixel shift to consider a strip 'scrolling'. Lower detects slower text; higher filters noise |
| `consistency_ratio` | float | `0.6` | 0.3–1.0 | Fraction of frame pairs that must show consistent shift for a band to count as scrolling (0.6 = 60%) |
| `pad` | integer | `8` | 0–50 | Pixel padding above/below the detected band when cropping for stitching |

## Configuration Examples

```json Default Detection theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "scrolling_text_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {}
  }
}
```

```json Fast Ticker (high fps, thin band) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "scrolling_text_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {
      "fps": 15.0,
      "strip_height": 30,
      "min_shift_px": 5.0
    }
  }
}
```

```json Slow Credits Roll (tall band, sensitive) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "scrolling_text_extractor",
    "version": "v1",
    "input_mappings": {
      "video": "video_url"
    },
    "parameters": {
      "fps": 3.0,
      "strip_height": 120,
      "min_shift_px": 1.0,
      "consistency_ratio": 0.5
    }
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Cost** | Billed per video minute (frame extraction + stitching + VLM OCR) — see [Billing & Pricing](/docs/platform/billing); rates come from `GET /v1/billing/pricing` |
| **External API** | Google Gemini (VLM OCR) |
| **Tradeoff** | Higher `fps` improves fast-scroll accuracy at the cost of processing time |

## Vector Index

This extractor produces **payload-only output** — no vector index. The recovered text lives in the `scrolling_text` field. To make it semantically searchable, run `text_extractor` against `scrolling_text`.

## Limitations

* **Video only**: Accepts video inputs exclusively.
* **No embedding**: Output is payload-only; semantic search requires chaining a text extractor.
* **Band-height sensitivity**: `strip_height` should approximate the actual band height for reliable detection.
* **VLM dependency**: OCR quality depends on Gemini VLM availability and panorama clarity.

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Text Extractor](/docs/processing/extractors/text)
* [Universal Extractor](/docs/processing/extractors/universal)


# Text Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/text

Dense multilingual (E5-Large) text embeddings for semantic and cross-lingual search — query in one language, match content in 100+ others

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["text_search"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/text_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Text extractor pipeline showing chunking, E5-Large embedding, and optional LLM extraction](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/text.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=5c9f4dd60090c5941a6283f448b4da86)

The text extractor generates dense vector embeddings from text using the E5-Large multilingual model. Optimized for semantic search, RAG applications, and general-purpose text retrieval. Supports text chunking/decomposition with multiple splitting strategies. Fast (5ms/doc), cost-effective (free), and supports 100+ languages.

> [!NOTE]
>
> **Cross-lingual retrieval.** Because all 100+ languages share one embedding space, a query in one language matches content in **any** language — an English query (e.g. "foreign language", "translate what they said") retrieves Farsi, Mandarin, or Spanish text with **no translation step and no per-language index**. Note that this is *retrieval*, not translation: matched text is returned in its original language. To render it in the reader's language, add an `llm_enrich` translate stage — see the [Search Across Languages recipe](/docs/retrieval/cookbook#search-across-languages-cross-lingual).

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/text\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/text_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Filter Dataset** (if collection\_id provided)
   * Filter to specified collection
2. **Apply Input Mappings**
   * Resolve text field from source (e.g., `transcription`, `content`, `data`)
3. **Text Chunking** (conditional: if `split_by != "none"`)
   * Split by: characters, words, sentences, paragraphs, pages, or markdown structure
   * Configure `chunk_size` and `chunk_overlap`
   * Each chunk becomes a separate document
4. **E5 Text Embedding Generation**
   * Multilingual E5-Large model (1024D)
   * L2 normalized vectors
   * Batch size: 4,096 texts
5. **Output**
   * Text documents with embeddings
   * One document per input (or per chunk if chunking enabled)

## When to Use

| Use Case | Description |
| - | - |
| **Product search** | Search products by natural language descriptions |
| **FAQ matching** | Match user questions to knowledge base articles |
| **Document retrieval** | Find relevant documents from large corpora |
| **Content discovery** | Recommend similar content based on semantic similarity |
| **RAG chunking** | Split documents into chunks for retrieval-augmented generation |
| **Multi-language / cross-lingual search** | Query in one language, match content in another (foreign-language, translate-intent queries) across 100+ languages with a single model |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Exact phrase / keyword matching (SKUs, codes, names) | Add a [lexical (BM25) search](/docs/retrieval/stages/feature-search#lexical-bm25-search) |
| Keyword-heavy queries (e.g. "iPhone 15 Pro Max 256GB") | Lexical (BM25) search alongside the dense one |
| Critical technical terms or short texts (1–5 words) | Lexical (BM25) search |

> [!NOTE]
>
> Dense embeddings and exact-keyword matching are complementary. Keep `text_extractor` for semantic recall and add a `lexical: true` search over a `text` index for exact tokens — fuse them with `rrf`. See [Lexical (BM25) Search](/docs/retrieval/stages/feature-search#lexical-bm25-search).

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `text` | string | **Yes** | Text content to process. Recommended: 10-400 words for optimal quality. Maximum: 512 tokens (\~400 words), longer text is truncated. |

```json theme={null}
{
  "text": "Premium wireless Bluetooth headphones with active noise cancellation, 30-hour battery life, and premium sound quality."
}
```

**Input Examples:**

| Type | Example |
| - | - |
| Product description | "Premium wireless Bluetooth headphones with active noise cancellation" |
| FAQ question | "How do I reset my password if I forgot it?" |
| Article paragraph | "Machine learning models have revolutionized natural language processing..." |
| User query | "best restaurants near Times Square" |

## Output Schema

| Field | Type | Description |
| - | - | - |
| `text` | string | The processed text content (full text or chunk) |
| `text_extractor_v1_embedding` | float\[1024] | Dense vector embedding, L2 normalized |

```json theme={null}
{
  "text": "Premium wireless Bluetooth headphones with active noise cancellation",
  "text_extractor_v1_embedding": [0.023, -0.041, 0.018, ...]
}
```

When chunking is enabled, each chunk becomes a separate document with tracking metadata stored in `metadata` (not in the document payload):

* `chunk_index` – Position of this chunk in the original document
* `chunk_text` – The text content of this chunk
* `total_chunks` – Total number of chunks from the source

With `split_by: "markdown"`, each document also carries three structure fields:

| Field | Type | Description |
| - | - | - |
| `heading_path` | string | Headings enclosing the chunk, outermost first, joined with `" > "` |
| `chunk_kind` | string | `"section"` for prose, `"code"` for a fenced code block |
| `code_language` | string | Language from the code fence's info string, when the fence names one. Set on code chunks only. |

## Parameters

### Chunking Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `split_by` | string | `"none"` | Strategy for splitting text into chunks |
| `chunk_size` | integer | `1000` | Target size for each chunk (units depend on `split_by`) |
| `chunk_overlap` | integer | `0` | Number of units to overlap between consecutive chunks. Not used for `markdown`. |

#### Split Strategies

| Strategy | Description | Best For |
| - | - | - |
| `characters` | Split by character count | Uniform sizes, quick testing |
| `words` | Split by word boundaries | General text, preserves words |
| `sentences` | Split by sentence boundaries | Q\&A, precise retrieval, preserves semantic units |
| `paragraphs` | Split by paragraph (double newlines) | Articles, documentation, natural structure |
| `pages` | Split by page breaks | PDFs, paginated documents |
| `markdown` | One document per heading section and one per fenced code block | Docs sites, READMEs, any markdown with code examples |
| `none` | No splitting (default) | Short texts \< 400 words |

**Recommended chunk sizes:**

* `characters`: 500-2000
* `words`: 100-400
* `sentences`: 3-10
* `paragraphs`: 1-3
* `pages`: 1
* `markdown`: `chunk_size` counts words of prose per section chunk

#### Markdown splitting

`split_by: "markdown"` follows the structure of the page:

* Each heading section becomes one document. The chunk starts with its heading, so it reads on its own.
* A section longer than `chunk_size` words splits at paragraph boundaries.
* Each fenced code block becomes its own document. Code blocks are never split, whatever their length.
* Front matter is dropped. Its `title` becomes the outermost heading in `heading_path`.
* A `#` comment inside a code block does not count as a heading.

Use it when your text has code examples. The other strategies count characters, words or sentences, so they can cut through a code fence and lose the code's language and section.

The ColBERT extractor (`colbert_extractor`) accepts `split_by: "markdown"` too.

**Chunk overlap:** 10-20% of `chunk_size` helps preserve context across boundaries. Example: `chunk_size: 1000`, `chunk_overlap: 100-200`.

### Embedding Model

| Parameter | Type | Default | Description |
| - | - | - | - |
| `embedding_model` | string | *current TEXT default* | Override the embedding model via the [model registry](/docs/processing/model-registry) |

The text extractor resolves its model through the central embedding registry. Leave unset to use the current TEXT modality default (`intfloat_e5_large_instruct_v1`, 1024d) — the registry swaps hot when a new frontier text model ships, so existing collections pick it up without a code change.

> [!WARNING]
>
> Dimensions are locked at namespace creation. Switching `embedding_model` on an existing namespace requires a migration since the vector index dimensionality is fixed.

### Embedding Task

Instruction-aware embedding models (E5, Gemini) use a **task hint** to optimize the embedding for a specific downstream use case. By default, all extractors use `retrieval_document` at ingestion time, which produces embeddings optimized for asymmetric search (queries find documents).

> [!NOTE]
>
> Set `embedding_task` at the **collection level**, not on the extractor. See [Collection Embedding Task](/docs/platform/processing#embedding-task) for full details and examples.

| Task | Use Case | Effect on E5 | Effect on Gemini |
| - | - | - | - |
| `retrieval_document` | **Default.** Search: find documents from queries | Prepends `"passage: "` | Instructs "represent for retrieval" |
| `retrieval_query` | Rare at index time. Query-side is automatic | Prepends `"query: "` | Instructs "represent this query" |
| `semantic_similarity` | Symmetric comparison (deduplication, matching) | Prepends `"query: "` | Instructs "represent for similarity" |
| `classification` | Document categorization pipelines | Prepends `"query: "` | Instructs "represent for classification" |
| `clustering` | Grouping documents into clusters | Prepends `"query: "` | Not applied |

> [!NOTE]
>
> You almost never need to set this. The default `retrieval_document` is correct for search, and at query time Mixpeek automatically uses `retrieval_query`. Only override if your collection is primarily used for clustering, classification, or symmetric similarity — not retrieval.

> [!NOTE]
>
> Non-instruction-aware models (SigLIP, CLIP, Vertex multimodal) ignore this parameter.

### LLM Structured Extraction Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `response_shape` | string \| object | `null` | Define custom structured output using LLM extraction |
| `llm_provider` | string | `null` | LLM provider: `openai`, `google`, `anthropic`, `typesafe` |
| `llm_model` | string | `null` | Specific model for extraction |

#### response\_shape Modes

**Natural Language Mode (string):**

```json theme={null}
{
  "response_shape": "Extract key entities, sentiment (positive/negative/neutral), and main topics from the text"
}
```

The service automatically infers JSON schema from your description.

**JSON Schema Mode (object):**

```json theme={null}
{
  "response_shape": {
    "type": "object",
    "properties": {
      "sentiment": {
        "type": "string",
        "enum": ["positive", "negative", "neutral"]
      },
      "entities": {
        "type": "array",
        "items": { "type": "string" }
      },
      "topics": {
        "type": "array",
        "items": { "type": "string" },
        "maxItems": 5
      }
    },
    "required": ["sentiment"]
  }
}
```

#### LLM Provider & Model Options

| Provider | Example Models |
| - | - |
| `openai` | `gpt-4o-mini-2024-07-18` (cost-effective), `gpt-4o-2024-08-06` (best quality) |
| `google` | `gemini-2.5-flash` (fastest), `gemini-1.5-flash-001` |
| `anthropic` | `claude-3-5-haiku-20241022` (fast), `claude-3-5-sonnet-20241022` (best reasoning) |
| `typesafe` | `jev-latest`: labels, flags and ratings from a fixed set, with probabilities |

With `llm_provider: "typesafe"`, [Jev](/docs/processing/jev) answers each `response_shape` field as a typed question. `response_shape` must be a JSON schema, and every field needs a closed set of values: a `boolean`, an `enum`, a `number` from 0 to 1, or an `integer` with `minimum` and `maximum`. Natural-language `response_shape` needs a generative model. The collection is rejected at creation if a field cannot be answered, and the error names the field.

## Configuration Examples

```json Basic Embedding (No Chunking) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "description"
    },
    "field_passthrough": [
      { "source_path": "metadata.product_id" }
    ],
    "parameters": {}
  }
}
```

```json Sentence Chunking for RAG theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "document_content"
    },
    "field_passthrough": [
      { "source_path": "metadata.document_id" }
    ],
    "parameters": {
      "split_by": "sentences",
      "chunk_size": 5,
      "chunk_overlap": 1
    }
  }
}
```

```json Paragraph Chunking for Articles theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "article_body"
    },
    "field_passthrough": [
      { "source_path": "metadata.title" },
      { "source_path": "metadata.author" }
    ],
    "parameters": {
      "split_by": "paragraphs",
      "chunk_size": 2,
      "chunk_overlap": 0
    }
  }
}
```

```json Markdown Chunking for Docs theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "page_markdown"
    },
    "parameters": {
      "split_by": "markdown",
      "chunk_size": 300
    }
  }
}
```

```json Word-Level Chunking with Overlap theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "content"
    },
    "parameters": {
      "split_by": "words",
      "chunk_size": 300,
      "chunk_overlap": 50
    }
  }
}
```

```json LLM Extraction (Natural Language) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "review_text"
    },
    "parameters": {
      "response_shape": "Extract sentiment (positive/negative/neutral), key product features mentioned, and overall rating impression",
      "llm_provider": "openai",
      "llm_model": "gpt-4o-mini-2024-07-18"
    }
  }
}
```

```json LLM Extraction (JSON Schema) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "document"
    },
    "parameters": {
      "response_shape": {
        "type": "object",
        "properties": {
          "entities": {
            "type": "array",
            "items": { "type": "string" },
            "description": "Named entities mentioned"
          },
          "sentiment": {
            "type": "string",
            "enum": ["positive", "negative", "neutral"]
          },
          "topics": {
            "type": "array",
            "items": { "type": "string" },
            "maxItems": 5
          }
        },
        "required": ["sentiment"]
      },
      "llm_provider": "anthropic",
      "llm_model": "claude-3-5-haiku-20241022"
    }
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Embedding latency** | \~5ms per document (batched: \~2ms/doc) |
| **Query latency** | 5-10ms for top-100 results |
| **Cost** | Free (self-hosted E5-Large) |
| **GPU required** | No (but 5-10x faster with GPU) |
| **Memory** | \~4GB per 1M documents |
| **Index build** | \~1 hour per 10M documents |

**LLM extraction** adds cost and latency based on provider pricing. Only use when structured extraction is needed.

## Dense Embeddings vs Lexical (BM25)

`text_extractor` produces **dense** embeddings — great for semantic recall, weak on exact tokens. For exact-keyword precision, pair it with a **lexical (BM25)** search (the `lexical: true` option on a `feature_search` stage, backed by a `text` payload index — not a separate extractor).

| Dimension | Dense (`text_extractor`) | Lexical (BM25, `lexical: true`) |
| - | - | - |
| **Matches** | Meaning / paraphrase | Exact tokens, SKUs, codes, prices |
| **Semantic recall** | Excellent | Poor |
| **Exact matching** | Poor | Excellent |
| **Multi-language** | Excellent (E5-Large) | Good (token-based) |
| **Requires** | Embedding index | `text` payload index |

> [!TIP]
>
> The strongest setup is **hybrid**: run a dense search and a lexical search in the same `feature_search` stage and fuse with `rrf`. See [Lexical (BM25) Search](/docs/retrieval/stages/feature-search#lexical-bm25-search).

## Vector Index

| Property | Value |
| - | - |
| **Feature URI** | `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1` |
| **Index name** | `text_extractor_v1_embedding` |
| **Dimensions** | 1024 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Datatype** | float32 |
| **Inference model** | `multilingual_e5_large_instruct_v1` |
| **Normalization** | L2 normalized |

> [!NOTE]
>
> In retrievers, reference this feature by its **Feature URI** above (the output name is `multilingual_e5_large_instruct_v1`, **not** the index name `text_extractor_v1_embedding`).

### Bring your own vectors

To store vectors you computed yourself in a `text_extractor` collection, embed the way the platform does.

| Property | Value |
| - | - |
| Model | `intfloat/multilingual-e5-large-instruct` |
| Dimensions | 1024 |
| Distance | Cosine |
| Document text | Prepend `passage: ` before you embed |
| Query text | Prepend `query: ` before you embed a query vector you pass with `input_mode: vector` |

The platform L2-normalizes its own vectors. Normalize yours the same way so they compare with the managed ones.

The model and dimensions above are the current text default. Check `index.dimensions` on the collection's `vector_indexes` entry before you embed, because the [default can change](#embedding-model).

Send each vector under a key the collection accepts. [Write vectors into a managed collection](/docs/vector-store/documents#write-vectors-into-a-managed-collection) lists both.

## Limitations

* **Token limit**: 512 tokens (\~400 words). Longer text is automatically truncated.
* **Exact phrases**: Cannot reliably match exact phrases or technical terms.
* **Domain jargon**: Struggles with very domain-specific jargon or acronyms.
* **Terminology variance**: May miss documents that use different terminology for the same concept.
* **Short texts**: Less effective for very short texts (1-5 words) where lexical matching is sufficient.
* **Keyword-heavy queries**: Less effective for queries like "iPhone 15 Pro Max 256GB".

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Passthrough Extractor](/docs/processing/extractors/passthrough)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)


# Universal Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/universal

All-in-one multimodal extractor — image, video, audio, and documents — producing 3072-d Gemini embeddings plus text descriptions, OCR, and transcription

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["video_search"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/universal_extractor/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

The universal extractor is an all-in-one feature extractor that handles **image, video, audio, and documents** through Google's Gemini APIs. It produces a single 3072-dimensional embedding (Gemini Embedding 2) per object alongside rich text extraction: AI-generated descriptions, OCR for images and documents, and transcription for audio and video. It runs on the fast path, as a work-runtime task instead of a batch RayJob, so small batches skip cluster startup.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/universal\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/universal_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Resolve input** — apply `input_mappings` to get the file URL/path from the source object (`content` field).
2. **Detect modality** — classify the object as image, video, audio, or document.
3. **Segment (if needed)** — video and audio are processed in up to `max_video_segments` 30s segments; documents up to `max_document_pages` pages.
4. **Gemini embedding** — generate a 3072-d Gemini Embedding 2 vector (`output_dimensionality` configurable 256–3072).
5. **Text extraction** (if `extract_text`) — OCR for images/documents, transcription for audio/video.
6. **Description** (if `generate_description`) — Gemini vision/understanding produces a natural-language description.
7. **Output** — one document per object (or per segment/page for chunked content).

## When to Use

| Use Case | Description |
| - | - |
| **Mixed-modality corpora** | A single bucket containing images, video, audio, and PDFs you want searchable with one extractor |
| **Fast onboarding** | The fast path skips RayJob startup, so small batches return quickly |
| **Cross-modal search** | One shared 3072-d embedding space across all four modalities |
| **Rich metadata** | Need descriptions, OCR text, and transcription alongside the vector |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| High-volume single-modality at lowest cost | Modality-specific extractor (`text_extractor`, `image_extractor`) |
| Audio fingerprinting / sound-mark matching | `audio_fingerprint_extractor` |
| Spatial/layout document analysis | `document_graph_extractor` |
| Self-hosted, no external API calls | `text_extractor` / `image_extractor` |

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `content` | string | **Yes** | URL or path to the file to process. Populated from `input_mappings`. |

```json theme={null}
{
  "content": "s3://my-bucket/assets/report.pdf"
}
```

Supported input types: **IMAGE, VIDEO, AUDIO, PDF, TEXT, STRING**.

## Output Schema

| Field | Type | Description |
| - | - | - |
| `universal_extractor_v1_embedding` | float\[3072] | Gemini Embedding 2 vector for the content |
| `modality` | string | Detected modality: `image`, `video`, `audio`, or `document` |
| `text` | string \| null | Extracted text (OCR, transcription, or document text) |
| `description` | string \| null | AI-generated description of the content |
| `segment_index` | integer \| null | Segment index (chunked video/audio/documents) |
| `segment_total` | integer \| null | Total segments for this source object |
| `page_number` | integer \| null | Page number (documents only) |
| `start_time_s` / `end_time_s` | float \| null | Segment start/end time in seconds (video/audio) |
| `start_time` / `end_time` | float \| null | The same segment times under the name the other video extractors use (video/audio) |
| `duration_s` | float \| null | Total file duration in seconds (video/audio) |

```json theme={null}
{
  "universal_extractor_v1_embedding": [0.012, -0.034, 0.008, ...],
  "modality": "document",
  "text": "Quarterly revenue grew 12% year over year...",
  "description": "A financial report page with a revenue bar chart",
  "page_number": 1,
  "segment_total": 12
}
```

## Output by Modality

The extractor writes one document per unit: an image is one document, video and audio split into 30-second segments, and a document splits into pages. Every document carries `universal_extractor_v1_embedding`, `modality`, `text`, `description`, `segment_index`, and `segment_total`. The table lists the fields each modality adds and what fills `text` and `description`.

| Modality | Documents per object | Extra fields | `text` holds | `description` holds |
| - | - | - | - | - |
| `image` | 1 | none | The description and the OCR text, joined by a newline | Gemini description of the image |
| `video` | One per 30-second segment, up to `max_video_segments` | `start_time_s`, `end_time_s`, `start_time`, `end_time`, `duration_s`, `covered_s` | The segment description and the speech transcript, joined by a newline | Gemini description of the segment |
| `audio` | One per 30-second segment, up to `max_video_segments` | `start_time_s`, `end_time_s`, `start_time`, `end_time`, `duration_s`, `covered_s` | The transcript of the segment | Always null |
| `document` | One per page, up to `max_document_pages` | `page_number` | The PDF text layer, or Gemini text extraction for a page without one | A one-sentence summary on page 1, null on every other page |

`covered_s` is the number of seconds the indexed segments span, capped by `duration_s`. It shows when `max_video_segments` cut a long file short.

Two parameters change the fields. `generate_description: false` sets `description` to null. `extract_text: false` leaves OCR text, the transcript, and page text out of `text`.

If no segment of a video or audio file produces a document, the extractor writes one document for the whole file. That document has a null `text` and no `start_time_s` or `end_time_s`.

## What the Extractor Does Not Emit

The output has no field for a hook segment, a music or tone descriptor, keyframes, scene boundaries, or faces. Video segments are fixed 30-second windows that start at 0. The first segment (`segment_index` 0) spans the first 30 seconds of the file and cannot be narrowed to the first 3. The video `description` is free text, so no music or tone value is searchable as its own field.

To get these fields, use one of two paths:

* **Face data:** the [Face Identity Extractor](/docs/processing/extractors/face-identity) detects and embeds faces.
* **Hook, music, or tone fields:** build a [custom extractor](/docs/processing/custom-extractors) that writes them. Upload and deploy run on dedicated infrastructure. On the shared API, submit the extractor through [Extractor Submissions](/docs/processing/extractor-marketplace).

## Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `output_dimensionality` | integer | `3072` | 256–3072 | Output embedding dimensions (Gemini Embedding 2 supports 256–3072) |
| `task_type` | string | `"RETRIEVAL_DOCUMENT"` | — | Embedding intent for Gemini Embedding 2. Common values: `RETRIEVAL_DOCUMENT`, `RETRIEVAL_QUERY`, `SEMANTIC_SIMILARITY` |
| `generate_description` | boolean | `true` | — | Generate a text description via Gemini vision/understanding |
| `extract_text` | boolean | `true` | — | Extract text (OCR for images/docs, transcription for audio/video) |
| `max_video_segments` | integer | `10` | 1–50 | Maximum number of 30s segments to process for video and audio files |
| `max_document_pages` | integer | `50` | 1–200 | Maximum number of pages to process for document files |
| `max_file_download_mb` | integer | `500` | 1–1024 | Maximum file download size (MB) for fast-path processing |
| `max_concurrency` | integer | `4` | 1–32 | Maximum per-task object concurrency for fast-path processing |

> [!WARNING]
>
> Dimensions are locked at namespace creation. Switching `output_dimensionality` on an existing namespace requires a migration since the vector index dimensionality is fixed.

## Configuration Examples

```json All-in-One Defaults theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "universal_extractor",
    "version": "v1",
    "input_mappings": {
      "content": "file_url"
    },
    "parameters": {}
  }
}
```

```json Embeddings Only (No Text/Description) theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "universal_extractor",
    "version": "v1",
    "input_mappings": {
      "content": "file_url"
    },
    "parameters": {
      "generate_description": false,
      "extract_text": false
    }
  }
}
```

```json Compact Embeddings for Clustering theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "universal_extractor",
    "version": "v1",
    "input_mappings": {
      "content": "file_url"
    },
    "parameters": {
      "output_dimensionality": 768,
      "task_type": "CLUSTERING"
    }
  }
}
```

```json Long-Form Video & Documents theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "universal_extractor",
    "version": "v1",
    "input_mappings": {
      "content": "file_url"
    },
    "parameters": {
      "max_video_segments": 30,
      "max_document_pages": 200
    }
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Compute** | Fast path (a work-runtime task, no RayJob startup) |
| **Cost** | See [Billing & Pricing](/docs/platform/billing) — rates come from `GET /v1/billing/pricing`. One charge covers all Gemini API calls (embedding, description, OCR/transcription) |
| **External API** | Google Gemini (embedding + vision/understanding) |
| **Max download** | 500 MB per object (configurable to 1024 MB) |

## Vector Index

| Property | Value |
| - | - |
| **Index name** | `universal_extractor_v1_embedding` |
| **Dimensions** | 3072 (configurable 256–3072) |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Inference model** | `google/gemini-embedding-2` |

## Limitations

* **External dependency**: Requires Google Gemini API availability; subject to its rate limits.
* **Per-object cost**: Higher per-object cost than self-hosted single-modality extractors.
* **Segment/page caps**: Video beyond `max_video_segments` and documents beyond `max_document_pages` are truncated.
* **Download ceiling**: Files larger than `max_file_download_mb` are skipped on the fast path.

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)
* [Gemini Multifile Extractor](/docs/processing/extractors/gemini-multifile)
* [Audio Fingerprint Extractor](/docs/processing/extractors/audio-fingerprint)


# Video Transcode Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/video-transcode

Convert RED R3D, ARRI RAW and mixed-codec video to a web-playable H.264 or H.265 MP4 mezzanine in object storage

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["video_transcode"]` as a video add-on. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

The transcoding extractor converts source video to a web-playable MP4 in your object
storage. It picks a decode path per file:

1. **RED R3D** through the REDline CLI.
2. **ARRI RAW** (`.ari`, `.arriraw`, `.arx`, or ARRI codecs inside MXF and MOV
   containers) through ART CMD.
3. **Everything else** through FFmpeg.

Containers are codec-probed with `ffprobe` before routing, so a misleading file
extension does not send a file down the wrong path.

Use it to make RAW camera archives ingestable and playable, and to normalize a
mixed-codec library to one mezzanine format. It does not produce scenes or embeddings.
Chain a collection with a video or multimodal extractor on top of the converted output
for those.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/transcoding\_extractor\_v1](https://api.mixpeek.com/v1/collections/features/extractors/transcoding_extractor_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Create the collection

```json theme={null}
{
  "collection_name": "playable-masters",
  "source": { "type": "bucket", "bucket_ids": ["bkt_raw_footage"] },
  "features": ["video_transcode"]
}
```

`video_transcode` is a companion add-on, so enabling it creates a companion collection
alongside the base video collection rather than adding a field to it.

## Parameters

Every parameter has a default, so `parameters` is optional.

| Parameter | Default | Values |
| - | - | - |
| `output_codec` | `libx264` | `libx264` for player compatibility, `libx265` for smaller files. |
| `crf` | `23` | 0 to 51. Lower is higher quality and larger. 18 to 23 is the usual H.264 range. |
| `preset` | `medium` | `ultrafast` through `veryslow`. Slower compresses better and costs more CPU. |
| `max_resolution` | `1080` | Caps output height in pixels; width scales with it. Set `null` to keep the source resolution. |
| `audio_codec` | `aac` | `aac`, `mp3`, or `copy` to pass the source audio through. |
| `require_native_sdks` | `false` | See below. |

> [!WARNING]
>
> **`require_native_sdks` decides what a RAW file does when the vendor SDK is absent.**
> RED and ARRI decoding need the REDline and ART CMD binaries on the engine image. When
> they are missing, the default `false` falls back to FFmpeg and produces a file
> anyway, which for a RAW source can mean wrong color or a failed decode you only notice
> downstream.
>
> Set `require_native_sdks: true` for RED or ARRI sources. The file fails loudly instead
> of degrading quietly, which is the outcome you want when the alternative is a
> silently wrong master.

To set parameters, declare the extractor explicitly. Register it on the namespace
first, then map the video field on the collection:

```bash theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/namespaces/$NAMESPACE_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "feature_extractors": [
      { "feature_extractor_name": "transcoding_extractor", "version": "v1" }
    ]
  }'
```

`feature_extractors` appends, so existing extractors on the namespace are kept.

Then on the collection:

```json theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "transcoding_extractor",
    "version": "v1",
    "input_mappings": { "video": "video" },
    "parameters": {
      "output_codec": "libx264",
      "crf": 20,
      "max_resolution": null,
      "require_native_sdks": true
    }
  }
}
```

## Output

One document per source object:

| Field | Meaning |
| - | - |
| `converted_url` | Object-storage URL of the transcoded MP4. |
| `source_format` | `r3d`, `arri`, or `standard`. |
| `width`, `height` | Output dimensions in pixels. |
| `duration_sec` | Output duration. |
| `converted_size_bytes` | Size of the converted file. |

Accepted inputs are R3D, ARI, ARRIRAW, ARX, MXF, MP4, MOV, AVI, MKV, WebM, and anything
else FFmpeg decodes.

## Pricing

`video_transcode` bills at **\$0.015 per minute** of video as a video add-on. Rates on
this page can change; the [billing page](/docs/platform/billing) and
`GET /v1/billing/pricing` are the live source.


# Web Scraper Extractor
Source: https://docs.mixpeek.com/docs/processing/extractors/web-scraper

Recursive website crawling with multimodal content extraction and semantic embeddings

> [!WARNING]
>
> Built-in extractor names are a **deprecated alias** — collections are now created by picking [features](/docs/processing/features). This pipeline is selected with `features: ["web_crawl"]`. Existing `feature_extractor` configs keep working; see the [migration guide](/docs/processing/extractor-migration).

> **[View on GitHub](https://github.com/mixpeek/mixpeek-extractors/blob/main/extractors/web_scraper/README.md)**
>
> Runnable reference for this extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry.

![Web scraper extractor pipeline showing crawling, content extraction, chunking, and multimodal embeddings](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/extractors/web-scraper.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=adb2204d2ca53dc092437f8866dec326)

The web scraper extractor recursively crawls websites to extract multimodal content with semantic embeddings. Automatically discovers and extracts text, code blocks, images, and asset links from web pages. Each extracted document receives E5-Large text embeddings (1024D) for semantic search, Jina Code embeddings (768D) for code snippets, and optional SigLIP visual embeddings (768D) for images. Supports JavaScript-rendered SPAs, includes resilience features like retry logic, proxy rotation, and captcha detection.

> [!NOTE]
>
> View extractor details at [api.mixpeek.com/v1/collections/features/extractors/web\_scraper\_v1](https://api.mixpeek.com/v1/collections/features/extractors/web_scraper_v1) or fetch programmatically with `GET /v1/collections/features/extractors/{feature_extractor_id}`.

## Pipeline Steps

1. **Filter Dataset** (if collection\_id provided)
   * Filter to specified collection
2. **Crawl Configuration & Setup**
   * Parse seed URL and configure crawl parameters
   * Set up URL filtering rules, rendering strategy, resilience options
3. **Recursive Web Crawling**
   * BFS-based link traversal with depth limit
   * JavaScript rendering support (auto-detect or explicit)
   * URL filtering (include/exclude patterns)
   * Resilience: retry logic, proxy rotation, captcha detection
4. **Content Extraction Per Page**
   * Extract text content, title, metadata
   * Identify and extract code blocks with language detection
   * Discover images with alt text, dimensions
   * Find asset links (PDFs, documents, archives)
   * Optional: Structured extraction via LLM (`response_shape`)
5. **Content Chunking** (optional)
   * Split page content by strategy: sentences, paragraphs, words, characters
   * Configurable chunk size and overlap
   * Track chunk metadata for joined results
6. **Document Expansion**
   * Create separate documents for page content, each code block, each image
   * Preserve parent URL and crawl depth metadata
7. **Multi-Modal Embedding Generation**
   * E5-Large (1024D) for page text content
   * Jina Code (768D) for code blocks
   * SigLIP (768D) for images (if `generate_image_embeddings=true`)
8. **Output**
   * Documents with text content, code blocks, images
   * Asset links discovered but not crawled
   * Multiple embeddings per document for hybrid search

## When to Use

| Use Case | Description |
| - | - |
| **API documentation** | Index technical documentation with code examples and diagrams |
| **Knowledge base crawling** | Extract FAQs, guides, and tutorials from support sites |
| **Job board scraping** | Find job listings with parsed content and structured fields |
| **News aggregation** | Collect and index articles with multimodal content |
| **Competitive analysis** | Monitor competitor websites for content changes |
| **Open source docs** | Index project documentation from GitHub Pages, ReadTheDocs |
| **Product research** | Gather product information from multiple websites |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Protected/authenticated content | Configure via `custom_headers` with auth tokens |
| PDF-only extraction | `document_graph_extractor` (better OCR, layout detection) |
| Social media scraping | Use platform-specific APIs (Twitter API, Instagram Graph API) |
| E-commerce product catalogs | Use platform APIs when available (better data structure) |
| Very large sites (10K+ pages) | Increase `max_pages`, implement crawl goal filtering |

## Input Schema

| Field | Type | Required | Description |
| - | - | - | - |
| `url` | string | **Yes** | Seed URL to start crawling from. Example: `https://docs.example.com/api/` |

```json theme={null}
{
  "url": "https://docs.example.com/getting-started"
}
```

**Input Examples:**

| Type | Example |
| - | - |
| API documentation | `https://docs.openai.com/api/` |
| Knowledge base | `https://help.example.com/` |
| Blog | `https://blog.example.com/` |
| Job board | `https://jobs.example.com/listings` |

## Output Schema

Each crawled page produces one or more documents depending on content extraction and expansion settings:

| Field | Type | Description |
| - | - | - |
| `content` | string | Extracted text content from page |
| `title` | string | Page title (from `<title>` tag or heading) |
| `page_url` | string | Full URL of crawled page |
| `code_blocks` | array | Code blocks found on page (structure: `[{language, code, line_start, line_end}]`) |
| `images` | array | Images found on page (structure: `[{src, alt, title, width, height}]`) |
| `asset_links` | array | Downloadable assets discovered (structure: `[{url, file_type, link_text, file_extension}]`) |
| `chunk_index` | integer | Position within page chunks (if chunking enabled) |
| `total_chunks` | integer | Total chunks from this page (if chunking enabled) |
| `crawl_depth` | integer | Depth from seed URL (0 = seed, 1 = links from seed, etc.) |
| `parent_url` | string | Referrer URL (previous page in crawl path) |
| `intfloat__multilingual_e5_large_instruct` | float\[1024] | E5-Large text embedding, L2 normalized |
| `jinaai__jina_embeddings_v2_base_code` | float\[768] | Jina Code embedding (if code blocks extracted) |
| `google__siglip_base_patch16_224` | float\[768] | SigLIP visual embedding (if `generate_image_embeddings=true`) |

```json theme={null}
{
  "content": "The REST API provides endpoints for creating, reading, updating, and deleting resources...",
  "title": "REST API Overview - Example Docs",
  "page_url": "https://docs.example.com/api/overview",
  "code_blocks": [
    {
      "language": "python",
      "code": "import requests\nresponse = requests.get('https://api.example.com/users')",
      "line_start": 1,
      "line_end": 2
    }
  ],
  "images": [
    {
      "src": "https://docs.example.com/images/api-flow.png",
      "alt": "API request flow diagram",
      "width": 800,
      "height": 600
    }
  ],
  "asset_links": [
    {
      "url": "https://docs.example.com/downloads/openapi.yaml",
      "file_type": "openapi",
      "link_text": "Download OpenAPI Spec",
      "file_extension": "yaml"
    }
  ],
  "crawl_depth": 2,
  "parent_url": "https://docs.example.com/api/",
  "intfloat__multilingual_e5_large_instruct": [0.023, -0.041, 0.018, ...],
  "jinaai__jina_embeddings_v2_base_code": [0.045, -0.023, ...],
  "google__siglip_base_patch16_224": [0.078, -0.091, ...]
}
```

## Parameters

### Crawl Configuration Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `max_depth` | integer | 2 | 0-1000 | Maximum link depth from seed (0 = seed URL only, higher = deeper crawl) |
| `max_pages` | integer | 50 | 1-1000000 | Maximum pages to crawl in single run |
| `crawl_timeout` | integer | 300 | 10-3600 | Maximum time for crawl in seconds (10s - 1h) |
| `crawl_mode` | enum | `"deterministic"` | deterministic, semantic | BFS deterministic or LLM-guided semantic crawling |
| `crawl_goal` | string | null | - | Goal for semantic crawling (e.g., "find all API endpoints"). Used with `crawl_mode: semantic` |

### Rendering Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `render_strategy` | enum | `"auto"` | Rendering method: `static` (HTML only), `javascript` (Puppeteer), `auto` (auto-detect) |

### URL Filtering Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `include_patterns` | array | null | Regex patterns for URLs to include (whitelist). Example: `["/docs/.*", "/api/.*"]` |
| `exclude_patterns` | array | null | Regex patterns for URLs to exclude (blacklist). Example: `["/admin/.*", ".*logout.*"]` |

### Content Chunking Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `chunk_strategy` | enum | `"none"` | none, sentences, paragraphs, words, characters | How to split page content |
| `chunk_size` | integer | 500 | 1-10000 | Target size per chunk (units depend on strategy) |
| `chunk_overlap` | integer | 50 | 0-5000 | Overlap between consecutive chunks |

### Document Identity Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `document_id_strategy` | enum | `"url"` | How to generate document IDs: `url` (unique per page), `position` (sequential), `content` (hash-based) |

### Embedding Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `generate_text_embeddings` | boolean | true | Generate E5-Large text embeddings for page content |
| `generate_code_embeddings` | boolean | true | Generate Jina Code embeddings for code blocks |
| `generate_image_embeddings` | boolean | true | Generate SigLIP embeddings for discovered images |

### LLM Structured Extraction Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `response_shape` | string or object | null | Define structured extraction: natural language description or JSON schema |
| `llm_provider` | string | null | LLM provider: `openai`, `google`, `anthropic` (required if using `response_shape`) |
| `llm_model` | string | null | Specific LLM model (e.g., `gpt-4o-mini`, `gemini-2.5-flash`) |
| `llm_api_key` | string | null | API key (supports secret vault references like `${vault:openai-key}`) |

### Resilience: Retry Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `max_retries` | integer | 3 | 0-10 | Maximum retry attempts on request failure |
| `retry_base_delay` | number | 1.0 | 0.1-30.0 | Base delay for exponential backoff (seconds) |
| `retry_max_delay` | number | 30.0 | 1.0-300.0 | Maximum delay between retries (seconds) |
| `respect_retry_after` | boolean | true | - | Respect `Retry-After` header from server |

### Resilience: Proxy Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `proxies` | array | null | Proxy URLs for rotation. Example: `["http://proxy1:8080", "http://proxy2:8080"]` |
| `rotate_proxy_on_error` | boolean | true | Rotate proxy when request fails |
| `rotate_proxy_every_n_requests` | integer | 0 | Rotate proxy every N requests (0 = no periodic rotation) |

### Resilience: Captcha Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `captcha_service_provider` | string | null | Captcha solving service: `2captcha`, `anti-captcha`, `capsolver` |
| `captcha_service_api_key` | string | null | API key for captcha service (supports secret vault references) |
| `detect_captcha` | boolean | true | Auto-detect captcha challenges and attempt to solve |

### Resilience: Session Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `persist_cookies` | boolean | true | Persist cookies across requests within single crawl |
| `custom_headers` | object | null | Custom HTTP headers. Example: `{"Authorization": "Bearer token", "User-Agent": "Custom"}` |

### Politeness Parameters

| Parameter | Type | Default | Range | Description |
| - | - | - | - | - |
| `delay_between_requests` | number | 0.0 | 0.0-60.0 | Delay between consecutive requests (seconds) |

## Configuration Examples

```json Basic Documentation Crawl theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "web_scraper",
    "version": "v1",
    "input_mappings": {
      "url": "docs_url"
    },
    "field_passthrough": [
      { "source_path": "metadata.vendor" },
      { "source_path": "metadata.product" }
    ],
    "parameters": {
      "max_depth": 2,
      "max_pages": 50,
      "crawl_timeout": 300,
      "render_strategy": "auto",
      "generate_text_embeddings": true,
      "generate_code_embeddings": true,
      "generate_image_embeddings": false,
      "delay_between_requests": 0.5
    }
  }
}
```

```json API Docs with Structured Extraction theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "web_scraper",
    "version": "v1",
    "input_mappings": {
      "url": "api_docs_url"
    },
    "parameters": {
      "max_depth": 3,
      "max_pages": 100,
      "include_patterns": ["/api/.*", "/reference/.*"],
      "exclude_patterns": ["/changelog/.*", "/deprecated/.*"],
      "render_strategy": "javascript",
      "response_shape": {
        "type": "object",
        "properties": {
          "endpoints": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "method": { "type": "string" },
                "path": { "type": "string" },
                "description": { "type": "string" },
                "parameters": { "type": "array", "items": { "type": "string" } }
              }
            }
          },
          "authentication": { "type": "string" },
          "rate_limits": { "type": "string" }
        }
      },
      "llm_provider": "openai",
      "llm_model": "gpt-4o-mini",
      "generate_text_embeddings": true,
      "generate_code_embeddings": true,
      "generate_image_embeddings": true
    }
  }
}
```

```json Knowledge Base with Semantic Crawling theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "web_scraper",
    "version": "v1",
    "input_mappings": {
      "url": "kb_url"
    },
    "parameters": {
      "max_depth": 4,
      "max_pages": 200,
      "crawl_mode": "semantic",
      "crawl_goal": "Find all articles related to troubleshooting and error resolution",
      "chunk_strategy": "paragraphs",
      "chunk_size": 1000,
      "chunk_overlap": 100,
      "generate_text_embeddings": true,
      "generate_code_embeddings": false,
      "generate_image_embeddings": false
    }
  }
}
```

```json Job Board with Resilience theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "web_scraper",
    "version": "v1",
    "input_mappings": {
      "url": "job_board_url"
    },
    "parameters": {
      "max_depth": 2,
      "max_pages": 500,
      "render_strategy": "javascript",
      "max_retries": 5,
      "retry_base_delay": 2.0,
      "retry_max_delay": 60.0,
      "proxies": ["http://proxy1:8080", "http://proxy2:8080"],
      "rotate_proxy_on_error": true,
      "rotate_proxy_every_n_requests": 10,
      "persist_cookies": true,
      "delay_between_requests": 1.0,
      "captcha_service_provider": "2captcha",
      "captcha_service_api_key": "${vault:captcha-api-key}",
      "generate_text_embeddings": true,
      "generate_code_embeddings": false
    }
  }
}
```

```json High-Volume Crawl with Filtering theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "web_scraper",
    "version": "v1",
    "input_mappings": {
      "url": "website_url"
    },
    "parameters": {
      "max_depth": 5,
      "max_pages": 1000,
      "crawl_timeout": 3600,
      "include_patterns": ["^https://example\\.com/.*"],
      "exclude_patterns": [".*login.*", ".*admin.*", ".*/search\\?.*"],
      "chunk_strategy": "sentences",
      "chunk_size": 500,
      "chunk_overlap": 50,
      "document_id_strategy": "url",
      "generate_text_embeddings": true,
      "generate_code_embeddings": true,
      "generate_image_embeddings": false,
      "max_retries": 3,
      "delay_between_requests": 0.2
    }
  }
}
```

```json Premium: Full Featured theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "web_scraper",
    "version": "v1",
    "input_mappings": {
      "url": "target_url"
    },
    "parameters": {
      "max_depth": 3,
      "max_pages": 250,
      "crawl_timeout": 1800,
      "crawl_mode": "semantic",
      "crawl_goal": "Find all technical content, code examples, and API documentation",
      "render_strategy": "javascript",
      "include_patterns": ["/docs/.*", "/api/.*", "/guide/.*"],
      "exclude_patterns": ["/legacy/.*"],
      "chunk_strategy": "paragraphs",
      "chunk_size": 1000,
      "chunk_overlap": 100,
      "response_shape": "Extract key topics, code language, and frameworks mentioned",
      "llm_provider": "anthropic",
      "llm_model": "claude-3-5-haiku-20241022",
      "generate_text_embeddings": true,
      "generate_code_embeddings": true,
      "generate_image_embeddings": true,
      "max_retries": 5,
      "retry_base_delay": 1.5,
      "proxies": ["http://proxy1:8080"],
      "rotate_proxy_every_n_requests": 5,
      "persist_cookies": true,
      "custom_headers": {
        "User-Agent": "Mozilla/5.0 (compatible; MixpeekBot/1.0)"
      },
      "delay_between_requests": 0.5
    }
  }
}
```

## Performance & Costs

| Metric | Value |
| - | - |
| **Average page load** | 2-5 seconds (depends on page complexity and rendering) |
| **Pages per minute** | 12-30 pages (with delays and retries) |
| **Code block extraction** | \~10ms per 1KB of code |
| **Image extraction** | \~50ms per 10 images |
| **Embedding latency** | \~5ms per text page (E5), \~10ms per code block (Jina), \~50ms per image (SigLIP) |
| **Cost** | Billed per crawled page — see [Billing & Pricing](/docs/platform/billing); rates come from `GET /v1/billing/pricing` |
| **Memory usage** | \~100MB base + \~1MB per 100 pages in crawl queue |

## Vector Indexes

All three embeddings are stored as [MVS](https://mixpeek.com/mvs) named vectors for hybrid search:

| Property | Value |
| - | - |
| **Index 1 name** | `intfloat__multilingual_e5_large_instruct` |
| **Dimensions** | 1024 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Datatype** | float32 |
| **Normalization** | L2 normalized |

| Property | Value |
| - | - |
| **Index 2 name** | `jinaai__jina_embeddings_v2_base_code` |
| **Dimensions** | 768 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Datatype** | float32 |
| **Normalization** | L2 normalized |

| Property | Value |
| - | - |
| **Index 3 name** | `google__siglip_base_patch16_224` |
| **Dimensions** | 768 |
| **Type** | Dense |
| **Distance metric** | Cosine |
| **Datatype** | float32 |
| **Status** | Optional (if `generate_image_embeddings=true`) |

## Comparison with Other Extractors

| Feature | web\_scraper | text\_extractor | multimodal\_extractor | document\_graph\_extractor |
| - | - | - | - | - |
| **Input types** | URLs (crawling) | Text only | Video, Image, Text | PDF only |
| **Recursive crawling** | ✅ Yes | ✗ | ✗ | ✗ |
| **Code extraction** | ✅ Yes | ✗ | ✗ | ✗ |
| **Image extraction** | ✅ Yes | ✗ | ✅ Yes | ✗ |
| **Multimodal embeddings** | ✅ Yes | Text only | ✅ Yes | Text only |
| **LLM extraction** | ✅ Yes | ✅ Yes | ✗ | ✗ |
| **Resilience features** | ✅ Yes | ✗ | ✗ | ✗ |
| **Best for** | Web crawling | Text search | Video/image/text | PDF analysis |
| **Cost** | Per crawled page — see [Billing & Pricing](/docs/platform/billing) | Per token | Per video minute | Per page |

## Resilience & Robustness

The web scraper includes enterprise-grade resilience features:

### Retry Strategy

* Exponential backoff with configurable base and max delays
* Respects server `Retry-After` headers
* Retries on network errors, timeouts, and temporary failures (5xx)

### Proxy Rotation

* Support for multiple proxies with automatic rotation
* Rotate on error or periodic rotation every N requests
* Helps avoid rate limiting and IP bans

### Captcha Detection & Solving

* Auto-detect common captcha types (reCAPTCHA, hCaptcha)
* Integration with 2captcha, Anti-Captcha, CapSolver services
* Fallback to manual review if solving fails

### Session Management

* Persistent cookies across requests within a single crawl
* Custom HTTP headers for authentication
* Support for API key and bearer token injection

### URL Filtering

* Include patterns (whitelist): Only crawl matching URLs
* Exclude patterns (blacklist): Skip URLs matching patterns
* Prevent crawling auth/admin pages, search results, etc.

## Limitations

* **Content-only crawling**: Does not execute custom JavaScript actions (clicking, form submission, scrolling)
* **Authentication**: Limited to HTTP headers (Bearer tokens, API keys). No interactive login flows.
* **Dynamic content**: JavaScript-rendering adds 2-3x latency per page
* **Large sites**: 10K+ page sites may require high `max_pages` and long timeouts
* **Robots.txt**: Does not parse `robots.txt`; respect via `delay_between_requests` and `max_pages`
* **Rate limiting**: May be blocked by aggressive rate limiting; use proxies and delays

## Related

* [Feature Extractors Overview](/docs/processing/feature-extractors)
* [Text Extractor](/docs/processing/extractors/text)
* [Document Graph Extractor](/docs/processing/extractors/document)
* [Image Extractor](/docs/processing/extractors/image)
* [Multimodal Extractor](/docs/processing/extractors/multimodal)


# Pipeline Configuration (Advanced)
Source: https://docs.mixpeek.com/docs/processing/feature-extractors

Advanced knobs behind a collection's processing pipeline — input mappings, field passthrough, parameters, and feature URIs

> [!NOTE]
>
> **Start with [Features](/docs/processing/features).** `features: ["image_search"]` is the way to create collections — the platform resolves the pipeline and defaults the wiring. This page covers the advanced configuration underneath: explicit input mappings, field passthrough, pipeline parameters, and the feature URIs that retrievers query. The `feature_extractor` config object shown here still works everywhere but is a **deprecated alias** — see the [migration guide](/docs/processing/extractor-migration).

![Processing pipeline turning objects into reusable features](https://mintcdn.com/mixpeek/o18CfVBlvyb5Tvo5/assets/mixpeek-feature-extractors.svg?fit=max\&auto=format\&n=o18CfVBlvyb5Tvo5\&q=85\&s=29ef229426dfbed63acc16e7cad9cec6)

Every collection runs one processing pipeline: Ray-powered workflows that read objects, run ML models, and write features into collection documents. Each pipeline output exposes a stable **feature URI** so retrievers, taxonomies, and clusters can reference it with confidence. When you create a collection with `features: [...]`, Mixpeek picks the pipeline, pins its version, and defaults its inputs; the knobs below are for when you need explicit control.

## Anatomy of an explicit pipeline config

The explicit config is attached via the `feature_extractor` field — mutually exclusive with `features` (provide one or the other):

```json theme={null}
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "product_text"
    },
    "field_passthrough": [
      { "source_path": "metadata.category" },
      { "source_path": "metadata.brand" }
    ],
    "parameters": {
      "embedding_model": "multilingual_e5_large_instruct_v1"
    }
  }
}
```

Four knobs control what the pipeline sees and produces.

### `feature_extractor_name` + `version`

Identifies the pipeline and pins its version per collection — a new `v2` ships without disturbing collections on `v1`. You roll forward deliberately via a [namespace migration](/docs/api-reference/namespace-migrations/list-migrations), not in place. Built-in names are accepted as deprecated aliases of their [feature](/docs/processing/extractor-migration); custom plugin names are yours and stay first-class.

### `input_mappings` — bind pipeline inputs to object fields

Every pipeline declares an `input_schema` (e.g., the text pipeline expects a `text` input). `input_mappings` bind those named inputs to fields in your bucket schema: `product_text`, `video_url`, etc. Use **flat field names** directly, not prefixed paths like `payload.product_text`.

On the `features: [...]` path these default to the standard `uploads` bucket properties (`image`, `video`, `audio`, `pdf`, `content`, `url`) — you only need explicit mappings for custom bucket schemas.

### `warnings` on the create response

Collection create validates `input_mappings` against the bucket **schema**. The batch runs against the bucket's **objects**. A sync with no `schema_mapping` writes its blob to the default property, so a field can be declared on the schema and absent from every object. Both surfaces look correct and the collection produces nothing.

When the create detects this it returns the collection with a `warnings` array:

```json theme={null}
{
  "collection_id": "col_...",
  "warnings": [
    "input_mappings names ['segment'], which the bucket schema declares but the objects in this bucket do not carry. A sampled object carries ['content'] instead. This collection will process nothing..."
  ]
}
```

The collection is created either way. `warnings` is `null` when there is nothing to report.

Fix it on the sync, not the collection: add a `schema_mapping` whose blob entry sets `blob_property` to the field you mapped.

> [!NOTE]
>
> The check samples one object and stays quiet on an empty bucket, since creating a collection before the first upload is the normal order. A bucket that passes today can still receive mismatched objects later, so a missing warning is not a guarantee.

### `field_passthrough` — carry source fields into documents

Pipeline outputs alone are rarely enough at query time. You want to filter by `metadata.category` or `status` without re-joining against the source bucket. `field_passthrough` copies selected source fields onto every output document so retrievers can use them directly in metadata filters.

> [!TIP]
>
> If you plan to filter by a field at query time, pass it through at ingestion time. Reaching back into the source bucket from a retriever stage is 100–1000x slower than a local metadata predicate.

### `parameters` — tune the pipeline

Parameters are pipeline-specific: chunk strategy, embedding model variant, OCR enable/disable, transcription language, thumbnail quality. Invalid parameters return a teaching `422` that includes the pipeline's parameter schema (also available via `GET /v1/collections/features/extractors`).

## The output schema

When you configure a collection, it immediately calculates an `output_schema` that merges passthrough fields with pipeline outputs. You can inspect this before any processing runs:

```bash theme={null}
curl "https://api.mixpeek.com/v1/collections/{collection_id}/features" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

The response lists every feature URI the collection will produce, its type, its dimensions (for embeddings), and its description. Use this to validate downstream retrievers, taxonomies, and clusters before you ingest a single object.

## Feature URIs: the stable contract

Every pipeline output publishes a URI in the form `mixpeek://{name}@{version}/{output}`:

* `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1`
* `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding`
* `mixpeek://universal_extractor@v1/gemini-embedding-2`

The URI is the contract between ingestion and everything downstream:

* **Retrievers** reference features by URI in `feature_search` stages.
* **Taxonomies** classify against a feature URI.
* **Clusters** group documents by a feature URI.
* **Migrations** swap one URI for another across retrievers atomically.

Because the URI pins the pipeline version, a retriever built against `v1` keeps working even after `v2` is released — it simply doesn't see `v2` features unless you explicitly migrate. URIs embed internal pipeline names; that's expected — they're version-pinned implementation references, not config vocabulary. Discover the URIs your collection actually produces with `GET /v1/collections/{id}/features` rather than constructing them by hand.

> [!TIP]
>
> Treat feature URIs like API versions: pin them in production retrievers, and roll them forward deliberately rather than in-place.

## Vector index registration

When a pipeline publishes an embedding, Mixpeek registers a vector index in [MVS](/docs/vector-store/overview) at the matching URI. Retrievers then query that exact index via `feature_search`.

For built-in pipelines this is automatic. For [custom extractors](/docs/processing/custom-extractors), it's the single biggest pitfall: the `features` list in `manifest.py` must use the exact keys `feature_type`, `feature_name`, `embedding_dim`, `distance_metric`. Intuitive-but-wrong names (`type`, `name`, `dimensions`, `distance`) silently create a collection with **zero vector indexes**, and the ingestion task reports `COMPLETED` with 0 documents written.

## What happens at runtime

For a single-pipeline collection, the end-to-end path is:

1. **Flatten.** API flattens the manifest into per-pipeline row artifacts (Parquet) and stores them in S3.
2. **Schedule.** Ray poller discovers pending batches and submits a job.
3. **Run.** Workers load the dataset, run the pipeline flow (GPU if available), and emit features plus passthrough fields.
4. **Write.** `MVSBatchProcessor` writes vectors and payloads to MVS, emits webhook events, and updates index signatures.

Every document records lineage metadata so you can trace any feature back to the object that produced it:

```json theme={null}
{
  "root_object_id": "obj_123",
  "source_collection_id": "col_source",
  "processing_tier": 1,
  "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
}
```

When pipelines are chained across collections, the `processing_tier` field becomes load-bearing — see [Multi-Tier Feature Extraction](/docs/processing/multi-tier-extractors) for how the DAG executes.

## Performance and scaling

* **GPU workers** deliver 5–10x faster throughput for embeddings, reranking, and video processing. CPU-only pipelines skip GPU allocation (saves \~3 minutes of cluster startup and \~6x on cost).
* **Ray Data** handles batching, shuffling, and parallelization automatically. Default batch size is 64 rows; tune via the pipeline's compute profile.
* **Autoscaling** maintains target utilization (`0.7` CPU, `0.8` GPU by default).
* **Inference cache** short-circuits repeated model calls when inputs hash to the same key — handy when reprocessing or when ingesting near-duplicates.
* **Normalization** happens once at ingest (720p video mezzanine, capped image edges, 16kHz mono audio) — it bounds per-unit cost and is included in the [modality rates](/docs/platform/billing). Opt out per collection with `full_res: true` (2x multiplier).

## List Available Extractors

Four routes list extractors. They read the built-in extractors from the same registry. They differ in the response envelope, the filters, and the credentials they need.

| Route | Auth | Returns | Query parameters |
| - | - | - | - |
| `GET /v1/collections/features/extractors` | None | An array. Each item has `feature_extractor_name`, `version`, `feature_extractor_id`, `input_schema`, `output_schema`, `parameter_schema`, `default_parameters`, `costs`, `required_vector_indexes`, and `example_usage`. | None |
| `GET /v1/discovery/extractors` | None | An array of summaries for manifest authors. Each item has `name`, `version`, `supported_modalities`, `output_features`, `parameter_schema`, `default_parameters`, `costs`, and `example_usage`. | None |
| `GET /v1/extractors` | API key | An object with `extractors`, `total`, `builtin_count`, and `custom_count`. | `source` (`builtin`, `custom`, or `all`) and `include_disabled` |
| `GET /v1/namespaces/{namespace_id}/extractors` | API key and the `X-Namespace` header | An object with `extractors`, `total`, `namespace_id`, `builtin_count`, and `custom_count`. It also lists custom plugins deployed for that namespace. | `source` and `include_disabled` |

To read one extractor's schemas, call `GET /v1/collections/features/extractors/{feature_extractor_id}`. Each extractor page in these docs links to it.

`/v1/features/extractors` does not exist and returns `404`. The `/v1/features` prefix serves feature search only. Custom plugins need dedicated infrastructure. See [Custom Extractors](/docs/processing/custom-extractors).

## Operational checklist

1. **Prefer `features: [...]`.** Reach for the explicit config only when you need custom input mappings, passthrough, or parameters.
2. **Pin versions.** Upgrade deliberately via new collection + migration, never in-place.
3. **Batch uploads.** Keep ingestion batches in the 1k–10k object range to maximize parallelism without overwhelming the scheduler.
4. **Use `field_passthrough` for every metadata filter.** If you plan to filter by `category` at query time, pass it through at ingestion time.
5. **Inspect features before querying.** `GET /v1/collections/{id}/features` returns the feature URIs, dimensions, and descriptions actually registered — use this to confirm the pipeline ran correctly before building retrievers.
6. **Watch lineage, not just counts.** A task reporting `COMPLETED` with 0 features written almost always means an `input_mappings` mistake or (for custom extractors) a bad `features` manifest. The [Document Lineage API](/docs/api-reference/document-lineage/get-document-lineage) and the `vector_indexes` field on the collection are your diagnostic tools.

## Reference

* **[Features](/docs/processing/features)** — the feature menu per modality, discovery endpoint, and cost estimates.
* **[Migration guide](/docs/processing/extractor-migration)** — mapping deprecated built-in extractor names to feature keys.
* **[Custom Extractors](/docs/processing/custom-extractors)** — bring your own pipeline; runs on the same Ray infrastructure with full GPU support, versioning, and observability, and prices per unit from its declared compute profile.


# Features
Source: https://docs.mixpeek.com/docs/processing/features

Features are what you want to search by — pick them per file type; Mixpeek resolves the pipeline, defaults the wiring, and prices per natural unit

Features answer the third of the [three pricing questions](/docs/platform/billing#how-pricing-works) — after *what kind of files* and *how much content* comes **what you want to search by**. Instead of choosing models or pipelines, you declare the capabilities you want — visual similarity (`image_search`), faces (`faces`), on-screen text (`onscreen_text`), document layout (`document_layout`) — and the platform resolves the implementation internally. Swapping or upgrading the models behind a feature never changes your config, your API calls, or your pricing.

Every feature applies to a **modality** (the *what kind of files* question) and is billed in that modality's natural unit (the *how much* question):

| Modality | Unit | Base feature (included in base rate) | Add-on features |
| - | - | - | - |
| `image` | per image | `image_search` | `faces`, `multimodal_understanding` |
| `video` | per minute | `video_search` | `faces`, `onscreen_text`, `audio_fingerprint`, `multimodal_understanding` |
| `audio` | per minute | `audio_search` | `multimodal_understanding` |
| `document` | per page | `document_search` | `faces`, `document_layout`, `multimodal_understanding` |
| `text` | per token | `text_search` | `multimodal_understanding` |
| `web` | per crawled page | `web_crawl` | — |

Clustering and taxonomy enrichment are **included** on every modality at no additional charge. For rates, tiers, and how usage is billed, see [Billing & Pricing](/docs/platform/billing).

## Discover features

`GET /v1/collections/features` returns the live feature menu — modalities, units, feature keys, display names, and current rates. No authentication required. This endpoint is the source of truth: studio, the homepage pricing page, and the billing engine all read the same catalog.

```bash theme={null}
curl "https://api.mixpeek.com/v1/collections/features"
```

```json theme={null}
{
  "modalities": [
    {
      "modality": "video",
      "unit": "minute",
      "unit_display": "per minute",
      "base": {
        "key": "video_search",
        "name": "Video search (scene embeddings)",
        "kind": "base",
        "included_in_base_rate": true,
        "rate_usd": 0.05,
        "per": 1
      },
      "addons": [
        { "key": "faces", "name": "Face detection + identity", "kind": "addon", "rate_usd": 0.10, "per": 1, "companion": true },
        { "key": "onscreen_text", "name": "On-screen text (video OCR)", "kind": "addon", "rate_usd": 0.10, "per": 1, "companion": true },
        { "key": "clustering", "name": "Clustering (included)", "kind": "included", "included": true, "pricing_note": "included — no additional charge" }
      ]
    }
  ],
  "custom_features": [],
  "full_res_multiplier": 2.0
}
```

Response abbreviated — the live payload covers all six modalities. Feature `kind` tells you how it's priced:

| Kind | Meaning |
| - | - |
| `base` | Covered by the modality's per-unit base rate — you get it by creating any collection for that modality |
| `addon` | Priced per the same unit, added on top of the base rate |
| `external` | Backed by external LLM inference — usage-based passthrough pricing, never flat-rated |
| `included` | Bundled enrichment (clustering, taxonomy) — no additional charge |

## Create a collection with features

Pass `features: [...]` to [Create Collection](/docs/api-reference/collections/create-collection). The platform resolves the pipeline, pins its version, and defaults the input wiring for you:

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "press-photos",
    "source": { "type": "bucket", "bucket_ids": ["'$BUCKET_ID'"] },
    "features": ["image_search"]
  }'
```

```python Python theme={null}
import os
from mixpeek import Mixpeek

# namespace= is required — it's sent as the X-Namespace header on every call
# (collection create returns 403 without it).
client = Mixpeek(
    api_key=os.environ["MIXPEEK_API_KEY"],
    namespace=os.environ["MIXPEEK_NAMESPACE"],
)

collection = client.collections.create(
    collection_name="press-photos",
    source={"type": "bucket", "bucket_ids": [os.environ["BUCKET_ID"]]},
    features=["image_search"],
)
```

```javascript JavaScript theme={null}
import { Mixpeek } from "mixpeek";

// namespace is sent as the X-Namespace header on every call (collection
// create is namespace-scoped).
const client = new Mixpeek({
  apiKey: process.env.MIXPEEK_API_KEY,
  namespace: process.env.MIXPEEK_NAMESPACE,
});

const { data: collection } = await client.collections.createCollection({
  createCollectionRequest: {
    collection_name: "press-photos",
    source: { type: "bucket", bucket_ids: [process.env.BUCKET_ID] },
    features: ["image_search"],
  },
});
```

> [!NOTE]
>
> Both SDKs accept `features` on collection create — Python **`mixpeek>=1.3.29`** (`client.collections.create(...)`) and JavaScript **`mixpeek>=0.81.28`** (`client.collections.createCollection({ createCollectionRequest: { ... } })`). Set the namespace when you construct the client (Python `namespace=`, JS `namespace:`, or the `MIXPEEK_NAMESPACE` env var) — collection create is namespace-scoped and returns 403 without it. The `feature_extractor` field is a [deprecated alias](/docs/processing/extractor-migration).

> [!WARNING]
>
> Creating the collection does not process anything on its own. Objects you upload
> to a bucket sit at `status: PENDING`, and a bucket-sourced collection returns zero
> documents until a batch runs. That reads as a failed create, and it is not one.
>
> Two paths start the work:
>
> * **Sync config**, the production path. The bucket submits batches as objects land.
> * **Manual trigger**, for on-demand runs. `POST /v1/collections/{collection_id}/trigger`
>   returns `202` with `batch_id` and `task_id`, plus the `object_count` it picked up.
>
> ```bash theme={null}
> curl -X POST "https://api.mixpeek.com/v1/collections/$COLLECTION_ID/trigger" \
>   -H "Authorization: Bearer $MIXPEEK_API_KEY" \
>   -H "X-Namespace: $NAMESPACE_ID"
> ```
>
> Track it with `GET /v1/tasks/{task_id}`. See [Triggers](/docs/platform/triggers) and
> [Syncs](/docs/platform/syncs).

### Default input wiring

Every organization's scaffolded `uploads` bucket exposes one standard source property per modality — `image`, `video`, `audio`, `pdf`, `content` (text), and `url` (web). A features-based create maps the pipeline's inputs to those properties automatically, so the collection is runnable with zero wiring.

If your bucket uses different property names, the create returns a teaching `422` naming the fields that exist so you can rename the property — or fall back to explicit `input_mappings` via the [advanced pipeline config](/docs/processing/feature-extractors).

### Errors teach

Validation on the features path is designed for humans and agents guessing their way in — every rejection tells you what to do instead:

* **Unknown key** — `features: ["face"]` responds with the full list of valid feature keys (and reminds you about `custom:<plugin>`).
* **One pipeline per collection** — some feature combinations need separate collections (e.g. a base feature plus a `companion: true` add-on). The error lists exactly which features group into which collection: create one collection per group over the same source bucket. Add-ons flagged `companion: true` in discovery always create a companion collection alongside the base.
* **`features` XOR `feature_extractor`** — provide one or the other, never both.
* **Not yet available** — a feature key the platform recognizes but does not serve is rejected explicitly rather than silently ignored.

### Warnings on a create that succeeded

A `features` create wires `input_mappings` to the standard `uploads` properties for you, so this rarely fires on the scaffolded bucket. On a custom bucket it can: the create returns `201` with a `warnings` array when a mapped field is declared on the bucket schema and missing from the objects already in it. The collection exists and will process nothing until the sync writes that property. See [`warnings` on the create response](/docs/processing/feature-extractors).

### Custom features (bring your own)

[Custom extractors](/docs/processing/custom-extractors) you publish are *your* vocabulary — they stay explicitly named and are selected as `custom:<plugin_name>`:

```json theme={null}
{
  "collection_name": "product-defects",
  "source": { "type": "bucket", "bucket_ids": ["bkt_123"] },
  "features": ["custom:defect_detector"]
}
```

Custom features are priced per unit from the compute profile the plugin declares — the same machinery that prices native features. See [Custom Extractors](/docs/processing/custom-extractors).

### Full resolution opt-out

By default, Mixpeek normalizes content once at ingest (video to a 720p mezzanine, images capped at \~1568px max edge, audio to 16kHz mono) — originals are always kept untouched in your bucket. If a workload needs extraction on the original resolution (fine print OCR, tiny logos), opt out per collection:

```json theme={null}
{
  "collection_name": "fine-print",
  "source": { "type": "bucket", "bucket_ids": ["bkt_123"] },
  "features": ["document_layout"],
  "full_res": true
}
```

Full-res processing bills at a **2x multiplier** on the modality's base and add-on rates (`full_res_multiplier` in the pricing payload).

### Chain collections (advanced)

Searching by one thing often produces content worth searching by another — e.g. search video by scenes, then feed those documents into a second collection to search the results by something else. Collection-to-collection pipelines do exactly this: a collection can read another collection's documents as its source, forming a processing DAG. See [Multi-Tier Feature Extraction](/docs/processing/multi-tier-extractors).

### Merge sources with different schemas (advanced)

A collection can read several buckets at once. When those buckets have different
schemas, `source.field_map` chooses which fields from each one land in the merged
collection, and what they are called there.

```json theme={null}
{
  "collection_name": "all-creative",
  "source": {
    "type": "bucket",
    "bucket_ids": ["bkt_video_ads", "bkt_static_ads"],
    "field_map": {
      "bkt_video_ads": [
        { "source_path": "headline", "target_path": "title" }
      ],
      "bkt_static_ads": [
        { "source_path": "name", "target_path": "display_name", "default": "untitled" }
      ]
    }
  },
  "features": ["image_search"]
}
```

Each entry accepts four fields:

| Field | Required | Behavior |
| - | - | - |
| `source_path` | Yes | Field to copy. Dot notation reaches nested fields, as in `metadata.author`. |
| `target_path` | No | Name in the merged collection. Defaults to `source_path`, or its last component for a nested path. |
| `default` | No | Value used when the source field is missing. Without it, the field is omitted from that document. |
| `required` | No | `true` fails processing when the source field is missing. Defaults to `false`. |

`field_map` narrows. A source listed in `bucket_ids` but absent from `field_map`
passes **all** of its fields through, so you only name the sources you want to trim.
The same key works for `source.type: "collection"`, mapping `collection_ids`.

> [!WARNING]
>
> Leaving a source out and giving it an empty list are opposites, and both return
> `200`:
>
> | You write | That source contributes |
> | - | - |
> | no key for the source | every one of its fields |
> | `"bkt_x": []` | nothing |
>
> The branch turns on whether the key is present, not on whether its list has
> entries. A source mapped to `[]` adds no fields to the collection, so its
> documents arrive carrying only `null` for the fields other sources named. That
> reads as broken ingestion rather than as the selection you wrote. Remove the key
> to pass the source through.

#### Two sources cannot write the same field

Every target name in a `field_map` must be unique across the whole map. Create returns
`422` when two entries resolve to the same one, whichever sources they belong to and
whatever their types:

```
field_map target-path collision(s): target_path 'title' is mapped by
sources ['bkt_static_ads', 'bkt_video_ads']. Two sources cannot map fields
to the same target_path (no silent overwrite). Rename one target_path to resolve.
```

A `field_map` key that is not one of the declared `bucket_ids` or `collection_ids` is
rejected the same way.

> [!WARNING]
>
> Names collide before you see them. When `target_path` is omitted the target is the
> last dot-component of `source_path`, so `metadata.author` on one source and
> `content.author` on another both resolve to `author` and fail. Set `target_path`
> explicitly whenever you map nested fields.

This makes `field_map` a selector, never a converger. Two sources that name the same
thing differently, `headline` and `title`, cannot be folded onto one field here. Land
them as separate fields, or align the names in the source schemas.

#### Merging without a field\_map

Leave `field_map` off and every source contributes all of its fields, unioned by name.
That path does converge, and it is the one to reach for when your sources already
agree on names:

| Sources declare | Result |
| - | - |
| Same field name, same type | One field. Each document fills it from its own source. |
| Same field name, different types | `422`: `Field 'title' has conflicting types across sources`. |

#### Reading a merged collection

The output schema is the union of what every source contributes, so a document
carries the fields its own source mapped and `null` for the rest. Given the example
above, a document from `bkt_video_ads` has a `title` value and `display_name` of
`null`, and one from `bkt_static_ads` has the reverse.

Filter on `root_bucket_id` to select the documents one source contributed.

> [!NOTE]
>
> A `null` here means the source never mapped that field, not that the value was
> empty. Set `default` on an entry when you want a placeholder instead, which fills
> only for documents from **that** source when its `source_path` is missing.

### Build a collection from a retriever pipeline (advanced)

A collection normally runs a feature extractor over its source. It can instead run a
**retriever pipeline** over an upstream collection, which makes it a materialized view
of that collection's documents.

```json theme={null}
{
  "collection_name": "scored-scenes",
  "source": { "type": "collection", "collection_ids": ["col_scenes"] },
  "retriever_transform": {
    "stages": [
      {
        "stage_name": "safety",
        "config": {
          "stage_id": "classify",
          "parameters": { "task": "nsfw", "document_field": "content" }
        }
      }
    ],
    "write_back_fields": [
      { "source_field": "classification.label", "target_field": "safety_label" },
      { "source_field": "classification.nsfw_score", "target_field": "safety_score" },
      { "source_field": "classification.is_nsfw", "target_field": "flagged" }
    ]
  }
}
```

| Field | Required | Purpose |
| - | - | - |
| `stages` | Yes | The pipeline, one or more stages, in the same shape a retriever uses. |
| `write_back_fields` | No | Which pipeline outputs become document fields. These define `output_schema`. |
| `input_mappings` | No | Feed upstream document fields into stage inputs. |

Each `write_back_fields` entry takes `source_field` (a dot path into the retriever
result), `target_field` (the field written on the document), and an optional `mode` of
`first`, `all_as_array`, or `concat` when a stage returns several results.

**A collection is extractor-based or transform-based, never both.** `feature_extractor`
became optional when `retriever_transform` arrived, and create returns `422` if you send
neither or both.

> [!WARNING]
>
> `retriever_transform` requires `source.type` of `collection`. A bucket source returns
> `422`. Point it at a collection that already holds the documents you want to transform.

Run it with `POST /v1/collections/{collection_id}/trigger`.

### How the output is shaped

Cardinality follows the pipeline, and the mechanism is the document id:

* **Without a `group_by` stage**, each output document takes its id from the upstream
  document, so the view is 1:1.
* **With one**, the id comes from the value of the grouped field, so upstream rows
  sharing that value collapse into exactly one document. The field is taken from the
  last `group_by` stage in the pipeline, defaulting to `source_object_id`.

Grouping keeps the highest-scoring result for each group, because `group_by` orders
groups by score before the collapse. The rest of that group's rows are not merged into
it; they are dropped.

Ids are deterministic either way, so a re-trigger upserts in place. Re-running after new
upstream documents arrive updates the view incrementally rather than appending a second
copy of it.

A trigger reads the **entire** source collection, cursor-paging through it. The
100-document cap on `POST /v1/documents/list` bounds that endpoint, not this, and there
is no paging option to set. A source of 50,000 documents produces a view over all
50,000.

> [!WARNING]
>
> Transform output is written **payload-only**, with no vectors. `write_back_fields`
> produces fields you can filter, sort, and read; it does not produce an embedding. To
> search the view by similarity, point a second collection with a feature extractor at
> it.

> [!NOTE]
>
> `feature_extractor` cannot be added to a transform collection afterward. It is a
> structural field on every collection, so `PATCH` rejects it with `422` and tells you
> to clone instead. Pick the build mode at create time.

## Find your `feature_uri` (to search)

A [`feature_search`](/docs/retrieval/stages/feature-search) stage needs a **`feature_uri`** — the exact vector index to search. You picked *features* by name; each one produces a vector index whose `feature_uri` looks like `mixpeek://<extractor>@<version>/<model>`.

> [!WARNING]
>
> The last segment is the embedding **model name**, not the literal word `embedding`. A wrong `feature_uri` (e.g. `.../embedding`) does **not** error — the stage matches nothing and silently returns **0 results** (with a self-correcting warning in the response). Always use the exact string.

**The reliable, version-proof way — read it from the collection.** After processing, `GET` the collection and copy the URI; this always matches what you actually built, even if model versions change:

```bash cURL theme={null}
curl "https://api.mixpeek.com/v1/collections/$COLLECTION_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
# → vector_indexes[].feature_uri  e.g. "mixpeek://universal_extractor@v1/gemini-embedding-2"
```

```python Python theme={null}
col = client.collections.get(collection_id)
for vi in col["vector_indexes"]:
    print(vi["feature_uri"])   # paste the one you want into your feature_search stage
```

> [!NOTE]
>
> **On a BYO or promoted collection the value is the bare index name, not a `mixpeek://` URI.**
> Those collections have no extractor, so their `vector_indexes` entries come from the
> namespace vector config and carry `source: "namespace_vector_config"`. A `feature_search`
> stage accepts either form, so paste whatever the `GET` returned.

**Common features → `feature_uri`** (the exact string a fresh `@v1` collection produces):

| Feature (`features: [...]`) | Extractor | `feature_uri` |
| - | - | - |
| `video_search` | `universal_extractor` | `mixpeek://universal_extractor@v1/gemini-embedding-2` |
| `text_search` | `text_extractor` | `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1` |
| `multimodal_understanding` | `multimodal_extractor` | `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding` |

> [!NOTE]
>
> **Video transcript search.** `video_search` produces **one** blended scene embedding (`gemini-embedding-2`); the transcript rides along as a `text` payload field, not a separate searchable vector. To search a transcript as its own vector, add the `multimodal_understanding` feature with transcription enabled — it creates a second index `mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1`. When in doubt, `GET` the collection and read the `feature_uri`s it actually has.

## Estimate before you run

`POST /v1/organizations/billing/estimate` quotes planned ingestion using the **same rating engine that bills you** — the quote and the charge can't disagree:

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/billing/estimate" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "items": [
      { "mime_type": "video/mp4", "minutes": 42 },
      { "mime_type": "application/pdf", "pages": 300 }
    ],
    "features": ["faces", "onscreen_text"]
  }'
```

```python Python theme={null}
resp = requests.post(
    "https://api.mixpeek.com/v1/organizations/billing/estimate",
    headers={"Authorization": f"Bearer {os.environ['MIXPEEK_API_KEY']}"},
    json={
        "items": [
            {"mime_type": "video/mp4", "minutes": 42},
            {"mime_type": "application/pdf", "pages": 300},
        ],
        "features": ["faces", "onscreen_text"],
    },
)
quote = resp.json()
```

```javascript JavaScript theme={null}
const resp = await fetch(
  "https://api.mixpeek.com/v1/organizations/billing/estimate",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.MIXPEEK_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      items: [
        { mime_type: "video/mp4", minutes: 42 },
        { mime_type: "application/pdf", pages: 300 },
      ],
      features: ["faces", "onscreen_text"],
    }),
  }
);
const quote = await resp.json();
```

```json theme={null}
{
  "pricing_model": "v2",
  "placeholder_rates": false,
  "line_items": [
    { "modality": "video", "key": "video_search", "units": 42.0, "rate_usd": 0.05, "per": 1, "amount_usd": 2.10 },
    { "modality": "video", "key": "faces", "units": 42.0, "rate_usd": 0.10, "per": 1, "amount_usd": 4.20 },
    { "modality": "video", "key": "onscreen_text", "units": 42.0, "rate_usd": 0.10, "per": 1, "amount_usd": 4.20 },
    { "modality": "document", "key": "document_search", "units": 300.0, "rate_usd": 1.50, "per": 1000, "amount_usd": 0.45 },
    { "modality": "document", "key": "faces", "units": 300.0, "rate_usd": 1.00, "per": 1000, "amount_usd": 0.30 }
  ],
  "total_usd": 11.25,
  "batch_minimum_applied": false,
  "allowance_pool_usd": 10.0,
  "allowance_covered_usd": 10.0,
  "estimated_overage_usd": 1.25
}
```

Base features are quoted automatically per the modality of each item — you only list add-ons. Unknown MIME types quote at the cheapest band rather than failing (be forgiving is a design rule here).

## How this relates to extractors

Under the hood every feature resolves to an extraction pipeline, and [custom extractors](/docs/processing/custom-extractors) let you register your own. But **built-in extractor names are no longer part of the public API surface** — they remain accepted in configs only as deprecated aliases. If you have existing configs using `feature_extractor`, see the [migration guide](/docs/processing/extractor-migration); for advanced pipeline knobs (input mappings, field passthrough, parameters), see [Pipeline Configuration](/docs/processing/feature-extractors).


# Jev decision model
Source: https://docs.mixpeek.com/docs/processing/jev

Use TypeSafe AI's Jev for classification, filtering, reranking, cluster labeling and hierarchical cluster search

Jev is an evaluation model from TypeSafe AI. It answers typed questions about a piece of text and returns calibrated probabilities. It does not generate text. Use it for steps that are a decision over a fixed set of options: keep or discard, pick a label, rate on a scale, route to a branch.

| Question type | Jev returns | Example |
| - | - | - |
| Yes/no | Probability that the answer is yes | Does this document mention side effects? |
| Choice | The chosen option and a probability for each option | Which category fits: footwear, apparel or electronics? |
| Score | A value across ordered levels you define | How severe is the complaint, 1 to 5? |

Jev reads text only, up to 32,000 tokens per request. Media documents are judged on their text fields, such as descriptions, transcripts and metadata. TypeSafe prices Jev at \$0.042 per million input tokens, and output is free.

## Jev in Mixpeek

| Surface | How to select Jev | What Jev decides |
| - | - | - |
| [Text extractor](/docs/processing/extractors/text) | `llm_provider: "typesafe"` | Each `response_shape` field, at ingestion |
| [Cluster labeling](/docs/enrichment/clusters) | `llm_labeling.provider: "typesafe"` + `candidate_labels` | The label for each cluster, from your vocabulary |
| [LLM Filter](/docs/retrieval/stages/llm-filter) | `model_name: "jev-latest"` | Keep or discard each document |
| [LLM Enrich](/docs/retrieval/stages/llm-enrich) | `model_name: "jev-latest"` + `output_schema` | Each schema field for each document |
| [Rerank](/docs/retrieval/stages/rerank) | `inference_name: "typesafe__jev"` | Relevance probability for each query and document pair |
| [Classify](/docs/retrieval/stages/classify) | `inference_name: "typesafe__jev"` + `labels` | Probability of each label for each document |
| [Agent Search](/docs/retrieval/stages/agent-search) | `strategy: "cluster_navigation"` + `model_name: "jev-latest"` | Which child cluster to descend into, at every level |
| Inference API | `inference_name: "typesafe__jev"` | Any of the question types, called directly |

## Swap the decision model

Jev belongs to a category Mixpeek calls decision models: a model that answers typed questions (yes/no, choice, score) and returns a probability with each answer. Every surface above asks its question through one factory, so any generative model can take Jev's place. A generative model answers the same questions and reports its own confidence, and Mixpeek marks those probabilities as uncalibrated.

| Scope | How to swap | Example |
| - | - | - |
| One request | `inference_name: "mixpeek__decision"` with `parameters.model` | `"model": "gemini-2.5-flash-lite"` |
| One stage | `inference_name: "mixpeek__decision"` plus `decision_model` on [Rerank](/docs/retrieval/stages/rerank) or [Classify](/docs/retrieval/stages/classify), `model_name` on [Agent Search](/docs/retrieval/stages/agent-search) | `"decision_model": "gemini-2.5-flash-lite"` |
| One cluster | `llm_labeling.provider` and `model_name` with `candidate_labels` | `"provider": "google", "model_name": "gemini-2.5-flash-lite"` |
| The deployment | `DECISION_MODEL_DEFAULT` on the API and engine | `DECISION_MODEL_DEFAULT=gemini-2.5-flash-lite` |

`mixpeek__decision` takes the same four input shapes as `typesafe__jev` and uses `DECISION_MODEL_DEFAULT` when `parameters.model` is unset. `typesafe__jev` always calls Jev. Every response names the model that answered in `model` and says whether its probabilities are calibrated in `calibrated`.

## Where the probabilities go

Mixpeek writes the decision model's probability next to every answer it produces, so you can threshold, audit or route on it later.

Rerank, Classify, cluster labeling and Agent Search record a probability from any decision model. The text extractor, LLM Filter and LLM Enrich record one when Jev answers. With a generative model those three run as ordinary LLM calls and write no probability.

| Surface | Field | Holds |
| - | - | - |
| Text extractor | `extraction_confidence` | Probability of each extracted field's value |
| Cluster labeling | `label_confidence`, `label_candidates`, `label_model`, `label_calibrated` | Probability of the chosen label, the top three labels, and the model |
| LLM Filter | `scores.<stage_name>` on each kept document, `llm_decisions[].keep_probability` in stage metadata | Probability that the document meets the condition |
| LLM Enrich | `<output_field>_confidence` | Probability of each enriched field's value |
| Rerank | `score` on each document, `model_used` and `probabilities_calibrated` in stage metadata | Probability that the document is relevant |
| Classify | `labels[].confidence` | Probability of each label |
| Agent Search | `cluster_path_score` on each document, `navigation_trace` and `probabilities_calibrated` in stage metadata | Geometric mean of the decisions on the path, and every decision's probabilities |

## Schema fields and the questions they become

Extraction, cluster labeling and LLM Enrich describe their output as a JSON schema. Jev answers each field as one question, and all questions for one document go in one request.

| Schema field | Question |
| - | - |
| `boolean` | Yes/no. The value is true when the probability is at least 0.5 |
| `number` with `minimum: 0` and `maximum: 1` | Yes/no. The value is the probability |
| `string` with `enum` (2 to 255 values) | Choice. Add `enumDescriptions` to describe each option |
| `integer` with `minimum` and `maximum` (2 to 10 levels) | Score |
| `array` of `enum` strings | One yes/no per value. The value lists each option at 0.5 or above |
| `object` | Its properties, answered the same way |

A string field named `reason`, `reasoning`, `explanation`, `rationale` or `justification` receives the probabilities behind the answer. Any other field without a closed set of values is rejected before a request is sent, and the error names the field. Use a generative model such as `gemini-2.5-flash-lite` for free-text fields.

## Call Jev through the inference API

`typesafe__jev` accepts four input shapes.

| Inputs | Parameters | Returns |
| - | - | - |
| `pairs`: list of `[query, document]` | `relevance_criteria` (optional) | `scores`: one probability per pair |
| `text` | `labels`: list, or map of label to description | `labels`: every label with its probability, highest first |
| `text` + `schema` | none | `data` shaped like the schema, and `answers` with the probabilities |
| `state` + `questions` | none | TypeSafe's response, unchanged |

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/inference" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inference_name": "typesafe__jev",
    "inputs": {
      "text": "Kickflip down the eight stair at the city skatepark",
      "schema": {
        "type": "object",
        "properties": {
          "sport": {"type": "string", "enum": ["skateboarding", "surfing", "basketball"]},
          "is_tutorial": {"type": "boolean"}
        }
      }
    }
  }'
```

```python Python theme={null}
import requests

resp = requests.post(
    f"{MP_API_URL}/v1/inference",
    headers={"Authorization": f"Bearer {MP_API_KEY}", "X-Namespace": MP_NAMESPACE},
    json={
        "inference_name": "typesafe__jev",
        "inputs": {
            "text": "Kickflip down the eight stair at the city skatepark",
            "schema": {
                "type": "object",
                "properties": {
                    "sport": {"type": "string", "enum": ["skateboarding", "surfing", "basketball"]},
                    "is_tutorial": {"type": "boolean"},
                },
            },
        },
    },
)
print(resp.json())
```

```javascript JavaScript theme={null}
const resp = await fetch(`${MP_API_URL}/v1/inference`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${MP_API_KEY}`,
    "X-Namespace": MP_NAMESPACE,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    inference_name: "typesafe__jev",
    inputs: {
      text: "Kickflip down the eight stair at the city skatepark",
      schema: {
        type: "object",
        properties: {
          sport: { type: "string", enum: ["skateboarding", "surfing", "basketball"] },
          is_tutorial: { type: "boolean" },
        },
      },
    },
  }),
});
console.log(await resp.json());
```

## Writing questions Jev answers well

* State the exact condition. Jev reads criteria word for word, so "The document states the refund policy for damaged items" works better than "relevant to refunds".
* Keep arithmetic, counting and date comparison in code. Ask Jev for the parts, then compute.
* Send only the text the question needs. Unrelated content lowers accuracy.
* Use the probabilities. Route answers below a threshold you choose to review or to a generative model.

## Bring your own key

The inference API accepts `parameters.api_key` to call Jev with your own TypeSafe key.


# Migrate Embedding Models
Source: https://docs.mixpeek.com/docs/processing/model-migration

Switch a namespace to a new embedding model by re-extracting into a target namespace — safely, with validation and dry-run

Embedding dimensions are **locked at namespace creation** — you can't swap the embedding model in place, because the vector index dimensionality is fixed and vectors from different models aren't comparable. To move to a new model, you **re-extract** your data into a target namespace using the new extractor config, validate it, then cut over.

> [!NOTE]
>
> This is the right path whenever you change the embedding model (e.g. a new frontier text model, or switching `text_extractor` → a higher-dim model). For routine model upgrades within the same family, the [model registry](/docs/processing/model-registry) hot-swaps the default for **new** collections without touching existing ones.

## How it works

A **migration** of type `re_extract` reads your source namespace's objects and re-runs extraction with a new `feature_extractors` config (the new model) into a target namespace. Your source stays live and untouched until you choose to cut over.

| Migration type | What it does |
| - | - |
| `re_extract` | Re-run extraction with new extractor/model config (use this to change embedding models) |
| `copy` | Copy resources as-is to another namespace (no re-extraction) |
| `extend` | Add new features to existing documents without a full re-extract |

## 1. Validate first (dry run)

Always validate the migration config before committing compute — it checks the source, target, and extractor config without extracting anything.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/namespaces/migrations/validate" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "migration_type": "re_extract",
      "source_namespace_id": "ns_prod",
      "target_namespace_name": "prod-v2",
      "feature_extractors": [
        { "feature_extractor_name": "text_extractor", "version": "v1",
          "params": { "embedding_model": "<new-model-id>" } }
      ]
    }
  }'
```

## 2. Create the migration

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/namespaces/migrations/" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "migration_type": "re_extract",
      "source_namespace_id": "ns_prod",
      "target_namespace_name": "prod-v2",
      "feature_extractors": [
        { "feature_extractor_name": "text_extractor", "version": "v1",
          "params": { "embedding_model": "<new-model-id>" } }
      ],
      "dry_run": false,
      "webhook_url": "https://example.com/migration-status"
    },
    "start_immediately": false
  }'
```

The response includes a `migration_id`.

| Config field | Purpose |
| - | - |
| `migration_type` | `re_extract` to change the model |
| `source_namespace_id` | Namespace to migrate from |
| `target_namespace_name` / `target_namespace_id` | Where the re-extracted data lands |
| `feature_extractors` | New extractor config — the new embedding model |
| `filters` | Optionally migrate a subset (by collection, date, etc.) |
| `batch_options` | Tune batch size / parallelism |
| `dry_run` | Validate only, don't execute |
| `webhook_url` | Get progress callbacks |

## 3. Start and monitor

```bash theme={null}
# Start (if you didn't set start_immediately)
curl -sS -X POST "$MP_API_URL/v1/namespaces/migrations/{migration_id}/start" \
  -H "Authorization: Bearer $MP_API_KEY"

# Poll status
curl -sS "$MP_API_URL/v1/namespaces/migrations/{migration_id}" \
  -H "Authorization: Bearer $MP_API_KEY"
```

Cancel a running migration with `POST /v1/namespaces/migrations/{migration_id}/cancel`.

## 4. Cut over

Re-extraction re-pays GPU/extraction cost for every document, so validate on a subset first (use `filters`) and confirm quality before migrating everything. Once the target namespace is populated and validated:

1. Re-run your [evaluations](/docs/retrieval/evaluations) against the target namespace to confirm relevance is at least as good. For a head-to-head old-vs-new read on real query logs, run the same query set through both namespaces' retrievers as [benchmarks](/docs/retrieval/benchmarks), or [generate an evaluation dataset from recorded interactions](/docs/retrieval/interactions) so the comparison reflects production traffic rather than synthetic queries.
2. Point your application's `X-Namespace` (and retrievers) at the target namespace.
3. Retire the source namespace when you're confident.

## Rollback

The source namespace stays live and untouched until you retire it — that **is** the rollback path. If post-cutover metrics regress, point `X-Namespace` back at the source namespace; no re-extraction is needed in either direction. Only retire the source once the target has survived real traffic for as long as your risk tolerance requires.

> [!WARNING]
>
> Vectors from different embedding models are **not** comparable — you cannot mix old and new vectors in the same index, and you cannot copy embeddings across models (only `re_extract` regenerates them). Plan for the full re-extraction cost.

## Related

* [Model Registry](/docs/processing/model-registry) — how default models are resolved and hot-swapped
* [Feature Extractors](/docs/processing/feature-extractors) — extractor + model configuration
* [Evaluations](/docs/retrieval/evaluations) — verify the new model before cutting over
* [Reprocess existing content](/docs/tutorials/reprocess-existing-content) — the underlying re-extraction flow, step by step
* [Monitoring ingestion](/docs/processing/tasks) — track re-extraction progress


# Model Registry
Source: https://docs.mixpeek.com/docs/processing/model-registry

Load HuggingFace models, built-in models, or your own fine-tuned weights inside custom extractors

Models run inside [custom extractors](/docs/processing/custom-extractors). The model registry handles downloading, caching, and serving — you just declare which model to use and the infrastructure shares it across all workers via Ray's object store.

> [!NOTE]
>
> **How models are consumed.** Built-in and HuggingFace models work on any plan — reference them by feature URI or load them in a pipeline. **Custom (uploaded) models** are consumed from inside a custom extractor (`model_source="namespace"`), so wiring one into ingest/retrieval also requires a [dedicated deployment](/docs/processing/custom-extractors#availability) (or an extractor merged via [Submissions](/docs/processing/extractor-marketplace)). Uploading + deploying a custom model is supported wherever your org has Enterprise infra.

## Three Ways to Load Models

| Approach | When to Use |
| - | - |
| **Built-in Models** | Common tasks — embeddings, transcription, reranking. No code needed, just reference the feature URI. |
| **HuggingFace Models** | Any public HF model. Cached cluster-wide on first download. |
| **Custom Models** (Enterprise) | Your own fine-tuned weights uploaded as `.tar.gz`. Stored in S3, deployed to Ray. |

## HuggingFace Models (Recommended)

Use `LazyModelMixin` in your extractor's pipeline. Models load on first batch, not at actor creation, and are shared zero-copy across all workers.

```python theme={null}
from engine.models.lazy import LazyModelMixin
from engine.inference.services import BaseBatchInferenceService

class MyEmbeddingProcessor(LazyModelMixin, BaseBatchInferenceService):
    model_id = "intfloat/multilingual-e5-large-instruct"
    model_class = "AutoModel"
    tokenizer_class = "AutoTokenizer"
    torch_dtype = "float16"
    model_source = "huggingface"

    def _process_batch(self, batch):
        model, tokenizer = self.get_model()

        inputs = tokenizer(
            batch["text"].tolist(),
            padding=True,
            truncation=True,
            return_tensors="pt",
        )

        with torch.no_grad():
            outputs = model(**inputs)

        batch["embedding"] = outputs.last_hidden_state.mean(dim=1).tolist()
        return batch
```

### LazyModelMixin Attributes

| Attribute | Type | Default | Description |
| - | - | - | - |
| `model_id` | str | `""` | HuggingFace model ID or namespace model ID |
| `model_class` | str | `"AutoModel"` | Transformers model class name |
| `tokenizer_class` | str \| None | `"AutoTokenizer"` | Tokenizer class, or `None` to skip |
| `torch_dtype` | str | `"float32"` | `"float16"`, `"float32"`, or `"bfloat16"` |
| `model_source` | str | `"huggingface"` | `"huggingface"` or `"namespace"` |

Call `self.get_model()` to get a `(model, tokenizer)` tuple. Override `_instantiate_model(cached_data)` for non-standard architectures.

## Custom Models (Enterprise)

> [!NOTE]
>
> Custom models require an **Enterprise** subscription. [Contact sales](https://mixpeek.com/contact) to enable.

Upload fine-tuned weights and use them in extractors. Three steps:

**1. Upload**

```bash theme={null}
tar -czvf my_model.tar.gz ./model_weights/

curl -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/models" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -F "file=@my_model.tar.gz" \
  -F "name=my-embedding-model" \
  -F "version=1.0.0" \
  -F "model_format=pytorch" \
  -F "task_type=embedding" \
  -F "num_gpus=0" \
  -F "memory_gb=4.0"
```

Supported formats: `pytorch` (`.pt`, `.pth`), `safetensors`, `onnx`, `huggingface` (directory).

**2. Deploy to Ray**

```bash theme={null}
curl -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/models/my-embedding-model_1_0_0/deploy" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

**3. Use in your extractor**

Set `model_source = "namespace"` and override `_instantiate_model()`:

```python theme={null}
class MyCustomProcessor(LazyModelMixin, BaseBatchInferenceService):
    model_id = "my-embedding-model_1_0_0"
    model_source = "namespace"

    def _instantiate_model(self, weights):
        import torch
        model = torch.nn.Linear(768, 256)
        model.load_state_dict(weights)
        model.to(self._detect_device())
        model.eval()
        return model, None

    def _process_batch(self, batch):
        model, _ = self.get_model()
        # Use model...
```

## Model Versioning

Models are versioned independently. Deploy a new version alongside the existing one, test in staging, then shift traffic:

```
my-embedding-model_1_0_0  (production)
my-embedding-model_2_0_0  (staging — validate before promoting)
```

The feature URI updates with the extractor version (`mixpeek://my_extractor@2.0.0/my_embedding`), so both versions can coexist.

## Python SDK

```python theme={null}
from mixpeek import Mixpeek
from mixpeek.api.custom_models_api import CustomModelsApi

client = Mixpeek(api_key="mxp_sk_...")
models = CustomModelsApi(client.api_client)

result = models.upload_model_namespaces(
    namespace_id="ns_abc123",
    file="my_model.tar.gz",   # path to the .tar.gz archive
    name="my-reranker",
    version="1.0.0",
    model_format="pytorch",
    task_type="reranking",
)

models.deploy_model_namespaces(
    namespace_id="ns_abc123",
    model_id=result.model_id,
)
```

> [!NOTE]
>
> The generated SDK exposes these under `CustomModelsApi` (`upload_model_namespaces`, `deploy_model_namespaces`, `list_models_namespaces`, …). See the [Models API reference](/docs/api-reference/custom-models/upload-a-custom-model) for exact parameters. The cURL form above is verified end-to-end.

## Limits

| Limit | Value |
| - | - |
| Max models per organization | 50 |
| Max archive size | 5 GB |
| Supported formats | pytorch, safetensors, onnx, huggingface |

## Related

> **[Custom Extractors](/docs/processing/custom-extractors)**
>
> Package and deploy extractors that use these models.

> **[Extractor Quickstart](/docs/tutorials/custom-extractor-quickstart)**
>
> Build a working extractor with model loading end-to-end.

> **[Model API Reference](/docs/api-reference/custom-models/upload-a-custom-model)**
>
> Upload, deploy, list, and delete model archives.

> **[Self-Improving CV Pipeline](/docs/tutorials/annotations-improve-features)**
>
> Full tutorial: deploy YOLO, annotate, fine-tune, redeploy.


# Multi-Tier Feature Extraction
Source: https://docs.mixpeek.com/docs/processing/multi-tier-extractors

How chained collections, dependency tiers, and cross-tier lineage turn extraction into a composable DAG

> **[Browse the extractor catalog on GitHub](https://github.com/mixpeek/mixpeek-extractors)**
>
> Runnable reference for every built-in Mixpeek extractor — inputs, parameters, output fields, embedding models, and copy-paste examples. Auto-generated from the live registry, so it always matches production.

![Multi-tier extraction: raw objects flow through Tier 1 (decompose), Tier 2 (embed), Tier 3 (classify) as a chained collection DAG](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-multi-tier.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=e26c58348ff5166c781ee407b76a9b81)

A single extractor call gives you one pass over raw data. That works for homogeneous inputs and a single model. It falls apart the moment you need to transcribe audio *and then* embed the transcription, or OCR a PDF *and then* chunk the text *and then* classify each chunk. Multi-tier extraction solves this by turning ingestion into a declarative DAG: collections chain together, each tier reads the output of the previous one, and the engine schedules them in the correct order.

> [!NOTE]
>
> This is the composition story for feature extractors. For the core concept — how a single extractor is configured — see [Feature Extractors](/docs/processing/feature-extractors). For the catalog, see the pages under **Built-in Feature Extractors**.

## Why One Extractor Isn't Enough

Three problems force you out of the single-extractor model:

**1. Heterogeneous inputs.** A single 30-minute video is not one document — it's dozens of segments, each with its own transcription, thumbnail, embedding, and OCR. A PDF is N pages with layout, text chunks, and figures. One extractor can't own both the decomposition *and* every downstream enrichment without becoming an unmaintainable god-model.

**2. Model boundaries.** Transcription is an audio model. Text embedding is a text model. Taxonomy classification is an LLM. Each has different hardware (GPU vs API), different batch sizes, different failure modes. Trying to run them in a single actor wastes GPU time on I/O-bound work and vice versa.

**3. Reusability.** You want to build one transcription step and reuse the output for embedding, for summarization, for diarization, for taxonomy classification. If transcription is buried inside a monolithic extractor, every downstream use triggers a redundant pass.

Multi-tier extraction moves composition into the ingestion graph itself. One extractor per job, one concern per extractor, composed via collections.

## The Object → Document → Feature Model

Three primitives do all the work in Mixpeek:

| Primitive | What It Is | Why It Matters |
| - | - | - |
| **Object** | Raw file or record in a bucket (a video, a PDF, a row of JSON). | The input boundary. You ingest objects. |
| **Document** | One row of output in a collection, produced by **decomposition**. | The query boundary. You search documents. |
| **Feature** | A named output attached to a document (embedding, transcription, OCR text, label, score). | The composition boundary. Retrievers, taxonomies, and clusters reference features by URI. |

The pipeline is always:

```
Object (bucket) → Decomposition → Document (collection) → Feature extractor → Feature (MVS + Mongo)
```

**Decomposition** is the step people miss. A 30-minute video isn't one document — it's dozens of segments with their own transcriptions, thumbnails, and embeddings. A PDF isn't one document — it's N pages with OCR, layout, and text chunks. The extractor decides the decomposition strategy (time/scene/silence for video; page/paragraph/sentence for text), and every resulting document gets its own row. Retrievers search at the document grain, not the object grain.

## Chaining Collections Across Tiers

To chain extractors, you chain collections: collection B's `source` is `{"type": "collection", "collection_id": "<A's collection_id>"}`, and B's extractor reads A's documents as input. The engine assigns a **processing tier** to each collection based on the dependency graph:

```
Tier 1  raw objects       → multimodal_extractor  → segments with transcription + scene embeddings
Tier 2  tier-1 documents  → text_extractor        → chunks with text embeddings
Tier 3  tier-2 documents  → taxonomy enrichment (taxonomy_applications) → classifications per chunk
```

Tier 1 runs first on raw objects. Tier 2 runs only once tier 1 has written results. Tier 3 only after tier 2. The engine respects this DAG automatically — you don't trigger tiers manually.

### Worked Example: Video → Transcription → Text Embeddings → Classifications

Each body below is a `POST /v1/collections` request sent with an `X-Namespace` header. The namespace must list both extractors in its `feature_extractors` (set when you create the namespace), and the taxonomy must already exist. Every create returns a `collection_id`; tier 2's `source` uses the one tier 1 returned.

```json theme={null}
// Tier 1: raw video objects → scene segments with transcription
{
  "collection_name": "segments",
  "source": { "type": "bucket", "bucket_ids": ["<your bucket_id>"] },
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": { "video": "file" },
    "parameters": { "split_method": "scene", "run_transcription": true }
  }
}

// Tier 2: segment transcriptions → sentence chunks with text embeddings,
// classified against a taxonomy as each chunk is written (tier 3)
{
  "collection_name": "chunks",
  "source": { "type": "collection", "collection_id": "<collection_id returned for segments>" },
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": { "text": "transcription" },
    "field_passthrough": [
      { "source_path": "start_time" },
      { "source_path": "end_time" },
      { "source_path": "root_object_id" }
    ],
    "parameters": { "split_by": "sentences" }
  },
  "taxonomy_applications": [{ "taxonomy_id": "<your taxonomy_id>" }]
}
```

At query time, a retriever can `feature_search` over the `chunks` collection (fine-grained, classified chunks) and trace each hit back to the original video segment through the `start_time`, `end_time` and `root_object_id` fields passed through from tier 1.

## How the Engine Schedules Tiers

1. **Flatten.** The API flattens the manifest into per-extractor row artifacts (Parquet) and stores them in S3, along with a dependency graph.
2. **Poll.** Ray pollers walk the graph tier-by-tier. A tier's batch is eligible only once every upstream tier has written its MVS rows.
3. **Submit.** When a tier's dependencies are satisfied, the poller submits a Ray job for that tier. Tiers at the same depth can run in parallel.
4. **Write.** Workers run the extractor, emit features and passthrough fields, and `MVSBatchProcessor` writes vectors and payloads to [MVS](/docs/vector-store/overview). Webhooks fire on completion.
5. **Cascade.** Downstream tiers become eligible as soon as their inputs land.

This is fundamentally a pull-based scheduler: nothing runs until the data it needs exists. It means you can re-ingest a single object at the top of the DAG and the entire chain cascades automatically.

## Cross-Tier Lineage

Every document carries the metadata needed to walk back up the chain:

```json theme={null}
{
  "document_id": "doc_chunk_42",
  "root_object_id": "obj_video_7",
  "source_collection_id": "col_segments",
  "source_document_id": "doc_segment_3",
  "processing_tier": 2,
  "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
}
```

Use the [Document Lineage API](/docs/api-reference/document-lineage/get-document-lineage) to walk the tree in either direction:

* **Down from an object** — "show me every document derived from `obj_video_7`" returns all tier-1 segments, all tier-2 chunks, all tier-3 classifications in a single tree.
* **Up from a document** — "where did `doc_chunk_42` come from?" returns the source segment, the source object, and the extractors that produced each.

The [decomposition tree visualization](/docs/api-reference/document-lineage/get-decomposition-tree-visualization) renders this as a graph. It's the primary tool for debugging "why is this document showing up in my retriever results?"

> [!TIP]
>
> When a tier reports `COMPLETED` with fewer documents than expected, the lineage tree tells you exactly where the drop happened — which input documents failed to produce output, and which extractor was running at the time.

## Decomposition Patterns

The shape of the DAG reflects the shape of your data:

| Pattern | Tier 1 | Tier 2+ | When to Use |
| - | - | - | - |
| **Fan-out** | One extractor decomposes objects into many documents (video → segments, PDF → pages). | Per-segment enrichments (embedding, classification). | Video libraries, document intelligence. |
| **Fan-in** | Multiple source collections. | One extractor joins or aggregates across them. | Multimodal catalogs that combine text + image embeddings into a single document per product. |
| **Sidecar** | Primary extractor runs. | Secondary tier adds a derived feature (e.g., a summary or classification) to the same documents. | Adding enrichments after the fact without reprocessing the primary content. |
| **Versioned swap** | `v1` extractor runs on all objects. | `v2` runs on the same objects in a parallel collection. | A/B testing a new model version before migrating retrievers. |

## Migrations Across a Chain

Upgrading a tier-1 extractor is not a local change — every downstream tier reads tier-1's output, so a schema change ripples through the chain. The safe migration pattern:

1. **Branch.** Create a parallel tier-1 collection (`col_segments_v2`) using the new extractor version. Ingest the same objects from the same bucket.
2. **Rebuild downstream.** Create tier-2 and tier-3 collections pointing at `col_segments_v2` as their source. You now have a second, fully independent chain running in parallel.
3. **Evaluate.** Point an evaluation dataset at both chains (two retrievers referencing the different feature URIs). Compare recall, precision, latency, and cost.
4. **Cut over.** Use a [namespace migration](/docs/api-reference/namespace-migrations/list-migrations) to swap retriever references from the `v1` feature URIs to the `v2` URIs atomically.
5. **Retire.** Once traffic is stable on the new chain, delete the old collections.

This is the same blue/green pattern as service deploys, scaled up to the DAG: the feature URI is the stable address, and the migration flips which version of the chain answers to it.

## Operational Gotchas

* **`field_passthrough` at every tier.** Each tier only carries forward what you explicitly pass through. If you want `metadata.category` visible at tier 3, pass it through at tier 1 and tier 2 as well. Missing passthroughs are the most common source of "why is this field gone?" questions.
* **Tier-0 `input_mappings` always read from the bucket schema; tier-N reads from the previous collection's document schema.** They look the same but the resolution rules differ.
* **Re-ingesting a tier-1 object cascades the whole chain.** Expect tier-2 and tier-3 jobs to fire automatically once tier 1 writes. This is usually what you want, but it's worth knowing before you kick off a 10M-object reprocess.
* **Deleting a tier-1 document tombstones everything downstream.** Lineage is preserved so retrievers can filter out orphans; the actual downstream features are garbage-collected asynchronously.
* **Custom plugins work at any tier.** The same `manifest.py` / `pipeline.py` / `realtime.py` contract applies whether the plugin is reading raw objects or the output of another extractor.


# Retrieval Cookbook
Source: https://docs.mixpeek.com/docs/retrieval/cookbook

Ready-to-copy, runnable multi-stage retriever configurations for common multimodal patterns

![Retrieval cookbook: ready-to-copy pipeline recipes for RAG, hybrid search, video moments, face search, and dedup](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-cookbook.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=db2f9d37741c47689a82aa5b87acc9ce)

Every recipe below is a complete, copy-paste retriever config. Create the retriever once, then execute it with runtime `inputs`.

> [!NOTE]
>
> **Stage shape.** Every stage is `{ "stage_name", "stage_type", "config": { "stage_id", "parameters" } }`. `stage_name` is your label; `stage_id` is the implementation (`feature_search`, `rerank`, `attribute_filter`, …); `stage_type` is the category (`filter`, `sort`, `reduce`, `group`, `enrich`, `apply`). See [Retrievers](/docs/retrieval/retrievers) and [Stages](/docs/retrieval/stages/overview).

> [!WARNING]
>
> **Point `feature_uri` at *your* collection's index.** Each recipe's `feature_search` uses a `feature_uri` for a specific extractor (e.g. `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding`). That string must match the collection you built — a `video_search` collection uses `mixpeek://universal_extractor@v1/gemini-embedding-2`, **not** the multimodal one. A mismatched `feature_uri` silently returns **0 results**. Find yours with `GET /v1/collections/{id}` → `vector_indexes[].feature_uri` — see [Find your feature\_uri](/docs/processing/features#find-your-feature_uri-to-search).

```bash Create + execute (every recipe uses this) theme={null}
# Create the retriever (body = one of the recipes below)
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" -d @recipe.json
# -> returns { "retriever_id": "ret_..." }

# Execute it with runtime inputs
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "inputs": { "query": "people discussing electric vehicles" } }'
```

***

## Multimodal RAG

Retrieve context for an LLM across a video's **visual** content and **transcript**, fuse with RRF, rerank, and format into a single prompt-ready context block.

```json theme={null}
{
  "retriever_name": "multimodal-rag",
  "collection_identifiers": ["col_videos"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    {
      "stage_name": "search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "fusion": "rrf",
          "final_top_k": 50,
          "searches": [
            { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 },
            { "feature_uri": "mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
          ]
        }
      }
    },
    { "stage_name": "rerank", "stage_type": "sort",
      "config": { "stage_id": "rerank", "parameters": { "inference_name": "BAAI__bge_reranker_v2_m3", "query": "{{INPUT.query}}", "top_k": 10 } } },
    { "stage_name": "prepare", "stage_type": "apply",
      "config": { "stage_id": "rag_prepare", "parameters": { "max_tokens": 8000, "output_mode": "single_context", "citation": { "style": "numbered" } } } }
  ]
}
```

**Result:** the 10 most relevant moments, reranked, formatted into a cited context block ready to paste into an LLM prompt.

***

## Hybrid Search (dense + keyword/BM25)

Fuse semantic recall (dense vectors) with exact-keyword precision (BM25) under RRF — so brand names, SKUs, and prices like `$9.99` still match. Requires a `text` payload index (see [Text Indexes](/docs/vector-store/namespaces#text-indexes-bm25)).

```json theme={null}
{
  "retriever_name": "hybrid-search",
  "collection_identifiers": ["col_docs"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    {
      "stage_name": "hybrid",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "fusion": "rrf",
          "final_top_k": 25,
          "searches": [
            { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 },
            { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "lexical": true, "top_k": 100 }
          ]
        }
      }
    }
  ]
}
```

**Result:** 25 results ranked by fused semantic + keyword relevance. Use `rrf` here — it ranks by position, immune to the cosine-vs-BM25 score-scale mismatch.

***

## Search Across Languages (cross-lingual)

Mixpeek's text and multimodal embeddings are **multilingual** (E5-Large / Gemini, 100+ languages) and every language lands in the **same** vector space. So a query in one language retrieves semantically matching content in **any** language — no translation step, no per-language index, no language parameter. Query in English, match Farsi, Mandarin, or Spanish transcripts and captions.

```json theme={null}
{
  "retriever_name": "cross-lingual-search",
  "collection_identifiers": ["col_content"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "search", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 25,
        "searches": [ { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 } ] } } }
  ]
}
```

**Result:** an English query like `"discussion of sanctions"` returns segments whose transcript or caption is in another language — ranked by meaning, not keywords. For **audio/video** collections, transcription is produced in the **source** language ([multimodal](/docs/processing/extractors/multimodal) with `run_transcription: true`, or [universal](/docs/processing/extractors/universal); Whisper detects the source language per segment and covers 99 of them) and the multilingual embedding is what makes it retrievable cross-lingually — just point `feature_uri` at that extractor's transcript embedding, e.g. `mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1`.

> [!NOTE]
>
> **Mixpeek retrieves across languages; it does not translate.** There is no translation output field — matched text comes back in its **original language**. If you need it rendered in the reader's language (e.g. original + English side by side), append an [`llm_enrich`](/docs/retrieval/stages/llm-enrich) stage that translates each result into a new field:
>
> ```json theme={null}
> { "stage_name": "translate", "stage_type": "enrich",
>   "config": { "stage_id": "llm_enrich", "parameters": {
>     "provider": "openai",
>     "model_name": "gpt-4o-mini",
>     "prompt": "Translate the following to English, preserving names and numbers. Return only the translation:\n\n{{DOC.content}}",
>     "output_field": "translation" } } }
> ```
>
> Each result then carries both `content` (original) and `translation` (English) for side-by-side display. This is the recommended path — reach for a [custom extractor](/docs/processing/custom-extractors) only if you need translation baked in at ingest time.

***

## Video Moment Localization

Search a video and collapse matching segments into a handful of seekable moments with start/end timestamps.

```json theme={null}
{
  "retriever_name": "video-moments",
  "collection_identifiers": ["col_videos"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "search", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 200,
        "searches": [ { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 200 } ] } } },
    { "stage_name": "moments", "stage_type": "reduce",
      "config": { "stage_id": "moment_group", "parameters": {
        "parent_field": "source_object_id",
        "time_field": "start_time",
        "merge_tolerance_ms": 2000,
        "max_moments_per_parent": 5,
        "score_strategy": "max",
        "output_mode": "annotated" } } }
  ]
}
```

**Result:** per video, up to 5 merged moments with timestamps you can seek to.

> [!TIP]
>
> `time_field` must point at the field holding each segment's timestamp. For a plain **text** query use the segment's own `start_time`. `query_chunks` is only present when you search with a **file** input via [query preprocessing](/docs/retrieval/stages/feature-search#query-preprocessing).

***

## Face Search (1:N identification)

Find every clip a person appears in by passing a reference face image as a `content` query against the face embedding.

```json theme={null}
{
  "retriever_name": "face-search",
  "collection_identifiers": ["col_videos"],
  "input_schema": { "reference_face_url": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "face", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 20,
        "searches": [ { "feature_uri": "mixpeek://face_identity_extractor@v1/insightface__arcface",
          "query": { "input_mode": "content", "value": "{{INPUT.reference_face_url}}" }, "top_k": 50 } ] } } }
  ]
}
```

**Result:** the 20 closest face matches. A cosine score above \~0.30 typically indicates the same person (see [Face Identity](/docs/processing/extractors/face-identity)).

This recipe chases **one** reference face. To match every new video against a maintained **roster of named identities** (a watchlist you add to and refine over time), build a reference collection instead — see [Bootstrap a Labeled Dataset](/docs/tutorials/bootstrap-labeled-dataset).

***

## Reverse Image Search + Dedup

Find visually similar items from *other* sources, deduplicated.

```json theme={null}
{
  "retriever_name": "reverse-image",
  "collection_identifiers": ["col_catalog"],
  "input_schema": {
    "image_url": { "type": "text", "required": true },
    "exclude_source": { "type": "text" }
  },
  "stages": [
    { "stage_name": "similar", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 100,
        "searches": [ { "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
          "query": { "input_mode": "content", "value": "{{INPUT.image_url}}" }, "top_k": 100, "min_score": 0.7 } ] } } },
    { "stage_name": "exclude_self", "stage_type": "filter",
      "config": { "stage_id": "attribute_filter", "parameters": {
        "field": "metadata.source", "operator": "ne", "value": "{{INPUT.exclude_source}}" } } },
    { "stage_name": "dedupe", "stage_type": "reduce",
      "config": { "stage_id": "deduplicate", "parameters": {
        "strategy": "field", "fields": ["metadata.source_url"], "keep": "first" } } }
  ]
}
```

**Result:** unique visually-similar items, excluding the original source.

***

## Search OCR Text from Scanned PDFs

Scanned and archival PDFs are OCR'd — and VLM-corrected for low-confidence blocks — by the [document graph extractor](/docs/processing/extractors/document), which embeds each text block. So OCR'd text is searchable exactly like any other text: no separate OCR index or step. Optionally keep only high-confidence blocks, or target a layout type (e.g. only `table` blocks).

```json theme={null}
{
  "retriever_name": "ocr-search",
  "collection_identifiers": ["col_scanned_docs"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "search", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 25,
        "searches": [ { "feature_uri": "mixpeek://document_graph_extractor@v1/intfloat__multilingual_e5_large_instruct",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 } ] } } },
    { "stage_name": "high_confidence", "stage_type": "filter",
      "config": { "stage_id": "attribute_filter", "parameters": {
        "field": "confidence_tag", "operator": "in", "value": ["A", "B"] } } }
  ]
}
```

**Result:** the 25 best-matching blocks (each with `page_number`, `bbox`, `object_type`), limited to high-confidence OCR. Drop the `high_confidence` stage to include fair/poor blocks, or filter `object_type` to `table`/`form` to target structured regions.

> [!TIP]
>
> For text burned into **video frames** (not PDFs), enable `run_ocr` on the [multimodal extractor](/docs/processing/extractors/multimodal) — it populates an `ocr_text` payload field you can filter on with [`attribute_filter`](/docs/retrieval/stages/attribute-filter). See also [feature\_search → Lexical (BM25)](/docs/retrieval/stages/feature-search#lexical-bm25-search).

***

## Search + Classify

Search, then attach an LLM/taxonomy label to each result in one pipeline.

```json theme={null}
{
  "retriever_name": "search-classify",
  "collection_identifiers": ["col_content"],
  "input_schema": { "query": { "type": "text", "required": true } },
  "stages": [
    { "stage_name": "search", "stage_type": "filter",
      "config": { "stage_id": "feature_search", "parameters": {
        "final_top_k": 25,
        "searches": [ { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 } ] } } },
    { "stage_name": "label", "stage_type": "enrich",
      "config": { "stage_id": "llm_enrich", "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Classify this result: {{DOC.content}}",
        "output_field": "label",
        "output_schema": { "category": "string", "confidence": "number" } } } }
  ]
}
```

**Result:** 25 results, each annotated with a structured `label`. For NSFW/safety classification use the [`classify`](/docs/retrieval/stages/classify) stage; for taxonomy-backed labeling see [Taxonomies](/docs/enrichment/taxonomies).

***

## Scan a batch of inputs

Any retriever can run many queries at once — ideal for moderating an upload queue or scanning a catalog:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute/batch" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "queries": [ {"inputs": {"image_url": "https://…/a.jpg"}}, {"inputs": {"image_url": "https://…/b.jpg"}} ] }'
```

The retriever is planned once and reused across the batch ([query optimization](/docs/retrieval/query-optimization)).

## Composing recipes

Stages are independent — add, remove, or reorder them. Common extensions: append a [`rerank`](/docs/retrieval/stages/rerank) for precision, an [`attribute_filter`](/docs/retrieval/stages/attribute-filter) for metadata scoping (the optimizer pushes it down for you), or a [`rag_prepare`](/docs/retrieval/stages/rag-prepare) to format for an LLM.


# Retrievers
Source: https://docs.mixpeek.com/docs/retrieval/retrievers

Compose stage-based search pipelines over your collections

![Mixpeek Retrievers](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-retrievers.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=3fba235556bd21f93dd44694527ba255)

Retrievers combine feature-aware search stages, structured filters, enrichment joins, and optional LLM post-processing into a single executable pipeline. Each retriever has an input schema, a list of target collections, and a deterministic set of stages executed in order.

> **[Build a retriever over your own data](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_retrievers\&utm_campaign=managed_onramp)**
>
> Create a namespace, ingest a few files, and compose your first search pipeline in the Studio — no API key setup required.

> [!NOTE]
>
> **Multi-stage retrieval is what makes Mixpeek a warehouse, not a database.** No other system offers composable filter → sort → reduce → enrich → apply pipelines over multimodal data. This is the query language for unstructured data — the equivalent of SQL for embeddings across modalities. See the [Retrieval Cookbook](/docs/retrieval/cookbook) for ready-to-copy pipeline configurations.

## Anatomy of a Retriever

```json theme={null}
{
  "retriever_name": "product_search_v2",
  "description": "Product search with enrichment and transformation",
  "collection_identifiers": ["col_products"],
  "input_schema": {
    "query_text": { "type": "text", "required": true },
    "max_price": { "type": "number" }
  },
  "stages": [
    {
      "stage_name": "enrich_catalog",
      "stage_type": "enrich",
      "config": {
        "stage_id": "document_enrich",
        "parameters": {
          "target_collection_id": "col_catalog",
          "source_field": "metadata.product_id",
          "target_field": "product_id",
          "fields_to_merge": ["name", "price", "category"],
          "output_field": "catalog_data"
        }
      }
    },
    {
      "stage_name": "fetch_billing",
      "stage_type": "apply",
      "config": {
        "stage_id": "api_call",
        "parameters": {
          "url": "https://api.stripe.com/v1/customers/{{DOC.metadata.customer_id}}",
          "method": "GET",
          "allowed_domains": ["api.stripe.com"],
          "auth": {
            "type": "bearer",
            "secret_ref": "stripe_api_key"
          },
          "output_field": "metadata.billing",
          "on_error": "skip"
        }
      }
    },
    {
      "stage_name": "reshape",
      "stage_type": "apply",
      "config": {
        "stage_id": "json_transform",
        "parameters": {
          "template": "{\"id\": \"{{DOC.document_id}}\", \"title\": \"{{DOC.metadata.title}}\", \"price\": {{DOC.catalog_data.price}}}",
          "fail_on_error": false
        }
      }
    }
  ],
  "cache_config": {
    "enabled": true,
    "ttl_seconds": 300
  }
}
```

## Minimal Working Example

The simplest retriever that performs a feature search. Note the required fields:

* **`collection_identifiers`** is required at the retriever root level when using `feature_search` stages — the search needs to know which collections to query. (Accepts collection names or IDs.)
* **`input_schema`** is a flat map of input field name → type definition (e.g. `{ "query": { "type": "text", "required": true } }`). It's required when your stages use `{{INPUT.*}}` template variables — Mixpeek validates inputs against this schema before execution. It is **not** JSON Schema: map each field name directly to its `{ "type": ..., "required": ... }` definition — do **not** wrap it in `{ "type": "object", "properties": { ... } }`. Field `type` is one of sixteen values: `string`, `text`, `number`, `integer`, `float`, `boolean`, `object`, `array`, `date`, `datetime`, `image`, `audio`, `video`, `pdf`, `excel`, `document_reference`. `required` defaults to `false`.

  A field takes more than `type` and `required`:

  | Key | Meaning |
  | - | - |
  | `type` | Required. One of the sixteen above. |
  | `required` | Defaults to `false`. |
  | `default` | Value used when the caller omits the input. |
  | `description` | Shown in generated UIs. |
  | `enum` | Restrict to a fixed set of values. |
  | `examples` | Sample values. Apps render these as example inputs, so give several diverse ones. |
  | `items` | Field definition for each element, when `type` is `array`. |
  | `properties` | Nested field definitions, when `type` is `object`. |

  `items` and `properties` nest the same field definition recursively, which is
  the one place the shape resembles JSON Schema. The top level does not: it stays
  a flat map of field name to definition.

  There is no `vector` type. Declare a precomputed embedding as `array` with `items` of type `float`: `{"embedding": {"type": "array", "items": {"type": "float"}, "required": true}}`. A `feature_search` query then reads it with `{"input_mode": "vector", "value": "{{INPUT.embedding}}"}`. See [Feature Search](/docs/retrieval/stages/feature-search).
* Each stage needs a **`stage_name`** and a **`config`** object containing the **`stage_id`** (e.g. `feature_search`) and its **`parameters`**. `stage_type` (the category, e.g. `filter`) is optional but recommended.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "basic_search",
    "collection_identifiers": ["col_my_collection"],
    "input_schema": {
      "query": { "type": "text", "required": true }
    },
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100
              }
            ],
            "final_top_k": 20
          }
        }
      }
    ]
  }'
```

Execute it:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/<retriever_id>/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"inputs": {"query": "search terms here"}, "limit": 10}'
```

## Stage Catalog

Stages are the building blocks of retriever pipelines. Each stage belongs to a **category** that defines its behavior:

| Category | Behavior | Example Use Cases |
| - | - | - |
| **filter** | Reduce the number of documents while preserving schema | Attribute filters, semantic search, hybrid search |
| **sort** | Reorder documents without changing the set | Attribute sort, score-based ordering, reranking |
| **reduce** | Collapse results into aggregated values | Top-k selection, deduplication, sampling, summarization |
| **group** | Reshape results by bucketing into logical groups | Group by field, semantic clustering |
| **apply** | Transform or restructure documents | JSON transforms, API calls, code execution, web scrape |
| **enrich** | Add knowledge to documents using AI or joins | LLM enrichment, taxonomy classification, cross-collection joins |

> [!NOTE]
>
> Retrieve the live registry with `GET /v1/retrievers/stages`. Each entry includes `stage_id`, category, icon, and parameter schema so you can dynamically build configuration UIs or validations.
>
> **Live stages:** [https://api.mixpeek.com/v1/retrievers/stages](https://api.mixpeek.com/v1/retrievers/stages)

```bash theme={null}
curl -s --request GET \
  --url "$MP_API_URL/v1/retrievers/stages" \
  --header "Authorization: Bearer $MP_API_KEY" \
  --header "X-Namespace: $MP_NAMESPACE"
```

```json theme={null}
[
  {
    "stage_id": "api_call",
    "description": "Enrich documents with external API calls",
    "category": "apply",
    "icon": "external-link"
  },
  {
    "stage_id": "json_transform",
    "description": "Transform document structure using Jinja2 templates",
    "category": "apply",
    "icon": "code"
  },
  {
    "stage_id": "external_web_search",
    "description": "Search the web using Exa AI-native search",
    "category": "apply",
    "icon": "globe"
  },
  {
    "stage_id": "document_enrich",
    "description": "Join and enrich documents with data from another collection",
    "category": "enrich",
    "icon": "link"
  },
  {
    "stage_id": "cross_compare",
    "description": "Multi-tier cross-collection comparison with classification",
    "category": "apply",
    "icon": "code-compare"
  }
]
```

### Filter Stages

Filter stages reduce the document set while preserving the document schema. Use these at the start of your pipeline to narrow down candidates.

> [!NOTE]
>
> Use `GET /v1/retrievers/stages?category=filter` to retrieve the current list of filter stages and their parameter schemas.

### Sort Stages

Sort stages reorder documents without changing the result set. Place these after filters to control ranking.

> [!NOTE]
>
> Use `GET /v1/retrievers/stages?category=sort` to retrieve the current list of sort stages and their parameter schemas.

### Reduce Stages

Reduce stages collapse results into aggregated values. Use these for deduplication, sampling, or summarization.

> [!NOTE]
>
> Use `GET /v1/retrievers/stages?category=reduce` to retrieve the current list of reduce stages and their parameter schemas.

### Group Stages

Group stages reshape results by bucketing documents into logical groups or clusters.

> [!NOTE]
>
> Use `GET /v1/retrievers/stages?category=group` to retrieve the current list of group stages and their parameter schemas.

### Apply Stages

Apply stages transform or restructure documents. Use these to reshape output, call external services, or run custom code.

| Stage ID | Description | Transformation |
| - | - | - |
| `cross_compare` | Multi-tier comparison against a reference collection | N → M (findings) or N → N (enrich) |
| `api_call` | Enrich documents with external API calls | N → N |
| `json_transform` | Transform document structure using Jinja2 templates | N → N |
| `external_web_search` | Search the web using Exa AI-native search | 0 → M (creates documents) |
| `code_execution` | Execute custom Python/TypeScript/JavaScript in sandboxes | N → M (custom logic) |

### Enrich Stages

Enrich stages add knowledge to documents using AI models, taxonomies, or cross-collection joins.

| Stage ID | Description | Transformation |
| - | - | - |
| `llm_enrich` | Generate new fields with LLM prompts | N → N (with extracted data added) |
| `taxonomy_enrich` | Classify documents against taxonomy nodes | N → N (with taxonomy labels added) |
| `document_enrich` | Join documents with data from another collection | N → N (LEFT JOIN) |

***

## Enrich Stage Details

### document\_enrich

Joins documents with data from another collection, similar to a SQL LEFT JOIN. Each input document produces exactly one output document with added fields from the target collection.

**When to use:**

* Combine data from multiple collections (e.g., products + catalog info)
* Attach user profiles, metadata, or related entities
* Denormalize data at query time

**Parameters:**

| Parameter | Required | Description |
| - | - | - |
| `target_collection_id` | Yes | Collection to join with |
| `source_field` | Yes\* | Field in current documents to match |
| `target_field` | Yes\* | Field in target collection to match against |
| `fields_to_merge` | No | Specific fields to merge (or entire document if omitted) |
| `output_field` | No | Where to place enrichment (root or nested path) |
| `retriever_id` | No | Use an existing retriever for lookup instead of direct field matching |
| `retriever_config` | No | Anonymous retriever definition for complex lookups |
| `retriever_inputs` | No | Template inputs when using retriever-based enrichment |
| `strategy` | No | `enrich` (merge fields) or `append` (add as nested object) |
| `allow_missing` | No | Keep documents without matches (default: true) |
| `when` | No | Conditional filter for selective enrichment |
| `cache_behavior` | No | `auto`, `disabled`, or `aggressive` |
| `cache_ttl_seconds` | No | Cache TTL in seconds |

\*Required for direct joins; not needed when using `retriever_id` or `retriever_config`.

**Examples:**

```json Direct Field Join theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "col_products",
      "source_field": "metadata.product_id",
      "target_field": "product_id",
      "fields_to_merge": ["name", "price", "category"],
      "output_field": "product_data"
    }
  }
}
```

```json Retriever-Based Enrichment theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "col_similar_items",
      "retriever_id": "ret_find_similar_products",
      "retriever_inputs": {
        "query": "{{DOC.description}}",
        "category": "{{DOC.metadata.category}}"
      },
      "fields_to_merge": ["name", "price", "image_url"],
      "output_field": "similar_products",
      "strategy": "append"
    }
  }
}
```

```json Conditional Enrichment theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "col_specs",
      "source_field": "metadata.sku",
      "target_field": "sku",
      "fields_to_merge": ["specifications", "warranty"],
      "output_field": "metadata.technical_details",
      "when": {
        "AND": [
          {"field": "metadata.category", "operator": "eq", "value": "electronics"}
        ]
      }
    }
  }
}
```

***

### api\_call

Enriches documents by calling external HTTP APIs. Enables integration with third-party services (Stripe, GitHub, weather APIs, etc.) to augment documents with real-time data.

> [!WARNING]
>
> **Security**: This stage makes external HTTP requests. Always use `allowed_domains` to prevent SSRF attacks. Never store credentials directly—use `auth.secret_ref` to reference vault-stored secrets.

**Parameters:**

| Parameter | Required | Description |
| - | - | - |
| `url` | Yes | API endpoint URL (supports `{DOC.field}` and `{INPUT.field}` templates) |
| `allowed_domains` | Yes | Domain allowlist for SSRF protection (never use `*`) |
| `output_field` | Yes | Dot-path where API response should be stored |
| `method` | No | HTTP method: GET, POST, PUT, PATCH, DELETE (default: GET) |
| `auth` | No | Authentication configuration (see below) |
| `headers` | No | Additional HTTP headers |
| `body` | No | Request body for POST/PUT/PATCH (JSON, supports templates) |
| `timeout` | No | Request timeout in seconds (1-60, default: 10) |
| `max_response_size` | No | Maximum response size in bytes (default: 10MB) |
| `response_path` | No | JSONPath to extract specific field from response |
| `rate_limit` | No | Rate limiting config (`requests_per_minute`, `requests_per_hour`) |
| `when` | No | Conditional filter for selective enrichment |
| `on_error` | No | Error handling: `skip`, `remove`, or `raise` (default: skip) |

**Authentication Types:**

| Type | Description | Required Fields |
| - | - | - |
| `none` | No authentication (public APIs) | — |
| `bearer` | Bearer token (OAuth 2.0, JWT) | `secret_ref` |
| `api_key` | API key in header or query param | `secret_ref`, `key`, `location` (header/query) |
| `basic` | HTTP Basic Auth (username:password in secret) | `secret_ref` |
| `custom_header` | Custom header with arbitrary name | `secret_ref`, `key` |

**Examples:**

```json Stripe Customer Lookup theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.stripe.com/v1/customers/{DOC.metadata.stripe_id}",
      "method": "GET",
      "allowed_domains": ["api.stripe.com"],
      "auth": {
        "type": "bearer",
        "secret_ref": "stripe_api_key"
      },
      "output_field": "metadata.stripe_data",
      "timeout": 10,
      "on_error": "skip"
    }
  }
}
```

```json GitHub API (Public) theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.github.com/repos/{INPUT.owner}/{INPUT.repo}",
      "method": "GET",
      "allowed_domains": ["api.github.com"],
      "output_field": "metadata.github_info",
      "response_path": "$.stargazers_count"
    }
  }
}
```

```json POST with API Key theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.example.com/v1/analyze",
      "method": "POST",
      "allowed_domains": ["api.example.com"],
      "auth": {
        "type": "api_key",
        "key": "X-API-Key",
        "location": "header",
        "secret_ref": "example_api_key"
      },
      "headers": {
        "Content-Type": "application/json"
      },
      "body": {
        "text": "{DOC.text}",
        "language": "en"
      },
      "output_field": "metadata.analysis"
    }
  }
}
```

***

### json\_transform

Applies a Jinja2 template to each document, rendering the template with full document context and replacing the document with the parsed JSON output. Use this to reformat documents for external APIs or reshape data for downstream consumers.

**Parameters:**

| Parameter | Required | Description |
| - | - | - |
| `template` | Yes | Jinja2 template string that must render to valid JSON |
| `fail_on_error` | No | Fail entire pipeline on transformation error (default: false) |

**Template Context:**

| Namespace | Description |
| - | - |
| `DOC` / `doc` | Current document fields and metadata |
| `INPUT` / `inputs` | Original query inputs from the search request |
| `CONTEXT` / `context` | Execution context (namespace\_id, internal\_id, etc.) |
| `STAGE` / `stage` | Current stage execution data |

**Examples:**

```json Field Selection theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"id\": \"{{ DOC.document_id }}\", \"content\": \"{{ DOC.text }}\", \"score\": {{ DOC.score }}}"
    }
  }
}
```

```json Conditional Fields theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"workflow_name\": \"process-asset\", \"inputs\": [{\"name\": \"id\", \"value\": \"{{ DOC.id }}\"}{% if DOC.asset_type == \"VIDEO\" %}, {\"name\": \"video\", \"value\": {\"src\": \"{{ DOC.url }}\"}}{% endif %}]}"
    }
  }
}
```

```json Array Iteration theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"title\": \"{{ DOC.title }}\", \"tags\": [{% for tag in DOC.tags %}\"{{ tag }}\"{% if not loop.last %}, {% endif %}{% endfor %}]}"
    }
  }
}
```

```json JSON Escaping theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"user_id\": \"{{ DOC.metadata.user_id }}\", \"raw_data\": {{ DOC.metadata.raw | tojson }}}"
    }
  }
}
```

***

### external\_web\_search

Performs AI-native web search using Exa's neural ranking system. Creates new documents from web search results, enabling retriever pipelines to incorporate real-time internet content.

> [!NOTE]
>
> This stage **creates** new documents (0 → M transformation) rather than enriching existing ones. Use it at the start of a pipeline or to augment internal results with external web sources.

**Parameters:**

| Parameter | Required | Description |
| - | - | - |
| `query` | Yes | Search query (supports `{INPUT.field}` and `{DOC.field}` templates) |
| `num_results` | No | Number of results (1-100, default: 10) |
| `use_autoprompt` | No | Enable Exa's query enhancement (default: true) |
| `start_published_date` | No | Filter by publication date (YYYY-MM-DD format) |
| `category` | No | Content category: `research paper`, `news`, `github`, `tweet`, `blog`, `company`, `pdf` |
| `include_text` | No | Include text snippets in results (default: true) |

**Output Schema:**

Each result becomes a document with:

* `metadata.url` – Web page URL
* `metadata.title` – Page title
* `metadata.text` – Text snippet (if `include_text=true`)
* `metadata.published_date` – Publication date (if available)
* `metadata.author` – Author name (if available)
* `metadata.search_query` – Original query used
* `metadata.search_position` – 0-indexed position in results
* `score` – Exa relevance score

**Examples:**

```json Basic Web Search theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{INPUT.query}",
      "num_results": 10,
      "include_text": true,
      "use_autoprompt": true
    }
  }
}
```

```json Research Papers theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "neural network architectures",
      "num_results": 20,
      "category": "research paper",
      "include_text": true
    }
  }
}
```

```json Recent News theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{INPUT.company_name} latest product launches",
      "num_results": 5,
      "category": "news",
      "start_published_date": "2024-10-01",
      "include_text": true
    }
  }
}
```

***

### cross\_compare

Compares source documents against a reference collection using a cascading match strategy (exact → fuzzy → semantic → visual). Each comparison produces a classified finding with a score and confidence level. Ideal for drift detection, deduplication, and compliance checking.

> [!NOTE]
>
> This stage can output either **findings** (N → M, one document per comparison) or **enriched documents** (N → N, results attached as a field). Set `output_mode` to control this behavior.

**Parameters:**

| Parameter | Required | Description |
| - | - | - |
| `reference_collection_id` | Yes | Collection containing reference documents to compare against |
| `source_field` | No | Field on source documents to extract comparison elements (default: `content`) |
| `reference_field` | No | Field on reference documents containing comparison content (default: `content`) |
| `extraction_mode` | No | Element extraction: `raw`, `lines`, `labels`, `list` (default: `raw`) |
| `match_tiers` | No | Ordered matching cascade of tier objects (default: `[{"type": "exact"}, {"type": "fuzzy", "threshold": 0.75}]`). Each tier takes a `type` (`exact`, `fuzzy`, or `vector`), a `threshold`, and a `feature_uri` for `vector` tiers |
| `classifications` | No | Score-to-label mapping rules (see below) |
| `output_mode` | No | `findings` (N→M) or `enrich` (N→N) (default: `findings`) |
| `output_field` | No | Field name for enrich mode results (default: `comparison_results`) |
| `reference_limit` | No | Max reference documents to fetch (default: 200) |
| `source_doc_type_filter` | No | Only process source docs with this doc\_type |

**Examples:**

```json Content Drift Detection theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_documentation",
      "source_field": "content",
      "reference_field": "content",
      "extraction_mode": "labels",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.75},
        {"type": "vector", "threshold": 0.85, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"},
        {"type": "vector", "threshold": 0.55, "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1"}
      ],
      "classifications": [
        {"min_score": 0.95, "label": "current"},
        {"min_score": 0.75, "label": "needs_review"},
        {"min_score": 0.0, "label": "outdated"}
      ]
    }
  }
}
```

```json Catalog Matching (Enrich Mode) theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_internal_catalog",
      "source_field": "product_name",
      "reference_field": "product_name",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.80}
      ],
      "output_mode": "enrich",
      "output_field": "catalog_match",
      "classifications": [
        {"min_score": 0.95, "label": "exact_match"},
        {"min_score": 0.80, "label": "likely_match"},
        {"min_score": 0.0, "label": "no_match"}
      ]
    }
  }
}
```

```json Deduplication Check theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_existing_corpus",
      "source_field": "content",
      "reference_field": "content",
      "extraction_mode": "lines",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.75},
        {"type": "vector", "threshold": 0.90, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"}
      ],
      "output_mode": "enrich",
      "output_field": "duplication_analysis",
      "classifications": [
        {"min_score": 0.95, "label": "duplicate"},
        {"min_score": 0.80, "label": "near_duplicate"},
        {"min_score": 0.0, "label": "unique"}
      ]
    }
  }
}
```

***

Call `GET /v1/retrievers/stages` to retrieve the latest stage metadata and parameter schemas.

## Execution Lifecycle

1. **Validate Inputs** – Mixpeek enforces the retriever’s `input_schema`.
2. **Walk Stages** – Each stage receives the current working set, runs, and outputs a new set.
3. **Apply Pagination** – `offset`, `cursor`, `scroll`, or `keyset` pagination is handled after the final stage.
4. **Return Telemetry** – Responses include `status`, `warnings`, `stage_statistics`, and `budget`.

> [!WARNING]
>
> **An empty result can mean success or silent failure. Read `warnings`.**
> A true no-match, a mismatched `feature_uri`, and a stage that timed out all
> return HTTP 200 with an empty `documents` array. You cannot tell them apart
> from `documents` alone.
>
> `warnings` is populated when a search returns zero for a reason the platform
> detected, and it is the only such signal in the response. Read it as evidence
> that something is off. Confirm the cause yourself before acting on the text,
> because a warning can name the wrong cause and prescribe an expensive remedy.
>
> Confirm cheaply first. Re-read your `feature_uri` against
> `GET /v1/collections/{id}` → `vector_indexes[].feature_uri`, and run a query
> you know works against the same collection. A collection that answers one
> query and not another is not missing its vectors.
>
> Read `warnings`, check `status`, and verify before you treat an empty result
> as an answer. A wrong `feature_uri` is the most common cause; see
> [Retriever returning zero results](/docs/resources/troubleshooting#retriever-returning-zero-results).

> [!NOTE]
>
> **Presigned media URLs are short-lived.** Media fields in a result
> (`thumbnail_url`, `video_segment_url`, `face_crop_url`, and similar) come back
> as signed URLs minted fresh on each request, and they **expire after 24 hours**.
> Do not persist or hard-code a URL value. Re-run the query for a fresh one, or
> fetch the blob through the document API.
>
> Retriever execute signs these whatever you pass for `return_presigned_urls`,
> which is not read on this path. Some values still come back as raw `gs://` or
> `s3://`: anything under `metadata`, values nested inside dicts or lists on a
> document carrying a `collection_id`, and the fields `original_url`,
> `source_blobs` and `presigned_urls`. See
> [Media URLs](/docs/vector-store/documents#media-urls) for the rule across endpoints.

Response headers include:

* `ETag` – cache validator; pair with `If-None-Match` for 304 responses.
* `Cache-Control` – TTL derived from `cache_config`.
* `X-Cache` – `HIT` or `MISS` for query-level caching.

## Filters & Templates

Structured filters support comparison operators (`eq`, `gt`, `lte`, `in`, etc.) and logical composition (`AND`, `OR`, `NOT`).

### Template Namespaces

Stages support dynamic configuration through template expressions using Jinja2 syntax. Both uppercase and lowercase namespace formats are supported and work identically:

| Namespace | Description | Examples |
| - | - | - |
| `INPUT` / `inputs` | User-provided query parameters and inputs | `{{INPUT.query_text}}`, `{{inputs.max_price}}` |
| `DOC` / `doc` | Current document fields (for per-document logic) | `{{DOC.metadata.category}}`, `{{doc.content_type}}` |
| `CONTEXT` / `context` | Execution state (budget, timing, retriever metadata) | `{{CONTEXT.budget_remaining}}`, `{{context.time_elapsed_ms}}` |
| `STAGE` / `stage` | Previous stage outputs (for cascading logic) | `{{STAGE.hybrid_search.top_score}}`, `{{stage.filter.count}}` |

> [!NOTE]
>
> Mixed usage within the same stage is supported. For example, you can use `{{INPUT.query}}` alongside `{{context.budget_remaining}}` in the same configuration.

**Conditional expressions:**

```json theme={null}
{
  "batch_size": "{{CONTEXT.budget_remaining > 50 ? 200 : 50}}",
  "field": "{{DOC.media_type == 'image' ? 'image_url' : 'video_url'}}"
}
```

**Templated batch size:**

```json theme={null}
{
  "batch_size": "{{20 * inputs.page_size}}"
}
```

## Retrievers & Caching

* **Query cache** – caches entire responses keyed by inputs, filters, pagination, and collection index signatures.
* **Stage cache** – reuse outputs of expensive stages by listing them under `cache_stage_names`.
* **Inference cache** – Engine deduplicates identical model calls.

Use `GET /v1/analytics/retrievers/{id}/cache-performance` to monitor hit rates and latency improvements.

## Pagination Options

Pass a `pagination` object. Each method takes its own keys, and `method` is required.

| Method | Keys (default) | Use case |
| - | - | - |
| `offset` | `page_size` (10), `page_number` (1) | Jump to a numbered page |
| `cursor` | `limit` (10), `cursor` (null on first call) | Stable iteration over large result sets |
| `scroll` | `limit` (100), `scroll_id` (null on first call), `scroll_ttl` (300) | Deep pagination for analytics workloads |
| `keyset` | `limit` (10), `after` (null on first call) | Paginated browsing, requires a sort key |

```json theme={null}
{ "inputs": { "query_text": "..." },
  "pagination": { "method": "offset", "page_size": 10, "page_number": 1 } }
```

> [!WARNING]
>
> The `offset` method takes `page_size` and `page_number`. It does **not** take
> `offset` or `limit`, despite the method name. Unknown keys inside `pagination`
> return a 422 that names the valid keys for the method you chose.
>
> A top-level `offset` beside `inputs` is not a request field, so it is ignored
> rather than rejected. Every page then returns the default page, which reads as
> duplicate documents rather than as an error.

Pagination is only available on the named-retriever path,
`POST /v1/retrievers/{retriever_id}/execute`. The adhoc path,
`POST /v1/retrievers/execute`, accepts `collection_identifiers`, `input_schema`,
`stages`, `inputs`, `budget_limits`, and `stream` — no `pagination` and no `limit`.
Build a named retriever when you need to page.

## Execute a Retriever

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/<retriever_id>/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "query_text": "wireless earbuds",
      "max_price": 150
    },
    "filters": {
      "field": "metadata.category",
      "operator": "eq",
      "value": "audio"
    },
    "limit": 10,
    "return_presigned_urls": true,
    "return_vectors": false
  }'
```

Response snippet:

```json theme={null}
{
  "execution_id": "exec_b8f31e0c",
  "documents": [...],
  "stage_statistics": {
    "hybrid_search": { "duration_ms": 180, "cache_hit": true },
    "filter": { "duration_ms": 8 },
    "rerank": { "duration_ms": 120 }
  },
  "budget": {
    "credits_used": 12.4,
    "credits_limit": 100,
    "time_elapsed_ms": 310
  }
}
```

> [!NOTE]
>
> `credits_used` and `credits_limit` are internal ledger units (1 credit = \$0.001). Dollar-based usage and the rate card live in [Billing](/docs/platform/billing).

## What the Platform Understood

Every execute response carries an optional `interpretation` object reporting what the
platform did with your request. It also appears on
`GET /v1/retrievers/{retriever_id}/executions/{execution_id}`.

```json theme={null}
"interpretation": {
  "provided_input_keys": ["query_text", "max_price"],
  "inferred_inputs": { "tone": "neutral" },
  "ignored_input_keys": ["brand"],
  "relaxations": [
    {
      "field": "metadata.region",
      "template": "{{INPUT.region}}",
      "reason": "the input this condition references was not provided",
      "where": "pre_filters"
    }
  ],
  "query_class": { "label": "keyword", "evidence": "2 token(s); keyword-style query" },
  "result_set": { "label": "diverse", "evidence": "10 documents with no dominant leader" }
}
```

| Key | Type | Meaning |
| - | - | - |
| `provided_input_keys` | `string[]` | The input names you sent. |
| `inferred_inputs` | `object` | Schema defaults applied because you omitted the field, as `name: value`. |
| `ignored_input_keys` | `string[]` | Inputs nothing in the pipeline consumes. They had no effect. |
| `relaxations` | `object[]` | Filter conditions removed because the input they reference was absent. |
| `query_class` | `{label, evidence}` | What kind of query you sent. |
| `result_set` | `{label, evidence}` | What shape the served result set came back in. |

`relaxations` and `inferred_inputs` are absent when empty, so read the keys defensively.

> [!WARNING]
>
> A non-empty `relaxations` means your result set is **broader** than the filter you
> wrote. The condition was dropped, not applied, and the same fact is repeated in the
> top-level `warnings` array. Provide the referenced input, or remove the condition.

`ignored_input_keys` is the counterpart. An input no stage reads is inert: the platform
accepts it, runs without it, and returns HTTP 200. Check this key before trusting that
an input took effect.

### Query and result-set labels

Both classifiers are deterministic and run on every execution at no model cost. Each
returns an `evidence` string stating why the label was chosen.

`query_class.label` is one of, checked most-specific-first:

| Label | Chosen when |
| - | - |
| `filter_only` | No non-empty string input. The run is filters and vectors only. |
| `exact_phrase` | The query is wrapped in matching quotes. |
| `identifier` | One id-shaped token containing at least one digit. |
| `question` | Ends with `?`, or opens with an interrogative such as `how` or `which`. |
| `keyword` | Three tokens or fewer. |
| `natural_language` | Four tokens or more. |

`result_set.label` is one of:

| Label | Chosen when |
| - | - |
| `zero` | 0 documents returned. |
| `single` | Exactly 1 document returned. |
| `sparse` | Fewer than `max(2, ceil(0.25 * top_k))` documents, when you requested `top_k` of 4 or more. |
| `dominated` | The top score leads the second by 25% or more of the top score. |
| `diverse` | 2 or more documents with no dominant leader. |

> [!NOTE]
>
> `sparse` cannot fire unless you send `top_k` or `limit`. Without a denominator the
> check is skipped rather than estimated, so a small result set from an unbounded
> request classifies as `dominated` or `diverse`.

The literal query text runs as provided. No spell-correction step exists, and
`interpretation` never rewrites your query. Stage-level rewrites such as
[query\_expand](/docs/retrieval/stages/query-expand) report their expansions under
`stage_statistics.stages.<stage>.metadata` instead.

### Unknown inputs fail loudly

An input name the retriever does not declare returns HTTP 400 before any stage runs.
The error names the offending field and lists every field the retriever accepts:

```json theme={null}
{
  "message": "Unknown input field(s) 'querytext'. This retriever accepts: ['query_text', 'max_price']",
  "details": {
    "unknown_fields": ["querytext"],
    "accepted_fields": ["query_text", "max_price"],
    "provided_inputs": ["querytext"],
    "hint": "Rename the unknown field(s) to one of the accepted input fields ['query_text', 'max_price'], or add them to the retriever's input_schema. Example: {\"inputs\": {\"query_text\": ..., \"max_price\": ...}}"
  }
}
```

A misspelled key aimed at a field with a default used to be discarded silently, which
returned HTTP 200 built from the default. Read `accepted_fields` to find the name you
meant. A missing **required** input returns 422, and that check fires first.

> [!NOTE]
>
> `session_id` and `user_id` are exempt. Send either on any retriever without
> declaring it in `input_schema`. They identify who and which task issued the call
> rather than what to search for, so making them a per-retriever opt-in meant an
> agent stamping `session_id` onto its executions got a `400` from every retriever
> that had no learned fusion configured.

## Batch Execution

Execute a retriever against multiple inputs in a single request. The retriever is fetched and optimized once, then executed concurrently across all queries with bounded parallelism.

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/<retriever_id>/execute/batch" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": [
      {"inputs": {"image": "https://example.com/suspect-1.jpg"}},
      {"inputs": {"image": "https://example.com/suspect-2.jpg"}},
      {"inputs": {"image": "https://example.com/suspect-3.jpg"}}
    ],
    "concurrency": 5
  }'
```

```python Python theme={null}
results = client.retrievers.execute_batch(
    retriever_id="ret_abc123",
    queries=[
        {"inputs": {"image": "https://example.com/suspect-1.jpg"}},
        {"inputs": {"image": "https://example.com/suspect-2.jpg"}},
        {"inputs": {"image": "https://example.com/suspect-3.jpg"}},
    ],
    concurrency=5,
)

for result in results:
    print(f"Query {result.query_index}: {result.status}, {len(result.documents)} docs")
```

| Parameter | Type | Default | Description |
| - | - | - | - |
| `queries` | array | required | 1–50 query objects, each with `inputs` and optional `filters` |
| `concurrency` | integer | `5` | Max parallel executions (1–20) |
| `settings` | object | `null` | Shared settings applied to every query (e.g., `limit`, `max_chunks`) |
| `stream` | boolean | `false` | Stream results via Server-Sent Events |

Response:

```json theme={null}
{
  "retriever_id": "ret_abc123",
  "total_queries": 3,
  "completed": 3,
  "failed": 0,
  "results": [
    {
      "query_index": 0,
      "status": "completed",
      "documents": [...],
      "execution_id": "exec_a1b2c3"
    }
  ],
  "total_duration_ms": 61200
}
```

### Streaming

For pipelines with LLM stages that take longer than a few seconds per query, use `"stream": true` to receive results as each query completes:

```bash theme={null}
curl -N -X POST "$MP_API_URL/v1/retrievers/<retriever_id>/execute/batch" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": [
      {"inputs": {"image": "https://example.com/suspect-1.jpg"}},
      {"inputs": {"image": "https://example.com/suspect-2.jpg"}}
    ],
    "concurrency": 5,
    "stream": true
  }'
```

The response is an SSE stream:

```
: stream-start
: keepalive

data: {"event_type":"query_complete","query_index":0,"total_queries":2,"status":"completed","documents":[...]}

data: {"event_type":"query_complete","query_index":1,"total_queries":2,"status":"completed","documents":[...]}

data: {"event_type":"batch_complete","retriever_id":"ret_abc123","total_queries":2,"completed":2,"failed":0,"total_duration_ms":61200}
```

Keepalive comments (`: keepalive`) are sent every 15 seconds to keep the connection alive through proxies. Results arrive out of order as each query finishes — use `query_index` to match results to inputs.

### When to use batch

| Scenario | Approach |
| - | - |
| Single query | `/execute` |
| 2–16 queries, fast pipeline (under 10s) | `/execute/batch` |
| 2–16 queries, LLM-heavy pipeline (over 10s) | `/execute/batch` with `"stream": true` |
| More than 16 queries | Split into batches of 16, call sequentially or from multiple threads |

## Write Results to a Collection

A retriever's results can become documents in a collection, so the next retriever, taxonomy or cluster can read them like any other collection. There are two ways to do it, and both use the same saved retriever.

### Push: one execution writes its results

Add `target_collection_id` to an execute request. After the retriever runs, each result is written to that collection as a document of its own. Any collection in the namespace can receive the writes. To search them on the source's features, write into a collection with the same feature extractor as the source, so it declares the same vector index; it fills only from the pushed writes unless you also trigger it.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/ret_abc123/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {"query": "comets"},
    "target_collection_id": "col_shortlist"
  }'
```

The response carries `written_to`:

```json theme={null}
{
  "written_to": {
    "collection_id": "col_shortlist",
    "documents_written": 25,
    "documents_skipped": 0
  }
}
```

Each written document holds:

* The result's fields as its payload, including any field a stage added (an enrich label, a `json_transform` output).
* The source document's stored vectors, so you can run `feature_search` on the new collection with the same feature URIs.
* Its lineage under `_internal.lineage`: the source document and collection, and a chain step with the `execution_id`, `retriever_id` and `retriever_version` that produced it.

Run the same retriever into the same collection again and the documents are updated in place: the document id comes from the source document and the target collection, so a source document maps to one written document. Writes are billed as document writes, at the number written. `target_collection_id` is not available with `"stream": true`, and the request answers `422` naming both fields.

### Pull: a collection fills itself from a retriever

Give a collection a `retriever_transform` that names the saved retriever, with a collection source. The collection runs the retriever over its source documents and keeps itself filled as they change. The `retriever_transform` section of [Features](/docs/processing/features) has the full shape, including the version pin.

```json theme={null}
{
  "collection_name": "astronomy-labeled",
  "source": { "type": "collection", "collection_ids": ["col_articles"] },
  "retriever_transform": { "retriever_id": "ret_abc123", "retriever_version": 1 }
}
```

## Anonymous Retrievers

Anonymous retrievers let you execute a retriever pipeline in a single API call without persisting it. This is ideal for:

* **Prototyping and experimentation** – Test stage configurations quickly without cluttering your retriever list
* **Dynamic pipelines** – Build pipelines on-the-fly based on user context or application logic
* **One-off queries** – Run complex searches that don't need to be reused
* **CI/CD testing** – Validate retriever configurations in automated tests without creating permanent resources

> [!TIP]
>
> Use anonymous retrievers during development to iterate quickly, then promote working configurations to named retrievers for production use.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_identifiers": ["col_products"],
    "input_schema": {
      "query_text": { "type": "text", "required": true }
    },
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query_text}}" },
                "top_k": 50
              }
            ],
            "final_top_k": 50
          }
        }
      },
      {
        "stage_name": "rerank",
        "stage_type": "sort",
        "config": {
          "stage_id": "rerank",
          "parameters": {
            "inference_name": "BAAI__bge_reranker_v2_m3"
          }
        }
      }
    ],
    "inputs": {
      "query_text": "noise cancelling headphones"
    }
  }'
```

The response format is identical to named retriever execution. The key difference: no `retriever_id` is created or stored.

## Publishing & Display Config

Retrievers can be published as public search interfaces hosted at `mxp.co`. Publishing requires a `display_config` that controls how the UI renders inputs, results, and styling.

Set `display_config` when creating or updating a retriever via `PATCH /v1/retrievers/{id}`:

```json theme={null}
{
  "display_config": {
    "title": "Product Search",
    "description": "Search through thousands of products",
    "logo_url": "https://example.com/logo.png",
    "theme": {
      "primary_color": "#4F46E5",
      "mode": "light"
    },
    "inputs": [
      {
        "field_name": "query_text",
        "field_schema": { "type": "text", "required": true }
      }
    ],
    "exposed_fields": ["title", "thumbnail_url", "price", "category"],
    "layout": {
      "results_layout": "grid",
      "columns": 3
    },
    "field_config": {
      "thumbnail_url": {
        "format": "image",
        "format_options": { "width": 400, "height": 300, "aspect_ratio": "16/9" }
      },
      "price": {
        "format": "number",
        "format_options": { "label": "Price", "decimals": 2, "prefix": "$" }
      }
    }
  }
}
```

**Key fields:**

| Field | Required | Description |
| - | - | - |
| `title` | Yes | Page heading for the public search interface |
| `description` | No | Subtitle/description text |
| `logo_url` | No | URL to logo image |
| `inputs` | Yes | List of input fields to render (maps to `input_schema`) |
| `exposed_fields` | Yes | Document fields shown in results (at least one required) |
| `layout` | No | Result layout: `grid`, `list`, or `table` with column count |
| `theme` | No | Primary color, dark/light mode |
| `field_config` | No | Per-field display format: `text`, `image`, `date`, `number`, `url`, `boolean`, `array` |
| `seo` | No | SEO metadata (auto-generated from title/description if omitted) |
| `template_type` | No | Built-in template: `portrait-gallery`, `media-search`, `document-search` |
| `field_mappings` | No | Maps template slots (e.g., `thumbnail`, `title`) to actual field names |

Once `display_config` is set, publish the retriever with `POST /v1/retrievers/{id}/publish`.

***

## Maintenance & Versioning

* Use `PATCH /v1/retrievers/{id}` to rename retrievers or adjust cache settings (stages and schema are immutable; create a new retriever for breaking changes).
* List retrievers with filters, search, and sort: `POST /v1/retrievers/list`.
* Retrieve execution history: `GET /v1/retrievers/{id}/executions`.
* Diagnose pipelines without executing: `POST /v1/retrievers/{id}/explain`.

## Interaction Feedback

Capture user feedback with `/v1/retrievers/interactions` to power downstream analytics, learning-to-rank, or personalized retrieval:

```json theme={null}
{
  "feature_id": "doc_abc123",
  "interaction_type": ["click", "long_view"],
  "position": 2,
  "metadata": { "duration_ms": 12000 },
  "user_id": "user_456",
  "session_id": "sess_xyz789"
}
```

## Best Practices

1. **Start narrow** – run a single search stage before adding rerankers or joins.
2. **Push filters early** – stage-level filters shrink the candidate set before expensive operations.
3. **Use JOIN strategies wisely** – `direct` for key-based joins, `retriever` for similarity joins; set `join_strategy` to control merge behavior.
4. **Enable caching** – stage caching plus query caching dramatically reduces latency for repeat queries.
5. **Monitor analytics** – use retriever analytics endpoints to optimize parameters, detect slow stages, and understand cache ROI.

Retrievers turn Mixpeek’s primitives—features, taxonomies, clusters, and models—into end-user search experiences. Configure once, execute anywhere, and evolve the pipeline with confidence.


# Agent Search
Source: https://docs.mixpeek.com/docs/retrieval/stages/agent-search

LLM-driven multi-step retrieval with iterative reasoning and tool orchestration

![Agent Search stage showing LLM reasoning loop with retriever stages as tools](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/agent-search.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=4cee3285137fc96359b6d06b82ef243b)

The Agent Search stage uses an LLM reasoning loop to orchestrate other retriever stages as callable tools. Instead of executing a fixed sequence of stages, the LLM dynamically decides which stages to invoke, with what arguments, and how many iterations to perform based on the query and intermediate results.

> [!NOTE]
>
> **Stage Category**: FILTER (Adaptive retrieval)
>
> **Transformation**: Query → LLM reasoning (1-N iterations) → Refined documents

## When to Use

| Use Case | Description |
| - | - |
| **Multi-hop queries** | Questions requiring following references across documents |
| **Iterative refinement** | Broad search followed by intelligent narrowing |
| **Complex filtering logic** | When the right filter conditions depend on query semantics |
| **Tree/hierarchy navigation** | Navigating hierarchical document indexes top-down |
| **Exploratory search** | When the best search strategy isn't known upfront |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple keyword or vector search | `feature_search` |
| Known metadata filter conditions | `attribute_filter` |
| Fixed pipeline with known stages | Chain stages directly |
| Latency-critical applications (\< 1s) | Direct stage execution |
| Cost-sensitive high-volume queries | Pre-configured pipelines |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `strategy` | string | `iterative_refinement` | Reasoning strategy (see Strategies below) |
| `stages` | list\[string] | *from strategy* | Which retriever stages the agent can invoke as tools |
| `system_prompt` | string | *from strategy* | Custom system prompt for the LLM. Supports `{{INPUT.*}}` variables |
| `max_iterations` | integer | `5` | Maximum reasoning iterations (1-20) |
| `timeout_seconds` | float | `60.0` | Total timeout for the reasoning loop (5-300s) |
| `provider` | string | *auto* | LLM provider (openai, google, anthropic, typesafe). `typesafe` runs `cluster_navigation` only |
| `model_name` | string | *auto* | Specific model to use |
| `feedback` | string | `null` | Feedback from a prior execution to inject into the agent's prompt. Use to correct or refine behavior based on previous results |
| `min_confidence` | float | `0.0` | Minimum confidence (0.0–1.0) required to accept results. Below this threshold, the agent keeps searching even if it signals done |
| `auto_strategy` | boolean | `false` | When true, a lightweight LLM call selects the best strategy automatically before the main loop. Adds \~0.5–1s latency |
| `context_source` | object | `null` | Structured context for the first prompt. `{"type": "manifest"}` gives the agent a map of the retriever's collections (see Namespace Map below) |

## Strategies

| Strategy | Default Tools | Best For |
| - | - | - |
| `iterative_refinement` | feature\_search, attribute\_filter | Progressive narrowing of results |
| `multi_hop` | feature\_search, attribute\_filter, llm\_filter | Following cross-document references |
| `tree_navigation` | feature\_search, attribute\_filter | Hierarchical index traversal |
| `cluster_navigation` | attribute\_filter | Walking a cluster hierarchy top-down to the clusters that answer the query |
| `full_catalog` | *every registered stage* | Open-ended queries where the model should compose the pipeline |
| `custom` | *user-specified* | Full control over tools and prompt |

## Namespace Map

Set `context_source` to `{"type": "manifest"}` to show the agent what it can search before it calls a tool. The stage reads the namespace manifest, keeps only the retriever's collections, and adds what is in the data:

* each collection's id, document count, extractor and fields
* which collection it is derived from, and the lineage field that links a document to its parent
* the most common values of its indexed fields and taxonomy output fields
* the clusters over it, with each group's label and size

The agent uses the map to choose a collection, a filter value or a cluster. Collections outside the retriever never appear in it. Building the map costs a few database reads before the first LLM call; `context_source.built_ms` in the stage metadata reports the time.

```json theme={null}
{
  "stage_id": "agent_search",
  "parameters": {
    "strategy": "iterative_refinement",
    "context_source": {"type": "manifest"},
    "max_iterations": 5
  }
}
```

The same map is available outside a retriever from [`GET /v1/manifest/export?detail=stats`](/docs/operations/manifests#describe-what-is-in-a-namespace).

## Cluster Navigation

`cluster_navigation` searches a cluster hierarchy the way a person browses one. A cluster run writes one centroid document per cluster, with its `label`, `summary` and `parent_cluster_id`. The stage reads those centroids, starts at the top-level clusters, and asks the model which child cluster most likely contains documents that answer the query. It repeats this until it reaches leaf clusters. It then returns the member documents of the best leaf clusters.

The walk is a beam search, and every decision goes to a [decision model](/docs/processing/jev#swap-the-decision-model). Each decision returns a probability for every child cluster. The stage keeps the best `beam_width` paths and ranks them by the geometric mean of their decision probabilities, so shallow and deep leaves compare fairly. All paths at one level are asked in one request.

`model_name` picks the decision model. Leave it unset to use the deployment default, which is `jev-latest`. Jev returns calibrated probabilities. A generative model such as `gemini-2.5-flash-lite` reports its own confidence for each option, and the stage marks those as uncalibrated in `probabilities_calibrated`.

A parent cluster is shown to the model as the leaves a descent through it can reach, such as `Sports: Skateboarding / Surfing`. Labeling picks one label per cluster, so a parent that holds species and landforms can come back labeled `Species`. Listing the leaves keeps a query about a mountain from skipping that branch.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `cluster_navigation.centroid_collection_ids` | list\[string] | retriever collections | Collections that hold the centroid documents, usually the cluster's output collection |
| `cluster_navigation.beam_width` | integer | `3` | Paths kept at each level, 1 to 8. `1` is greedy descent |
| `cluster_navigation.max_leaf_clusters` | integer | `3` | Leaf clusters whose members are returned, 1 to 20 |
| `cluster_navigation.membership_field` | string | `cluster_id` | Field on member documents that names their cluster |
| `cluster_navigation.label_field` | string | `label` | Centroid field shown to the model as the option name |
| `cluster_navigation.summary_field` | string | `summary` | Centroid field shown to the model as the option description |
| `cluster_navigation.parent_field` | string | `parent_cluster_id` | Centroid field that names the parent cluster |

The stage matches members to clusters by `membership_field`, so cluster ids must be unique within the collections it reads. Point `centroid_collection_ids` at one cluster's output collection. If several cluster runs write `cl_0` into the same collection, members of every `cl_0` come back.

When the stage receives documents from an earlier stage, it keeps the ones in the chosen clusters and preserves their order. When it runs first, it reads the members from the collections. Each returned document carries `cluster_path` and `cluster_path_score`. Stage metadata includes `chosen_clusters` (each leaf with its path and score), `navigation_trace` (the candidates and probabilities at every level), `model_used` and `probabilities_calibrated`.

```json theme={null}
{
  "stage_name": "browse_clusters",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "cluster_navigation",
      "model_name": "jev-latest",
      "cluster_navigation": {
        "centroid_collection_ids": ["col_cluster_output"],
        "beam_width": 3,
        "max_leaf_clusters": 2
      }
    }
  }
}
```

See [Hierarchical search over clusters](/docs/tutorials/hierarchical-cluster-search) for an end-to-end example.

## Available Tools (Stages)

The agent can invoke any registered retriever stage as a tool. Each stage is presented to the LLM with a simplified parameter schema:

| Tool | What the LLM Sees | Typical Use |
| - | - | - |
| `feature_search` | Semantic similarity search with query and top\_k | Vector search |
| `attribute_filter` | Filter by field, operator, and value | Metadata filtering |
| `llm_filter` | Filter by natural language criteria | Semantic filtering |
| `rerank` | Re-score documents by relevance to query | Result refinement |
| `llm_enrich` | Generate new fields using LLM analysis | Information extraction |
| `taxonomy_enrich` | Classify documents against a taxonomy | Categorization |

## Configuration Examples

```json Iterative Refinement theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "iterative_refinement",
      "max_iterations": 5,
      "timeout_seconds": 30.0
    }
  }
}
```

```json Custom Strategy with Specific Tools theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "custom",
      "stages": ["feature_search", "attribute_filter", "rerank"],
      "system_prompt": "You are a document retrieval agent. Start with feature_search to find initial matches, then use attribute_filter to narrow by metadata, and rerank for final ordering.",
      "max_iterations": 4,
      "timeout_seconds": 45.0
    }
  }
}
```

```json Multi-Hop with LLM Filter theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "multi_hop",
      "max_iterations": 7,
      "timeout_seconds": 60.0
    }
  }
}
```

```json Tree Navigation theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "tree_navigation",
      "max_iterations": 10,
      "timeout_seconds": 90.0
    }
  }
}
```

```json With Feedback + Confidence Threshold theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "iterative_refinement",
      "max_iterations": 5,
      "timeout_seconds": 45.0,
      "min_confidence": 0.7,
      "feedback": "Previous results included marketing documents — focus only on technical specs."
    }
  }
}
```

```json Auto-Strategy Selection theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "auto_strategy": true,
      "max_iterations": 5,
      "timeout_seconds": 60.0
    }
  }
}
```

```json Full Catalog (model composes pipeline) theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "full_catalog",
      "max_iterations": 8,
      "timeout_seconds": 90.0
    }
  }
}
```

> [!NOTE]
>
> `full_catalog` hands the LLM every registered filter/sort/rerank/enrich stage as a tool and lets it decide the pipeline shape per query. Higher cost and latency than the pinned strategies — use it when composition quality matters more than cost.

> [!TIP]
>
> Use the `custom` strategy when you need precise control over which tools the agent can access and how it should reason. The built-in strategies provide sensible defaults for common patterns.

> [!TIP]
>
> Use `feedback` to create a human-in-the-loop refinement cycle: execute a retriever, review the results, then re-execute with feedback describing what was wrong. The agent adjusts its strategy based on your corrections. Pair with `min_confidence` to ensure the agent keeps searching until results meet your quality bar.

## How It Works

1. **Strategy selection**: If `auto_strategy` is enabled, a lightweight LLM call picks the best strategy for the query
2. **Prompt assembly**: The system prompt is built from the strategy defaults, with `feedback` prepended if provided
3. **Reasoning loop**: Each iteration, the LLM receives the query, available tools, and a budget note showing remaining iterations and seconds
4. **Tool execution**: The LLM calls retriever stages as tools. Results are summarized and fed back
5. **Confidence gating**: The LLM calls `finish_search` to declare done with a confidence score (0.0–1.0). If confidence is below `min_confidence`, the loop continues
6. **Context compression**: When the conversation grows long, older messages are replaced with a compact working-memory summary to stay within context limits
7. **Return**: Accumulated results and metadata (confidence, summary, reasoning trace) are returned

Each tool call creates a sub-state execution of the target stage, inheriting namespace and collection context from the parent. Non-empty results from each iteration replace the previous working set. If a refinement query returns zero results, the previous results are preserved.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 2-30s (depends on iterations and sub-stages) |
| **Memory** | O(N) per iteration result set |
| **Cost** | LLM API calls + sub-stage costs per iteration |
| **Complexity** | O(iterations \* sub-stage complexity) |

## Common Pipeline Patterns

### Agent as First Stage

```json theme={null}
[
  {
    "stage_name": "agent_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "agent_search",
      "parameters": {
        "strategy": "iterative_refinement",
        "stages": ["feature_search", "attribute_filter"],
        "max_iterations": 3
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 10
      }
    }
  }
]
```

### Complex Query Decomposition

```json theme={null}
[
  {
    "stage_name": "agent_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "agent_search",
      "parameters": {
        "strategy": "custom",
        "stages": ["feature_search", "attribute_filter", "llm_filter"],
        "system_prompt": "Break down the user's complex query into sub-queries. Use feature_search for each sub-query, attribute_filter to narrow by metadata, and llm_filter to validate relevance. Combine the best results.",
        "max_iterations": 5,
        "timeout_seconds": 45.0
      }
    }
  },
  {
    "stage_name": "deduplicate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "deduplicate",
      "parameters": {
        "strategy": "field",
        "fields": ["document_id"]
      }
    }
  }
]
```

## Response Metadata

The stage returns execution metadata in `stage_statistics`:

| Field | Description |
| - | - |
| `iterations_used` | Number of reasoning iterations completed |
| `max_iterations` | Maximum iterations that were configured |
| `timeout_hit` | Whether the timeout was reached |
| `strategy` | Strategy that was used |
| `auto_strategy_used` | Whether auto-strategy routing was active |
| `stages_invoked` | List of tool calls made (name + arguments + result count) |
| `total_llm_cost` | Total LLM API cost for the reasoning loop |
| `model_used` | LLM model that was used |
| `final_confidence` | Agent's self-reported confidence in results (0.0–1.0) |
| `final_summary` | Agent's explanation of what was found and why it stopped |
| `reasoning_trace` | Per-iteration trace: agent reasoning, tool calls, durations |
| `context_source` | With `context_source.type` `manifest`: collections and clusters in the map, its size in characters, and `built_ms`. On failure, `error` says why and the search runs without the map |

## Error Handling

| Error | Behavior |
| - | - |
| Sub-stage execution fails | Error message returned to LLM; it can retry or try a different approach |
| Timeout reached | Returns results accumulated so far (graceful degradation) |
| Max iterations reached | Runs a recovery call to capture confidence, then returns accumulated results |
| Unknown stage in tools list | Stage is skipped (logged as warning) |
| LLM returns no tool calls | Runs a recovery call to capture confidence, then returns current results |
| Empty collection | Returns empty result set (no error) |

## Related

* [Feature Search](/docs/retrieval/stages/feature-search) - Vector similarity search (commonly used as agent tool)
* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Metadata filtering (commonly used as agent tool)
* [LLM Filter](/docs/retrieval/stages/llm-filter) - Natural language filtering
* [Rerank](/docs/retrieval/stages/rerank) - Result re-scoring by relevance


# Aggregate
Source: https://docs.mixpeek.com/docs/retrieval/stages/aggregate

Compute statistical aggregations and metrics across document results

![Aggregate stage showing statistical computations on search results](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/aggregate.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=13e879d5297474a4835667f5ff221382)

The Aggregate stage computes statistical aggregations across your search results, including counts, sums, averages, min/max values, and custom metrics. This is useful for analytics, faceted search, and understanding result distributions.

> [!NOTE]
>
> **Stage Category**: REDUCE (Aggregates results)
>
> **Transformation**: N documents → aggregation results + optional documents

> [!NOTE]
>
> **Aggregating a whole collection?** Use
> `POST /v1/collections/{collection_identifier}/documents/aggregate`. It runs the
> same functions over every matching document, not just the current pipeline's
> results. It returns **exact** counts at any collection size, and `count`,
> `count_distinct`, `sum`, `avg`, `min` and `max` stream without a row cap. Pair
> it with the `is_null` filter operator for field-presence checks, such as
> counting documents missing a `from_collection` ancestor across 180k+ assets.
>
> The two request bodies share `group_by` (an array of objects), `function` and
> `alias`. The endpoint adds `filters`, `having`, `unwind` and `range_buckets`, and
> calls its sort direction `sort_direction` where this stage uses `sort_order`.
>
> `aggregations` is required, with at least one entry. Every entry needs
> `function` and `alias`. `field` is required for every function except `count`.

> [!WARNING]
>
> **Read the results from the stage statistics.** This stage consumes the documents
> and, unless `include_documents` is `true`, emits none. The groups and metrics are in
> `stage_statistics.stages.<your stage_name>.metadata.aggregations`.
> A response with an empty `documents` array and a `null` top-level `facets` is
> what a working aggregate stage returns. Top-level `facets` is filled only by the
> `feature_search` stage's own `facets` option. See [Output](#output).

## When to Use

| Use Case | Description |
| - | - |
| **Faceted search** | Count documents by category |
| **Analytics** | Compute metrics across results |
| **Price ranges** | Min/max/avg calculations |
| **Distributions** | Understand result characteristics |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Grouping with full docs | `group_by` |
| Simple counting | Use search facets |
| Per-document stats | `code_execution` |
| LLM-based analysis | `summarize` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `aggregations` | array | *Required* | Aggregation operations to compute. See below. |
| `group_by` | array | *none* | Fields to group by, as `{"field": "metadata.category", "alias": "category"}`. Omit `alias` and the last path segment names the group key. Omit `group_by` for one global group. |
| `sort_by` | string | *none* | An `alias` from `aggregations` to sort groups by. Applies only with `group_by`. |
| `sort_order` | string | `desc` | `asc` or `desc`. |
| `limit` | integer | *none* | Keep the first N groups after sorting (1 to 1000). Applies only with `group_by`. |
| `include_documents` | boolean | `false` | Pass the input documents through to the next stage. |

Each entry in `aggregations`:

| Field | Type | Description |
| - | - | - |
| `function` | string | *Required.* One of the functions below. |
| `alias` | string | *Required.* The key for this metric in the output. Unique within the stage. |
| `field` | string | The field path to aggregate, in dot notation. Required for every function except `count`. |
| `field_b` | string | Second field for `co_occurrence` and `correlation`. |
| `percentile_value` | number | Percentile to compute, 0 to 100. Only for `percentile`; defaults to 50. |
| `top_k` | integer | Values or pairs to keep for `frequency` and `co_occurrence`, 1 to 100. Defaults to 10. |

A field path resolves against the document the way the [Group By](/docs/retrieval/stages/group-by)
stage does. `score` and `document_id` are read directly. A bare name such as
`client_id` is looked up on the document, then in `metadata`. A dotted path such as
`metadata.price` walks the document.

## Aggregation Functions

| Function | Description | Example |
| - | - | - |
| `count` | Number of documents in the group | Total results |
| `count_distinct` | Number of unique values of `field` | Unique authors |
| `sum` | Sum of a numeric field | Total revenue |
| `avg` | Average of a numeric field | Average price |
| `min` | Smallest value of a numeric field | Lowest price |
| `max` | Largest value of a numeric field | Highest price |
| `first` | Value from the first document in the group | Top-ranked title |
| `last` | Value from the last document in the group | Lowest-ranked title |
| `collect` | List of all values, duplicates kept | All tags |
| `collect_distinct` | List of unique values | Distinct tags |
| `percentile` | One percentile of a numeric field | P50, P95 |
| `stddev` | Standard deviation (sample) | Score spread |
| `variance` | Variance (sample) | Price volatility |
| `frequency` | Top-K values with counts and percent | Top categories |
| `co_occurrence` | Top-K pairs of two fields with counts | Brand and category pairs |
| `correlation` | Pearson correlation of two numeric fields | Price vs. rating |

## Configuration Examples

```json Basic Aggregations theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "count", "alias": "total"},
        {"function": "avg", "field": "metadata.price", "alias": "avg_price"},
        {"function": "min", "field": "metadata.price", "alias": "min_price"},
        {"function": "max", "field": "metadata.price", "alias": "max_price"}
      ]
    }
  }
}
```

```json Grouped Aggregations theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "group_by": [{"field": "metadata.category", "alias": "category"}],
      "aggregations": [
        {"function": "count", "alias": "count"},
        {"function": "avg", "field": "metadata.rating", "alias": "avg_rating"}
      ],
      "sort_by": "count",
      "sort_order": "desc",
      "limit": 10
    }
  }
}
```

```json Metrics With The Documents Kept theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "count", "alias": "total"},
        {"function": "count_distinct", "field": "metadata.brand", "alias": "brand_count"},
        {"function": "frequency", "field": "metadata.brand", "alias": "top_brands", "top_k": 5}
      ],
      "include_documents": true
    }
  }
}
```

```json Percentile Analysis theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "percentile", "field": "score", "percentile_value": 95, "alias": "p95_score"},
        {"function": "avg", "field": "score", "alias": "avg_score"}
      ]
    }
  }
}
```

```json Multi-Field Aggregations theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "count", "alias": "total_docs"},
        {"function": "sum", "field": "metadata.views", "alias": "total_views"},
        {"function": "avg", "field": "metadata.engagement_rate", "alias": "avg_engagement"},
        {"function": "count_distinct", "field": "metadata.author_id", "alias": "unique_authors"}
      ]
    }
  }
}
```

## Output

The groups and metrics come back in the stage's statistics:

```json theme={null}
{
  "documents": [],
  "facets": null,
  "stage_statistics": {
    "stages": {
      "aggregate": {
        "input_count": 25,
        "output_count": 0,
        "metadata": {
          "num_documents_in": 25,
          "num_groups": 2,
          "aggregations": [
            {"group": {"category": "electronics"}, "metrics": {"count": 15, "avg_rating": 4.2}},
            {"group": {"category": "clothing"}, "metrics": {"count": 10, "avg_rating": 4.5}}
          ]
        }
      }
    }
  }
}
```

`stages` is keyed by the `stage_name` you gave the stage. `output_count` is `0` when
`include_documents` is `false`, because the stage consumes the documents. With
`include_documents: true` it equals `input_count`, and the documents continue to
the next stage unchanged.

Without `group_by` there is one entry and `group` is `null`:

```json theme={null}
{"group": null, "metrics": {"total": 150, "avg_price": 49.99}}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-50ms |
| **Memory** | O(groups × aggregations) |
| **Cost** | Free |
| **Scalability** | Efficient for large result sets |

## Common Pipeline Patterns

### Search with Facets

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 500 }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "count", "alias": "total"},
          {"function": "frequency", "field": "metadata.category", "alias": "categories", "top_k": 10},
          {"function": "percentile", "field": "metadata.price", "percentile_value": 50, "alias": "median_price"}
        ],
        "include_documents": true
      }
    }
  }
]
```

### Analytics Pipeline

```json theme={null}
[
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "AND": [
            {"field": "metadata.date", "operator": "gte", "value": "2024-01-01"},
            {"field": "metadata.status", "operator": "eq", "value": "published"}
          ]
        }
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "count", "alias": "total_published"},
          {"function": "sum", "field": "metadata.views", "alias": "total_views"},
          {"function": "avg", "field": "metadata.engagement", "alias": "avg_engagement"},
          {"function": "percentile", "field": "metadata.views", "percentile_value": 90, "alias": "p90_views"}
        ],
        "group_by": [{"field": "metadata.author", "alias": "author"}]
      }
    }
  }
]
```

### E-Commerce Product Analytics

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 500 }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.in_stock",
        "operator": "eq",
        "value": true
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "count", "alias": "available_products"},
          {"function": "min", "field": "metadata.price", "alias": "lowest_price"},
          {"function": "max", "field": "metadata.price", "alias": "highest_price"},
          {"function": "avg", "field": "metadata.rating", "alias": "avg_rating"},
          {"function": "count_distinct", "field": "metadata.brand", "alias": "brand_count"}
        ],
        "group_by": [{"field": "metadata.category", "alias": "category"}],
        "include_documents": true
      }
    }
  }
]
```

## Aggregation Details

### Count

```json theme={null}
{"function": "count", "alias": "total"}
```

Counts documents in the group. No field required.

### Sum / Avg / Min / Max

```json theme={null}
{"function": "avg", "field": "metadata.price", "alias": "average_price"}
```

Reads numeric values only. A document whose value is missing or not a number is skipped. With no numeric values the metric is `null`.

### Count Distinct

```json theme={null}
{"function": "count_distinct", "field": "metadata.author", "alias": "unique_authors"}
```

Counts unique values exactly. A list-valued field contributes each element.

### Percentile

```json theme={null}
{"function": "percentile", "field": "score", "percentile_value": 95, "alias": "p95_score"}
```

Returns one percentile per entry, interpolated between neighbouring values. Add one entry per percentile you want.

## Statistical Aggregation Examples

```json Percentile Analysis theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "percentile", "field": "score", "alias": "median_score", "percentile_value": 50},
        {"function": "percentile", "field": "score", "alias": "p90_score", "percentile_value": 90},
        {"function": "stddev", "field": "score", "alias": "score_spread"},
        {"function": "variance", "field": "score", "alias": "score_variance"}
      ]
    }
  }
}
```

```json Frequency Distribution theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "frequency", "field": "category", "alias": "top_categories", "top_k": 10},
        {"function": "frequency", "field": "brand", "alias": "top_brands", "top_k": 5}
      ]
    }
  }
}
```

```json Co-occurrence Analysis theme={null}
{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "co_occurrence", "field": "brand", "field_b": "category", "alias": "brand_category_pairs"},
        {"function": "correlation", "field": "price", "field_b": "rating", "alias": "price_rating_corr"}
      ]
    }
  }
}
```

### Statistical Output Examples

**Frequency** returns value counts with percentages:

```json theme={null}
{
  "top_categories": [
    {"value": "electronics", "count": 45, "percent": 30.0},
    {"value": "clothing", "count": 38, "percent": 25.3},
    {"value": "books", "count": 22, "percent": 14.7}
  ]
}
```

**Co-occurrence** returns field pair counts:

```json theme={null}
{
  "brand_category_pairs": [
    {"field_a": "Nike", "field_b": "footwear", "count": 12, "percent": 8.0},
    {"field_a": "Nike", "field_b": "apparel", "count": 8, "percent": 5.3}
  ]
}
```

**Correlation** returns the Pearson coefficient (-1 to 1):

```json theme={null}
{
  "price_rating_corr": 0.342156
}
```

## Error Handling

| Error | Behavior |
| - | - |
| Missing field | The document is skipped for that aggregation |
| Non-numeric value in a numeric aggregation | The value is skipped; with no numeric values the metric is `null` |
| Empty results | `count` is `0`; other metrics are `null` or an empty list |
| Unknown `function`, or a missing `alias` | Rejected when the retriever is created |
| Insufficient data for stddev, variance, correlation | Returns `null` (needs 2 or more values) |

## Related

* [Group By](/docs/retrieval/stages/group-by) - Group with full documents
* [Sample](/docs/retrieval/stages/sample) - Statistical sampling
* [Summarize](/docs/retrieval/stages/summarize) - LLM-powered analysis


# API Call
Source: https://docs.mixpeek.com/docs/retrieval/stages/api-call

Enrich documents with external API calls (Stripe, GitHub, weather APIs, etc.)

![API Call stage showing documents enriched with external HTTP API data](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/api-call.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=f08353c8c864984c827a74cee6f93088)

The API Call stage enriches documents by calling external HTTP APIs. This enables integration with third-party services like Stripe, GitHub, weather APIs, and more to augment documents with real-time data.

> [!NOTE]
>
> **Stage Category**: APPLY (1-1 Enrichment)
>
> **Transformation**: N documents → N documents (same count, expanded schema)

## When to Use

| Use Case | Description |
| - | - |
| **Customer data lookup** | Enrich with Stripe billing data, CRM info |
| **Repository info** | Fetch GitHub commit stats, stars |
| **Real-time data** | Add weather, stock prices, currency rates |
| **Data validation** | Verify addresses, phone numbers |
| **External context** | Lookup additional context from any API |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Untrusted/user-provided URLs | Major security risk (SSRF) |
| API credentials can't be secured | Use organization secrets vault |
| High-volume enrichment | Rate limits apply |
| Time-critical responses | Network latency adds 100-500ms |
| Internal-only APIs behind firewalls | Use `sql_lookup` for databases |

## Parameters

### Required Parameters

| Parameter | Type | Description |
| - | - | - |
| `url` | string | API endpoint URL. Supports `{DOC.field}` and `{INPUT.field}` templates. |
| `allowed_domains` | string\[] | Domain allowlist for SSRF protection. **Never use `*`**. |
| `output_field` | string | Dot-path where API response should be stored (e.g., `metadata.stripe_data`). |

### Optional Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `method` | string | `GET` | HTTP method: `GET`, `POST`, `PUT`, `PATCH`, `DELETE` |
| `auth` | object | `null` | Authentication configuration (see below) |
| `headers` | object | `{}` | Additional HTTP headers (supports templates) |
| `body` | object | `null` | Request body for POST/PUT/PATCH (JSON) |
| `timeout` | integer | `10` | Request timeout in seconds (1-60) |
| `max_response_size` | integer | `10485760` | Maximum response size in bytes (default: 10MB) |
| `response_path` | string | `null` | JSONPath to extract specific field from response |
| `rate_limit` | object | `null` | Rate limiting per domain |
| `when` | object | `null` | Conditional filter for selective enrichment |
| `on_error` | string | `skip` | Error handling: `skip`, `remove`, or `raise` |

## Authentication Types

**Bearer Token**

**OAuth 2.0, JWT tokens** - Most modern APIs (GitHub, OpenAI, Stripe)

```json theme={null}
{
  "auth": {
    "type": "bearer",
    "secret_ref": "stripe_api_key"
  }
}
```

Adds header: `Authorization: Bearer {secret_value}`

**API Key**

**Header or query parameter** - Weather APIs, Maps, etc.

```json theme={null}
{
  "auth": {
    "type": "api_key",
    "key": "X-API-Key",
    "location": "header",
    "secret_ref": "weather_api_key"
  }
}
```

| Field | Description |
| - | - |
| `key` | Header name or query param name |
| `location` | `header` (recommended) or `query` |
| `secret_ref` | Secret name in vault |

**Basic Auth**

**HTTP Basic Authentication** - Legacy systems

```json theme={null}
{
  "auth": {
    "type": "basic",
    "secret_ref": "basic_credentials"
  }
}
```

Secret format: `username:password`

Adds header: `Authorization: Basic {base64(username:password)}`

**Custom Header**

**Non-standard auth headers**

```json theme={null}
{
  "auth": {
    "type": "custom_header",
    "key": "X-Custom-Auth",
    "secret_ref": "custom_token"
  }
}
```

Adds header: `{key}: {secret_value}`

## Error Handling

| Strategy | Behavior | Best For |
| - | - | - |
| `skip` | Keep document unchanged | Optional enrichment |
| `remove` | Remove document from results | Mandatory enrichment |
| `raise` | Fail entire pipeline | Debugging, critical failures |

## Configuration Examples

```json Stripe Customer Lookup theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.stripe.com/v1/customers/{DOC.metadata.stripe_id}",
      "method": "GET",
      "allowed_domains": ["api.stripe.com"],
      "auth": {
        "type": "bearer",
        "secret_ref": "stripe_api_key"
      },
      "output_field": "metadata.stripe_data",
      "timeout": 10,
      "on_error": "skip"
    }
  }
}
```

```json GitHub Repository Stats theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.github.com/repos/{INPUT.owner}/{INPUT.repo}",
      "method": "GET",
      "allowed_domains": ["api.github.com"],
      "output_field": "metadata.github_info",
      "response_path": "$.stargazers_count"
    }
  }
}
```

```json POST with JSON Body theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.example.com/v1/analyze",
      "method": "POST",
      "allowed_domains": ["api.example.com"],
      "auth": {
        "type": "api_key",
        "key": "X-API-Key",
        "location": "header",
        "secret_ref": "example_api_key"
      },
      "headers": {
        "Content-Type": "application/json"
      },
      "body": {
        "text": "{DOC.content}",
        "language": "en"
      },
      "output_field": "metadata.analysis"
    }
  }
}
```

```json Conditional Enrichment theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.stripe.com/v1/customers/{DOC.metadata.customer_id}",
      "method": "GET",
      "allowed_domains": ["api.stripe.com"],
      "auth": {
        "type": "bearer",
        "secret_ref": "stripe_api_key"
      },
      "output_field": "metadata.billing",
      "when": {
        "AND": [
          {"field": "metadata.is_premium", "operator": "eq", "value": true}
        ]
      },
      "on_error": "skip"
    }
  }
}
```

```json Rate Limited with JSONPath theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.weatherapi.com/v1/current.json?q={DOC.metadata.location}",
      "method": "GET",
      "allowed_domains": ["api.weatherapi.com"],
      "auth": {
        "type": "api_key",
        "key": "key",
        "location": "query",
        "secret_ref": "weather_api_key"
      },
      "output_field": "metadata.weather",
      "response_path": "$.current",
      "rate_limit": {
        "requests_per_minute": 60
      },
      "timeout": 5
    }
  }
}
```

## Security

> [!WARNING]
>
> **SSRF Protection Required**
>
> This stage makes external HTTP requests which can be exploited for Server-Side Request Forgery attacks. **Always use `allowed_domains`** to whitelist permitted domains.

### Security Best Practices

1. **Never use `*` in allowed\_domains** - Explicitly list each domain
2. **Never store credentials in configuration** - Always use `auth.secret_ref` to reference vault secrets
3. **Set rate limits** - Prevent abuse and excessive costs
4. **Use HTTPS** - HTTP URLs are automatically upgraded
5. **Audit configurations** - Review before deployment to prevent data exfiltration

### Storing Secrets

```bash theme={null}
# Create a secret in the organization vault
curl -X POST "$MP_API_URL/v1/organizations/secrets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "stripe_api_key",
    "secret_value": "mxp_sk_..."
  }'
```

Then reference in configuration:

```json theme={null}
{
  "auth": {
    "type": "bearer",
    "secret_ref": "stripe_api_key"
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency per request** | 100-500ms (network dependent) |
| **Timeout range** | 1-60 seconds |
| **Max response size** | 10MB (configurable) |
| **Parallelization** | Documents processed concurrently |

## Template Variables

URLs, headers, and body values support template variables:

| Namespace | Description | Example |
| - | - | - |
| `DOC` | Current document fields | `{DOC.metadata.customer_id}` |
| `INPUT` | Query inputs | `{INPUT.api_version}` |

## Related

* [SQL Lookup](/docs/retrieval/stages/sql-lookup) - For database enrichment
* [Document Enrich](/docs/retrieval/stages/document-enrich) - For collection joins
* [Organization Secrets](/docs/api-reference/organization-secrets/create-secret) - Managing API credentials


# Attribute Filter
Source: https://docs.mixpeek.com/docs/retrieval/stages/attribute-filter

Filter documents by metadata field conditions with boolean logic support

![Attribute Filter stage showing metadata-based document filtering](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/attribute-filter.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=8d3295fc1e60c387cdab775efc4fb901)

The Attribute Filter stage filters documents based on metadata field conditions. It supports simple single-field filtering and complex boolean logic (AND/OR/NOT). When used as a first stage, it retrieves documents directly from the database; when used after other stages, it filters in-memory results.

> [!NOTE]
>
> **Stage Category**: FILTER (Reduces document set)
>
> **Transformation**: N documents → M documents (where M ≤ N, based on conditions)

## When to Use

| Use Case | Description |
| - | - |
| **Metadata filtering** | Filter by status, category, date, etc. |
| **Post-search refinement** | Narrow semantic search results |
| **Access control** | Filter by user permissions |
| **Business logic** | Active items, published content |
| **Initial retrieval** | Fetch documents by attributes (no embeddings) |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Semantic similarity | `feature_search` |
| Content-based filtering | `llm_filter` |
| Complex text matching | `feature_search` with text features |
| Scoring/ranking | Use sort stages after filtering |

## Parameters

### Simple Mode

Use for single-condition filtering:

| Parameter | Type | Default | Description |
| - | - | - | - |
| `field` | string | *Required* | Field path to filter on |
| `operator` | string | `eq` | Comparison operator |
| `value` | any | *Required* | Value to compare against |
| `case_insensitive` | boolean | `false` | Case-insensitive string matching |

### Boolean Mode

Use for complex multi-condition filtering:

| Parameter | Type | Default | Description |
| - | - | - | - |
| `conditions` | object | *Required* | Boolean condition object |
| `batch_size` | integer | `100` | Documents per batch (first-stage only) |

### Scan limit

Both modes accept:

| Parameter | Type | Default | Description |
| - | - | - | - |
| `max_scan_documents` | integer | `10000` | First stage only: the most documents scanned per collection, up to `100000`. Raise it when a grouped or aggregated result needs every matching document. Ignored when a later stage needs vectors. |

When the scan stops with matches left, the response carries a warning and a `first_stage_scan_documents` entry in `limits`.

### Natural-Language Mode

Supply a plain-English filter and let an LLM compile it to a structured `conditions` tree at runtime. Useful when the filter shape is derived from user input or when field names are unknown in advance.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `natural_language` | string | *Required* | Plain-English filter description (e.g. `"active premium customers with priority >= 5"`) |
| `field_hints` | list\[string] | `null` | Optional list of allowed field paths to constrain the LLM's output |

> [!NOTE]
>
> NL mode rides the wave — better LLMs produce better filter trees. Takes precedence over simple-mode fields; ignored if `conditions` is also set. Adds one LLM call per execution (\~200-500ms).

## Supported Operators

| Operator | Description | Example Value |
| - | - | - |
| `eq` | Equals | `"active"`, `42`, `true` |
| `ne` | Not equals | `"deleted"` |
| `gt` | Greater than | `100` |
| `gte` | Greater than or equal | `4.5` |
| `lt` | Less than | `50` |
| `lte` | Less than or equal | `10` |
| `in` | In array | `["tech", "science"]` |
| `nin` | Not in array | `["spam", "deleted"]` |
| `contains` | Contains substring | `"guide"` |
| `starts_with` | Starts with | `"intro"` |
| `ends_with` | Ends with | `".pdf"` |
| `regex` | Regular expression | `"^[A-Z].*"` |
| `exists` | Field exists | `true` or `false` |
| `is_null` | Field is null | `true` or `false` |
| `text` | Case-insensitive substring match, the same as `contains` | `"machine learning"` |
| `words` | Every word appears as a whole word, matched on its English stem | `"drinking coffee"` |
| `phrase` | Exact phrase — matches the words in order, word-boundary safe | `"machine learning model"` |
| `geo_radius` | Within N meters of a center point | `{"center": {"lat": 40.758, "lon": -73.9855}, "radius": 15000}` |
| `geo_bounding_box` | Inside a lat/lon box (antimeridian-safe) | `{"top_left": {"lat": 42.0, "lon": -74.5}, "bottom_right": {"lat": 40.5, "lon": -72.5}}` |
| `geo_polygon` | Inside an arbitrary polygon (≥3 points) | `{"exterior": {"points": [{"lat": 41.2, "lon": -74.5}, ...]}}` |

> [!NOTE]
>
> The three `geo_*` operators take an **object** as their `value`, not a scalar. See [Geospatial filtering](#geospatial-filtering) for the field format and exact value shapes.

## Configuration Examples

```json Simple Equality theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.status",
      "operator": "eq",
      "value": "published"
    }
  }
}
```

```json Numeric Comparison theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.price",
      "operator": "lte",
      "value": 100
    }
  }
}
```

```json In Array theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.category",
      "operator": "in",
      "value": ["electronics", "computers", "phones"]
    }
  }
}
```

```json Contains Substring theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.title",
      "operator": "contains",
      "value": "guide",
      "case_insensitive": true
    }
  }
}
```

```json Field Exists theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.premium_features",
      "operator": "exists",
      "value": true
    }
  }
}
```

```json Dynamic Value theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.category",
      "operator": "eq",
      "value": "{{INPUT.category}}"
    }
  }
}
```

```json Natural Language theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "natural_language": "published blog posts from 2024 that are not archived",
      "field_hints": ["metadata.status", "metadata.published_at", "metadata.archived"]
    }
  }
}
```

## Boolean Conditions

For complex filtering, use the `conditions` parameter with AND/OR/NOT logic:

```json AND Conditions theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "AND": [
          {"field": "metadata.status", "operator": "eq", "value": "active"},
          {"field": "metadata.in_stock", "operator": "eq", "value": true},
          {"field": "metadata.price", "operator": "lte", "value": 500}
        ]
      }
    }
  }
}
```

```json OR Conditions theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "OR": [
          {"field": "metadata.category", "operator": "eq", "value": "featured"},
          {"field": "metadata.rating", "operator": "gte", "value": 4.5}
        ]
      }
    }
  }
}
```

```json NOT Conditions theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "NOT": [
          {"field": "metadata.status", "operator": "eq", "value": "deleted"},
          {"field": "metadata.spam", "operator": "eq", "value": true}
        ]
      }
    }
  }
}
```

```json Nested Boolean Logic theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "AND": [
          {"field": "metadata.status", "operator": "eq", "value": "active"},
          {
            "OR": [
              {"field": "metadata.category", "operator": "eq", "value": "premium"},
              {"field": "metadata.featured", "operator": "eq", "value": true}
            ]
          },
          {
            "NOT": [
              {"field": "metadata.region", "operator": "eq", "value": "restricted"}
            ]
          }
        ]
      }
    }
  }
}
```

## Geospatial filtering

Filter documents by geographic location. Three operators — `geo_radius`,
`geo_bounding_box`, and `geo_polygon` — restrict results to documents whose
location falls within a circle, a box, or an arbitrary polygon. They work in
both simple mode and inside boolean `conditions`, exactly like any other
operator.

### The location field

A document is geo-filterable when it carries a location field. By convention the
field is named `location`, and Mixpeek resolves it wherever it lives — it tries
top-level `location`, then `metadata.location`, then `_internal.metadata.location`
— so you write `"field": "location"` regardless of how the document was ingested.

A location value can be either of two forms:

| Form | Example | Notes |
| - | - | - |
| Object | `{"lat": 40.758, "lon": -73.9855}` | Recommended. `lat` in `[-90, 90]`, `lon` in `[-180, 180]`. |
| GeoJSON array | `[-73.9855, 40.758]` | **Longitude first**, then latitude — the GeoJSON order. |

> [!WARNING]
>
> **GeoJSON arrays are `[lon, lat]`, not `[lat, lon]`.** This is the single most
> common geo footgun. `[-73.9855, 40.758]` is Manhattan; `[40.758, -73.9855]` is
> an invalid point (latitude 40.758 is fine, but longitude 40.758 places it in
> the Indian Ocean, and if either value exceeds its range the point is dropped).
> Prefer the `{"lat": ..., "lon": ...}` object form, which is unambiguous.

A document with **no** location field never matches a geo filter (it is treated
as a non-match, not an error). If a document stores a **list** of points, it
matches when **any** point satisfies the filter.

### The three operators

```json geo_radius theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "location",
      "operator": "geo_radius",
      "value": {
        "center": {"lat": 40.758, "lon": -73.9855},
        "radius": 15000
      }
    }
  }
}
```

```json geo_bounding_box theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "location",
      "operator": "geo_bounding_box",
      "value": {
        "top_left": {"lat": 42.0, "lon": -74.5},
        "bottom_right": {"lat": 40.5, "lon": -72.5}
      }
    }
  }
}
```

```json geo_polygon theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "location",
      "operator": "geo_polygon",
      "value": {
        "exterior": {
          "points": [
            {"lat": 41.2, "lon": -74.5},
            {"lat": 41.2, "lon": -73.5},
            {"lat": 40.4, "lon": -73.5},
            {"lat": 40.4, "lon": -74.5}
          ]
        }
      }
    }
  }
}
```

| Operator | `value` shape | Semantics |
| - | - | - |
| `geo_radius` | `{"center": {"lat", "lon"}, "radius": <meters>}` | Within `radius` **meters** of `center`. Distance is haversine great-circle (mean Earth radius 6,371,000 m). |
| `geo_bounding_box` | `{"top_left": {"lat", "lon"}, "bottom_right": {"lat", "lon"}}` | Inside the box. Antimeridian-safe: if `top_left.lon > bottom_right.lon` the box wraps 180° (e.g. Fiji). |
| `geo_polygon` | `{"exterior": {"points": [{"lat", "lon"}, ...]}}` | Inside the exterior ring by ray-casting. Needs **≥ 3** points. |

> [!NOTE]
>
> `radius` is always in **meters** — `15000` is 15 km, not 15,000 km.

### Combine with a selective filter (recommended)

Geo operators are evaluated by a payload scan — they are correct but **not
index-accelerated** (there is no geo postings index yet). To keep the candidate
set small and latency low, pair a geo condition with a selective non-geo
condition (a brand, category, or status equality) inside an `AND`:

```json theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "AND": [
          {"field": "metadata.category", "operator": "eq", "value": "store"},
          {
            "field": "location",
            "operator": "geo_radius",
            "value": {"center": {"lat": 40.758, "lon": -73.9855}, "radius": 5000}
          }
        ]
      }
    }
  }
}
```

The selective `eq` narrows the set first; the geo condition is then applied to a
small candidate list. Don't rely on a geo-only filter over a large collection
for sub-millisecond latency.

### Validation

Malformed geo specs fail **loudly at request time** with a message that names
the expected shape — you get a clear `400`, not a silent empty result set.
Common mistakes that are rejected:

* `value` is not an object (e.g. a bare number or string)
* `center` / `top_left` / `bottom_right` missing `lat` or `lon`, or out of range
* `radius` missing or negative
* `geo_polygon` with fewer than 3 points, or a point missing `lat`/`lon`

### Worked example

Six documents, each an office with a `location` (one has none):

| Document | Location | lat, lon |
| - | - | - |
| Manhattan | NYC | 40.7580, -73.9855 |
| Brooklyn | NYC | 40.6782, -73.9442 |
| Hartford | CT | 41.7658, -72.6734 |
| Boston | MA | 42.3601, -71.0589 |
| Philadelphia | PA | 39.9526, -75.1652 |
| Virtual | *(no location)* | — |

Applying each operator from the examples above:

| Filter | Matches | Why |
| - | - | - |
| `geo_radius` center=Manhattan, 15 km | **Manhattan, Brooklyn** | Brooklyn is \~8 km away; Hartford (\~160 km), Boston (\~306 km), Philadelphia (\~130 km) are outside. |
| `geo_bounding_box` (42.0,-74.5)→(40.5,-72.5) | **Manhattan, Brooklyn, Hartford** | Boston's lon -71.06 is east of -72.5; Philadelphia's lat 39.95 is south of 40.5. |
| `geo_polygon` NYC-metro rectangle | **Manhattan, Brooklyn** | Hartford's lat 41.77 and Philadelphia's lat 39.95 fall outside the ring. |

In every case the Virtual office (no location) is excluded.

## First-Stage vs Later-Stage Behavior

| Position | Behavior |
| - | - |
| **First stage** | Fetches documents directly from the database (up to `max_scan_documents` per collection, 10,000 by default) |
| **Later stage** | Filters in-memory results from previous stages |

### First-Stage Example

When no documents exist in the pipeline yet:

```json theme={null}
[
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.status",
        "operator": "eq",
        "value": "published",
        "batch_size": 100
      }
    }
  }
]
```

### Later-Stage Example

After semantic search:

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.query}}"}],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.in_stock",
        "operator": "eq",
        "value": true
      }
    }
  }
]
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-20ms |
| **First-stage limit** | 10,000 documents per collection by default, up to 100,000 with `max_scan_documents` |
| **In-memory filtering** | \< 5ms for 1,000 docs |
| **Index utilization** | Uses indexes when available |

> [!TIP]
>
> For best performance with `feature_search`, use pre-filters in the search stage instead of a separate `attribute_filter` stage. Pre-filters are applied at the vector index level.

## Common Pipeline Patterns

### Search + Filter + Sort

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 100
          }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "AND": [
            {"field": "metadata.status", "operator": "eq", "value": "active"},
            {"field": "metadata.category", "operator": "in", "value": "{{INPUT.categories}}"}
          ]
        }
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.created_at",
        "direction": "desc"
      }
    }
  }
]
```

### Attribute-Only Retrieval (No Embeddings)

```json theme={null}
[
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "AND": [
            {"field": "metadata.type", "operator": "eq", "value": "product"},
            {"field": "metadata.featured", "operator": "eq", "value": true}
          ]
        },
        "batch_size": 50
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.priority",
        "direction": "desc"
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "count": 10
      }
    }
  }
]
```

### Multi-Stage Filtering

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.query}}"}],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.category",
        "operator": "eq",
        "value": "{{INPUT.category}}"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 20
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.in_stock",
        "operator": "eq",
        "value": true
      }
    }
  }
]
```

## Comparison: attribute\_filter vs feature\_search Pre-Filters

| Aspect | attribute\_filter (stage) | feature\_search pre-filters |
| - | - | - |
| **When applied** | After search results | During vector search |
| **Performance** | Good | Best (index-level) |
| **Use case** | Post-filtering, first-stage retrieval | Always when possible |
| **Flexibility** | Can be placed anywhere | Only with feature\_search |

**Recommendation:** When filtering during semantic search, prefer pre-filters in `feature_search`. Use `attribute_filter` for:

* Post-search refinement based on previous stage outputs
* First-stage attribute-only retrieval
* Dynamic filters that depend on earlier stage results

## Error Handling

| Error | Behavior |
| - | - |
| Field not found | Document excluded (treated as no match) |
| Invalid operator | Stage fails with error |
| Type mismatch | Attempts type coercion, then excludes |
| Invalid regex | Stage fails with error |
| Malformed geo spec | Request rejected with a clear error naming the expected shape (fails loud, not a silent empty result) |
| Document has no location field | Document excluded from geo filters (treated as no match) |

## Related Stages

* [Feature Search](/docs/retrieval/stages/feature-search) - Semantic vector search
* [LLM Filter](/docs/retrieval/stages/llm-filter) - Content-based LLM filtering
* [Sort Attribute](/docs/retrieval/stages/sort-attribute) - Sort by field values


# Cluster
Source: https://docs.mixpeek.com/docs/retrieval/stages/cluster

Group documents by embedding similarity into semantic clusters

![Cluster stage showing semantic grouping of search results](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/cluster.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=2e8b5f4a714a1beefd9733f7fbb0acbd)

The Cluster stage groups documents based on embedding similarity, creating semantic clusters of related content. This helps organize search results into meaningful groups and discover themes within your results.

> [!NOTE]
>
> **Stage Category**: GROUP (Groups documents)
>
> **Transformation**: N documents → K clusters with documents

## When to Use

| Use Case | Description |
| - | - |
| **Theme discovery** | Find topics within search results |
| **Result organization** | Group similar items together |
| **Deduplication** | Find near-duplicate content |
| **Exploration** | Understand result diversity |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Grouping by field value | `group_by` |
| Removing duplicates | `deduplicate` |
| Pre-defined categories | `taxonomy_enrich` |
| Single representative | `sample` per group |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `algorithm` | string | `hdbscan` | Clustering algorithm (see below) |
| `n_clusters` | integer | `null` | Number of clusters, 2 to 100. Used by `kmeans`, `agglomerative`, `spectral` and `gaussian_mixture`, and ignored by `hdbscan` and `dbscan`. When omitted for an algorithm that needs it, the stage picks `min(8, N/10)` with a floor of 2, where N is the number of documents |
| `min_cluster_size` | integer | `5` | Minimum documents per cluster, 2 to 100. Used by `hdbscan` and `dbscan`. Lowered automatically for small result sets |
| `feature_uri` | string | *auto* | Embedding to cluster; auto-inherited from a prior `feature_search`, otherwise the first vector field found on the documents |
| `output_mode` | string | `clusters` | `clusters`, `labeled`, or `representatives` |
| `include_centroids` | boolean | `true` | Include each cluster's centroid vector in `clusters` output. The centroid is returned only when the query also requests vectors |
| `max_members_per_cluster` | integer | `50` | Maximum members listed per cluster in `clusters` output, 1 to 500. Members are ordered closest to the centroid first |

## Clustering Algorithms

| Algorithm | Description | Best For |
| - | - | - |
| `hdbscan` | Density-based clustering, chooses the cluster count and marks noise (default) | Unknown cluster count |
| `kmeans` | K-means clustering | Fixed number of clusters |
| `dbscan` | Density-based with fixed epsilon | Noisy data with clear density separation |
| `agglomerative` | Hierarchical clustering | Nested clusters |
| `spectral` | Graph-based clustering | Non-convex cluster shapes |
| `gaussian_mixture` | Probabilistic clustering | Overlapping clusters |

Documents that `hdbscan` or `dbscan` treat as noise get the cluster ID `-1`. The `clusters` and `representatives` output modes leave them out, and `labeled` keeps them with `cluster_id: -1`.

## Configuration Examples

```json Basic Clustering theme={null}
{
  "stage_name": "cluster",
  "stage_type": "reduce",
  "config": {
    "stage_id": "cluster",
    "parameters": {
      "algorithm": "kmeans",
      "n_clusters": 5
    }
  }
}
```

```json KMeans With Known K theme={null}
{
  "stage_name": "cluster",
  "stage_type": "reduce",
  "config": {
    "stage_id": "cluster",
    "parameters": {
      "algorithm": "kmeans",
      "n_clusters": 8
    }
  }
}
```

```json Density-Based Clustering theme={null}
{
  "stage_name": "cluster",
  "stage_type": "reduce",
  "config": {
    "stage_id": "cluster",
    "parameters": {
      "algorithm": "hdbscan",
      "min_cluster_size": 3
    }
  }
}
```

```json Custom Embedding Field theme={null}
{
  "stage_name": "cluster",
  "stage_type": "reduce",
  "config": {
    "stage_id": "cluster",
    "parameters": {
      "algorithm": "kmeans",
      "n_clusters": 6,
      "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
    }
  }
}
```

```json Fine-Grained Clustering theme={null}
{
  "stage_name": "cluster",
  "stage_type": "reduce",
  "config": {
    "stage_id": "cluster",
    "parameters": {
      "n_clusters": 15,
      "min_cluster_size": 2,
      "algorithm": "kmeans"
    }
  }
}
```

```json Representative Documents theme={null}
{
  "stage_name": "cluster",
  "stage_type": "reduce",
  "config": {
    "stage_id": "cluster",
    "parameters": {
      "algorithm": "kmeans",
      "n_clusters": 8,
      "output_mode": "representatives"
    }
  }
}
```

## How Clustering Works

1. **Extract Embeddings**: Get embedding vectors from each document
2. **Apply Algorithm**: Run the clustering algorithm (e.g., k-means)
3. **Assign Documents**: Each document gets a cluster ID
4. **Compute Centroids**: Calculate cluster centers
5. **Format Output**: Return cluster documents, labeled documents, or one representative per cluster, depending on `output_mode`

The stage needs at least 3 documents with an embedding. With fewer, it returns no results and reports the reason under `skipped` in the stage metadata.

## Output Schema

The shape depends on `output_mode`.

### `clusters` (default)

The stage returns one document per cluster. Each has a generated `document_id`, the `collection_id` of the source documents, and a `score` equal to the member count. The members and the centroid are carried in `payload`.

```json theme={null}
{
  "document_id": "doc_a1b2c3",
  "collection_id": "col_articles",
  "score": 12.0,
  "cluster_id": 0,
  "member_count": 12,
  "representative_id": "doc_123",
  "payload": {
    "members": [
      {
        "document_id": "doc_123",
        "content": "Introduction to neural networks...",
        "cluster_distance": 0.15
      }
    ],
    "centroid": null
  }
}
```

`representative_id` is the member closest to the centroid. `centroid` holds the vector only when `include_centroids` is `true` and the query requests vectors. Otherwise it is `null`. At most `max_members_per_cluster` members are listed, and `member_count` counts all of them.

### `labeled`

The stage returns the original documents with two fields added.

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Introduction to neural networks...",
  "cluster_id": 0,
  "cluster_distance": 0.15
}
```

`cluster_distance` is the distance to the cluster centroid, so lower is closer. It is absent when the algorithm returns no centroid, and for noise documents (`cluster_id: -1`).

### `representatives`

The stage returns one document per cluster, the one closest to the centroid.

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Introduction to neural networks...",
  "cluster_id": 0,
  "cluster_distance": 0.15,
  "is_representative": true
}
```

The stage metadata reports `algorithm`, `n_clusters`, `n_documents_in`, `n_documents_out`, `n_valid_embeddings` and `n_invalid_documents`.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 50-200ms |
| **Memory** | O(N × embedding\_dim) |
| **Cost** | Free |
| **Scalability** | Built for result sets of 10 to 500 documents |

> [!WARNING]
>
> Clustering large document sets (over 1,000) can be slow. Consider pre-filtering or sampling before clustering.

## Common Pipeline Patterns

### Search + Cluster

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "cluster",
    "stage_type": "reduce",
    "config": {
      "stage_id": "cluster",
      "parameters": {
        "algorithm": "kmeans",
        "n_clusters": 5
      }
    }
  }
]
```

### Cluster + Sample Representatives

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 200 }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "cluster",
    "stage_type": "reduce",
    "config": {
      "stage_id": "cluster",
      "parameters": {
        "algorithm": "kmeans",
        "n_clusters": 10,
        "output_mode": "labeled"
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "strategy": "stratified",
        "stratify_by": "cluster_id",
        "min_per_stratum": 2
      }
    }
  }
]
```

### Theme Discovery Pipeline

```json theme={null}
[
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.date",
        "operator": "gte",
        "value": "2024-01-01"
      }
    }
  },
  {
    "stage_name": "cluster",
    "stage_type": "reduce",
    "config": {
      "stage_id": "cluster",
      "parameters": {
        "algorithm": "hdbscan",
        "min_cluster_size": 5
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "google",
        "model_name": "gemini-2.5-flash-lite",
        "prompt": "Summarize the main themes found in these clusters:\n\n{{DOCUMENTS}}"
      }
    }
  }
]
```

### Diverse Results Pipeline

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "cluster",
    "stage_type": "reduce",
    "config": {
      "stage_id": "cluster",
      "parameters": {
        "algorithm": "kmeans",
        "n_clusters": 5,
        "output_mode": "representatives"
      }
    }
  }
]
```

## Choosing n\_clusters

| Result Size | Recommended Clusters |
| - | - |
| \< 50 docs | 3-5 clusters |
| 50-200 docs | 5-10 clusters |
| 200-500 docs | 8-15 clusters |
| 500+ docs | 10-20 clusters |

> [!TIP]
>
> Start with fewer clusters and increase if clusters are too broad. Use `hdbscan` if you don't know the optimal number.

## Error Handling

| Condition | Behavior |
| - | - |
| Document has no embedding | Left out of clustering |
| Fewer than 3 documents with embeddings | No results, with the reason under `skipped` in the metadata |
| No embedding can be identified | Request fails. Set `feature_uri` or run `feature_search` first |
| Clustering service fails | Request fails with a clustering error |

## Related

* [Group By](/docs/retrieval/stages/group-by) - Group by field values
* [Sample](/docs/retrieval/stages/sample) - Select representatives
* [MMR](/docs/retrieval/stages/mmr) - Diversity in ranking
* [Deduplicate](/docs/retrieval/stages/deduplicate) - Remove duplicates


# Cross Compare
Source: https://docs.mixpeek.com/docs/retrieval/stages/cross-compare

Multi-tier cross-collection content matching with configurable classification

![Cross Compare stage showing multi-tier matching cascade between source and reference collections](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/cross-compare.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=469f62624aa2d633d1f989b6fcec9910)

The Cross Compare stage compares source documents against a reference collection using a cascading match strategy: exact → fuzzy → semantic → visual. Each match is classified using configurable rules, enabling drift detection, deduplication, and compliance checking workflows.

> [!NOTE]
>
> **Stage Category**: APPLY (Cross-collection comparison)
>
> **Transformation**: N documents → M finding documents (`findings` mode) or N documents → N enriched documents (`enrich` mode)

## When to Use

| Use Case | Description |
| - | - |
| **Content drift detection** | Compare video UI against documentation to find outdated content |
| **Product catalog matching** | Match supplier products against internal catalog |
| **Content deduplication** | Check new content against existing corpus |
| **Compliance checking** | Verify content against requirements or standards |
| **Cross-reference validation** | Validate labels, features, or terms across sources |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple field joins | `document_enrich` |
| External API enrichment | `api_call` |
| Single-collection filtering | `attribute_filter` or `feature_search` |
| Semantic similarity search | `feature_search` |

## Parameters

> [!NOTE]
>
> These tables are generated from `CrossCompareParameters`, the model the stage
> validates against. `GET /v1/retrievers/stages` serves the same schema, so
> Studio's stage form and any agent reading the API see exactly these fields.

### Core Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `reference_collection_id` | string | *Required* | Collection containing the reference documents to compare against |
| `source_field` | string | `content` | Field on source documents holding the comparison elements. A string is used as-is; a list is treated as pre-extracted elements. |
| `reference_field` | string | `content` | Field on reference documents holding the comparison content |
| `extraction_mode` | string | `raw` | How to split the field into elements: `raw`, `lines`, `labels`, or `list` |

### Matching Configuration

`match_tiers` is a list of tier objects, not a list of strings. Each entry
carries its own threshold, so different tiers can accept at different scores.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `match_tiers` | MatchTierConfig\[] | `[{"type": "exact"}, {"type": "fuzzy", "threshold": 0.75}]` | Ordered cascade. Matching stops at the first tier that succeeds. |

Fields on each `MatchTierConfig`:

| Field | Type | Default | Description |
| - | - | - | - |
| `type` | string | *Required* | `exact` (case-insensitive equality), `fuzzy` (SequenceMatcher ratio), or `vector` (embedding search) |
| `threshold` | float | `0.75` | Minimum score for this tier to accept a match |
| `feature_uri` | string | `null` | Feature URI to search, for `vector` tiers |
| `vectors` | object\[] | `null` | Several weighted vectors combined into one score, for `vector` tiers |
| `classifications` | ClassificationRule\[] | `null` | Rules for this tier only. Falls back to the global `classifications`. |
| `min_words` | integer | `1` | Skip elements shorter than this, for `vector` tiers |
| `cross_page_fallback` | boolean | `true` | If an aligned-page search finds nothing, retry across all reference pages |

### Classification

| Parameter | Type | Default | Description |
| - | - | - | - |
| `classifications` | ClassificationRule\[] | See below | Score-to-label rules, evaluated in order, first match wins |
| `no_match_label` | string | `no_match` | Label applied when no tier matches |

A `ClassificationRule` is `{"min_score": float, "label": string}`, both required.

Default rules:

```json theme={null}
[
  {"min_score": 0.95, "label": "exact_match"},
  {"min_score": 0.85, "label": "close_match"},
  {"min_score": 0.65, "label": "partial_match"},
  {"min_score": 0.0, "label": "no_match"}
]
```

### Output Configuration

| Parameter | Type | Default | Description |
| - | - | - | - |
| `output_mode` | string | `findings` | `findings` (N-to-M, one document per comparison) or `enrich` (1-to-1, results attached to the source document) |
| `output_field` | string | `comparison_results` | Field the results land on, in `enrich` mode |

### Reference & Source Configuration

| Parameter | Type | Default | Description |
| - | - | - | - |
| `reference_limit` | integer | `200` | Maximum reference documents to fetch |
| `reference_doc_type` | string | `null` | Only use reference docs of this `doc_type`. `null` fetches all. |
| `source_location_field` | string | `start_time` | Field holding the source location reference (timestamp, page) |
| `source_doc_type_field` | string | `doc_type` | Field that names the source document's type |
| `source_doc_type_filter` | string | `null` | Only process source docs whose type matches. `null` processes all. |
| `filter_generic_labels` | boolean | `true` | In `labels` mode, drop generic UI labels (`Save`, `Cancel`, `Next`) that cannot meaningfully differ |
| `signal_fields` | object | `{}` | Map of signal name to source payload field, e.g. `{"transcript": "content"}` |

### Alignment

Used when source documents should be compared against a specific reference page
rather than the whole collection.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `alignment_mode` | string | `none` | `none` for a flat comparison, `grouped` to align source groups to reference pages |
| `alignment_group_field` | string | `vlm_page_context` | Field to group source documents by |
| `alignment_feature_uri` | string | `null` | Feature URI used for the alignment vector search |
| `alignment_limit` | integer | `5` | Reference pages considered for each alignment group |
| `alignment_query_fields` | string\[] | `["vlm_page_context", "content"]` | Source fields combined into the alignment query text |

## Extraction Modes

**raw**

Use the field value as a single element. Best for comparing whole content blocks.

```json theme={null}
{"extraction_mode": "raw"}
```

**lines**

Split by newlines. Each line becomes a comparison element. Useful for step-by-step instructions or structured text.

```json theme={null}
{"extraction_mode": "lines"}
```

**labels**

Extract UI/feature labels via pattern matching. Identifies instruction patterns ("Click **Settings**"), em-dash separators ("Label — description"), and action labels ("Configure X").

Generic labels like "Save", "Cancel", "Next" are filtered by default.

```json theme={null}
{"extraction_mode": "labels", "filter_generic_labels": true}
```

**list**

Field is already a list of elements. Used directly without extraction.

```json theme={null}
{"extraction_mode": "list"}
```

## Matching Cascade

Tiers run in the order you list them and matching stops at the first success:

```
For each source element:
  ├─ exact:  case-insensitive string equality        → score 1.0
  ├─ fuzzy:  SequenceMatcher ratio  ≥ tier threshold
  └─ vector: embedding similarity   ≥ tier threshold
```

There are three tier types. Text and image matching are both `vector` tiers,
distinguished by the `feature_uri` you point the tier at, so a visual comparison
is a `vector` tier on an image feature rather than a separate tier type.

If no tier matches, the element gets `match_tier: "none"` and the
`no_match_label` classification.

## Configuration Examples

```json Content Drift Detection theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_documentation",
      "source_field": "content",
      "reference_field": "content",
      "extraction_mode": "labels",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.75},
        {"type": "vector", "threshold": 0.85, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"},
        {"type": "vector", "threshold": 0.55, "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1"}
      ],
      "source_doc_type_filter": "scene",
      "source_location_field": "start_time",
      "classifications": [
        {"min_score": 0.95, "label": "current"},
        {"min_score": 0.75, "label": "needs_review"},
        {"min_score": 0.0, "label": "outdated"}
      ]
    }
  }
}
```

```json Product Catalog Matching theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_internal_catalog",
      "source_field": "product_name",
      "reference_field": "product_name",
      "extraction_mode": "raw",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.80}
      ],
      "output_mode": "enrich",
      "output_field": "catalog_match",
      "classifications": [
        {"min_score": 0.95, "label": "exact_match"},
        {"min_score": 0.80, "label": "likely_match"},
        {"min_score": 0.0, "label": "no_match"}
      ]
    }
  }
}
```

```json Content Deduplication theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_existing_corpus",
      "source_field": "content",
      "reference_field": "content",
      "extraction_mode": "lines",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.75},
        {"type": "vector", "threshold": 0.90, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"}
      ],
      "output_mode": "enrich",
      "output_field": "duplication_analysis",
      "classifications": [
        {"min_score": 0.95, "label": "duplicate"},
        {"min_score": 0.80, "label": "near_duplicate"},
        {"min_score": 0.0, "label": "unique"}
      ]
    }
  }
}
```

```json Compliance Checking theme={null}
{
  "stage_name": "cross_compare",
  "stage_type": "apply",
  "config": {
    "stage_id": "cross_compare",
    "parameters": {
      "reference_collection_id": "col_requirements",
      "source_field": "content",
      "reference_field": "requirement_text",
      "extraction_mode": "lines",
      "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.70},
        {"type": "vector", "threshold": 0.80, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"}
      ],
      "output_mode": "findings",
      "classifications": [
        {"min_score": 0.90, "label": "compliant"},
        {"min_score": 0.70, "label": "partial"},
        {"min_score": 0.0, "label": "non_compliant"}
      ]
    }
  }
}
```

## Output Schema

### Findings Mode

Each comparison produces a finding document:

```json theme={null}
{
  "element_type": "text",
  "source_content": "Configure API Keys",
  "source_location": "00:01:23",
  "reference_match": "API Key Configuration",
  "reference_url": "https://docs.example.com/api-keys",
  "match_tier": "fuzzy",
  "match_score": 0.87,
  "classification": "close_match",
  "confidence": 0.92,
  "signals": {
    "context_match": true,
    "workflow_match": false,
    "transcript_match": true
  }
}
```

### Enrich Mode

Comparison results attached as a field on source documents:

```json theme={null}
{
  "document_id": "doc_source_123",
  "content": "...",
  "comparison_results": [
    {
      "element_type": "text",
      "source_content": "Configure API Keys",
      "match_tier": "fuzzy",
      "match_score": 0.87,
      "classification": "close_match",
      "confidence": 0.92
    }
  ]
}
```

### Finding Fields

| Field | Type | Description |
| - | - | - |
| `element_type` | string | Type of element: `text`, `code`, `visual`, or custom |
| `source_content` | string | Content from the source document |
| `source_location` | string | Location reference (timestamp, page number) |
| `reference_match` | string | Best matching content from reference |
| `reference_url` | string | URL or ID of matched reference document |
| `match_tier` | string | Tier used: `exact`, `fuzzy`, `semantic`, `visual`, `none` |
| `match_score` | float | Match score (0.0 - 1.0) |
| `classification` | string | Label from classification rules |
| `confidence` | float | Multi-signal confidence (0.0 - 1.0) |
| `signals` | object | Corroborating signals used in confidence |

## Performance

| Scenario | Expected Latency | Notes |
| - | - | - |
| Exact + fuzzy only (50 docs) | 50-200ms | In-memory string matching |
| With semantic tier (50 docs) | 200-500ms | MVS vector queries |
| With visual comparison (50 docs) | 500-1500ms | Multiple vector queries |
| Large reference set (200 docs) | 300-800ms | More candidates to compare |

> [!TIP]
>
> Reference documents are fetched once and reused across all source documents. The matching cascade short-circuits at the first successful tier, so ordering `match_tiers` from fastest to slowest (exact → fuzzy → vector) is optimal.

**Limits:**

* Max source documents per execution: 50
* Max reference documents fetched: 200 (configurable via `reference_limit`)

## Common Pipeline Patterns

### Drift Detection Pipeline

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
          "top_k": 50
        }],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "cross_compare",
    "stage_type": "apply",
    "config": {
      "stage_id": "cross_compare",
      "parameters": {
        "reference_collection_id": "col_documentation",
        "source_field": "content",
        "reference_field": "content",
        "extraction_mode": "labels",
        "match_tiers": [
          {"type": "exact"},
          {"type": "fuzzy", "threshold": 0.75},
          {"type": "vector", "threshold": 0.85, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"},
          {"type": "vector", "threshold": 0.55, "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1"}
        ],
        "classifications": [
          {"min_score": 0.95, "label": "current"},
          {"min_score": 0.75, "label": "needs_review"},
          {"min_score": 0.0, "label": "outdated"}
        ]
      }
    }
  }
]
```

### Catalog Match + Transform

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
          "top_k": 30
        }],
        "final_top_k": 30
      }
    }
  },
  {
    "stage_name": "cross_compare",
    "stage_type": "apply",
    "config": {
      "stage_id": "cross_compare",
      "parameters": {
        "reference_collection_id": "col_reference_catalog",
        "source_field": "product_name",
        "reference_field": "product_name",
        "match_tiers": [
        {"type": "exact"},
        {"type": "fuzzy", "threshold": 0.75}
      ],
        "output_mode": "enrich",
        "output_field": "match_result"
      }
    }
  },
  {
    "stage_name": "json_transform",
    "stage_type": "apply",
    "config": {
      "stage_id": "json_transform",
      "parameters": {
        "template": "{\"product\": \"{{ DOC.product_name }}\", \"match_status\": \"{{ DOC.match_result[0].classification }}\", \"score\": {{ DOC.match_result[0].match_score }}}"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Reference collection not found | Stage fails with error |
| No reference documents found | All elements classified as `no_match` |
| Vector index not available | Semantic/visual tiers skipped silently |
| Source field missing on document | Document skipped |
| Exceeds max\_working\_documents | Extra documents passed through unchanged |

## vs Other Enrichment Stages

| Feature | cross\_compare | document\_enrich | api\_call |
| - | - | - | - |
| **Purpose** | Multi-tier comparison with classification | Simple field join/lookup | External API enrichment |
| **Data source** | Internal MVS namespaces | Internal MVS namespaces | External HTTP APIs |
| **Matching** | Cascading: exact → fuzzy → semantic → visual | Top-1 vector or key match | N/A |
| **Output** | Classified findings with scores | Joined fields | API response |
| **Latency** | 50-1500ms | 5-20ms | 100-500ms |
| **Best for** | Drift detection, dedup, compliance | Cross-collection joins | Third-party data |

## Related

* [Document Enrich](/docs/retrieval/stages/document-enrich) - Simple cross-collection joins
* [Feature Search](/docs/retrieval/stages/feature-search) - Vector search (often used before cross\_compare)
* [JSON Transform](/docs/retrieval/stages/json-transform) - Transform comparison output
* [Taxonomy Enrich](/docs/retrieval/stages/taxonomy-enrich) - Classification enrichment


# Deduplicate
Source: https://docs.mixpeek.com/docs/retrieval/stages/deduplicate

Remove duplicate documents by field match or content similarity

![Deduplicate stage showing removal of duplicate documents](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/deduplicate.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=38b32d27d3be3567bcb3599d6630cbff)

The Deduplicate stage removes duplicate documents from the result set based on exact field matching or content similarity. This is analogous to SQL's `DISTINCT`, MongoDB's `$group` with `$first`, and Elasticsearch's field collapsing.

> [!NOTE]
>
> **Stage Category**: REDUCE (Removes duplicates)
>
> **Transformation**: N documents → M documents (M ≤ N, duplicates removed)

## When to Use

| Use Case | Description |
| - | - |
| **URL deduplication** | One result per source URL after web enrichment |
| **Author collapse** | Keep one result per author |
| **Content dedup** | Remove near-identical text chunks |
| **Multi-source merge** | Remove overlapping results from multiple searches |
| **Query expansion cleanup** | Remove duplicates from expanded query results |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Grouping with aggregation | `group_by` stage |
| Sampling unique categories | `sample` with stratified |
| Limiting result count | `limit` stage |
| Filtering by criteria | `attribute_filter` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `strategy` | string | `field` | Dedup method: `field` (exact match) or `content` (similarity) |
| `fields` | list\[string] | *required for field* | Field paths to compare for deduplication |
| `content_field` | string | `content` | Text field for content-based dedup |
| `similarity_threshold` | float | `0.95` | Similarity threshold for content dedup (0.0-1.0) |
| `keep` | string | `first` | Which duplicate to keep: `first` or `last` |
| `case_sensitive` | boolean | `true` | Whether string comparisons are case-sensitive |

## Deduplication Strategies

| Strategy | Performance | Best For |
| - | - | - |
| `field` | O(N) hash-based | Exact field matching (URL, ID, title) |
| `content` | O(N²) pairwise | Near-duplicate text detection |

## Configuration Examples

```json Deduplicate by URL theme={null}
{
  "stage_name": "deduplicate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "deduplicate",
    "parameters": {
      "strategy": "field",
      "fields": ["metadata.source_url"],
      "keep": "first"
    }
  }
}
```

```json Case-Insensitive Author Dedup theme={null}
{
  "stage_name": "deduplicate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "deduplicate",
    "parameters": {
      "strategy": "field",
      "fields": ["metadata.author"],
      "case_sensitive": false
    }
  }
}
```

```json Multi-Field Dedup theme={null}
{
  "stage_name": "deduplicate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "deduplicate",
    "parameters": {
      "strategy": "field",
      "fields": ["metadata.author", "metadata.title"]
    }
  }
}
```

```json Content Similarity Dedup theme={null}
{
  "stage_name": "deduplicate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "deduplicate",
    "parameters": {
      "strategy": "content",
      "content_field": "content",
      "similarity_threshold": 0.9,
      "keep": "first"
    }
  }
}
```

> [!TIP]
>
> For best results, place deduplicate after sorting/reranking so that `keep: "first"` retains the highest-scored duplicate. This ensures you keep the most relevant version of each document.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 5ms (field) / 10-100ms (content) |
| **Memory** | O(N) hash set (field) / O(N) content cache (content) |
| **Cost** | Free |
| **Complexity** | O(N) field / O(N²) content |

## Common Pipeline Patterns

### Web Search Deduplication

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 50}],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "external_web_search",
    "stage_type": "apply",
    "config": {
      "stage_id": "external_web_search",
      "parameters": {
        "query": "{{INPUT.query}}",
        "num_results": 10
      }
    }
  },
  {
    "stage_name": "deduplicate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "deduplicate",
      "parameters": {
        "strategy": "field",
        "fields": ["metadata.source_url"]
      }
    }
  }
]
```

### Cross-Collection Dedup

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 100}],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  },
  {
    "stage_name": "deduplicate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "deduplicate",
      "parameters": {
        "strategy": "content",
        "content_field": "content",
        "similarity_threshold": 0.85
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Field doesn't exist | Documents with missing fields have `None` as key value |
| All unique documents | Returns all documents unchanged |
| Empty input | Returns empty result set |
| Single document | Returned as-is (no duplicates possible) |

## Related

* [Group By](/docs/retrieval/stages/group-by) - Group documents with aggregation
* [Limit](/docs/retrieval/stages/limit) - Truncate results after deduplication
* [Sample](/docs/retrieval/stages/sample) - Random sampling (different from dedup)
* [Unwind](/docs/retrieval/stages/unwind) - Inverse: expand grouped items


# External Web Search
Source: https://docs.mixpeek.com/docs/retrieval/stages/external-web-search

Augment results with real-time web search using Exa's neural search API

![External Web Search stage showing Exa API integration for web results](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/external-web-search.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=992fd46917a19a8e6cb0aa09d7cda089)

The External Web Search stage integrates Exa's neural search API to augment your results with real-time web content. This enables hybrid retrieval combining your indexed documents with fresh web results.

> [!NOTE]
>
> **Stage Category**: APPLY (Enriches pipeline with web results)
>
> **Transformation**: N documents → N + M documents (web results added)

## When to Use

| Use Case | Description |
| - | - |
| **Knowledge augmentation** | Supplement internal docs with web content |
| **Real-time information** | Access current events, news, updates |
| **Research expansion** | Broaden search beyond your corpus |
| **Competitive intelligence** | Include competitor content in results |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Internal-only search | Skip this stage |
| Sensitive/confidential queries | Use only indexed content |
| Low-latency requirements | Web search adds 200-500ms |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `query` | string | `{{INPUT.query}}` | Search query (supports templates) |
| `num_results` | integer | `10` | Number of web results to retrieve (1-100) |
| `start_published_date` | string | `null` | Filter by publish date (`YYYY-MM-DD`) |
| `category` | string | `null` | Content category filter |
| `use_autoprompt` | boolean | `true` | Let Exa optimize the query |
| `include_text` | boolean | `true` | Include text snippets in results |

## Available Categories

| Category | Description |
| - | - |
| `company` | Company websites and profiles |
| `research_paper` | Academic and research content |
| `news` | News articles |
| `pdf` | PDF documents |
| `github` | GitHub repositories |
| `tweet` | Twitter/X content |
| `personal_site` | Personal websites and blogs |

## Configuration Examples

```json Basic Web Search theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{{INPUT.query}}",
      "num_results": 10
    }
  }
}
```

```json Category-Filtered Search theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{{INPUT.query}}",
      "num_results": 5,
      "category": "research_paper"
    }
  }
}
```

```json Recent News Search theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{{INPUT.query}}",
      "num_results": 10,
      "category": "news",
      "start_published_date": "2024-01-01"
    }
  }
}
```

```json Research Papers theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{{INPUT.query}}",
      "num_results": 20,
      "category": "research_paper",
      "include_text": true
    }
  }
}
```

```json GitHub Code Search theme={null}
{
  "stage_name": "external_web_search",
  "stage_type": "apply",
  "config": {
    "stage_id": "external_web_search",
    "parameters": {
      "query": "{{INPUT.query}} implementation",
      "num_results": 10,
      "category": "github",
      "use_autoprompt": false
    }
  }
}
```

## Content Extraction

Set `include_text` to `true` (default) to include text snippets in each result. Disable it to reduce API costs and response size.

## Output Schema

Web results are added to the document set with a `source: "web"` marker:

```json theme={null}
{
  "document_id": "web_abc123",
  "source": "web",
  "url": "https://example.com/article",
  "title": "Article Title",
  "content": "Full extracted text content...",
  "published_date": "2024-03-15T10:30:00Z",
  "author": "John Doe",
  "score": 0.95,
  "metadata": {
    "domain": "example.com",
    "category": "news"
  }
}
```

## Exa Neural Search

Exa uses neural search rather than keyword matching:

| Feature | Description |
| - | - |
| **Semantic understanding** | Understands query intent |
| **Neural ranking** | ML-based relevance scoring |
| **Content extraction** | Automatic text extraction |
| **Autoprompt** | Query optimization for better results |

> [!TIP]
>
> Enable `use_autoprompt` (default) for natural language queries. Disable it when you need exact phrase matching or have already optimized your query.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 200-500ms |
| **Rate limits** | Based on Exa plan |
| **Parallel execution** | Concurrent with pipeline |
| **Caching** | Results cached for 1 hour |

## Common Pipeline Patterns

### Internal + Web Hybrid Search

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 20 }
        ],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "external_web_search",
    "stage_type": "apply",
    "config": {
      "stage_id": "external_web_search",
      "parameters": {
        "query": "{{INPUT.query}}",
        "num_results": 10
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]
```

### Web-Augmented RAG

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 30 }
        ],
        "final_top_k": 30
      }
    }
  },
  {
    "stage_name": "external_web_search",
    "stage_type": "apply",
    "config": {
      "stage_id": "external_web_search",
      "parameters": {
        "query": "{{INPUT.query}}",
        "num_results": 5,
        "category": "news",
        "start_published_date": "{{INPUT.date_filter}}"
      }
    }
  },
  {
    "stage_name": "rag_prepare",
    "stage_type": "apply",
    "config": {
      "stage_id": "rag_prepare",
      "parameters": {
        "max_tokens": 8000,
        "output_mode": "single_context"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| API rate limit | Retry with backoff |
| Network timeout | Stage fails gracefully, no web results |
| Invalid domain | Ignored, other domains searched |
| No results found | Empty web result set |

> [!WARNING]
>
> Web search results may include content from untrusted sources. Consider filtering or validating web content before using in sensitive applications.

## Related

* [Web Scrape](/docs/retrieval/stages/web-scrape) - Extract content from specific URLs
* [Feature Search](/docs/retrieval/stages/feature-search) - Search your indexed content
* [Rerank](/docs/retrieval/stages/rerank) - Combine and rank mixed results


# Feature Search
Source: https://docs.mixpeek.com/docs/retrieval/stages/feature-search

Unified semantic and hybrid search across multiple embedding features with configurable fusion strategies

![Feature Search stage showing multi-vector semantic search with fusion](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/feature-search.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=abdaa3f5f627921b4c614a25a2fe11eb)

The Feature Search stage is the **primary search stage** for retrieval pipelines. It performs vector similarity search across one or more embedding features, supporting single-modal, multimodal, and hybrid search patterns. Results from multiple searches are fused using configurable strategies (RRF, DBSF, weighted, max, or learned).

> [!NOTE]
>
> **Stage Category**: FILTER (Retrieves documents)
>
> **Transformation**: 0 documents → N documents (retrieves from collection based on vector similarity)

> **[Run feature search on your own data](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_feature_search\&utm_campaign=managed_onramp)**
>
> Create a managed namespace, extract embeddings from your own files, then compose this stage into a retriever.

## When to Use

| Use Case | Description |
| - | - |
| **Semantic search** | Find documents similar in meaning to a query |
| **Image search** | Search by image embeddings |
| **Video search** | Search by video frame embeddings |
| **Multimodal search** | Combine text + image + video in one query |
| **Hybrid search** | Fuse results from multiple embedding types |
| **Decompose/recompose** | Group results by parent document |
| **Faceted search** | Get result counts by field values |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Exact field matching | `attribute_filter` |
| Full-text keyword search | Combine with text features |
| No embeddings in collection | `attribute_filter` |
| Post-search filtering only | Use after `feature_search` |

## Core Concepts

### Feature URIs

Feature URIs identify which embedding index to search. They follow the pattern:

```
mixpeek://{extractor_name}@{version}/{output_name}
```

**Examples:**

* `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding` - Multimodal text/image/video embeddings
* `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1` - Text-only embeddings
* `mixpeek://image_extractor@v1/google_siglip_base_v1` - Image embeddings
* `mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1` - Speech/transcript embeddings (from video/audio)

### Fusion Strategies

When searching multiple features, results are combined using fusion:

| Strategy | Description | Best For |
| - | - | - |
| `rrf` | Reciprocal Rank Fusion | General purpose, balanced results |
| `dbsf` | Distribution-Based Score Fusion | When scores have different distributions |
| `weighted` | Weighted combination | When you know relative importance |
| `max` | Maximum score wins | When any match is sufficient |
| `learned` | Thompson Sampling bandit | Automatically adapts per-user from [interaction data](/docs/retrieval/auto-tune) |

### Learned Fusion Configuration

When `fusion` is set to `"learned"`, you can provide a `learning_config` object to control how the bandit adapts. See [Auto-Tune](/docs/retrieval/auto-tune) for a full walkthrough.

```json Learned Fusion with learning_config theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
          "top_k": 100
        }
      ],
      "fusion": "learned",
      "learning_config": {
        "context_features": ["INPUT.user_id"],
        "demographic_features": ["INPUT.user_segment"],
        "reward_map": {
          "click": 1.0,
          "purchase": 3.0,
          "add_to_cart": 2.0,
          "bookmark": 1.5,
          "positive_feedback": 2.0,
          "negative_feedback": -2.0,
          "skip": -0.5
        },
        "min_interactions": 5,
        "exploration_bonus": 1.0,
        "exploration_decay": 0.99,
        "exploration_floor": 0.1,
        "decay_factor": 0.995,
        "decay_window_days": 365,
        "min_weight": 0.05,
        "max_weight": 0.95,
        "rollout_pct": 100.0,
        "shadow_mode": false
      },
      "final_top_k": 25
    }
  }
}
```

#### learning\_config Fields

| Field | Type | Default | Description |
| - | - | - | - |
| `context_features` | `string[]` | `["INPUT.user_id"]` | Input fields for personal-level learning. References `INPUT.*` fields from the retriever's `input_schema`. |
| `demographic_features` | `string[]` | `[]` | Input fields for segment-level fallback (e.g., `"INPUT.user_segment"`). |
| `reward_signal` | `string` | `"click"` | *Deprecated.* Use `reward_map` instead. |
| `reward_map` | `object` | [See defaults](/docs/retrieval/reward-signals) | Maps interaction types to reward magnitudes. Positive values reinforce the associated feature; negative values penalize it. |
| `min_interactions` | `integer` | `5` | Minimum interactions before personal-level weights are used. Below this, falls back to demographic or global. |
| `exploration_bonus` | `float` | `1.0` | Initial multiplier for weight distribution variance. |
| `exploration_decay` | `float` | `0.99` | Per-interaction decay of `exploration_bonus`. |
| `exploration_floor` | `float` | `0.1` | Minimum exploration bonus (prevents full exploitation). |
| `decay_factor` | `float` | `0.995` | Per-day exponential decay on older interactions. `1.0` = no decay. |
| `decay_window_days` | `integer` | `365` | Interactions older than this are excluded entirely. |
| `min_weight` | `float` | `0.05` | Floor for any feature's weight after sampling. |
| `max_weight` | `float` | `0.95` | Ceiling for any feature's weight after sampling. |
| `rollout_pct` | `float` | `100.0` | Percentage of requests using learned weights (0-100). |
| `shadow_mode` | `boolean` | `false` | Compute learned weights but serve static results. |

> [!NOTE]
>
> See [Auto-Tune](/docs/retrieval/auto-tune) for the full concept overview, [Reward Signals](/docs/retrieval/reward-signals) for reward map customization, and [Rollout & Safety](/docs/retrieval/auto-tune-rollout) for traffic splitting and kill switch details.

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `searches` | array | *Required* | Array of search configurations |
| `final_top_k` | integer | `25` | Total results to return after fusion |
| `fusion` | string | `rrf` | Fusion strategy for multi-search |
| `group_by` | object | `null` | Group results by field |
| `facets` | array | `null` | Facet configs, one per field. Each is `{key, limit?, exact?}`, not a bare field name. |

### Search Object Parameters

Each item in the `searches` array supports:

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | *Required* | Embedding index to search |
| `query` | string/object | *Required* | Query text or embedding |
| `top_k` | integer | `100` | Candidates per search |
| `filters` | object | `null` | Pre-filter conditions |
| `weight` | number | `1.0` | Weight for fusion (weighted strategy) |
| `lexical` | boolean | `false` | Run this search as keyword/BM25 instead of vector (see [Lexical (BM25) Search](#lexical-bm25-search)) |
| `query_preprocessing` | object | `null` | Large file decomposition config |

### Query Input Modes

The `query` field on each search object accepts either a plain string (shorthand for `text` mode) or an object with an explicit `input_mode`:

| Mode | `input_mode` | Value | Supported by |
| - | - | - | - |
| **Text** | `"text"` | Plain text string | All text-capable extractors |
| **Content** | `"content"` | Single URL or base64 data URI | All multimodal extractors |
| **Document** | `"document"` | Reference to an existing document | All extractors |
| **Vector** | `"vector"` | Pre-computed embedding (list of floats) | All extractors |
| **Multi-content** | `"multi_content"` | List of URLs and/or text strings | `gemini_multifile_extractor` only |

**Text** — embed a string and search:

```json theme={null}
{"input_mode": "text", "value": "{{INPUT.query}}"}
```

**Content** — fetch a URL and embed it:

```json theme={null}
{"input_mode": "content", "value": "{{INPUT.image_url}}"}
```

**Vector** — use a pre-computed embedding directly (no inference at query time):

```json theme={null}
{"input_mode": "vector", "value": "{{INPUT.embedding}}"}
```

**Multi-content** — embed multiple files together in one API call. Only valid when the `feature_uri` points to an extractor whose vector index has `supports_multi_query=True` (currently: `gemini_multifile_extractor`). Attempting this with any other feature URI returns a `400` error.

```json theme={null}
{
  "input_mode": "multi_content",
  "values": ["{{INPUT.image_url}}", "{{INPUT.description}}"]
}
```

Each item in `values` is auto-detected: URLs (`http://`, `https://`, `s3://`) are fetched and embedded as files; all other strings are embedded as text. All items are passed to the underlying model in one call, producing a single query vector that mirrors how objects were indexed.

### Lexical (BM25) Search

Set `lexical: true` on a search to run **keyword/BM25** matching instead of vector similarity. The query text is matched against the namespace's [full-text index](/docs/vector-store/namespaces#text-indexes-bm25) — it is **not** embedded into a vector. BM25 catches exact tokens that dense embeddings routinely miss: brand names, SKUs, prices like `$9.99`, promo codes, error strings, and CTAs.

| Behavior | Detail |
| - | - |
| **Input** | Must be text (`input_mode: "text"`). The query string is used verbatim. |
| **Matching** | Across **all** `text`-indexed string payload fields — not a single field. |
| **`feature_uri`** | Used only for collection scoping; no vector index is queried. |
| **Prerequisite** | A `text` payload index must exist (see [Text Indexes (BM25)](/docs/vector-store/namespaces#text-indexes-bm25)). |

> [!NOTE]
>
> **Searching only one field (e.g. OCR text).** BM25 matches across *all* `text`-indexed string fields — it can't be scoped to a single field like `ocr_text`. To make one field independently searchable, give it its own dense index by running a `text_extractor` over it (map the extractor's input to `ocr_text`), then `feature_search` that feature URI directly. For coarse exact-substring filtering on a single field, an [`attribute_filter`](/docs/retrieval/stages/attribute-filter) with the `contains` operator works but is not relevance-ranked.

The real power is **hybrid retrieval** — fuse a dense (vector) search with a lexical (BM25) search under `rrf` so semantic recall and exact-keyword precision reinforce each other:

```json Dense + Lexical Hybrid (RRF) theme={null}
{
  "stage_name": "hybrid_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "fusion": "rrf",
      "final_top_k": 25,
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
          "lexical": true,
          "top_k": 100
        }
      ]
    }
  }
}
```

> [!TIP]
>
> Use `rrf` fusion for dense+lexical hybrid — it ranks by position, so it is immune to the score-scale mismatch between cosine similarity and BM25. Avoid `weighted`/`max` here unless you have a specific reason.

### Query Preprocessing

When searching with large files (videos, PDFs, long documents) as input, `query_preprocessing` decomposes the file into chunks using the same extractor pipeline that indexed your data, runs parallel searches for each chunk, and fuses the results.

This is **ingestion applied to the query** — same decomposition and embedding, but vectors are used for search instead of storage.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | `null` | Extractor pipeline for decomposition (inherits from search `feature_uri` if not set) |
| `params` | object | `null` | Extractor parameters — **identical schema to the collection's extractor config for that `feature_uri`** |
| `max_chunks` | integer | `20` | Max chunks to search (1-100). Each chunk runs its own search and adds query cost — reads are metered (see [Billing](/docs/platform/billing)) |
| `aggregation` | string | `rrf` | Fusion strategy: `rrf`, `max`, or `avg` |
| `dedup_field` | string | `null` | Field to deduplicate results by |

> [!NOTE]
>
> **`params` uses the extractor's own parameter schema.** Whatever parameters the extractor accepts during ingestion (e.g. `split_method`, `time_split_interval` for video; `chunk_size`, `chunk_overlap` for text) are the same parameters you pass here. There is no separate preprocessing-specific schema — the extractor drives the decomposition exactly as it would during collection processing. Refer to the extractor's own documentation for valid parameter names.

> [!NOTE]
>
> You can also set `query_preprocessing` at the **stage level** (on `parameters`) to apply it to all searches as a default. Per-search settings override the stage default.

**Aggregation strategies:**

| Strategy | Best For | How It Works |
| - | - | - |
| `rrf` | General purpose (recommended) | Rank-based fusion, immune to score magnitude differences |
| `max` | "Find this exact moment" | Keeps highest score per document across chunks |
| `avg` | "Find similar overall content" | Averages scores — consistent matches win |

## Configuration Examples

```json Basic Text Search theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 100
        }
      ],
      "final_top_k": 25
    }
  }
}
```

```json Image Search theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
          "query": "{{INPUT.image_url}}",
          "top_k": 50
        }
      ],
      "final_top_k": 20
    }
  }
}
```

```json Multimodal Hybrid Search theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
          "query": "{{INPUT.image_url}}",
          "top_k": 100
        }
      ],
      "fusion": "rrf",
      "final_top_k": 25
    }
  }
}
```

```json Weighted Fusion theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": "{{INPUT.query}}",
          "top_k": 100,
          "weight": 0.7
        },
        {
          "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
          "query": "{{INPUT.image_url}}",
          "top_k": 100,
          "weight": 0.3
        }
      ],
      "fusion": "weighted",
      "final_top_k": 20
    }
  }
}
```

```json With Pre-Filters theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 100,
          "filters": {
            "AND": [
              {"field": "metadata.status", "operator": "eq", "value": "published"},
              {"field": "metadata.category", "operator": "in", "value": ["tech", "science"]}
            ]
          }
        }
      ],
      "final_top_k": 25
    }
  }
}
```

```json With Grouping (Decompose/Recompose) theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 200
        }
      ],
      "group_by": {
        "field": "metadata.parent_id",
        "limit": 10,
        "group_size": 3
      },
      "final_top_k": 30
    }
  }
}
```

```json With Facets theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 100
        }
      ],
      "facets": [
        { "key": "metadata.category" },
        { "key": "metadata.author", "limit": 20 },
        { "key": "metadata.year", "exact": true }
      ],
      "final_top_k": 25
    }
  }
}
```

```json Multi-file Query (Gemini Multifile) theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://gemini_multifile_extractor@v1/gemini-embedding-exp-03-07",
          "query": {
            "input_mode": "multi_content",
            "values": [
              "{{INPUT.image_url}}",
              "{{INPUT.spec_sheet_url}}",
              "{{INPUT.description}}"
            ]
          },
          "top_k": 20
        }
      ],
      "final_top_k": 10
    }
  }
}
```

## Query Preprocessing Examples

Search with a large video — decompose it into 10-second segments, search each, and fuse:

```json Video Search with Preprocessing theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": {"input_mode": "content", "value": "{{INPUT.video}}"},
          "top_k": 100,
          "query_preprocessing": {
            "params": {"split_method": "time", "time_split_interval": 10},
            "max_chunks": 20,
            "aggregation": "max"
          }
        }
      ],
      "final_top_k": 25
    }
  }
}
```

```json PDF Search (Page-by-Page) theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://image_extractor@v1/siglip_embedding",
          "query": {"input_mode": "content", "value": "{{INPUT.pdf_document}}"},
          "top_k": 100,
          "query_preprocessing": {
            "max_chunks": 50,
            "aggregation": "rrf"
          }
        }
      ],
      "final_top_k": 25
    }
  }
}
```

```json Mixed: Video (Preprocessed) + Text (Normal) theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": {"input_mode": "content", "value": "{{INPUT.video}}"},
          "query_preprocessing": {
            "params": {"time_split_interval": 10},
            "aggregation": "max"
          }
        },
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": {"input_mode": "text", "value": "{{INPUT.text_query}}"}
        }
      ],
      "final_top_k": 25
    }
  }
}
```

```json Stage-Level Default (All Searches) theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "query_preprocessing": {
        "max_chunks": 15,
        "aggregation": "rrf"
      },
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": {"input_mode": "content", "value": "{{INPUT.video1}}"}
        },
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": {"input_mode": "content", "value": "{{INPUT.video2}}"}
        }
      ],
      "final_top_k": 25
    }
  }
}
```

> [!NOTE]
>
> Preprocessing uses the same extractor pipeline that indexed your data. The `params` accept the same fields you configured on your collection's feature extractor (e.g., `split_method`, `chunk_size`). If you don't specify `params`, extractor defaults are used.

The response includes preprocessing metadata showing what happened:

```json theme={null}
{
  "metadata": {
    "preprocessing": {
      "content_type": "video/mp4",
      "extractor": "multimodal_extractor@v1",
      "chunks_generated": 18,
      "chunks_searched": 18,
      "aggregation": "rrf",
      "preprocessing_ms": 12450
    }
  }
}
```

Each result also includes `query_chunks` showing which parts of your query matched:

```json theme={null}
{
  "document_id": "doc_abc123",
  "score": 0.89,
  "query_chunks": [
    {"chunk_index": 0, "start_ms": 0, "end_ms": 10000, "score": 0.92},
    {"chunk_index": 2, "start_ms": 20000, "end_ms": 30000, "score": 0.87}
  ]
}
```

## Grouping (Decompose/Recompose)

When documents are decomposed into chunks (e.g., video frames, document pages), use `group_by` to recompose results by parent:

```json theme={null}
{
  "group_by": {
    "field": "metadata.parent_id",
    "limit": 10,
    "group_size": 3
  }
}
```

| Parameter | Description |
| - | - |
| `field` | Field to group by (e.g., parent document ID) |
| `limit` | Maximum number of groups to return |
| `group_size` | Maximum documents per group |

**Use cases:**

* Video search: Group frames by video, return top 3 frames per video
* Document search: Group chunks by document, return best chunks per doc
* Product search: Group variants by product family

## Faceted Search

Get counts of results by field values for building filter UIs. Each entry is a
config object, so a bare field name is not accepted:

```json theme={null}
{
  "facets": [
    { "key": "metadata.category" },
    { "key": "metadata.brand", "limit": 20 },
    { "key": "metadata.price_range", "exact": true }
  ]
}
```

| Field | Default | Meaning |
| - | - | - |
| `key` | *required* | Field path to facet on. Dot notation for nested fields. |
| `limit` | `10` | Unique values returned, sorted by count descending then value ascending. |
| `exact` | `false` | Exact counts instead of approximate. |

Facets run in parallel with the search under the same filters, and they count
the **entire filtered result set**, not the page you asked for. So a `page_size`
of 10 still gives counts across every match.

**Response.** `facets` is an **array**, one entry per faceted field, each
carrying its `key` and its value/count buckets:

```json theme={null}
{
  "facets": [
    {
      "key": "metadata.category",
      "buckets": [
        {"value": "electronics", "count": 45},
        {"value": "clothing", "count": 23}
      ]
    }
  ]
}
```

> [!WARNING]
>
> **Every faceted field needs a keyword index in MVS.** Faceting fails without
> one. `metadata.*`, `status` and `collection_id` are commonly auto-indexed; a
> custom field needs indexing configured first. See
> [payload indexes](/docs/retrieval/filters#payload-indexes).

> [!NOTE]
>
> `facets` is surfaced at the top level of the execution response so a consumer
> has one predictable place to read it. The same data also appears per-stage
> under `stage_statistics.stages.<stage>.metadata.facets`.
>
> **When more than one stage computes facets, only the last stage's reach the
> top level.** Read the per-stage path if you need facets from an earlier stage.
>
> The field is absent when no facets were requested. The spec types each entry
> as an open object, so read the bucket keys defensively.

## Filter Syntax

> [!NOTE]
>
> Filtered fields must have [payload indexes](/docs/retrieval/filters#payload-indexes) on your namespace. Without indexes, filtering is slow and the response includes warnings about unindexed fields.

Pre-filters use boolean logic with AND/OR/NOT. The key is `pre_filters`; a key
named `filters` is rejected with a 422 naming the valid placements.

```json theme={null}
{
  "pre_filters": {
    "AND": [
      {"field": "metadata.status", "operator": "eq", "value": "active"},
      {
        "OR": [
          {"field": "metadata.category", "operator": "eq", "value": "tech"},
          {"field": "metadata.category", "operator": "eq", "value": "science"}
        ]
      }
    ]
  }
}
```

### Filter-Only Search

Leave `searches` empty and supply `pre_filters` to select documents by attribute
alone, with no vector query and no embedding cost.

```json theme={null}
{
  "stage_name": "search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": []
    },
    "pre_filters": {
      "AND": [
        {"field": "metadata.status", "operator": "eq", "value": "active"}
      ]
    }
  }
}
```

`pre_filters` is a sibling of `parameters` on the stage config. This is the
placement to use for filter-only search and for a stage that runs a vector query
and a filter together. A `pre_filters` placed inside `parameters` on a
`feature_search` stage is lifted to the stage config before validation.

> [!WARNING]
>
> `pre_filters` is required in this mode. An empty `searches` with no
> `pre_filters` gives the stage nothing to select on and returns a 400.

### Supported Operators

| Operator | Description | Example |
| - | - | - |
| `eq` | Equals | `{"field": "status", "operator": "eq", "value": "active"}` |
| `ne` | Not equals | `{"field": "status", "operator": "ne", "value": "deleted"}` |
| `gt` | Greater than | `{"field": "price", "operator": "gt", "value": 100}` |
| `gte` | Greater than or equal | `{"field": "rating", "operator": "gte", "value": 4}` |
| `lt` | Less than | `{"field": "age", "operator": "lt", "value": 30}` |
| `lte` | Less than or equal | `{"field": "count", "operator": "lte", "value": 10}` |
| `in` | In array | `{"field": "category", "operator": "in", "value": ["a", "b"]}` |
| `nin` | Not in array | `{"field": "status", "operator": "nin", "value": ["deleted", "archived"]}` |
| `contains` | Contains substring | `{"field": "title", "operator": "contains", "value": "guide"}` |
| `exists` | Field exists | `{"field": "metadata.optional", "operator": "exists", "value": true}` |

## Performance

| Metric | Value |
| - | - |
| **Latency** | 10-50ms (single search) |
| **Latency** | 20-80ms (multi-search with fusion) |
| **Optimal top\_k** | 100-500 per search |
| **Maximum top\_k** | 10,000 per search |
| **Fusion overhead** | \< 5ms |

> [!TIP]
>
> For best performance, use pre-filters to reduce the search space. Filtering at the vector index level is much faster than post-filtering in later stages.

## Common Pipeline Patterns

### Basic Search + Rerank

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 100
          }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]
```

### Multimodal Search + Filter + Limit

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 100
          },
          {
            "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
            "query": "{{INPUT.image}}",
            "top_k": 100
          }
        ],
        "fusion": "rrf",
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.in_stock",
        "operator": "eq",
        "value": true
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "count": 20
      }
    }
  }
]
```

### Video Search with Frame Grouping

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 500
          }
        ],
        "group_by": {
          "field": "metadata.video_id",
          "limit": 10,
          "group_size": 5
        },
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "google",
        "model_name": "gemini-2.5-flash-lite",
        "prompt": "Summarize why these video segments match the query:\n\n{{DOCUMENTS}}"
      }
    }
  }
]
```

### E-commerce Search with Facets

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 200,
            "filters": {
              "AND": [
                {"field": "metadata.in_stock", "operator": "eq", "value": true},
                {"field": "metadata.price", "operator": "lte", "value": "{{INPUT.max_price}}"}
              ]
            }
          }
        ],
        "facets": [
          { "key": "metadata.category" },
          { "key": "metadata.brand" },
          { "key": "metadata.color" }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "{{INPUT.sort_by}}",
        "direction": "{{INPUT.sort_order}}"
      }
    }
  }
]
```

## Output Schema

Each result includes:

| Field | Type | Description |
| - | - | - |
| `document_id` | string | Unique document identifier |
| `score` | float | Combined similarity score |
| `content` | string | Document content |
| `metadata` | object | Document metadata |
| `features` | object | Feature data and scores per search |

**Example output:**

```json theme={null}
{
  "document_id": "doc_abc123",
  "score": 0.892,
  "content": "Document content here...",
  "metadata": {
    "title": "Example Document",
    "category": "tech",
    "created_at": "2024-01-15T10:30:00Z"
  },
  "features": {
    "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding": {
      "score": 0.91
    },
    "mixpeek://image_extractor@v1/google_siglip_base_v1": {
      "score": 0.87
    }
  }
}
```

## Comparison: feature\_search vs attribute\_filter

| Aspect | feature\_search | attribute\_filter |
| - | - | - |
| **Purpose** | Semantic similarity | Exact matching |
| **Input** | Query text/embedding | Field conditions |
| **Scoring** | Vector similarity | Binary match |
| **Speed** | 10-50ms | 5-20ms |
| **Use when** | Finding similar content | Filtering by metadata |

## Error Handling

| Error | Behavior |
| - | - |
| Invalid feature\_uri | Stage fails with error |
| Empty query | Returns empty results |
| Filter syntax error | Stage fails with error |
| No matching documents | Returns empty results |

## Creating a Retriever with feature\_search

The following is a **complete working example** of creating a retriever that uses the `feature_search` stage, then executing it. Pay close attention to the field names — several are easy to confuse.

> [!WARNING]
>
> **Common mistakes:**
>
> * Use `collection_identifiers` (not `collection_ids`) in the retriever body.
> * `input_schema` is a **flat map keyed by field name** (`{"query": {"type": "text"}}`) — do **not** wrap it in a JSON Schema object (`{"properties": {...}, "type": "object"}`).
> * Use `type: "text"` (not `"string"`) in `input_schema` values.
> * `stage_type` at the outer level must be `"filter"` — passing `stage_type: "feature_search"` is rejected (`feature_search` is a `stage_id`, not a `stage_type`).
> * `stage_id: "feature_search"` lives inside the `config` object, **not** at the outer `stage_id`.
> * Inside each search, the query value uses `{"input_mode": "text", "value": "..."}` — the `value` key, not a bare `text` key.
> * `final_top_k` lives inside `config.parameters`, not at the top level.
> * If a `feature_uri` is wrong, the error lists the `available_feature_uris` for your target collections — copy the exact URI (e.g. `multilingual_e5_large_instruct_v1`, not `embedding`).

### Step 1 — Create the Retriever

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "my-semantic-retriever",

    "collection_identifiers": ["col_abc123"],

    "input_schema": {
      "query": {
        "type": "text",
        "description": "Search query",
        "required": true
      }
    },

    "stages": [
      {
        "stage_name": "Semantic Search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": {
                  "input_mode": "text",
                  "value": "{{INPUT.query}}"
                },
                "top_k": 50
              }
            ],
            "final_top_k": 10
          }
        }
      }
    ]
  }'
```

```python Python theme={null}
import httpx

response = httpx.post(
    "https://api.mixpeek.com/v1/retrievers",
    headers={
        "Authorization": f"Bearer {api_key}",
        "X-Namespace": namespace_id,
    },
    json={
        "retriever_name": "my-semantic-retriever",
        # ✅ correct field: collection_identifiers
        "collection_identifiers": ["col_abc123"],
        "input_schema": {
            "query": {
                "type": "text",       # ✅ "text", not "string"
                "description": "Search query",
                "required": True,
            }
        },
        "stages": [
            {
                "stage_name": "Semantic Search",
                "stage_type": "filter",   # ✅ required at outer stage
                "config": {
                    "stage_id": "feature_search",  # ✅ inside config
                    "parameters": {
                        "searches": [
                            {
                                "feature_uri": (
                                    "mixpeek://text_extractor@v1/"
                                    "multilingual_e5_large_instruct_v1"
                                ),
                                "query": {
                                    "input_mode": "text",
                                    "value": "{{INPUT.query}}",
                                },
                                "top_k": 50,
                            }
                        ],
                        "final_top_k": 10,  # ✅ inside parameters
                    },
                },
            }
        ],
    },
)
data = response.json()
retriever_id = data["retriever_id"]  # retriever_id is top-level on the response
```

### Step 2 — Execute the Retriever

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "query": "machine learning for fashion brand compliance"
    }
  }'
```

```python Python theme={null}
response = httpx.post(
    f"https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute",
    headers={
        "Authorization": f"Bearer {api_key}",
        "X-Namespace": namespace_id,
    },
    json={
        "inputs": {"query": "machine learning for fashion brand compliance"}
    },
)
results = response.json()
```

### Finding the Right feature\_uri

The `feature_uri` must match an embedding index that exists in your namespace. To discover available feature URIs, list the vector indexes in a collection:

```bash theme={null}
curl "https://api.mixpeek.com/v1/collections/$COLLECTION_ID" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

Each item in the `vector_indexes` array has a `feature_uri` field — use that value directly in your retriever stage.

## Embedding Task Conditioning

Feature search automatically applies **task-aware embedding conditioning** to instruction-aware models (E5, Gemini) at query time. This means query embeddings are optimized for asymmetric retrieval without any configuration.

**How it works:**

* **Index time:** Extractors embed documents with `retrieval_document` task (configurable via `embedding_task` on the extractor — see [Text Extractor](/docs/processing/extractors/text#embedding-task))
* **Query time:** Feature search automatically uses `retrieval_query` task for all query embeddings

This asymmetric pairing (document vs. query) improves retrieval quality by \~10% for instruction-aware models like E5-Large.

The `embedding_task` used is included in the stage response metadata:

```json theme={null}
{
  "metadata": {
    "embedding_task": "retrieval_query",
    "num_features": 1,
    "fusion_strategy": "rrf",
    "total_results": 25
  }
}
```

**Task-aware models:**

| Model | Task Support | Used By |
| - | - | - |
| E5-Large (`intfloat_e5_large_instruct_v1`) | Prefix-based (`"query: "` / `"passage: "`) | `text_extractor`, `multimodal_extractor` transcription |
| Gemini Embedding 2 | Instruction-based | `universal_extractor` |
| Vertex Multimodal | Not task-aware (ignored) | `multimodal_extractor` visual |
| SigLIP / CLIP | Not task-aware (ignored) | `image_extractor` |

## Related Stages

* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Metadata-based filtering
* [Rerank](/docs/retrieval/stages/rerank) - Neural re-ranking
* [Query Expand](/docs/retrieval/stages/query-expand) - Query expansion before search
* [MMR](/docs/retrieval/stages/mmr) - Diversity-optimized selection


# Group By
Source: https://docs.mixpeek.com/docs/retrieval/stages/group-by

Aggregate documents by shared field values into logical groups

![Group By stage showing document aggregation by field values](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/group-by.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=bd0f5eff607fb54559e4a282ee779403)

The Group By stage aggregates documents that share the same value for a specified field, creating logical groups. This is useful for organizing results by category, author, date, or any other attribute.

> [!NOTE]
>
> **Stage Category**: GROUP (Groups documents)
>
> **Transformation**: N documents → G groups (where G = unique field values)

## When to Use

| Use Case | Description |
| - | - |
| **Category grouping** | Group products by category |
| **Author aggregation** | Group articles by author |
| **Date grouping** | Group by day/month/year |
| **Source organization** | Group by data source |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Semantic similarity grouping | `cluster` |
| Statistical aggregations only | `aggregate` |
| Removing duplicates | `deduplicate` |
| Top-N per group | Use with `sample` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `group_by_field` | string | `source_object_id` | Field to group by (dot notation supported) |
| `max_per_group` | integer | `10` | Maximum documents to keep per group |
| `output_mode` | string | `all` | `first` (top doc per group), `all` (grouped), `flatten` (flat list) |

> [!WARNING]
>
> Use `group_by_field`. This stage ignores unknown keys, so a wrong key name
> returns HTTP 200 and wrong groups.
>
> Send `field` here and the stage groups by the `source_object_id` default
> instead. `field` belongs to a different model: the `group_by` object nested
> inside `feature_search` parameters, which requires it.

## Configuration Examples

```json Basic Group By theme={null}
{
  "stage_name": "group_by",
  "stage_type": "reduce",
  "config": {
    "stage_id": "group_by",
    "parameters": {
      "group_by_field": "metadata.category"
    }
  }
}
```

```json Limited Docs Per Group theme={null}
{
  "stage_name": "group_by",
  "stage_type": "reduce",
  "config": {
    "stage_id": "group_by",
    "parameters": {
      "group_by_field": "metadata.author",
      "max_per_group": 5
    }
  }
}
```

```json Deduplicate (top doc per group) theme={null}
{
  "stage_name": "group_by",
  "stage_type": "reduce",
  "config": {
    "stage_id": "group_by",
    "parameters": {
      "group_by_field": "metadata.brand",
      "max_per_group": 1,
      "output_mode": "first"
    }
  }
}
```

```json Date Grouping theme={null}
{
  "stage_name": "group_by",
  "stage_type": "reduce",
  "config": {
    "stage_id": "group_by",
    "parameters": {
      "group_by_field": "metadata.publish_date"
    }
  }
}
```

```json Nested Field Grouping theme={null}
{
  "stage_name": "group_by",
  "stage_type": "reduce",
  "config": {
    "stage_id": "group_by",
    "parameters": {
      "group_by_field": "metadata.source.type",
      "max_per_group": 3
    }
  }
}
```

## Output Schema

```json theme={null}
{
  "groups": [
    {
      "key": "electronics",
      "count": 25,
      "documents": [
        {
          "document_id": "doc_123",
          "content": "Latest smartphone review...",
          "score": 0.95,
          "metadata": {"category": "electronics", "price": 999}
        },
        {
          "document_id": "doc_456",
          "content": "Laptop comparison guide...",
          "score": 0.89,
          "metadata": {"category": "electronics", "price": 1299}
        }
      ]
    },
    {
      "key": "clothing",
      "count": 18,
      "documents": [...]
    }
  ],
  "metadata": {
    "total_groups": 5,
    "total_documents": 100,
    "field": "metadata.category"
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-20ms |
| **Memory** | O(N) |
| **Cost** | Free |
| **Scalability** | Efficient |

## Common Pipeline Patterns

### Search + Group by Category

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "group_by",
    "stage_type": "reduce",
    "config": {
      "stage_id": "group_by",
      "parameters": {
        "group_by_field": "metadata.category",
        "max_per_group": 5
      }
    }
  }
]
```

### Grouped Results with Aggregations

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 200 }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "group_by",
    "stage_type": "reduce",
    "config": {
      "stage_id": "group_by",
      "parameters": {
        "group_by_field": "metadata.brand",
        "max_per_group": 10
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "avg", "field": "metadata.price", "alias": "avg_price"},
          {"function": "avg", "field": "metadata.rating", "alias": "avg_rating"}
        ],
        "group_by": [{"field": "metadata.brand", "alias": "brand"}]
      }
    }
  }
]
```

### Author-Grouped Search

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "document_enrich",
      "parameters": {
        "target_collection_id": "authors",
        "target_field": "author_id",
        "source_field": "metadata.author_id",
        "output_field": "author"
      }
    }
  },
  {
    "stage_name": "group_by",
    "stage_type": "reduce",
    "config": {
      "stage_id": "group_by",
      "parameters": {
        "group_by_field": "author.name",
        "max_per_group": 3
      }
    }
  }
]
```

### Time-Based Grouping

```json theme={null}
[
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.date",
        "operator": "gte",
        "value": "2024-01-01"
      }
    }
  },
  {
    "stage_name": "code_execution",
    "stage_type": "apply",
    "config": {
      "stage_id": "code_execution",
      "parameters": {
        "code": "def transform(doc):\n    date = doc.get('metadata', {}).get('date', '')\n    doc['metadata']['month'] = date[:7]  # YYYY-MM\n    return doc"
      }
    }
  },
  {
    "stage_name": "group_by",
    "stage_type": "reduce",
    "config": {
      "stage_id": "group_by",
      "parameters": {
        "group_by_field": "metadata.month"
      }
    }
  }
]
```

## Document Sorting Within Groups

Documents within each group are automatically sorted by relevance `score` (highest first), then limited to `max_per_group`.

## Output Modes

| `output_mode` | Description |
| - | - |
| `all` (default) | Return all documents (up to `max_per_group`) grouped by field |
| `first` | Return only the top-scoring document per group (deduplication) |
| `flatten` | Return all documents as a flat list (drops group structure) |

## Handling Missing Values

Documents missing the `group_by_field` value are grouped under a `null` key.

## Error Handling

| Error | Behavior |
| - | - |
| Missing field | Documents grouped under "null" key |
| Empty results | Return empty groups array |
| Invalid field path | Stage fails |

## Group By vs Cluster

| Aspect | Group By | Cluster |
| - | - | - |
| Grouping basis | Field value | Embedding similarity |
| Groups known | Yes (field values) | No (discovered) |
| Speed | Fast | Slower |
| Use case | Category organization | Theme discovery |

## Related

* [Aggregate](/docs/retrieval/stages/aggregate) - Statistical aggregations
* [Cluster](/docs/retrieval/stages/cluster) - Semantic grouping
* [Sample](/docs/retrieval/stages/sample) - Select from groups
* [Sort Attribute](/docs/retrieval/stages/sort-attribute) - Simple sorting


# JSON Transform
Source: https://docs.mixpeek.com/docs/retrieval/stages/json-transform

Transform document structure using Jinja2 templates for API payloads or custom schemas

![JSON Transform stage showing Jinja2 template transformation of documents](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/json-transform.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=93329a9b257ac7ef01e4d8c261b4fac1)

The JSON Transform stage applies a Jinja2 template to each document, rendering the template with full document context and replacing the document with the parsed JSON output. Use this to reformat documents for external APIs or reshape data for downstream consumers.

> [!NOTE]
>
> **Stage Category**: APPLY (1-1 Transformation)
>
> **Transformation**: N documents → N documents (or fewer with `fail_on_error=False`)

## When to Use

| Use Case | Description |
| - | - |
| **External API formatting** | Format documents for webhook payloads |
| **Response optimization** | Remove unused fields to reduce bandwidth |
| **Schema adaptation** | Convert internal format to client-specific format |
| **Conditional outputs** | Include fields based on document properties |
| **Array flattening** | Transform nested structures to flat arrays |
| **Field renaming** | Rename or reorganize document fields |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Filtering documents | `attribute_filter` or `llm_filter` |
| Sorting documents | `sort_by_field` or `rerank` |
| Enriching with new data | `document_enrich` or `api_call` |
| Joining external data | `taxonomy_enrich` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `template` | string | *Required* | Jinja2 template that must render to valid JSON |
| `fail_on_error` | boolean | `false` | Fail entire pipeline on transformation error |

## Template Context

Templates have access to the full retriever execution context:

| Namespace | Description | Example |
| - | - | - |
| `DOC` / `doc` | Current document fields and metadata | `{{ DOC.document_id }}` |
| `INPUT` / `inputs` | Original query inputs from search request | `{{ INPUT.query }}` |
| `CONTEXT` / `context` | Execution context (namespace\_id, etc.) | `{{ CONTEXT.namespace_id }}` |
| `STAGE` / `stage` | Current stage execution data | `{{ STAGE.name }}` |
| `SECRET` / `secret` | Vault secrets (API keys, credentials) | `{{ SECRET.api_key }}` |

> [!NOTE]
>
> Both uppercase and lowercase namespace formats work identically (`DOC` == `doc`).

### Built-in Functions

These numeric helpers are available directly in templates — useful for clamping
and rounding computed values (e.g. ranking scores):

| Function | Description | Example |
| - | - | - |
| `max` / `min` | Largest / smallest of the arguments | `{{ max(0, 1 - days_old / 14) }}` |
| `abs` | Absolute value | `{{ abs(DOC.delta) }}` |
| `round` | Round to nearest (optional digits) | `{{ round(score, 4) }}` |
| `ceil` / `floor` | Round up / down to an integer | `{{ ceil(DOC.count / 10) }}` |

Read tunable inputs with a default using `INPUT.get`, so callers can override a
weight per request but omit it otherwise: `{{ INPUT.get('recency_weight', 0.3) }}`.

## Template Features

### Jinja2 Syntax

| Feature | Syntax | Description |
| - | - | - |
| Variables | `{{ DOC.field }}` | Output field values |
| Conditionals | `{% if %}...{% endif %}` | Conditional content |
| Loops | `{% for item in items %}` | Iterate over arrays |
| Filters | `{{ value \| tojson }}` | Transform values |
| Comments | `{# comment #}` | Template comments |

### Useful Filters

| Filter | Description | Example |
| - | - | - |
| `tojson` | JSON-safe encoding | `{{ DOC.data \| tojson }}` |
| `length` | Get array/string length | `{{ DOC.tags \| length }}` |
| `default` | Fallback value | `{{ DOC.optional \| default('N/A') }}` |
| `first` / `last` | Array element | `{{ DOC.items \| first }}` |
| `join` | Join array | `{{ DOC.tags \| join(', ') }}` |

## Configuration Examples

```json Simple Field Selection theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"id\": \"{{ DOC.document_id }}\", \"content\": \"{{ DOC.text }}\", \"score\": {{ DOC.score }}}"
    }
  }
}
```

```json With JSON Escaping theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"id\": \"{{ DOC.document_id }}\", \"content\": {{ DOC.text | tojson }}, \"metadata\": {{ DOC.metadata | tojson }}}"
    }
  }
}
```

```json Conditional Field Inclusion theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"workflow_name\": \"process-asset\", \"inputs\": [{\"name\": \"id\", \"value\": \"{{ DOC.id }}\"}{% if DOC.asset_type == \"VIDEO\" %}, {\"name\": \"video\", \"value\": {\"src\": \"{{ DOC.url }}\"}}{% endif %}]}"
    }
  }
}
```

```json Array Iteration theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"title\": \"{{ DOC.title }}\", \"tags\": [{% for tag in DOC.tags %}\"{{ tag }}\"{% if not loop.last %}, {% endif %}{% endfor %}]}"
    }
  }
}
```

```json Nested Field Access theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"user_id\": \"{{ DOC.metadata.user_id }}\", \"category\": \"{{ DOC.metadata.category }}\", \"raw_data\": {{ DOC.metadata.raw | tojson }}}"
    }
  }
}
```

```json Strict Mode (Fail on Error) theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"required_field\": \"{{ DOC.must_exist }}\", \"value\": {{ DOC.number }}}",
      "fail_on_error": true
    }
  }
}
```

```json External Workflow API Format theme={null}
{
  "stage_name": "json_transform",
  "stage_type": "apply",
  "config": {
    "stage_id": "json_transform",
    "parameters": {
      "template": "{\"workflow\": \"{{ DOC.workflow_name }}\", \"inputs\": [{\"name\": \"variant_id\", \"value\": \"{{ DOC.variant_id }}\"}{% if DOC.asset_type == \"VIDEO\" %}, {\"name\": \"video\", \"value\": {\"src\": \"{{ DOC.asset_url }}\"}}{% endif %}]}"
    }
  }
}
```

## Error Handling

| Setting | Behavior |
| - | - |
| `fail_on_error: false` (default) | Skip failed documents with warning, continue processing |
| `fail_on_error: true` | Fail entire retrieval on first transformation error |

**Common failure causes:**

* Invalid template syntax
* Template rendering errors (missing fields)
* Invalid JSON output from template
* Document missing required fields

> [!TIP]
>
> Use `fail_on_error: false` for public APIs where partial results are acceptable. Use `fail_on_error: true` for internal workflows where data integrity is critical.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 1ms per document |
| **Processing** | Sequential (fast, no caching needed) |
| **Schema** | Output completely defined by template |

## Multi-line Templates

For complex templates, use HEREDOC syntax in the API call:

```bash theme={null}
curl -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -d '{
    "retriever_name": "json_transform_demo",
    "stages": [{
      "stage_name": "json_transform",
      "stage_type": "apply",
      "config": {
        "stage_id": "json_transform",
        "parameters": {
          "template": "{\n  \"id\": \"{{ DOC.document_id }}\",\n  \"title\": {{ DOC.title | tojson }},\n  \"items\": [\n    {% for item in DOC.items %}{\n      \"name\": \"{{ item.name }}\",\n      \"value\": {{ item.value }}\n    }{% if not loop.last %},{% endif %}\n    {% endfor %}\n  ]\n}"
        }
      }
    }]
  }'
```

## Common Patterns

### Drop Unused Fields

```json theme={null}
{
  "template": "{\"id\": \"{{ DOC.document_id }}\", \"title\": \"{{ DOC.title }}\", \"url\": \"{{ DOC.url }}\"}"
}
```

### Flatten Nested Metadata

```json theme={null}
{
  "template": "{\"doc_id\": \"{{ DOC.document_id }}\", \"user_id\": \"{{ DOC.metadata.user_id }}\", \"category\": \"{{ DOC.metadata.category }}\", \"score\": {{ DOC.score }}}"
}
```

### Add Query Context

```json theme={null}
{
  "template": "{\"query\": \"{{ INPUT.query }}\", \"result_id\": \"{{ DOC.document_id }}\", \"score\": {{ DOC.score }}}"
}
```

### Compute a custom ranking signal (recency boost + used-demotion)

`json_transform` is the general-purpose way to compute a **custom ranking score**
at query time: read the relevance score and any document fields, blend in your own
signals, emit the result as a `score`, then order by it with
[`sort_relevance`](/docs/retrieval/stages/sort-relevance). Because every weight is read
from `INPUT`, the same retriever is tunable per request — including an instant
off-switch — with no redeploy.

The pipeline is three stages:

```
feature_search  →  json_transform  →  sort_relevance
  (relevance)      (blend the score)   (order by "score")
```

A worked example: boost fresh documents, and demote any document that already
carries a `used_in_ad` edge. The blended score is

```
score = relevance × (1 + recency_weight × recency_factor) × (1 − used_demotion if used)
```

where `recency_factor` is a gentle linear decay `max(0, 1 − days_old / freshness_days)`.
The template reads each weight from `INPUT` with a default, so callers override only
what they want. This is the exact template that runs in production, prettified for
readability (the verbatim single-line version is in the block below):

```jinja theme={null}
{% set md = DOC['_internal']['metadata'] %}
{% set d  = (md.get('date_created') or '1970-01-01')[:10] %}

{# pseudo-ordinal day count: Y*372 + M*31 + D — monotonic, months≈31d, ±1d vs true days #}
{% set ord_doc  = (d[:4]|int)*372 + (d[5:7]|int)*31 + (d[8:10]|int) %}
{% set asof     = INPUT.get('as_of') or '2026-07-15' %}
{% set ord_asof = (asof[:4]|int)*372 + (asof[5:7]|int)*31 + (asof[8:10]|int) %}
{% set days_old = ord_asof - ord_doc %}

{# tunables — `x if x is not none else default` also catches a client-passed null #}
{% set fw = (INPUT.get('freshness_days') or 14) | int %}
{% set rw = (INPUT.get('recency_weight') if INPUT.get('recency_weight') is not none else 0.3) | float %}
{% set ud = (INPUT.get('used_demotion')  if INPUT.get('used_demotion')  is not none else 0.4) | float %}
{% set on = INPUT.get('boost_enabled')   if INPUT.get('boost_enabled')  is not none else true %}

{# edge-driven, not a flag: does this doc carry a used_in_ad edge? #}
{% set used = ((DOC.get('edges') or []) | selectattr('type', 'equalto', 'used_in_ad') | list | length) > 0 %}

{% set rf    = max(0.0, 1.0 - (days_old / fw)) %}
{% set base  = DOC.get('score') or 0 %}
{% set final = (base * (1 + rw * rf) * (1 - (ud if used else 0))) if on else base %}
{
  "document_id": {{ DOC.document_id | tojson }},
  "title": {{ md.get('title') | tojson }},
  "brand": {{ md.get('brand') | tojson }},
  "job_id": {{ md.get('job_id') | tojson }},
  "date_created": {{ d | tojson }},
  "days_old": {{ days_old }},
  "base_score": {{ base | round(4) }},
  "recency_factor": {{ rf | round(4) }},
  "is_used_in_ad": {{ used | tojson }},
  "new_badge": {{ (days_old <= fw) | tojson }},
  "score": {{ final | round(4) }}
}
```

A few things worth noticing in the real template:

* **User metadata is read from `_internal.metadata`** here (`md`), the path managed-pipeline
  documents use. If your documents carry metadata at the top level or under `metadata`
  instead, adjust the path to match how they were ingested.
* **The default idiom guards `null`, not just missing.** `INPUT.get('recency_weight') if
  INPUT.get('recency_weight') is not none else 0.3` falls back both when the caller omits
  the input **and** when it passes an explicit `null` — `INPUT.get('x', 0.3)` alone would
  keep a client-sent `null`.
* **`days_old` is a pseudo-ordinal** (`Y*372 + M*31 + D`), monotonic and accurate to about
  ±1 day versus true calendar days — deliberately cheap, since exact date math isn't needed
  to rank by freshness.

> **Verbatim single-line template (paste as the template string)**
>
> The production template is authored on one line (newlines are optional in Jinja).
> This is byte-for-byte what ran in prod:
>
> ```jinja theme={null}
> {% set md = DOC['_internal']['metadata'] %}{% set d = (md.get('date_created') or '1970-01-01')[:10] %}{% set ord_doc = (d[:4]|int)*372 + (d[5:7]|int)*31 + (d[8:10]|int) %}{% set asof = INPUT.get('as_of') or '2026-07-15' %}{% set ord_asof = (asof[:4]|int)*372 + (asof[5:7]|int)*31 + (asof[8:10]|int) %}{% set days_old = ord_asof - ord_doc %}{% set fw = (INPUT.get('freshness_days') or 14) | int %}{% set rw = (INPUT.get('recency_weight') if INPUT.get('recency_weight') is not none else 0.3) | float %}{% set ud = (INPUT.get('used_demotion') if INPUT.get('used_demotion') is not none else 0.4) | float %}{% set on = INPUT.get('boost_enabled') if INPUT.get('boost_enabled') is not none else true %}{% set used = ((DOC.get('edges') or []) | selectattr('type','equalto','used_in_ad') | list | length) > 0 %}{% set rf = max(0.0, 1.0 - (days_old / fw)) %}{% set base = DOC.get('score') or 0 %}{% set final = (base * (1 + rw * rf) * (1 - (ud if used else 0))) if on else base %}{"document_id": {{ DOC.document_id | tojson }}, "title": {{ md.get('title') | tojson }}, "brand": {{ md.get('brand') | tojson }}, "job_id": {{ md.get('job_id') | tojson }}, "date_created": {{ d | tojson }}, "days_old": {{ days_old }}, "base_score": {{ base | round(4) }}, "recency_factor": {{ rf | round(4) }}, "is_used_in_ad": {{ used | tojson }}, "new_badge": {{ (days_old <= fw) | tojson }}, "score": {{ final | round(4) }}}
> ```

Then sort by the computed field (it is the default, shown here for clarity):

```json theme={null}
{
  "stage_name": "sort_relevance",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_relevance",
    "parameters": { "score_field": "score", "direction": "desc" }
  }
}
```

> [!NOTE]
>
> `json_transform` **replaces** each document with the template's JSON output, so
> emit every field you want downstream — including the `score` that `sort_relevance`
> reads. Pass the template to the stage as its `template` string (escape it as JSON,
> or use the heredoc form shown under [Multi-line Templates](#multi-line-templates)).

Tunable inputs (declare them in the retriever's `input_schema` as optional):

| Input | Default | Effect |
| - | - | - |
| `recency_weight` | `0.3` | How strongly freshness lifts the score. Raise to let recent docs dominate. |
| `used_demotion` | `0.4` | Fraction to subtract when the doc has a `used_in_ad` edge. |
| `freshness_days` | `14` | Decay window, and the cutoff for the `new_badge` flag. |
| `as_of` | today | Reference date the age is measured from. |
| `boost_enabled` | `true` | Set `false` to bypass all boosting and return pure relevance order — an instant off-switch. |

```bash theme={null}
# default ranking
curl -X POST "$MP_API_URL/v1/retrievers/$RET_ID/execute" -H "Authorization: Bearer $MP_API_KEY" \
  -d '{"inputs": {"query": "footage scene"}}'
# heavier recency, wider window
curl ... -d '{"inputs": {"query": "footage scene", "recency_weight": 2.0, "freshness_days": 30}}'
# off-switch — pure relevance
curl ... -d '{"inputs": {"query": "footage scene", "boost_enabled": false}}'
```

## Related

* [Sort Relevance](/docs/retrieval/stages/sort-relevance) - Order by a score, including one computed here
* [RAG Prepare](/docs/retrieval/stages/rag-prepare) - Format documents for LLM context
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - Extract structured data with LLMs
* [API Call](/docs/retrieval/stages/api-call) - Format for external API calls


# Limit
Source: https://docs.mixpeek.com/docs/retrieval/stages/limit

Truncate results to a maximum count with optional offset for pagination

![Limit stage showing result truncation to top-N documents](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/limit.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=a70b2fed7d43107a58472fcddab2ec5a)

The Limit stage truncates the document set to a maximum number of results, optionally with an offset for pagination-style behavior. This is the retriever pipeline equivalent of SQL's `LIMIT/OFFSET` clause.

> [!NOTE]
>
> **Stage Category**: REDUCE (Truncates documents)
>
> **Transformation**: N documents → min(N, limit) documents

## When to Use

| Use Case | Description |
| - | - |
| **Top-K results** | Return only the best N results after reranking |
| **Pagination** | Implement page-based result access with offset |
| **Cost control** | Cap document count before expensive LLM stages |
| **Fixed output** | Guarantee exactly N results for downstream consumers |
| **Mid-pipeline trim** | Reduce candidates between expensive stages |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Random sampling | `sample` stage |
| Filtering by criteria | `attribute_filter` or `llm_filter` |
| Initial retrieval limit | Set `limit` in `feature_search` directly |
| Statistical reduction | `aggregate` stage |
| Grouping results | `group_by` stage |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `limit` | integer | `10` | Maximum number of documents to return (1-10000) |
| `offset` | integer | `0` | Number of documents to skip from the beginning (0-10000) |

## Configuration Examples

```json Top 10 Results theme={null}
{
  "stage_name": "limit",
  "stage_type": "reduce",
  "config": {
    "stage_id": "limit",
    "parameters": {
      "limit": 10
    }
  }
}
```

```json Pagination (Page 3) theme={null}
{
  "stage_name": "limit",
  "stage_type": "reduce",
  "config": {
    "stage_id": "limit",
    "parameters": {
      "limit": 10,
      "offset": 20
    }
  }
}
```

```json Single Best Result theme={null}
{
  "stage_name": "limit",
  "stage_type": "reduce",
  "config": {
    "stage_id": "limit",
    "parameters": {
      "limit": 1
    }
  }
}
```

```json Cap Before LLM Processing theme={null}
{
  "stage_name": "limit",
  "stage_type": "reduce",
  "config": {
    "stage_id": "limit",
    "parameters": {
      "limit": 25
    }
  }
}
```

> [!TIP]
>
> Place the limit stage after sorting/reranking to ensure you're keeping the highest-quality results. Limiting before reranking loses potentially relevant documents.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 1ms |
| **Memory** | O(1) |
| **Cost** | Free |
| **Complexity** | O(1) list slicing |

## Common Pipeline Patterns

### Rerank Then Limit

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 100}],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 10
      }
    }
  }
]
```

### Cost-Controlled LLM Pipeline

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 200}],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 20
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Summarize: {{DOC.content}}",
        "output_field": "summary"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Limit > input count | Returns all available documents |
| Offset > input count | Returns empty result set |
| Empty input | Returns empty result set |
| Offset + Limit > count | Returns documents from offset to end |

## Related

* [Sample](/docs/retrieval/stages/sample) - Random or stratified sampling
* [Deduplicate](/docs/retrieval/stages/deduplicate) - Remove duplicates before limiting
* [Rerank](/docs/retrieval/stages/rerank) - Re-score before limiting to ensure best results


# LLM Filter
Source: https://docs.mixpeek.com/docs/retrieval/stages/llm-filter

Filter documents using LLM-based content evaluation and criteria matching

![LLM Filter stage showing content-based filtering with language models](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/llm-filter.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=a25e99aac2e2ead150a2ab912d8f6109)

The LLM Filter stage uses language models to evaluate document content against specified criteria, filtering based on semantic understanding rather than metadata fields.

> [!NOTE]
>
> **Stage Category**: FILTER (Reduces document set)
>
> **Transformation**: N documents → M documents (where M ≤ N, based on LLM evaluation)

## When to Use

| Use Case | Description |
| - | - |
| **Content quality filtering** | Remove low-quality or irrelevant content |
| **Semantic criteria** | Filter by meaning, not just keywords |
| **Complex requirements** | "Only technical documentation" |
| **Subjective evaluation** | Tone, style, or sentiment filtering |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple metadata filtering | `attribute_filter` (faster) |
| Large result sets (100+) | Too slow, pre-filter first |
| Deterministic rules | `attribute_filter` |
| Low latency requirements | Use metadata filters |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `provider` | string | `google` | LLM provider: `openai`, `google`, `anthropic`, or `typesafe`. Inferred from `model_name` when omitted |
| `model_name` | string | provider default | LLM model to use |
| `criteria` | string | `Keep only documents relevant to {{INPUT.query}}` | Natural language filter criteria. An empty value skips the stage |
| `include_reasoning` | boolean | `false` | Include the LLM's reasoning for each decision in the stage metadata |
| `include_media` | boolean | `false` | Send each document's media to the model with its fields, so the criteria are judged on what the media shows. A video segment goes to Gemini as video; other providers get the segment's thumbnail. Requests carrying media skip the semantic cache |

Documents are batched automatically based on how many reach the stage.

## Available Models

| Model | Speed | Quality | Cost |
| - | - | - | - |
| `gpt-4o-mini` | Fast | Good | Low |
| `gpt-4o` | Medium | Excellent | Medium |
| `claude-3-haiku` | Fast | Good | Low |
| `claude-3-sonnet` | Medium | Excellent | Medium |
| `jev-latest` | Fast | Calibrated yes/no | Lowest |

`jev-latest` is [Jev](/docs/processing/jev), an evaluation model from TypeSafe AI. It answers "does this document meet the criteria" as a probability instead of generating text. A document is kept when the probability is at least 0.5, and `reason` reports the probability. Jev reads text only, so media documents are judged on their text fields: descriptions, transcripts and metadata.

## Configuration Examples

```json Filter with Jev theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "model_name": "jev-latest",
      "criteria": "Keep only drug advertisements that mention side effects"
    }
  }
}
```

```json Basic Content Filter theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "criteria": "Keep only documents that contain technical information about software development"
    }
  }
}
```

```json Quality Filter theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "criteria": "Filter out documents that are: incomplete, contain mostly ads/spam, or are not in English",
      "include_reasoning": true
    }
  }
}
```

```json Topic Relevance theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-3-haiku",
      "criteria": "Keep documents specifically about {{INPUT.topic}}. Exclude tangentially related or off-topic content."
    }
  }
}
```

```json Professional Tone Filter theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "criteria": "Include only documents with professional, formal tone suitable for business communication. Exclude casual, informal, or inappropriate content."
    }
  }
}
```

```json Factual Content Only theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "criteria": "Keep only factual, informative content. Remove: opinions without evidence, speculation, promotional content, and entertainment-focused material.",
      "include_reasoning": true
    }
  }
}
```

## Writing Effective Criteria

### Good Criteria Examples

```
✓ "Keep documents about machine learning algorithms and their implementations"
✓ "Filter out content that is primarily promotional or marketing material"
✓ "Include only documents written in the last 5 years about cloud computing"
✓ "Keep technical documentation; remove blog posts and news articles"
```

### Poor Criteria Examples

```
✗ "Good documents" (too vague)
✗ "Relevant content" (not specific)
✗ "High quality" (subjective without definition)
```

> [!TIP]
>
> Be specific about what to include AND exclude. The LLM makes a binary keep/discard decision for each document.

## Output Schema

### Filter Output

Documents that pass the filter are returned unchanged:

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Technical documentation about...",
  "metadata": {...}
}
```

### With Reasoning

Documents are still returned unchanged. When `include_reasoning` is `true`, the stage metadata records each decision with the LLM's reason:

```json theme={null}
{
  "llm_decisions": [
    {
      "document_id": "doc_123",
      "keep": true,
      "reason": "Document contains detailed technical information about API implementation."
    }
  ]
}
```

### Filtered Out (not in results)

Documents that don't match criteria are removed from the result set.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 200-500ms per batch |
| **Batch size** | Chosen automatically from the number of input documents |
| **Token usage** | \~100 tokens per document |
| **Parallel** | Batches processed concurrently |

> [!WARNING]
>
> LLM filtering is expensive and slow. Always apply `attribute_filter` or use search `top_k` limits to reduce the document set before LLM filtering.

## Common Pipeline Patterns

### Search + Metadata Filter + LLM Filter

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.type",
        "operator": "eq",
        "value": "documentation"
      }
    }
  },
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Keep only documents that provide actionable, step-by-step instructions"
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 5
      }
    }
  }
]
```

### Quality + Relevance Pipeline

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 30 }
        ],
        "final_top_k": 30
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 15
      }
    }
  },
  {
    "stage_name": "llm_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "llm_filter",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "criteria": "Keep only high-quality, authoritative sources. Remove: user-generated content without verification, outdated information (pre-2020), and incomplete documents."
      }
    }
  }
]
```

## Cost Optimization

| Strategy | Impact |
| - | - |
| Pre-filter with metadata | Reduce documents before LLM |
| Use cheaper models | `gpt-4o-mini` vs `gpt-4o` |
| Limit input documents | Use `top_k` in search |

## Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys. This gives you control over costs, rate limits, and API usage.

### Why Use BYOK?

| Benefit | Description |
| - | - |
| **Cost Control** | Use your own LLM provider account and billing |
| **Rate Limits** | Use your own rate limits instead of shared |
| **Compliance** | Keep API calls under your own account |
| **Key Rotation** | Rotate keys without changing retrievers |

### Setup

**1. Store your API key as a secret**

Store your LLM provider API key in the organization secrets vault:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

**2. Reference the secret in your stage**

Use the `api_key` parameter with template syntax:

```json theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "criteria": "Keep only technical documentation",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}
```

### BYOK Configuration Example

```json OpenAI BYOK theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "openai",
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "criteria": "Keep only high-quality, professional content",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}
```

```json Anthropic BYOK theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "anthropic",
      "provider": "anthropic",
      "model_name": "claude-3-haiku-20240307",
      "criteria": "Filter out promotional or marketing content",
      "api_key": "{{secrets.anthropic_api_key}}"
    }
  }
}
```

```json Google BYOK theme={null}
{
  "stage_name": "llm_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "provider": "google",
      "provider": "google",
      "model_name": "gemini-3.1-flash-lite",
      "criteria": "Keep only factual, informative documents",
      "api_key": "{{secrets.google_api_key}}"
    }
  }
}
```

### Supported Providers

| Provider | Secret Name Example | Models |
| - | - | - |
| OpenAI | `openai_api_key` | gpt-4o, gpt-4o-mini |
| Anthropic | `anthropic_api_key` | claude-3-haiku, claude-3-sonnet, claude-3-opus |
| Google | `google_api_key` | gemini-3.1-flash-lite, gemini-2.5-pro |

> [!NOTE]
>
> When `api_key` is not specified, the stage uses Mixpeek's default API keys and usage is charged to your Mixpeek account.

## Custom Filter Model (BYO Plugin)

Use your own filter model deployed as a [custom extractor](/docs/processing/custom-extractors) instead of a hosted LLM provider. Set `feature_uri` to route filtering through your extractor's inference endpoint.

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | `null` | Feature URI of a custom filter plugin. Overrides `provider`/`model_name` when set. |

Your plugin must accept `{prompt: str, document: dict}` and return `{keep: bool, reason: str}`.

### Configuration Example

```json theme={null}
{
  "stage_name": "my_filter",
  "config": {
    "stage_id": "llm_filter",
    "parameters": {
      "feature_uri": "mixpeek://my_filter_model@1.0.0/filter",
      "criteria": "Keep only documents relevant to the query"
    }
  }
}
```

> [!TIP]
>
> Set `inference_type: "generate"` in your plugin's manifest to declare compatibility with LLM stages.

## Error Handling

| Error | Behavior |
| - | - |
| LLM timeout | Retry once, then fail |
| Rate limit | Automatic backoff |
| Invalid model | Stage fails |
| Empty criteria | All documents pass |
| Invalid API key | Error returned with auth failure |

## Related

* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Metadata-based filtering
* [Rerank](/docs/retrieval/stages/rerank) - Relevance-based ordering
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - Extract data with LLM


# MaxSim Rerank
Source: https://docs.mixpeek.com/docs/retrieval/stages/maxsim-rerank

Rescore candidates with late-interaction token vectors, such as ColBERT's

The MaxSim Rerank stage takes the documents an earlier stage found and scores each one by MaxSim: every query token vector is matched to its most similar token vector in the document, and those best matches are summed. Token vectors come from a multi-vector feature, for example the ColBERT extractor's `mixpeek://colbert_extractor@v1/colbertv2_v1`.

Only the candidates are scored. The vector store reads the token vectors of the documents you pass in and returns one score per document, so the cost grows with the number of candidates.

> [!NOTE]
>
> **Stage Category**: SORT (Rescores and reorders)
>
> **Transformation**: N documents → N documents (rescored, reordered)

## When to Use

| Use Case | Description |
| - | - |
| **Dense search, then late interaction** | A fast dense search finds 100 candidates; MaxSim orders them with ColBERT's token-level precision |
| **Keyword or fused candidates** | Any candidate list works, as long as its documents (or their source objects) have token vectors |
| **Large collections** | Rescoring the top candidates reads far fewer token vectors than exhaustive MaxSim over every document |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | required | The multi-vector feature to score with |
| `query` | string | `{{INPUT.query}}` | Query text, encoded by the feature's own model |
| `top_n` | integer | all, up to 1000 | Rescore only the first `top_n` documents; the rest keep their order after them |
| `match_on` | string | `document_id` | `document_id`: the candidates themselves hold the token vectors. `root_object_id`: the token vectors live in another collection built over the same bucket objects |
| `missing` | string | `bottom` | A candidate with no token vectors: `bottom` keeps it after the scored ones, `drop` removes it |
| `query_vectors` | list of vectors | none | The query's token vectors, if you encoded the query yourself |

## Two collections over one bucket

A collection runs one extractor. To search densely and rerank with ColBERT, create two collections over the same bucket: one with the text extractor, one with the ColBERT extractor. Search the text collection, then rerank with `match_on: root_object_id`. Each candidate is scored with the best MaxSim among the ColBERT documents made from its source object.

```json theme={null}
{
  "stages": [
    {
      "stage_name": "dense",
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    },
    {
      "stage_name": "late-interaction",
      "stage_id": "maxsim_rerank",
      "parameters": {
        "feature_uri": "mixpeek://colbert_extractor@v1/colbertv2_v1",
        "query": "{{INPUT.query}}",
        "top_n": 100,
        "match_on": "root_object_id"
      }
    }
  ]
}
```

## What the stage reports

The stage's metadata in the execution response carries `candidates`, `scored`, `missing_tokens` and `query_tokens`. With `match_on: root_object_id` it also reports `source_objects` and `token_documents_scored`, the number of token documents the vector store scored.


# MMR (Maximal Marginal Relevance)
Source: https://docs.mixpeek.com/docs/retrieval/stages/mmr

Diversify search results by balancing relevance with result variety

![MMR stage showing diversity-aware result selection](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/mmr.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=8fefbc19aa3cb6a1bc6afcd028b5aff8)

The MMR (Maximal Marginal Relevance) stage diversifies search results by iteratively selecting documents that are both relevant to the query and different from already-selected documents. This prevents redundant results and surfaces a broader range of relevant content.

> [!NOTE]
>
> **Stage Category**: SORT (Reorders with diversity)
>
> **Transformation**: N documents → top\_k documents (diverse selection)

## When to Use

| Use Case | Description |
| - | - |
| **Reduce redundancy** | Avoid showing near-duplicate results |
| **Exploration** | Surface different aspects of a topic |
| **Coverage** | Ensure results span multiple subtopics |
| **Recommendation diversity** | Show varied options |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Pure relevance ranking | `rerank` |
| Simple sorting | `sort_by_field` |
| Already diverse corpus | Skip MMR |
| Very small result sets | Not enough to diversify |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `lambda` | float | `0.7` | Balance: 0 = max diversity, 1 = max relevance |
| `top_k` | integer | `25` | Number of results to return |
| `diversity_feature_uri` | string | *auto* | Feature URI of the embedding used for diversity |

## Lambda Parameter

The `lambda` parameter controls the relevance-diversity trade-off:

| Lambda | Behavior |
| - | - |
| `1.0` | Pure relevance (no diversification) |
| `0.7` | Slightly favor relevance |
| `0.5` | Balanced (default) |
| `0.3` | Favor diversity |
| `0.0` | Maximum diversity |

## Configuration Examples

```json Balanced MMR theme={null}
{
  "stage_name": "mmr",
  "stage_type": "sort",
  "config": {
    "stage_id": "mmr",
    "parameters": {
      "lambda": 0.5,
      "top_k": 10
    }
  }
}
```

```json High Diversity theme={null}
{
  "stage_name": "mmr",
  "stage_type": "sort",
  "config": {
    "stage_id": "mmr",
    "parameters": {
      "lambda": 0.3,
      "top_k": 15
    }
  }
}
```

```json Relevance-Focused theme={null}
{
  "stage_name": "mmr",
  "stage_type": "sort",
  "config": {
    "stage_id": "mmr",
    "parameters": {
      "lambda": 0.8,
      "top_k": 10
    }
  }
}
```

```json Custom Diversity Feature theme={null}
{
  "stage_name": "mmr",
  "stage_type": "sort",
  "config": {
    "stage_id": "mmr",
    "parameters": {
      "lambda": 0.5,
      "top_k": 20,
      "diversity_feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
    }
  }
}
```

## How MMR Works

The MMR algorithm iteratively selects documents using:

```
MMR = argmax[λ × Sim(doc, query) - (1-λ) × max(Sim(doc, selected_docs))]
```

1. **First Selection**: Choose the most relevant document
2. **Subsequent Selections**: Balance relevance against similarity to already-selected documents
3. **Repeat**: Until top\_k documents are selected

### Example Selection Process

| Iteration | Selected | Reason |
| - | - | - |
| 1 | Doc A (0.95) | Highest relevance |
| 2 | Doc C (0.82) | High relevance, different from A |
| 3 | Doc E (0.78) | Good relevance, different from A & C |
| 4 | Doc B (0.90) | Skipped earlier due to similarity to A |

## Output Schema

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Document content...",
  "score": 0.85,
  "mmr": {
    "relevance_score": 0.92,
    "diversity_penalty": 0.07,
    "mmr_score": 0.85,
    "selection_order": 3
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 10-50ms |
| **Complexity** | O(N × top\_k) |
| **Memory** | O(N) embeddings |
| **Cost** | Free (no API calls) |

## Common Pipeline Patterns

### Search + MMR

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "mmr",
    "stage_type": "sort",
    "config": {
      "stage_id": "mmr",
      "parameters": {
        "lambda": 0.5,
        "top_k": 10
      }
    }
  }
]
```

### Search + Filter + MMR + Enrich

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.category",
        "operator": "in",
        "value": ["tech", "science", "business"]
      }
    }
  },
  {
    "stage_name": "mmr",
    "stage_type": "sort",
    "config": {
      "stage_id": "mmr",
      "parameters": {
        "lambda": 0.6,
        "top_k": 15
      }
    }
  },
  {
    "stage_name": "llm_enrichment",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "model_name": "gpt-4o-mini",
        "prompt": "Summarize in one sentence",
        "output_field": "summary"
      }
    }
  }
]
```

### Diverse RAG Pipeline

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "mmr",
    "stage_type": "sort",
    "config": {
      "stage_id": "mmr",
      "parameters": {
        "lambda": 0.4,
        "top_k": 8
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "prompt": "Synthesize diverse perspectives on: {{INPUT.query}}\n\n{{DOCUMENTS}}"
      }
    }
  }
]
```

## MMR vs Rerank

| Aspect | MMR | Rerank |
| - | - | - |
| Goal | Diversity + relevance | Maximum relevance |
| Method | Embedding similarity | Cross-encoder scoring |
| Speed | Fast (10-50ms) | Slower (50-100ms) |
| Best for | Exploration, coverage | Precision, accuracy |

## Tuning Lambda

| Use Case | Recommended Lambda |
| - | - |
| News aggregation | 0.3-0.4 (high diversity) |
| Product search | 0.5-0.6 (balanced) |
| Technical docs | 0.7-0.8 (relevance focus) |
| Legal/compliance | 0.8-0.9 (high precision) |

> [!TIP]
>
> Start with lambda=0.5 and adjust based on user feedback. If users complain about redundant results, lower lambda. If they miss relevant results, raise it.

## Error Handling

| Error | Behavior |
| - | - |
| Missing embeddings | Fall back to relevance sort |
| Empty input | Return empty |
| top\_k > input size | Return all documents |
| Invalid lambda | Clamp to \[0, 1] |

## Related

* [Rerank](/docs/retrieval/stages/rerank) - Pure relevance re-scoring
* [Sort Relevance](/docs/retrieval/stages/sort-relevance) - Simple score sorting
* [Sample](/docs/retrieval/stages/sample) - Random diversity


# Moment Group
Source: https://docs.mixpeek.com/docs/retrieval/stages/moment-group

Merge contiguous temporal intervals into consolidated video moments, grouped by parent object

![Moment Group stage merging frame-level intervals into consolidated video moments](https://mintcdn.com/mixpeek/Bj-7wkJ91nGXYgQD/assets/retrievers/moment-group.svg?fit=max\&auto=format\&n=Bj-7wkJ91nGXYgQD\&q=85\&s=0391797d858a7a01e09d04318bffa886)

The Moment Group stage takes frame-level or chunk-level documents with temporal metadata and merges contiguous intervals into consolidated video moments (time ranges), grouped by parent object. Instead of returning individual frames that matched a query, you get precise start/end time ranges pinpointing where in a video the match occurs.

> [!NOTE]
>
> **Stage Category**: REDUCE (Merges intervals into moments)
>
> **Transformation**: N frame/chunk documents → M consolidated moments per parent video

## When to Use

| Use Case | Description |
| - | - |
| **Sub-scene localization** | Pinpoint exactly where in a video a query matches |
| **Moment extraction** | Return time-range clips instead of individual frames |
| **Video search results** | Provide start/end timestamps for player seek |
| **Highlight reels** | Extract the top-scoring moments across a video library |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Grouping by non-temporal fields | `group_by` |
| Time-window aggregations (hour/day/week) | `temporal` |
| Results without temporal metadata | `aggregate` |
| Semantic clustering of results | `cluster` |

## Requirements

* Input documents must have temporal intervals — either `query_chunks` with `start_ms`/`end_ms` (attached by `feature_search` preprocessing) or document-level time fields.
* The `parent_field` must exist on the documents so results can be grouped by source video/object.

> [!WARNING]
>
> **Text query vs file query — pick the right `time_field`.** The default `time_field: "query_chunks"` is only populated when you search with a **file** input (via [query preprocessing](/docs/retrieval/stages/feature-search#query-preprocessing)). For a plain **text** query, the matched video segments carry top-level `start_time`/`end_time` in **seconds**. Set `time_field: "start_time"`; the stage derives the end field by swapping `start`→`end` and scales seconds to milliseconds by the field name, so `merge_tolerance_ms` and the returned moments stay in milliseconds. Leaving `query_chunks` with a text query yields **empty moments** — the most common silent failure here. See the [Video Moment Localization recipe](/docs/retrieval/cookbook#video-moment-localization).

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `parent_field` | string | `"source_object_id"` | Field to group results by parent video/object. Common values: `source_object_id` (standard ingest lineage), `root_object_id` (top-level ancestor), or any custom field. |
| `time_field` | string | `"query_chunks"` | Source of temporal intervals. `"query_chunks"` reads the array `feature_search` attaches (each entry has `start_ms`, `end_ms`, `score`). For document-level timestamps, use a dot-path like `"metadata.start_ms"` — the stage derives the end field by replacing `"start"` with `"end"`. The unit is keyed off the field name: `*_time`, `*_s`, `*_sec`, `*_seconds` are read as seconds and scaled to milliseconds; `*_ms` and anything else pass through as milliseconds. |
| `merge_tolerance_ms` | integer | `2000` | Maximum gap in milliseconds between intervals before they are split into separate moments. `0` = exact-overlap only. Range: 0–60000. |
| `max_moments_per_parent` | integer | `10` | Maximum moments returned per parent, sorted by score (highest first). Range: 1–100. |
| `score_strategy` | string | `"max"` | How to aggregate scores across merged intervals: `"max"`, `"avg"`, or `"sum"`. |
| `min_score` | float | `null` | Drop moments scoring below this threshold. Range: 0.0–1.0. |
| `output_mode` | string | `"annotated"` | `"annotated"` keeps the best-scoring document per parent and attaches a `moments` array (preserves all original document fields). `"moments_only"` emits standalone moment documents with IDs like `moment_{parent}_{start_ms}`. |

## Configuration Examples

```json Basic Moment Grouping theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 2000,
      "max_moments_per_parent": 5,
      "score_strategy": "max",
      "output_mode": "annotated"
    }
  }
}
```

```json Tight Scene Localization theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 1500,
      "max_moments_per_parent": 3,
      "score_strategy": "max",
      "min_score": 0.6,
      "output_mode": "annotated"
    }
  }
}
```

```json Standalone Moment Documents theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "root_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 3000,
      "max_moments_per_parent": 10,
      "score_strategy": "avg",
      "output_mode": "moments_only"
    }
  }
}
```

```json Document-Level Timestamps theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "metadata.start_ms",
      "merge_tolerance_ms": 2000,
      "max_moments_per_parent": 5,
      "score_strategy": "max",
      "output_mode": "annotated"
    }
  }
}
```

## Output Schema

### Annotated Mode (default)

In `annotated` mode, the best-scoring document per parent is preserved with a `moments` array attached:

```json theme={null}
{
  "document_id": "doc_abc123",
  "source_object_id": "video_001",
  "score": 0.92,
  "content": "...",
  "metadata": { "..." : "..." },
  "moments": [
    {
      "start_ms": 12000,
      "end_ms": 18500,
      "score": 0.92,
      "match_count": 4,
      "document_ids": ["doc_abc123", "doc_abc124", "doc_abc125", "doc_abc126"]
    },
    {
      "start_ms": 45000,
      "end_ms": 51000,
      "score": 0.78,
      "match_count": 2,
      "document_ids": ["doc_abc130", "doc_abc131"]
    }
  ]
}
```

### Moments-Only Mode

In `moments_only` mode, standalone moment documents are emitted:

```json theme={null}
{
  "document_id": "moment_video_001_12000",
  "start_ms": 12000,
  "end_ms": 18500,
  "score": 0.92,
  "match_count": 4,
  "document_ids": ["doc_abc123", "doc_abc124", "doc_abc125", "doc_abc126"]
}
```

### Moment Object Fields

| Field | Type | Description |
| - | - | - |
| `start_ms` | integer | Start of the moment in milliseconds |
| `end_ms` | integer | End of the moment in milliseconds |
| `score` | float | Aggregated score (per `score_strategy`) |
| `match_count` | integer | Number of source intervals merged into this moment |
| `document_ids` | array | IDs of the original documents that contributed to this moment |

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-50ms |
| **Memory** | O(N) where N = input documents |
| **Cost** | Free |
| **Scalability** | Efficient — runs in the API layer, no engine call |

## Common Pipeline Patterns

### Video Search with Moment Localization

The canonical use case: search across video frames, then consolidate matches into seekable moments.

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 200
          }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "moment_group",
    "stage_type": "reduce",
    "config": {
      "stage_id": "moment_group",
      "parameters": {
        "parent_field": "source_object_id",
        "time_field": "start_time",
        "merge_tolerance_ms": 1500,
        "max_moments_per_parent": 3,
        "score_strategy": "max",
        "min_score": 0.6,
        "output_mode": "annotated"
      }
    }
  }
]
```

### Moment Extraction with Reranking

Search, rerank for precision, then group into moments:

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 500
          }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "top_k": 100
      }
    }
  },
  {
    "stage_name": "moment_group",
    "stage_type": "reduce",
    "config": {
      "stage_id": "moment_group",
      "parameters": {
        "parent_field": "source_object_id",
        "time_field": "start_time",
        "merge_tolerance_ms": 2000,
        "max_moments_per_parent": 5,
        "score_strategy": "max",
        "output_mode": "annotated"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Missing `parent_field` on document | Document skipped |
| Missing temporal metadata | Document skipped |
| No intervals pass `min_score` | Parent omitted from output |
| Empty input | Empty results returned |

## Related

* [Temporal](/docs/retrieval/stages/temporal) - Time-window aggregations with drift detection
* [Group By](/docs/retrieval/stages/group-by) - Group documents by any field value
* [Aggregate](/docs/retrieval/stages/aggregate) - Statistical aggregations
* [Deduplicate](/docs/retrieval/stages/deduplicate) - Remove duplicate documents


# Retriever Stages
Source: https://docs.mixpeek.com/docs/retrieval/stages/overview

The core of the warehouse query layer: composable retriever stages for multi-stage search pipelines

Retriever stages are the building blocks of search pipelines in Mixpeek. Each stage performs a specific operation on the document set, allowing you to compose complex retrieval workflows from simple, reusable components. Where a traditional database returns rows, the warehouse query layer returns ranked, enriched, and fused multimodal results assembled stage by stage.

> **[Compose stages over your own data](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_stages_overview\&utm_campaign=managed_onramp)**
>
> Create a managed namespace, index your files, then chain these stages into a retriever you can query.

## Stage Categories

Stages are organized into six categories based on how they transform the document set:

> **FILTER**
>
> Reduce the document set by matching criteria. Outputs a subset of input documents.
>
> **Stages**: feature\_search, attribute\_filter, llm\_filter, agent\_search, query\_expand

> **SORT**
>
> Reorder documents by relevance or field values. Same documents, different order.
>
> **Stages**: sort\_relevance, sort\_attribute, mmr, rerank, score\_normalize

> **REDUCE**
>
> Collapse results into aggregated values. Produces a single value or smaller set from the input.
>
> **Stages**: aggregate, temporal, sample, summarize, limit, deduplicate, moment\_group, score\_threshold

> **GROUP**
>
> Reshape results by bucketing documents into logical groups or clusters.
>
> **Stages**: group\_by, cluster

> **APPLY**
>
> Transform or restructure documents. May reshape fields, create new documents, or call external services.
>
> **Stages**: json\_transform, rag\_prepare, external\_web\_search, api\_call, sql\_lookup, cross\_compare, web\_scrape, unwind, code\_execution

> **ENRICH**
>
> Add knowledge to documents using AI models, taxonomies, or cross-collection joins.
>
> **Stages**: llm\_enrich, taxonomy\_enrich, document\_enrich, agentic\_enrich

## All Stages

### Filter Stages

| Stage | Description |
| - | - |
| [Feature Search](/docs/retrieval/stages/feature-search) | Search by vector similarity using multimodal embeddings |
| [Attribute Filter](/docs/retrieval/stages/attribute-filter) | Filter by metadata fields with boolean logic (AND/OR/NOT) |
| [LLM Filter](/docs/retrieval/stages/llm-filter) | Semantic filtering using LLM-based evaluation |
| [Agent Search](/docs/retrieval/stages/agent-search) | LLM-driven multi-step retrieval with iterative reasoning and tool orchestration |
| [Query Expand](/docs/retrieval/stages/query-expand) | LLM-powered query expansion with RRF result fusion |

### Sort Stages

| Stage | Description |
| - | - |
| [Sort Relevance](/docs/retrieval/stages/sort-relevance) | Reorder by relevance scores |
| [Sort Attribute](/docs/retrieval/stages/sort-attribute) | Order by any metadata field (dates, price, etc.) |
| [MMR](/docs/retrieval/stages/mmr) | Diversify results with Maximal Marginal Relevance |
| [Rerank](/docs/retrieval/stages/rerank) | Re-score with cross-encoder models (e.g., BGE reranker) |
| [Score Normalize](/docs/retrieval/stages/score-normalize) | Rescale scores to a common range for consistent comparison |

### Reduce Stages

| Stage | Description |
| - | - |
| [Aggregate](/docs/retrieval/stages/aggregate) | Compute COUNT, SUM, AVG, percentile, stddev, frequency, correlation on results |
| [Temporal](/docs/retrieval/stages/temporal) | Group by time windows (hour/day/week/month/quarter/year) with drift detection |
| [Sample](/docs/retrieval/stages/sample) | Random or stratified sampling of results |
| [Summarize](/docs/retrieval/stages/summarize) | Condense documents into an LLM-generated summary |
| [Limit](/docs/retrieval/stages/limit) | Truncate results to a maximum count with optional offset |
| [Deduplicate](/docs/retrieval/stages/deduplicate) | Remove duplicate documents by field or content similarity |
| [Moment Group](/docs/retrieval/stages/moment-group) | Merge contiguous temporal intervals into consolidated video moments |
| [Score Threshold](/docs/retrieval/stages/score-threshold) | Drop results below an absolute score; return none when nothing qualifies |

### Group Stages

| Stage | Description |
| - | - |
| [Group By](/docs/retrieval/stages/group-by) | Group documents by field value (decompose/recompose) |
| [Cluster](/docs/retrieval/stages/cluster) | Discover themes via embedding-based clustering |

### Apply Stages

| Stage | Description |
| - | - |
| [JSON Transform](/docs/retrieval/stages/json-transform) | Reshape documents using Jinja2 templates |
| [RAG Prepare](/docs/retrieval/stages/rag-prepare) | Format for LLM context with token management and citations |
| [External Web Search](/docs/retrieval/stages/external-web-search) | Augment with Exa AI-native web search |
| [API Call](/docs/retrieval/stages/api-call) | Enrich with external REST API responses |
| [SQL Lookup](/docs/retrieval/stages/sql-lookup) | Join with PostgreSQL data |
| [Cross Compare](/docs/retrieval/stages/cross-compare) | Multi-tier cross-collection matching with classification |
| [Web Scrape](/docs/retrieval/stages/web-scrape) | Extract full page content from URLs |
| [Unwind](/docs/retrieval/stages/unwind) | Decompose array fields into separate documents |
| [Code Execution](/docs/retrieval/stages/code-execution) | Execute Python/TypeScript/JavaScript in sandboxes |

### Enrich Stages

| Stage | Description |
| - | - |
| [LLM Enrich](/docs/retrieval/stages/llm-enrich) | Generate new fields with LLM prompts |
| [Taxonomy Enrich](/docs/retrieval/stages/taxonomy-enrich) | Classify documents against taxonomy nodes |
| [Document Enrich](/docs/retrieval/stages/document-enrich) | Cross-collection joins (LEFT JOIN) |
| [Agentic Enrich](/docs/retrieval/stages/agentic-enrich) | Multi-turn agent with tool access for complex classification |

## Pipeline Patterns

### Basic RAG Pipeline

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 50}],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content",
        "top_k": 10
      }
    }
  },
  {
    "stage_name": "rag_prepare",
    "stage_type": "apply",
    "config": {
      "stage_id": "rag_prepare",
      "parameters": {
        "max_tokens": 8000,
        "output_mode": "single_context"
      }
    }
  }
]
```

### E-Commerce Search

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 100}],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "AND": [
            {"field": "metadata.in_stock", "operator": "eq", "value": true},
            {"field": "metadata.price", "operator": "lte", "value": "{{INPUT.max_price}}"}
          ]
        }
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.{{INPUT.sort_by}}",
        "direction": "{{INPUT.sort_order}}"
      }
    }
  }
]
```

### Research Assistant

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 100}],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "external_web_search",
    "stage_type": "apply",
    "config": {
      "stage_id": "external_web_search",
      "parameters": {
        "query": "{{INPUT.query}}",
        "num_results": 10,
        "category": "research_paper"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "top_k": 15
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "google",
        "model_name": "gemini-2.5-flash-lite",
        "prompt": "Synthesize findings on: {{INPUT.query}}\n\n{{DOCUMENTS}}"
      }
    }
  }
]
```

### Enriched Document Retrieval

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 20}],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "document_enrich",
      "parameters": {
        "target_collection_id": "col_users",
        "source_field": "metadata.author_id",
        "target_field": "user_id",
        "output_field": "author"
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Extract key topics and entities from: {{DOC.content}}",
        "output_field": "analysis"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "top_k": 10
      }
    }
  }
]
```

## Stage Selection Guide

| Goal | Recommended Stage |
| - | - |
| Find semantically similar documents | feature\_search |
| Filter by metadata fields | attribute\_filter |
| Filter by content meaning | llm\_filter |
| Improve recall with query variations | query\_expand |
| Get best relevance ranking | rerank |
| Order by price/date/rating | sort\_attribute |
| Re-sort by relevance scores | sort\_relevance |
| Diversify results | mmr |
| Normalize scores across sources | score\_normalize |
| Suppress weak results / "no good results" | score\_threshold |
| Truncate to top-N results | limit |
| Remove duplicate results | deduplicate |
| Expand array fields to documents | unwind |
| Answer questions from docs | summarize |
| Compute statistics on results | aggregate |
| Find themes in results | cluster |
| Locate moments/scenes in video | moment\_group |
| Group by category/author | group\_by |
| Random/stratified sampling | sample |
| Add external API data | api\_call |
| Add database data | sql\_lookup |
| Join Mixpeek collections | document\_enrich |
| Classify documents | taxonomy\_enrich |
| Complex multi-step classification | agentic\_enrich |
| Generate new fields with LLM | llm\_enrich |
| Transform document structure | json\_transform |
| Prepare for LLM context | rag\_prepare |
| Custom code transformations | code\_execution |
| Add web search results | external\_web\_search |
| Extract URL content | web\_scrape |

## Performance Considerations

| Stage | Typical Latency | Cost |
| - | - | - |
| feature\_search | 5-50ms | Index storage |
| attribute\_filter | \< 5ms | Free |
| llm\_filter | 200-500ms | LLM API |
| query\_expand | 300-800ms | LLM API |
| rerank | 50-100ms | Inference |
| sort\_attribute | \< 5ms | Free |
| sort\_relevance | \< 5ms | Free |
| mmr | 10-50ms | Free |
| score\_normalize | \< 1ms | Free |
| score\_threshold | \< 1ms | Free |
| limit | \< 1ms | Free |
| deduplicate | 5-50ms | Free |
| unwind | \< 5ms | Free |
| summarize | 500-2000ms | LLM API |
| aggregate | 5-50ms | Free |
| cluster | 50-200ms | Inference |
| moment\_group | 5-50ms | Free |
| group\_by | 5-20ms | Free |
| sample | \< 5ms | Free |
| llm\_enrich | 300-800ms | LLM API |
| agentic\_enrich | 2-30s | LLM API (multi-turn) |
| api\_call | 50-500ms | External API |
| sql\_lookup | 10-100ms | Database |
| code\_execution | 5-50ms | Free |
| rag\_prepare | \< 10ms | Free |
| json\_transform | \< 5ms | Free |
| external\_web\_search | 100-500ms | Exa API |
| taxonomy\_enrich | 20-100ms | Inference |
| document\_enrich | 10-50ms | Database |
| web\_scrape | 500-5000ms | External |

> [!TIP]
>
> Order stages efficiently: cheap operations (filters, sorts) before expensive ones (rerank, LLM calls). This reduces the document count before costly processing.

## Template Variables

All stages support template variables for dynamic configuration:

| Variable | Description |
| - | - |
| `{{INPUT.*}}` | Input parameters from retriever call |
| `{{DOC.*}}` | Document fields (in APPLY, ENRICH, and GROUP stages) |
| `{{CONTEXT.*}}` | Pipeline context (index, citations) |

```json theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.tenant_id",
      "operator": "eq",
      "value": "{{INPUT.tenant_id}}"
    }
  }
}
```


# Query Expand
Source: https://docs.mixpeek.com/docs/retrieval/stages/query-expand

Generate query variations using LLMs and fuse results for improved recall

![Query Expand stage showing LLM-powered query variations and result fusion](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/query-expand.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=6c5d62605e1456dee20bb14c69a306ce)

The Query Expand stage uses language models to generate multiple query variations from the original query, executes searches for each variation, and fuses the results. This improves recall by capturing different phrasings and aspects of the user's intent.

![Query expansion explained: an ambiguous two-word query (jaguar speed) is expanded by an LLM into multiple reformulations covering each interpretation, each searches independently, and results fuse via Reciprocal Rank Fusion into one broader ranked list; HyDE instead generates a hypothetical answer document and embeds that paragraph for more signal. Multi-query improves recall for ambiguous queries, HyDE improves precision for short single-intent queries, and both can be combined.](https://mintcdn.com/mixpeek/7Pcnzu2AXsKNt5KO/assets/query-expansion.png?fit=max\&auto=format\&n=7Pcnzu2AXsKNt5KO\&q=85\&s=2562fa1e32816cb4898f15ee8a9a5448)

> [!NOTE]
>
> **Stage Category**: FILTER (Generates and fuses search results)
>
> **Transformation**: 1 query → N query variations → fused results

## When to Use

| Use Case | Description |
| - | - |
| **Improved recall** | Capture documents that match alternative phrasings |
| **Ambiguous queries** | Handle queries with multiple interpretations |
| **Synonym expansion** | Find documents using different terminology |
| **Multi-aspect search** | Break complex queries into sub-queries |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple keyword search | `feature_search` directly |
| Low latency requirements | Pre-compute expansions |
| Precise single-intent queries | Standard search |
| Cost-sensitive applications | Use simpler search |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_search_config` | object | *Required* | Full [`feature_search`](/docs/retrieval/stages/feature-search) stage parameters. The query text in its first search is the original query, and each variation replaces the query text in every search |
| `num_expansions` | integer | `3` | Number of query variations to generate (1 to 10) |
| `expansion_model` | string | `gpt-4o-mini` | LLM model for query generation |
| `expansion_prompt` | string | *auto* | Custom prompt for query generation. Use `{{QUERY}}` for the original query and `{{NUM_EXPANSIONS}}` for the count |
| `include_original` | boolean | `true` | Also run the original query alongside the variations |
| `fusion_strategy` | string | `rrf` | Result fusion: `rrf` or `linear` |
| `rrf_k` | integer | `60` | RRF constant k (1 to 1000). Lower values (20 to 40) favor precision, higher values (80 to 100) favor recall |
| `deduplicate` | boolean | `true` | Deduplicate the fused results by `document_id` |
| `expansion_timeout_ms` | integer | server default | Time budget in milliseconds for the expansion LLM call, up to 5000. On timeout the search runs on the original query only and the response carries a warning |

## Fusion Methods

| Method | Description | Best For |
| - | - | - |
| `rrf` | Reciprocal Rank Fusion | General purpose, balanced |
| `linear` | Average of a document's scores across the queries it appears in | When scores are comparable |

## Configuration Examples

```json Basic Query Expansion theme={null}
{
  "stage_name": "query_expand",
  "stage_type": "filter",
  "config": {
    "stage_id": "query_expand",
    "parameters": {
      "num_expansions": 3,
      "feature_search_config": {
        "searches": [
          {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 30}
        ],
        "final_top_k": 30
      }
    }
  }
}
```

```json Custom Expansion Prompt theme={null}
{
  "stage_name": "query_expand",
  "stage_type": "filter",
  "config": {
    "stage_id": "query_expand",
    "parameters": {
      "num_expansions": 5,
      "expansion_prompt": "Generate {{NUM_EXPANSIONS}} alternative phrasings of this query, focusing on different synonyms and related concepts: {{QUERY}}",
      "feature_search_config": {
        "searches": [
          {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 20}
        ],
        "final_top_k": 20
      }
    }
  }
}
```

```json High-Recall Configuration theme={null}
{
  "stage_name": "query_expand",
  "stage_type": "filter",
  "config": {
    "stage_id": "query_expand",
    "parameters": {
      "expansion_model": "gpt-4o",
      "num_expansions": 5,
      "fusion_strategy": "rrf",
      "feature_search_config": {
        "searches": [
          {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 50}
        ],
        "final_top_k": 50
      }
    }
  }
}
```

```json Domain-Specific Expansion theme={null}
{
  "stage_name": "query_expand",
  "stage_type": "filter",
  "config": {
    "stage_id": "query_expand",
    "parameters": {
      "num_expansions": 4,
      "expansion_prompt": "Generate {{NUM_EXPANSIONS}} medical search queries that capture different aspects of: {{QUERY}}. Include medical terminology and layman's terms.",
      "feature_search_config": {
        "searches": [
          {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 25}
        ],
        "final_top_k": 25
      }
    }
  }
}
```

## How Query Expansion Works

1. **Original Query**: "how to fix memory leaks"
2. **LLM Generates Variations**:
   * "memory leak detection and resolution"
   * "debugging memory issues in applications"
   * "preventing memory leaks in code"
3. **Execute Searches**: Run the `feature_search_config` for the original query (unless `include_original` is `false`) and for each variation, concurrently
4. **Fuse Results**: Combine using RRF or linear fusion
5. **Return**: Deduplicated, ranked result set

## Reciprocal Rank Fusion (RRF)

RRF combines results from multiple queries using the formula:

```
score(doc) = Σ 1 / (k + rank_i)
```

Where `k` is `rrf_k` (60 by default), and `rank_i` is the document's rank in query i's results.

| Advantage | Description |
| - | - |
| Score-agnostic | Works with different scoring scales |
| Rank-based | Focuses on relative ordering |
| Self-balancing | No manual weight tuning |

## Output Schema

Each returned document carries the fused result in its `score`. With `rrf`, a document ranked first by all four queries scores 4 / (60 + 1), about 0.066.

The stage metadata reports how the run went:

```json theme={null}
{
  "original_query": "how to fix memory leaks",
  "expansions": [
    "memory leak detection and resolution",
    "debugging memory issues in applications",
    "preventing memory leaks in code"
  ],
  "queries_executed": 4,
  "llm_calls": 1,
  "expansion_timed_out": false,
  "fusion_strategy": "rrf",
  "rrf_k": 60,
  "total_results_before_fusion": 120,
  "final_result_count": 74
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 300-800ms (LLM + searches) |
| **LLM calls** | 1 per execution |
| **Search calls** | `num_expansions` + 1, run concurrently (`num_expansions` when `include_original` is `false`) |
| **Token usage** | \~50-100 tokens |

> [!WARNING]
>
> Query expansion adds latency due to LLM generation and multiple searches. Use judiciously for queries where recall improvement justifies the cost.

## Common Pipeline Patterns

### Expanded Search + Rerank

```json theme={null}
[
  {
    "stage_name": "query_expand",
    "stage_type": "filter",
    "config": {
      "stage_id": "query_expand",
      "parameters": {
        "num_expansions": 3,
        "feature_search_config": {
          "searches": [
            {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 50}
          ],
          "final_top_k": 50
        }
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]
```

### Expansion + Filter + Summarize

```json theme={null}
[
  {
    "stage_name": "query_expand",
    "stage_type": "filter",
    "config": {
      "stage_id": "query_expand",
      "parameters": {
        "num_expansions": 4,
        "feature_search_config": {
          "searches": [
            {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 30}
          ],
          "final_top_k": 30
        }
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.verified",
        "operator": "eq",
        "value": true
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "google",
        "model_name": "gemini-2.5-flash-lite",
        "prompt": "Answer this question: {{INPUT.query}}\n\nBased on these documents:\n{{DOCUMENTS}}"
      }
    }
  }
]
```

## Cost Optimization

| Strategy | Impact |
| - | - |
| Reduce `num_expansions` | Fewer searches |
| Use cheaper LLM | `expansion_model` of `gpt-4o-mini` vs `gpt-4o` |
| Lower `top_k` in `feature_search_config` | Less fusion overhead |
| Cache common expansions | Reduce LLM calls |

## Error Handling

| Error | Behavior |
| - | - |
| LLM failure | Fall back to original query, with a warning in the response |
| Empty expansions | Use original query only |
| Expansion timeout | Search runs on the original query only, with a warning that names `expansion_timeout_ms` |

## Related

* [Feature Search](/docs/retrieval/stages/feature-search) - Single query search
* [Feature Search](/docs/retrieval/stages/feature-search) - Vector + text search
* [Rerank](/docs/retrieval/stages/rerank) - Re-score fused results


# RAG Prepare
Source: https://docs.mixpeek.com/docs/retrieval/stages/rag-prepare

Prepare documents for LLM context windows with token management and source citations — evidence/provenance per result so every claim traces back to a source clip

![RAG Prepare stage showing document formatting for LLM context](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/rag-prepare.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=d27d7a01f367d2643a5d9bf2737a654a)

The RAG Prepare stage formats search results for LLM consumption by managing token budgets, formatting documents, and adding citations. This is a **preparation stage that does NOT call an LLM** - it prepares content for downstream LLM stages or external LLM calls.

> [!NOTE]
>
> **Stage Category**: APPLY
>
> **Transformation**:
>
> * `single_context` mode: N documents → 1 combined context document
> * `formatted_list` mode: N documents → N formatted documents

## When to Use

| Use Case | Description |
| - | - |
| **Before LLM generation** | Prepare context for summarization or Q\&A |
| **Token budget management** | Fit multiple docs into context window |
| **Citation tracking** | Enable source attribution in responses |
| **Consistent formatting** | Standardize document format for LLM input |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Want LLM to generate summary | `summarize` stage (calls LLM) |
| Don't need token management | Pass documents directly |
| Simple pass-through | Skip this stage |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `max_tokens` | integer | `8000` | Maximum tokens for combined output |
| `tokenizer` | string | `cl100k_base` | Tokenizer to use (GPT-4 compatible) |
| `truncation_strategy` | string | `priority_truncate` | How to handle token overflow |
| `output_mode` | string | `single_context` | Output format |
| `document_template` | string | `[{{CONTEXT.INDEX}}] {{DOC.content}}\n\n` | Template for each document |
| `content_field` | string | `content` | Field to extract content from |
| `separator` | string | `\n` | Separator between documents |
| `citation` | object | `{style: "numbered"}` | Citation configuration |

### Truncation Strategies

| Strategy | Behavior |
| - | - |
| `priority_truncate` | Include docs in score order, truncate last to fit |
| `proportional` | Give each doc proportional token budget |
| `drop_last` | Include complete docs until limit, drop remaining |

### Output Modes

| Mode | Output | Use Case |
| - | - | - |
| `single_context` | 1 document with combined `context` string | Direct LLM input |
| `formatted_list` | N documents with `formatted_content` field | Custom processing |

## Configuration Examples

```json Basic RAG Context theme={null}
{
  "stage_name": "rag_prepare",
  "stage_type": "apply",
  "config": {
    "stage_id": "rag_prepare",
    "parameters": {
      "max_tokens": 8000,
      "output_mode": "single_context"
    }
  }
}
```

```json With Numbered Citations theme={null}
{
  "stage_name": "rag_prepare",
  "stage_type": "apply",
  "config": {
    "stage_id": "rag_prepare",
    "parameters": {
      "max_tokens": 4000,
      "document_template": "[{{CONTEXT.INDEX}}] {{DOC.metadata.title}}\n{{DOC.content}}\n\n",
      "truncation_strategy": "priority_truncate",
      "citation": {
        "style": "numbered",
        "include_title": true
      }
    }
  }
}
```

```json Large Context Window (GPT-4 Turbo) theme={null}
{
  "stage_name": "rag_prepare",
  "stage_type": "apply",
  "config": {
    "stage_id": "rag_prepare",
    "parameters": {
      "max_tokens": 32000,
      "tokenizer": "cl100k_base",
      "truncation_strategy": "proportional"
    }
  }
}
```

```json Formatted List Mode theme={null}
{
  "stage_name": "rag_prepare",
  "stage_type": "apply",
  "config": {
    "stage_id": "rag_prepare",
    "parameters": {
      "output_mode": "formatted_list",
      "document_template": "Source: {{DOC.metadata.source}}\n{{DOC.content}}"
    }
  }
}
```

```json Custom Template with URL theme={null}
{
  "stage_name": "rag_prepare",
  "stage_type": "apply",
  "config": {
    "stage_id": "rag_prepare",
    "parameters": {
      "max_tokens": 8000,
      "document_template": "Document {{CONTEXT.INDEX}}:\nTitle: {{DOC.metadata.title}}\nURL: {{DOC.metadata.url}}\n\n{{DOC.content}}\n\n---\n",
      "citation": {
        "style": "bracketed",
        "include_url": true
      }
    }
  }
}
```

## Template Placeholders

| Placeholder | Description |
| - | - |
| `{{CONTEXT.INDEX}}` | 1-based position in result set (1, 2, 3...) |
| `{{CONTEXT.CITATION}}` | Citation marker based on citation.style |
| `{{DOC.*}}` | Any document field (e.g., `{{DOC.content}}`, `{{DOC.metadata.title}}`) |

## Citation Styles

| Style | Output | Example |
| - | - | - |
| `numbered` | `[1]`, `[2]`, `[3]` | Default, clean |
| `bracketed` | `[doc_id]` | Document ID references |
| `footnote` | Superscript numbers | Academic style |
| `none` | No citations | When not needed |

## Output Schema

### single\_context Mode

```json theme={null}
{
  "rag_context": "[1] First document content...\n\n[2] Second document content...",
  "citations": [
    {"index": 1, "title": "Document Title", "document_id": "doc_123"},
    {"index": 2, "title": "Another Title", "document_id": "doc_456"}
  ]
}
```

### formatted\_list Mode

Each document gets:

```json theme={null}
{
  "document_id": "doc_123",
  "formatted_content": "[1] Title\nContent here...",
  "original_content": "Content here...",
  "metadata": {...}
}
```

## Citing Timestamped Source Clips (Video / Audio)

The default `citation` object is `{index, title, document_id}` — enough to name the
source, but it does **not** inline the timestamp or a link back to the source clip.
For investigative / RAG use cases where every claim must resolve to a *timestamped,
clickable source* (e.g. "watch this at 03:12"), surface the anchor yourself — the
data is already on every document, so this needs **no new stage or endpoint**:

* Video/audio segment documents carry `start_time_s` / `end_time_s`
  (see [Universal Extractor → Output](/docs/processing/extractors/universal)).
* Every document carries `_internal.lineage.root_object_id` pointing back to the
  original source object (see [Lineage Traversal](/docs/retrieval/lineage-traversal)).

Reference those fields directly in `document_template` so the timestamp travels into
the LLM context (and into any answer the LLM cites):

```json Timestamped clip citations theme={null}
{
  "stage_name": "rag_prepare",
  "stage_type": "apply",
  "config": {
    "stage_id": "rag_prepare",
    "parameters": {
      "max_tokens": 8000,
      "output_mode": "single_context",
      "document_template": "[{{CONTEXT.INDEX}}] {{DOC.metadata.title}} @ {{DOC.start_time_s}}s–{{DOC.end_time_s}}s (source: {{DOC._internal.lineage.root_object_id}})\n{{DOC.text}}\n\n",
      "citation": { "style": "numbered", "include_title": true }
    }
  }
}
```

To render a **clickable** clip link, hydrate the `document_id` returned in the
`citations[]` array (or the `root_object_id` above) into a playable URL with
`GET /v1/documents/{id}?expand=root_object` — see
[Lineage Traversal](/docs/retrieval/lineage-traversal). Do this in the app layer after
`execute`; the timestamp and source object are preserved end-to-end in the document,
they are simply not inlined into the structured `citations[]` array by default.

> [!NOTE]
>
> Citations attach at the **document** level. `rag_prepare` (and `summarize`'s
> `source_document_ids`) cite the set of source documents behind the context, not
> individual sentences — per-claim/per-sentence attribution is up to the downstream
> LLM prompt. Ask the LLM to emit the `[n]` marker inline with each claim so the
> numbered citation resolves back to the timestamped clip above.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 10ms |
| **Token counting** | Uses tiktoken (accurate) |
| **No LLM calls** | Pure formatting |

> [!TIP]
>
> This stage does NOT call an LLM. It only formats content for LLM consumption. Use the `summarize` stage if you want LLM-generated summaries.

## Common Pipeline Patterns

### Search + Prepare + External LLM

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  },
  {
    "stage_name": "rag_prepare",
    "stage_type": "apply",
    "config": {
      "stage_id": "rag_prepare",
      "parameters": {
        "max_tokens": 8000,
        "output_mode": "single_context",
        "citation": {"style": "numbered"}
      }
    }
  }
]
```

The output `rag_context` can then be passed to an external LLM call.

### vs Summarize Stage

| Feature | rag\_prepare | summarize |
| - | - | - |
| Calls LLM | No | Yes |
| Output | Formatted context | Generated summary |
| Latency | \< 10ms | 500-2000ms |
| Cost | Free | LLM API costs |
| Use case | Prepare for external LLM | End-to-end RAG |

## Related

* [Summarize](/docs/retrieval/stages/summarize) - LLM-powered summarization
* [JSON Transform](/docs/retrieval/stages/json-transform) - Custom document formatting
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - Extract structured data with LLM


# Rerank
Source: https://docs.mixpeek.com/docs/retrieval/stages/rerank

Re-score and reorder search results using cross-encoder models for higher precision

![Rerank stage showing cross-encoder model re-scoring search results](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/rerank.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=21889adcc72377962074d258050939f9)

The Rerank stage uses cross-encoder models to re-score and reorder search results. Unlike bi-encoder models (used in semantic search), cross-encoders process the query and document together, enabling more accurate relevance scoring at the cost of higher latency.

> [!NOTE]
>
> **Stage Category**: SORT (Reorders documents)
>
> **Transformation**: N documents → top\_k documents (re-ranked by relevance)

## When to Use

| Use Case | Description |
| - | - |
| **Two-stage retrieval** | Fast recall (search) + precise ranking (rerank) |
| **High-precision requirements** | When ranking quality is critical |
| **Top-N optimization** | Improve quality of final displayed results |
| **RAG applications** | Better context selection for LLM generation |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Large result sets (1000+) | Too slow; use `sort_by_field` |
| Real-time requirements (\< 20ms) | Use search scores directly |
| Simple attribute sorting | `sort_by_field` |

> [!WARNING]
>
> **Budget for this stage before you add it.** A cross-encoder scores every candidate against the query, so its cost grows with the number of candidates you hand it, not with `top_k`.
>
> Measured on a production pipeline, `BAAI__bge_reranker_v2_m3` took **p50 3.26s** on its own, against a retriever whose search, filter and index stages together took **\~38ms**. The rerank was 99% of a 4.4s p50.
>
> Set `top_k` on the stage feeding it. Reranking 20 candidates and keeping 10 costs a fraction of reranking 200.

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `inference_name` | string | `BAAI__bge_reranker_v2_m3` | Reranking inference service. List options with `GET /engine/inference`. Ignored when `feature_uri` is set. |
| `feature_uri` | string | `null` | Custom reranker plugin (`mixpeek://...`). Overrides `inference_name`. |
| `top_k` | integer | `null` | Number of results to keep after reranking (omit to keep all, reordered). |
| `max_candidates` | integer | `10` | Score at most this many incoming documents, in the order the previous stage returned them; the rest follow unscored in that order. Set a larger number (up to 1000), or `null` to score everything, when you need deeper reranking and can accept the latency. |
| `query` | string | `{{INPUT.query}}` | Query for relevance scoring |
| `document_field` | string | `content` | Document field to rerank against |
| `relevance_criteria` | string | `null` | What counts as relevant, stated as a condition. Used by `typesafe__jev` and `mixpeek__decision`; cross-encoders ignore it. |
| `decision_model` | string | `null` | Model that answers for `mixpeek__decision` or `typesafe__jev`. `jev-latest` is Jev; a generative model such as `gemini-2.5-flash-lite` answers the same question. Unset means the deployment default for `mixpeek__decision` and Jev for `typesafe__jev`. |
| `learning_config` | object | `null` | The stage's learning block. `evaluation_dataset_id` names the evaluation a trained reranker is scored on: a trained set ships only when it scores higher there than the live configuration, and without it nothing ships. `shipped_weight_set` is read-only, written by the evaluation gate (`weight_set_id`, `evaluation_id`, `baseline_evaluation_id`, `shipped_at`); a request that sets it is refused. |

## Available Models

| `inference_name` | Model | Score |
| - | - | - |
| `BAAI__bge_reranker_v2_m3` | Multilingual cross-encoder | Normalized relevance score |
| `typesafe__jev` | [Jev](/docs/processing/jev), TypeSafe AI's evaluation model | Probability (0 to 1) that the document meets `relevance_criteria` |
| `mixpeek__decision` | The [decision model](/docs/processing/jev#swap-the-decision-model) named by `decision_model` | Probability (0 to 1) that the document meets `relevance_criteria`, calibrated only when Jev answers |

Jev scores each query and document pair in its own request, so no document affects another document's score. Pairs run concurrently. Set `relevance_criteria` to the exact condition you rank on. Jev reads it word for word.

With a decision model, stage metadata names the model that scored the documents in `model_used` and says whether its probabilities are calibrated in `probabilities_calibrated`.

On 100 [SciFact](https://github.com/beir-cellar/beir) claims, reranking the top 30 vector results with Jev raised nDCG\@10 from 0.817 to 0.880 at 669 ms per query. The same rerank through `mixpeek__decision` with `gemini-2.5-flash-lite` scored 0.806 at 2.1 s.

For other models, deploy your own via a [custom extractor](/docs/processing/custom-extractors) and reference it with `feature_uri`.

## Configuration Examples

```json Rerank with Jev theme={null}
{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "typesafe__jev",
      "relevance_criteria": "The text is a drug advertisement that discloses side effects",
      "document_field": "text",
      "top_k": 10
    }
  }
}
```

```json Basic Reranking theme={null}
{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "BAAI__bge_reranker_v2_m3",
      "top_k": 10
    }
  }
}
```

```json High-Quality Reranking theme={null}
{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "BAAI__bge_reranker_v2_m3",
      "top_k": 5
    }
  }
}
```

```json Custom Query theme={null}
{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "BAAI__bge_reranker_v2_m3",
      "query": "{{INPUT.question}}",
      "top_k": 20
    }
  }
}
```

```json Large Candidate Pool theme={null}
{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "BAAI__bge_reranker_v2_m3",
      "top_k": 50
    }
  }
}
```

## How Cross-Encoders Work

| Bi-Encoder (Search) | Cross-Encoder (Rerank) |
| - | - |
| Query and doc encoded separately | Query + doc encoded together |
| Pre-compute doc embeddings | Must process each pair |
| Fast (\< 10ms for millions) | Slower (50-100ms for 100 docs) |
| Good approximate ranking | Precise relevance scoring |

Cross-encoders see the full context of both query and document together, enabling better understanding of semantic relationships.

## Two-Stage Retrieval Pattern

The recommended pattern is fast recall followed by precise reranking:

```json theme={null}
[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]
```

**Why this works:**

1. **Search stage**: Fast, retrieves 100 candidates (\< 20ms)
2. **Rerank stage**: Slower but precise, picks best 10 (50-100ms)
3. **Total**: High-quality results in 70-120ms

## Performance

| Metric | Value |
| - | - |
| **Latency** | 50-100ms (depends on candidate count) |
| **Optimal input size** | 50-200 documents |
| **Maximum practical** | \~500 documents |
| **Batching** | Automatic |

> [!WARNING]
>
> Reranking 1000+ documents is not recommended. Use `top_k` limits in the search stage to control candidate pool size.

## Output

Each returned document includes:

| Field | Type | Description |
| - | - | - |
| `document_id` | string | Unique document identifier |
| `score` | float | Reranker relevance score |
| `original_score` | float | Score from previous stage |
| `rerank_position` | integer | Position after reranking |

## Common Pipeline Patterns

### Search + Rerank + Limit

```json theme={null}
[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 20
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 5
      }
    }
  }
]
```

### Search + Filter + Rerank

```json theme={null}
[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 200
          }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "category_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.category",
        "operator": "eq",
        "value": "{{INPUT.category}}"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]
```

## Custom Reranker (BYO Model)

Use your own reranker model deployed as a [custom extractor](/docs/processing/custom-extractors) instead of the built-in models. Set `feature_uri` to route reranking through your extractor's inference endpoint.

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | `null` | Feature URI of a custom reranker plugin. Overrides `inference_name` when set. |

Your plugin must accept `{pairs: [[query, doc], ...]}` and return `{scores: [float, ...]}`.

### Configuration Example

```json theme={null}
{
  "stage_name": "my_rerank",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "feature_uri": "mixpeek://my_reranker@1.0.0/rerank",
      "top_k": 10
    }
  }
}
```

> [!TIP]
>
> Set `inference_type: "rerank"` in your plugin's manifest to declare compatibility with the rerank stage.

## Trade-offs

| Aspect | Impact |
| - | - |
| **Higher precision** | Better relevance scoring |
| **Higher latency** | 50-100ms per batch |
| **Limited scale** | Best for \< 500 candidates |
| **API costs** | Per-document scoring |

## Related

* [Feature Search](/docs/retrieval/stages/feature-search) - Initial candidate retrieval
* [Feature Search](/docs/retrieval/stages/feature-search) - Combined vector + text search
* [Sort Attribute](/docs/retrieval/stages/sort-attribute) - Simple attribute sorting


# Sample
Source: https://docs.mixpeek.com/docs/retrieval/stages/sample

Select a random or stratified sample of documents from results

![Sample stage showing random and stratified sampling of results](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/sample.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=19f8bd5a9bd579e271fe9afaed60a975)

The Sample stage selects a subset of documents from your results using random or stratified sampling. This is useful for creating representative samples, reducing result sets, or ensuring diversity across categories.

> [!NOTE]
>
> **Stage Category**: REDUCE (Samples documents)
>
> **Transformation**: N documents → S sampled documents (where S ≤ N)

## When to Use

| Use Case | Description |
| - | - |
| **Representative samples** | Get a sample of large result sets |
| **A/B testing** | Random document selection |
| **Stratified selection** | Equal representation per category |
| **Cost reduction** | Sample before expensive operations |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Top-N by relevance | `limit` or `rerank` |
| Diversity by similarity | `mmr` |
| Remove duplicates | `deduplicate` |
| All results needed | Skip sampling |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `strategy` | string | `random` | Sampling strategy: `random`, `stratified`, `reservoir` |
| `count` | integer | `10` | Number of documents to sample |
| `seed` | integer | *random* | Random seed for reproducibility |
| `stratify_by` | string | *none* | Field for stratified sampling |
| `min_per_stratum` | integer | `1` | Minimum samples per stratum |
| `preserve_top_k` | integer | `0` | Always keep top K by score, sample the rest |

## Sampling Strategies

| Strategy | Description | Best For |
| - | - | - |
| `random` | Uniform random selection | General sampling |
| `stratified` | Proportional samples per group | Category balance |
| `reservoir` | Memory-efficient sampling | Very large result sets |

## Configuration Examples

```json Random Sample theme={null}
{
  "stage_name": "sample",
  "stage_type": "reduce",
  "config": {
    "stage_id": "sample",
    "parameters": {
      "strategy": "random",
      "count": 20
    }
  }
}
```

```json Reproducible Random Sample theme={null}
{
  "stage_name": "sample",
  "stage_type": "reduce",
  "config": {
    "stage_id": "sample",
    "parameters": {
      "strategy": "random",
      "count": 10,
      "seed": 42
    }
  }
}
```

```json Stratified by Category theme={null}
{
  "stage_name": "sample",
  "stage_type": "reduce",
  "config": {
    "stage_id": "sample",
    "parameters": {
      "strategy": "stratified",
      "stratify_by": "metadata.category",
      "count": 30,
      "min_per_stratum": 3
    }
  }
}
```

```json Reservoir Sampling theme={null}
{
  "stage_name": "sample",
  "stage_type": "reduce",
  "config": {
    "stage_id": "sample",
    "parameters": {
      "strategy": "reservoir",
      "count": 25
    }
  }
}
```

```json Stratified with Total Limit theme={null}
{
  "stage_name": "sample",
  "stage_type": "reduce",
  "config": {
    "stage_id": "sample",
    "parameters": {
      "strategy": "stratified",
      "stratify_by": "metadata.source",
      "count": 30
    }
  }
}
```

## How Sampling Works

### Random Sampling

Selects documents with uniform probability:

```
Input: [A, B, C, D, E, F, G, H, I, J] (10 docs)
Sample(count=3): [D, G, B] (random selection)
```

### Stratified Sampling

Ensures representation from each group:

```
Input:
  - Category A: [A1, A2, A3, A4, A5]
  - Category B: [B1, B2, B3]
  - Category C: [C1, C2]

Stratified(min_per_stratum=2):
  [A1, A3, B2, B1, C1, C2]
```

### Reservoir Sampling

Memory-efficient uniform sampling for very large or streaming result sets:

```
Input: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] (processed one at a time)
Reservoir(count=3): [2, 6, 9] (uniform random, single pass)
```

## Output Schema

```json theme={null}
{
  "documents": [
    {
      "document_id": "doc_123",
      "content": "Sampled document content...",
      "score": 0.85,
      "sample": {
        "method": "stratified",
        "stratum": "electronics",
        "sample_index": 0
      }
    }
  ],
  "metadata": {
    "method": "stratified",
    "total_input": 100,
    "sample_size": 15,
    "strata": {
      "electronics": {"input": 45, "sampled": 5},
      "clothing": {"input": 35, "sampled": 5},
      "books": {"input": 20, "sampled": 5}
    }
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 5ms |
| **Memory** | O(N) |
| **Cost** | Free |
| **Complexity** | O(N) random, O(N log N) stratified |

## Common Pipeline Patterns

### Search + Sample for Testing

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 1000 }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "strategy": "random",
        "count": 50,
        "seed": 42
      }
    }
  }
]
```

### Balanced Category Sample

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 500 }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "strategy": "stratified",
        "stratify_by": "metadata.category",
        "count": 25,
        "min_per_stratum": 5
      }
    }
  }
]
```

### Sample Before LLM Processing

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 200 }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 50
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "strategy": "random",
        "count": 10
      }
    }
  },
  {
    "stage_name": "llm_enrichment",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "model_name": "gpt-4o",
        "prompt": "Extract key insights",
        "output_field": "insights"
      }
    }
  }
]
```

### Cluster + Sample Representatives

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 200 }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "cluster",
    "stage_type": "reduce",
    "config": {
      "stage_id": "cluster",
      "parameters": {
        "n_clusters": 10
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "strategy": "stratified",
        "stratify_by": "cluster.cluster_id",
        "count": 20,
        "min_per_stratum": 2
      }
    }
  }
]
```

### Multi-Source Balanced Sample

```json theme={null}
[
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.date",
        "operator": "gte",
        "value": "2024-01-01"
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "strategy": "stratified",
        "stratify_by": "metadata.source",
        "count": 30,
        "min_per_stratum": 10
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "prompt": "Compare perspectives from different sources on this topic\n\n{{DOCUMENTS}}"
      }
    }
  }
]
```

## Stratified Sampling Details

### Minimum Per Stratum

```json theme={null}
{
  "strategy": "stratified",
  "stratify_by": "metadata.category",
  "count": 30,
  "min_per_stratum": 5
}
```

Each group is guaranteed at least 5 samples (if available), with the remainder allocated proportionally up to `count`.

### Proportional Allocation

```json theme={null}
{
  "strategy": "stratified",
  "stratify_by": "metadata.category",
  "count": 30
}
```

Stratified sampling allocates samples proportional to group size by default.

## Reproducibility

Use `seed` for reproducible results:

```json theme={null}
{
  "strategy": "random",
  "count": 20,
  "seed": 12345
}
```

Same seed + same input = same output.

## Error Handling

| Error | Behavior |
| - | - |
| count > input | Return all documents |
| Empty stratum | Skip that stratum |
| Invalid stratify\_by | Fall back to random |
| count = 0 | Return empty |

## Sample vs Other Reduction Stages

| Stage | Selection Basis | Deterministic |
| - | - | - |
| `sample` | Random/Stratified | With seed |
| `limit` | Position | Yes |
| `mmr` | Diversity + relevance | Yes |
| `deduplicate` | Uniqueness | Yes |

## Related

* [Aggregate](/docs/retrieval/stages/aggregate) - Statistical analysis
* [Group By](/docs/retrieval/stages/group-by) - Group before sampling
* [Cluster](/docs/retrieval/stages/cluster) - Semantic grouping
* [MMR](/docs/retrieval/stages/mmr) - Diversity-based selection


# Score Normalize
Source: https://docs.mixpeek.com/docs/retrieval/stages/score-normalize

Rescale document scores to a common range for consistent comparison

![Score normalize stage showing score rescaling from raw to normalized range](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/score-normalize.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=eaecf3f5c9379dc1d76f3e6044d70b86)

The Score Normalize stage rescales document scores using statistical normalization methods, enabling meaningful comparison across different scoring sources and consistent downstream thresholding.

> [!NOTE]
>
> **Stage Category**: SORT (Rescales scores)
>
> **Transformation**: N documents → N documents (same order, normalized scores)

## When to Use

| Use Case | Description |
| - | - |
| **Hybrid search fusion** | Normalize text and vector scores before combining |
| **Score thresholding** | Set consistent cutoffs across different retrievers |
| **Cross-model comparison** | Make scores from different models comparable |
| **Probability ranking** | Convert scores to probability distribution |
| **Multi-stage pipelines** | Normalize between reranking stages |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Reordering by relevance | `sort_relevance` |
| Reranking with cross-encoders | `rerank` |
| Filtering by score threshold | `attribute_filter` on score field |
| Single scoring source | Scores are already comparable |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `method` | string | `min_max` | Normalization method: `min_max`, `z_score`, `softmax`, `l2` |
| `score_field` | string | `score` | Field containing the score to normalize |
| `output_field` | string | `null` | Write normalized score to this field (preserves original) |
| `min_value` | float | `null` | Custom minimum for min\_max (uses actual min if null) |
| `max_value` | float | `null` | Custom maximum for min\_max (uses actual max if null) |

## Normalization Methods

| Method | Formula | Output Range | Best For |
| - | - | - | - |
| `min_max` | (x - min) / (max - min) | \[0, 1] | Bounded comparison |
| `z_score` | (x - mean) / std | (-∞, +∞) | Statistical thresholding |
| `softmax` | exp(x) / Σexp | (0, 1), sum=1 | Probability distribution |
| `l2` | x / ‖x‖₂ | \[-1, 1] | Geometric comparison |

## Configuration Examples

```json Min-Max to [0,1] theme={null}
{
  "stage_name": "score_normalize",
  "stage_type": "sort",
  "config": {
    "stage_id": "score_normalize",
    "parameters": {
      "method": "min_max",
      "score_field": "score"
    }
  }
}
```

```json Z-Score with Separate Output theme={null}
{
  "stage_name": "score_normalize",
  "stage_type": "sort",
  "config": {
    "stage_id": "score_normalize",
    "parameters": {
      "method": "z_score",
      "score_field": "score",
      "output_field": "z_score"
    }
  }
}
```

```json Softmax Probability theme={null}
{
  "stage_name": "score_normalize",
  "stage_type": "sort",
  "config": {
    "stage_id": "score_normalize",
    "parameters": {
      "method": "softmax",
      "score_field": "score",
      "output_field": "probability"
    }
  }
}
```

```json Custom Range Bounds theme={null}
{
  "stage_name": "score_normalize",
  "stage_type": "sort",
  "config": {
    "stage_id": "score_normalize",
    "parameters": {
      "method": "min_max",
      "min_value": 0.0,
      "max_value": 1.0
    }
  }
}
```

> [!TIP]
>
> Use `output_field` to preserve the original score alongside the normalized value. This is useful for debugging or when you need both raw and normalized scores downstream.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 1ms |
| **Memory** | O(N) for score array |
| **Cost** | Free |
| **Complexity** | O(N) (two passes: stats + normalize) |

## Common Pipeline Patterns

### Hybrid Search Fusion

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 50}],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "score_normalize",
    "stage_type": "sort",
    "config": {
      "stage_id": "score_normalize",
      "parameters": {
        "method": "min_max",
        "score_field": "score"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  }
]
```

### Score Thresholding After Normalization

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 100}],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "score_normalize",
    "stage_type": "sort",
    "config": {
      "stage_id": "score_normalize",
      "parameters": {
        "method": "min_max"
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "AND": [
            {"field": "score", "operator": "gte", "value": 0.5}
          ]
        }
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Single document | min\_max returns 1.0; z\_score returns 0.0 |
| All same scores | min\_max returns 1.0 for all; z\_score returns 0.0 for all |
| Score field missing | Treated as 0.0 |
| Non-numeric score | Treated as 0.0 |

## Related

* [Sort Relevance](/docs/retrieval/stages/sort-relevance) - Reorder by relevance scores
* [Rerank](/docs/retrieval/stages/rerank) - Re-score with cross-encoder models
* [Sort Attribute](/docs/retrieval/stages/sort-attribute) - Sort by any metadata field


# Score Threshold
Source: https://docs.mixpeek.com/docs/retrieval/stages/score-threshold

Drop results below an absolute score and return no results when nothing qualifies

The Score Threshold stage applies an **absolute** quality gate: it drops every document whose score on a chosen field fails a minimum bar. When nothing clears the bar it returns an **empty** result set and sets `all_below_threshold` — the signal your UI uses to show a "no good results" state instead of presenting weak matches.

> [!NOTE]
>
> **Stage Category**: REDUCE (N documents → ≤ N documents)
>
> **Transformation**: keeps only documents meeting `min_score`; the set may become empty.

## Why not just normalize and filter?

`score_normalize` with `min_max` always rescales the **top** result to `1.0` — so a threshold on the normalized score can *never* reject an all-bad result set (the best match is always 1.0). Score Threshold gates on the **raw or calibrated** score, so "everything is below the bar" is expressible. Threshold on a calibrated score — the [`rerank`](/docs/retrieval/stages/rerank) cross-encoder score is ideal (`score_field: "scores.rerank"`), since it is far more absolute and comparable than a raw cosine similarity.

## When to Use

| Use Case | Description |
| - | - |
| **Suppress weak matches** | Don't show results that aren't good enough to be useful |
| **"No good results" UX** | Branch to a no-results / suggestion state via `all_below_threshold` |
| **Quality gate after rerank** | Hard-gate on the calibrated cross-encoder score |
| **Confidence cutoffs** | Only surface high-confidence matches to end users |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Rescale scores for comparison | [`score_normalize`](/docs/retrieval/stages/score-normalize) |
| Keep top-N regardless of quality | [`limit`](/docs/retrieval/stages/limit) |
| Filter by metadata fields | [`attribute_filter`](/docs/retrieval/stages/attribute-filter) |
| Reorder by score | [`sort_relevance`](/docs/retrieval/stages/sort-relevance) |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `min_score` | float | *(required)* | Absolute minimum score a document must meet to be kept |
| `score_field` | string | `score` | Score field to gate on. Dot-paths supported (e.g. `scores.rerank`, `metadata.quality`) |
| `comparison` | string | `gte` | Keep docs whose score is `gte` (≥) or `gt` (strictly >) `min_score` |
| `missing_score` | string | `drop` | What to do with documents lacking `score_field`: `drop` or `keep` |

## Response Metadata

| Field | Description |
| - | - |
| `all_below_threshold` | `true` when **no** document met the bar (empty result set) |
| `input_count` / `output_count` | Documents in / kept |
| `dropped` | Documents removed |
| `missing_score_docs` | Documents that lacked `score_field` |

## Configuration Examples

```json Gate on the rerank score theme={null}
{
  "stage_name": "score_threshold",
  "stage_type": "reduce",
  "config": {
    "stage_id": "score_threshold",
    "parameters": {
      "min_score": 0.5,
      "score_field": "scores.rerank"
    }
  }
}
```

```json Strict cutoff on raw score (calibrate the number first) theme={null}
{
  "stage_name": "score_threshold",
  "stage_type": "reduce",
  "config": {
    "stage_id": "score_threshold",
    "parameters": {
      "min_score": 0.35,
      "comparison": "gt"
    }
  }
}
```

```json Keep docs missing the score field theme={null}
{
  "stage_name": "score_threshold",
  "stage_type": "reduce",
  "config": {
    "stage_id": "score_threshold",
    "parameters": {
      "min_score": 0.5,
      "score_field": "scores.rerank",
      "missing_score": "keep"
    }
  }
}
```

> [!WARNING]
>
> **A raw cosine score does not span 0 to 1, so do not copy a threshold between retrievers.**
> Each embedding model occupies its own narrow band, and a value that works on one model
> rejects every document on another.
>
> Measured on a production multimodal retriever (`multimodal_extractor@v2/gemini-embedding-2`,
> video segments, 27 text queries): on-topic queries scored 0.3755 to 0.4311 and off-topic
> controls scored 0.3136 to 0.3456. The separation is clean, and the whole range sits between
> 0.31 and 0.44. A `min_score` of 0.5 or 0.7 on that retriever returns zero documents for every
> query, including exact matches, and sets `all_below_threshold` every time.
>
> Read your own numbers before you pick a bar. Run the retriever without this stage, look at the
> top-1 score for a query that should match and one that should not, and put the bar between them.
> On the retriever above that puts the bar at 0.35, which was verified in both directions:
> `cars driving on a highway` keeps all 10 documents with `all_below_threshold` false, and
> `a library reading room with wooden shelves` drops all 10 and sets it true.

## "No Good Results → Suggestions" Pattern

Gate on the rerank score; when nothing qualifies, the empty result + `all_below_threshold` tells your application to fall back to [`query_expand`](/docs/retrieval/stages/query-expand) for adjacent suggestions instead of showing weak matches.

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}}
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  },
  {
    "stage_name": "score_threshold",
    "stage_type": "reduce",
    "config": {
      "stage_id": "score_threshold",
      "parameters": {
        "min_score": 0.5,
        "score_field": "scores.rerank"
      }
    }
  }
]
```

> [!TIP]
>
> Calibrate `min_score` empirically: run representative queries (including ones that *should* return nothing) and pick the value that separates good from bad on the rerank score. The right cutoff is query-domain specific.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 1ms |
| **Memory** | O(N) |
| **Cost** | Free |
| **Complexity** | O(N) (single pass) |

## Related

* [Score Normalize](/docs/retrieval/stages/score-normalize) - Rescale scores to a common range
* [Rerank](/docs/retrieval/stages/rerank) - Calibrated cross-encoder scores to gate on
* [Query Expand](/docs/retrieval/stages/query-expand) - Adjacent suggestions when nothing qualifies
* [Limit](/docs/retrieval/stages/limit) - Truncate to top-N regardless of score


# Sort Attribute
Source: https://docs.mixpeek.com/docs/retrieval/stages/sort-attribute

Reorder documents by any metadata field value

![Sort Attribute stage showing attribute-based document ordering](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/sort-attribute.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=08a095cd3221ca92e9351162374f6d19)

The Sort Attribute stage reorders documents based on metadata field values. It supports ascending/descending order, nested fields, and multiple sort criteria.

> [!NOTE]
>
> **Stage Category**: SORT (Reorders documents)
>
> **Transformation**: N documents → N documents (reordered by field value)

## When to Use

| Use Case | Description |
| - | - |
| **Price sorting** | Order products by price |
| **Recency sorting** | Most recent first |
| **Rating sorting** | Highest rated first |
| **Alphabetical** | Sort by name or title |
| **Custom ranking** | Sort by any numeric field |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Relevance ranking | `rerank` (neural re-scoring) |
| Complex scoring logic | `api_call` to custom service |
| Already sorted by search | Skip this stage |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `field` | string | `metadata.created_at` | Field path to sort by |
| `direction` | string | `desc` | Sort direction: `asc` or `desc` |
| `nulls_last` | boolean | `true` | Place null or missing values at the end of the results regardless of direction. When `false`, nulls follow the sort order: first for `asc`, last for `desc` |

## Configuration Examples

```json Price Low to High theme={null}
{
  "stage_name": "sort_attribute",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_attribute",
    "parameters": {
      "field": "metadata.price",
      "direction": "asc"
    }
  }
}
```

```json Newest First theme={null}
{
  "stage_name": "sort_attribute",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_attribute",
    "parameters": {
      "field": "metadata.created_at",
      "direction": "desc"
    }
  }
}
```

```json Highest Rated theme={null}
{
  "stage_name": "sort_attribute",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_attribute",
    "parameters": {
      "field": "metadata.rating",
      "direction": "desc",
      "nulls_last": true
    }
  }
}
```

```json Alphabetical theme={null}
{
  "stage_name": "sort_attribute",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_attribute",
    "parameters": {
      "field": "metadata.title",
      "direction": "asc"
    }
  }
}
```

```json Nested Field theme={null}
{
  "stage_name": "sort_attribute",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_attribute",
    "parameters": {
      "field": "metadata.reviews.average_rating",
      "direction": "desc"
    }
  }
}
```

```json By Relevance Score theme={null}
{
  "stage_name": "sort_attribute",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_attribute",
    "parameters": {
      "field": "score",
      "direction": "desc"
    }
  }
}
```

## Field Types

The stage handles various field types:

| Type | Sort Behavior |
| - | - |
| Number | Numeric comparison |
| String | Lexicographic (alphabetical) |
| Date/ISO8601 | Chronological |
| Boolean | false \< true |

## Null Handling

| `nulls_last` | Behavior |
| - | - |
| `true` | Null/missing values always appear last, whatever the direction (default) |
| `false` | Null/missing values follow the sort order: first for `asc`, last for `desc` |

```json theme={null}
{
  "field": "metadata.optional_field",
  "direction": "desc",
  "nulls_last": true
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 5ms |
| **Complexity** | O(n log n) |
| **Memory** | In-place sort |
| **Index support** | Uses indexes when available |

> [!TIP]
>
> Sorting is very fast. Unlike `rerank`, it doesn't require model inference, making it ideal for simple ordering by attributes.

## Common Pipeline Patterns

### Search + Filter + Sort

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 50
          }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.in_stock",
        "operator": "eq",
        "value": true
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.price",
        "direction": "asc"
      }
    }
  },
  {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
      "stage_id": "sample",
      "parameters": {
        "count": 10
      }
    }
  }
]
```

### Rerank + Sort (Tie-Breaking)

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 20
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.created_at",
        "direction": "desc"
      }
    }
  }
]
```

### Dynamic Sort Direction

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "{{INPUT.sort_by}}",
        "direction": "{{INPUT.sort_order}}"
      }
    }
  }
]
```

## Comparison: sort\_attribute vs rerank

| Feature | sort\_attribute | rerank |
| - | - | - |
| Based on | Field values | Query relevance |
| Speed | \< 5ms | 50-100ms |
| Model required | No | Yes |
| Use case | Attribute ordering | Relevance scoring |
| Cost | Free | API calls |

## Multiple Sort Criteria

For multi-field sorting, chain multiple sort stages (last sort is primary):

```json theme={null}
[
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.created_at",
        "direction": "desc"
      }
    }
  },
  {
    "stage_name": "sort_attribute",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_attribute",
      "parameters": {
        "field": "metadata.featured",
        "direction": "desc"
      }
    }
  }
]
```

This sorts by `featured` first, then by `created_at` for ties.

## Error Handling

| Error | Behavior |
| - | - |
| Field not found | Treated as null |
| Type mismatch | String comparison fallback |
| Invalid order | Defaults to `desc` |

## Related

* [Rerank](/docs/retrieval/stages/rerank) - Neural relevance ranking
* [Sample](/docs/retrieval/stages/sample) - Reduce result count
* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Filter before sorting


# Sort Relevance
Source: https://docs.mixpeek.com/docs/retrieval/stages/sort-relevance

Reorder documents by their relevance scores from previous stages

![Sort Relevance stage showing document reordering by relevance scores](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/sort-relevance.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=feed410c224a3f23047428fbdb2c904f)

The Sort Relevance stage reorders documents based on their relevance scores from previous stages. This is useful when documents have been modified or filtered and need to be re-sorted by their original search scores.

> [!NOTE]
>
> **Stage Category**: SORT (Reorders documents)
>
> **Transformation**: N documents → N documents (reordered by relevance)

## When to Use

| Use Case | Description |
| - | - |
| **Post-filter sorting** | Re-sort after attribute\_filter removes documents |
| **Score normalization** | Apply consistent sorting after multi-stage processing |
| **Restoring order** | Return to relevance order after other sort operations |
| **Combining scores** | Sort by combined scores from multiple sources |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Sorting by metadata field | `sort_by_field` |
| Neural re-scoring | `rerank` |
| Results already sorted | Skip this stage |
| Diversifying results | `mmr` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `score_field` | string | `score` | Field containing relevance score |
| `direction` | string | `desc` | Sort direction: `desc` (highest first) or `asc` |
| `missing_score` | string | `bottom` | Where docs lacking a score go: `bottom`, `top`, or `preserve` |

## Configuration Examples

```json Basic Relevance Sort theme={null}
{
  "stage_name": "sort_relevance",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_relevance",
    "parameters": {
      "direction": "desc"
    }
  }
}
```

```json Custom Score Field theme={null}
{
  "stage_name": "sort_relevance",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_relevance",
    "parameters": {
      "score_field": "search_score",
      "direction": "desc"
    }
  }
}
```

```json Missing Score Handling theme={null}
{
  "stage_name": "sort_relevance",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_relevance",
    "parameters": {
      "score_field": "score",
      "direction": "desc",
      "missing_score": "bottom"
    }
  }
}
```

```json Ascending Order theme={null}
{
  "stage_name": "sort_relevance",
  "stage_type": "sort",
  "config": {
    "stage_id": "sort_relevance",
    "parameters": {
      "score_field": "distance",
      "direction": "asc"
    }
  }
}
```

## How It Works

1. **Extract Scores**: Read the score field from each document
2. **Sort**: Order documents by score (descending by default)
3. **Optionally Normalize**: Scale scores to 0-1 range
4. **Return**: Documents in new order

## Output Schema

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Document content...",
  "score": 0.95,
  "sort_relevance": {
    "original_position": 3,
    "new_position": 1,
    "normalized_score": 1.0
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 5ms |
| **Memory** | O(N) |
| **Cost** | Free |
| **Complexity** | O(N log N) |

## Common Pipeline Patterns

### Search + Filter + Re-sort

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.status",
        "operator": "eq",
        "value": "published"
      }
    }
  },
  {
    "stage_name": "sort_relevance",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_relevance",
      "parameters": {
        "direction": "desc"
      }
    }
  }
]
```

### Multi-Stage with Score Normalization

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "llm_enrichment",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "model_name": "gpt-4o-mini",
        "prompt": "Extract key topics",
        "output_field": "topics"
      }
    }
  },
  {
    "stage_name": "sort_relevance",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_relevance",
      "parameters": {
        "direction": "desc"
      }
    }
  }
]
```

## Comparison with Other Sort Stages

| Stage | Purpose | Score Source |
| - | - | - |
| `sort_relevance` | Relevance scores | Search/fusion scores |
| `sort_by_field` | Metadata values | Any document field |
| `rerank` | Neural re-scoring | Cross-encoder model |
| `mmr` | Diversity | Relevance + diversity |

## Error Handling

| Error | Behavior |
| - | - |
| Missing score field | Use 0 as default |
| Non-numeric score | Move to end |
| Empty input | Return empty |
| Equal scores | Maintain original order |

## Related

* [Rerank](/docs/retrieval/stages/rerank) - Neural re-scoring
* [Sort Attribute](/docs/retrieval/stages/sort-attribute) - Metadata sorting
* [MMR](/docs/retrieval/stages/mmr) - Diversity-aware sorting


# SQL Lookup
Source: https://docs.mixpeek.com/docs/retrieval/stages/sql-lookup

Enrich documents with data from SQL databases using parameterized queries

![SQL Lookup stage showing database enrichment with parameterized queries](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/sql-lookup.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=d2f0f6d36e4fa48114bd6ab9741e9fc6)

The SQL Lookup stage enriches documents by querying external SQL databases. It executes parameterized queries using document fields as inputs, joining external structured data with your search results.

> [!NOTE]
>
> **Stage Category**: APPLY (Enriches documents)
>
> **Transformation**: N documents → N documents (with SQL data added)

## When to Use

| Use Case | Description |
| - | - |
| **Customer data enrichment** | Join customer details from CRM database |
| **Inventory lookups** | Add real-time stock levels to product results |
| **Pricing data** | Enrich with current pricing from ERP |
| **Cross-system joins** | Combine vector search with relational data |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple key-value lookups | `document_enrich` (collection join) |
| Read-heavy, cacheable data | Consider pre-indexing in Mixpeek |
| Real-time transactional queries | Direct database access |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `connection_id` | string | *Required* | Database connection identifier |
| `query` | string | *Required* | SQL query with parameter placeholders |
| `parameters` | object | `{}` | Mapping of placeholder names to document fields |
| `output_field` | string | `sql_result` | Dot-path where query results are stored |
| `result_handling` | string | `first` | How to handle multiple rows (`first`, `all`, `error_if_empty`, `error_if_multiple`) |
| `on_no_results` | string | `null` | Behavior when no rows returned (`skip`, `null`, `error`) |
| `timeout` | integer | `30` | Query timeout in seconds (1-300) |
| `on_error` | string | `skip` | Error handling strategy (`skip`, `remove`, `raise`) |

## Supported Databases

| Database | Connection Type | Notes |
| - | - | - |
| PostgreSQL | `postgresql` | Create the connection as shown in [PostgreSQL](/docs/integrations/databases/postgresql) |

A connection to any other provider fails with `Connection does not support SQL queries`. For Snowflake or another warehouse, use the [API Call](/docs/retrieval/stages/api-call) stage against an endpoint that runs the query. See [Join Warehouse Metrics onto Results](#join-warehouse-metrics-onto-results).

## Configuration Examples

```json Basic Customer Lookup theme={null}
{
  "stage_name": "sql_lookup",
  "stage_type": "apply",
  "config": {
    "stage_id": "sql_lookup",
    "parameters": {
      "connection_id": "crm_postgres",
      "query": "SELECT name, email, tier FROM customers WHERE id = $1",
      "parameters": {
        "customer_id": "{{DOC.metadata.customer_id}}"
      },
      "output_field": "customer_data"
    }
  }
}
```

```json Inventory Enrichment theme={null}
{
  "stage_name": "sql_lookup",
  "stage_type": "apply",
  "config": {
    "stage_id": "sql_lookup",
    "parameters": {
      "connection_id": "inventory_db",
      "query": "SELECT stock_level, warehouse_location, last_updated FROM inventory WHERE sku = $1",
      "parameters": {
        "sku": "{{DOC.metadata.sku}}"
      },
      "output_field": "inventory",
      "timeout": 5
    }
  }
}
```

```json Multiple Row Results theme={null}
{
  "stage_name": "sql_lookup",
  "stage_type": "apply",
  "config": {
    "stage_id": "sql_lookup",
    "parameters": {
      "connection_id": "orders_db",
      "query": "SELECT order_id, status, total FROM orders WHERE customer_id = $1 ORDER BY created_at DESC LIMIT 5",
      "parameters": {
        "cid": "{{DOC.metadata.customer_id}}"
      },
      "output_field": "recent_orders",
      "result_handling": "all"
    }
  }
}
```

```json Join with Multiple Fields theme={null}
{
  "stage_name": "sql_lookup",
  "stage_type": "apply",
  "config": {
    "stage_id": "sql_lookup",
    "parameters": {
      "connection_id": "product_db",
      "query": "SELECT price, currency, discount_pct FROM pricing WHERE product_id = $1 AND region = $2",
      "parameters": {
        "pid": "{{DOC.document_id}}",
        "region": "{{INPUT.user_region}}"
      },
      "output_field": "pricing"
    }
  }
}
```

## Query Syntax

### Parameter Placeholders

Use `$1`, `$2`, and so on for placeholders. The stage sorts the keys of `parameters` in alphabetical order and binds their values to the placeholders in that order. `$1` takes the first key alphabetically.

```sql theme={null}
SELECT * FROM users WHERE status = $1 AND id = $2
```

With `parameters` set to `{"user_id": "...", "status": "active"}`, `$1` is `status` and `$2` is `user_id`. The database binds the values, so they are not concatenated into the query text.

### Template Variables

| Variable | Description |
| - | - |
| `{{DOC.*}}` | Any document field |
| `{{INPUT.*}}` | Input parameters from retriever call |
| `{{CONTEXT.*}}` | Pipeline context variables |

## Output Schema

### Single Row (default)

```json theme={null}
{
  "document_id": "doc_123",
  "content": "...",
  "sql_result": {
    "name": "John Doe",
    "email": "john@example.com",
    "tier": "premium"
  }
}
```

### Multiple Rows

```json theme={null}
{
  "document_id": "doc_123",
  "content": "...",
  "recent_orders": [
    {"order_id": "ord_1", "status": "delivered", "total": 99.99},
    {"order_id": "ord_2", "status": "shipped", "total": 149.99}
  ]
}
```

### No Results

```json theme={null}
{
  "document_id": "doc_123",
  "content": "...",
  "sql_result": null
}
```

## Security

> [!WARNING]
>
> SQL queries are parameterized to prevent injection attacks. Never concatenate user input directly into query strings.

| Security Feature | Description |
| - | - |
| **Parameterized queries** | All parameters are escaped |
| **Connection isolation** | Each connection uses dedicated credentials |
| **Read-only option** | Configure connections as read-only |
| **Query timeout** | Prevent long-running queries |

## Performance

| Metric | Value |
| - | - |
| **Latency** | 10-100ms (depends on query complexity) |
| **Connection pooling** | Automatic |
| **Execution** | One query per document, run one after another |
| **Timeout handling** | Graceful with null result |

> [!TIP]
>
> For high-volume lookups, ensure your database has appropriate indexes on the queried columns. Consider caching frequently accessed data.

## Common Pipeline Patterns

### Search + SQL Enrichment

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 20 }
        ],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "sql_lookup",
    "stage_type": "apply",
    "config": {
      "stage_id": "sql_lookup",
      "parameters": {
        "connection_id": "product_db",
        "query": "SELECT price, stock FROM products WHERE id = $1",
        "parameters": {
          "id": "{{DOC.metadata.product_id}}"
        },
        "output_field": "product_data"
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "product_data.stock",
        "operator": "gt",
        "value": 0
      }
    }
  }
]
```

## Join Warehouse Metrics onto Results

A creative library often carries an ID such as `creative_id` that keys a metrics table. The table holds spend, ROAS, and thumb-stop rate. Two stages join that table onto search results. Where the table lives picks the stage.

| Table location | Stage |
| - | - |
| PostgreSQL | `sql_lookup` with a PostgreSQL connection |
| Snowflake or another warehouse | `api_call` against an HTTP endpoint that runs the query |

Both stages run after the search stage and add fields to each returned document. They do not change which documents were retrieved or how they scored. A search cannot rank by a joined value. To rank by ROAS, ingest it as document data.

### From PostgreSQL

Read `creative_id` from the document metadata and bind it to `$1`.

```json theme={null}
{
  "stage_name": "sql_lookup",
  "stage_type": "apply",
  "config": {
    "stage_id": "sql_lookup",
    "parameters": {
      "connection_id": "ad_metrics_postgres",
      "query": "SELECT spend, roas, thumbstop_rate FROM creative_performance WHERE creative_id = $1",
      "parameters": {
        "creative_id": "{{DOC.metadata.creative_id}}"
      },
      "output_field": "warehouse_metrics"
    }
  }
}
```

Each result gains a `warehouse_metrics` object. A creative with no row gets `null`, because `on_no_results` defaults to `null`.

```json theme={null}
{
  "document_id": "doc_123",
  "score": 0.91,
  "warehouse_metrics": {
    "spend": 1840.5,
    "roas": 3.2,
    "thumbstop_rate": 0.31
  }
}
```

### From Another Warehouse

Put an HTTP endpoint in front of the warehouse. The endpoint takes the ID in its path and returns one JSON object. The stage reads the token from an organization secret. See [API Call](/docs/retrieval/stages/api-call#storing-secrets).

```json theme={null}
{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://metrics.example.com/v1/creatives/{{DOC.metadata.creative_id}}/performance",
      "method": "GET",
      "allowed_domains": ["metrics.example.com"],
      "auth": {
        "type": "bearer",
        "secret_ref": "metrics_api_token"
      },
      "output_field": "warehouse_metrics",
      "timeout": 10,
      "on_error": "skip"
    }
  }
}
```

The endpoint needs a public address and must return JSON. The stage sends one request per document, one after another. A retriever that returns 20 documents sends 20 requests.

## Error Handling

| Error | Behavior |
| - | - |
| Query timeout | Handled per `on_error` (default: document passes through unchanged) |
| Connection failure | Handled per `on_error` (default: `skip`) |
| No rows returned | `output_field` set to `null` (default `on_no_results`) |
| Invalid SQL | Stage fails with error message |

## Related

* [Document Enrich](/docs/retrieval/stages/document-enrich) - Collection-based joins
* [API Call](/docs/retrieval/stages/api-call) - REST API enrichment
* [JSON Transform](/docs/retrieval/stages/json-transform) - Transform enriched data


# Summarize
Source: https://docs.mixpeek.com/docs/retrieval/stages/summarize

Generate LLM-powered summaries from document sets

![Summarize stage showing LLM-powered document summarization](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/summarize.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=2c06e3bfcd4ca916055c4a613b69d911)

The Summarize stage uses language models to generate summaries from document sets. It can create single summaries from multiple documents, per-document summaries, or answer questions based on the retrieved content.

> [!NOTE]
>
> **Stage Category**: REDUCE (Aggregates documents)
>
> **Transformation**: N documents → 1 summary document (or N documents with summaries)

## When to Use

| Use Case | Description |
| - | - |
| **RAG summarization** | Generate answers from search results |
| **Document synthesis** | Combine multiple sources into one summary |
| **Key points extraction** | Distill long documents to essentials |
| **Question answering** | Answer user questions from retrieved docs |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Just formatting for LLM | `rag_prepare` (no LLM call) |
| Extracting structured data | `llm_enrich` |
| Real-time low-latency | Pre-compute summaries |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `prompt` | string | *Required* | Summarization instructions (must include `{{DOCUMENTS}}`) |
| `provider` | string | `google` | LLM provider: `openai`, `google`, `anthropic` |
| `model_name` | string | *provider default* | Specific LLM model to use |
| `content_field` | string | `content` | Field containing text to summarize |
| `group_by` | string | *none* | Field to group by (one summary per group); omit for a single summary |
| `max_input_tokens` | integer | `8000` | Max tokens to send to LLM |
| `include_sources` | boolean | `true` | Add source document IDs to output |
| `output_field` | string | `summary` | Field for summary output |

## Available Models

Set `provider` and `model_name` together. If `provider` is omitted, it is inferred from `model_name` (defaults to `google` / `gemini-2.5-flash-lite`).

| Provider | `model_name` examples | Speed | Quality |
| - | - | - | - |
| `google` | `gemini-2.5-flash-lite` | Fast | Good |
| `openai` | `gpt-4o-mini` | Fast | Good |
| `openai` | `gpt-4o` | Medium | Excellent |
| `anthropic` | `claude-haiku-4-5-20251001` | Fast | Good |

## Configuration Examples

```json Basic RAG Summary theme={null}
{
  "stage_name": "summarize",
  "stage_type": "reduce",
  "config": {
    "stage_id": "summarize",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Based on the provided documents, answer the user's question: {{INPUT.query}}\n\n{{DOCUMENTS}}",
      "include_sources": true
    }
  }
}
```

```json Detailed Summary theme={null}
{
  "stage_name": "summarize",
  "stage_type": "reduce",
  "config": {
    "stage_id": "summarize",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Synthesize the following documents into a comprehensive summary. Include key findings, important details, and any contradictions between sources.\n\n{{DOCUMENTS}}",
      "max_input_tokens": 16000,
      "include_sources": true
    }
  }
}
```

```json Per-Category Summaries theme={null}
{
  "stage_name": "summarize",
  "stage_type": "reduce",
  "config": {
    "stage_id": "summarize",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Summarize the documents for '{{GROUP_VALUE}}' in 2-3 sentences, focusing on the main points.\n\n{{DOCUMENTS}}",
      "group_by": "metadata.category",
      "output_field": "category_summary"
    }
  }
}
```

```json Executive Brief theme={null}
{
  "stage_name": "summarize",
  "stage_type": "reduce",
  "config": {
    "stage_id": "summarize",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-haiku-4-5-20251001",
      "prompt": "Create an executive summary of these documents. Include: key takeaways (3-5 bullet points), recommendations, and any risks or concerns mentioned.\n\n{{DOCUMENTS}}",
      "include_sources": true
    }
  }
}
```

```json Q&A with Sources theme={null}
{
  "stage_name": "summarize",
  "stage_type": "reduce",
  "config": {
    "stage_id": "summarize",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Answer the following question based only on the provided documents. If the answer cannot be found in the documents, say so. Question: {{INPUT.question}}\n\n{{DOCUMENTS}}",
      "include_sources": true,
      "max_input_tokens": 12000
    }
  }
}
```

## Grouping

### Single Summary (default)

With no `group_by`, all documents are combined into one summary (N→1):

```
[Doc1, Doc2, Doc3] → "Combined summary of all documents..."
```

### Per-Group Summaries

Set `group_by` to a field path to produce one summary per unique group value (N→M).
Use `{{GROUP_VALUE}}` in the prompt to reference the current group:

```
group_by: "metadata.category"
[Doc1(A), Doc2(A), Doc3(B)] → ["A" summary, "B" summary]
```

## Output Schema

The summary is written to `output_field` (default `summary`). When `include_sources` is
true, `source_document_ids` is added; when `include_metadata` is true, `document_count`
and `tokens_used` are added.

### Single Summary (no `group_by`)

```json theme={null}
{
  "summary": "Based on the documents, the answer is...",
  "source_document_ids": ["doc_123", "doc_456"],
  "document_count": 2,
  "tokens_used": 1250
}
```

### Per-Group (with `group_by`)

One summary document per unique group value:

```json theme={null}
[
  {
    "summary": "Summary for the electronics category...",
    "source_document_ids": ["doc_123", "doc_456"],
    "document_count": 2
  },
  {
    "summary": "Summary for the clothing category...",
    "source_document_ids": ["doc_789"],
    "document_count": 1
  }
]
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 500-2000ms |
| **Token usage** | Depends on input size |
| **Max input** | Model context window |
| **Streaming** | Supported |

> [!WARNING]
>
> Summarization calls the LLM and incurs API costs. Use `rag_prepare` if you only need to format content for external LLM calls.

## Common Pipeline Patterns

### Full RAG Pipeline

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "prompt": "Answer the user's question based on the provided documents: {{INPUT.query}}\n\n{{DOCUMENTS}}",
        "include_sources": true
      }
    }
  }
]
```

### Multi-Document Synthesis

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.topic}}" }, "top_k": 20 }
        ],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.type",
        "operator": "eq",
        "value": "research_paper"
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "anthropic",
        "model_name": "claude-haiku-4-5-20251001",
        "prompt": "Synthesize the research findings from these papers on {{INPUT.topic}}. Identify common themes, contradictions, and gaps in the research.\n\n{{DOCUMENTS}}",
        "max_input_tokens": 32000
      }
    }
  }
]
```

### Preview Summaries

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 10 }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Create a one-sentence summary for the '{{GROUP_VALUE}}' group:\n\n{{DOCUMENTS}}",
        "group_by": "metadata.source",
        "output_field": "preview"
      }
    }
  }
]
```

## Comparison: summarize vs rag\_prepare

| Feature | summarize | rag\_prepare |
| - | - | - |
| Calls LLM | Yes | No |
| Output | Generated summary | Formatted context |
| Latency | 500-2000ms | \< 10ms |
| Cost | LLM API costs | Free |
| Use case | End-to-end RAG | Prepare for external LLM |

## Error Handling

| Error | Behavior |
| - | - |
| Token limit exceeded | Truncates input, continues |
| LLM timeout | Retry once, then fail |
| Rate limit | Automatic backoff |
| Empty input | Returns empty summary |

## Related

* [RAG Prepare](/docs/retrieval/stages/rag-prepare) - Format for LLM without calling
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - Structured extraction
* [Rerank](/docs/retrieval/stages/rerank) - Improve input quality first


# Temporal
Source: https://docs.mixpeek.com/docs/retrieval/stages/temporal

Group documents by time windows and compute per-window aggregations with drift detection

![Temporal stage showing time-window grouping with drift detection](https://mintcdn.com/mixpeek/5nxzvCsxVltsWfsI/assets/retrievers/temporal.svg?fit=max\&auto=format\&n=5nxzvCsxVltsWfsI\&q=85\&s=365959d43f4fd79d1aeee07877f4211b)

The Temporal stage groups documents into time windows (hour, day, week, month, quarter, year) and computes aggregations per window. It can also detect drift between consecutive windows, flagging spikes or drops that exceed a threshold.

> [!NOTE]
>
> **Stage Category**: REDUCE (Groups results by time)
>
> **Transformation**: N documents → M time-window results (with optional drift detection)

## When to Use

| Use Case | Description |
| - | - |
| **Trend analysis** | Track how metrics change over time |
| **Spike detection** | Flag windows where a metric jumps or drops significantly |
| **Content velocity** | Count new documents per day/week/month |
| **Temporal distribution** | Understand when content was created or modified |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple counting | `aggregate` |
| Full time-series from database | Aggregation API |
| Grouping by non-time fields | `group_by` |
| LLM-based trend analysis | `summarize` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `time_field` | string | *Required* | Document field containing the timestamp |
| `window` | string | *Required* | Window granularity: `hour`, `day`, `week`, `month`, `quarter`, `year` |
| `aggregations` | array | *Required* | List of aggregation operations per window |
| `drift` | object | `null` | Drift detection configuration |
| `sort_order` | string | `"asc"` | Sort windows `asc` (oldest first) or `desc` (newest first) |
| `limit` | integer | `null` | Max number of windows to return |
| `include_documents` | boolean | `false` | Include original documents in output |

## Aggregation Types

| Type | Field Required | Description |
| - | - | - |
| `count` | No | Number of documents in window |
| `sum` | Yes | Sum of field values |
| `avg` | Yes | Average value |
| `min` | Yes | Minimum value |
| `max` | Yes | Maximum value |
| `count_distinct` | Yes | Unique values count |
| `collect_distinct` | Yes | Gather unique values into list |

## Drift Detection

Drift detection compares a metric between consecutive windows and computes the percent change.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `drift.enabled` | boolean | `false` | Enable drift detection |
| `drift.metric` | string | *Required if enabled* | Which aggregation alias to track |
| `drift.threshold` | float | `null` | Percent change to flag (e.g., `50.0` flags changes > 50%) |

## Timestamp Formats

The stage parses timestamps automatically:

| Format | Example |
| - | - |
| ISO 8601 | `"2026-04-01T10:30:00Z"` |
| ISO date | `"2026-04-01"` |
| Epoch seconds (int) | `1743465600` |
| Epoch seconds (float) | `1743465600.123` |

Documents with missing or unparseable timestamps are skipped (counted in `num_documents_skipped`).

## Configuration Examples

```json Daily Trend Analysis theme={null}
{
  "stage_name": "temporal",
  "stage_type": "reduce",
  "config": {
    "stage_id": "temporal",
    "parameters": {
      "time_field": "created_at",
      "window": "day",
      "aggregations": [
        {"function": "count", "alias": "posts_per_day"},
        {"function": "avg", "field": "score", "alias": "avg_relevance"}
      ]
    }
  }
}
```

```json Spike Detection with Drift theme={null}
{
  "stage_name": "temporal",
  "stage_type": "reduce",
  "config": {
    "stage_id": "temporal",
    "parameters": {
      "time_field": "published_at",
      "window": "day",
      "aggregations": [
        {"function": "count", "alias": "count"}
      ],
      "drift": {
        "enabled": true,
        "metric": "count",
        "threshold": 50.0
      }
    }
  }
}
```

```json Monthly Content Velocity theme={null}
{
  "stage_name": "temporal",
  "stage_type": "reduce",
  "config": {
    "stage_id": "temporal",
    "parameters": {
      "time_field": "created_at",
      "window": "month",
      "aggregations": [
        {"function": "count", "alias": "total"},
        {"function": "count_distinct", "field": "author", "alias": "unique_authors"},
        {"function": "collect_distinct", "field": "category", "alias": "categories"}
      ],
      "sort_order": "desc",
      "limit": 12
    }
  }
}
```

```json Quarterly Revenue Trends theme={null}
{
  "stage_name": "temporal",
  "stage_type": "reduce",
  "config": {
    "stage_id": "temporal",
    "parameters": {
      "time_field": "transaction_date",
      "window": "quarter",
      "aggregations": [
        {"function": "sum", "field": "amount", "alias": "revenue"},
        {"function": "count", "alias": "transactions"},
        {"function": "avg", "field": "amount", "alias": "avg_transaction"}
      ],
      "drift": {
        "enabled": true,
        "metric": "revenue",
        "threshold": 20.0
      }
    }
  }
}
```

```json Hourly Activity Monitoring theme={null}
{
  "stage_name": "temporal",
  "stage_type": "reduce",
  "config": {
    "stage_id": "temporal",
    "parameters": {
      "time_field": "timestamp",
      "window": "hour",
      "aggregations": [
        {"function": "count", "alias": "events"},
        {"function": "max", "field": "latency_ms", "alias": "peak_latency"}
      ],
      "drift": {
        "enabled": true,
        "metric": "events"
      },
      "sort_order": "desc",
      "limit": 24
    }
  }
}
```

## Output Schema

### Window Results

```json theme={null}
{
  "metadata": {
    "windows": [
      {
        "window": "2026-04-01",
        "metrics": {
          "count": 3,
          "avg_score": 0.85
        }
      },
      {
        "window": "2026-04-02",
        "metrics": {
          "count": 1,
          "avg_score": 0.72
        },
        "drift": {
          "absolute_change": -2,
          "percent_change": -66.67,
          "flagged": true
        }
      }
    ],
    "num_windows": 2,
    "num_documents_in": 4,
    "num_documents_skipped": 0,
    "window_granularity": "day"
  }
}
```

### Window Key Formats

| Window | Format | Example |
| - | - | - |
| `hour` | `YYYY-MM-DDTHH:00:00` | `2026-04-01T15:00:00` |
| `day` | `YYYY-MM-DD` | `2026-04-01` |
| `week` | `YYYY-WNN` | `2026-W14` |
| `month` | `YYYY-MM` | `2026-04` |
| `quarter` | `YYYY-QN` | `2026-Q2` |
| `year` | `YYYY` | `2026` |

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-50ms |
| **Memory** | O(windows x aggregations) |
| **Cost** | Free |
| **Scalability** | Efficient for large result sets |

## Common Pipeline Patterns

### Search + Temporal Analysis

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 500
          }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "temporal",
    "stage_type": "reduce",
    "config": {
      "stage_id": "temporal",
      "parameters": {
        "time_field": "created_at",
        "window": "day",
        "aggregations": [
          {"function": "count", "alias": "matches_per_day"},
          {"function": "avg", "field": "score", "alias": "avg_relevance"}
        ],
        "drift": {
          "enabled": true,
          "metric": "matches_per_day",
          "threshold": 100.0
        }
      }
    }
  }
]
```

### Brand Monitoring with Spike Detection

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.brand_name}}",
            "top_k": 1000
          }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "google",
        "model_name": "gemini-2.5-flash-lite",
        "prompt": "Classify sentiment as positive, neutral, or negative: {{DOC.content}}",
        "output_field": "sentiment"
      }
    }
  },
  {
    "stage_name": "temporal",
    "stage_type": "reduce",
    "config": {
      "stage_id": "temporal",
      "parameters": {
        "time_field": "published_at",
        "window": "week",
        "aggregations": [
          {"function": "count", "alias": "mentions"},
          {"function": "count_distinct", "field": "sentiment", "alias": "sentiment_spread"}
        ],
        "drift": {
          "enabled": true,
          "metric": "mentions",
          "threshold": 50.0
        }
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Missing timestamp field | Document skipped |
| Unparseable timestamp | Document skipped |
| Non-numeric field for sum/avg | Document skipped for that aggregation |
| Empty results | 0 windows returned |
| Unknown aggregation | Returns null |

## Related

* [Aggregate](/docs/retrieval/stages/aggregate) - Statistical aggregations without time grouping
* [Group By](/docs/retrieval/stages/group-by) - Group documents by any field
* [Sample](/docs/retrieval/stages/sample) - Statistical sampling


# Buckets
Source: https://docs.mixpeek.com/docs/studio/buckets



![Studio Buckets page listing buckets with schema fields, object counts, and storage size](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/buckets.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=1102b2f48b0d53a80f0927db585be9bb)

[Buckets](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/buckets.mp4)

Buckets are the warehouse's ingestion layer, where raw files enter before being decomposed into features.

### Manage buckets and objects

* **Create bucket**: Define a JSON-like schema to validate objects. Required for object registration. API parity: [Create Bucket](/docs/api-reference/buckets/create-bucket).
* **View bucket**: Inspect schema, object counts, and metadata. API parity: [Get Bucket](/docs/api-reference/buckets/get-bucket).
* **Update bucket**: Rename or adjust description and metadata. API parity: [Update Bucket](/docs/api-reference/buckets/update-bucket).
* **Delete bucket**: Enqueues a task to remove bucket and objects. Track under Tasks. API parity: [Delete Bucket](/docs/api-reference/buckets/delete-bucket).
* **Add objects**: Register raw inputs and blobs without processing. Use single or batch create. API: [Create Object](/docs/api-reference/bucket-objects/create-object), [Create Objects in Batch](/docs/api-reference/bucket-objects/create-objects-in-batch).
* **Batch flows**: Group object IDs, then submit for processing. Track task status. API: [Create Batch](/docs/api-reference/bucket-batches/create-batch), [Add Objects](/docs/api-reference/bucket-batches/add-objects-to-batch), [Submit Batch](/docs/api-reference/bucket-batches/submit-batch-for-processing).

### Tips

* Object creation does not process files; processing happens when Engine runs extraction based on collections pointing to this bucket.
* Use batch submit for large ingestions to parallelize Engine work and monitor via Tasks in Studio.

**Bucket lifecycle**

**1. Create**

Define schema and metadata.

**2. Update**

Adjust name, description, metadata.

**3. Delete**

Enqueue task; track in Tasks.

**Objects & batches**

**1. Register objects**

Single or batch registration without processing.

**2. Create batch**

Group object IDs for processing.

**3. Submit batch**

Trigger Engine processing and monitor progress.

Processing runs in the Engine and writes to your active namespace; ensure it’s set correctly in the top bar.


# Clusters
Source: https://docs.mixpeek.com/docs/studio/clusters



![Studio Clusters page listing clusters with algorithm, results, and completion status](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/clusters.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=ef817918dae9ba17f80703bba77b6cac)

[Clusters](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/clusters.mp4)

Clusters provide warehouse-native grouping, the multimodal equivalent of SQL GROUP BY.

### Create and run clustering jobs

* **Create**: Click New Cluster, select collections, pick vector or attribute clustering, and configure algorithm params. API: [Create Cluster](/docs/api-reference/clusters/create-cluster).
* **Execute**: Run the clustering pipeline on the Engine. `mode` picks the run: `full` reclusters everything, `assign` adds new documents to existing centroids, `composite` clusters the centroids of earlier runs. API: [Execute Clustering](/docs/api-reference/clusters/execute-clustering).
* **Inspect**: Review centroids, metrics, and members if saved. Under Artifacts, download a run's rows as CSV or Parquet. API: [Export Cluster Run Data](/docs/api-reference/cluster-executions/export-cluster-run-data), which takes a `run_id` and returns a presigned download URL.
* **List/Get/Delete**: Manage clustering configurations and results. API: [List](/docs/api-reference/clusters/list-clusters), [Get](/docs/api-reference/clusters/get-cluster), [Delete](/docs/api-reference/clusters/delete-cluster).
* **Browse results**: Read every centroid plus a sample of members, the data behind the scatter plot. API: [Cluster Visualization Data](/docs/api-reference/clusters/cluster-visualization-data). For group labels, summaries, and document counts, use [List Cluster Groups](/docs/api-reference/clusters/list-cluster-groups).
* **Enrich the source collection**: Create the cluster with `enrich_source_collection` set to true, and each execution writes cluster labels back onto the source documents. API: [Create Cluster](/docs/api-reference/clusters/create-cluster).

**Choosing an algorithm at scale.** For collections over \~100K documents, pick a **linear** algorithm — **K-Means**, **Gaussian Mixture**, or **Leiden**. The density/graph algorithms that build a pairwise distance matrix — HDBSCAN, DBSCAN, Agglomerative, Spectral, OPTICS, Mean Shift — do not scale past 100K and will error on larger datasets (an N×N distance matrix at 1M rows would need \~7,000 GB of RAM). To run one of those on a large collection, set a `sample_size` to cluster a representative subset instead. The create-cluster wizard surfaces this guidance inline when you choose the algorithm.

### Visualization

The cluster scatter plot maps reduced coordinates to position and size:

* **x, y** → point position on the chart
* **z** (when `dimension_reduction.components` is `3`) → **dot size**, where larger dots represent higher z-values

This depth-cue approach surfaces the third dimension without requiring a full 3D renderer, making it easy to spot structure that would be lost in a flat 2D projection.

### Tips

* Start with a sample size to validate parameters before full runs.
* Use LLM labeling for human-friendly labels when vectors are dense and unlabeled.
* Set `dimension_reduction.components` to `3` to see depth-based sizing in the scatter plot.

**1. Create a cluster job**

Choose collections and configure algorithm parameters; optionally set dimensionality reduction.

**2. Execute or submit**

Run in real-time or submit as an asynchronous job and track via Tasks.

**3. Inspect and enrich**

Review centroids and metrics. With `enrich_source_collection` on, the execution has already written labels onto the source documents.

Artifacts such as parquet paths allow downstream analytics and reproducible exploration.


# Collections
Source: https://docs.mixpeek.com/docs/studio/collections



![Studio Collections page listing collections with document counts and enabled status](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/collections.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=42f19152e578aa4413754213b62bc194)

[Collections](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/collections.mp4)

Collections define how the warehouse decomposes raw files into queryable features.

### Build collections

* **Create**: Click New Collection, name it, provide a source (bucket or collection), and pick the [features](/docs/processing/features) you want (per your content's modality). API: [Create Collection](/docs/api-reference/collections/create-collection).
* **Features**: Studio shows available feature addresses and vector indexes. Use this to configure retrievers. API: [Describe Features](/docs/api-reference/collections/describe-collection-features).
* **Update**: Modify description, enablement, and taxonomy applications. API: [Update Collection](/docs/api-reference/collections/update-collection).
* **List/Get/Delete**: Browse, inspect, or remove collections. API: [List](/docs/api-reference/collections/list-collections), [Get](/docs/api-reference/collections/get-collection), [Delete](/docs/api-reference/collections/delete-collection).

### Documents

* View documents produced by extraction with metadata and vectors.
* Create or update documents directly for advanced workflows. API: [Create](/docs/api-reference/collection-documents/create-a-document), [Update](/docs/api-reference/collection-documents/update-document), [Get](/docs/api-reference/collection-documents/get-a-document-by-id), [List](/docs/api-reference/collection-documents/list-documents), [Delete](/docs/api-reference/collection-documents/delete-a-document-by-id).

### Tips

* Collections are the searchable view of your data; choose features that match your modalities.
* Ensure `internal_id` and `X-Namespace` context is correct—Studio handles this automatically.

**1. Create a collection**

Select a source, pick features, and save. Studio bootstraps vector indexes as needed.

**2. Inspect features**

Use Describe Features to confirm vector names, dimensions, and index metadata for retriever setup.

**3. Attach taxonomies (optional)**

Configure on-demand or materialized applications from the collection page.

**4. Attach retriever enrichments (optional)**

Configure retriever enrichments to run retriever pipelines on documents during post-processing. Each enrichment maps document fields to retriever inputs, executes the retriever, and writes selected result fields back to the document. Use for LLM classification, cross-collection joins, or multi-stage enrichment at ingestion time.

Documents are stored as MVS records with payload and vectors; the record ID is the document\_id in results.


# Explorer
Source: https://docs.mixpeek.com/docs/studio/explorer



[Explorer](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/explorer.mp4)

### Explore and search your documents

* **Search**: Enter queries that map to your retriever inputs or perform fielded searches using filters. For retriever-backed search, use the `Research` area; Explorer provides ad-hoc discovery across collections.
* **Filters**: Apply AND/OR/NOT filters, ranges, and keyword matches on metadata. API filter model mirrors \[LogicalOperator] fields used across list and execute endpoints.
* **Sort and paginate**: Order by score or any document field and control page size. Mirrors retriever execution sort behaviour.
* **Select fields**: Toggle columns to tailor the view. Useful to inspect vector presence, enrichments, and metadata.
* **Preview assets**: Enable presigned URLs to open media directly from S3-compatible storage.

### Tips

* Switch namespaces to change the collection set you are exploring.
* For production search UX, prefer saved retrievers; Explorer is ideal for QA and data auditing.

Explorer is optimized for ad-hoc discovery. For structured pipelines, build a retriever and test in Research.

`eq`, `ne`, `gt`, `lt`, `gte`, `lte`, `in`, `nin`, `contains`, `starts_with`, `ends_with`, `regex`, `exists`, `is_null`, `text`.


# Namespaces
Source: https://docs.mixpeek.com/docs/studio/namespaces



![Studio Namespaces page listing namespaces with document and object counts](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/namespaces.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=6a2ee282a02e6ea5ed691211ebdad5b3)

[Namespaces](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/namespace.mp4)

Namespaces are isolated warehouse environments, each with its own storage, collections, and retrieval pipelines.

### Manage namespaces in Studio

* **Create**: Click New Namespace, provide a name and optional description. You can attach feature extractors and payload indexes up-front. API parity: [Create Namespace](/docs/api-reference/namespaces/create-namespace).
* **Switch active namespace**: Use the top bar to scope every page. This sets the `X-Namespace` in API calls made by Studio.
* **Update**: Edit payload indexes or rename. API parity: [Update Namespace](/docs/api-reference/namespaces/update-namespace).
* **List and search**: Quickly find namespaces you have access to. API parity: [List Namespaces](/docs/api-reference/namespaces/list-namespaces).
* **Get details**: Inspect configured feature extractors and indexes. API parity: [Get Namespace](/docs/api-reference/namespaces/get-namespace).
* **Delete**: Remove a namespace that you no longer need. This deletes the associated MVS namespace. API parity: [Delete Namespace](/docs/api-reference/namespaces/delete-namespace).

### Tips

* Namespaces map 1 to 1 with [MVS](https://mixpeek.com/mvs) namespaces. Choose clear names per environment like `acme_dev`, `acme_prod`.
* Keep one or more feature extractors attached; Engine ensures the collection is bootstrapped on first ingestion.

**1. Create a namespace**

Click New Namespace, add details, and optionally attach feature extractors and payload indexes.

**2. Switch active namespace**

Use the top bar switcher; this sets the `X-Namespace` applied by Studio.

**3. Manage and clean up**

Update indexes as needs evolve; delete unused namespaces to keep your workspace tidy.

Deleting a namespace removes its backing vector store namespace for the org scope.

On an Enterprise plan the platform takes a snapshot first, including a copy of your
bucket objects, and keeps it for 30 days. You can restore from it into a new
namespace, or over the original. On every other plan the delete is not covered by
anything, so take your own copy before you delete.

A snapshot is a copy of the namespace's data, taken while writes continue. It
restores the documents, vectors and objects it captured. It does not reproduce a
single instant, so treat it as a backup rather than as a rewind. See
[Create Namespace Snapshot](/docs/api-reference/namespace-snapshots/create-namespace-snapshot)
and [Restore From Snapshot](/docs/api-reference/namespace-snapshots/restore-from-snapshot).


# Quickstart
Source: https://docs.mixpeek.com/docs/studio/quickstart



![Studio home showing the what-to-make-searchable picker, file drop zone, and sample-data card](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/quickstart.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=10bbff0e993c0028c02046478c18823f)

[Quickstart overview](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/quickstart.mp4)

Studio is the visual interface for the multimodal data warehouse, where you manage namespaces, collections, and retrievers.

### Use the Studio to manage your organization

* **Set active namespace**: Use the top bar namespace switcher. All actions scope to this selection. Namespaces mirror API behavior for `X-Namespace`. Create one in Studio or via [Create Namespace](/docs/api-reference/namespaces/create-namespace).
* **Create buckets**: Define schemas to validate your objects. Buckets are the entry point for raw data. See [Create Bucket](/docs/api-reference/buckets/create-bucket) and [Create Object](/docs/api-reference/bucket-objects/create-object).
* **Create collections**: Point to a source and pick [features](/docs/processing/features). Collections produce searchable documents and vectors. See [Create Collection](/docs/api-reference/collections/create-collection) and [Describe Features](/docs/api-reference/collections/describe-collection-features).
* **Ingest data**: Upload objects to buckets, optionally in batches. Processing runs in the Engine and writes documents to your namespace.
* **Explore results**: Use `Explorer` to search, filter, and preview documents with presigned URLs.
* **Build retrievers**: Configure stages, test inputs, and save pipelines to power your apps. See [Create Retriever](/docs/api-reference/retrievers/create-retriever) and [Execute Retriever](/docs/api-reference/retrievers/execute-retriever).
* **Enrich with taxonomies**: Attach flat or hierarchical taxonomies on-demand or materialized. See [Create Taxonomy](/docs/api-reference/taxonomies/create-taxonomy) and [Test Taxonomy](/docs/api-reference/taxonomies/test-taxonomy-configuration-validation-only).
* **Discover with clusters**: Run clustering jobs and apply enrichment back to collections. See [Create Cluster](/docs/api-reference/clusters/create-cluster) and [Execute Clustering](/docs/api-reference/clusters/execute-clustering).

### Tips

* **Auth is automatic in Studio**: Your API key and `X-Namespace` header are handled for you. When calling APIs directly, include both.
* **Rate limits and health**: Check `Operations` for service health and limits. API reference: [Healthcheck](/docs/api-reference/health/healthcheck) and [Limits](/docs/troubleshoot/limits).

**1. Set active namespace**

Use the top bar switcher. This scopes all actions and mirrors the `X-Namespace` header.

**2. Create buckets**

Define schemas, then register objects. Buckets are your raw input boundary.

**3. Create collections**

Pick features and index configs; this produces searchable documents and vectors.

**4. Ingest and process**

Upload objects or submit batches; the Engine writes vectors and payloads to [MVS](https://mixpeek.com/mvs).

**5. Explore and retrieve**

Use Explorer for ad-hoc discovery and Retrievers for production search.

**6. Enrich with taxonomies and clusters**

Attach taxonomies on-demand or materialize; run clustering for discovery and analytics.

Studio follows Mintlify component patterns for clarity and scannability; see the Mintlify starter for examples.


# Research
Source: https://docs.mixpeek.com/docs/studio/research



[Research](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/research.mp4)

### Research and debug

* **Ad-hoc retrieval**: Quickly prototype queries against selected retrievers. Tweak inputs, filters, sorts, and grouping to validate pipeline behavior without code.
* **Inspect stage outputs**: Compare per-stage results, scores, and timings to tune parameters.
* **Debug inference**: Call inference backends directly to inspect embeddings and model outputs. API: [Raw Inference](/docs/api-reference/inference/execute-raw-inference).
* **Iterate**: Promote successful experiments to saved retrievers and wire them into your apps.

### Tips

* Use small limits and targeted filters while tuning to keep responses fast.
* Validate feature availability in your collections via `Describe Features` before testing embedding stages.

**1. Pick a retriever**

Select the pipeline to test and provide sample inputs.

**2. Run and inspect**

Compare per-stage results, scores, and timings to tune configs.

**3. Debug inference (optional)**

Call inference backends to inspect embeddings and raw outputs.

Promote stable experiments to saved retrievers to get governance and reuse across environments.


# Retrievers
Source: https://docs.mixpeek.com/docs/studio/retrievers



![Studio Retrievers page listing retrievers with their stage badges and collections](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/retrievers.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=c18b118ee9feeee245388d121263ff7b)

[Retrievers](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/retrievers.mp4)

Retrievers are the warehouse's Reassemble layer, where you compose multi-stage pipelines to query across all your features.

### Build and execute retrievers

* **Create**: Click New Retriever, name it, select collections, and add stages from the catalog. API: [Create Retriever](/docs/api-reference/retrievers/create-retriever) and [List Stages](/docs/api-reference/retriever-stages/list-available-retriever-stages).
* **Stages**: Configure parameters per stage, including pre/post filters. Common stages include KNN search, hybrid fusion, reranking, and grouping.
* **Test**: Use the Run panel to execute with sample inputs, filters, sorts, and pagination. API parity: [Execute Retriever](/docs/api-reference/retrievers/execute-retriever).
* **Inspect**: View per-stage results, timing, and final results. Toggle URL presigning when you need asset links.
* **Manage**: List, view, update, or delete retrievers as they evolve. API: [Get](/docs/api-reference/retrievers/get-retriever), [List](/docs/api-reference/retrievers/list-retrievers), [Delete](/docs/api-reference/retrievers/delete-retriever), [Raw Inference](/docs/api-reference/inference/execute-raw-inference).

### Tips

* Ensure your collections expose the vectors you reference; confirm via `Describe Features` on the collection.
* Use `group_by` to collapse results into logical groups, then control member sorting via stage or final sort.

**1. Create a retriever**

Name it, select collections, and add stages from the catalog.

**2. Configure stages**

Set parameters and optional pre/post filters; chain multiple stages for hybrid or rerank flows.

**3. Test and iterate**

Run with sample inputs and inspect per-stage timing and results.

Use Debug Inference to inspect raw model outputs and embeddings when tuning parameters.


# Taxonomies
Source: https://docs.mixpeek.com/docs/studio/taxonomies



![Studio Taxonomies page listing a hierarchical taxonomy with its version and status](https://mintcdn.com/mixpeek/Wd_dr3_jo8gqgFxV/assets/studio/taxonomies.png?fit=max\&auto=format\&n=Wd_dr3_jo8gqgFxV\&q=85\&s=f799891c61c30d01e39487fc991f3438)

[Taxonomies](https://d8ts56gtwtyh1.cloudfront.net/docs/warehouse/taxonomies.mp4)

Taxonomies provide warehouse-native enrichment, the multimodal equivalent of a SQL JOIN.

### Create and apply taxonomies

* **Create**: Click New Taxonomy, choose flat or hierarchical, and define config. API parity: [Create Taxonomy](/docs/api-reference/taxonomies/create-taxonomy).
* **Flat**: Join a source collection to a taxonomy collection via a retriever. Configure input mappings and enrichment fields.
* **Hierarchical**: Define nodes explicitly or infer via schema, clusters, or LLM. Per-node retrievers and input mappings are supported.
* **Execute on-demand**: Validate configs against sample documents. API: [Test Taxonomy](/docs/api-reference/taxonomies/test-taxonomy-configuration-validation-only). Note this is for testing only; on-demand enrichment typically happens inside retrievers.
* **Materialize**: Attach to collections with `execution_mode` set to materialize so Engine enriches after ingestion. Manage under Collections → Taxonomy Applications. API: [Update Collection](/docs/api-reference/collections/update-collection).
* **Versions**: Snapshot configurations and browse history. API: [Create Version](/docs/api-reference/taxonomies/create-taxonomy-version), [List Versions](/docs/api-reference/taxonomies/list-taxonomy-versions).
* **Manage**: Get, list, or delete taxonomies. API: [Get](/docs/api-reference/taxonomies/get-taxonomy), [List](/docs/api-reference/taxonomies/list-taxonomies), [Delete](/docs/api-reference/taxonomies/delete-taxonomy).

### Tips

* Use on-demand for interactive retrieval flows and materialize for post-ingestion enrichment.
* Keep enrichment fields minimal to avoid payload bloat; prefer app-side joins when possible.

**Flat**

**1. Create**

Select retriever, input mappings, and source collection.

**2. Validate**

Use Execute for on-demand checks with sample docs.

**3. Apply**

Attach to a collection as on-demand or materialize.

**Hierarchical**

**1. Define or infer**

Add nodes or choose schema/cluster/LLM inference.

**2. Per-node options**

Override retriever and input mappings as needed.

**3. Version**

Snapshot configs and track changes over time.

Materialized enrichments are executed by the Engine post-ingestion—no separate API call is needed.


# LangChain
Source: https://docs.mixpeek.com/docs/agent-integrations/langchain

Give your AI agents eyes, ears, and memory with Mixpeek's LangChain integration

The `langchain-mixpeek` package gives LangChain agents the ability to see video, hear audio, search images, and act on unstructured content — all through Mixpeek's multimodal infrastructure.

> **[Connect LangChain to your own content](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_langchain\&utm_campaign=managed_onramp)**
>
> Create a managed namespace to get the `api_key`, `namespace`, and `retriever_id` that every example on this page needs.

## Installation

```bash theme={null}
pip install langchain-mixpeek
```

## Quick Start

### 1. Search (Retriever)

```python theme={null}
from langchain_mixpeek import MixpeekRetriever

retriever = MixpeekRetriever(
    api_key="mxp_...",
    retriever_id="ret_abc123",
    namespace="my-namespace",
)
docs = retriever.invoke("find the red cup")
```

Each result is a LangChain `Document` with `page_content` and metadata (`document_id`, `score`, `namespace`).

### 2. Agent Tool

```python theme={null}
from langchain_mixpeek import MixpeekRetriever

retriever = MixpeekRetriever(
    api_key="mxp_...",
    retriever_id="ret_abc123",
    namespace="my-namespace",
)

# One line — retriever becomes an agent tool
tool = retriever.as_tool()
```

### 3. Full Toolkit (search + ingest + classify + cluster + alert)

```python theme={null}
from langchain_mixpeek import MixpeekToolkit
from langgraph.prebuilt import create_react_agent
from langchain_anthropic import ChatAnthropic

toolkit = MixpeekToolkit(
    api_key="mxp_...",
    namespace="my-namespace",
    bucket_id="bkt_abc123",
    collection_id="col_def456",
    retriever_id="ret_ghi789",
)

agent = create_react_agent(
    ChatAnthropic(model="claude-sonnet-4-20250514"),
    toolkit.get_tools(),
)

result = agent.invoke({
    "messages": [("user", "Scan these product URLs and alert me about counterfeits")]
})
```

The toolkit gives your agent 6 capabilities:

| Tool | What it does |
| - | - |
| `mixpeek_search` | Search video, images, audio, documents by natural language |
| `mixpeek_ingest` | Upload text, images, video, audio, PDFs, spreadsheets |
| `mixpeek_process` | Trigger feature extraction (embedding, OCR, transcription, face detection) |
| `mixpeek_classify` | Run taxonomy classification on documents |
| `mixpeek_cluster` | Group similar documents (kmeans, dbscan, hdbscan, etc.) |
| `mixpeek_alert` | Set up monitoring with webhook, Slack, or email notifications |

### 4. VectorStore (full pipeline)

```python theme={null}
from langchain_mixpeek import MixpeekVectorStore

store = MixpeekVectorStore(
    api_key="mxp_...",
    namespace="my-namespace",
    bucket_id="bkt_abc123",
    collection_id="col_def456",
    retriever_id="ret_ghi789",
)

# Ingest any content type
store.add_texts(["product description..."])
store.add_images(["https://example.com/photo.jpg"])
store.add_videos(["https://example.com/clip.mp4"])
store.add_audio(["https://example.com/recording.mp3"])
store.add_pdfs(["https://example.com/doc.pdf"])
store.add_excel(["https://example.com/data.xlsx"])

# Trigger processing (embedding, OCR, face detection, etc.)
store.trigger_processing()

# Search
docs = store.similarity_search("red cup on the table")

# Convert to agent tools anytime
tool = store.as_tool()
toolkit = store.as_toolkit()
retriever = store.as_retriever()
```

### 5. Search-Only (minimal config)

If you only need search, skip the bucket/collection config:

```python theme={null}
store = MixpeekVectorStore.from_retriever(
    api_key="mxp_...",
    namespace="my-namespace",
    retriever_id="ret_abc123",
)
docs = store.similarity_search("red cup")
```

## Configuration

| Parameter | Type | Default | Description |
| - | - | - | - |
| `api_key` | str | required | Mixpeek API key (`mxp_...`) |
| `retriever_id` | str | required | Retriever ID for search (`ret_...`) |
| `namespace` | str | required | Namespace to operate in |
| `bucket_id` | str | required\* | Bucket for uploads (`bkt_...`) |
| `collection_id` | str | required\* | Collection for processing (`col_...`) |
| `top_k` | int | `10` / `5` | Max results (retriever / tool) |
| `content_field` | str | `"text"` | Field to use as `page_content` |
| `filters` | dict | `None` | Attribute filters (retriever only) |

\*Required for ingest/processing. Not needed for search-only via `from_retriever()`.

> [!TIP]
>
> The `content_field` can reference any field in your retriever results — including enrichment fields like `trend_insight` or `brand_alignment`. If the field contains a dict with a `text` key, the text is automatically extracted.

## Examples

### Brand Protection Agent

An agent that scans marketplace listings and alerts on counterfeits:

```python theme={null}
from langchain_mixpeek import MixpeekToolkit
from langgraph.prebuilt import create_react_agent
from langchain_anthropic import ChatAnthropic

toolkit = MixpeekToolkit(
    api_key="mxp_...",
    namespace="brand-protection",
    bucket_id="bkt_...",
    collection_id="col_...",
    retriever_id="ret_...",
)

# Only give the agent the tools it needs
agent = create_react_agent(
    ChatAnthropic(model="claude-sonnet-4-20250514"),
    toolkit.get_tools(actions=["search", "ingest", "process", "alert"]),
    prompt="You are a brand protection agent. Scan products and flag counterfeits.",
)

result = agent.invoke({
    "messages": [("user", "Check if these 5 Amazon listings are selling counterfeit Stanley cups")]
})
```

### RAG Chain

Standard retrieval-augmented generation:

```python theme={null}
from langchain_core.prompts import ChatPromptTemplate
from langchain_anthropic import ChatAnthropic
from langchain_mixpeek import MixpeekRetriever

retriever = MixpeekRetriever(
    api_key="mxp_...",
    retriever_id="ret_...",
    namespace="my-namespace",
)
llm = ChatAnthropic(model="claude-sonnet-4-20250514")

prompt = ChatPromptTemplate.from_template(
    "Answer using this context:\n{context}\n\nQuestion: {question}"
)

chain = {"context": retriever, "question": lambda x: x} | prompt | llm
response = chain.invoke("what happens at 2 minutes?")
```

### Multi-Retriever Agent

Different retrievers for different content types:

```python theme={null}
from langchain_mixpeek import MixpeekTool
from langgraph.prebuilt import create_react_agent

video_search = MixpeekTool(
    api_key="mxp_...",
    retriever_id="ret_video_archive",
    namespace="archive",
    name="search_video_archive",
    description="Search video archive for specific scenes, faces, or moments.",
)

image_search = MixpeekTool(
    api_key="mxp_...",
    retriever_id="ret_product_images",
    namespace="catalog",
    name="search_product_images",
    description="Search product image catalog by visual similarity.",
)

agent = create_react_agent(llm, [video_search, image_search])
```

## Platform Features

The VectorStore exposes the full Mixpeek platform:

### Taxonomies (document classification)

```python theme={null}
# Create a taxonomy
store.create_taxonomy(
    name="product-categories",
    config={
        "taxonomy_type": "flat",
        "retriever_id": "ret_...",
        "collection_id": "col_...",
        "input_mappings": [...],
        "enrichment_fields": [...],
    },
)

# List and execute
taxonomies = store.list_taxonomies()
results = store.execute_taxonomy("tax_abc123")
```

### Clusters (unsupervised grouping)

```python theme={null}
# Create and run clustering
cluster = store.create_cluster(
    cluster_type="vector",
    vector_config={
        "algorithm": "kmeans",  # or dbscan, hdbscan, spectral, etc.
        "algorithm_params": {"n_clusters": 10},
    },
)
store.execute_cluster(cluster["cluster_id"])
groups = store.get_cluster_groups(cluster["cluster_id"])
```

### Alerts (match notifications)

```python theme={null}
# Create an alert with webhook + Slack
store.create_alert(
    name="counterfeit-detection",
    notification_config={
        "channels": [
            {"channel_type": "webhook", "config": {"url": "https://..."}},
            {"channel_type": "slack", "channel_id": "#alerts"},
        ],
        "include_matches": True,
        "include_scores": True,
    },
)

# Check results
results = store.get_alert_results("alt_abc123")
```

### Custom Plugins

```python theme={null}
# List deployed plugins
plugins = store.list_plugins()

# Check deployment status
status = store.get_plugin_status("plg_abc123")

# Test a realtime plugin
result = store.test_plugin("plg_abc123", inputs={"text": "hello"})
```

## Tips

### Selecting Toolkit Actions

Don't give agents tools they don't need. Use `actions` to scope:

```python theme={null}
# Search-only agent
toolkit.get_tools(actions=["search"])

# Ingest + search agent
toolkit.get_tools(actions=["search", "ingest", "process"])

# Full platform agent
toolkit.get_tools()  # all 6 tools
```

### Error Handling

All toolkit tools catch exceptions and return error strings instead of crashing the agent. The retriever raises exceptions normally.

### Token Efficiency

Set `top_k` to limit results. Large result sets waste tokens without improving quality. Start with `top_k=5`.

## Source Code

* **PyPI**: [langchain-mixpeek](https://pypi.org/project/langchain-mixpeek/) (Python)
* **npm**: [@mixpeek/langchain](https://www.npmjs.com/package/@mixpeek/langchain) (JavaScript)
* **GitHub**: [mixpeek/langchain-mixpeek](https://github.com/mixpeek/langchain-mixpeek)
* **Connector Page**: [mixpeek.com/connectors/langchain](https://mixpeek.com/connectors/langchain)

## Next Steps

> **[MCP Server](/docs/agent-integrations/mcp)**
>
> Connect Claude directly via the Model Context Protocol

> **[OpenAI Function Calling](/docs/agent-integrations/openai-function-calling)**
>
> Wire Mixpeek into OpenAI assistants

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> 15+ extractors: text, image, video, audio, face, PDF, web scraper

> **[Python SDK](/docs/integrations/developer-tools/python-sdk)**
>
> Full SDK reference


# MCP Server
Source: https://docs.mixpeek.com/docs/agent-integrations/mcp

Give AI agents access to video, image, and audio search through the Model Context Protocol

Connect Claude (or any MCP-compatible client) to Mixpeek so it can search video, image, and audio content.

![Mixpeek MCP server architecture: an MCP client connects over HTTPS with a Bearer API key to the scope-gated Mixpeek MCP server (87 tools), which exposes retrievers as tools — agent\_search for autonomous multimodal retrieval, retriever\_execute for multi-stage filter/join/rerank pipelines, features\_get for faces, scenes, OCR, transcripts and embeddings, and agentic\_enrich (ingestion scope, off by default) — operating on a Mixpeek namespace of video, images, audio and documents stored as features plus dense, sparse and BM25 vectors on object storage.](https://mintcdn.com/mixpeek/ugDIgs1A5t6TC3Za/assets/mixpeek-mcp-multimodal.png?fit=max\&auto=format\&n=ugDIgs1A5t6TC3Za\&q=85\&s=a4c3018620ac7e1b91e56fdf5598af99)

## Hosted Server URLs

Mixpeek runs five hosted MCP servers. Each exposes a different subset of tools, so you load only what your agent needs.

| Scope | URL | Auth | Tools |
| - | - | - | - |
| **Full** | `https://mcp.mixpeek.com/mcp` | API key | 66: everything |
| **Ingestion** | `https://mcp.mixpeek.com/ingestion/mcp` | API key | 27: buckets, collections, documents, storage connections |
| **Retrieval** | `https://mcp.mixpeek.com/retrieval/mcp` | API key | 18: retrievers, agents, marketplace, search |
| **Admin** | `https://mcp.mixpeek.com/admin/mcp` | API key | 23: namespaces, taxonomies, clusters, storage connections |
| **App** | `https://mcp.mixpeek.com/app/mcp` | OAuth (`mixpeek:read`) | 9: read-only browse and search |

The App server is read-only. You sign in with your Mixpeek account in the browser, so you never paste a key into the client. See [Sign in with OAuth](#sign-in-with-oauth).

> [!TIP]
>
> For a focused search agent, use the [Per-Retriever Server](#per-retriever-mcp-server) instead. It exposes a single typed `search` tool generated from your retriever's input schema.

***

## Setup

Replace `YOUR_API_KEY` with your key from the [Mixpeek Studio](https://studio.mixpeek.com). To connect without a key, use the [App server over OAuth](#sign-in-with-oauth).

**Claude Desktop**

Add this to your Claude Desktop config file (`claude_desktop_config.json`):

```json Full theme={null}
{
  "mcpServers": {
    "mixpeek": {
      "url": "https://mcp.mixpeek.com/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

```json Ingestion theme={null}
{
  "mcpServers": {
    "mixpeek-ingestion": {
      "url": "https://mcp.mixpeek.com/ingestion/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

```json Retrieval theme={null}
{
  "mcpServers": {
    "mixpeek-retrieval": {
      "url": "https://mcp.mixpeek.com/retrieval/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

```json Admin theme={null}
{
  "mcpServers": {
    "mixpeek-admin": {
      "url": "https://mcp.mixpeek.com/admin/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

**Claude Code**

Run this in your terminal. To set up Claude Code, Codex, Gemini CLI and Cursor in one step instead, run `curl -fsSL https://mixpeek.com/install | sh`.

```bash Full theme={null}
claude mcp add --scope user --transport http mixpeek https://mcp.mixpeek.com/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"
```

```bash Ingestion theme={null}
claude mcp add --scope user --transport http mixpeek-ingestion https://mcp.mixpeek.com/ingestion/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"
```

```bash Retrieval theme={null}
claude mcp add --scope user --transport http mixpeek-retrieval https://mcp.mixpeek.com/retrieval/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"
```

```bash Admin theme={null}
claude mcp add --scope user --transport http mixpeek-admin https://mcp.mixpeek.com/admin/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"
```

**Cursor**

Add this to `~/.cursor/mcp.json`, or to `.cursor/mcp.json` in your project:

```json App (OAuth) theme={null}
{
  "mcpServers": {
    "mixpeek": {
      "url": "https://mcp.mixpeek.com/app/mcp"
    }
  }
}
```

```json Full (API key) theme={null}
{
  "mcpServers": {
    "mixpeek": {
      "url": "https://mcp.mixpeek.com/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

With the App config, Cursor opens a Mixpeek sign-in page the first time it connects. To use a scoped key-based server instead, swap the URL for one from the [table above](#hosted-server-urls).

### Sign in with OAuth

The App server at `https://mcp.mixpeek.com/app/mcp` uses OAuth 2.1. It exposes 9 read-only tools: list and read namespaces, collections, documents and retrievers, run a retriever, and search a namespace.

1. Add the URL to your client with no `Authorization` header.
2. On first connect, the server answers `401` with a `WWW-Authenticate` header that names its metadata at `https://mcp.mixpeek.com/.well-known/oauth-protected-resource/app/mcp`.
3. Your client opens the Mixpeek sign-in page at `auth.mixpeek.com` and requests the `mixpeek:read` scope.
4. After you approve, the client stores the token and the tools load.

The sign-in acts as you in your Mixpeek organization, with your role's permissions. The App server exposes read tools only, so an OAuth connection can search and read but not change anything. For testing, the App server also accepts a Mixpeek API key in `Authorization: Bearer YOUR_API_KEY`.

***

## Per-Retriever MCP Server

The retriever server is a lightweight MCP server scoped to a **single retriever**. It reads your retriever's `input_schema` at startup and generates a typed `search` tool whose parameters match exactly.

**1. Install**

```bash theme={null}
pip install mixpeek-mcp-retriever
```

**2. Run**

```bash theme={null}
mixpeek-mcp-retriever \
  --retriever-id ret_xxx \
  --namespace-id ns_xxx \
  --api-key YOUR_API_KEY
```

**3. Connect**

Add to your Claude Desktop config:

```json theme={null}
{
  "mcpServers": {
    "my-search": {
      "command": "mixpeek-mcp-retriever",
      "args": [
        "--retriever-id", "ret_xxx",
        "--namespace-id", "ns_xxx",
        "--api-key", "YOUR_API_KEY"
      ]
    }
  }
}
```

The retriever server exposes three tools:

| Tool | Description |
| - | - |
| `search` | Execute the retriever. Parameters are generated from the retriever's `input_schema` -- field names, types, required flags, enums, and descriptions. Pagination (`page`, `page_size`) is added automatically. |
| `describe` | Returns structured metadata: retriever ID, name, collections, input fields, and stage configuration. |
| `explain` | Returns a human-readable explanation of the pipeline: what each stage does, in order. |

> [!NOTE]
>
> If your retriever's `input_schema` has a field named `page` or `page_size`, the pagination parameters are automatically renamed to `_pagination_page` and `_pagination_page_size` to avoid conflicts.

You can also run the retriever server over HTTP for deployed agents:

```bash theme={null}
mixpeek-mcp-retriever \
  --retriever-id ret_xxx \
  --namespace-id ns_xxx \
  --api-key YOUR_API_KEY \
  --transport http \
  --port 8081
```

Or configure everything via environment variables (prefixed with `RETRIEVER_MCP_`):

```bash theme={null}
export RETRIEVER_MCP_RETRIEVER_ID=ret_xxx
export RETRIEVER_MCP_NAMESPACE_ID=ns_xxx
export RETRIEVER_MCP_API_KEY=YOUR_API_KEY
export RETRIEVER_MCP_TRANSPORT=stdio
mixpeek-mcp-retriever
```

***

## Available Tools by Scope

The `full` scope serves all 87 tools. The App scope is a read-only subset of them. The three key-based scopes below overlap: `list_storage_connections` and `get_storage_connection` are in two of them, which is why the per-scope counts sum to more than the total.

### Ingestion -- 29 tools

| Tool | Description |
| - | - |
| `clone_collection` | Clone an existing collection with optional overrides |
| `create_bucket` | Create a new bucket for file storage and processing |
| `create_bucket_sync` | Attach a storage connection to a bucket so its objects are ingested |
| `create_collection` | Create a new collection for processing and searching documents |
| `create_document` | Create a new document in a collection |
| `delete_bucket` | Delete a bucket and all its objects |
| `delete_collection` | Delete a collection and all its documents |
| `delete_document` | Delete a document from a collection |
| `get_bucket` | Get details of a specific bucket |
| `get_bucket_sync` | Get one sync configuration, including its status and counters |
| `get_collection` | Get details of a specific collection |
| `get_collection_failures` | Why documents in one collection failed to process, grouped by cause |
| `get_collection_lifecycle` | Get the storage lifecycle status of a collection, including current tier (active/cold/archived), vector counts, and last transition timestamp |
| `get_collection_overview` | Document counts and processing state for one collection |
| `get_document` | Get a specific document by ID |
| `get_storage_connection` | Get one storage connection by id or name |
| `list_bucket_syncs` | List the sync configurations on a bucket |
| `list_buckets` | List all buckets in a namespace |
| `list_collections` | List all collections in a namespace |
| `list_documents` | List documents in a collection with filters |
| `list_storage_connections` | List storage connections in the organization |
| `transition_collection_lifecycle` | Transition a collection's storage tier |
| `trigger_bucket_sync` | Run a sync configuration now instead of waiting for its schedule |
| `trigger_collection` | Trigger collection processing on bucket objects |
| `update_bucket` | Update bucket configuration |
| `update_collection` | Update collection configuration |
| `update_document` | Update a document's data |
| `upload_object` | Upload an object (file) to a bucket |
| `upsert_documents_byo` | Upsert documents with pre-computed vectors directly into a namespace |

### Retrieval -- 21 tools

| Tool | Description |
| - | - |
| `agent_signup` | Get a Mixpeek API key and namespace without a browser, for an agent acting for a named person |
| `clone_retriever` | Clone an existing retriever with optional modifications |
| `create_agent_session` | Create a new conversational agent session |
| `create_interaction` | Record a user interaction (click, view, conversion) against a retriever result |
| `create_retriever` | Create a multi-stage search pipeline |
| `delete_retriever` | Delete a retriever |
| `execute_marketplace_retriever` | Run a public marketplace retriever by name against its hosted dataset |
| `execute_retriever` | Execute a retriever pipeline with inputs and get search results |
| `get_agent_history` | Get conversation history for an agent session |
| `get_marketplace_listing` | Get one marketplace listing, including the inputs its execute call expects |
| `get_marketplace_template` | Get a marketplace listing's stages, to create the same retriever over your own collections |
| `get_retriever` | Get details of a specific retriever including all stages |
| `get_retriever_performance` | Latency and volume for one retriever over a time range |
| `get_retriever_slow_queries` | The slowest recent queries on one retriever |
| `list_interactions` | List recorded interactions, for inspecting or auditing the learning signal |
| `list_marketplace` | List the public retrievers Mixpeek hosts over sample datasets |
| `list_retrievers` | List all retrievers in a namespace |
| `respond_to_agent_confirmation` | Approve or reject the write an agent session paused on, and get the rest of its response |
| `search_namespace` | Search across all resources in a namespace (buckets, collections, retrievers, etc.) |
| `send_agent_message` | Send a message to an agent session and get a response; a write pauses for `respond_to_agent_confirmation` |
| `update_retriever` | Update retriever configuration |

### Admin -- 39 tools

| Tool | Description |
| - | - |
| `apply_manifest` | Apply a manifest: create or update the resources it declares |
| `create_cluster` | Create a document clustering configuration |
| `create_namespace` | Create a new namespace (workspace) for organizing collections and resources |
| `create_snapshot` | Take a point-in-time snapshot of a namespace |
| `create_standalone_namespace` | Create a standalone namespace for BYO vectors |
| `create_storage_connection` | Create a storage connection to an external source (S3, Google Drive, RTSP camera, and others) |
| `create_taxonomy` | Create a hierarchical classification taxonomy |
| `create_webhook` | Create a webhook that posts events (batch finished, documents updated and others) to a URL |
| `delete_cluster` | Delete a cluster configuration |
| `delete_namespace` | Delete a namespace and all its resources |
| `delete_storage_connection` | Delete a storage connection |
| `delete_taxonomy` | Delete a taxonomy |
| `delete_webhook` | Delete a webhook |
| `diff_manifest` | Show what applying a manifest would create, change or leave alone |
| `execute_cluster` | Execute clustering algorithm on a collection |
| `execute_taxonomy` | Apply taxonomy classification to document data |
| `export_manifest` | Export a namespace's resources as a manifest |
| `get_billing_transactions` | List individual credit transactions (the per-operation ledger): each debit, what consumed it, the namespace it's attributed to, and when |
| `get_credit_balance` | Get the organization's current credit balance, account tier, daily burn rate, and estimated days remaining |
| `get_namespace` | Get details of a specific namespace by ID or name |
| `get_namespace_errors` | Recent errors across a namespace |
| `get_snapshot` | Get one snapshot, including its status and size |
| `get_storage_connection` | Get one storage connection by id or name |
| `get_taxonomy` | Get details of a specific taxonomy |
| `get_usage_breakdown` | Get itemized credit usage for a billing month — broken down by operation, by extractor, by namespace (per-client/team chargeback), and by retriever (per-product cost) |
| `get_usage_summary` | Credit usage summary for the organization |
| `get_webhook` | Get one webhook |
| `list_clusters` | List all cluster configurations in a namespace |
| `list_namespaces` | List all namespaces in the organization |
| `list_snapshots` | List a namespace's snapshots |
| `list_storage_connections` | List storage connections in the organization |
| `list_taxonomies` | List all taxonomies in a namespace |
| `list_webhooks` | List the organization's webhooks |
| `restore_snapshot` | Restore a namespace from one of its snapshots |
| `test_storage_connection` | Test a storage connection against the live source and report what it found |
| `update_namespace` | Update namespace configuration |
| `update_webhook` | Update a webhook's URL, events or enabled state |
| `validate_manifest` | Validate a Mixpeek manifest (namespaces, buckets, collections, retrievers) without applying it |
| `verify_snapshot` | Check that a snapshot is complete and restorable |

### App -- 9 tools

Served at `https://mcp.mixpeek.com/app/mcp` behind OAuth. Every tool is read-only.

| Tool | Description |
| - | - |
| `execute_retriever` | Execute a retriever pipeline with inputs and get search results |
| `get_collection` | Get details of a specific collection |
| `get_document` | Get a specific document by ID |
| `get_retriever` | Get details of a specific retriever including all stages |
| `list_collections` | List all collections in a namespace |
| `list_documents` | List documents in a collection with filters |
| `list_namespaces` | List all namespaces in the organization |
| `list_retrievers` | List all retrievers in a namespace |
| `search_namespace` | Search across all resources in a namespace (buckets, collections, retrievers, etc.) |

## Authentication

The Full, Ingestion, Retrieval and Admin servers authenticate with your Mixpeek API key. The key carries the same RBAC permissions as the REST API. The App server uses OAuth instead.

* **App server:** Your client signs you in through OAuth 2.1 with the `mixpeek:read` scope. See [Sign in with OAuth](#sign-in-with-oauth).
* **Hosted servers (HTTP):** Pass the key in the `Authorization: Bearer YOUR_API_KEY` header. The server injects it into every tool call.
* **Stdio servers:** Set the `MIXPEEK_API_KEY` environment variable, or pass `--api-key` as a CLI argument.

> [!WARNING]
>
> Never commit API keys to version control. For deployed agents, use environment variables or a secrets manager.

***

## Example Conversation

Here is what a session looks like with the Retrieval server connected:

```
You: "Find all video clips where someone mentions quarterly revenue"
Claude: [calls execute_retriever with query="quarterly revenue"]
--> Returns 8 matching video segments with timestamps and transcript excerpts.

You: "Summarize the top 3 results"
Claude: [calls execute_retriever with query="quarterly revenue", then processes results]
--> "1. Q3 earnings call (2:34) -- CFO reports 22% YoY growth.
     2. Board presentation (14:12) -- Revenue breakdown by region.
     3. Team standup (0:45) -- Quick mention of hitting quarterly target."

You: "Create a retriever that searches product images by description and filters by brand"
Claude: [calls create_retriever with feature_search + attribute_filter stages]
--> "Created retriever ret_abc123 with 2 stages: feature search on image
     embeddings, then attribute filter on the brand field."
```

And with the per-retriever server:

```
You: "What does this retriever do?"
Claude: [calls describe]
--> "This is 'Product Search' -- searches your products collection
     by text query with an optional category filter, using
     feature search -> attribute filter -> reranking."

You: "Search for wireless headphones under electronics"
Claude: [calls search with query="wireless headphones", category="electronics"]
--> Returns top 10 matching products with scores and metadata.

You: "Explain the pipeline"
Claude: [calls explain]
--> "1. feature_search: embeds your query and finds the top 50 matches.
     2. attribute_filter: filters by category.
     3. rerank: reranks with Cohere down to the top 10."
```

***

## Architecture

```
+-----------------------------------------------------------+
|                   Claude / AI Agent                       |
+-----------------------------+-----------------------------+
                              | MCP Protocol (HTTP / Stdio)
                              v
+-----------------------------------------------------------+
|              Mixpeek MCP Server                           |
|  +--------+ +-----------+ +-----------+ +--------+       |
|  |  Full  | | Ingestion | | Retrieval | | Admin  |       |
|  | (66)   | |   (27)    | |   (18)    | |  (23)  |       |
|  +---+----+ +-----+-----+ +-----+-----+ +---+----+       |
|      +------------+-------------+-----------+             |
|                    Tool Handlers                          |
+-----------------------------+-----------------------------+
                              |       +--------------------+
                              |       | Retriever Server   |
                              |       | (per-retriever, 3) |
                              |       +---------+----------+
                              +--------+--------+
                                       | API calls
                                       v
                            +----------------------+
                            |   Mixpeek Platform   |
                            +----------------------+
```

The scoped servers share the same codebase. Scoping controls which tools are registered, not how they execute. Each scope is mounted at its own path prefix (`/ingestion`, `/retrieval`, `/admin`) while the full server runs at `/mcp`. The App server runs at `/app/mcp`.

***

## Troubleshooting

> **Claude can't connect to the MCP server**
>
> * Verify the URL is correct (e.g. `https://mcp.mixpeek.com/ingestion/mcp`)
> * Check that the `Authorization` header format is `Bearer YOUR_API_KEY`
> * Restart Claude Desktop or Claude Code after changing config

> **Tools return 'Unauthorized' or 'Invalid API key'**
>
> * Verify your API key at [studio.mixpeek.com](https://studio.mixpeek.com)
> * Check that the key has permissions for the namespace you're accessing
> * Make sure there are no extra spaces in the key

> **Tool not found (404)**
>
> * You may be calling a tool on the wrong scoped server (e.g. `execute_retriever` on `/ingestion`)
> * Use the full server if you need all tools

> **Retriever Server fails to start**
>
> * Ensure `--retriever-id` and `--namespace-id` are correct
> * Verify the API key has access to that namespace
> * Check that the retriever exists via the API

> **Search returns empty results**
>
> * Confirm your collection has processed documents (not just uploaded files)
> * Check that the retriever's `feature_uri` matches your collection's extractor
> * Try a broader query or remove optional filters

## Next Steps

> **[LangChain Integration](/docs/agent-integrations/langchain)**
>
> Use Mixpeek as a LangChain tool

> **[Retriever Stages](/docs/retrieval/stages/overview)**
>
> Configure multi-stage search pipelines

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Choose the right extractor for your data

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand namespaces, collections, and documents


# OpenAI Function Calling
Source: https://docs.mixpeek.com/docs/agent-integrations/openai-function-calling

Wire Mixpeek retrievers into OpenAI's function calling API so GPT models can search multimodal content

Wire Mixpeek retrievers into OpenAI's function calling API so GPT models can search video, image, and audio content on demand.

> [!NOTE]
>
> There is no `langchain-mixpeek` package. You use the standard `mixpeek` SDK and wrap it as an OpenAI function. This gives you full control over input parsing, error handling, and response formatting.

## The Pattern

OpenAI function calling lets GPT models decide when to invoke external tools during a conversation. You define a function schema, register it as a tool, and handle the call in your completion loop.

**1. Define the function schema**

Describe `search_mixpeek` with a name, description, and parameters so the model knows when and how to call it.

**2. Register it as a tool**

Pass the schema in the `tools` array when calling `chat.completions.create()`.

**3. Handle tool calls in the completion loop**

When the model returns `tool_calls`, execute `client.retrievers.execute()` with the provided arguments and append the results as a tool message.

**4. Get the final response**

Call `chat.completions.create()` again with the tool results. The model incorporates the search results into its answer.

## Installation

```bash Python theme={null}
pip install mixpeek openai
```

```bash JavaScript theme={null}
npm install mixpeek openai
```

## Function Schema

Define a function that tells GPT what Mixpeek search does and what inputs it accepts:

```json theme={null}
{
  "type": "function",
  "function": {
    "name": "search_mixpeek",
    "description": "Search across video, image, and audio content indexed in Mixpeek. Use this when the user asks about visual content, media files, or multimedia information.",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "Natural language search query describing what to find"
        },
        "limit": {
          "type": "integer",
          "description": "Maximum number of results to return (default 5)"
        }
      },
      "required": ["query"]
    }
  }
}
```

> [!TIP]
>
> Write a specific `description` that tells the model *when* to call this tool. Mention the content types your retriever handles (video, images, audio). Generic descriptions like "search for things" cause the model to over- or under-use the function.

## Full Working Example: Chat Completions API

```python Python theme={null}
import json
from openai import OpenAI
from mixpeek import Mixpeek

openai_client = OpenAI(api_key="YOUR_OPENAI_KEY")
mixpeek_client = Mixpeek(api_key="YOUR_MIXPEEK_KEY")

# Define the tool
tools = [
    {
        "type": "function",
        "function": {
            "name": "search_mixpeek",
            "description": (
                "Search across video, image, and audio content. "
                "Use when the user asks about visual content, media files, "
                "or multimedia information."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "Natural language search query",
                    },
                    "limit": {
                        "type": "integer",
                        "description": "Max results to return (default 5)",
                    },
                },
                "required": ["query"],
            },
        },
    }
]


def execute_search(query: str, limit: int = 5) -> str:
    """Call the Mixpeek retriever and return results as JSON."""
    results = mixpeek_client.retrievers.execute(
        retriever_id="ret_abc123",
        inputs={"query": query},
        namespace="my-namespace",
    )
    # Trim to limit and keep only essential fields for token efficiency
    trimmed = [
        {
            "document_id": doc["document_id"],
            "score": doc["score"],
            "metadata": doc.get("metadata", {}),
        }
        for doc in results[:limit]
    ]
    return json.dumps(trimmed, indent=2)


def chat(user_message: str):
    messages = [
        {
            "role": "system",
            "content": "You help users find and analyze multimedia content.",
        },
        {"role": "user", "content": user_message},
    ]

    # First call -- model may request a tool call
    response = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

    # Handle tool calls
    if message.tool_calls:
        messages.append(message)

        for tool_call in message.tool_calls:
            args = json.loads(tool_call.function.arguments)
            result = execute_search(
                query=args["query"],
                limit=args.get("limit", 5),
            )
            messages.append(
                {
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "content": result,
                }
            )

        # Second call -- model generates final answer with results
        response = openai_client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools,
        )
        message = response.choices[0].message

    return message.content


print(chat("Find clips where someone mentions the product launch"))
```

```javascript JavaScript theme={null}
import OpenAI from "openai";
import Mixpeek from "mixpeek";

const openai = new OpenAI({ apiKey: "YOUR_OPENAI_KEY" });
const mixpeek = new Mixpeek({ apiKey: "YOUR_MIXPEEK_KEY" });

const tools = [
  {
    type: "function",
    function: {
      name: "search_mixpeek",
      description:
        "Search across video, image, and audio content. " +
        "Use when the user asks about visual content, media files, " +
        "or multimedia information.",
      parameters: {
        type: "object",
        properties: {
          query: {
            type: "string",
            description: "Natural language search query",
          },
          limit: {
            type: "integer",
            description: "Max results to return (default 5)",
          },
        },
        required: ["query"],
      },
    },
  },
];

async function executeSearch(query, limit = 5) {
  const results = await mixpeek.retrievers.execute({
    retrieverId: "ret_abc123",
    inputs: { query },
    namespace: "my-namespace",
  });

  const trimmed = results.slice(0, limit).map((doc) => ({
    document_id: doc.document_id,
    score: doc.score,
    metadata: doc.metadata || {},
  }));

  return JSON.stringify(trimmed, null, 2);
}

async function chat(userMessage) {
  const messages = [
    { role: "system", content: "You help users find and analyze multimedia content." },
    { role: "user", content: userMessage },
  ];

  let response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages,
    tools,
  });

  let message = response.choices[0].message;

  if (message.tool_calls) {
    messages.push(message);

    for (const toolCall of message.tool_calls) {
      const args = JSON.parse(toolCall.function.arguments);
      const result = await executeSearch(args.query, args.limit || 5);
      messages.push({
        role: "tool",
        tool_call_id: toolCall.id,
        content: result,
      });
    }

    response = await openai.chat.completions.create({
      model: "gpt-4o",
      messages,
      tools,
    });
    message = response.choices[0].message;
  }

  return message.content;
}

console.log(await chat("Find clips where someone mentions the product launch"));
```

## Assistants API

You can register Mixpeek search as a tool on an OpenAI Assistant. The Assistants API manages conversation state and persistent threads for you.

```python theme={null}
import json
import time
from openai import OpenAI
from mixpeek import Mixpeek

openai_client = OpenAI(api_key="YOUR_OPENAI_KEY")
mixpeek_client = Mixpeek(api_key="YOUR_MIXPEEK_KEY")

# Create an assistant with the Mixpeek tool
assistant = openai_client.beta.assistants.create(
    name="Media Search Assistant",
    instructions=(
        "You help users search and analyze video, image, and audio content. "
        "Use the search_mixpeek tool whenever the user asks about media files."
    ),
    model="gpt-4o",
    tools=[
        {
            "type": "function",
            "function": {
                "name": "search_mixpeek",
                "description": (
                    "Search across video, image, and audio content indexed in Mixpeek."
                ),
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "type": "string",
                            "description": "Natural language search query",
                        },
                        "limit": {
                            "type": "integer",
                            "description": "Max results to return (default 5)",
                        },
                    },
                    "required": ["query"],
                },
            },
        }
    ],
)

# Create a thread and send a message
thread = openai_client.beta.threads.create()
openai_client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Find video frames showing the CEO on stage at the keynote",
)

# Start a run
run = openai_client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id,
)

# Poll until complete, handling tool calls
while True:
    run = openai_client.beta.threads.runs.retrieve(
        thread_id=thread.id, run_id=run.id
    )

    if run.status == "requires_action":
        tool_outputs = []
        for tool_call in run.required_action.submit_tool_outputs.tool_calls:
            args = json.loads(tool_call.function.arguments)
            results = mixpeek_client.retrievers.execute(
                retriever_id="ret_abc123",
                inputs={"query": args["query"]},
                namespace="my-namespace",
            )
            trimmed = [
                {
                    "document_id": doc["document_id"],
                    "score": doc["score"],
                    "metadata": doc.get("metadata", {}),
                }
                for doc in results[: args.get("limit", 5)]
            ]
            tool_outputs.append(
                {
                    "tool_call_id": tool_call.id,
                    "output": json.dumps(trimmed),
                }
            )

        run = openai_client.beta.threads.runs.submit_tool_outputs(
            thread_id=thread.id,
            run_id=run.id,
            tool_outputs=tool_outputs,
        )

    elif run.status == "completed":
        break
    elif run.status in ("failed", "cancelled", "expired"):
        print(f"Run ended with status: {run.status}")
        break
    else:
        time.sleep(1)

# Get the assistant's response
messages = openai_client.beta.threads.messages.list(thread_id=thread.id)
print(messages.data[0].content[0].text.value)
```

## Tips

### Write Descriptive Function Schemas

The `description` field on your function and its parameters directly affects when and how the model calls your tool. Be specific about what content types your retriever handles.

```python theme={null}
# Good -- tells the model exactly when to use it
"Search video frames, audio transcripts, and image content in the media library. Returns timestamped results with relevance scores."

# Bad -- too vague, model won't know when to call it
"Search for stuff in the database."
```

### Limit Result Size for Token Efficiency

Every result you return gets added to the conversation context. Strip unnecessary fields and cap the number of results to avoid wasting tokens.

```python theme={null}
def execute_search(query: str, limit: int = 5) -> str:
    results = mixpeek_client.retrievers.execute(
        retriever_id="ret_abc123",
        inputs={"query": query},
        namespace="my-namespace",
    )
    # Return only what the model needs to formulate an answer
    trimmed = [
        {
            "id": doc["document_id"],
            "score": round(doc["score"], 3),
            "title": doc.get("metadata", {}).get("title", ""),
            "summary": doc.get("metadata", {}).get("description", "")[:200],
        }
        for doc in results[:limit]
    ]
    return json.dumps(trimmed)
```

### Handle Errors Gracefully

Return error messages as tool output instead of raising exceptions. This lets the model recover or ask the user to rephrase.

```python theme={null}
def execute_search(query: str, limit: int = 5) -> str:
    try:
        results = mixpeek_client.retrievers.execute(
            retriever_id="ret_abc123",
            inputs={"query": query},
            namespace="my-namespace",
        )
        trimmed = [
            {"id": doc["document_id"], "score": doc["score"]}
            for doc in results[:limit]
        ]
        return json.dumps(trimmed)
    except Exception as e:
        return json.dumps({"error": str(e), "suggestion": "Try a different query."})
```

> [!NOTE]
>
> If you use `parallel_tool_calls` (enabled by default in the Chat Completions API), the model may issue multiple `search_mixpeek` calls in a single response. The handler loop in the examples above already supports this -- each tool call is processed independently.

## Next Steps

> **[LangChain](/docs/agent-integrations/langchain)**
>
> Use Mixpeek as a LangChain tool for agent workflows

> **[MCP Server](/docs/agent-integrations/mcp)**
>
> Connect Claude directly via the Model Context Protocol

> **[Retriever Stages](/docs/retrieval/stages/overview)**
>
> Configure multi-stage search pipelines

> **[Python SDK](/docs/integrations/developer-tools/python-sdk)**
>
> Full SDK reference


# Apps
Source: https://docs.mixpeek.com/docs/canvas/apps

Deploy your own frontend code — React, vanilla JS, or any static site — connected to your Mixpeek retrievers. Served on your custom domain with auth built in.

> [!NOTE]
>
> Canvas Apps require an **Enterprise** plan. [Contact us](https://mixpeek.com/contact) to upgrade.

## Overview

Canvas is Mixpeek's **application hosting platform**. Build a web app using any frontend framework — React, Vue, Svelte, vanilla JS, or plain HTML — and deploy it as a zip bundle. Mixpeek handles hosting, CDN, auth, versioning, and the entire backend.

Your app is live at `{slug}.mxp.co` the moment you deploy. No servers to manage, no infrastructure to configure, no API keys in your frontend code.

### How it works

You write frontend code. Mixpeek provides everything else:

* **Hosting** — your built assets (HTML, JS, CSS, images) are stored on S3 and served via CDN with immutable caching
* **Backend proxy** — all `/api/*` requests are forwarded to `api.mixpeek.com` with your org's API key and namespace injected server-side. Your credentials never reach the browser.
* **Runtime injection** — the Canvas runtime injects `window.__MIXPEEK__` into your HTML at serve time with auth config, environment variables, and monitoring hooks
* **Auth** — optionally enable Clerk-powered sign-in (Google, GitHub, email) with per-app user management. The auth SDK is auto-injected — no libraries to install.
* **Environments** — deploy to staging (`staging-{slug}.mxp.co`) or production (`{slug}.mxp.co`) independently
* **Versioning** — every deploy creates an immutable version with content hash, commit message, and source files. Rollback instantly.
* **Monitoring** — error boundaries, Sentry, and PostHog are auto-injected for crash detection and usage analytics

You bring the UI, Mixpeek brings search, retrieval, ingestion, and multimodal AI.

> **[Deploy from code](/docs/canvas/apps/deploy)**
>
> Upload any React, vanilla JS, or static site. Mixpeek serves it from S3 with CDN caching.

> **[Custom domains](/docs/canvas/apps/domains)**
>
> Default domain at `{slug}.mxp.co`. Add your own subdomain via CNAME with auto TLS.

> **[Built-in auth](/docs/canvas/apps/authentication)**
>
> Clerk-powered sign-in (Google, GitHub, email) — auto-injected into your app with zero config.

> **[User management](/docs/canvas/apps/users)**
>
> Invite members, assign roles, and control per-app access — powered by Clerk organizations.

> **[Git-like version control](/docs/canvas/apps/versions)**
>
> Every publish and deploy creates an immutable version with a content hash, message, and diffable snapshot.

***

## Architecture

When a user visits `{slug}.mxp.co`, here's the request lifecycle:

1. **DNS** — resolves to the Canvas runtime (Cloudflare-proxied, DDoS protected)
2. **Routing** — the Express server maps the hostname to your `app_id` (cached in Redis). Supports `{slug}.mxp.co`, `staging-{slug}.mxp.co`, and custom domains.
3. **HTML** — `index.html` is fetched from S3 and dynamically injected with `window.__MIXPEEK__` (runtime config, auth, monitoring)
4. **Assets** — JS, CSS, and images are served from S3 via CDN with immutable cache headers (content-hashed filenames → permanent caching)
5. **SPA routing** — non-asset paths fall through to `index.html` so client-side routing (React Router, etc.) works out of the box
6. **API proxy** — `/api/*` requests are forwarded to `api.mixpeek.com` with `Authorization` and `X-Namespace` headers injected server-side

> [!NOTE]
>
> Your API key is **never** sent to the browser. The `/api` proxy runs server-side and injects credentials on every request. Your frontend code only calls relative paths like `/api/v1/retrievers/execute`.

### Runtime config injection

The canvas runtime injects a `window.__MIXPEEK__` object into your app's HTML at serve time:

```javascript theme={null}
window.__MIXPEEK__ = {
  apiUrl: "https://api.mixpeek.com",  // Always present
  // Your custom env vars from build_config.env_vars
  MY_CUSTOM_VAR: "value",
  // Auth config (if enabled)
  auth: {
    mode: "clerk",
    publishableKey: "pk_live_...",
    orgId: "org_xyz",
    providers: ["google", "github", "email"],
  },
}
```

Access these values in your app code:

```jsx theme={null}
const apiUrl = window.__MIXPEEK__?.apiUrl
const customVar = window.__MIXPEEK__?.MY_CUSTOM_VAR
```

> [!WARNING]
>
> Do not use `process.env` in your app — it will crash in the browser. Use `window.__MIXPEEK__` for runtime config or `import.meta.env` for Vite build-time variables.

***

## Quickstart

**1. Create an App in Studio**

Go to **Apps** → **Create App**. Give it a name and a globally-unique slug — this becomes your default URL at `https://{slug}.mxp.co`.

**2. Build your frontend**

Write a React app (or any static site) that calls Mixpeek APIs via the canvas proxy. See the [example below](#canvas-sdk).

```bash theme={null}
# Bootstrap a React app
npm create vite@latest my-search-app -- --template react
cd my-search-app
npm install
npm run build
# Zip the dist/ folder
zip -r my-search-app.zip dist/
```

**3. Deploy the bundle**

In the App details page, drag & drop your `.zip` onto the Deploy panel and click **Deploy**. Your build is queued immediately.

Or via API:

```bash theme={null}
# 1. Get a presigned upload URL
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy/upload-url \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"filename": "my-app.zip", "content_type": "application/zip"}'

# 2. PUT the zip to the returned upload_url
curl -X PUT "$UPLOAD_URL" \
  -H "Content-Type: application/zip" \
  --data-binary @my-app.zip

# 3. Trigger the build
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"source": "cli_upload", "bundle_s3_key": "$BUNDLE_KEY", "environment": "production", "message": "Initial deploy"}'
```

**4. Visit your App**

Your app is live at `https://{slug}.mxp.co` within seconds of a successful build. Add a custom domain to use your own URL.

***

## Canvas SDK

The canvas runtime injects credentials **server-side** — your API key never reaches the browser. Call any Mixpeek API through the `/api` proxy:

```jsx theme={null}
// src/App.jsx — a minimal React search app
import { useState } from 'react'

async function search(query) {
  // No auth headers needed — the canvas proxy injects them server-side
  const res = await fetch('/api/v1/retrievers/ret_abc123/execute', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      inputs: { query },
      settings: { limit: 20 },
    }),
  })
  return res.json()
}

export default function App() {
  const [query, setQuery] = useState('')
  const [results, setResults] = useState([])
  const [loading, setLoading] = useState(false)

  const handleSearch = async (e) => {
    e.preventDefault()
    setLoading(true)
    const data = await search(query)
    setResults(data.documents ?? [])
    setLoading(false)
  }

  return (
    <div style={{ maxWidth: 800, margin: '0 auto', padding: 24 }}>
      <h1>Search</h1>
      <form onSubmit={handleSearch}>
        <input
          value={query}
          onChange={(e) => setQuery(e.target.value)}
          placeholder="Search..."
          style={{ width: '100%', padding: 8, fontSize: 16 }}
        />
        <button type="submit" disabled={loading}>
          {loading ? 'Searching...' : 'Search'}
        </button>
      </form>
      <ul>
        {results.map((r) => (
          <li key={r.document_id}>
            <strong>{r.fields?.title ?? r.document_id}</strong>
            <p>{r.fields?.description}</p>
          </li>
        ))}
      </ul>
    </div>
  )
}
```

### What `/api` supports

The proxy forwards **all** Mixpeek API methods with your org's credentials injected:

| Category | Example paths |
| - | - |
| **Retrievers** | `/api/v1/retrievers/{id}/execute`, `/api/v1/retrievers/list` |
| **Collections** | `/api/v1/collections/list`, `/api/v1/collections/{id}` |
| **Documents** | `/api/v1/documents/{id}`, `/api/v1/documents/list` |
| **Namespaces** | `/api/v1/namespaces/list` |
| **Batches** | `/api/v1/batches/list`, `/api/v1/batches/{id}` |
| **Tasks** | `/api/v1/tasks/{id}`, `/api/v1/tasks/list` |
| **Marketplace** | `/api/v1/marketplace/catalog/{name}/execute` |
| **Taxonomies** | `/api/v1/taxonomies/list`, `/api/v1/taxonomies/{id}/classify` |

> [!NOTE]
>
> Use `/api/v1/...` (relative path) instead of `https://api.mixpeek.com/v1/...` — the canvas proxy injects `Authorization` and `X-Namespace` headers automatically, avoiding CORS and keeping API keys out of your bundle.

### Using the Mixpeek JS SDK

You can also use the `mixpeek` npm package pointed at `/api`:

```jsx theme={null}
import { Mixpeek } from 'mixpeek'

// apiKey is a placeholder — the proxy injects the real key server-side
const client = new Mixpeek({ apiKey: 'canvas', baseUrl: '/api' })

// Execute a retriever
const results = await client.request({
  method: 'POST',
  url: '/v1/retrievers/ret_abc123/execute',
  data: { inputs: { query: 'red shoes' }, settings: { limit: 10 } },
})

// Execute a marketplace retriever
const marketplace = await client.request({
  method: 'POST',
  url: '/v1/marketplace/catalog/brand-safety/execute',
  data: { inputs: { query: 'test content' }, settings: { limit: 5 } },
})
```

### Billing

All API calls through `/api` are billed to the organization that owns the canvas app. End-users don't need their own Mixpeek API keys or accounts — usage is attributed to your org automatically.

***

## Environments

Each app supports two independent environments:

| Environment | URL | Use case |
| - | - | - |
| **Staging** | `staging-{slug}.mxp.co` | Test changes before going live |
| **Production** | `{slug}.mxp.co` | Live, user-facing deployment |

Each environment has its own S3 asset prefix, so staging and production can serve different versions simultaneously. Deploy to either environment from Studio or via the API.

### Deploying to staging

Set `"environment": "staging"` in the deploy request. Your staging build is live at `https://staging-{slug}.mxp.co`.

### Promoting to production

Deploy the same bundle to production, or use the restore endpoint to point production at a staging version's assets:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/versions/$STAGING_VERSION/restore \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment": "production"}'
```

***

## Creating an App

### Via Studio

Go to **Apps** → **Create App**. The wizard collects:

1. **Name + Slug** — the name is display-only; the slug becomes your URL (`{slug}.mxp.co`) and must be globally unique
2. **Access Control** — choose how end-users authenticate (public, password, API key, JWT, or SSO)

### Via API

Only `slug` and `meta` are required. After creation, deploy your code via the deploy pipeline.

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

app = client.apps.create(
    slug="product-search",
    meta={"title": "Product Search"},
    auth_config={"mode": "clerk"},  # optional — enable Clerk auth
)
print(f"App created: {app.app_id}")
print(f"URL: https://{app.slug}.mxp.co")
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/apps \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "slug": "product-search",
    "meta": { "title": "Product Search" },
    "auth_config": { "mode": "clerk" }
  }'
```

***

## Deploy lifecycle

Every deploy creates a new versioned build. The lifecycle is:

| Stage | Description |
| - | - |
| `queued` | Bundle uploaded, build queued |
| `building` | Mixpeek is packaging and deploying your bundle |
| `complete` | Build complete — app is live |
| `failed` | Build failed — previous version stays live, check error message |

### Operations

| Operation | What it does | Creates a version? |
| - | - | - |
| **Deploy** (`POST /v1/apps/{id}/deploy`) | Upload and serve a new code bundle | Yes — with asset manifest and message |
| **Restore** (`POST /v1/apps/{id}/versions/{v}/restore`) | Roll back to any previous version's assets | No — instant redirect |
| **Promote** (`POST /v1/apps/{id}/promote`) | Promote staging deploy to production | No — swaps environments |
| **Update** (`PATCH /v1/apps/{id}`) | Update app config (auth, meta, etc.) | No |

Each deploy requires a commit message describing what changed. See [Version History](/docs/canvas/apps/versions) for the full version control system.

***

## Authentication modes

| Mode | Description |
| - | - |
| `public` | Anyone can access — no login required |
| `password` | Shared password gate |
| `api_key` | Requires `X-App-Key` header |
| `clerk` | Clerk-powered sign-in with [per-app user management](/docs/canvas/apps/users) |
| `jwt` | Customer-managed tokens |
| `sso_oidc` | Okta, Auth0, Azure AD |
| `sso_saml` | Enterprise SAML 2.0 IdP |

> [!NOTE]
>
> Auth enforcement activates when the canvas runtime is deployed. Config is stored now and takes effect automatically.

***

## Custom domains

Every app gets a default URL at `https://{slug}.mxp.co`. You can also add your own subdomain:

1. Enter your subdomain (e.g., `search.yourdomain.com`) in the **Domains** panel
2. Add a DNS `CNAME` record pointing to the target shown in the response
3. Click **Verify** — Mixpeek provisions a TLS certificate automatically via Cloudflare

See [Custom Domains](/docs/canvas/apps/domains) for the full setup guide.

***

## Example apps

> **E-commerce product search**
>
> React app with multimodal search — text queries, image upload, and faceted filters backed by a Mixpeek retriever.

> **Internal knowledge base**
>
> Password-protected semantic search over company documents, PDFs, and meeting transcripts.

> **Media library**
>
> Video + image search portal with scene-level results, thumbnails, and timestamp previews.

> **Content moderation dashboard**
>
> Real-time moderation queue pulling from alert webhooks and showing flagged content with similarity scores.

***

## Related

* [Version History](/docs/canvas/apps/versions)
* [Deploy from Code](/docs/canvas/apps/deploy)
* [Authentication](/docs/canvas/apps/authentication)
* [User Management](/docs/canvas/apps/users)
* [Custom Domains](/docs/canvas/apps/domains)
* [Create App (API)](/docs/api-reference/apps/create-app)
* [Deploy App (API)](/docs/api-reference/apps/deploy-app)


# Authentication
Source: https://docs.mixpeek.com/docs/canvas/apps/authentication

Add Clerk-based sign-in to your Canvas app with zero configuration. Users authenticate via Google, GitHub, or email — the SDK is auto-injected.

## Overview

Canvas apps support built-in authentication powered by Clerk. When you enable auth, your users can sign up and sign in via Google, GitHub, or email/password — with no extra libraries or configuration on your part.

The auth SDK is **auto-injected** into your app at runtime. You get a `window.MixpeekAuth` object with methods to check sign-in state, show modals, and access user profiles.

***

## Enabling Auth

Enable Clerk authentication by updating your app's `auth_config`:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

client.apps.update(
    app_id="app_abc123",
    auth_config={"mode": "clerk"},
)
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'your-api-key' })

await client.apps.update({
  appId: 'app_abc123',
  authConfig: { mode: 'clerk' },
})
```

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"auth_config": {"mode": "clerk"}}'
```

Setting mode to `"clerk"` automatically:

* Injects `window.__MIXPEEK__` with Clerk configuration
* Loads the `/_auth/sdk.js` script into your app
* Enforces authentication on all `/api` proxy requests

> [!NOTE]
>
> You do not need to install or import any auth libraries. The canvas runtime handles everything.

***

## Using MixpeekAuth in Your App

The SDK exposes `window.MixpeekAuth` with the following API:

| Property / Method | Type | Description |
| - | - | - |
| `MixpeekAuth.onReady(callback)` | `(fn) => void` | Called when auth is initialized |
| `MixpeekAuth.isSignedIn` | `boolean` | Whether the current user is signed in |
| `MixpeekAuth.user` | `object` | `{ id, email, name, avatar_url }` |
| `MixpeekAuth.showSignIn()` | `() => void` | Opens the Clerk sign-in modal |
| `MixpeekAuth.showSignUp()` | `() => void` | Opens the Clerk sign-up modal |
| `MixpeekAuth.signOut()` | `() => void` | Signs out and reloads the page |

### React Example

```jsx theme={null}
// src/App.jsx — auth-gated Canvas app
import { useState, useEffect } from 'react'

export default function App() {
  const [user, setUser] = useState(null)
  const [ready, setReady] = useState(false)

  useEffect(() => {
    window.MixpeekAuth.onReady(() => {
      setReady(true)
      if (window.MixpeekAuth.isSignedIn) {
        setUser(window.MixpeekAuth.user)
      }
    })
  }, [])

  if (!ready) return <p>Loading...</p>

  if (!user) {
    return (
      <div>
        <h1>Welcome</h1>
        <p>Sign in to access this app.</p>
        <button onClick={() => window.MixpeekAuth.showSignIn()}>
          Sign In
        </button>
        <button onClick={() => window.MixpeekAuth.showSignUp()}>
          Sign Up
        </button>
      </div>
    )
  }

  return (
    <div>
      <h1>Hello, {user.name}</h1>
      <p>{user.email}</p>
      <button onClick={() => window.MixpeekAuth.signOut()}>
        Sign Out
      </button>
      {/* Your app content here */}
    </div>
  )
}
```

***

## Auth Endpoints

The canvas runtime exposes three auth endpoints on your app's domain:

| Endpoint | Method | Description |
| - | - | - |
| `/_auth/me` | GET | Returns the current user's profile |
| `/_auth/users` | GET | Lists all users for this app (requires auth) |
| `/_auth/sdk.js` | GET | The auth SDK script (auto-loaded, but you can load it manually) |

> [!TIP]
>
> Use `/_auth/me` from your frontend to verify auth state on page load without relying solely on the client-side SDK.

***

## API Access and Billing

All `/api` requests are authenticated with the **app owner's API key** — your end-users never need their own Mixpeek credentials.

* Usage is billed to the organization that owns the Canvas app
* The `/api` proxy supports all Mixpeek API methods: retrievers, collections, documents, and marketplace
* End-users interact with your app; your API key handles the backend calls transparently

> [!WARNING]
>
> Because all API usage is billed to your organization, monitor your usage in Studio to avoid unexpected costs from high-traffic apps.

***

## User Storage

When users sign up through your app, their metadata (email, name, avatar) is stored automatically. Each app maintains its own user list, scoped independently from other apps.

Access user data via the `/_auth/users` endpoint:

```bash theme={null}
curl https://your-app.mxp.co/_auth/users \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
[
  {
    "id": "user_abc123",
    "email": "jane@example.com",
    "name": "Jane Smith",
    "avatar_url": "https://..."
  }
]
```

***

## Auth Providers

By default, Clerk authentication supports:

* **Google** — OAuth sign-in
* **GitHub** — OAuth sign-in
* **Email / password** — standard credentials

Configure allowed providers via `auth_config.clerk_allowed_providers`:

```python Python theme={null}
client.apps.update(
    app_id="app_abc123",
    auth_config={
        "mode": "clerk",
        "clerk_allowed_providers": ["google", "github"],
    },
)
```

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "auth_config": {
      "mode": "clerk",
      "clerk_allowed_providers": ["google", "github"]
    }
  }'
```

> [!NOTE]
>
> Omit `clerk_allowed_providers` to enable all default providers (Google, GitHub, and email/password).

***

## Related

* [User Management](/docs/canvas/apps/users) — invite members, assign roles, remove users
* [Apps overview](/docs/canvas/apps)
* [Deploy from Code](/docs/canvas/apps/deploy)
* [Custom Domains](/docs/canvas/apps/domains)


# Build Logs
Source: https://docs.mixpeek.com/docs/canvas/apps/build-logs

Stream real-time build output during deploys and diagnose failures with persistent log history.

## Overview

Every Canvas deploy produces a detailed build log — from downloading your bundle through validation, asset upload, and completion. Logs stream in real time via Server-Sent Events (SSE) and persist in the deploy record for later review.

***

## Streaming logs

Subscribe to live build output during a deploy:

```bash theme={null}
curl -N https://api.mixpeek.com/v1/apps/$APP_ID/deploys/$DEPLOY_ID/logs/stream \
  -H "Authorization: Bearer $API_KEY"
```

The response is an SSE stream (`text/event-stream`). Each event contains a JSON log entry:

```
data: {"ts": 1711741200.123, "line": "Downloading bundle from S3..."}
data: {"ts": 1711741201.456, "line": "Extracting bundle..."}
data: {"ts": 1711741202.789, "line": "Validating bundle (index.html, size)..."}
data: {"ts": 1711741203.012, "line": "Uploading 14 assets to S3..."}
data: {"ts": 1711741205.345, "line": "Deploy complete!"}
event: done
data: {}
```

### Consuming in JavaScript

```javascript theme={null}
const response = await fetch(
  `https://api.mixpeek.com/v1/apps/${appId}/deploys/${deployId}/logs/stream`,
  { headers: { Authorization: `Bearer ${apiKey}` } }
)

const reader = response.body.getReader()
const decoder = new TextDecoder()

while (true) {
  const { done, value } = await reader.read()
  if (done) break

  const text = decoder.decode(value)
  for (const line of text.split('\n')) {
    if (line.startsWith('data: ')) {
      const entry = JSON.parse(line.slice(6))
      console.log(entry.line)
    }
  }
}
```

> [!NOTE]
>
> Use `fetch` + `ReadableStream` instead of `EventSource` — fetch supports `Authorization` headers, while `EventSource` does not.

***

## Reconnecting mid-deploy

If you disconnect and reconnect, the SSE endpoint replays all previously emitted log lines (from MongoDB) before switching to live streaming. No log lines are lost on reconnect.

***

## Log lifecycle

| Deploy stage | Log lines emitted |
| - | - |
| `queued` | None yet |
| `building` | Real-time build progress (download, extract, validate, upload) |
| `complete` | All lines + "Deploy complete!" |
| `failed` | All lines + "Deploy failed: " |

After a deploy reaches a terminal state (`complete` or `failed`), the full log is persisted in the deploy record and can be retrieved at any time.

***

## Git deploy logs

Deploys triggered via GitHub push include additional log lines from the build process:

```
Cloning https://github.com/org/repo (branch: main)...
Installing dependencies (npm ci)...
npm warn deprecated ...
Building project (npm run build)...
vite v5.4.2 building for production...
✓ 42 modules transformed.
Zipping dist/...
Uploading bundle to S3...
Deploy complete!
```

Real `npm ci` and `npm run build` output is streamed line-by-line, so you see the same output you'd see locally.

***

## Studio

In the App details page, the **Deploy** panel shows a terminal-style log viewer during active deploys. Logs auto-scroll as new lines arrive and remain viewable after completion.

***

## Related

* [Deploy from Code](/docs/canvas/apps/deploy)
* [GitHub Integration](/docs/canvas/apps/github)
* [CLI: `mixpeek apps logs`](/docs/canvas/apps/cli)


# CLI
Source: https://docs.mixpeek.com/docs/canvas/apps/cli

Deploy, stream logs, manage previews, and run a local dev server from the command line with @mixpeek/cli.

## Overview

The Mixpeek CLI (`@mixpeek/cli`) lets you deploy Canvas apps, stream build logs, list preview environments, and run a local dev server — all from your terminal.

***

## Installation

```bash theme={null}
npm install -g @mixpeek/cli
```

***

## Authentication

```bash theme={null}
mixpeek login
```

This stores your API key locally for use in subsequent commands. You can also pass `--api-key` to any command.

***

## Commands

### `mixpeek apps deploy`

Build your app locally and deploy it to Canvas.

```bash theme={null}
cd my-app
mixpeek apps deploy --app-id $APP_ID
```

What it does:

1. Detects your package manager (npm, yarn, pnpm)
2. Runs `npm run build` (or equivalent)
3. Zips the `dist/` output directory
4. Uploads the bundle to Mixpeek
5. Triggers a deploy

Options:

| Flag | Description | Default |
| - | - | - |
| `--app-id` | App ID to deploy to | Required |
| `--environment` | Target environment | `production` |
| `--message` | Commit message for the version | Auto-generated |
| `--api-key` | API key (overrides stored key) | From `mixpeek login` |

> [!NOTE]
>
> The CLI builds `dist/` (not `src/`). Make sure your `build` script outputs to `dist/`, `build/`, or `out/`.

***

### `mixpeek apps logs`

Stream real-time build logs for an active deploy.

```bash theme={null}
mixpeek apps logs --app-id $APP_ID --deploy-id $DEPLOY_ID
```

Log lines are printed with timestamps and colored output. The stream closes automatically when the deploy completes or fails.

***

### `mixpeek apps preview`

List active preview deployments for an app.

```bash theme={null}
mixpeek apps preview --app-id $APP_ID
```

Output:

```
Preview Environments:
  preview-42  →  https://pr-42-my-app.mxp.co
  preview-17  →  https://pr-17-my-app.mxp.co
```

***

### `mixpeek apps dev`

Start a local development server with Mixpeek environment variables pre-configured.

```bash theme={null}
mixpeek apps dev --app-id $APP_ID
```

This spawns your project's dev server (e.g., `npm run dev`) with `VITE_MIXPEEK_API_URL` set, so API proxy calls work locally.

***

## CI/CD integration

Use the CLI in GitHub Actions or any CI pipeline:

```yaml theme={null}
# .github/workflows/deploy.yml
name: Deploy to Canvas
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm install -g @mixpeek/cli
      - run: mixpeek apps deploy --app-id ${{ secrets.APP_ID }} --api-key ${{ secrets.MIXPEEK_API_KEY }} --message "Deploy from CI (${{ github.sha }})"
```

***

## Related

* [Deploy from Code](/docs/canvas/apps/deploy)
* [Build Logs](/docs/canvas/apps/build-logs)
* [Preview Deploys](/docs/canvas/apps/preview-deploys)


# Deploy from Code
Source: https://docs.mixpeek.com/docs/canvas/apps/deploy

Upload a zip of your frontend code and Mixpeek builds and hosts it automatically. Deploy to staging or production with full versioning.

## Overview

Deploy any React app, vanilla JS site, or static bundle to a Canvas app. Upload a `.zip` of your build output and Mixpeek handles hosting, CDN distribution, and versioning — no infrastructure needed.

Every deploy creates an immutable version record with a commit message, asset manifest, and content hashes. See [Version History](/docs/canvas/apps/versions) for the full version control system.

***

## Quickstart

**1. Build your frontend**

Build your app to a static output directory.

```bash theme={null}
# React / Vite
npm run build
# Output: dist/

# Next.js (static export)
npm run build && npm run export
# Output: out/
```

**2. Zip the output directory**

```bash theme={null}
zip -r my-app.zip dist/
```

The zip should contain an `index.html` at the root (or inside a single top-level folder). Mixpeek auto-detects the entry point.

**3. Get a presigned upload URL**

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy/upload-url \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"filename": "my-app.zip", "content_type": "application/zip"}'
```

Response:

```json theme={null}
{
  "upload_url": "https://s3.us-east-2.amazonaws.com/...",
  "bundle_s3_key": "app-builds/<app_id>/<token>/my-app.zip",
  "expires_in": 3600
}
```

Save `upload_url` for the next step and `bundle_s3_key` for the deploy request.

**4. Upload the zip**

```bash theme={null}
curl -X PUT "$UPLOAD_URL" \
  -H "Content-Type: application/zip" \
  --data-binary @my-app.zip
```

**5. Trigger the deploy**

A commit `message` is required — it describes what changed in this version.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "cli_upload",
    "bundle_s3_key": "$BUNDLE_KEY",
    "environment": "production",
    "message": "Initial deploy",
    "source_files": {
      "index.html": "<!DOCTYPE html>...",
      "src/app.js": "// your source"
    }
  }'
```

> [!WARNING]
>
> `source_files` is **required** for `cli_upload` deploys. Without it the deploy returns 400.
> Pass a `{path: content}` map of your source files so versions can be diffed and re-deployed.

Include `git_commit_sha` / `git_author` for CI/CD traceability.

Your app is live at `https://{slug}.mxp.co` within seconds of a successful build.

***

## Environments

Each app supports two independent environments with separate URLs and asset prefixes:

| Environment | URL pattern | Deploy field |
| - | - | - |
| **Production** | `{slug}.mxp.co` | `"environment": "production"` |
| **Staging** | `staging-{slug}.mxp.co` | `"environment": "staging"` |

Staging and production can serve different versions simultaneously. Deploy to staging first to test, then promote to production.

### Deploying to staging

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "cli_upload",
    "bundle_s3_key": "$BUNDLE_KEY",
    "environment": "staging",
    "message": "Test new search layout"
  }'
```

Your staging build is live at `https://staging-{slug}.mxp.co`.

### Promoting staging to production

Once you've verified staging, promote it to production by deploying the same bundle to the production environment, or use the restore endpoint to point production at the staging version:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/versions/$STAGING_VERSION/restore \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment": "production"}'
```

Restore is instant — no rebuild required.

***

## Deploy lifecycle

| Stage | Description |
| - | - |
| `queued` | Bundle uploaded, deploy queued |
| `building` | Mixpeek is validating, packaging, and uploading assets |
| `complete` | Deploy complete — new version is live |
| `failed` | Deploy failed — previous version stays live, error returned |

### Check deploy status

`GET /v1/apps/{app_id}/deploys/{deploy_id}` returns the current status of a deploy:

```bash theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/deploys/$DEPLOY_ID \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
{
  "deploy_id": "dep_abc123",
  "status": "building",
  "environment": "production",
  "message": "Initial deploy",
  "created_at": "2025-01-15T10:30:00Z"
}
```

The `status` field will be one of: `queued`, `building`, `complete`, or `failed`.

### Stream build logs

`GET /v1/apps/{app_id}/deploys/{deploy_id}/logs/stream` returns a Server-Sent Events (SSE) stream of build logs in real time. Use this to monitor progress or debug failed deploys:

```bash theme={null}
curl -N https://api.mixpeek.com/v1/apps/$APP_ID/deploys/$DEPLOY_ID/logs/stream \
  -H "Authorization: Bearer $API_KEY"
```

Each SSE event contains a log line from the build process. The stream closes automatically when the deploy reaches `complete` or `failed`.

***

## Multi-file output

Your zip can contain any number of files — HTML, JS, CSS, images, fonts. The only requirement is an `index.html` at the root. All files are uploaded to S3 and served with appropriate cache headers:

* **`index.html`** — no-cache (always fresh)
* **Content-hashed files** (e.g., `app-a1b2c3.js`) — immutable, permanent cache
* **Other assets** — standard cache headers

***

## Canvas SDK

Your app runs inside the Mixpeek canvas runtime. Call Mixpeek APIs through the built-in `/api` proxy — credentials are injected **server-side**, so your API key never reaches the browser:

```jsx theme={null}
// src/App.jsx — no auth headers needed
async function search(query) {
  const res = await fetch('/api/v1/retrievers/ret_abc123/execute', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ inputs: { query }, settings: { limit: 20 } }),
  })
  return res.json()
}
```

> [!NOTE]
>
> Use `/api/v1/...` (relative path) instead of `https://api.mixpeek.com/v1/...` — the canvas proxy injects `Authorization` and `X-Namespace` headers automatically, avoiding CORS and keeping API keys out of your bundle.

***

## Pre-deploy validation

Mixpeek runs automatic checks on every bundle before deploying:

| Check | What it catches |
| - | - |
| **index.html exists** | Missing entry point (zip structure wrong) |
| **Script tags present** | Blank page (no executable code) |
| **Asset references valid** | Broken `src`/`href` links in index.html |
| **No bare `process.env`** | Runtime crash in browser (use `window.__MIXPEEK__` instead) |
| **Bundle size \< 50 MB** | Accidentally included `node_modules` or large assets |
| **JS bundles non-empty** | Failed build that produced empty files |

If any check fails, the deploy is rejected with a descriptive error message.

***

## Deploy via Studio

In the App details page, drag & drop your `.zip` file onto the **Deploy** panel and click **Deploy**. The build is queued immediately and status updates in real time.

***

## Rollback

Every deploy is versioned. You have two rollback options:

**Quick rollback** — restore the previous config:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/rollback \
  -H "Authorization: Bearer $API_KEY"
```

**Restore any version** — point an environment to a specific version's assets:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/versions/3/restore \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment": "production"}'
```

Restore is instant — no rebuild required. All previous assets are retained in S3.

See [Version History](/docs/canvas/apps/versions) for the full version control system including diffs, downloads, and git metadata.

***

## Source files

When you include `source_files` in your deploy request, those files are stored alongside the version record. This enables:

* **Source-level diffs** — compare actual source code between versions, not just built output
* **Download and re-deploy** — download a version's source files and deploy modified code

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "cli_upload",
    "bundle_s3_key": "$BUNDLE_KEY",
    "environment": "production",
    "message": "Add dark mode",
    "source_files": {
      "src/App.tsx": "import React from ...",
      "src/components/Header.tsx": "export function Header() ...",
      "src/styles.css": "body { ... }"
    }
  }'
```

***

## Related

* [Apps overview](/docs/canvas/apps)
* [Version History](/docs/canvas/apps/versions)
* [Custom Domains](/docs/canvas/apps/domains)
* [Deploy App (API)](/docs/api-reference/apps/deploy-app)
* [List Versions (API)](/docs/api-reference/apps/list-versions)


# Custom Domains
Source: https://docs.mixpeek.com/docs/canvas/apps/domains

Point your own subdomain at a Mixpeek App. TLS is provisioned automatically.

## Overview

Every App gets a default URL at `{slug}.mxp.co`. You can also serve it on your own subdomain — for example, `search.yourcompany.com` — by adding a custom domain and creating a DNS CNAME record.

***

## Add a custom domain

**1. Open the Domains panel**

In the App details page, click the **Domains** tab.

**2. Enter your subdomain**

Type the subdomain you want to use (e.g. `search.yourcompany.com`) and click **Add Domain**.

**3. Add a DNS TXT record for verification**

Add a TXT record to prove domain ownership. The record name and value are returned in the API response:

| Type | Name | Value |
| - | - | - |
| TXT | `_mixpeek-verify.search.yourcompany.com` | `mixpeek-site-verification={verification_token}` |

**4. Create a CNAME record**

In your DNS provider, add a `CNAME` record pointing your subdomain to the `cname_target` returned by the API:

| Type | Name | Value |
| - | - | - |
| CNAME | search.yourcompany.com | `{slug}.mxp.co` |

**5. Trigger verification**

Call `POST /v1/apps/{app_id}/domains/{domain}/verify` to start DNS polling. Mixpeek polls every 30 minutes for up to 72 hours, then provisions a TLS certificate automatically via Cloudflare.

Domain status changes to `active` once the certificate is issued.

***

## Domain statuses

| Status | Meaning |
| - | - |
| `pending` | Domain added, waiting for DNS verification |
| `verifying` | TXT record found, polling for CNAME propagation |
| `provisioning_tls` | CNAME verified, issuing TLS certificate |
| `active` | Domain is live and serving HTTPS |
| `failed` | Verification failed after 72 hours — check your DNS records |

***

## Via API

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key", namespace_id="ns_...")

domain = client.apps.domains.add(
    app_id="app_...",
    domain="search.yourcompany.com",
)
print(domain.status)  # "pending"
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/domains \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{"domain": "search.yourcompany.com"}'
```

***

## Notes

* Only subdomain CNAMEs are supported (not apex/root domains).
* Each domain can be attached to one App at a time.
* Removing a domain does not affect the App's default `{slug}.mxp.co` URL.

***

## Related

* [Apps overview](/docs/canvas/apps)
* [Deployment guide](/docs/canvas/apps/deploy)
* [Add Domain (API)](/docs/api-reference/apps/add-domain)
* [List Domains (API)](/docs/api-reference/apps/list-domains)


# Environment Variables
Source: https://docs.mixpeek.com/docs/canvas/apps/environment-variables

Configure environment variables for your Canvas app — available in both the browser runtime and server functions.

## Overview

Canvas apps have two ways to access configuration:

1. **Browser runtime** — `window.__MIXPEEK__` (injected into every HTML response)
2. **Server functions** — `ctx.env` (available in your handler's context)

***

## Browser runtime variables

Canvas automatically injects a `window.__MIXPEEK__` object into every HTML page. This includes system variables and any custom environment variables you configure.

### System variables (always available)

| Variable | Description |
| - | - |
| `window.__MIXPEEK__.apiBaseUrl` | The API proxy URL (`/api`) |
| `window.__MIXPEEK__.appId` | Your app's ID |
| `window.__MIXPEEK__.slug` | Your app's slug |
| `window.__MIXPEEK__.environment` | Current environment (`production`, `staging`, or `preview-N`) |

### Accessing in your app

```jsx theme={null}
// React example
function App() {
  const mixpeek = window.__MIXPEEK__

  async function search(query) {
    const res = await fetch(`${mixpeek.apiBaseUrl}/v1/retrievers/ret_abc/execute`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ inputs: { query } }),
    })
    return res.json()
  }

  return <SearchBar onSearch={search} />
}
```

> [!NOTE]
>
> Use `window.__MIXPEEK__.apiBaseUrl` (which resolves to `/api`) instead of hardcoding `https://api.mixpeek.com`. The Canvas proxy injects your API key and namespace header server-side, keeping credentials out of the browser.

***

## Custom environment variables

Set custom environment variables on your app via the API:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "build_config": {
      "env_vars": {
        "FEATURE_FLAG_NEW_UI": "true",
        "ANALYTICS_ID": "UA-12345"
      }
    }
  }'
```

Custom env vars are:

* Injected into `window.__MIXPEEK__` for browser access
* Available as `ctx.env` in [server functions](/docs/canvas/apps/server-functions)
* Stored encrypted and never exposed in build logs

***

## Server function environment

Server functions receive all environment variables through `ctx.env`:

```typescript theme={null}
// api/config.ts
export default async function handler(ctx) {
  return {
    body: {
      feature_flag: ctx.env.FEATURE_FLAG_NEW_UI,
      api_key_set: !!ctx.env.MIXPEEK_API_KEY,
    },
  }
}
```

Server functions also have access to system environment variables like `MIXPEEK_API_KEY` and `MIXPEEK_NAMESPACE_ID` — these are injected server-side and never reach the browser.

***

## Environment-specific variables

Since Canvas supports multiple environments (production, staging, preview), you can set different variables per environment by deploying with different configurations.

***

## Related

* [Server Functions](/docs/canvas/apps/server-functions)
* [Deploy from Code](/docs/canvas/apps/deploy)
* [Apps Overview](/docs/canvas/apps)


# GitHub Integration
Source: https://docs.mixpeek.com/docs/canvas/apps/github

Connect a GitHub repo to your Canvas app for automatic deploys on push and preview URLs on pull requests.

## Overview

Connect a GitHub repository to a Canvas app and every push to your branch triggers an automatic build and deploy. Pull requests get [preview URLs](/docs/canvas/apps/preview-deploys) for live review before merging.

***

## Connecting a repo

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/connect-repo \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "repo_url": "https://github.com/your-org/your-repo",
    "branch": "main"
  }'
```

This configures a GitHub webhook that listens for `push` and `pull_request` events on the specified branch.

***

## Automatic deploys on push

When you push to the connected branch:

1. GitHub sends a `push` webhook to Canvas
2. Canvas clones your repo and checks out the pushed commit
3. Runs `npm ci` to install dependencies
4. Runs `npm run build` to produce the build output
5. Zips the `dist/` directory and deploys it

Build output is streamed in real time — see [Build Logs](/docs/canvas/apps/build-logs).

> [!NOTE]
>
> Canvas auto-detects your build output directory. It looks for `dist/`, `build/`, or `out/` in order.

***

## What gets built

Canvas runs a standard Node.js build process:

```
git clone <repo_url> --branch <branch> --depth 1
cd <repo>
npm ci
npm run build
```

Your `package.json` must include a `build` script. The build output must contain an `index.html`.

***

## Git metadata in versions

Every deploy from GitHub automatically records git metadata in the version record:

| Field | Description |
| - | - |
| `git_commit_sha` | The commit SHA that triggered the deploy |
| `git_author` | Author of the commit |
| `git_branch` | Branch name |

This metadata is visible in [Version History](/docs/canvas/apps/versions) and in Studio.

***

## Webhook events

| Event | Canvas behavior |
| - | - |
| `push` to connected branch | Build and deploy to production |
| `pull_request` opened/updated | Build and deploy [preview](/docs/canvas/apps/preview-deploys) |
| `pull_request` closed | Clean up preview environment |

***

## Disconnecting

To stop automatic deploys, remove the webhook from your GitHub repository settings or update the app's configuration to remove the `repo_url`.

***

## Related

* [Deploy from Code](/docs/canvas/apps/deploy)
* [Preview Deploys](/docs/canvas/apps/preview-deploys)
* [Build Logs](/docs/canvas/apps/build-logs)
* [Version History](/docs/canvas/apps/versions)


# Runtime Logs & Analytics
Source: https://docs.mixpeek.com/docs/canvas/apps/logs

View request logs, server function output, and performance metrics for your Canvas app.

## Overview

Canvas captures two types of runtime data for every app:

* **Request logs** — every HTTP request hitting your app (method, path, status, duration)
* **Runtime logs** — `console.log/warn/error/info` output from [server functions](/docs/canvas/apps/server-functions)

Both are kept for 30 days and are queryable via API or Studio.

***

## Request logs

Every request to your Canvas app is logged automatically — no configuration needed.

### Query via API

```bash theme={null}
curl "https://api.mixpeek.com/v1/apps/$APP_ID/logs?log_type=requests&hours=1&limit=100" \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
{
  "app_id": "app_1433d93d7a95",
  "log_type": "requests",
  "entries": [
    {
      "timestamp": "2026-03-29 21:21:02.750",
      "method": "GET",
      "path": "/",
      "status_code": 200,
      "duration_ms": 74,
      "user_agent": "Mozilla/5.0 ...",
      "ip": "68.175.65.201"
    }
  ]
}
```

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `log_type` | `string` | `runtime` | `runtime` or `requests` |
| `hours` | `integer` | `1` | Lookback window (1–168 hours) |
| `level` | `string` | — | Filter by log level (runtime only): `log`, `warn`, `error`, `info` |
| `limit` | `integer` | `100` | Max entries returned (1–1000) |

***

## Runtime logs

Server function `console` output is automatically captured and stored. Each log entry includes the function route and log level.

### Query via API

```bash theme={null}
# All runtime logs from the last hour
curl "https://api.mixpeek.com/v1/apps/$APP_ID/logs?log_type=runtime&hours=1" \
  -H "Authorization: Bearer $API_KEY"

# Only errors
curl "https://api.mixpeek.com/v1/apps/$APP_ID/logs?log_type=runtime&hours=24&level=error" \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
{
  "app_id": "app_1433d93d7a95",
  "log_type": "runtime",
  "entries": [
    {
      "timestamp": "2026-03-29 22:15:03.456",
      "level": "error",
      "message": "Failed: connection timeout",
      "route_path": "search",
      "request_id": "req_abc123"
    }
  ]
}
```

***

## Studio

In Studio, the App details page has an **Analytics** panel with three tabs:

| Tab | What it shows |
| - | - |
| **Overview** | Error rates, web vitals, custom events |
| **Runtime Logs** | Server function console output with level badges |
| **Requests** | HTTP request log with method, path, status, and duration |

Both log tabs auto-refresh every 10 seconds and support filtering by time range and log level.

***

## What's captured

### Request log fields

| Field | Description |
| - | - |
| `timestamp` | When the request was received |
| `method` | HTTP method (GET, POST, etc.) |
| `path` | Request path |
| `status_code` | HTTP response status |
| `duration_ms` | Time to process the request (ms) |
| `user_agent` | Client user agent string |
| `ip` | Client IP address |

### Runtime log fields

| Field | Description |
| - | - |
| `timestamp` | When the log was emitted |
| `level` | `log`, `warn`, `error`, or `info` |
| `message` | The logged message (truncated to 2000 chars) |
| `route_path` | Which server function produced the log |
| `request_id` | Correlates logs to a specific request |

***

## Retention

All log data is retained for **30 days** and automatically purged after that.

***

## Related

* [Server Functions](/docs/canvas/apps/server-functions)
* [Build Logs](/docs/canvas/apps/build-logs)
* [Deploy from Code](/docs/canvas/apps/deploy)


# Monitoring
Source: https://docs.mixpeek.com/docs/canvas/apps/monitoring

Automatic error tracking, web vitals, and custom event reporting for Canvas apps — with optional Sentry and PostHog integration.

## Overview

Canvas automatically captures client-side errors, web performance metrics, and custom events for every app. Data flows to Mixpeek's analytics store and optionally to your own Sentry or PostHog instance.

***

## Built-in error tracking

Canvas injects a lightweight error boundary that captures:

* **Unhandled exceptions** — `window.onerror` and `unhandledrejection` events
* **Error details** — message, stack trace, source URL, line/column numbers
* **Context** — app ID, user agent, page URL

Errors are kept for 30 days and visible in Studio's Analytics panel.

### Querying errors via API

```bash theme={null}
curl "https://api.mixpeek.com/v1/apps/$APP_ID/analytics/errors?minutes=60" \
  -H "Authorization: Bearer $API_KEY"
```

***

## Web vitals

Canvas automatically reports [Core Web Vitals](https://web.dev/vitals/) for every page load:

| Metric | What it measures |
| - | - |
| **LCP** (Largest Contentful Paint) | Loading performance |
| **FID** (First Input Delay) | Interactivity |
| **CLS** (Cumulative Layout Shift) | Visual stability |

Vitals are stored in `canvas_vitals` and visible in Studio.

***

## Custom events

Track custom events from your app using the Canvas analytics API:

```javascript theme={null}
// In your app code
fetch('/_canvas/events', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    event_type: 'search',
    event_name: 'query_executed',
    event_data: JSON.stringify({ query: 'red shoes', results: 42 }),
  }),
})
```

***

## Sentry integration

Send errors to your own Sentry project by configuring the monitoring settings:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "monitoring_config": {
      "sentry_dsn": "https://abc123@o456.ingest.sentry.io/789"
    }
  }'
```

Canvas injects the Sentry SDK automatically — no code changes needed in your app.

***

## PostHog integration

Enable PostHog product analytics:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "monitoring_config": {
      "posthog_api_key": "phc_abc123",
      "posthog_host": "https://app.posthog.com"
    }
  }'
```

Canvas injects the PostHog snippet automatically. Page views, sessions, and custom events are tracked.

***

## Studio

The **Analytics** panel in Studio shows:

* **Error rate** — errors per minute over the selected time range
* **Web vitals** — LCP, FID, CLS distributions
* **Custom events** — event counts by type
* **Runtime logs** — server function console output (see [Runtime Logs](/docs/canvas/apps/logs))
* **Request logs** — HTTP request log with status codes and latency

***

## Related

* [Runtime Logs & Analytics](/docs/canvas/apps/logs)
* [Server Functions](/docs/canvas/apps/server-functions)
* [Apps Overview](/docs/canvas/apps)


# Preview Deploys
Source: https://docs.mixpeek.com/docs/canvas/apps/preview-deploys

Get a unique preview URL for every pull request, automatically built and cleaned up when the PR closes.

## Overview

When you connect a GitHub repo to a Canvas app, every pull request gets its own preview deployment at a unique URL. Reviewers can test changes live before merging — no manual deploy needed.

***

## How it works

1. **Open a PR** targeting your app's connected branch
2. GitHub sends a `pull_request` webhook to Canvas
3. Canvas clones the PR's head branch, builds it, and deploys to a preview environment
4. The preview is live at `https://pr-{N}-{slug}.mxp.co`
5. **Push more commits** to the PR — the preview auto-updates
6. **Close or merge the PR** — the preview environment is cleaned up

***

## Preview URLs

Preview deploys follow a predictable URL pattern:

| PR | App slug | Preview URL |
| - | - | - |
| #42 | `my-app` | `https://pr-42-my-app.mxp.co` |
| #17 | `taste` | `https://pr-17-taste.mxp.co` |
| #103 | `dashboard` | `https://pr-103-dashboard.mxp.co` |

Preview environments are fully functional — they use the same API proxy, auth configuration, and environment variables as your production app.

***

## Setup

Preview deploys work automatically once your repo is connected. No additional configuration is needed.

**1. Connect your repo**

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/connect-repo \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "repo_url": "https://github.com/your-org/your-repo",
    "branch": "main"
  }'
```

**2. Open a pull request**

Open a PR targeting the connected branch (e.g., `main`). Canvas automatically detects the PR and starts building.

**3. Review the preview**

Visit `https://pr-{N}-{slug}.mxp.co` to see the live preview. The URL is also posted as a GitHub deployment status on the PR.

***

## Supported webhook events

| Event | Action | Canvas behavior |
| - | - | - |
| `pull_request` | `opened` | Build and deploy preview |
| `pull_request` | `synchronize` | Rebuild preview with new commits |
| `pull_request` | `reopened` | Rebuild preview |
| `pull_request` | `closed` | Clean up preview environment |

***

## Preview vs staging vs production

| Environment | URL pattern | Use case |
| - | - | - |
| **Production** | `{slug}.mxp.co` | Live site |
| **Staging** | `staging-{slug}.mxp.co` | Manual staging deploys |
| **Preview** | `pr-{N}-{slug}.mxp.co` | Automatic per-PR previews |

Each preview environment is independent — multiple PRs can have active previews simultaneously.

***

## Cleanup

When a PR is closed (merged or not), Canvas automatically:

1. Removes the preview environment from the app's configuration
2. Deletes preview assets from S3

No manual cleanup is needed.

***

## Related

* [Deploy from Code](/docs/canvas/apps/deploy)
* [Build Logs](/docs/canvas/apps/build-logs)
* [GitHub Integration](/docs/canvas/apps/github)


# Server Functions
Source: https://docs.mixpeek.com/docs/canvas/apps/server-functions

Run server-side JavaScript and TypeScript in your Canvas app — access environment variables, key-value storage, and the Mixpeek API without exposing secrets.

## Overview

Server functions let you run backend logic inside your Canvas app. Write TypeScript or JavaScript handlers that execute server-side with access to environment variables, a built-in KV store, and `fetch` — without managing infrastructure.

***

## How it works

1. Include server function source files in your deploy's `source_files` map
2. Files under `api/` become HTTP endpoints at `/functions/`
3. Canvas transpiles TypeScript → JavaScript via esbuild and executes your handler in a sandboxed async context

| Source file | Endpoint |
| - | - |
| `api/hello.ts` | `GET/POST /functions/hello` |
| `api/search.js` | `GET/POST /functions/search` |
| `api/data/export.ts` | `GET/POST /functions/data/export` |

***

## Writing a server function

Export a default function that receives a context object and returns a response:

```typescript theme={null}
// api/hello.ts
export default async function handler(ctx) {
  const name = ctx.req.query.name || 'world'
  return {
    status: 200,
    body: { message: `Hello, ${name}!` },
  }
}
```

### Context object

Your handler receives a `ctx` object with:

| Property | Type | Description |
| - | - | - |
| `ctx.req.method` | `string` | HTTP method (GET, POST, etc.) |
| `ctx.req.path` | `string` | Request path |
| `ctx.req.query` | `object` | URL query parameters |
| `ctx.req.body` | `any` | Parsed JSON body (for POST/PUT) |
| `ctx.req.headers` | `object` | Request headers (excluding internal headers) |
| `ctx.env` | `object` | App environment variables |
| `ctx.kv` | `object` | Key-value store (see below) |

### Return value

Return an object with:

| Field | Type | Default | Description |
| - | - | - | - |
| `status` | `number` | `200` | HTTP status code |
| `body` | `any` | — | Response body (object → JSON, string → text) |
| `headers` | `object` | `{}` | Custom response headers |

***

## Key-value store

Every app has a built-in Redis-backed KV store, accessible in server functions via `ctx.kv`:

```typescript theme={null}
// api/counter.ts
export default async function handler(ctx) {
  const current = await ctx.kv.get('visit_count')
  const count = parseInt(current || '0') + 1
  await ctx.kv.set('visit_count', String(count))

  return {
    body: { visits: count },
  }
}
```

| Method | Signature | Description |
| - | - | - |
| `get` | `kv.get(key: string): Promise<string \| null>` | Read a value |
| `set` | `kv.set(key: string, value: string, ttl?: number): Promise<void>` | Write a value (optional TTL in seconds, default 30 days) |
| `del` | `kv.del(key: string): Promise<void>` | Delete a key |
| `keys` | `kv.keys(): Promise<string[]>` | List all keys for this app |

***

## Calling the Mixpeek API

Use `fetch` to call the Mixpeek API through the Canvas proxy — credentials are injected automatically:

```typescript theme={null}
// api/search.ts
export default async function handler(ctx) {
  const { query } = ctx.req.body

  const res = await fetch('https://api.mixpeek.com/v1/retrievers/ret_abc123/execute', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${ctx.env.MIXPEEK_API_KEY}`,
      'X-Namespace': ctx.env.MIXPEEK_NAMESPACE_ID,
    },
    body: JSON.stringify({ inputs: { query }, settings: { limit: 10 } }),
  })

  return { body: await res.json() }
}
```

***

## Deploying with source files

Include your server functions in the `source_files` field when deploying:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "cli_upload",
    "bundle_s3_key": "$BUNDLE_KEY",
    "environment": "production",
    "message": "Add search API route",
    "source_files": {
      "api/hello.ts": "export default async function handler(ctx) {\n  return { body: { message: \"Hello!\" } }\n}",
      "api/search.ts": "export default async function handler(ctx) {\n  // search logic here\n}"
    }
  }'
```

***

## Runtime logging

`console.log`, `console.warn`, `console.error`, and `console.info` calls in server functions are captured and sent to the [runtime logs](/docs/canvas/apps/logs) system. View them in Studio or query via API.

```typescript theme={null}
// api/process.ts
export default async function handler(ctx) {
  console.log('Processing request', ctx.req.method, ctx.req.path)

  try {
    const result = await doWork(ctx.req.body)
    console.info('Success:', result.id)
    return { body: result }
  } catch (err) {
    console.error('Failed:', err.message)
    return { status: 500, body: { error: err.message } }
  }
}
```

***

## Limitations

* **Execution timeout:** 30 seconds per request
* **Request body size:** 1 MB max
* **No filesystem access:** Server functions run in a sandboxed context
* **No npm imports:** Only `fetch` and the `ctx` APIs are available in the execution context
* **TypeScript only:** `.ts` and `.js` files are supported (transpiled via esbuild)

***

## Related

* [Deploy from Code](/docs/canvas/apps/deploy)
* [Runtime Logs](/docs/canvas/apps/logs)
* [Environment Variables](/docs/canvas/apps/environment-variables)


# User Management
Source: https://docs.mixpeek.com/docs/canvas/apps/users

Manage users for your Canvas app with Clerk organizations. Invite members, assign roles, and control access — all through the API or Studio.

## Overview

Each Canvas app with Clerk authentication gets a dedicated **Clerk organization**. This organization is the single source of truth for who has access to the app and what role they hold.

User management lets you:

* **List members** of a Canvas app
* **Invite users** by email with a specific role
* **Update roles** (admin or member)
* **Remove users** to revoke access immediately
* **Manage invitations** — list pending and revoke unused invitations

> [!NOTE]
>
> User management requires `auth_config.mode` set to `"clerk"`. Apps using `public`, `password`, or other auth modes do not have per-user access control.

***

## Enabling User Management

Set your app's auth mode to `clerk`. Mixpeek automatically provisions a Clerk organization for the app:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

client.apps.update(
    app_id="app_abc123",
    auth_config={"mode": "clerk"},
)
```

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{"auth_config": {"mode": "clerk"}}'
```

Once enabled, the **Users** tab appears in the Studio app detail page.

***

## Roles

Each member has one of two roles:

| Role | Slug | Description |
| - | - | - |
| **Member** | `org:member` | Standard access to the app |
| **Admin** | `org:admin` | Full access including user management |

***

## API Reference

All user management endpoints are scoped to a single app and require your API key with namespace header.

### List Members

Returns all users in the app's Clerk organization.

```bash theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/users \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

**Response:**

```json theme={null}
{
  "users": [
    {
      "user_id": "user_abc123",
      "membership_id": "mem_xyz789",
      "email": "jane@example.com",
      "name": "Jane Smith",
      "avatar_url": "https://img.clerk.com/...",
      "role": "org:admin",
      "joined_at": "2026-03-15T10:30:00Z"
    }
  ],
  "total": 1
}
```

### Invite a User

Send an email invitation to join the app's organization.

```python Python theme={null}
import httpx

resp = httpx.post(
    f"https://api.mixpeek.com/v1/apps/{app_id}/users/invite",
    headers={
        "Authorization": f"Bearer {api_key}",
        "X-Namespace": namespace_id,
    },
    json={
        "email_address": "newuser@example.com",
        "role": "org:member",
    },
)
print(resp.json())
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/users/invite \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "email_address": "newuser@example.com",
    "role": "org:member"
  }'
```

**Response:**

```json theme={null}
{
  "invitation_id": "inv_abc123",
  "email_address": "newuser@example.com",
  "role": "org:member",
  "status": "pending",
  "created_at": "2026-03-27T14:00:00Z"
}
```

### Update a User's Role

Change a member's role between `org:member` and `org:admin`.

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/apps/$APP_ID/users/$USER_ID/role \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{"role": "org:admin"}'
```

### Remove a User

Immediately revokes the user's access to the app.

```bash theme={null}
curl -X DELETE https://api.mixpeek.com/v1/apps/$APP_ID/users/$USER_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

### List Pending Invitations

```bash theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/users/invitations \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

**Response:**

```json theme={null}
{
  "invitations": [
    {
      "invitation_id": "inv_abc123",
      "email_address": "pending@example.com",
      "role": "org:member",
      "status": "pending",
      "created_at": "2026-03-27T14:00:00Z"
    }
  ],
  "total": 1
}
```

### Revoke an Invitation

Cancel a pending invitation before the recipient accepts it.

```bash theme={null}
curl -X DELETE https://api.mixpeek.com/v1/apps/$APP_ID/users/invitations/$INVITATION_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

***

## Studio

When a Canvas app has `auth_config.mode = "clerk"`, a **Users** tab appears in the app detail page. From there you can:

1. **View members** — see all users with their name, email, avatar, and role
2. **Invite users** — click **Invite**, enter an email and role, then send
3. **Change roles** — use the role dropdown next to any member
4. **Remove users** — click the trash icon with a confirmation dialog
5. **Manage invitations** — view pending invitations and revoke them

***

## How It Works

Each Canvas app maps to a **Clerk organization**. When you enable `auth_config.mode = "clerk"`, Mixpeek auto-provisions a Clerk org and stores the `clerk_org_id` in the app record.

All user management operations go directly to Clerk — there is no separate user database to maintain. Clerk is the single source of truth.

```
Your App (auth_config.mode = "clerk")
    └── Clerk Organization (auto-provisioned)
            ├── Members (org:admin, org:member)
            └── Pending Invitations
```

> [!WARNING]
>
> Removing a user revokes access immediately. The user will be signed out on their next request.

***

## Related

* [Authentication](/docs/canvas/apps/authentication) — enable Clerk auth and use the MixpeekAuth SDK
* [Apps overview](/docs/canvas/apps) — Canvas app architecture and quickstart
* [Custom Domains](/docs/canvas/apps/domains) — add your own domain to a Canvas app


# Version History
Source: https://docs.mixpeek.com/docs/canvas/apps/versions

Every change to a Canvas app is versioned with a content hash, commit message, and diffable snapshot — like git for deployed applications.

## Overview

Canvas apps have built-in version control. Every time you **publish** config changes or **deploy** a new code bundle, an immutable version record is created with:

* A **content hash** (SHA-256) — like a git commit SHA
* A **commit message** (required) — describes what changed
* A **full config snapshot** — serialized as diffable JSON files
* **Who** published and **when**

You can list all versions, diff any two versions, download previous bundles, and restore (rollback) to any version instantly.

***

## How versioning works

There are two ways a new version is created:

| Action | What creates the version | What's captured |
| - | - | - |
| **Publish** (`POST /v1/apps/{id}/publish`) | Config snapshot (sections, meta, auth, theme, etc.) | Content hash of config, message, config as JSON files |
| **Deploy** (`POST /v1/apps/{id}/deploy`) | Code bundle upload (React/JS build) | Asset manifest with file hashes, message, optional source files |

Both paths create a `VersionRecord` in the app's `versions` array. The version number auto-increments.

### Content hashing

When you publish, Mixpeek generates a deterministic SHA-256 hash of your entire config snapshot. This hash changes only when the config actually changes — identical publishes produce the same hash.

```
v1  →  hash: d733b11021fa  →  "Initial release"
v2  →  hash: e186b986ff96  →  "feat: add custom HTML search interface"
v3  →  hash: d733b11021fa  →  "revert: back to original config"  (same hash as v1!)
```

### Config as diffable files

Your app config is serialized into individual JSON files for each top-level key. This makes version diffs meaningful:

```
meta.json          →  {"title": "Product Search", "logo_url": "..."}
theme.json         →  {"colors": {"primary": "#FC5185"}}
sections.json      →  [{"type": "search", ...}]
auth_config.json   →  {"mode": "clerk", "clerk_allowed_providers": [...]}
custom_html.json   →  "<div id=\"app\">...</div>"
```

***

## Listing versions

```bash cURL theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/versions \
  -H "Authorization: Bearer $API_KEY"
```

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")
versions = client.apps.versions.list(app_id="app_abc123")

for v in versions.versions:
    print(f"v{v.version}: {v.s3_version_id} — {v.message}")
```

Response:

```json theme={null}
{
  "versions": [
    {
      "version": 2,
      "s3_version_id": "e186b986ff96",
      "message": "feat: add custom HTML search interface",
      "deployed_by": "int_40ed22c147907235",
      "deployed_at": "2026-03-27T13:10:00Z",
      "environment": "production"
    },
    {
      "version": 1,
      "s3_version_id": "d733b11021fa",
      "message": "Initial release",
      "deployed_by": "int_40ed22c147907235",
      "deployed_at": "2026-03-27T13:09:00Z",
      "environment": "production"
    }
  ],
  "total": 2
}
```

***

## Viewing version details

Get full metadata for a specific version, including the source files snapshot and which environments it's active in:

```bash theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/versions/1 \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
{
  "version": 1,
  "s3_version_id": "d733b11021fa",
  "message": "Initial release",
  "deployed_by": "int_40ed22c147907235",
  "deployed_at": "2026-03-27T13:09:00Z",
  "environment": "production",
  "source_files": {
    "meta.json": "{\"title\": \"Product Search\"}",
    "theme.json": "{\"colors\": {\"primary\": \"#FC5185\"}}",
    "sections.json": "[]"
  },
  "is_active": {
    "staging": false,
    "production": true
  }
}
```

***

## Diffing versions

Compare any two versions to see what changed — like `git diff v1..v2`:

```bash theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/versions/1/diff/2 \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
{
  "app_id": "app_abc123",
  "from_version": 1,
  "to_version": 2,
  "summary": {
    "added": 1,
    "removed": 0,
    "modified": 0,
    "unchanged": 7
  },
  "source_diff": {
    "custom_html.json": "--- v1/custom_html.json\n+++ v2/custom_html.json\n@@ -0,0 +1 @@\n+\"<div><h1>Custom Search</h1></div>\""
  }
}
```

The `summary` counts file-level changes (based on content hashes). The `source_diff` provides unified diffs of the actual content — the same format as `git diff`.

***

## Rollback

### Quick rollback (one level)

Restore the previous published config instantly:

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/rollback \
  -H "Authorization: Bearer $API_KEY"
```

```python Python theme={null}
app = client.apps.rollback(app_id="app_abc123")
print(f"Rolled back to v{app.version}")
```

### Restore any version

For deploy-based versions (code bundles), restore any specific version to any environment:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/versions/3/restore \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment": "production"}'
```

This is instant — no rebuild required. The previous assets are always retained in S3.

***

## Download a version's bundle

Download the original zip bundle for any deployed version and work locally:

**1. Download the bundle**

```bash theme={null}
curl https://api.mixpeek.com/v1/apps/$APP_ID/versions/2/download \
  -H "Authorization: Bearer $API_KEY"
```

Response:

```json theme={null}
{
  "download_url": "https://s3.amazonaws.com/...",
  "version": 2,
  "asset_prefix": "apps/my-app/dep_abc123",
  "expires_in": 3600
}
```

**2. Download and extract**

```bash theme={null}
curl -o bundle.zip "$DOWNLOAD_URL"
unzip bundle.zip -d my-app/
```

**3. Edit files locally**

Make your changes to the extracted source.

**4. Re-zip and deploy**

```bash theme={null}
cd my-app && zip -r ../updated.zip .
# Upload and deploy as usual (see Deploy from Code)
```

***

## Git metadata

When deploying via CI/CD or the CLI, you can attach git metadata to each version for full traceability:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/apps/$APP_ID/deploy \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "cli_upload",
    "bundle_s3_key": "$BUNDLE_KEY",
    "message": "fix: search results pagination",
    "git_commit_sha": "a1b2c3d4e5f6",
    "git_commit_message": "fix: search results pagination",
    "git_author": "Jane Smith <jane@example.com>"
  }'
```

Git metadata appears in version detail responses and Studio's version history panel.

> [!TIP]
>
> If you connect a GitHub repository via `POST /v1/apps/{id}/connect-repo`, deploys are triggered automatically on push. Git metadata is captured from the webhook payload — no manual fields needed.

***

## Version record fields

| Field | Type | Description |
| - | - | - |
| `version` | `int` | Auto-incrementing version number |
| `s3_version_id` | `string` | Content hash (publish) or S3 version ID (deploy) |
| `asset_prefix` | `string` | S3 path prefix for deployed assets |
| `asset_manifest` | `object` | Map of `{relative_path: {s3_key, hash, size}}` |
| `source_files` | `object` | Map of `{path: content}` for source-level diffs |
| `deployed_by` | `string` | Internal ID of the user who published/deployed |
| `deployed_at` | `string` | ISO 8601 timestamp |
| `environment` | `string` | `"staging"` or `"production"` |
| `message` | `string` | Commit message (required) |
| `build_duration_ms` | `int` | Build time in milliseconds (deploy only) |
| `git_commit_sha` | `string` | Git SHA (if provided) |
| `git_commit_message` | `string` | Git commit message (if provided) |
| `git_author` | `string` | Git author (if provided) |

***

## Related

* [Apps overview](/docs/canvas/apps)
* [Deploy from Code](/docs/canvas/apps/deploy)
* [List Versions (API)](/docs/api-reference/apps/list-versions)
* [Publish App (API)](/docs/api-reference/apps/publish-app)
* [Rollback App (API)](/docs/api-reference/apps/rollback-app)


# Supported environments
Source: https://docs.mixpeek.com/docs/customer-hosted/compatibility

The Kubernetes, GPU, storage and KubeRay configurations Customer-Hosted Mixpeek supports

Customer-Hosted Mixpeek supports qualified Kubernetes environments. The list is
deliberately narrow and grows when a deployment asks it to, because every entry
here is something we can run a qualification against and stand behind.

"Runs on any Kubernetes" is a claim nobody can test, so we do not make it.

## What a deployment requires

| | Requirement |
| - | - |
| **Kubernetes** | EKS, GKE |
| **Compute** | NVIDIA GPU nodes at a known driver baseline |
| **Storage** | S3 or GCS for objects, plus approved CSI storage classes for persistent volumes |
| **KubeRay** | operator versions Mixpeek qualifies (see the hard floor below) |
| **Install** | the Mixpeek Helm chart |

## Qualification status

The table above is the configuration a deployment needs. This one is what a
qualification establishes per environment.

| Environment | Status |
| - | - |
| **GKE** | Mixpeek's own production runs here, and the qualification fixture builds a GKE cluster from scratch |
| **EKS** | Supported. A deployment starts with a qualification run against your cluster, which is what establishes the configuration a general table cannot |
| **Other distributions** | Not qualified. Tell us what you run and we will scope the qualification |

We keep these separate on purpose. A row here describes how a deployment on that
distribution is established, which is a more useful answer than a version number.

## Structural constraints

Two requirements are properties of the software rather than matrix results, so
they hold everywhere:

| | Requirement | Why |
| - | - | - |
| **KubeRay** | **≥ 1.1**, hard | `submissionMode: HTTPMode` does not exist below it, so batch cannot be submitted at all |
| **Kubernetes** | **≥ 1.33** optional | Enables in-place vertical scaling (`pods/resize`) for Ray's autoscaler. On older clusters the rule is inert and Ray replaces pods to resize them |

## Your Ray version is not an input

Mixpeek ships its own Ray, Python and CUDA userspace in one image, and the
`RayService` it runs is ours. Whatever Ray you run elsewhere in the cluster has no
bearing on whether Mixpeek works.

That changes only for the
[shared-cluster integration](/docs/customer-hosted/existing-raycluster), which is scoped
per cluster. There the cluster is yours, so its runtime is an input, and that page
says what it costs.

## Getting a direct answer for your cluster

Ask Mixpeek to run the [qualification suite](/docs/customer-hosted/qualification)
against it. The report names the Kubernetes, KubeRay and storage configuration it
found and whether each check passed on them. That is an answer about your cluster,
which is worth more than a general claim about a matrix.

> [!NOTE]
>
> Compatibility results come from qualification runs rather than from a
> hand-maintained table, which is why this page sends you to a run against your own
> cluster instead of a list of version numbers.


# Configuration and secrets
Source: https://docs.mixpeek.com/docs/customer-hosted/configuration

The Secret the chart expects you to create, and the values only you can supply

The chart does **not** create any Secret. Every workload it renders reads its
configuration from one you create yourself:

```yaml theme={null}
envFrom:
  - secretRef:
      name: mixpeek-engine-secrets
```

Create that Secret in the same namespace before you install. If it is missing the
pods will not start, which is the safe failure: you get a clear event and nothing
runs against the wrong configuration.

## The values only you can supply

These five name infrastructure. Each one has a built-in default that points at
**Mixpeek's** own buckets and projects, so leaving one out does not fail loudly.
The pods start and that component talks to a resource you do not own.

| Key | What it is | If you omit it |
| - | - | - |
| `AWS_BUCKET` | Bucket holding your source objects and extraction artifacts | Requests use a Mixpeek bucket name against **your** endpoint, so they fail with a `NoSuchBucket` naming a bucket you have never created |
| `AWS_PUBLIC_BUCKET` | Bucket for publicly addressable assets | Same shape as above |
| `MONGODB_API_DB` | Database name inside your MongoDB | Your data is written to a database named for Mixpeek's dev environment |
| `GOOGLE_BUCKET_NAME` | GCS bucket, only if you enable Google-backed extractors | Uploads target a Mixpeek bucket name |
| `GEMINI_PROJECT_ID` | GCP project for Gemini inference, only if you enable it | Inference is billed against a Mixpeek project id, which your credentials cannot open |
| `GCS_STORAGE_BUCKET` | GCS bucket for processed artifacts, only with Google-backed storage | Writes target a Mixpeek bucket name |
| `S3_VECTORS_BUCKET_NAME` | Bucket backing the S3 vector store, if you use that backend | Vector writes target a Mixpeek bucket name |
| `S3_VECTORS_LOCAL_BUCKET` | Local-mode counterpart of the above | Same |
| `LANGSMITH_PROJECT` | LangSmith tracing project, only if you enable tracing | Traces are attributed to a Mixpeek project |
| `ENGINE_HEAD_SVC` | In-cluster address of the Ray head's Serve service | Defaults to a hostname in **Mixpeek's** namespace, which does not resolve in yours. The chart derives this for you when it installs the engine, so set it only if you run the engine separately. |

Your credentials will not open a Mixpeek-owned resource, so the practical outcome
is a permission or not-found error rather than data crossing between tenants. The
problem is that the error names **our** resource and gives you nothing to act on.

## Connecting to your own object store

| Key | Notes |
| - | - |
| `AWS_ENDPOINT_URL` | Set this to your own S3-compatible endpoint. Leave it unset only if you are using AWS S3 itself. |
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | Credentials for that endpoint |
| `AWS_REGION` | Region string your store expects |

## Everything else

The Secret Mixpeek runs internally carries around 120 keys. Most are
integrations the runtime does not need in your environment: payments, support
desks, analytics, social connectors and error reporting. Set only what you use.

Most keys you leave out stay unset, and the feature that reads them stays off.
`SENTRY_DSN` is the exception, and it is the one to read the next section for.

## What leaves your environment

Short answer: error reports, unless you turn them off. Nothing else.

**`SENTRY_DSN` has a non-empty default that points at Mixpeek's Sentry project.**
It is not part of the chart, so it appears in no manifest you can inspect, and
leaving it unset does not disable it. The runtime only skips error reporting when
the DSN is empty or `ENV` is `local`, and a deployed install is neither. Exceptions
raised in your environment, with their stack traces and the context Sentry attaches,
are sent to us.

To stop that, set it explicitly to an empty value:

```yaml theme={null}
SENTRY_DSN: ""
```

Point it at your own Sentry project instead if you want the reports and want them to
stay yours.

> [!NOTE]
>
> This default is being reconsidered so that error reporting is off unless you opt in.
> Until that ships, setting `SENTRY_DSN: ""` is what turns it off, and this page will
> say so plainly for as long as it is true.

A fuller version of this, with per-model sizes and the source line for each claim,
is in the Customer-Hosted control-plane boundary write-up your account team can share.

### What does not leave

Checked the same way, so this list means something:

| Channel | Behaviour with the shipped configuration |
| - | - |
| Product analytics (PostHog) | Off. The client is a no-op without an API key, and the key has no default. |
| Mixpeek control plane | Off. Every `CONTROL_PLANE_*` setting defaults to unset, and the request-forwarding prefix does nothing without a control-plane URL. |
| Your objects and documents | Never leave. They are read from and written to the endpoints you configure above. |
| Model weights | Two model sets are baked into the engine image. Four more are downloaded from `huggingface.co` the first time a replica loads them, so this path is outbound but carries nothing of yours. An environment with no route to `huggingface.co` starts Ray healthy and then leaves those applications unable to load. |

`MIXPEEK_PRIVATE_TOKEN` is worth calling out: the API and the engine authenticate
to each other with it, so set it to a value you generate, and use the same value
for both.


# End-to-End Data Flow
Source: https://docs.mixpeek.com/docs/customer-hosted/data-flow

One object, from arrival to a moderation decision: what runs where, what it costs, and what leaves your account.

One object entering the system, through every layer, to a decision and a queryable index.
Written extractor-agnostic: the extractor is a variable, and this page shows where that
variable is resolved and what it changes.

The primary walk is customer-hosted (BYOC), because that is the shape where "what runs
where" matters. Mixpeek Cloud is the same pipeline with every plane in Mixpeek's account.

![End-to-end data flow: an object arrives by presigned PUT or sync, becomes a bucket object, joins a batch, is scheduled to a Ray profile, executes on an ephemeral RayJob calling long-lived Ray Serve models, persists through MVS to object storage, is enriched by taxonomies, and produces a pass, flag or hold decision](https://mintcdn.com/mixpeek/HWi1bY8aTG_9gk9z/assets/mixpeek-data-flow.svg?fit=max\&auto=format\&n=HWi1bY8aTG_9gk9z\&q=85\&s=08a63e6053989ce9d22e7502633a4605)

> **Same diagram as Mermaid**
>
> ```mermaid theme={null}
> flowchart TB
>   A["① ARRIVAL<br/>presigned PUT or sync connector"] --> B["② OBJECT<br/>bucket row + blobs + schema validation"]
>   B --> C["③ BATCH<br/>work unit over N objects, deduped"]
>   C --> D["④ SCHEDULE<br/>work scheduler → profile resolve → sizing"]
>   D --> E["⑤ EXECUTE<br/>ephemeral RayJob, tier 0..N"]
>   E -->|"model calls"| S["Ray Serve<br/>long-lived, autoscaled"]
>   E --> F["⑥ PERSIST<br/>SyncVectorWriter → MVS → WAL → object storage"]
>   F --> G["⑦ ENRICH / CLASSIFY<br/>taxonomy · clusters"]
>   G --> H["⑧ DECIDE<br/>retriever + threshold + trigger"]
>   F --> I["QUERY<br/>retriever stages"]
>   H -.->|"pass / flag / hold"| OUT["your destination<br/>object storage or webhook"]
> ```

> **Same system as a UML deployment diagram**
>
> Nodes are execution environments and artifacts are what runs on them, so this view
> answers "what is deployed where" rather than "what happens next".
>
> ![UML deployment diagram: a Mixpeek account node holding the API gateway, a customer Kubernetes cluster node holding Mixpeek services, Ray on KubeRay and stateful services, a customer object storage node holding source objects, sealed WAL and MVS snapshots, and a hosted LLM provider node outside the customer VPC](https://mintcdn.com/mixpeek/HWi1bY8aTG_9gk9z/assets/mixpeek-data-flow-uml.svg?fit=max\&auto=format\&n=HWi1bY8aTG_9gk9z\&q=85\&s=e79796635b7e75a8ffb1aa3a965718a4)
>
> PlantUML source, so the next editor regenerates rather than redraws:
>
> ```plantuml theme={null}
> @startuml
> node "Mixpeek account" as MXP {
>   artifact "API gateway · auth · billing · plan gates" as GW
> }
> node "Your Kubernetes cluster" as K8S {
>   node "Mixpeek services" as SVC {
>     artifact "API pods" as API
>     artifact "Work runtime · ledger in Mongo, runs on Ray" as WORK
>     artifact "Redis · locks, caches" as REDIS
>   }
>   node "Ray on KubeRay" as RAY {
>     artifact "RayJob · one per batch\nephemeral · holds no models" as JOB
>     artifact "RayService · Serve\nlong-lived · holds the models" as SERVE
>   }
>   node "Stateful services" as STATE {
>     artifact "MVS shards · Rust" as MVS
>     artifact "MongoDB · metadata and analytics" as MONGO
>   }
> }
> node "Your object storage" as OS {
>   artifact "source objects" as OBJ
>   artifact "sealed WAL" as WAL
>   artifact "MVS snapshots" as SNAP
> }
> node "Hosted LLM provider" as LLM {
>   artifact "OpenAI · Anthropic · Vertex" as PROV
> }
> GW ..> API : "https (managed control plane only)"
> JOB ..> OBJ : "reads blobs"
> JOB ..> SERVE : "inference RPC"
> MVS ..> WAL : "ships sealed WAL"
> MVS ..> SNAP : "ships snapshots"
> SERVE ..> PROV : "egress, only if a stage uses a hosted model"
> @enduml
> ```

## The boundary, first

BYOC splits on one line: you provide the substrate, Mixpeek provides and operates the
software.

> [!WARNING]
>
> **Where the control plane sits is an install-time choice.** The diagrams above draw the
> managed shape.
>
> Six collections are control-plane by definition: API keys, organizations, users, tenant
> routing, the Canvas app registry, and an internal repair queue. On a **managed** install
> those resolve to a Mixpeek-operated database and your namespace egresses to it, which is
> the egress the install kit's network policy opens. On a **fully self-hosted** install the
> control-plane database is left unset, the same six resolve to the database in your own
> namespace, and no pipeline stage needs outbound to Mixpeek.
>
> The eight stages below are identical in both shapes. Two things outside the pipeline
> differ, and both are worth knowing before you design around self-hosted:
>
> * **Canvas apps.** The build worker, the bundle store and the app runtime all operate on
>   one shared registry. A registry sitting in your own namespace is not reachable by them,
>   so Canvas apps are unavailable on that shape.
> * **Usage reporting.** Your organization record and your usage records both sit in your
>   namespace, so nothing reports consumption back to Mixpeek. Metering becomes a
>   commercial arrangement rather than something the platform does for you.
>
> Which shape you get is set at install time, so tell us which one your environment
> requires and it is configured that way from the first deploy.

The eight stages below each answer the same four questions: input, work, output, resources.

## Stage by stage

### ① Arrival

| | |
| - | - |
| **Input** | Bytes, from one of two doors |
| **Work** | Door A: `POST /v1/buckets/{id}/uploads` returns `upload_id` and `presigned_url`; the client PUTs straight to object storage, so bytes never transit the API. Door B: a sync pulls from a source you already have. |
| **Output** | An object in your bucket, at the storage class the bucket's `storage_tier` names |
| **Resources** | Object storage write. No compute. No Mixpeek pod touches the payload. |
| **Where** | Your cloud. The API only mints and validates the URL. |

Sync providers today: `google_drive`, `s3`, `snowflake`, `sharepoint`, `tigris`, `iconik`.
Modes are `initial_only` or `continuous` with a `polling_interval_seconds`. **Continuous
sync is polling, not an event subscription**, which sets your latency expectation for "a
video arrives".

### ② Object

| | |
| - | - |
| **Input** | The stored bytes plus caller metadata |
| **Work** | Create the bucket row: `object_id`, `key_prefix`, `blobs[]`, `source_details[]`. Declared fields are type-checked against `bucket_schema` on write; undeclared fields pass through. Unique-key policy dedupes here. |
| **Output** | An object record in Mongo, blobs referencing storage locations |
| **Resources** | API pod CPU (small), one Mongo write, object-storage metadata read |
| **Where** | Mixpeek software in your namespace; Mongo in your data plane |

**A video is one object with blobs, not many objects.** Decomposition into chunks, frames
and scenes happens in stage ⑤, which is what makes tiers necessary.

### ③ Batch

| | |
| - | - |
| **Input** | `object_ids[]` (or a bucket-wide selector), `dedup_strategy` |
| **Work** | Plan gates check `max_batch_size` and `max_active_batches`; the cost estimator prices it; the ledger excludes objects already extracted at the same content and config hash. |
| **Output** | A batch record with one task per processing tier |
| **Resources** | API pod CPU, Mongo write, credit ledger read |
| **Where** | Control-plane logic in your namespace; billing meters back to Mixpeek |

`dedup_strategy` is the cost-multiplying knob: a re-run with the wrong value re-pays full
extraction. **Tier 0 is bucket to collection; tier N above 0 is collection to collection.**
A pipeline that turns a video into chunks, chunks into frames, and frames into embeddings is
three tiers, each independently tracked and retryable.

### ④ Schedule

This is where the extractor stops being a name and becomes a machine.

| | |
| - | - |
| **Input** | The collection's `feature_extractor`, the batch's object count, whether the namespace is on shared or dedicated tenancy |
| **Work** | Queue a work task, resolve a GKE compute profile, compute worker counts |
| **Output** | A RayJob spec: head plus CPU workers plus optional GPU workers, with requests, limits and replica bounds |
| **Resources** | A row in the work ledger (Mongo); Ray runs the unit; Redis holds locks |
| **Where** | Your namespace |

Profile resolution is a six-rule ladder:

| # | Rule | Result |
| - | - | - |
| 1 | explicit `profile_name` | that profile |
| 2 | extractor declares `gke_profile` | that profile |
| 3 | plugin job and not `requires_gpu` | `plugin`, hard-isolated on a tainted node |
| 4 | not `requires_gpu` | `cpu-only`, skips the GPU pool |
| 5 | job-type default | batch → `default` · cluster → `small` · benchmark → `small` |
| 6 | fallback | `default`, which carries a GPU worker group |

**Only 6 of 12 built-in extractors declare a profile.** For the rest, `requires_gpu` decides
the hardware, not anything named in the definition. Choosing an extractor chooses a node
pool and therefore a bill.

Sizing is `object_count ÷ objects_per_worker → replicas`, clamped by the tenancy slice and a
350Gi ceiling. Shared tenancy gets gpu 2 / cpu 32 / api 4; a dedicated GPU tenant rises to
gpu 8 / api 12. Tenancy is decided by which tenant manifest deploys the namespace, not by
a field the namespace's own configuration carries.

### ⑤ Execute

| | |
| - | - |
| **Input** | RayJob spec, object references, extractor config |
| **Work** | KubeRay materializes a per-batch cluster; workers read blobs from object storage, decompose, call models, emit documents |
| **Output** | Document payloads with vectors and `_internal` lineage, in memory |
| **Resources** | Head pod plus N CPU workers plus optional GPU workers; the node autoscaler provisions from your pools |
| **Where** | Your Kubernetes, your nodes, under your taints and quotas. A model-backed extractor is the exception, and it sends the media itself: see [What leaves your account](#what-leaves-your-account) |

```mermaid theme={null}
flowchart LR
  subgraph EPH["Ephemeral RayJob · dies with the batch"]
    HEAD["head<br/>1 cpu / 4–12Gi"]
    CW["cpu workers<br/>N × 1 cpu / 2–6Gi"]
    GW["gpu workers<br/>0–4 × 2 cpu / 12–32Gi + 1×GPU<br/>(only if the profile has the group)"]
  end
  subgraph LIVE["RayService · long-lived, autoscaled, shared"]
    SERVE["Serve deployments<br/>embedders · rerankers · VLM adapters"]
  end
  OS[("object storage")] -->|"read blobs"| CW
  CW -->|"inference RPC"| SERVE
  GW -->|"inference RPC"| SERVE
  CW --> OUT["documents"]
  GW --> OUT
```

**Two Ray shapes, and confusing them is the usual mistake.** The RayService is long-lived
and holds the models. The RayJob is ephemeral, one per batch, and holds no models at all for
Serve-backed extractors. A `cpu-only` worker's real peak is about 0.27 core and 0.77 GiB,
which is why its requests are 1 cpu / 2Gi and why one node holds seven or eight concurrent
batches instead of three.

Two mechanics worth carrying:

* **Kubernetes `cpu_req` is not Ray `num-cpus`.** `cpu_workers.ray_num_cpus` decouples them,
  so a worker can advertise 2 schedulable CPU to Ray for placement groups while requesting 1
  from Kubernetes. Without it, multi-step streaming placement cannot schedule and the job
  orphans.
* **Requests drive the autoscaler; limits are the OOM gate.** Limits stay generous so a
  decode or frame burst is absorbed rather than OOMKilled.

### ⑥ Persist

| | |
| - | - |
| **Input** | Document payloads from the workers |
| **Work** | Writer → MVS coordinator → primary shard gRPC → WAL append → staging buffer → index flush → PQ train → seal WAL → ship to object storage |
| **Output** | Queryable points; metadata in Mongo; usage events to Mongo's analytics time-series collections |
| **Resources** | MVS pod 2–4 CPU / 4–64 GiB; object-storage writes for WAL and snapshots |
| **Where** | Your Kubernetes, your disks, your object storage |

```mermaid theme={null}
flowchart LR
  W["Ray worker<br/>SyncVectorWriter"] --> C["MVS coordinator<br/>(Python, per namespace)"]
  C -->|"UpsertDocuments gRPC :50051"| P["primary shard (Rust)"]
  P --> WAL["WAL append<br/>local disk"]
  P --> STG["staging buffer"]
  STG -->|"≥1000 or interval"| LIRE["LIRE index flush"]
  LIRE -->|"partition ≥ 50k vectors"| PQ["train PQ 32×8"]
  WAL -->|"seal + ship"| OBJ[("object storage<br/>sealed WAL + snapshot")]
  OBJ -->|"cold-start restore ~seconds"| P
  C -->|"SearchDense reads"| R["replica shard(s)"]
  OBJ -.->|"restore"| R
```

The knobs that decide cost and recall:

| Knob | Default | Trades |
| - | - | - |
| `MVS_NPROBE` | 8 | partitions searched per query: the recall against latency dial |
| `pq_subvectors / pq_bits` | 32 / 8 | \~32× memory reduction; PQ serves the hot path, raw vectors retained for reconstruction |
| `pq_training_threshold` | 50,000 | below it, compression buys nothing |
| `max_vectors` per shard | 50,000,000 | split point: more shards, more fan-out |
| `MVS_IDLE_EVICT_MINUTES` | 30 | idle namespaces evict and restore on demand. This is what makes the cost math work: one pod amortized across hundreds of idle namespaces |
| `MVS_WAL_MAX_SHIP_AGE_SECS` | 30 | bounds RPO. A sealed WAL segment ships to object storage within this many seconds, so worst-case loss is the writes acknowledged in the last 30 seconds |

**Durability, stated plainly:** the recovery source is object storage. A rescheduled primary
replays the shipped WAL over its last snapshot, so the vector store's RPO is 30 seconds, the
WAL ship age. Metadata in MongoDB is recoverable to any point inside its 10-minute oplog
capture window (RPO 600 seconds), and versioned object storage loses nothing (RPO 0). A
namespace restore is held to 30 minutes (RTO 1800 seconds). Replicas serve reads and do not
shorten the vector store's gap.

> [!NOTE]
>
> Replica counts vary by namespace, and a namespace under maintenance can run a shape that
> does not match the steady state above. Treat this section as the design, and read the
> actual pod counts in your own namespace before depending on them.

### ⑦ Enrich and classify

| | |
| - | - |
| **Input** | Persisted documents |
| **Work** | Taxonomy enrichment (flat or hierarchical) writes labels back onto documents; clustering runs group them |
| **Output** | Documents carrying classification fields |
| **Resources** | A Serve `taxonomy_join` deployment, or LLM providers for label generation; clustering runs as its own RayJob on the `small` profile |
| **Where** | Your namespace. LLM-based hierarchy inference egresses; flat enrichment does not. See [What leaves your account](#what-leaves-your-account) |

Flat taxonomy enrichment calls no language model. It matches against a reference collection
and writes labels back. Hierarchy inference has three methods, and only the LLM-based one
calls a provider.

Taxonomy is a small part of the egress picture, and the section below is the whole of it.

### ⑧ Decide

**This stage composes primitives; there is no moderation resource in the API.**

| | |
| - | - |
| **Input** | Enriched documents |
| **Work** | A retriever with a `score_threshold` reduce stage, or a taxonomy assignment read, produces a verdict. A trigger (`cron`, `interval`, `event`, `conditional`) acts on it. |
| **Output** | A pass, flag or hold decision, and whatever the trigger's action writes |
| **Resources** | MVS query CPU; reranker Serve replicas if the retriever sorts; LLM tokens if it enriches |
| **Where** | Your namespace, plus any external provider a stage calls |

> [!WARNING]
>
> The platform computes the verdict and fires the trigger, but writing that decision back out
> as a destination object with lineage is the weak link. If moderation must end with "a file
> lands in a `reviewed/` bucket", treat that as integration work today rather than a
> primitive.

## What leaves your account

Egress is not a property of one stage. It is a property of which extractors and which
retriever stages a pipeline is configured with, and it exists at **both ingest and query
time**. Enumerated from the call sites rather than characterised, because every attempt to
describe this as a category has been wrong:

**At ingest**, a model-backed extractor sends the content itself. `gemini_multifile_extractor`
builds Gemini parts with `Part.from_bytes(data=content, mime_type=...)`, so for an image or a
PDF the **media** goes to the provider, not a label derived from it. The same holds for
`multimodal_extractor` v2 Gemini embedding, `universal_extractor`, `scrolling_text_extractor`,
`document_graph_extractor`'s VLM corrector, and `web_scraper`'s semantic router. Cluster
labelling calls a model too.

**At query time**, seven retriever stages call providers: `enrich/llm`, `enrich/agentic`,
`filter/llm`, `filter/agent_search`, `filter/attribute`, `filter/query_expand` and
`reduce/summarize`. The `enrich` family is the volume path, once per result, so an enrich
stage over 100 results is 100 calls.

**Those same seven also run at ingest**, whenever a taxonomy or a collection's
`retriever_enrichments` points at a retriever. That path fires once per *document* rather
than once per result, which makes it the larger volume of the two on a bulk load. Reading
the collection will not tell you, because its config carries only a `retriever_id`; you
have to follow the reference to the retriever and read its stages.

**Which of these are in your pipeline is configuration.** A pipeline built on self-hosted
models in your own Serve deployments makes none of these calls. One built on
`gemini_multifile_extractor` sends every object's bytes to Google at ingest. Both are
supported, and the difference is a choice you make per collection and per retriever rather
than a property of the platform.

Three things change the answer, and all three are available today:

* **Your own provider credentials.** Organization-default LLM credentials resolve per request,
  so these calls can run on your OpenAI, Anthropic or Vertex account. The egress becomes your
  contract with the provider rather than ours.
* **Self-hosted models.** Anything served from your own Ray Serve deployments stays inside
  your VPC end to end.
* **Replace the path.** A custom plugin can take over the enrich batch entirely, so the
  built-in provider call is never made.

> [!WARNING]
>
> If a reviewer needs the exhaustive list for a given namespace, derive it from that
> namespace's own configuration rather than from this page. Ask us to run the namespace
> egress check against it: the check reads your collections, taxonomies, clusters and
> retrievers, names every configured component that reaches a provider, and refuses to
> report a clean result when it hits a reference it could not follow. The set above grows
> whenever an extractor or a stage is added, so a page cannot stay exhaustive.

### Model weights are a different kind of egress

No customer content leaves on a self-hosted-model pipeline, which is the commitment
above. The **weights** do arrive from outside, carrying nothing of yours, from
HuggingFace and similar hosts.

Two of the six model sets are baked into the engine image and need no network at
all. The other four are fetched **the first time each replica loads them**, not at
image build:

| Model | Size | When |
| - | - | - |
| ArcFace (`buffalo_l`) | 601 MB | baked into the image |
| `multilingual-e5-large-instruct` | 1.1 GB | baked into the image |
| `bge-reranker-v2-m3` | 2.2 GB | first replica load |
| `siglip-base-patch16-224` | 779 MB | first replica load |
| `all-MiniLM-L6-v2` | 88 MB | first replica load |
| CLAP `630k-audioset-best.pt` | 1.86 GB | first audio object |

It matters operationally rather than for a privacy review, and the failure is easy
to misread. A namespace with no outbound to the public internet brings the Ray
cluster up healthy and then leaves those four applications unable to load, which
reads as a slow install for as long as it takes someone to look at the Serve status.

Decide it before the first deploy. Either allow egress to `huggingface.co` on first
start, or ask us for an image with all six baked.

## Query path

Retrieval reuses the index the ingest path built.

| Stage family | Executes against | Cost driver |
| - | - | - |
| **filter** | MVS gRPC (vector and payload pushdown); `llm` and `query_expand` hit the LLM service | MVS CPU × `nprobe`; LLM tokens |
| **sort** | `rerank` goes to a Ray Serve reranker; the rest run in-process | reranker replica time |
| **reduce** | database-level aggregation; `cluster` goes to Serve; `summarize` to an LLM | mostly free; summarize is tokens |
| **enrich** | LLM providers, `taxonomy_join` Serve, sandbox for code | **per result**: an enrich stage on 100 results is 100 LLM calls |
| **apply** | external HTTP, your SQL, web search | third-party limits and egress |

Read mechanics: coordinator → `SearchDense` RPC (fan-out across the hash ring for
multi-shard namespaces) → shard probes `nprobe` partitions via PQ codes → re-ranks top-N
against raw vectors → returns point id, score and payload.

**Filter predicates are evaluated coordinator-side, after the shard returns.** That is known
waste, and it is what compute pushdown is for.

## Resource map

| Stage | Compute | Storage touched | State | Scales with |
| - | - | - | - | - |
| ① Arrival | none (presigned) or sync worker | object storage write | none | upload concurrency |
| ② Object | API pod, small | Mongo write | Mongo | objects/sec |
| ③ Batch | API pod, small | Mongo write | Mongo, credit ledger | batch creation rate |
| ④ Schedule | work ledger row | Mongo (ledger), Redis (locks) | Mongo | active batches |
| ⑤ Execute | RayJob head plus N workers, optionally GPU | object storage read | ephemeral | `object_count ÷ objects_per_worker` |
| ⑤b Inference | RayService Serve replicas, autoscaled | model cache | long-lived | concurrent inference RPS |
| ⑥ Persist | MVS pod 2–4 CPU / 4–64 GiB | WAL local disk plus object storage | **the durable one** | vectors, partitions |
| ⑦ Enrich | Serve replicas and/or LLM providers | Mongo, MVS | documents | documents × labels |
| ⑧ Decide | MVS query plus optional Serve or LLM | reads only | none | verdicts/sec |
| Query | MVS plus Serve plus providers | reads only | none | QPS × `nprobe` × stages |

| Store | Role | Loss impact |
| - | - | - |
| **Object storage** | bucket objects, MVS snapshots and sealed WAL, exports | catastrophic: it is the durability floor |
| **MVS** | vectors plus the `_internal` payload envelope | rebuildable from source objects, expensively |
| **Mongo** | resource metadata, batch and tier state, non-vector document fields | control-plane truth |
| **Redis** | distributed locks, caches, heartbeats | in-flight work, not durable state |
| **Mongo (time-series collections)** | usage and analytics events | billing and analytics history |

## Where it breaks

Back-pressure and failure points, in the order they bite:

| Point | Symptom | Absorbed by |
| - | - | - |
| Plan gate | batch refused at creation | `max_batch_size`, `max_active_batches` |
| Tenancy clamp | fewer workers than sizing asked for | shared slice (gpu 2 / cpu 32 / api 4) |
| Node autoscaler | RayJob pods pend | your pool max and quota, your policy rather than ours |
| Spot preemption | worker dies mid-batch | Ray retry; drivers pinned to on-demand in dedicated tenancy |
| Serve saturation | inference RPC latency climbs | Serve autoscaling within replica bounds |
| MVS staging | write latency spikes at flush | staging threshold 1000, flush interval |
| MVS eviction | first query after idle is slow | 30-minute idle evict, restore on demand |
| Tier failure | one tier fails, the batch continues | per-tier tracking, retry without reprocessing successful tiers |

One correctness note, because it is invisible otherwise: a batch that finishes with a failed
tier reports `COMPLETED_WITH_ERRORS`, not `COMPLETED`. **If you build a gate that polls batch
status, poll for the honest values.**

## What this page does not establish

* **Where the control plane sits.** The section above states the disagreement rather than
  picking a side. The install kit deploys no API, so it is the permission grant rather than
  the workload and does not settle it either.
* **A live BYOC deployment.** Stage-by-stage resource numbers are read from the shared
  production cluster, which is the same software on Mixpeek's substrate. Your pools, quotas
  and storage classes are yours and will differ.
* **Networking and egress detail.** VPC layout, private-link options and per-provider network
  paths are untraced. What is enumerated is the set of call sites that reach a model provider,
  listed above. That list was wrong three times before it was right, each time by describing
  the category instead of counting: treat it as accurate on its date and derive the live set
  from a namespace's own configured extractors and stages.
* **Moderation as a product surface.** It does not exist as a resource. Stage ⑧ is a
  composition, and the destination-write gap is real.
* **Continuous sync latency.** Sync polls on `polling_interval_seconds`. End-to-end arrival
  latency is unmeasured for every provider.

> **[Deployment models](/docs/resources/deployment-models)**
>
> The shapes Mixpeek runs in, and which planes sit where.

> **[Single tenant](/docs/resources/single-tenant)**
>
> Dedicated tenancy, and what changes about the numbers above.


# Running inside your existing Ray cluster
Source: https://docs.mixpeek.com/docs/customer-hosted/existing-raycluster

Why Mixpeek brings its own Ray, and what a shared-cluster deployment would cost

> [!WARNING]
>
> This is not part of Customer-Hosted Mixpeek v1. Mixpeek deploys its own
> `RayService` into your namespace, running our Ray, our image and our CUDA
> userspace. Running Mixpeek workloads inside a `RayCluster` you already operate is
> a separate enterprise integration, scoped and priced per deployment.
>
> This page explains what we measured when we scoped it, so the decision is
> inspectable instead of a policy you have to accept.

## What the shared-cluster contract would cost

A Ray cluster runs one image. When we bring the cluster the image is ours, so
your Ray, Python and CUDA versions are not inputs to anything. When the cluster is
yours, our workloads run in your runtime, and three things follow.

**Mixpeek would have to publish an engine image built against your Ray version.**
Your version becomes part of our build matrix and stays there, so a Ray upgrade on
your side turns into a Mixpeek compatibility event on ours. That fails the test we
hold the product to: can Mixpeek upgrade itself without asking your platform team
to upgrade a dependency first?

**Your CUDA and Python userspace become ours to match.** Model code that we
qualify once against our own pinned userspace would need qualifying against
each customer's.

**Serve lifecycle becomes shared.** The detail below is the one we would most
want a customer to read before asking for this.

## The Serve lifecycle hazard, measured

Ray Serve's configuration API is declarative over the whole application set. A
config that does not name an application removes it. Whether that reaches your
applications depends on whether Serve is holding a config for them.

| How your app was deployed | Serve holds its config | A whole-config deploy |
| - | - | - |
| `serve.run(...)` in Python, imperative | No | **Leaves it alone** |
| A config `PUT` or `serveConfigV2`, declarative | Yes | **Removes it** |

You can check which you have. An application with an empty `deployed_app_config`
in `GET /api/serve/applications/` was deployed imperatively and sits outside the
lifecycle that config governs.

> [!NOTE]
>
> Measured on Ray 2.53.0 with KubeRay 1.5.1, on a single-head cluster with one
> application of each kind: the same whole-config deploy removed the declaratively
> deployed application and left the imperative one with a byte-identical deploy
> timestamp. That is enough to state the asymmetry. It is not a promise that a
> whole-config deploy is safe on your cluster, and we do not make one.

Mixpeek's deploy path never uses the whole-config route on a cluster it does not
own. We add, scale and remove one named application at a time, which cannot reach
another application however yours were deployed. That mitigation works, and it
exists because the hazard is real. Owning the `RayService` removes the hazard
rather than mitigating it, which is the cleaner answer and the one v1 ships.

Route prefixes carry a smaller version of the same problem. Serve routes on
longest matching prefix, so `/` and `/forecast` coexist. Our deploy path refuses a
prefix that would collide with one of yours, and refuses to take `/` on a shared
cluster unless explicitly told to, because every request your applications do not
match, including their 404s, would start arriving at ours.

## If you need this anyway

Some deployments have a hard requirement to use an existing cluster. Tell us your
Ray, Python and CUDA versions first, because those decide whether the integration
is feasible before anything else is worth discussing.

The qualification suite already carries the rows this contract needs, and they
run today on our fixtures:

* `serve-lifecycle` deploys, scales and removes one named Mixpeek application and
  asserts your application kept its status, replica count and **deploy
  timestamp**. The timestamp is the sharp signal, because a reconcile that took
  your app down and put it back leaves the first two unchanged.
* `non-interference` snapshots every foreign application, job and actor before and
  after.
* Both are preceded by a quiet-window control. The suite watches your cluster with
  nothing of its own in flight, and if anything moves on its own it reports
  INCONCLUSIVE rather than attributing the movement to Mixpeek.

> [!WARNING]
>
> If your cluster is under memory pressure or still settling, the isolation rows
> report INCONCLUSIVE rather than a verdict. That is deliberate. A foreign
> application restarting because the cluster is struggling looks identical to one
> disturbed by us, and reporting the second when the first is true would be a
> finding about your cluster that we invented.


# What you provide
Source: https://docs.mixpeek.com/docs/customer-hosted/kuberay

The substrate Customer-Hosted Mixpeek expects from your Kubernetes cluster

You grant Mixpeek a namespace. Mixpeek deploys the whole stack into it: the API
and services, MVS, Redis, and a `RayService` running our own Ray, image and CUDA
userspace. Batch runs as ephemeral `RayJob`s in the same namespace.

This page is the list of what has to be true of your cluster first. The
[qualification suite](/docs/customer-hosted/qualification) checks every item on it and
gives you the report before anything is installed.

## Cluster

* **KubeRay already installed**, either cluster-wide or watching the namespace you
  grant. Mixpeek does not install it and does not need to own it. This is the one
  cluster-scoped prerequisite, and
  [Security and RBAC](/docs/customer-hosted/security-rbac) covers why and for how long.
* **KubeRay 1.1 or newer.** Hard requirement. Below it `submissionMode: HTTPMode`
  does not exist and batch cannot be submitted at all.
* **Kubernetes 1.33 or newer is optional.** From 1.33 Ray's autoscaler can use
  in-place vertical scaling (`pods/resize`), which the kit's autoscaler Role
  grants. On older clusters that rule is inert and Ray replaces pods to resize
  them.
* One namespace, and the two Roles from
  [Security and RBAC](/docs/customer-hosted/security-rbac).
* A `ResourceQuota` you are comfortable with. One ships with the kit. Change the
  numbers; Mixpeek reads it and sizes batches to fit.

## Compute

Node capacity the namespace can schedule onto, with the taints and labels we
should target. Tell us the labels and we render them into the deployment rather
than assuming yours look like ours.

GPU nodes at a known NVIDIA driver baseline for inference and extraction. The
suite reports GPU nodes it can see and whether an untolerating pod can actually
reach them, which is a different question from whether the nodes exist.

### The floor, before a single document

The install brings up thirteen Ray Serve applications. Three stay warm: the engine's
entry point, the work runtime and the text embedding model, so your first upload
and your first search answer. The others start
when a call first needs them, and that call gets a 503 with a `Retry-After` header
while the replica starts; retry after the interval it gives. To keep one of them
warm, set its `min_replicas` in your values. The replica minimums reserve
**0.85 CPU sitting idle**. At maximum replicas, they
reserve **32.20 CPU**. These are logical Ray reservations; actual CPU use depends
on the workload. That is the engine alone. It does not count the API, Redis,
or the MongoDB you bring.

None of the thirteen requests a GPU. The extractors that use one take it through
the batch path, not through a warm Serve replica, so a cluster with no GPU nodes
brings the whole plane up healthy and then fails at extraction time rather than
at install time. Bring the GPU nodes; the qualification suite will tell you
whether an untolerating pod can reach them.

### Size the worker group

Serve runs its HTTP proxy on the head only, and the head is started with `num-cpus: 0`
because it holds the cluster's metadata rather than running work. Every one of
the thirteen applications asks for CPU, so none of them can land there. They all
go to a worker.

For the supported configuration, give each worker replica at least
**11 CPU and 28 GiB** available to Ray. This leaves CPU headroom above the
7.20 CPU reserved by the warm replicas. Let autoscaling add workers for batch work.

Replicas can run across workers. Each replica's CPU, memory, and custom resource
requirements must fit on its assigned worker, as described in
[Ray's scheduling rules](https://docs.ray.io/en/latest/ray-core/scheduling/resources.html).

Ray takes its logical CPU count from the container's limit, so requests and
limits both have to reach that number. Requests above limits is refused by the
Kubernetes API server outright, and the pod never appears at all: the RayCluster
still reports `RayClusterProvisioned: True`, because that condition counts the
pods that exist. Look at the `ReplicaFailure` condition when a worker is missing.

The applications also ask for a custom Ray resource named `serve_node`, which the
chart declares on both node groups through `rayStartParams`. If you replace the
cluster config with your own, carry that declaration across. Without it every
application stays unschedulable, the RayService sits in `Initializing` with an
empty `SERVICE STATUS`, and nothing reports an error.

### Disk, which is larger than the registry suggests

Two images land on every node that runs Mixpeek: the engine image and the
platform image that carries the API and MVS.

A registry size is compressed. What a node needs is the unpacked size, and on
our own fixture the platform image measured **4.96 GB in the registry and 14.2 GB
on disk**, a factor of 2.85. The engine image is larger still, around 15.4 GB
unpacked. Pulling the platform image took 130 seconds on a well-connected VM.

Size nodes for roughly **60 GB of image storage** before their own workload, and
expect the first pull on a cold node to take minutes rather than seconds.

## Egress on first start

An install with no route to the public internet does not come up, and the way it
fails is easy to misread as slowness.

Three of the thirteen Serve applications fetch their model weights from
`huggingface.co` the first time each replica loads: the reranker (2.2 GB), the
SigLIP image encoder (779 MB), and the MiniLM sentence encoder (88 MB). A fourth,
the CLAP audio encoder, fetches a checkpoint from the same host the first time it
sees audio. Two other models, ArcFace and the multilingual E5 encoder, are baked
into the image and need nothing.

So you have two options, and we would rather you pick one deliberately:

* **Allow egress to `huggingface.co`** from the Mixpeek namespace, at least on
  first start. Once a replica has loaded, it does not fetch again.
* **Ask us for a pre-baked image.** The engine image already bakes two of the six
  model sets by exactly this mechanism, so baking the rest is additive. The four
  together measure 4.9 GB, which is what it would add to the image.

One application also calls a hosted API at request time: `universal__api`,
which backs `POST /v1/inference`, dispatches to OpenAI, Google and Anthropic by
model name. It is not on the path for the extractors and retrievers a standard
install uses, so leaving it unreachable does not affect indexing or search. It
also accepts a self-hosted OpenAI-compatible endpoint, which keeps
`POST /v1/inference` working with no outbound traffic. Bring the requirement to
your deployment conversation if your environment forbids reaching a hosted
model provider.

This install does not ship an embedding model that calls out to a hosted
provider on its own: the one that used to (Gemini, through Google's Generative
Language and Vertex endpoints) is omitted here on purpose, because it cannot
serve on an install with no egress. It is also not selectable when you create a
collection on this deployment, so choosing it fails at configuration time
rather than the first time you try to extract on real data.

## Storage

Two different things, and it is worth separating them because they fail
differently.

**Object storage is where your index lives.** MVS keeps durable state in S3 or
GCS, not on a disk. Losing a pod does not lose the index; it loses at most the
un-replayed write-ahead window, which MVS keeps in a local `emptyDir` sized per
shard. That is a deliberate design: the vector store is object-storage-native, and
it is why a node drain is a recovery rather than a restore.

**A CSI storage class is still required, for the stateful services beside it.**
Redis runs as a StatefulSet with `volumeClaimTemplates` and needs a real
PersistentVolume. A cluster that cannot provision one cannot run the plane, even
though MVS itself would not have noticed.

* **A CSI storage class** that provisions persistent volumes surviving pod
  restarts and rescheduling. The suite writes a nonce from one pod, deletes it,
  and reads it back from a second pod on the same claim.
* **Throughput and IOPS above the floor for your workload size.** The suite
  measures sequential write and read against a stated floor and fails below it.
  It does not infer the floor from what it observed, because a bound derived from
  the measurement can never fail.
* **Object storage**, S3 or GCS, reachable from the namespace. This is the one
  MVS actually depends on, so treat it as the load-bearing requirement rather
  than the afterthought its position in this list once implied.

> [!WARNING]
>
> **On EKS this is not automatic.** A stock EKS 1.31 cluster cannot provision a
> PersistentVolume at all: it ships a `gp2` StorageClass that is not marked default,
> whose provisioner is the in-tree `kubernetes.io/aws-ebs` that Kubernetes removed in
> 1.31, and the EBS CSI driver is not among the default addons. It looks like working
> storage in `kubectl get storageclass` and provisions nothing. Install the
> `aws-ebs-csi-driver` addon and mark a class default before we arrive. GKE hides this
> by shipping a working default, so an EKS install is where it surfaces.

## A MongoDB you provide

The chart ships Redis and does not ship MongoDB. Mixpeek stores its metadata in
Mongo: objects, collections, batches, retriever definitions, and the six
control-plane collections. The API reads its connection string from the Secret you
supply at install. So a database has to exist and be reachable from the
namespace before the first install, and it is the one prerequisite on this page
that is not a property of your cluster.

* **A MongoDB the namespace can reach**, managed or self-run, inside the cluster
  or outside it. We do not pin a minimum server version: our driver is pymongo
  4.9, and what we run and test against is current MongoDB. If you are standing up
  something older, or a wire-compatible service rather than MongoDB itself, tell us
  which and we will qualify it rather than guess.
* **A connection string in the install Secret.** The API takes its whole
  environment from a Secret, so the URI goes there rather than into chart values.
* **Sized for metadata, not for media.** Documents are records and references;
  your video stays in object storage. This database stays small relative to the
  index.

If you intend the fully self-hosted shape, where nothing egresses to Mixpeek, this
database is also where the control-plane collections land. See
[the boundary](/docs/customer-hosted/data-flow#the-boundary-first) for what that
changes.

## Networking and identity

* Egress from the namespace to your object storage and to the Mixpeek control
  plane.
* In-cluster DNS and pod-to-pod routing inside the namespace. A default-deny
  `NetworkPolicy` with no allow rule for us is the common cause of a namespace
  that looks healthy and cannot talk to itself, and the suite separates that from
  a DNS failure rather than reporting one verdict for both.
* **Workload identity** if you want Mixpeek's pods to reach cloud resources
  without static credentials. GKE Workload Identity, EKS IRSA and AKS Workload
  Identity are all recognised.
* Image pull access to the Mixpeek engine image, either a pull secret in the
  namespace or a mirror into a registry your nodes already trust.

## Which of your customisations we accept

You asked whether we require customisations or direct control of the
`RayCluster`/`RayService` objects. We own those objects in the granted namespace,
and this is the honest list of what your platform's conventions can change about
them.

| | Accepted | How |
| - | - | - |
| **Node selectors** | Yes | Values, per worker group. Tell us the labels and we render them rather than assuming ours |
| **Tolerations** | Yes | Values, merged with the ones our GPU groups need |
| **Affinity / anti-affinity** | Yes | Values, rendered verbatim |
| **priorityClassName** | Yes | Values |
| **A scheduler you inject** (`schedulerName`) | Yes | Values |
| **Extra labels and annotations** | Yes | Values, on the pod template |
| **topologySpreadConstraints** | Yes | Values |
| **Sidecar injection by your mesh** | Yes, and we do not fight it | We register no admission webhooks, so your injector runs with nothing of ours in front of it |
| **Resource requests and limits** | Negotiated, not free | They are sized to what the extractors actually need; too small is a Pending pod rather than a slower one |
| **Replacing our container image** | No | The pinned Ray, Python and CUDA userspace is the thing that lets us upgrade without your change window |

The mechanism is one sentence: the chart renders the `rayClusterConfig` from
values, so anything expressible in a Kubernetes pod template reaches the cluster
unmodified. There is no allowlist to extend.

> [!WARNING]
>
> **One asymmetry worth knowing before you plan around it.** In Mixpeek's own
> managed deployments a generator injects scheduling into the pod spec, and there it
> **replaces** `nodeSelector` while it **merges** `tolerations`. That path is not the
> one a customer-hosted install uses, and the chart replaces nothing. We are calling
> it out because "we accept your node selectors" and "a code path exists that
> overwrites them" would both be true statements about Mixpeek today, and you
> should not have to discover which one applies to you.

## What turns the chart on, and the one value that must be right

`values.yaml` is an all-off skeleton. `enabled: false` for api, mvs, ray and
redis, each template gated on its own flag, so a bare `helm install ./chart`
renders essentially nothing. What turns components on is an **install-values overlay**
applied with `-f`. Your deployment ships with an overlay written for your
environment, so the values that govern it are the ones in that file.

Two consequences worth knowing before your first install.

**A default written under a disabled component is not a default you get.** A storage
class, a node selector or a resource request sitting in `values.yaml` beneath a
component that ships off is not what your cluster will do. Your overlay is the file
that answers "what will this actually run", and it is the one to review.

> [!WARNING]
>
> **`platform` is the value that must be right, because the failure is silent.** It
> keeps GKE-specific defaults when set to `gke`, and anything else turns them off. Left
> at the GKE default on EKS the install **succeeds** and the Deployments sit at 0 Ready
> with healthy containers, because `readinessGates` names
> `cloud.google.com/load-balancer-neg-ready`, which nothing outside Google satisfies.
> The other three are louder: gcsfuse volume mounts leave pods in `ContainerCreating`,
> and the `BackendConfig` and GMP `PodMonitoring` API groups do not exist off-GKE, so
> `helm install` fails outright. The readiness gate is the dangerous one precisely
> because it presents as a broken deployment rather than as a wrong value. Your
> overlay sets it for your distribution.

**Portability is not evenly distributed across the chart, and it is not readable from
the file names.** `api.yaml` and the three observability templates branch on `platform`.
`ray.yaml`, `mvs.yaml` and `redis.yaml` carry no such
branch. Separately, the generator that emits the Ray cluster spec writes Google node
selectors (`cloud.google.com/gke-nodepool`), so on a non-GKE cluster the node placement
in that spec is rendered data we replace for you rather than a value you override.

## Your Ray version is not an input

A Ray cluster runs one image and that image is ours. Your Ray, Python and CUDA
versions have no bearing on whether Mixpeek works, because we run our own runtime
under your scheduler rather than inside yours.

That constraint applies only to the
[shared-cluster integration](/docs/customer-hosted/existing-raycluster), which is scoped
per cluster.

## What stays yours

Node pools, quotas, autoscaler policy, taints and network policy. The
[qualification suite](/docs/customer-hosted/qualification) tests that they stay
authoritative: work that must land on your designated pool, work that must be
refused for want of a toleration, and work your quota must reject.

## Teardown

Removing the kit's objects removes what Mixpeek created. The suite's
`--expect-clean` mode runs afterwards and fails if residue remains, so the claim
gets checked.


# Ray infrastructure lifecycle
Source: https://docs.mixpeek.com/docs/customer-hosted/lifecycle

What happens to a Customer-Hosted install when your platform team deletes, drains or upgrades underneath it

Your platform team will delete things. Node pools get recreated on a schedule,
operators get patched, clusters get upgraded, and none of that will pause for
Mixpeek. This page says what each of those does to a running install, and links
the fixture run that measured it.

Everything below was measured on a throwaway Kubernetes cluster with KubeRay on
it, running the same install kit and chart a customer applies. Where something
has not been measured, this page says so rather than reasoning about it.

## The short version

| What you do | What happens | Recovery |
| - | - | - |
| Delete the `RayCluster` | KubeRay rebuilds it under the same name | Automatic, about a second |
| Delete the `RayService` | The cluster is garbage collected with it | Redeploy the chart |
| Drain the node the head is on | Head goes unschedulable, then returns | Automatic, once capacity is back |
| Upgrade KubeRay underneath | The running cluster is untouched | Nothing to do |
| Upgrade the Kubernetes control plane | Not yet measured | See below |

## Deleting the RayCluster is a non-event

Delete the `RayCluster` under a live `RayService` and KubeRay's controller builds
a new one. Measured at about one second of no Ray capacity, unattended, with a
running head at the end.

The rebuilt cluster carries the same name and a new UID. That matters if you are
writing your own health check: a check keyed on the name cannot tell "this
survived" from "this was destroyed and replaced", and those are different answers.
Key yours on `.metadata.uid`.

## Deleting the RayService needs a redeploy

This one goes the other way, so do not generalise from the case above.

Nothing reconciles a deleted `RayService` back. Only the Helm release owns it, so
removing it is a redeploy rather than something that heals. Measured: the object
was deleted, nothing recreated it, and a `helm upgrade --install` is what brings
it back.

The cascade is clean, which is the part worth stating positively. The `RayCluster`
carries an owner reference to the `RayService`, so deleting the parent collects
the child. Removing Mixpeek's `RayService` does not leave Ray pods running and
billing on your nodes with no object left to explain them.

## Draining the head's node recovers on its own

Cordon the node the Ray head is on and delete the head. While the node is
cordoned the head cannot schedule. Uncordon it and the head returns to `Running`
by itself, with no redeploy and no manual step.

That is the answer to the node-pool question: a routine drain costs you the time
the capacity is gone. It does not wedge the engine.

One caveat about how this was measured. The fixture has two nodes and the second
carries a `NoSchedule` taint for an unrelated test, so the head had nowhere to
move while its node was cordoned. On your cluster, with spare schedulable
capacity, the head reschedules instead of waiting. The transferable half is the
recovery, and that is what was asserted.

## Upgrading KubeRay does not restart your engine

KubeRay 1.5.1 upgraded to 1.6.2 underneath a live `RayService`. The running
`RayCluster` was untouched: same UID before and after, head still `Running`.

So you can patch KubeRay on your own schedule without coordinating a Mixpeek
window. If a future KubeRay release does recreate the cluster, the qualification
suite reports it as a compatibility finding rather than passing quietly.

## What is not measured yet

**In-flight work.** Every result on this page was measured on an idle cluster.
Seconds of missing Ray capacity is a non-event when nothing is running, and it may
not be when a batch is mid-flight. What happens to a batch that was executing
across a deletion, a drain or an upgrade has not been measured, and nothing here
should be read as a claim about it.

**Kubernetes control-plane upgrades.** The KubeRay half of the upgrade question is
answered above. The Kubernetes half is not. The fixture cannot answer it, because
throwaway local clusters do not upgrade their control plane in place, so a pass
there would describe the fixture rather than your cluster. Measuring it needs a
real managed cluster and it is scheduled work.

## Where your data actually lives

None of the above touches your index. Ray holds no durable state in a
Customer-Hosted install:

* Your vectors are durable in your object storage. MVS keeps an unreplayed
  write-ahead window on node-local disk, so losing a pod costs a rebuild of that
  window rather than data.
* Metadata is in Mongo, queues are in Redis, and source media stays in your own
  bucket by reference.

The stateful thing a node drain actually threatens is Redis, which uses a
persistent volume. Size and back it up the way you would any other stateful
workload in your cluster.

See also [what you provide](/docs/customer-hosted/kuberay),
[the qualification suite](/docs/customer-hosted/qualification), and
[the compatibility table](/docs/customer-hosted/compatibility).


# Customer-Hosted Mixpeek
Source: https://docs.mixpeek.com/docs/customer-hosted/overview

Run the Mixpeek runtime inside your own Kubernetes environment

You provide Kubernetes, compute, storage and IAM. Mixpeek deploys and operates the
Mixpeek **runtime** inside a namespace you grant, in your account, on your network:
the services that ingest your data, the vector store that holds your index, the Ray
cluster that runs extraction and inference, and the models.

Every byte of your content and every GPU it touches stays inside your environment.

What does not move is the part that has no data in it. The pipeline compiler, the
evaluation and tuning logic, our prompts and model-selection rules, the source and
the build chain stay with Mixpeek. That split is why your platform team is not in
our release cycle, and why we can ship a fix on a Tuesday without asking you for a
change window.

> [!NOTE]
>
> **The short version, for a security review.** Your data plane and your compute
> plane are yours and local. Our authoring plane is ours and remote. Nothing that
> carries your content crosses that line, and the [data
> flow](/docs/customer-hosted/data-flow) traces every stage that could.

## What runs where

```text theme={null}
Your account / VPC
│
└── Your Kubernetes cluster
    │
    └── Mixpeek namespace
        ├── Mixpeek API and services
        ├── MVS (the vector store)
        ├── Redis and required stateful services
        └── Mixpeek RayService
            ├── our Ray
            ├── our image
            ├── our Python and CUDA userspace
            └── our models

You provide:
├── CPU and GPU nodes
├── persistent volumes / approved storage classes
├── object storage
├── networking
└── IAM
```

The whole plane lands in the namespace, so there is no second install to reason
about and no component you have to stand up yourself before we arrive.

## What stays with Mixpeek, and why

A platform team's first question about a vendor running software in their cluster
is usually "what are you hiding in there". The honest answer is that we are not
hiding anything in there, because the parts worth protecting are not shipped.

| Runs in your namespace | Stays with Mixpeek |
| - | - |
| API and services handling your requests | The pipeline compiler |
| MVS, holding your index | Evaluation and tuning logic |
| Ray, running your extraction and inference | Prompts and model-selection rules |
| The models, executing on your GPUs | Source repositories and the build chain |
| Redis and required stateful services | Release engineering |

The line is not drawn around sensitivity, it is drawn around **data**. Everything
that touches your content runs where your content already is. Everything that
decides *how* Mixpeek works, and touches no customer data to do it, stays on our
side where we can change it without a deploy into your cluster.

That is also what makes the upgrade story work. A pipeline change is ours to make
and ship; it does not become a Kubernetes change in your environment.

## Why we bring our own runtime

Mixpeek pins a Ray version, a Python version and a CUDA userspace, and ships them
together in one image. That pin is the reason an upgrade is ours to perform.

The test we hold ourselves to: **can Mixpeek upgrade itself without asking your
platform team to upgrade a dependency first?** With our own runtime the answer is
yes. If we adapted to whatever Ray, Python and CUDA you already run, every
Mixpeek release would be gated on your cluster's version matrix, and your Ray
upgrade would become a Mixpeek compatibility event.

So you own where the bytes and the GPUs live. We own how Mixpeek works.

## Supported environments

Customer-Hosted Mixpeek supports qualified Kubernetes environments. We start
narrow on purpose and widen as real deployments ask for it.

| | Supported today |
| - | - |
| **Kubernetes** | EKS, GKE |
| **Compute** | NVIDIA GPU nodes at a known driver baseline |
| **Storage** | S3 or GCS, plus approved CSI storage classes |
| **KubeRay** | operator versions Mixpeek qualifies |
| **Install** | the Mixpeek Helm chart |

If your environment is outside this list, tell us what it is. Widening the matrix
is a qualification exercise with a real answer at the end of it, and we would
rather run it than guess.

### Air-gapped and restricted networks

A fully self-hosted install keeps the control plane in your namespace, so no pipeline stage needs outbound access to Mixpeek. Some extractors and retriever stages do call hosted model providers, and they send the content itself. That includes `universal_extractor`, the default for new namespaces, which embeds with Gemini. Which pipelines can run with no egress at all depends on the extractors and stages you choose; [What leaves your account](/docs/customer-hosted/data-flow#what-leaves-your-account) lists every call site. Talk to us about the pipeline you need before you plan a disconnected deployment.

## What Mixpeek asks for

One Kubernetes namespace, and two namespace-scoped Roles inside it.

There is one cluster-scoped prerequisite: the KubeRay operator, which installs
CRDs and ClusterRoles. It is installed once, by you, before Mixpeek arrives, and
it is the only thing outside the namespace.
[Security and RBAC](/docs/customer-hosted/security-rbac) lists every verb we ask for,
that one prerequisite and its lifetime, and how to check all of it with one
command rather than take our word for it.

## What stays yours

Node pools, quotas, taints, tolerations, autoscaler policy and network policy.
Mixpeek's workloads schedule under your rules.

That is a testable claim, and the
[qualification suite](/docs/customer-hosted/qualification) tests it. It submits work
that must land on the pool you designate, work that must be refused for want of a
toleration, and work your quota must reject, then reports which of those actually
happened.

## Before anything is deployed

The suite runs against your bare cluster first and gives you the report: a
plain-text table of checks and verdicts covering reachability, storage
durability and throughput, permissions, networking, capacity, scheduling,
isolation and teardown.

Rows that could not be run say so. A row that cannot establish something reports
what it could not determine rather than reporting an absence, because "we found
no problem" and "we could not look" are different sentences and only one of them
is evidence.

## Teardown

Uninstalling removes the namespace-scoped objects and the workloads. The suite
has an `--expect-clean` mode that runs afterwards and fails if Mixpeek residue
remains, so "we left your cluster as we found it" gets checked.

> [!NOTE]
>
> Customer-Hosted Mixpeek sits at the far end of the [deployment
> ladder](/docs/resources/deployment-models). If you would rather not operate Kubernetes
> at all, BYO Cloud keeps the data in your cloud account while Mixpeek operates the
> infrastructure.

> [!NOTE]
>
> Already running your own Ray cluster and want Mixpeek inside it? That is a
> different contract from this one, and it is
> [not part of v1](/docs/customer-hosted/existing-raycluster).


# Qualification
Source: https://docs.mixpeek.com/docs/customer-hosted/qualification

What Mixpeek checks on your cluster before deploying anything, and how to read the report

Qualification runs before any Mixpeek workload is deployed. It answers one
question: can Mixpeek be attached to this cluster safely, and if not, which part
is the problem.

You get the report. It is plain text, one line per check.

## Two reports, at two different times

There are two suites, and the difference is when they can run.

| | Substrate qualification | Ray qualification |
| - | - | - |
| **Asks** | Can the Mixpeek plane be installed here, and what does it cost in permissions | Does Mixpeek behave correctly here, and did it disturb anything of yours |
| **Runs** | Before anything of ours exists | After the plane is installed |
| **Needs** | A kubeconfig | A running Ray cluster |
| **You can run it yourself** | Yes, read-only | Not until we are installed |

Start with the substrate report. Everything it asks has an answer that should
stop an install rather than be discovered halfway through one.

### Substrate checks

| Check | What it establishes |
| - | - |
| `k8s-reachable` | The API server answers, and which cluster this is |
| `k8s-version` | Whether your version is one we test against. Below it is untested rather than known-broken, so this warns rather than fails |
| `kuberay` | Whether KubeRay is present and at least 1.1. Absent is fine and expected before an install |
| `storage-class` | A class exists that can provision volumes, with its binding mode, expansion and reclaim policy written out |
| `pvc-durability` | **Data written by one pod is readable by another after the first is gone.** This is what MVS needs from your storage and it is the row worth reading first |
| `storage-throughput` | Sequential write and read throughput on that class. Informational unless you give us a floor, because it measures burst on a fresh volume, once, sequentially, and MVS serves small random reads |
| `crd-install-grant` | Who holds the cluster-scoped grant that installing KubeRay needs. Never fails; both answers are supported |
| `namespace-grant` | The identity can create what the install creates, so a missing grant is named now rather than partway through |
| `workload-identity` | Whether a ServiceAccount here is bound to a cloud identity, so we reach your object storage as ourselves rather than with static keys you hand us |
| `pod-security` | Whether PodSecurity admission on the namespace will reject our pods, and what to tell us so it does not |
| `node-capacity` | Enough **untainted** allocatable capacity for what we asked for. Tainted and cordoned nodes are excluded and listed |
| `gpu-nodes` | Whether any node advertises a GPU |
| `namespace-census` | What your cluster held before we touched it, for comparison afterwards |

> [!NOTE]
>
> `pvc-durability` and `storage-throughput` are the only rows that write anything.
> Between them they create two small PersistentVolumeClaims and three short-lived
> pods, and delete all of it. Both are opt-in behind `--allow-provision`; everything
> else in the substrate report is read-only.
>
> It is worth understanding why it goes to that trouble. A pod that writes to its
> mount and reads the value back passes on an `emptyDir`, which persists nothing.
> So the row writes a marker unique to that run, deletes the pod, and requires a
> second pod on the same claim to read that exact marker back. A volume returning
> some other marker fails as loudly as an empty one: a claim quietly carrying
> another workload's data is the worse finding of the two.

## What the Ray suite checks

Twenty checks in five groups. Every one is a claim about *your* cluster, not
about Mixpeek.

### Cluster

| Check | What it establishes |
| - | - |
| `ray-reachable` | The Ray dashboard answers, and reports its version and node count |
| `fingerprint` | The cluster the suite is talking to is the cluster you think it is |
| `kuberay` | The KubeRay CRDs are served, at a version whose fields we emit |

### Security

| Check | What it establishes |
| - | - |
| `rbac-minimal-sa` | The submitter ServiceAccount can do what it needs and **cannot** do what it must not. Five required verbs, five forbidden ones |
| `rayjob-cr-secrets` | Whether credentials are readable from RayJob specs in your namespace |
| `dashboard-secrets` | Whether credential-shaped values are exposed through the Ray dashboard |
| `network-policy` | Whether any NetworkPolicy governs the namespace, so a deliberate choice and an oversight stop looking identical |
| `canary-secret` | A planted value is not readable where it should not be |

### Connectivity

Every one of these asks *can the cluster reach it*, not *can this laptop reach
it*. They run as jobs inside your cluster and report from there, because those
differ in exactly the cases that matter: a private endpoint, a workload identity,
an egress policy.

| Check | What it establishes |
| - | - |
| `registry-pull` | Your **nodes** can pull the Mixpeek engine image |
| `object-storage` | The cluster can reach your object storage, and whether it did so with its own identity or with static keys |
| `control-plane` | The cluster can reach the endpoints the engine needs |
| `api-health` | The Mixpeek API is reachable |
| `engine-on-target` | The engine you reached is running on the cluster under test, not a different one |

### Workloads

| Check | What it establishes |
| - | - |
| `scheduling` | Your node pool, taint and quota bind. A pod with the right toleration lands; the same pod without it is refused and the reason names the taint; an impossible selector is refused; an oversized pod is rejected by your quota |
| `autoscaling` | Worker count rises under load and falls after, never past your `maxReplicas` |
| `failure-recovery` | Different infrastructure faults produce **different, named** diagnoses through the API |
| `serve-lifecycle` | One named Serve application can be deployed, scaled and removed without moving yours |
| `golden-e2e` | A real document flows through ingestion to retrieval |

### Isolation

| Check | What it establishes |
| - | - |
| `non-interference` | Every foreign application, job and actor is unchanged after Mixpeek runs |
| `cleanup` | No Mixpeek residue after uninstall |
| `serve-full-config-hazard` | A destructive positive control, opt-in and throwaway-only, that proves the hazard the deploy path avoids is real |

## Fault handling, and where the boundary of the suite sits

The suite reports on your cluster. Some platform behaviour is a property of the
software and holds on any cluster, so it is stated here rather than measured per
install.

| Behaviour | Where it is established |
| - | - |
| Different infrastructure faults produce different, named diagnoses through the API | `failure-recovery`, on your cluster |
| A batch orphaned by node death, preemption or SIGTERM is resubmitted automatically, bounded; one orphaned by a deploy is not | Platform behaviour, identical on every install |
| Removing and recreating the `RayCluster` costs a cold start and the work in flight, and nothing durable | Platform behaviour; `upgradeStrategy: NewCluster` makes recreation the routine upgrade path |
| A Serve application of ours can be deployed, scaled and removed without moving yours | `serve-lifecycle`, on your cluster |

The distinction matters when you read a report. A row that passed is a statement
about your cluster. A behaviour in the second group is a statement about Mixpeek,
and a green report does not re-derive it.

## Reading a verdict

```
RESULT: QUALIFIED       every required check passed
RESULT: NOT QUALIFIED   a required check failed. The row names which
RESULT: INCOMPLETE      nothing failed, but not everything ran
RESULT: UNREACHABLE     the cluster did not answer at all
```

`INCOMPLETE` is the one worth understanding. It means no check contradicted
anything and some checks could not run: a missing credential, a permission the
running identity lacks, or a check that could not run. The row names which. It is
not a pass, and it is
not a failure. Treating it as either is the mistake the verdict exists to
prevent.

## Rows that skip, and why that is deliberate

A skipped row states what it could not do:

```
scheduling  SKIP | this identity cannot create pods, so scheduling policy
                   cannot be measured from here. That is the submitter Role
                   working as intended (it grants rayjobs.ray.io and nothing
                   on pods); run this row as an identity that can create pods.
```

This matters more than it looks. The alternative is a check that reports a
failure of *your* cluster when the truth is that our probe lacked a permission,
and a report that blames the customer for the auditor's access is worse than no
report.

The same discipline applies to absence. `network-policy` will tell you it could
not read your NetworkPolicies rather than telling you there are none. `kuberay`
distinguishes "not installed" from "could not read whether it is installed", and
only the first is a statement about your cluster.

## Running it yourself

The substrate suite is the one you can run before granting us anything. It needs
a kubeconfig, `kubectl`, and the `python3` already on your machine. No Ray, no
Mixpeek, no dependencies to install.

```bash theme={null}
# read-only
python3 substrate_e2e.py --namespace mixpeek

# with the storage probe, which creates one 1Gi PVC and two pods and deletes them
python3 substrate_e2e.py --namespace mixpeek --allow-provision \
  --require-cpu 8 --require-memory 32Gi
```

`--require-cpu` and `--require-memory` are the capacity the install needs; ask us
for the numbers for your configuration. Without them the capacity row skips
rather than guessing, because a check that capacity exceeds the capacity it just
observed cannot fail.

The one-liner that checks the most important RBAC claim needs nothing at all:

```bash theme={null}
kubectl auth can-i --list \
  --as=system:serviceaccount:mixpeek:mixpeek-submitter -n mixpeek
```

Ask us to run both suites against your cluster and send you the reports, or ask
for the suites and run them yourself.

> [!NOTE]
>
> The compatibility matrix of Ray, KubeRay and Kubernetes versions we publish as
> supported is generated from CI results rather than written by hand. See
> [Compatibility](/docs/customer-hosted/compatibility).


# Security and RBAC
Source: https://docs.mixpeek.com/docs/customer-hosted/security-rbac

Every permission Mixpeek asks for in your cluster, and how to verify the claim yourself

One namespace. This page is the whole ask, with the verbs written out.

## What gets created

Applying the install kit creates six objects, all namespaced:

* one `ServiceAccount` (`mixpeek-submitter`)
* two `Role`s and their `RoleBinding`s
* one `ResourceQuota`

You can read every object in about five minutes.

## The short answer, for a security review

| | Does Mixpeek install one? |
| - | - |
| CustomResourceDefinitions | **No.** Mixpeek defines no CRDs. KubeRay's three are the prerequisite below |
| Operators | **No.** Mixpeek runs no operator or controller of its own |
| ClusterRoles or ClusterRoleBindings | **No** |
| Admission webhooks, mutating or validating | **No.** Mixpeek registers nothing in the admission chain, so your own admission and scheduling policy applies to our pods with nothing of ours in front of it |
| Aggregated API servers, `APIService` objects | **No** |
| Anything outside the granted namespace | **No** |

There is exactly one cluster-scoped prerequisite and it is not ours to install.

## The one cluster-scoped prerequisite

KubeRay is cluster-scoped, and there is no namespace-only way to install it.
Measured on kuberay-operator 1.5.1: **3 CustomResourceDefinitions**
(`rayclusters`, `rayjobs`, `rayservices`) and **5 ClusterRoles with 1
ClusterRoleBinding**. CRDs and ClusterRoles are cluster-scoped by definition.

The distinction that matters to a security review is lifetime:

| | Scope | Frequency | Who normally holds it |
| - | - | - | - |
| Installing KubeRay | Cluster | Once, then on upgrade | You |
| Everything Mixpeek does after that | One namespace | Continuous | Mixpeek |

Most customers reading this page already run KubeRay, in which case there is
nothing to grant. Where you do not, your platform team installs it once with a
published Helm command, and the steady-state ask is unchanged.

You can also grant it to us, and some customers prefer to. The permissions are
`customresourcedefinitions`, `clusterroles` and `clusterrolebindings`, all
`create`. The trade is straightforward: holding it yourself means a Mixpeek
release that requires a newer KubeRay waits for your change window.

> [!NOTE]
>
> On GKE these permissions are separable, which is worth knowing if you are
> scoping a custom role. An identity can hold
> `container.customResourceDefinitions.create` and not
> `container.clusterRoles.create`, so a check that reads only the CRD verb reports
> the grant as held on a cluster where the install will fail halfway. The
> qualification suite checks all three.

## The two Roles, in full

### `rayjob-manager`: submitting batch work

| API group | Resources | Verbs |
| - | - | - |
| `ray.io` | `rayjobs` | `create`, `get`, `list`, `watch`, `delete`, `patch` |
| core | `secrets` | `get`, `list` |
| core | `resourcequotas` | `get`, `list` |
| core | `services` | `list` |

`resourcequotas` is read so batches can be sized to fit under your quota rather
than discovering it by rejection. `services` is listed to resolve the Ray head.

### `ray-autoscaler-generation-stable`: letting Ray scale its own workers

| API group | Resources | Verbs |
| - | - | - |
| core | `pods` | `get`, `list`, `watch`, `patch` |
| core | `pods/resize` | `patch` |
| `ray.io` | `rayclusters` | `get`, `patch` |

This is Ray's own autoscaler, not Mixpeek code. `pods/resize` is in-place vertical
scaling, available from Kubernetes 1.33; on older clusters the rule is inert and
Ray falls back to replacing pods.

> [!NOTE]
>
> The binding is named for a generation-stable subject on purpose. KubeRay creates
> a new ServiceAccount per cluster generation, so a name-bound Role breaks on the
> first `RayService` swap and the symptom is that worker scale-up silently stops.
> If you intend to share this namespace with your own workloads, tell us and we
> will pin the generation instead.

## What Mixpeek does not ask for

No `ClusterRole`. No cluster-scoped read of any kind. No ability to create
namespaces, read other namespaces, or touch nodes.

Verify it rather than believing it:

```bash theme={null}
kubectl auth can-i --list \
  --as=system:serviceaccount:mixpeek:mixpeek-submitter -n mixpeek
```

The qualification suite's `rbac-minimal-sa` row asserts five verbs we need and
five we must not have, including `create clusterroles`, `create namespaces`,
`delete nodes`, and creating RayJobs in a namespace other than ours. It fails if
any of the five forbidden answers `yes`.

> [!WARNING]
>
> `kubectl auth can-i --as` requires impersonation rights. If you run it as an
> identity that cannot impersonate, every answer comes back `no` and the output
> reads like a clean bill of health. Check that you got a mix of answers.

## A disclosure we would rather you heard from us

**`get`/`list` on `rayjobs` currently also reads Mixpeek credentials.** Our batch
RayJob CRs carry credential-shaped environment variables as inline values on their
container specs, so anyone holding `rayjob-manager` can read them out of the CR.

The verbs are genuinely required for batch to work. The problem is what the CR
contains, and it is ours. It is tracked, being fixed in two phases (deliver via
`envFrom` from a Secret, then remove the inline values), and the qualification
suite **fails** on `Security/rayjob-cr-secrets` until it is done.

**How long the exposure lasts.** Batch RayJob CRs are submitted with
`ttlSecondsAfterFinished: 120`, so a finished job's CR self-deletes two minutes
after it completes. What is readable is in-flight and just-finished jobs, rather
than an accumulating pile of credential-bearing objects.

## Network policy

`networkpolicy.yaml` ships as a **template with the endpoints blank**, and is
deliberately not applied by default. An incomplete egress policy does not fail at
apply time; it fails hours later, in the middle of a batch, as a timeout that
looks like something else.

That leaves a third outcome, where you apply the kit, it works, and the policy is
never revisited. So the qualification suite reports whether *any* NetworkPolicy
governs the namespace. It is a **warning** rather than a failure, since running
without one is a legitimate choice on a flat internal cluster. The point is that
deciding against a policy and forgetting about it stop looking identical.

The suite also distinguishes a policy that exists from a policy that governs *our*
pods. One carrying a `podSelector` that matches something else leaves our
workloads exactly as open as none at all, while still counting as one.

## Using a namespace that is not `mixpeek`

Two edits before your first apply: the `namespace:` line in `kustomization.yaml`,
and the Group subject `system:serviceaccounts:mixpeek` in
`role-ray-autoscaler.yaml`, which kustomize does not rewrite.

Then `kubectl apply -k .` with no `-n`. Passing `-n` instead does not work:
kustomize stamps every object with the pinned namespace and kubectl refuses the
mismatch object by object.


# Troubleshooting
Source: https://docs.mixpeek.com/docs/customer-hosted/troubleshooting

Every failure the qualification suite reports, what it means, and what to do

One section per verdict either suite can produce. If you are holding a report with
a `FAIL` or a `WARN`, find the row here.

## Substrate

Rows from `substrate_e2e.py`, which runs before anything of ours is installed.

### `pvc-durability`: the volume was empty on a second pod

Writes do not survive the pod that made them. MVS keeps its index on a
PersistentVolume, so this is disqualifying rather than inconvenient. Usually the
claim bound to something ephemeral, or the StorageClass provisioner is not
actually provisioning.

### `pvc-durability`: the volume returned a marker that is not this run's

The claim came back carrying data the suite did not write. Durability was not
demonstrated, and separately, a volume being reused across workloads is worth
understanding before anything of ours stores an index on it. Check whether the
StorageClass is handing out a pre-existing volume.

### `pvc-durability`: SKIP, this identity cannot create PVCs

The row needs to create one claim and two pods. Re-run as an identity with
`create` on `persistentvolumeclaims` in the namespace, or accept that the
durability claim is untested.

### `storage-class`: no StorageClass exists

Nothing can provision a PersistentVolume, so MVS cannot run on this cluster as it
stands.

### `storage-class`: WARN, no class is marked default

A PVC without an explicit `storageClassName` will never bind. Tell us which class
to use and we render it into the chart rather than assuming.

### `storage-class`: WARN, `allowVolumeExpansion` is false

An MVS shard that outgrows its volume has to be replaced rather than expanded.
Workable, and worth knowing before it happens at 3am.

### `storage-throughput`: WARN, no floor given

The row measured a number and asserted nothing about it, which is what it does
without `--require-write-mbps`. Ask us for the floor your configuration needs.

### `storage-throughput`: FAIL, below the stated floor

Worth reading the caveat with the number: this is a sequential write to a freshly
provisioned volume, measured once. Cloud block storage sells burst credits, so
sustained indexing will be slower than the figure that already failed. If the
class is sized by capacity (GKE `pd-balanced`, `gp3` without provisioned IOPS), a
larger volume is often the whole fix.

### `workload-identity`: WARN, no ServiceAccount is bound to a cloud identity

Mixpeek would need static credentials in a Secret to reach object storage. That
is workable and is a posture worth choosing on purpose rather than by default. If
you have wired an identity some other way, this row cannot see it: it reads
ServiceAccount annotations only and does not attempt a bucket call. The Ray
suite's `object-storage` row does that end to end, after install.

### `crd-install-grant`: WARN, this identity cannot create ClusterRoles

Not a failure, and both answers are supported. It means your platform team
installs and upgrades KubeRay, and the cost is that a Mixpeek release requiring a
newer KubeRay needs a change on your side first. See
[Security and RBAC](/docs/customer-hosted/security-rbac).

On GKE, note that `container.customResourceDefinitions.create` and
`container.clusterRoles.create` are separate permissions. Holding one does not
imply the other, which is why the row checks three verbs and not one.

### `namespace-grant`: FAIL, cannot create some resource in the namespace

The install creates it, so an install would fail partway through and leave
objects behind. The row names each missing verb.

### `pod-security`: WARN, the namespace enforces `restricted`

Our pods must run non-root with a `seccompProfile` and no added capabilities.
Tell us before the install so the chart renders a compliant `securityContext`.
Otherwise the first failure is an admission webhook message that reads like a
Mixpeek bug.

### `node-capacity`: FAIL, not enough untainted allocatable capacity

Tainted and cordoned nodes are excluded and listed in the evidence. If Mixpeek
should be scheduling onto them, tell us the tolerations and that capacity counts.
A cluster that is merely fenced should not read as too small.

### `node-capacity`: SKIP, no requirement given

The row will not infer its own bound. Ask us for the CPU and memory your
configuration needs and pass them.

### `kuberay`: WARN, not installed

Expected on a cluster where KubeRay is not installed. Somebody has to install
it; `crd-install-grant` says who can.

## Cluster

### `ray-reachable`: UNREACHABLE, or the dashboard does not answer

The suite could not reach the Ray dashboard at the address it was given. Nothing
downstream ran, so the rest of the report is empty rather than clean.

Usually one of: the dashboard is not exposed outside the cluster, a port-forward
died, or the address points at a different cluster. Confirm with
`curl <dashboard>/api/version` from wherever the suite runs.

### `fingerprint`: the cluster is not the one you think

The suite ran a job on the target and compared what came back against the address
it was given. A mismatch means two different clusters are involved, which makes
every other row describe something you did not intend to test.

### `kuberay`: CRDs not served, or below 1.1

KubeRay **1.1 or newer is required**. Below it, `submissionMode: HTTPMode` does
not exist and batch cannot be submitted at all. Check with
`kubectl get crd rayjobs.ray.io`.

## Security

### `rbac-minimal-sa`: a forbidden verb answered yes

The submitter ServiceAccount can do something it must not: create a
`ClusterRole`, create namespaces, delete nodes, or create RayJobs outside its own
namespace. The row names which.

This usually means the kit was applied on top of a pre-existing binding that
granted more, or the ServiceAccount was reused. It is the one row where a `FAIL`
means *you have given us too much* rather than too little.

### `rbac-minimal-sa`: every answer is `no`

`kubectl auth can-i --as` needs impersonation rights. Run as an identity that
cannot impersonate, everything answers `no`, and the row reads like a clean bill
of health. The suite calls this out; re-run as an identity that can impersonate.

### `rayjob-cr-secrets`: FAIL

Expected today, and it is our defect rather than yours. Batch RayJob CRs carry
credential-shaped environment variables inline, so anyone with `get`/`list` on
`rayjobs` can read them. See [Security and RBAC](/docs/customer-hosted/security-rbac) for the
fix in flight and the two-minute TTL that bounds it. The row stays red until it
lands, deliberately.

### `network-policy`: WARN, no NetworkPolicy exists

Not a failure. Mixpeek's workloads in that namespace have unrestricted egress,
which is a legitimate choice on a flat internal cluster. The row exists so that
choosing it and forgetting stop looking the same.

### `network-policy`: WARN, every policy carries a podSelector

A NetworkPolicy exists but the suite cannot resolve whether it selects *our*
pods. A policy that selects something else leaves our workloads as open as none at
all while still counting as one. Confirm the selector matches the pods in the
Mixpeek namespace.

## Connectivity

### `registry-pull`: the nodes could not pull the image

Your **nodes**, not the machine running the suite. Almost always a missing
`imagePullSecret` in the namespace, or a registry your nodes cannot authenticate
to. Mirror the image into a registry they already trust, or add the pull secret.

### `registry-pull` / `scheduling` / `failure-recovery`: SKIP, cannot create pods

Not a defect in your cluster. The identity running the suite lacks `create` on
pods, which the submitter Role does not grant by design. Re-run those rows as an
identity that can create pods in the namespace.

### `object-storage`: WARN, static keys in the environment

The bucket was reachable, but the job carried static credentials, so the run does
not demonstrate the cluster's own workload identity. If you intend to use
Workload Identity, this is the row that tells you it is not actually in use yet.

### `control-plane`: probe never reported from inside the cluster

The job that was supposed to reach your control-plane endpoints never came back.
Usually egress: a NetworkPolicy, a firewall rule, or a private endpoint without a
route. The suite runs this from inside the cluster on purpose, because reachable
from your laptop and reachable from a Ray worker are different questions.

### `engine-on-target`: the engine claims services not on this dashboard

The Mixpeek engine you reached is running somewhere other than the cluster under
test. The report would otherwise describe two machines at once.

## Workloads

### `scheduling`: a pod without a toleration still landed

Your taint is not being honoured. The suite submitted the same pod twice,
differing only by a toleration, and both scheduled.

### `scheduling`: did not schedule, but the reason does not mention a taint

The pod was blocked by something else, most often insufficient CPU. The row fails
because a pod failing to schedule is not evidence your taint did it, and crediting
it to the taint would report a working policy that was never exercised.

### `scheduling`: a pod requesting 999 CPU was accepted

There is no effective `ResourceQuota` in the namespace. Nothing bounds what
Mixpeek can ask for.

### `autoscaling`: SKIP, no maximum given

The row needs to be told your `maxReplicas`. It will not infer it: an assertion
that the count never exceeded the highest value observed is true by construction
and cannot fail.

### `autoscaling`: went past maxReplicas

The autoscaler exceeded the ceiling in your own configuration. Worth escalating;
this is your policy not being honoured by Ray, not by Mixpeek.

### `autoscaling`: scaled up and never came back down

Workers remained after the load finished. You are paying for idle capacity. Check
`idleTimeoutSeconds` in the cluster's autoscaler options.

### `failure-recovery`: two faults are indistinguishable

Two different infrastructure failures produced the same diagnosis through the API,
so a customer hitting one cannot tell which they hit. The row names the pair.

### `failure-recovery`: a fault never reached a settled state

The suite read a transition rather than an outcome, and says so rather than
reporting the transition as a diagnosis.

### `serve-lifecycle`: the customer's app moved

Deploying, scaling or removing a Mixpeek Serve application changed one of yours:
its status, its replica count, or its deploy timestamp. This is the row that must
never fail on a shared cluster. See
[Existing RayCluster](/docs/customer-hosted/existing-raycluster) for the deployment path and
why it is per-application.

### `serve-lifecycle`: INCONCLUSIVE, cluster unstable

The suite watched your cluster before writing anything and something moved on its
own. Nothing can be attributed to Mixpeek in either direction on that run. Usually
memory pressure or a cluster still settling; re-run once it is quiet.

## Isolation

### `non-interference`: a foreign app, job or actor changed

Something of yours is no longer as it was after Mixpeek ran. The row names it. If
the cluster was also unstable, the row says INCONCLUSIVE instead, because
interference and a cluster falling over produce the same observation.

### `cleanup`: Mixpeek residue after uninstall

Objects we created are still present after teardown. The row lists them.

## After install: the plane is up and a batch will not run

### `Engine submission failed ... [Errno 111] Connection refused`

The API accepted the batch and could not hand it to the engine. That errno means
a hostname resolved and nothing answered on the port, so the address is usually
right in form and wrong in destination. The message names the address the API
tried; start there.

Three things produce it, in the order worth checking.

**No Serve endpoint yet.** `kubectl get rayservice -n <namespace>` shows a blank
`SERVICE STATUS` and `NUM SERVE ENDPOINTS`. The engine is still importing, or it
cannot schedule. Check the next two.

**No Ray worker.** `DESIRED WORKERS` above `AVAILABLE WORKERS`, and the reason is
in the RayCluster's conditions rather than on any pod:

```bash theme={null}
kubectl get raycluster -n <namespace> \
  -o jsonpath='{.items[0].status.conditions[?(@.type=="ReplicaFailure")].message}'
```

A worker whose resource requests exceed its limits is refused by the Kubernetes
API server, so no pod is ever created and there is nothing to describe.
`RayClusterProvisioned` still reads `True`, because it counts the pods that exist.

**The applications have nowhere to go.** The engine's Serve deployments request a
custom Ray resource named `serve_node`. If you supplied your own cluster config
and dropped the `rayStartParams` that declare it, every application stays
unschedulable and no status field says so.

### The address itself

`kubectl exec -n <namespace> deploy/api -- env | grep ENGINE_API` shows what the
API resolved. The chart sets it to the head-only Serve service it installs
alongside the `RayService`. Point it somewhere else only if you are running the
engine outside this release.

The service is head-only on purpose. KubeRay labels every pod `ray.io/serve=true`,
workers included, and the engine runs its HTTP proxy on the head alone. A service
selecting on that label by itself sends a share of requests to pods with nothing
listening.

## Verdicts

### `INCOMPLETE`

Nothing failed and not everything ran. Look for `SKIP` rows: a missing credential,
a permission the running identity lacks, or a check that could not run. It is not a
pass and not a failure, and treating it as either is the mistake it exists to
prevent.

### `UNREACHABLE`

The cluster did not answer. No other row in the report means anything.


# Alerts
Source: https://docs.mixpeek.com/docs/enrichment/alerts

Monitor ingested content with retriever-powered alerts and real-time notifications

![Alerts: batch ingested documents checked against retriever in parallel, matches trigger webhook, Slack, or email notifications](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-alerts.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=9f6293ecc7fd647de5ac0335782a077f)

Alerts let you attach retriever pipelines to collections so that every ingested document is automatically checked against your search criteria. When matches are found, Mixpeek notifies your configured channels (webhook, Slack, or email) or processes the matching objects into a collection you define.

## How It Works

1. **Create** a retriever that defines your search criteria (e.g., semantic similarity, attribute filters)
2. **Create** an alert referencing that retriever, with notification channels configured
3. **Attach** the alert to a collection via `alert_applications` with input mappings
4. **Ingest** documents — alerts execute automatically during post-processing (Phase 3)
5. **Receive** notifications when matches are found

### Alerting on a Taxonomy-Assigned Category

If your goal is "notify my team when content lands in a specific category" rather than a semantic search, the retriever in step 1 does not need embeddings at all. Taxonomies run in Phase 1, before Alerts in Phase 3 (see [Architecture](#architecture) below), so by the time your alert's retriever executes, any category a taxonomy assigned is already a field on the document. A retriever with a single filter stage is enough:

```json theme={null}
{
  "stage_name": "attribute_filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "field": "metadata.category",
      "operator": "eq",
      "value": "counterfeit"
    }
  }
}
```

Replace `metadata.category` with the `field_path` your taxonomy's `enrichment_fields` writes (see [Taxonomies](/docs/enrichment/taxonomies)), and `value` with the category you want to alert on. Use the `in` operator instead of `eq` to alert on more than one category at once. See [Attribute Filter](/docs/retrieval/stages/attribute-filter) for the full operator list.

## Architecture

Alerts execute during the post-processing pipeline after document ingestion completes:

| Phase | System | Purpose |
| - | - | - |
| 1 | Taxonomies | Vector-based classification |
| 2 | Clusters | Document grouping |
| **3** | **Alerts** | **Retriever execution + notifications** |
| 4 | Retriever Enrichments | Field write-back |

### Parallel Execution

Within a single alert, document-level retriever calls execute **in parallel** as independent Ray tasks. If a batch ingests 100 documents, all 100 retriever calls fan out simultaneously rather than running sequentially. Results are aggregated after all calls complete, and a single notification is sent if any document produced matches.

Multiple alerts on the same collection execute **sequentially** to avoid race conditions in notification delivery.

## Configuration

### Create an Alert

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

alert = client.alerts.create(
    name="safety_content_monitor",
    description="Detect potentially harmful content on ingestion",
    retriever_id="ret_safety_classifier",
    notification_config={
        "channels": [
            {
                "channel_type": "webhook",
                "config": {"url": "https://your-app.com/webhooks/alerts"}
            },
            {
                "channel_type": "slack",
                "config": {"channel": "#content-alerts"}
            }
        ],
        "include_matches": True,
        "include_scores": True
    },
    enabled=True
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/alerts \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "safety_content_monitor",
    "retriever_id": "ret_safety_classifier",
    "notification_config": {
      "channels": [
        {"channel_type": "webhook", "config": {"url": "https://your-app.com/webhooks/alerts"}},
        {"channel_type": "slack", "config": {"channel": "#content-alerts"}}
      ],
      "include_matches": true,
      "include_scores": true
    },
    "enabled": true
  }'
```

### Conditions

`trigger_on` decides what the alert's retriever has to return for the alert to fire. The same condition is applied wherever the alert runs: when a document is written, in the batch pipeline, and on the periodic sweep.

| `trigger_on` | Fires when |
| - | - |
| `results` (default) | the retriever returns at least one row, or at least `min_count` rows when that is set, and the first row meets `threshold` when that is set |
| `no_results` | the retriever returns nothing |

Each run of the retriever is judged on its own rows. A retriever that ends in an aggregate stage returns its groups as rows, so `min_count: 3` means three groups and a `threshold` reads the first group's metrics. A run that errors or comes back degraded gives no verdict, so it neither fires a `no_results` alert nor clears one that is firing.

A `threshold` compares one field of the first row the retriever returns. The field can be a top-level key such as `score` or an aggregate column, a dot path such as `metadata.price`, or a bare name that is looked up in the row's metadata. The operators are `gt`, `gte`, `lt`, `lte`, `eq` and `ne`. A missing field, or a value of a different type, does not fire.

```json theme={null}
{
  "name": "error_burst",
  "retriever_id": "ret_errors_by_service",
  "trigger_on": "results",
  "min_count": 3,
  "notification_config": {"channels": [{"channel_type": "slack", "config": {"channel": "#ops"}}]}
}
```

```json theme={null}
{
  "name": "top_match_is_close",
  "retriever_id": "ret_prohibited_content",
  "threshold": {"field": "score", "operator": "gte", "value": 0.92},
  "notification_config": {"channels": [{"channel_type": "email"}]}
}
```

Sending `min_count` or `threshold` with `trigger_on: no_results` is refused with a 422 that names the field.

### Cooldown and Re-notify

`cooldown_seconds` keeps an alert from notifying again within that many seconds of its last notification. A fire inside the cooldown is recorded with `suppressed: cooldown` and sends nothing. The cooldown is shared by every path, so an alert that fired at ingest stays quiet on the sweep until the cooldown has passed.

`renotify_interval_seconds` applies to the sweep. While the condition stays true, the alert notifies again at this interval, and never sooner than the cooldown. Leave it unset to be told once each time the condition starts to hold.

`last_fired_at` on the alert shows when it last notified.

### Alert Inputs

`inputs` are the retriever inputs the alert runs with. The sweep has no document to map from, so it runs the retriever with exactly these. At ingest, inputs mapped from the document are laid over them, so a document field wins where both set the same key.

This is how a time window carries over to the sweep. A retriever filter on `{{time_ago(minutes=INPUT.window_minutes)}}` with `"inputs": {"window_minutes": 60}` on the alert looks back one hour whichever path runs it.

### Attach to a Collection

Attach alerts to collections via `alert_applications` when creating or updating a collection:

```python Python theme={null}
client.collections.create(
    collection_name="user_uploads",
    source={"type": "bucket", "bucket_ids": [bucket_id]},
    feature_extractor={
        "feature_extractor_name": "text_extractor",
        "version": "v1",
        "input_mappings": {"text": "content"},
    },
    alert_applications=[
        {
            "alert_id": alert.alert_id,
            "execution_mode": "on_ingest",
            "input_mappings": [
                {
                    "input_key": "query_text",
                    "source": {
                        "source_type": "document_field",
                        "path": "content"
                    }
                }
            ]
        }
    ]
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/collections \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "user_uploads",
    "source": {"type": "bucket", "bucket_ids": ["bucket_id"]},
    "feature_extractor": {
      "feature_extractor_name": "text_extractor",
      "version": "v1",
      "input_mappings": {"text": "content"}
    },
    "alert_applications": [
      {
        "alert_id": "alert_id",
        "execution_mode": "on_ingest",
        "input_mappings": [
          {
            "input_key": "query_text",
            "source": {"source_type": "document_field", "path": "content"}
          }
        ]
      }
    ]
  }'
```

### Input Mappings

Input mappings connect document fields to retriever input parameters:

| Source Type | Description | Example |
| - | - | - |
| `document_field` | Extract value from the ingested document using a dot-notation path | `{"source_type": "document_field", "path": "metadata.category"}` |
| `constant` | Pass a fixed value to the retriever | `{"source_type": "constant", "value": "safety_check"}` |

### Execution Modes

| Mode | Behavior |
| - | - |
| `on_ingest` | Execute automatically when documents are ingested (default) |
| `scheduled` | Execute on a schedule (does not trigger on ingest) |
| `on_demand` | Execute only when manually triggered |

## Notification Channels

### Webhook

```json theme={null}
{
  "channel_type": "webhook",
  "config": {"url": "https://your-app.com/webhooks/alerts"}
}
```

The webhook receives a JSON payload with alert details, matched documents, and scores.

### Slack

```json theme={null}
{
  "channel_type": "slack",
  "config": {"channel": "#alerts-channel"}
}
```

### Email

```json theme={null}
{
  "channel_type": "email",
  "config": {"to": ["alerts@your-company.com"]}
}
```

### Batch

The `batch` channel processes the bucket objects behind the matching documents into a collection you define.

```json theme={null}
{
  "channel_type": "batch",
  "config": {"collection_id": "col_alert_frames"}
}
```

After the batch that fired the alert completes, Mixpeek submits one batch per source bucket. Each batch holds the objects whose documents matched. The output documents keep lineage back to those objects. To copy capture fields such as `captured_at` onto the output documents, list them in the target collection's `field_passthrough` (see [Feature extractors](/docs/processing/feature-extractors)).

| Key | Required | Description |
| - | - | - |
| `collection_id` | Yes | Collection that receives the processed objects. Mixpeek also accepts `target_collection_id`. |
| `bucket_id` | No | Limits the batch to objects from this bucket. Defaults to the bucket in each matching document's lineage. |
| `dedup_strategy` | No | `skip` (default), `replace`, or `force`. With `skip`, Mixpeek processes an object once, even when it matches again. |

The target collection must meet three conditions:

* It reads from a bucket, and that bucket holds the matched objects.
* It is user-defined. Names that start with `_` or `mxp_` are system collections.
* It differs from the collection the alert watches. Processing back into it would fire the alert again.

Mixpeek validates the config keys when you save the alert. A missing `collection_id`, an unknown key, or an invalid `dedup_strategy` returns a 422. Mixpeek checks the three conditions after the alert fires.

The batch that fired the alert must leave the target out. A batch created without `collection_ids` covers every enabled collection that reads the bucket, the target included. `POST /v1/buckets/{bucket_identifier}/objects?auto_process=true` and `POST /v1/buckets/{bucket_identifier}/objects/batch?auto_process=true` create their batch this way. The target then holds the objects' documents before the alert fires, so with `dedup_strategy: skip` the batch the channel submits skips those objects. To exercise the channel, create the batch with `POST /v1/buckets/{bucket_identifier}/batches` and list the collections to process in `collection_ids`, leaving the target out.

Attach a second alert to the target collection to monitor the processed output. It is an ordinary retriever alert with the same channels as any other.

Each batch task records its origin in `additional_data.triggered_by_alert`. The record holds the alert ID, the execution ID, the matching document IDs, and each object's capture fields (`captured_at`, `segment_index`, `segment_duration_seconds`) when the source is an RTSP feed.

If the batch cannot start, for example because the target collection reads from another collection, Mixpeek sends an `alert.execution.failed` webhook event with the reason. Mixpeek retries transient errors and stops after 3 attempts.

## Monitoring Executions

Track alert execution history to monitor performance and debug issues:

```python Python theme={null}
executions = client.alerts.list_executions(alert_id="alert_id")

for execution in executions.results:
    print(f"Execution {execution.execution_id}: "
          f"triggered={execution.triggered}, "
          f"matches={execution.match_count}, "
          f"duration={execution.duration_ms}ms")
```

```bash cURL theme={null}
curl https://api.mixpeek.com/v1/alerts/ALERT_ID/executions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```

## Default Alerts

Every new organization gets a set of system alerts, one per built-in condition, across all its namespaces. They email the organization's `notifications_email`, which is set to the signup address when the organization is created and can be changed with `PATCH /v1/organizations`.

| Alert | Fires when | Ships |
| - | - | - |
| Batch failed | An ingestion batch ends in a failed state | enabled |
| Batch stalled | A batch makes no progress past the stall threshold | enabled |
| Batch error rate | The share of failed objects in a batch passes the threshold | enabled |
| Dead-letter queue growth | Sync objects exhaust their retries and land in the dead-letter queue | enabled |
| Provider quota exhausted | A batch hits a hard provider quota | enabled |
| Retriever returning no results | A retriever's recent executions all return zero results | enabled |
| Retriever getting no interactions | A retriever ran at least ten times in the window and none of its results drew an interaction. Quiet in a namespace that reports no interactions | enabled |
| Collection went stale | A collection that holds documents has had the same count for a week | enabled |
| Ingest stalled | A bucket that was receiving objects has received none for a day | enabled |
| Spend reached 80 percent of the monthly budget | Month-to-date spend reaches 80 percent of the monthly budget. Quiet with no budget set | enabled |
| Collection went empty | A collection that held documents drops to zero | disabled |
| Collection drift | A collection's document count drops sharply | disabled |
| Retriever latency spike | A retriever's p95 latency rises well above its baseline | disabled |
| Retriever accuracy regression | A retriever's latest evaluation metric falls below its floor | disabled |
| Retriever alignment regression | A retriever's benchmark ndcg drops sharply from the prior run | disabled |
| Task queue degraded | A processing queue's backlog is not draining | disabled |

List them with `GET /v1/organizations/alerts`, and turn one on or off with `PATCH /v1/organizations/alerts/{alert_identifier}` and `{"enabled": true}` or `false`. Provisioning finds each default by name, so a default you change or disable is not recreated.

Each condition is evaluated by a saved retriever in the namespace, named `system-<condition>` (for example `system-collection-stale`) and tagged `system-alert`, over the system collection that records it. Run it like any retriever to see which collections, batches or retrievers meet the condition now. Mixpeek never overwrites a shipped retriever you have edited.

## Comparison with Other Enrichment Types

| Feature | Taxonomies | Clusters | Alerts | Retriever Enrichments |
| - | - | - | - | - |
| Purpose | Vector-based classification | Document grouping | Notifications on match | Arbitrary retriever pipelines |
| Output | Label + score fields | Cluster assignments | Webhook/Slack/email notifications, or a batch into a collection | Configurable field write-back |
| Phase | 1 | 2 | 3 | 4 |
| Execution | Parallel per document | Batch | Parallel per document | Sequential per document |
| Use cases | Face matching, entity linking | Segmentation, pattern discovery | Content monitoring, safety checks | LLM classification, cross-collection joins |


# Clusters
Source: https://docs.mixpeek.com/docs/enrichment/clusters

Group documents by semantic similarity or metadata attributes, then label, visualize, and enrich

![Cluster visualization showing document groupings with centroids and member assignments](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-clusters.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=02851987f503c317b2a27e939c80d70b)

Clusters automatically group documents into meaningful categories. Define what to cluster on, pick an algorithm, execute, and get back labeled groups you can visualize, enrich into collections, or promote to taxonomies.

## Two Clustering Types

Mixpeek supports two fundamentally different ways to cluster documents:

> **Vector (Semantic)**
>
> Groups documents by **embedding similarity** — what they mean, not what metadata they have. Uses vector embeddings from any extractor (text, image, multimodal) and supports 8 algorithms.
>
> Best for: topic discovery, content deduplication, visual similarity, finding themes across modalities.

> **Attribute (Metadata)**
>
> Groups documents by **metadata field values** — like a `GROUP BY` on structured columns. No embeddings needed; operates directly on payload fields.
>
> Best for: categorical grouping, hierarchical organization by brand/category/status, faceted analytics.

### Minimal Request

A vector cluster needs `collection_ids`, `vector_config.feature_uris`, and `vector_config.clustering_method`. `clustering_method` has no default.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "product_topics",
    "collection_ids": ["col_products"],
    "vector_config": {
      "feature_uris": ["mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"],
      "clustering_method": "hdbscan"
    }
  }'
```

```python Python theme={null}
cluster = mp.clusters.create(
    cluster_name="product_topics",
    collection_ids=["col_products"],
    vector_config={
        "feature_uris": ["mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"],
        "clustering_method": "hdbscan",
    },
)
```

* `collection_ids` is a list, even for one collection.
* Each entry in `feature_uris` has the form `mixpeek://{extractor}@{version}/{output}`. Every collection in `collection_ids` must hold that feature.
* `cluster_type` defaults to `vector`. To create an attribute cluster, send `attribute_config` with an `attributes` list instead of `vector_config`.
* A full vector run needs at least 2 documents in the collections. K-Means needs at least `n_clusters` and HDBSCAN needs at least `min_cluster_size`. Below the minimum, the execute call fails with `Insufficient documents for clustering`.

### Vector Clustering

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "product_topics",
    "collection_ids": ["col_products"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uris": ["mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"],
      "clustering_method": "hdbscan",
      "algorithm_params": { "min_cluster_size": 10, "min_samples": 5 },
      "preprocessing_steps": [
        { "method": "whitening" },
        { "method": "umap", "n_components": 50, "n_neighbors": 30 }
      ]
    },
    "llm_labeling": {
      "provider": "openai",
      "model_name": "gpt-4o-mini"
    }
  }'
```

### Attribute Clustering

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "product_categories",
    "collection_ids": ["col_products"],
    "cluster_type": "attribute",
    "attribute_config": {
      "attributes": ["category", "brand"],
      "hierarchical_grouping": true
    }
  }'
```

With `hierarchical_grouping: true`, this creates nested groups: "Electronics" containing "Apple", "Samsung", etc. Without it, you get flat groups like "Electronics\_Apple", "Electronics\_Samsung".

## Algorithms

Vector clustering supports 8 algorithms. Pick based on whether you know how many clusters to expect:

| Algorithm | Best When | Key Parameters |
| - | - | - |
| **HDBSCAN** | You don't know the number of clusters. Handles variable density, auto-detects noise. | `min_cluster_size`, `min_samples` |
| **K-Means** | You know K. Fast, spherical clusters. | `n_clusters`, `max_iter` |
| **DBSCAN** | You want density-based grouping with a fixed distance threshold. | `eps`, `min_samples` |
| **Agglomerative** | You want hierarchical merging with a specific linkage strategy. | `n_clusters`, `linkage` (ward/complete/average/single) |
| **Spectral** | Clusters have complex, non-convex shapes. | `n_clusters_spectral` |
| **Gaussian Mixture** | You need soft (probabilistic) assignments. | `n_components_gmm` |
| **Mean Shift** | You want automatic cluster count via bandwidth-based mode finding. | bandwidth params |
| **OPTICS** | Similar to DBSCAN but handles varying density better. | `eps`, `min_samples` |

> [!TIP]
>
> Start with **HDBSCAN** if you don't know how many clusters to expect. Use **K-Means** when you have a target count and want fast results.

## Multi-Feature Strategy

When clustering on multiple embeddings (e.g., text + image), choose how to combine them:

| Strategy | How It Works | Use When |
| - | - | - |
| **`concatenate`** (default) | Fuses all embeddings into a single vector, then clusters once. Supports per-feature weights. | Features are complementary and you want one set of clusters. |
| **`independent`** | Runs separate clustering per feature. Produces one output per modality. | You want to compare how text clusters vs image clusters differ. |
| **`weighted`** | Auto-learns optimal feature weights via Bayesian optimization. | You're not sure which modality matters more — let the algorithm decide. |

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "multimodal_themes",
    "collection_ids": ["col_ads"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uris": [
        "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
        "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
      ],
      "multi_feature_strategy": "weighted",
      "clustering_method": "hdbscan",
      "algorithm_params": { "min_cluster_size": 15 }
    }
  }'
```

## Visualization Dimensions

In Studio, the cluster scatter plot encodes three visual dimensions so you can explore cluster structure at a glance:

| Visual Dimension | What It Represents | Controlled By |
| - | - | - |
| **Position (X, Y)** | Semantic proximity — nearby points are similar in embedding space | First two coordinates from dimensionality reduction (UMAP, PCA, or t-SNE) |
| **Color** | Cluster membership — each cluster gets a distinct color | Automatic assignment based on cluster ID |
| **Dot size** | Depth (Z axis) — larger dots have higher Z values, creating a depth cue | Third coordinate when `dimension_reduction.components` is set to `3` |

To enable the size dimension, set 3 components in your dimensionality reduction config:

```json theme={null}
{
  "dimension_reduction": {
    "method": "umap",
    "components": 3
  }
}
```

> [!NOTE]
>
> Without the third component, all dots render at the same size. With it, the Z value is linearly mapped to dot radius (10px–50px), so visually prominent points sit "closer" in the third principal axis.

## Centroid Methods

Control how cluster centers are calculated:

| Method | Description |
| - | - |
| **`mean`** (default) | Average of all member vectors. Smooth, stable centroids. |
| **`median`** | Median vector. More robust to outliers than mean. |
| **`medoid`** | The actual cluster member closest to the center. Most interpretable — the centroid is a real document. |

## Preprocessing

High-dimensional embeddings (1408d, 3072d) benefit from preprocessing before density-based algorithms. The `preprocessing_steps` field accepts an ordered list:

> **Whitening (ZCA)**
>
> Decorrelates embedding dimensions, removing redundant structure that causes density-based algorithms to over-fragment.
>
> ```json theme={null}
> { "method": "whitening", "regularization": 1e-5 }
> ```

> **UMAP reduction**
>
> Reduces dimensionality while preserving neighborhood structure. Critical for HDBSCAN on high-dimensional data.
>
> ```json theme={null}
> { "method": "umap", "n_components": 50, "n_neighbors": 30, "min_dist": 0.0, "metric": "cosine" }
> ```

> **Chained (recommended for high-dim)**
>
> Whitening + UMAP together typically improves HDBSCAN cluster purity by 15–30% on embeddings above 1000 dimensions.
>
> ```json theme={null}
> {
>   "preprocessing_steps": [
>     { "method": "whitening", "regularization": 1e-5 },
>     { "method": "umap", "n_components": 50, "n_neighbors": 30, "min_dist": 0.0 }
>   ]
> }
> ```

## Execution Modes

| Mode | What It Does | When To Use |
| - | - | - |
| **`full`** (default) | Clusters all documents from scratch. | Initial run, or when you want fresh clusters. |
| **`assign`** | Assigns new documents to existing centroids without re-clustering. O(n×k). | Streaming ingestion — run `full` periodically, `assign` for new docs in between. |
| **`composite`** | Clusters the **centroids** from prior executions together. | Cross-modality comparison, temporal drift detection, parameter tuning. |

### Incremental Assignment

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters/{cluster_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "assign",
    "assignment_threshold": 0.5
  }'
```

Documents below the `assignment_threshold` cosine similarity are marked as noise (`cluster_id = -1`).

### Composite Clustering

Clusters centroids from prior runs to reveal higher-order patterns. A run with 10,000 documents and 50 clusters contributes only 50 vectors, so composite execution is fast.

**Map your library along multiple dimensions at once.** A common analyst question — "how do my *visual-style* groups relate to my *messaging-theme* groups?" — is expressed with exactly this machinery, no tagging required:

1. Run **several independent clusterings on the same collection** (one per dimension — e.g. one over visual embeddings, one over transcript embeddings). Each is its own cluster resource; a collection can have as many as you need.
2. **Composite them**: pass the runs as `source_execution_ids` with `mode: "composite"`. The composite groups the centroids from every input run, so groups that land together across dimensions surface as one higher-order pattern (e.g. a "street-interview format" centroid clustering next to a "risk-reversal claim" centroid = a combination worth briefing).
3. Read the result on the cluster's **visualization map** in Studio (lasso a region to search inside it, or create a retriever scoped to a group directly from the map).

A first-class per-document cross-tab (counts of *group A × group B* membership) isn't a built-in view yet — the composite map shows how the groupings relate at the pattern level, and each input clustering keeps its own per-document groups.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters/{cluster_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "composite",
    "source_execution_ids": ["run_abc123", "run_def456"]
  }'
```

```python Python theme={null}
composite = mp.clusters.execute(
    cluster_id="cluster_id",
    mode="composite",
    source_execution_ids=[run_a.run_id, run_b.run_id],
)
```

## Hierarchical Sub-Clustering

Enable recursive sub-clustering when top-level clusters are too broad. Each cluster with at least twice `min_cluster_size` members is further divided using UMAP + HDBSCAN. `min_cluster_size` defaults to 50, so a top-level cluster needs 100 members before it splits.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "content_hierarchy",
    "collection_ids": ["col_videos"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uris": ["mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"],
      "clustering_method": "hdbscan",
      "algorithm_params": { "min_cluster_size": 50, "min_samples": 10 },
      "hierarchical": true,
      "max_hierarchy_depth": 3
    }
  }'
```

Sub-clusters get IDs like `cl_0_sub_1_sub_0`. Example: "sports", then "basketball", then "NBA highlights".

The output collection holds the whole tree:

| Document | Field | Value |
| - | - | - |
| Centroid | `parent_cluster_id` | The cluster one level up. Absent on top-level centroids |
| Centroid | `hierarchy_level` | `0` for top-level clusters, `1` and deeper for sub-clusters |
| Centroid | `child_cluster_ids` | The sub-clusters one level down |
| Member | `cluster_id` | The deepest cluster the document belongs to |
| Member | `cluster_path` | Every cluster from the top level down to `cluster_id` |

To read every member under a top-level cluster, including the ones in its sub-clusters, filter on `cluster_path` rather than `cluster_id`. [Agent Search](/docs/retrieval/stages/agent-search#cluster-navigation) walks this tree with `cluster_navigation`.

## Quality Metrics

Every execution returns metrics that tell you whether your clusters are meaningful:

| Metric | Range | What It Means |
| - | - | - |
| `silhouette_score` | -1 to 1 | Cluster separation quality. Above 0.5 is good. |
| `mean_cosine_to_centroid` | 0 to 1 | Average assignment confidence. Higher = tighter clusters. |
| `noise_ratio` | 0 to 1 | Fraction classified as noise. High values suggest parameters are too strict. |
| `cluster_size_entropy` | 0 to 1 | How balanced cluster sizes are. 1.0 = perfectly even. |
| `should_recluster` | 0 or 1 | Automatic recommendation based on metric thresholds. |

Per-member similarity is also tracked:

| Field | Level | Description |
| - | - | - |
| `cosine_similarity_to_centroid` | Member | How well this document fits its cluster (0–1) |
| `mean_cosine_similarity` | Centroid | Average member similarity (cohesion) |
| `min_cosine_similarity` | Centroid | Weakest member (indicates outliers) |

## LLM Labeling

Generate human-readable names, summaries, and keywords for each cluster. Control which document fields the LLM sees using **input mappings**.

### Text-Only Labeling

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "script_archetypes",
    "collection_ids": ["col_ad_scripts"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uris": ["mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"],
      "clustering_method": "kmeans",
      "kmeans_parameters": { "n_clusters": 20 }
    },
    "llm_labeling": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "labeling_inputs": {
        "input_mappings": [
          { "input_key": "text", "source_type": "payload", "path": "headline" },
          { "input_key": "text", "source_type": "payload", "path": "primary_text" },
          { "input_key": "text", "source_type": "payload", "path": "description" }
        ]
      }
    }
  }'
```

```python Python theme={null}
cluster = mp.clusters.create(
    cluster_name="script_archetypes",
    collection_ids=["col_ad_scripts"],
    cluster_type="vector",
    vector_config={
        "feature_uris": ["mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"],
        "clustering_method": "kmeans",
        "kmeans_parameters": {"n_clusters": 20},
    },
    llm_labeling={
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "labeling_inputs": {
            "input_mappings": [
                {"input_key": "text", "source_type": "payload", "path": "headline"},
                {"input_key": "text", "source_type": "payload", "path": "primary_text"},
                {"input_key": "text", "source_type": "payload", "path": "description"},
            ]
        },
    },
)
```

### Labeling From a Fixed Vocabulary

Set `candidate_labels` to label every cluster from a list you control. Each cluster gets the candidate that best fits its representative documents. Use a list of labels, or a map of label to a one-line description of what belongs under it. Up to 255 labels.

A fixed vocabulary keeps labels consistent across runs and across the levels of a hierarchical run. It also lets you map clusters onto an existing taxonomy level.

A [decision model](/docs/processing/jev#swap-the-decision-model) picks the label from the list. With `provider: "typesafe"` and `model_name: "jev-latest"`, [Jev](/docs/processing/jev) returns a calibrated probability. Any other model, such as `gemini-2.5-flash-lite`, answers the same question and reports its own confidence. Each centroid stores:

| Field | Value |
| - | - |
| `label_confidence` | Probability of the chosen label |
| `label_candidates` | The three most likely labels, each with its probability |
| `label_model` | The model that answered, such as `jev-1.13.0` |
| `label_calibrated` | `true` for Jev. `false` for a generative model's own estimate |

Jev does not write text, so it leaves `summary` and `keywords` empty, and it requires `candidate_labels` or a `response_shape` schema of closed-set fields. Jev reads text only, so point `labeling_inputs` at text fields.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "ad_categories",
    "collection_ids": ["col_ad_scripts"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
      "clustering_method": "kmeans",
      "algorithm_params": {"n_clusters": 3},
      "hierarchical": true,
      "max_hierarchy_depth": 2
    },
    "llm_labeling": {
      "enabled": true,
      "provider": "typesafe",
      "model_name": "jev-latest",
      "candidate_labels": {
        "Pharmaceutical Ads": "Prescription or OTC drug commercials",
        "Car Ads": "Vehicle sales and lease offers",
        "Product Demos": "Someone shows how a product works"
      },
      "labeling_inputs": {
        "input_mappings": [
          {"input_key": "text", "source_type": "payload", "path": "content"}
        ]
      }
    }
  }'
```

### Multimodal Labeling

Send images or video alongside text for richer labels. Use a vision-capable model:

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "scene_themes",
    "collection_ids": ["col_ad_scenes"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uris": ["mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"],
      "clustering_method": "kmeans",
      "kmeans_parameters": { "n_clusters": 25 }
    },
    "llm_labeling": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-preview-04-17",
      "labeling_inputs": {
        "input_mappings": [
          { "input_key": "text", "source_type": "payload", "path": "headline" },
          { "input_key": "text", "source_type": "payload", "path": "primary_text" },
          { "input_key": "image_url", "source_type": "blob", "path": "document_blobs.0.url" }
        ]
      }
    }
  }'
```

```python Python theme={null}
cluster = mp.clusters.create(
    cluster_name="scene_themes",
    collection_ids=["col_ad_scenes"],
    cluster_type="vector",
    vector_config={
        "feature_uris": ["mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"],
        "clustering_method": "kmeans",
        "kmeans_parameters": {"n_clusters": 25},
    },
    llm_labeling={
        "provider": "google",
        "model_name": "gemini-2.5-flash-preview-04-17",
        "labeling_inputs": {
            "input_mappings": [
                {"input_key": "text", "source_type": "payload", "path": "headline"},
                {"input_key": "text", "source_type": "payload", "path": "primary_text"},
                {"input_key": "image_url", "source_type": "blob", "path": "document_blobs.0.url"},
            ]
        },
    },
)
```

### Input Mapping Reference

| Field | Description |
| - | - |
| `input_key` | Key the LLM receives: `text`, `image_url`, `video_url`, `audio_url` |
| `source_type` | Where to pull the value: `payload` (document fields), `blob` (stored assets), `literal` (static value) |
| `path` | Dot-notation path into the document (for `payload` and `blob`) |
| `override` | Static value (only with `literal` source type) |

> [!NOTE]
>
> Without `labeling_inputs`, the full document payload is serialized as JSON. Input mappings let you send only the fields that matter.

### Custom Prompts and Response Shapes

Override the default prompt for domain-specific labels:

```json theme={null}
{
  "llm_labeling": {
    "provider": "openai",
    "model_name": "gpt-4o",
    "custom_prompt": "Analyze these ad clusters and label each creative archetype (e.g. 'UGC Testimonial', 'Problem-Solution Demo').",
    "response_shape": {
      "label": "string",
      "keywords": ["string"],
      "sentiment": "positive | negative | neutral",
      "target_audience": "string"
    }
  }
}
```

### Labeling Settings

| Setting | Default | Description |
| - | - | - |
| `max_samples_per_cluster` | auto (3–20) | Representative documents sent to the LLM per cluster. |
| `sample_text_max_length` | — | Truncate text inputs to this character length. |
| `include_summary` | `true` | Generate a longer summary alongside the label. |
| `include_keywords` | `true` | Generate keyword tags for each cluster. |
| `use_embedding_dedup` | `false` | Merge similar labels across clusters using embedding similarity. |
| `embedding_similarity_threshold` | 0.8 | Threshold for label dedup. |
| `cache_ttl_seconds` | 604800 | Cache labels for 7 days. Set to 0 to disable. |

## Enrichment

Write cluster membership back into your source collections:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters/{cluster_id}/enrich" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "run_id": "run_xyz789",
    "target_collection_id": "col_products_enriched",
    "fields": ["cluster_id", "label", "summary", "keywords"]
  }'
```

This writes `cluster_id` and labels into document payloads, enabling cluster-based filters and facets in retrievers.

## Execution & Triggers

* **Manual**: `POST /v1/clusters/{id}/execute`
* **Async job**: `POST /v1/clusters/{id}/execute/submit`
* **Automated**: create cron, interval, or event-based triggers under `/v1/clusters/triggers`
* Every run yields a `run_id` and exposes status via `GET /v1/clusters/{id}/executions`

## Where Assignments Live

Cluster assignments are documents in the cluster's output collections. `GET /v1/clusters/{cluster_id}` returns their IDs in `output_collection_ids`. A cluster with `multi_feature_strategy: "independent"` and several feature URIs has one output collection per feature URI.

Each output collection holds two kinds of document, told apart by `is_centroid`:

| Document | `is_centroid` | What it carries |
| - | - | - |
| Centroid | `true` | One per cluster: `cluster_id`, `cluster_label`, and `member_count`. |
| Member | `false` | One per clustered source document: `cluster_id`, `cluster_label`, and `source_document_id`. |

A member document has its own `document_id`. Its `source_document_id` is the ID of the document you clustered. To find the cluster of one source document, call `POST /v1/collections/{output_collection_id}/documents/list` and filter on `source_document_id`.

```bash Find the cluster of one document theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections/$OUTPUT_COLLECTION_ID/documents/list" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "filters": {
      "AND": [
        { "field": "source_document_id", "operator": "eq", "value": "doc_abc123" },
        { "field": "is_centroid", "operator": "eq", "value": false }
      ]
    }
  }'
```

```bash List the members of one cluster theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections/$OUTPUT_COLLECTION_ID/documents/list" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "filters": {
      "AND": [
        { "field": "cluster_id", "operator": "eq", "value": "cl_3" },
        { "field": "is_centroid", "operator": "eq", "value": false }
      ]
    }
  }'
```

The output collection holds the latest run only. To read an earlier run, export it.

With `enrich_source_collection: true` on create, the cluster writes no output collection. `output_collection_ids` then lists your source collections, and each source document gets `cluster_id` and `cluster_label` fields. Set `source_enrichment_config.field_mappings` to rename them or add more.

| To read | Endpoint |
| - | - |
| Labels, summaries, and document counts of the latest run | `GET /v1/clusters/{cluster_id}/groups` |
| Scatter-plot coordinates | `GET /v1/clusters/{cluster_id}/visualization` |
| Every member and centroid of one run, as CSV or Parquet | `GET /v1/clusters/{cluster_id}/executions/{run_id}/export` |

## Management

| Operation | Endpoint |
| - | - |
| Inspect definition | `GET /v1/clusters/{id}` |
| List clusters | `POST /v1/clusters/list` |
| Execution history | `GET /v1/clusters/{id}/executions` |
| Delete | `DELETE /v1/clusters/{id}` |

## Best Practices

1. **Start with HDBSCAN + whitening + UMAP** for vector clustering on high-dimensional embeddings.
2. **Prototype on samples** — tune parameters with a small `sample_size` before running at scale.
3. **Use incremental assignment for streaming data** — `full` periodically, `assign` for new documents in between.
4. **Monitor quality metrics** — check `silhouette_score` and `noise_ratio` after each run. Recluster when `should_recluster` fires.
5. **Use attribute clustering for categorical grouping** — don't force embeddings when metadata fields already capture the structure.
6. **Try multi-feature `weighted` strategy** when combining modalities — let Bayesian optimization find the right blend.
7. **Enable 3-component dimensionality reduction** to get the depth (size) dimension in Studio visualizations.

## Clusters vs Taxonomies vs Alerts

| I want to… | Use |
| - | - |
| Discover what categories exist in my data | **Clusters** |
| Apply known categories to new documents | [Taxonomies](/docs/enrichment/taxonomies) |
| Get notified when something specific appears | [Alerts](/docs/enrichment/alerts) |
| Turn discovered groups into reusable labels | Clusters → promote to taxonomy |


# Retriever Enrichments
Source: https://docs.mixpeek.com/docs/enrichment/retriever-enrichments

Run retriever pipelines on documents at ingestion time

![Retriever enrichments: document ingested, inputs mapped to retriever, pipeline executes, selected fields written back to document](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-retriever-enrichments.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=fbdd4b6942f4e4891f366bfbacfa0c38)

Retriever enrichments let you attach arbitrary retriever pipelines to collections. When documents are ingested, the configured retrievers execute against each document and write selected result fields back to the document. This enables LLM classification, cross-collection joins, and multi-stage enrichment without building custom extractor plugins.

## How It Works

1. **Attach** a retriever enrichment to a collection with input mappings and write-back field configuration
2. **Ingest** documents via batch processing as usual
3. **Post-processing** executes the retriever for each document, mapping document fields to retriever inputs
4. **Write-back** extracts specified fields from retriever results and writes them to the document

Retriever enrichments run in **Phase 4** of post-processing by default (after taxonomies, clusters, and alerts), but you can configure them to run in any phase.

## Configuration

Each retriever enrichment has three main sections:

### Input Mappings

Map document fields or constant values to retriever input parameters:

```json theme={null}
{
  "input_mappings": [
    {
      "input_key": "query",
      "source": {
        "source_type": "document_field",
        "path": "title"
      }
    },
    {
      "input_key": "collection_id",
      "source": {
        "source_type": "constant",
        "value": "col_reference_data"
      }
    }
  ]
}
```

### Write-Back Fields

Configure which retriever result fields to write back to documents:

```json theme={null}
{
  "write_back_fields": [
    {
      "source_field": "category",
      "target_field": "_enrichment_category",
      "mode": "first"
    },
    {
      "source_field": "related_items",
      "target_field": "_related_ids",
      "mode": "all_as_array"
    }
  ]
}
```

**Write-back modes:**

| Mode | Behavior |
| - | - |
| `first` | Write value from the first result only (default) |
| `all_as_array` | Collect values from all results into a list |
| `concat` | Concatenate string values from all results with ", " separator |

### Execution Control

| Field | Description | Default |
| - | - | - |
| `execution_phase` | Post-processing phase (1-4) | 4 (Enrichment) |
| `priority` | Priority within phase (higher = runs first) | 0 |
| `scroll_filters` | Filter which documents to enrich | None (all documents) |
| `enabled` | Whether enrichment is active | true |

## Example: LLM Classification at Ingestion

Attach a retriever with an `llm_enrich` stage to classify documents as they're ingested:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# Update collection to add retriever enrichment
client.collections.update(
    collection_id="col_articles",
    retriever_enrichments=[
        {
            "retriever_id": "ret_classifier",
            "input_mappings": [
                {
                    "input_key": "query",
                    "source": {
                        "source_type": "document_field",
                        "path": "content"
                    }
                }
            ],
            "write_back_fields": [
                {
                    "source_field": "category",
                    "target_field": "_llm_category",
                    "mode": "first"
                },
                {
                    "source_field": "sentiment",
                    "target_field": "_llm_sentiment",
                    "mode": "first"
                }
            ],
            "enabled": True
        }
    ]
)
```

## Example: Cross-Collection Join

Use a retriever enrichment to join data from a reference collection:

```python Python theme={null}
client.collections.update(
    collection_id="col_products",
    retriever_enrichments=[
        {
            "retriever_id": "ret_brand_lookup",
            "input_mappings": [
                {
                    "input_key": "query",
                    "source": {
                        "source_type": "document_field",
                        "path": "brand_name"
                    }
                }
            ],
            "write_back_fields": [
                {
                    "source_field": "brand_logo_url",
                    "target_field": "_brand_logo",
                    "mode": "first"
                },
                {
                    "source_field": "brand_category",
                    "target_field": "_brand_category",
                    "mode": "first"
                }
            ],
            "enabled": True
        }
    ]
)
```

> [!NOTE]
>
> Retriever enrichments execute sequentially within each collection to avoid race conditions. For collections with many documents, enrichment time scales linearly with document count.

## Comparison with Other Enrichment Types

| Feature | Taxonomies | Clusters | Alerts | Retriever Enrichments |
| - | - | - | - | - |
| Purpose | Vector-based classification | Document grouping | Notifications | Arbitrary retriever pipelines |
| Output | Label + score fields | Cluster assignments | Webhook notifications | Configurable field write-back |
| Phase | 1 | 2 | 3 | 4 (default) |
| Execution | Parallel per document | Batch | Parallel per document | Sequential per document |
| Use cases | Face matching, entity linking | Segmentation, pattern discovery | Content monitoring | LLM classification, cross-collection joins |


# Taxonomies
Source: https://docs.mixpeek.com/docs/enrichment/taxonomies

Enrich documents with similarity-based joins

Taxonomies let you attach structured metadata to documents by matching them against a reference collection. They are implemented as retriever-powered joins and can run on demand or be materialized into collections. Taxonomies are warehouse-native enrichment: the multimodal equivalent of a SQL JOIN, linking documents to canonical entities via embedding similarity rather than key equality.

![From clusters to taxonomies: documents are embedded into a shared vector space, unsupervised clustering discovers natural structure, an LLM labels each cluster (product reviews, technical docs, legal contracts, support tickets), and stable clusters are promoted into reusable hierarchical taxonomy nodes. The taxonomy then auto-classifies new documents at ingest, becomes search filters at query time, and reclassifies existing documents retroactively when the hierarchy changes.](https://mintcdn.com/mixpeek/7Pcnzu2AXsKNt5KO/assets/taxonomy-retrieval.png?fit=max\&auto=format\&n=7Pcnzu2AXsKNt5KO\&q=85\&s=3ac59885238e11b09dd02d900b383c50)

## Taxonomy Types

| Type | Structure | When to Use |
| - | - | - |
| Flat | Single-level reference collection | Face enrollment, entity linking, simple lookups |
| Hierarchical | Parent/child nodes with inheritance | Org charts, product categories, multi-level labeling |

Each node references a collection, retriever, and list of enrichment fields. Child nodes inherit parent properties automatically.

### Flat Taxonomy: Product Catalog Recognition

![Flat taxonomy showing multimodal documents matched against a product catalog reference collection](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-flat-taxonomy.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=9902f166d2881ee94901487218cd4674)

In a flat taxonomy, documents from any modality (video, image, audio, text) are matched against a single reference collection. Each document uses its appropriate feature embedding (CLIP for visual, text embeddings for audio transcripts) to find the best match. Enrichment fields (SKU, category, price) are attached when similarity exceeds the threshold.

### Hierarchical Taxonomy: Media Content Classification

![Hierarchical taxonomy showing media content classification with multiple levels from brand to campaign](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-hierarchical-taxonomy.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=35fa9d2cf3a194faaf4b59a62a4dc655)

In a hierarchical taxonomy, documents traverse multiple levels of progressive refinement. Starting from a broad brand classification (1 node), through content category (2 nodes), sport/style type (4 nodes), audience segmentation (5 nodes), to specific campaigns (6 nodes). Each level narrows the classification using different multimodal features—CLIP for brand detection, scene classification for categories, activity detection for sport types, demographic models for audiences, and campaign-specific patterns at the final level. Documents inherit all properties from parent nodes as they traverse down the tree.

## Execution Modes

A taxonomy runs one of three ways, each through its own surface, not a shared `mode` field:

| Mode | Description | Configured via | Use Case |
| - | - | - | - |
| On-demand | Enrich documents at query time inside a retriever | A `taxonomy_enrich` stage on a [retriever](/docs/retrieval/retrievers) | Exploratory workflows, testing, dynamic reference data |
| Materialized | Batch enrichment after extraction; results persisted in the collection | `taxonomy_applications[].execution_mode: "materialize"` on the collection | Production search, low-latency retrieval, analytics |
| Retroactive | Apply taxonomy to existing documents in a collection | `POST /v1/collections/{id}/apply-taxonomy` | Backfilling, taxonomy updates, schema migrations |

`taxonomy_applications[].execution_mode` only accepts `materialize`; the other two modes are not values of this field. See [Application Methods](#application-methods) below for each mode's request shape.

### How Hierarchical Taxonomies Execute

Hierarchical taxonomies are executed like **Common Table Expressions (CTEs)** in SQL—each level builds on the results of the previous level, creating a recursive evaluation chain from root to leaf nodes.

```
Level 1 (Root)     →  Match against brand collection
    ↓ passes matched docs
Level 2            →  Match against category collection (filtered by L1 result)
    ↓ passes matched docs
Level 3            →  Match against subcategory collection (filtered by L2 result)
    ↓ passes matched docs
Level N (Leaf)     →  Final enrichment fields attached
```

At each level:

1. Documents that matched the parent node are passed down
2. The child node's retriever executes against its reference collection
3. Enrichment fields from matching nodes are accumulated
4. Only documents exceeding the similarity threshold continue to child nodes

This CTE-style execution ensures that a document classified as "Nike → Athletic → Running" inherits enrichment fields from all three levels, not just the leaf node.

### Application Methods

Hierarchical taxonomies can be applied through three methods:

| Method | When It Runs | Use Case |
| - | - | - |
| **On-demand** | Query time, as a [retriever stage](/docs/retrieval/retrievers#apply-stages) | Dynamic classification, A/B testing taxonomy versions, low-volume queries |
| **Materialized** | During collection processing (post-extraction) | Production search requiring low latency, analytics dashboards |
| **Retroactive** | Manually triggered via API | Backfilling existing documents, applying updated taxonomy versions |

**On-demand** enrichment adds a `taxonomy_enrich` stage to your retriever pipeline:

```json theme={null}
{
  "stage_name": "taxonomy_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "taxonomy_enrich",
    "parameters": {
      "taxonomy_id": "tax_org_roles",
      "min_score": 0.4
    }
  }
}
```

**Materialized** enrichment runs automatically after document extraction completes. Configure it in the collection's `taxonomy_applications`:

```json theme={null}
{
  "taxonomy_applications": [
    {
      "taxonomy_id": "tax_product_hierarchy",
      "execution_mode": "materialize"
    }
  ]
}
```

**Retroactive** enrichment applies a taxonomy to documents already in the collection:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections/<collection_id>/apply-taxonomy" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "taxonomy_id": "tax_product_hierarchy",
    "scroll_filters": {
      "field": "metadata.needs_reclassification",
      "operator": "eq",
      "value": true
    }
  }'
```

Use retroactive application when:

* You've updated a taxonomy and need to reclassify existing documents
* You're migrating from a flat taxonomy to a hierarchical one
* You've added new reference data to taxonomy collections

## Internals: JOIN Stage

Taxonomies reuse the `join@v1` stage under the hood:

* **Direct join** – key-based match (`join_type: "direct"`).
* **Retriever join** – similarity match using a nested retriever (`join_type: "retriever"`).
* **Join strategies** – `replace`, `enrich`, `left`, or `append` control how fields merge.

Parallel execution (`asyncio.gather`) makes retrieval joins 10–50× faster than sequential lookups.

## Create a Flat Taxonomy

Only `taxonomy_name` and `config` are required at the top level. Everything that
describes the taxonomy goes inside `config`.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/taxonomies" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "taxonomy_name": "employee_faces",
    "config": {
      "taxonomy_type": "flat",
      "retriever_id": "ret_face_matcher",
      "input_mappings": [
        { "input_key": "query", "source_type": "payload", "path": "text" }
      ],
      "source_collection": {
        "collection_id": "col_employee_embeddings",
        "enrichment_fields": [
          { "field_path": "name", "merge_mode": "replace" },
          { "field_path": "department", "merge_mode": "replace" }
        ]
      }
    }
  }'
```

`retriever_id` and `input_mappings` are required for a flat taxonomy.

> [!NOTE]
>
> **`input_mappings` maps a retriever input key to where the value comes from.**
> Each entry is `{ input_key, source_type, path }`. `input_key` must be a key the
> retriever's own `input_schema` declares. Use `path` as a dot-path for
> `payload` and `vector` sources, and `override` to pass a static value for
> `literal`.
>
> It does **not** take a feature URI. A `feature_uri` identifies a vector index,
> not an input source.

> [!WARNING]
>
> **A collection's `retriever_enrichments[].input_mappings` is a different shape
> with a different vocabulary, despite the identical field name.** Reaching for
> the wrong one produces a body that is accepted and ignored.
>
> | | Taxonomy `config.input_mappings` | Collection `retriever_enrichments[].input_mappings` |
> | - | - | - |
> | Entry | `{ input_key, source_type, path, override }` | `{ input_key, source: { source_type, path, value } }` |
> | `source_type` | `payload`, `literal`, `vector`, `blob` | `document_field`, `constant`, `source_blob` |
>
> The taxonomy form is flat; the enrichment form nests everything but `input_key`
> under `source`. The two `source_type` vocabularies share no values, so a wrong
> guess is rejected rather than silently misread. A wrong *shape* is not: an
> entry sent flat to an enrichment fails its `source` requirement.

> [!WARNING]
>
> Every `enrichment_fields[].field_path` must already exist in the source
> collection's output schema. These fields are copied **from** the source, so a
> path that is not there returns a 422 listing the fields that are.
>
> `merge_mode` is `replace` or `append`.

## Create a Hierarchical Taxonomy

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/taxonomies" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "taxonomy_name": "org_roles",
    "config": {
      "taxonomy_type": "hierarchical",
      "retriever_id": "ret_face_matcher",
      "input_mappings": [
        { "input_key": "query", "source_type": "payload", "path": "text" }
      ],
      "hierarchical_nodes": [
        {
          "collection_id": "col_employee_embeddings",
          "parent_collection_id": null
        },
        {
          "collection_id": "col_executives",
          "parent_collection_id": "col_employee_embeddings",
          "retriever_id": "ret_executive_face"
        }
      ]
    }
  }'
```

Hierarchical nodes inherit parent enrichment properties; children can override or extend them.

> [!NOTE]
>
> **A hierarchy is a flat list, not nested JSON.** Nodes never contain child
> nodes. Each node names its parent with `parent_collection_id`, `null` marks a
> root, and the tree is rebuilt from those pointers. Order does not matter.
>
> `collection_id` is the only required field on a node, and it identifies the
> node. There is no separate node id.
>
> On a hierarchical taxonomy, `retriever_id` and `input_mappings` are optional at
> the `config` level and can be set per node. You can also let Mixpeek infer the
> shape with `inference_strategy` (`schema`, `cluster`, or `llm`) plus
> `inference_collections`, instead of listing nodes by hand.

## Attach to a Collection

`taxonomy_applications[].execution_mode` only accepts `materialize`. To attach a taxonomy for materialized enrichment:

```json theme={null}
{
  "taxonomy_applications": [
    {
      "taxonomy_id": "tax_employee_faces",
      "execution_mode": "materialize"
    }
  ]
}
```

Materialized enrichment updates documents \~30 seconds after ingestion completes (debounced to avoid thrashing). For on-demand classification, add a `taxonomy_enrich` stage to a retriever instead (see [Application Methods](#application-methods) above); do not put the taxonomy in `taxonomy_applications`.

## Test On Demand

`taxonomy_enrich` is an `apply`-category stage: it enriches documents already flowing through the retriever, so it needs a document-producing stage (for example `feature_search`) before it in the pipeline, not a standalone call. Add it to a retriever as shown under [Application Methods](#application-methods) above, then run `POST /v1/retrievers/{retriever_id}/execute`.

## Inference Strategies

* **Manual** – Define nodes explicitly (IDs, collections, retrievers).
* **Schema-based** – Infer nodes from existing collection schemas (planned).
* **Cluster-based** – Create nodes from clustering output.
* **LLM-based** – Generate hierarchical structure from sample documents.

Combining strategies is encouraged: bootstrap via inference, then fine-tune manually.

## Monitoring

* List taxonomies: `POST /v1/taxonomies/list`
* Inspect hierarchy and node metadata: `GET /v1/taxonomies/{id}?expand_nodes=true`
* Track materialized enrichment progress via webhook events (`collection.documents.written`)
* Use retriever analytics to ensure taxonomy stages don’t dominate latency.

## Best Practices

1. **Start flat** for quick wins; layer hierarchies once value is proven.
2. **Keep enrichment minimal**—copy only fields needed at query time.
3. **Cache taxonomy stages** in retrievers when reference collections rarely change.
4. **Version taxonomies** (via snapshots) before major structural changes.
5. **Combine with clusters** to discover candidate nodes and measure coverage.

Taxonomies let you inject domain knowledge into multimodal search—link documents to canonical entities without relying on brittle key joins.


# Claude Code Skill
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/claude-skill

Stand up a complete Mixpeek namespace, buckets, collections, retrievers, taxonomies, clusters, alerts, and triggers from a single slash command

The `/mixpeek` Claude Code skill is a setup wizard that turns a plain-English description of your data into a fully-configured Mixpeek workspace. Run it once, answer nine questions, and every resource is created for you via the API.

> [!NOTE]
>
> **What is a Claude Code skill?** Skills are slash commands that extend [Claude Code](https://claude.ai/code) — Anthropic's CLI for AI-assisted development. A skill is a markdown file saved to `~/.claude/commands/` that gives Claude a specialized prompt. Install once, use from any session.

***

## Install

One-liner install from the public Gist:

```bash theme={null}
mkdir -p ~/.claude/commands && curl -o ~/.claude/commands/mixpeek.md \
  https://gist.githubusercontent.com/esteininger/95a3d92dbae12177367cb8c13126f029/raw/mixpeek.md
```

Or copy the full skill content below manually into `~/.claude/commands/mixpeek.md`:

> **View full skill file — copy this into \~/.claude/commands/mixpeek.md**
>
> ````markdown theme={null}
> ---
> description: Set up Mixpeek resources from scratch — namespace, buckets, collections, retrievers, taxonomies, clusters, alerts, triggers, and webhooks — via a guided interview about your data and goals
> allowed-tools: Bash
> argument-hint: [setup|status] [--api-key KEY]
> ---
>
> # /mixpeek — Mixpeek Resource Setup Wizard
>
> You are a Mixpeek setup assistant. Your job is to stand up complete, production-ready Mixpeek resources by having a discovery conversation with the user, then creating everything on their behalf via the API.
>
> ---
>
> ## Step 1 — API Key
>
> The user's request: **$ARGUMENTS**
>
> Check if an API key was passed in the arguments. Otherwise check the environment:
>
> ```bash
> echo "${MIXPEEK_API_KEY:-not_set}"
> ```
>
> If no key is found, ask:
> > "What's your Mixpeek API key? You can find it at https://studio.mixpeek.com → Settings → API Keys."
>
> Store as `API_KEY`. All requests go to `https://api.mixpeek.com`.
>
> ---
>
> ## Step 2 — Discovery Interview
>
> Ask these questions conversationally. You can batch related ones. Listen carefully — answers drive every resource decision.
>
> ---
>
> ### DATA SECTION
>
> **Q1 — What data?**
> "Describe your data in plain English. What are the items?
> *Examples: 'product catalog', 'security camera frames', 'support tickets', 'PDF contracts', 'social media posts with images'*"
>
> **Q2 — Multiple datasets?**
> "Do you have more than one dataset? (e.g., products AND customer reviews AND vendor images)
> If yes, describe each one separately — I'll create a separate bucket and collections for each."
>
> **Q3 — Schema per dataset**
> "For each dataset, list the field names and their types:
> - text / string — names, descriptions, titles, content
> - image — URLs pointing to photos or images
> - video — URLs pointing to video files
> - audio — URLs pointing to audio files
> - float / number — prices, scores, ratings
> - integer / count — quantities, IDs, counts
> - boolean — flags like in_stock, is_active
> - date — ISO date strings
>
> *Example: name (text), description (text), photo_url (image), price (float), in_stock (boolean)*"
>
> **Q4 — Data location**
> "Where does this data live?
> - **URLs** — I have HTTP/HTTPS links to each item
> - **S3** — AWS S3 bucket (provide bucket name + prefix)
> - **Google Drive** — folder ID or URL
> - **SharePoint / OneDrive** — site URL + folder path
> - **Snowflake** — database.schema.table
> - **Upload later** — I'll push data via API after setup"
>
> ---
>
> ### RETRIEVAL SECTION
>
> **Q5 — Search & retrieval goals**
> "What kinds of queries do you want to run? (pick all that apply)
>
> a) **Semantic text search** — 'find items matching a text query'
> b) **Image search by text** — 'find images that match a text description'
> c) **Visual similarity** — 'find images/videos similar to this image'
> d) **Cross-modal** — 'query with text and match against both text and image embeddings'
> e) **Filtered search** — 'search + filter by field values (e.g., category=electronics, price<100)'
> f) **Question answering** — 'ask natural language questions, get synthesized answers'
> g) **Re-ranking** — 'use a cross-encoder to improve result ordering'"
>
> ---
>
> ### CLASSIFICATION SECTION
>
> **Q6 — Taxonomy / classification?**
> "Do you want to automatically classify or tag your documents with labels?
>
> - **Flat taxonomy** — each document gets one or more labels from a flat list (e.g., IAB content categories, product types, sentiment labels). You provide example items per label as a reference collection.
> - **Hierarchical taxonomy** — labels have a parent-child structure (e.g., Electronics → Smartphones → iPhone). The hierarchy can be explicit or inferred from your data.
> - **None** — skip classification"
>
> If yes: "What are the labels you want to assign? List them (e.g., 'electronics, clothing, food, sports') — or describe the hierarchy."
>
> ---
>
> ### CLUSTERING SECTION
>
> **Q7 — Clustering / grouping?**
> "Do you want to automatically group similar items together?
>
> - **Vector clustering** — group by semantic/visual similarity using embeddings. Algorithm options:
>   - `hdbscan` — auto-detects number of clusters (best for unknown structure)
>   - `kmeans` — you specify number of clusters K
>   - `agglomerative` — hierarchical bottom-up grouping
> - **Attribute clustering** — group by metadata field values (e.g., group by category + brand, creating 'Electronics > Apple', 'Electronics > Samsung', etc.)
> - **None** — skip clustering
>
> If clustering: Should clusters have **LLM-generated labels** (e.g., 'High-Performance Laptops' instead of 'Cluster 0')? If yes, which model? (gpt-4o-mini recommended, or claude-3-5-haiku)
>
> Should cluster labels be written back to the source documents as enrichment fields?"
>
> ---
>
> ### AUTOMATION SECTION
>
> **Q8 — Scheduled automation?**
> "Do you want any recurring automated operations?
>
> - **Re-cluster on a schedule** — re-run clustering daily/hourly as new data arrives
> - **Re-run taxonomy enrichment on a schedule** — re-classify documents periodically
> - **None** — trigger manually
>
> If yes: how often? (hourly / every 6 hours / daily at midnight / custom cron like '0 2 * * *')"
>
> ---
>
> ### ALERTS & WEBHOOKS SECTION
>
> **Q9 — Monitoring & alerts?**
> "Do you want to be notified when specific content is found or when jobs complete?
>
> - **Content alerts** — run a retriever query on new documents; notify if matches exceed a threshold (e.g., 'alert when prohibited content is detected', 'alert when competitor mentions appear')
> - **Job completion webhooks** — get notified when batches, clusters, or taxonomy jobs complete
> - **None** — skip notifications
>
> If alerts: describe what to watch for and provide a webhook URL to receive notifications.
> If webhooks: provide a URL and select event types (batch.completed, cluster.execution.completed, alert.triggered, etc.)"
>
> ---
>
> ## Step 3 — Design the Resource Plan
>
> Use the user's answers to determine exactly what to create. Apply these rules:
>
> ### Namespace Extractors
> - Any dataset has text fields → `text_extractor@v1`
> - Any dataset has image fields → `image_extractor@v1`
> - Any dataset has video fields → `image_extractor@v1` (video frames are images)
> - Include all that apply
>
> ### Buckets (one per dataset)
> Map field types to bucket schema types:
> - text/string/description/title/content → `"type": "string"`
> - image/photo/picture (URL) → `"type": "image"`
> - video (URL) → `"type": "string"` (stored as URL reference)
> - float/number/price/score → `"type": "float"`
> - integer/count/quantity → `"type": "integer"`
> - boolean → `"type": "string"` (serialize as "true"/"false")
> - date/datetime → `"type": "string"` (ISO-8601 format)
>
> ### Collections (one per extractor type per dataset)
> - Text field(s) in dataset → `{dataset}-text` collection with `text_extractor@v1`, `input_mappings: {"text": "field_name"}`
> - Image field in dataset → `{dataset}-images` collection with `image_extractor@v1`, `input_mappings: {"image": "image_url_field"}`
> - `field_passthrough`: all fields except the extractor input (those are stored as payload)
>
> ### Retrievers (from Q5)
> - Semantic text search → `feature_search` stage, `input_mode: "text"`, text_extractor URI
> - Image search by text → `feature_search` stage, `input_mode: "text"`, image_extractor URI
> - Visual similarity → `feature_search` stage, `input_mode: "content"`, image_extractor URI, `value: "{{INPUT.image_url}}"`
> - Cross-modal → `feature_search` stage with multiple searches (text + image URIs), fusion: "rrf"
> - Filtered search → add `attribute_filter` stage after feature_search
> - Q&A → `feature_search` + `llm_filter` stages chained
> - Re-ranking → add `rerank` stage after feature_search
>
> Default feature URIs (may be overridden post-batch):
> - Text: `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1`
> - Image: `mixpeek://image_extractor@v1/google_siglip_base_v1`
>
> **Always auto-detect actual URIs** from the collection's `vector_indexes` before creating retrievers.
>
> ### Taxonomies (from Q6)
> Flat taxonomy needs:
> - A **reference collection** — embeddings of the label examples (created from a label bucket)
> - A **retriever** that searches the reference collection
> - A **source collection** — the collection to enrich with labels
> - `input_mappings` — how to extract the query from source documents
>
> Hierarchical taxonomy:
> - Same structure, but `taxonomy_type: "hierarchical"` with `hierarchy` dict (child_collection_id → parent_collection_id)
> - Or use `inference_strategy: "llm"` with `inference_collections` to auto-infer hierarchy
>
> ### Clusters (from Q7)
> Vector cluster: `cluster_type: "vector"`, `vector_config: {feature_uris: [...], clustering_method: "hdbscan"|"kmeans", ...}`
> Attribute cluster: `cluster_type: "attribute"`, `attribute_config: {attributes: ["field1", "field2"], hierarchical_grouping: true|false}`
> LLM labeling: include `llm_labeling: {enabled: true, model_name: "gpt-4o-mini-2024-07-18", provider: "openai"}`
> Enrich source: `enrich_source_collection: true` to write cluster_id/label back to documents
>
> ### Triggers (from Q8)
> For clusters: `action_type: "cluster"`, `action_config: {cluster_id: "..."}`, `trigger_type: "cron"|"interval"`
> For taxonomy enrichment: `action_type: "taxonomy_enrichment"`, `action_config: {taxonomy_id: "...", collection_id: "..."}`
> Cron schedule: `schedule_config: {cron_expression: "0 2 * * *", timezone: "UTC"}`
> Interval: `schedule_config: {interval_seconds: 3600}` (hourly)
>
> ### Alerts (from Q9)
> Alert references a retriever (the search logic lives there). When the retriever returns results, the alert fires.
> Notification channels:
> - Inline webhook: `{channel_type: "webhook", config: {url: "https://..."}}`
> - Slack: `{channel_type: "slack", config: {channel: "#alerts"}}`
> - Email: `{channel_type: "email", config: {to: ["admin@example.com"]}}`
>
> ### Webhooks (from Q9)
> `POST /v1/organizations/webhooks/` with `webhook_name`, `event_types`, `channels: [{channel_type: "webhook", config: {url: "..."}}]`
> Event types: `object.created`, `collection.documents.written`, `cluster.execution.completed`, `cluster.execution.failed`, `trigger.execution.completed`, `trigger.execution.failed`, `alert.triggered`, `taxonomy.created`
>
> ---
>
> ## Step 4 — Show the Plan & Confirm
>
> Present a clear resource tree before creating anything:
>
> ```
> 📋 MIXPEEK SETUP PLAN — {project-name}
> ══════════════════════════════════════════════════════
>
> NAMESPACE: {project-name}
>   Extractors: text_extractor@v1, image_extractor@v1
>
> DATASET 1: {dataset1-name}
>   BUCKET: {dataset1-name}-data
>     Schema: field1 (string), field2 (image), field3 (float)
>   COLLECTION: {dataset1-name}-text
>     Extractor: text_extractor@v1  ← {text_field}
>     Passthrough: field1, field2, field3
>   COLLECTION: {dataset1-name}-images
>     Extractor: image_extractor@v1  ← {image_field}
>     Passthrough: field1, field2, field3
>
> RETRIEVER: {project-name}-search
>   Stage 1: feature_search (text + image, RRF)
>   Input: query (text)
>
> TAXONOMY: {project-name}-categories  [if classification requested]
>   Type: flat
>   Labels: electronics, clothing, food, ...
>   Source: {collection-id}
>
> CLUSTER: {project-name}-vector-clusters  [if vector clustering requested]
>   Algorithm: hdbscan
>   Feature: text_extractor URI
>   LLM labels: enabled (gpt-4o-mini)
>   Enrich source: yes → cluster_id, cluster_label
>
> TRIGGER: daily-recluster  [if automation requested]
>   Action: cluster → {cluster-id}
>   Schedule: cron "0 2 * * *" (daily at 2am UTC)
>
> ALERT: {alert-name}  [if monitoring requested]
>   Retriever: {retriever-id}
>   Notify: webhook → https://your-endpoint.com/hook
>
> WEBHOOK: job-notifications  [if webhooks requested]
>   Events: cluster.execution.completed, batch.completed
>   URL: https://your-endpoint.com/events
>
> ══════════════════════════════════════════════════════
> ```
>
> Ask: **"Does this look right? (yes / adjust X / skip Y)"**
>
> Wait for confirmation. Let the user adjust before creating.
>
> ---
>
> ## Step 5 — Create the Resources
>
> Use Python 3 with `httpx` (fallback to `requests` if needed). Run each as an inline script. Capture IDs from outputs.
>
> ### 5a — Namespace
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> API_KEY = "REPLACE_API_KEY"
> BASE = "https://api.mixpeek.com"
> PROJECT = "REPLACE_PROJECT_NAME"
>
> extractors = [
>     {"feature_extractor_name": "text_extractor", "version": "v1"},
>     # {"feature_extractor_name": "image_extractor", "version": "v1"},
> ]
>
> headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
> resp = httpx.post(f"{BASE}/v1/namespaces", headers=headers, json={
>     "namespace_name": PROJECT,
>     "feature_extractors": extractors,
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> data = resp.json()
> print(f"namespace_id={data['namespace_id']}")
> PYEOF
> ```
>
> Capture `namespace_id`. All subsequent requests include `X-Namespace: {namespace_id}`.
>
> ### 5b — Bucket (repeat for each dataset)
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> API_KEY = "REPLACE_API_KEY"
> BASE = "https://api.mixpeek.com"
> NS_ID = "REPLACE_NAMESPACE_ID"
> DATASET = "REPLACE_DATASET_NAME"
>
> headers = {
>     "Authorization": f"Bearer {API_KEY}",
>     "X-Namespace": NS_ID,
>     "Content-Type": "application/json",
> }
>
> schema_properties = {
>     # "field_name": {"type": "string"},
>     # "image_url": {"type": "image"},
>     # "price": {"type": "float"},
> }
>
> resp = httpx.post(f"{BASE}/v1/buckets", headers=headers, json={
>     "bucket_name": f"{DATASET}-data",
>     "bucket_schema": {"properties": schema_properties},
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> print(f"bucket_id={resp.json()['bucket_id']}")
> PYEOF
> ```
>
> ### 5c — Data Source Setup (if not manual upload)
>
> **S3 sync:**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json
>
> headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
> conn_resp = httpx.post(f"{BASE}/v1/organizations/connections", headers=headers, json={
>     "name": "s3-source",
>     "provider_type": "s3",
>     "provider_config": {
>         "bucket": "REPLACE_S3_BUCKET",
>         "region": "us-east-1",
>         "prefix": "",
>     },
>     "test_before_save": True,
> })
> print("connection:", conn_resp.json().get("connection_id"))
>
> headers["X-Namespace"] = NS_ID
> sync_resp = httpx.post(f"{BASE}/v1/buckets/{BUCKET_ID}/syncs", headers=headers, json={
>     "connection_id": conn_resp.json()["connection_id"],
>     "source_path": "optional/prefix/",
>     "sync_mode": "continuous",
>     "polling_interval_seconds": 3600,
> })
> print("sync_id:", sync_resp.json().get("sync_config_id"))
> PYEOF
> ```
>
> **If URLs (manual):** tell the user to `POST /v1/buckets/{bucket_id}/objects` with:
> ```json
> {
>   "field1": "value",
>   "blobs": [
>     {"property": "image_url", "type": "image", "data": "https://..."},
>     {"property": "description", "type": "text", "data": "text content here"}
>   ]
> }
> ```
>
> ### 5d — Collections (repeat for each extractor type per dataset)
>
> **Text collection:**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> resp = httpx.post(f"{BASE}/v1/collections", headers=headers, json={
>     "collection_name": f"{DATASET}-text",
>     "source": {"type": "bucket", "bucket_ids": [BUCKET_ID]},
>     "feature_extractor": {
>         "feature_extractor_name": "text_extractor",
>         "version": "v1",
>         "input_mappings": {"text": "REPLACE_TEXT_FIELD"},
>         "parameters": {},
>         "field_passthrough": ["REPLACE_ALL_OTHER_FIELDS"],
>     },
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> print(f"collection_id={resp.json()['collection_id']}")
> PYEOF
> ```
>
> **Image collection:**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> resp = httpx.post(f"{BASE}/v1/collections", headers=headers, json={
>     "collection_name": f"{DATASET}-images",
>     "source": {"type": "bucket", "bucket_ids": [BUCKET_ID]},
>     "feature_extractor": {
>         "feature_extractor_name": "image_extractor",
>         "version": "v1",
>         "input_mappings": {"image": "REPLACE_IMAGE_URL_FIELD"},
>         "parameters": {},
>         "field_passthrough": ["REPLACE_ALL_OTHER_FIELDS"],
>     },
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> print(f"collection_id={resp.json()['collection_id']}")
> PYEOF
> ```
>
> ### 5e — Retrievers
>
> **Semantic text search:**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> TEXT_URI = "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
>
> resp = httpx.post(f"{BASE}/v1/retrievers", headers=headers, json={
>     "retriever_name": f"{PROJECT}-search",
>     "collection_identifiers": [TEXT_COLLECTION_ID],
>     "stages": [{
>         "stage_name": "semantic_search",
>         "stage_type": "filter",
>         "config": {
>             "stage_id": "feature_search",
>             "parameters": {
>                 "searches": [{
>                     "feature_uri": TEXT_URI,
>                     "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
>                     "top_k": 10,
>                 }],
>                 "final_top_k": 5,
>                 "fusion": "rrf",
>                 "collection_identifiers": [TEXT_COLLECTION_ID],
>             },
>         },
>     }],
>     "input_schema": {"query": {"type": "text", "description": "Search query"}},
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> print(f"retriever_id={resp.json()['retriever']['retriever_id']}")
> PYEOF
> ```
>
> **Cross-modal (text query → text + image results, RRF fusion):**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> TEXT_URI = "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
> IMAGE_URI = "mixpeek://image_extractor@v1/google_siglip_base_v1"
> ALL_COLLECTIONS = [TEXT_COLLECTION_ID, IMAGE_COLLECTION_ID]
>
> resp = httpx.post(f"{BASE}/v1/retrievers", headers=headers, json={
>     "retriever_name": f"{PROJECT}-multimodal",
>     "collection_identifiers": ALL_COLLECTIONS,
>     "stages": [{
>         "stage_name": "multimodal_search",
>         "stage_type": "filter",
>         "config": {
>             "stage_id": "feature_search",
>             "parameters": {
>                 "searches": [
>                     {"feature_uri": TEXT_URI, "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 10},
>                     {"feature_uri": IMAGE_URI, "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 10},
>                 ],
>                 "final_top_k": 5,
>                 "fusion": "rrf",
>                 "collection_identifiers": ALL_COLLECTIONS,
>             },
>         },
>     }],
>     "input_schema": {"query": {"type": "text"}},
> })
> print(f"retriever_id={resp.json()['retriever']['retriever_id']}")
> PYEOF
> ```
>
> **Q&A retriever (retrieve + LLM synthesize):**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> TEXT_URI = "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
>
> resp = httpx.post(f"{BASE}/v1/retrievers", headers=headers, json={
>     "retriever_name": f"{PROJECT}-qa",
>     "collection_identifiers": [TEXT_COLLECTION_ID],
>     "stages": [
>         {
>             "stage_name": "retrieve_context",
>             "stage_type": "filter",
>             "config": {
>                 "stage_id": "feature_search",
>                 "parameters": {
>                     "searches": [{"feature_uri": TEXT_URI, "query": {"input_mode": "text", "value": "{{INPUT.question}}"}, "top_k": 10}],
>                     "final_top_k": 10,
>                     "fusion": "rrf",
>                     "collection_identifiers": [TEXT_COLLECTION_ID],
>                 },
>             },
>         },
>         {
>             "stage_name": "synthesize_answer",
>             "stage_type": "filter",
>             "config": {
>                 "stage_id": "llm_filter",
>                 "parameters": {
>                     "prompt": "Using only the retrieved documents, answer concisely: {{INPUT.question}}",
>                     "model": "gpt-4o-mini",
>                     "output_field": "answer",
>                 },
>             },
>         },
>     ],
>     "input_schema": {"question": {"type": "text", "description": "Question to answer from the corpus"}},
> })
> print(f"retriever_id={resp.json()['retriever']['retriever_id']}")
> PYEOF
> ```
>
> ### 5f — Batch Processing
>
> Trigger each collection separately to start feature extraction:
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json
>
> for col_id in [TEXT_COLLECTION_ID]:  # add IMAGE_COLLECTION_ID if applicable
>     r = httpx.post(f"{BASE}/v1/collections/{col_id}/trigger", headers=headers, json={}, timeout=30)
>     data = r.json()
>     print(f"  {col_id}: {r.status_code} → batch_id={data.get('batch_id')} objects={data.get('object_count')}")
> PYEOF
> ```
>
> ### 5g — Taxonomy (flat)
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> # Step 1: Reference bucket for label examples
> ref_resp = httpx.post(f"{BASE}/v1/buckets", headers=headers, json={
>     "bucket_name": f"{PROJECT}-taxonomy-labels",
>     "bucket_schema": {"properties": {"label_name": {"type": "string"}, "description": {"type": "string"}}},
> })
> ref_bucket_id = ref_resp.json()["bucket_id"]
>
> # Step 2: Upload label examples
> LABELS = [
>     # {"label_name": "electronics", "description": "consumer electronics and gadgets",
>     #  "blobs": [{"property": "description", "type": "text", "data": "consumer electronics and gadgets"}]}
> ]
> for label in LABELS:
>     httpx.post(f"{BASE}/v1/buckets/{ref_bucket_id}/objects", headers=headers, json=label)
>
> # Step 3: Reference collection
> ref_col_resp = httpx.post(f"{BASE}/v1/collections", headers=headers, json={
>     "collection_name": f"{PROJECT}-taxonomy-reference",
>     "source": {"type": "bucket", "bucket_ids": [ref_bucket_id]},
>     "feature_extractor": {
>         "feature_extractor_name": "text_extractor",
>         "version": "v1",
>         "input_mappings": {"text": "description"},
>         "parameters": {},
>         "field_passthrough": ["label_name"],
>     },
> })
> ref_col_id = ref_col_resp.json()["collection_id"]
>
> # Step 4: Taxonomy retriever
> tax_ret_resp = httpx.post(f"{BASE}/v1/retrievers", headers=headers, json={
>     "retriever_name": f"{PROJECT}-taxonomy-matcher",
>     "collection_identifiers": [ref_col_id],
>     "stages": [{"stage_name": "label_search", "stage_type": "filter", "config": {
>         "stage_id": "feature_search",
>         "parameters": {
>             "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 3}],
>             "final_top_k": 1,
>             "collection_identifiers": [ref_col_id],
>         },
>     }}],
>     "input_schema": {"query": {"type": "text"}},
> })
> tax_ret_id = tax_ret_resp.json()["retriever_id"]
>
> # Step 5: Create taxonomy
> tax_resp = httpx.post(f"{BASE}/v1/taxonomies", headers=headers, json={
>     "taxonomy_name": f"{PROJECT}-categories",
>     "description": "Automatically classify documents into predefined categories",
>     "config": {
>         "taxonomy_type": "flat",
>         "retriever_id": tax_ret_id,
>         "input_mappings": [{"input_key": "query", "source_type": "payload", "path": "REPLACE_TEXT_FIELD"}],
>         "source_collection": {
>             "collection_id": TEXT_COLLECTION_ID,
>             # enrichment_fields: only include if those fields already exist in the source schema
>         },
>     },
> })
> if tax_resp.status_code != 200:
>     print(f"ERROR {tax_resp.status_code}: {tax_resp.text}", file=sys.stderr); sys.exit(1)
> print(f"taxonomy_id={tax_resp.json()['taxonomy_id']}")
> PYEOF
> ```
>
> ### 5h — Clusters
>
> **Vector cluster (HDBSCAN + LLM labels):**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json, sys
>
> TEXT_URI = "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
>
> resp = httpx.post(f"{BASE}/v1/clusters", headers=headers, json={
>     "cluster_name": f"{PROJECT}-semantic-groups",
>     "collection_ids": [TEXT_COLLECTION_ID],
>     "cluster_type": "vector",
>     "vector_config": {
>         "feature_uris": [TEXT_URI],
>         "clustering_method": "hdbscan",
>     },
>     "llm_labeling": {"enabled": True, "provider": "openai", "model_name": "gpt-4o-mini-2024-07-18"},
>     "enrich_source_collection": True,
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> data = resp.json()
> print(f"cluster_id={data['cluster_id']}")
>
> exec_resp = httpx.post(f"{BASE}/v1/clusters/{data['cluster_id']}/execute", headers=headers, json={})
> print(f"execution_task_id={exec_resp.json().get('task_id')}")
> PYEOF
> ```
>
> ### 5i — Triggers
>
> **Daily re-cluster (cron):**
> ```bash
> python3 - <<'PYEOF'
> import httpx, json
>
> resp = httpx.post(f"{BASE}/v1/triggers", headers=headers, json={
>     "action_type": "cluster",
>     "action_config": {"cluster_id": CLUSTER_ID},
>     "trigger_type": "cron",
>     "schedule_config": {"cron_expression": "0 2 * * *", "timezone": "UTC"},
>     "description": "Re-cluster daily at 2am UTC",
> })
> # NOTE: POST /v1/triggers returns 201 Created
> if resp.status_code not in (200, 201):
>     print(f"ERROR {resp.status_code}: {resp.text}")
> else:
>     print(f"trigger_id={resp.json()['trigger_id']}")
> PYEOF
> ```
>
> ### 5j — Alerts
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json
>
> resp = httpx.post(f"{BASE}/v1/alerts", headers=headers, json={
>     "name": f"{PROJECT}-content-monitor",
>     "description": "Alert when specific content is detected in new documents",
>     "retriever_id": ALERT_RETRIEVER_ID,
>     "enabled": True,
>     "notification_config": {
>         "channels": [{"channel_type": "webhook", "config": {"url": "REPLACE_WEBHOOK_URL"}}],
>         "include_matches": True,
>         "include_scores": True,
>     },
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> print(f"alert_id={resp.json()['alert_id']}")
> PYEOF
> ```
>
> ### 5k — Webhooks
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json
>
> org_headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
>
> resp = httpx.post(f"{BASE}/v1/organizations/webhooks/", headers=org_headers, json={
>     "webhook_name": f"{PROJECT}-job-notifications",
>     "event_types": [
>         "cluster.execution.completed",
>         "cluster.execution.failed",
>         "trigger.execution.completed",
>         "trigger.execution.failed",
>         "alert.triggered",
>         "collection.documents.written",
>     ],
>     "channels": [{"channel_type": "webhook", "config": {"url": "REPLACE_WEBHOOK_URL"}}],
>     "enabled": True,
> })
> if resp.status_code != 200:
>     print(f"ERROR {resp.status_code}: {resp.text}", file=sys.stderr); sys.exit(1)
> print(f"webhook_id={resp.json()['webhook_id']}")
> PYEOF
> ```
>
> ---
>
> ## Step 6 — Auto-Detect Feature URIs
>
> After triggering the collection, confirm the actual feature URIs registered:
>
> ```bash
> python3 - <<'PYEOF'
> import httpx, json
>
> resp = httpx.get(f"{BASE}/v1/collections/{COLLECTION_ID}", headers=headers)
> for vi in resp.json().get("vector_indexes", []):
>     print(f"  vector: {vi.get('vector_name')}  uri: {vi.get('feature_uri')}")
> PYEOF
> ```
>
> If the detected URI differs from the default, patch the retriever stages accordingly.
>
> ---
>
> ## Step 7 — Final Summary
>
> After everything is created, output a complete summary:
>
> ```
> ✅ MIXPEEK SETUP COMPLETE — {project-name}
>
> ┌──────────────────────────────────────────────────────────┐
> │  Namespace:    {namespace_id}                            │
> │  Bucket:       {bucket_id}                               │
> │  Collection:   {text_col_id}   (text embeddings)         │
> │  Collection:   {image_col_id}  (image embeddings)        │
> │  Retriever:    {retriever_id}  (semantic search)         │
> │  Taxonomy:     {taxonomy_id}   (flat categories)         │
> │  Cluster:      {cluster_id}    (vector HDBSCAN)          │
> │  Trigger:      {trigger_id}    (daily re-cluster)        │
> │  Alert:        {alert_id}      (content monitor)         │
> │  Webhook:      {webhook_id}    (job notifications)       │
> └──────────────────────────────────────────────────────────┘
>
> 📡 SEARCH YOUR DATA (once batch completes):
>   curl -X POST https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute \
>     -H "Authorization: Bearer {api_key}" \
>     -H "X-Namespace: {namespace_id}" \
>     -H "Content-Type: application/json" \
>     -d '{"inputs": {"query": "your search here"}, "limit": 5}'
>
> 📚 DOCS: https://docs.mixpeek.com
> ```
>
> ---
>
> ## Error Handling
>
> For any non-200 response:
> 1. Print the full error body
> 2. Explain what went wrong in plain English
> 3. Suggest the fix
>
> Common errors:
> - `401` → bad/missing API key
> - `409 Conflict` → name already taken → ask user for a new name or offer to use the existing resource
> - `422 Unprocessable Entity` → bad request body → show the exact validation error field
> - `429 Too Many Requests` → wait 5s, retry once
> - `400` on taxonomy with `input_mappings` → check that `path` field exists in source document payload
>
> ---
>
> ## Key API Reference
>
> | Resource | Create | List | Execute |
> |----------|--------|------|---------|
> | Namespace | `POST /v1/namespaces` | `POST /v1/namespaces/list` | — |
> | Bucket | `POST /v1/buckets` | `POST /v1/buckets/list` | — |
> | Bucket Sync | `POST /v1/buckets/{id}/syncs` | `POST /v1/buckets/{id}/syncs/list` | `POST /v1/buckets/{id}/syncs/{sid}/trigger` |
> | Collection | `POST /v1/collections` | `POST /v1/collections/list` | `POST /v1/collections/{id}/trigger` |
> | Retriever | `POST /v1/retrievers` | `POST /v1/retrievers/list` | `POST /v1/retrievers/{id}/execute` |
> | Taxonomy | `POST /v1/taxonomies` | `POST /v1/taxonomies/list` | `POST /v1/collections/{id}/apply-taxonomy` |
> | Cluster | `POST /v1/clusters` | `POST /v1/clusters/list` | `POST /v1/clusters/{id}/execute` |
> | Trigger | `POST /v1/triggers` | `POST /v1/triggers/list` | `POST /v1/triggers/{id}/execute` |
> | Alert | `POST /v1/alerts` | `POST /v1/alerts/list` | — |
> | Webhook | `POST /v1/organizations/webhooks/` | `POST /v1/organizations/webhooks/list` | — |
>
> All requests except webhooks require `Authorization: Bearer {api_key}`.
> All requests except namespace creation and webhooks require `X-Namespace: {namespace_id}`.
> ````

> [!TIP]
>
> After saving the file, restart Claude Code. The `/mixpeek` command will appear in tab-complete.

***

## Usage

```
/mixpeek
```

Or pass your API key directly to skip the first prompt:

```
/mixpeek sk-mxp-...
```

***

## What It Asks

> **Q1 — What data?**
>
> Describe your dataset in plain English.
>
> *Examples: "product catalog with photos and descriptions", "security camera footage", "support tickets", "PDF contracts"*

> **Q2 — Multiple datasets?**
>
> If you have more than one dataset (e.g., products AND customer reviews AND vendor images), describe each separately. The skill creates a dedicated bucket and collection set for each.

> **Q3 — Schema**
>
> For each dataset, list field names and types:
>
> | Type | Examples |
> | - | - |
> | `text` / `string` | names, descriptions, titles, content |
> | `image` | URLs to photos |
> | `video` | URLs to video files |
> | `float` | prices, scores, ratings |
> | `integer` | quantities, IDs, counts |
> | `boolean` | in\_stock, is\_active |
> | `date` | ISO date strings |

> **Q4 — Where does the data live?**
>
> * **URLs** — HTTP/HTTPS links to each item
> * **S3** — AWS S3 bucket with optional prefix
> * **Google Drive** — folder ID or URL
> * **SharePoint / OneDrive** — site URL + folder path
> * **Snowflake** — database.schema.table
> * **Upload later** — set up the schema now, push data later via API

> **Q5 — Search & retrieval goals**
>
> Pick all that apply: semantic text search, image search by text, visual similarity, cross-modal, filtered search, question answering, re-ranking.

> **Q6 — Classification / taxonomy?**
>
> Flat (label list) or hierarchical (parent-child structure). You provide example items per label; the skill creates the reference collection and wiring automatically.

> **Q7 — Clustering / grouping?**
>
> Vector clustering (hdbscan / kmeans / agglomerative) or attribute clustering (group by field values). Optional LLM-generated cluster labels and enrichment back to source documents.

> **Q8 — Scheduled automation?**
>
> Re-cluster or re-classify on a schedule. Supports cron expressions and interval-based triggers.

> **Q9 — Monitoring & alerts?**
>
> Content alerts (notify when a retriever query matches new documents) and job completion webhooks.

***

## Resources Created

| Resource | What it does |
| - | - |
| **Namespace** | Isolated workspace; one per project |
| **Bucket(s)** | Raw data storage with typed schema |
| **Collection(s)** | Processing pipeline — one per extractor type per dataset |
| **Batch** | Triggers feature extraction across all bucket objects |
| **Retriever(s)** | Multi-stage search pipeline matching your retrieval goals |
| **Taxonomy** | Flat or hierarchical classifier applied to documents |
| **Cluster** | Groups similar documents; supports LLM-generated labels |
| **Trigger** | Scheduled re-clustering or taxonomy enrichment |
| **Alert** | Fires a webhook when a retriever query matches new content |
| **Webhook** | Event notifications for job completion, object creation, etc. |

***

## Requirements

* [Claude Code](https://claude.ai/code) installed
* A Mixpeek API key from [studio.mixpeek.com](https://studio.mixpeek.com) → Settings → API Keys
* Python 3 with `httpx` (`pip install httpx`)

***

## Next Steps

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand namespaces, collections, and documents

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Choose the right extractor for your data type

> **[Retriever Stages](/docs/retrieval/stages/overview)**
>
> Build custom multi-stage search pipelines

> **[MCP Server](/docs/integrations/developer-tools/mcp-server)**
>
> Connect Claude to Mixpeek via MCP for ongoing management


# IAB Tech Lab buyer agent
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/iab-buyer-agent

Connect IAB Tech Lab's reference buyer agent to Mixpeek for IAB Content Taxonomy 3.1 classification, brand-safety checks and contextual search.

IAB Tech Lab's [buyer agent](https://github.com/IABTechLab/buyer-agent) includes a Mixpeek client. With it configured, the agent gets three MCP tools:

| Tool | What it does |
| - | - |
| `classify_content` | Classifies text into IAB Content Taxonomy categories, returning name, tier, path and score |
| `check_brand_safety` | Classifies the text and flags matches against a list of sensitive categories |
| `contextual_search` | Runs any retriever in your namespace over indexed inventory |

## Configure the agent

Set these in the buyer agent's `.env`:

```bash theme={null}
MIXPEEK_API_KEY=your_api_key
MIXPEEK_BASE_URL=https://api.mixpeek.com
MIXPEEK_NAMESPACE=your_namespace_id
```

`classify_content` and `check_brand_safety` need a retriever over IAB categories. If you do not pass `retriever_id`, the agent uses a retriever in the namespace whose name contains both `iab` and `text`.

## Build the IAB namespace

The classifier is semantic search over the taxonomy: each IAB Content Taxonomy 3.1 category becomes a document whose text is its name and path, embedded with the text extractor. Each document must keep `iab_category_name`, `iab_tier` and `iab_path`, which are the fields the buyer agent reads from the results.

1. [Create a namespace](/docs/api-reference/namespaces/create-namespace) with the `text_extractor`.
2. Set a [spend budget](/docs/platform/budgets) on it. A new namespace starts at \$0.00 per day, and the embedding batch fails on that budget.
3. [Create a bucket](/docs/api-reference/buckets/create-bucket) and upload one object per category, with the text in a `category_text` blob and `iab_category_id`, `iab_category_name`, `iab_tier` and `iab_path` as fields. Leave `unique_key` off the bucket if you load it with the batch endpoint, which refuses buckets that declare one.
4. [Create a collection](/docs/api-reference/collections/create-collection) with `text_extractor`, `input_mappings: {"text": "category_text"}` and `field_passthrough` for the four `iab_` fields, then trigger it.
5. [Create a retriever](/docs/api-reference/retrievers/create-retriever) named `iab_text_search` with one `feature_search` stage on `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1`, querying `{{INPUT.query}}`.

A complete, tested script is in the guide [Using Mixpeek in the IAB Tech Lab buyer agent](https://mixpeek.com/guides/iab-buyer-agent-contextual-classification).

## Behavior to know

* The number of results comes from the retriever's `final_top_k`. The tools' `limit` argument does not reduce it.
* `check_brand_safety` matches categories by exact name. At buyer-agent commit `47ac319`, seven of its fourteen names are not Content Taxonomy 3.1 names, so content in those areas is classified but not flagged. Compare the returned `categories` against your own list of 3.1 names if you rely on the flag.
* `risk_level` is `high` only when a flagged score is 0.85 or above.


# JavaScript SDK
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/javascript-sdk

Official TypeScript/JavaScript SDK for the Mixpeek API

> [!NOTE]
>
> The Mixpeek JavaScript SDK is auto-generated from our OpenAPI specification and always stays in sync with the latest API features.

## Features

* **100% Type-Safe** - Built with TypeScript for complete type safety
* **Runtime Validation** - Zod schemas for request/response validation
* **Modern** - Supports both CommonJS and ESM
* **Promise-Based** - Clean async/await API
* **Developer-Friendly** - Intuitive method names and excellent autocomplete

## Installation

```bash npm theme={null}
npm install mixpeek
```

```bash yarn theme={null}
yarn add mixpeek
```

```bash pnpm theme={null}
pnpm add mixpeek
```

## Quick Start

```typescript theme={null}
import { Mixpeek } from 'mixpeek';

// Initialize the client
const client = new Mixpeek({
  apiKey: process.env.MIXPEEK_API_KEY,
  namespace: 'my-namespace'
});

// List collections
const collections = await client.collections.listCollections();
console.log('Collections:', collections);

// Create a collection
const newCollection = await client.collections.createCollection({
  alias: 'my-collection',
  description: 'My first collection'
});

// Execute a retriever
const results = await client.retrievers.executeRetriever({
  retrieverId: 'ret_abc123',
  query: 'find relevant documents'
});
```

## Configuration

### Environment Variables

```bash theme={null}
# Required
MIXPEEK_API_KEY=mxp_sk_your_api_key_here

# Optional
MIXPEEK_BASE_URL=https://api.mixpeek.com  # Default
MIXPEEK_NAMESPACE=default                 # Default
```

### Constructor Options

```typescript theme={null}
const client = new Mixpeek({
  apiKey: 'sk_...',              // Required (or set MIXPEEK_API_KEY)
  baseUrl: 'https://...',        // Optional: custom API endpoint
  namespace: 'my-namespace',     // Optional: namespace for isolation
  timeout: 30000,                // Optional: request timeout in ms
  axiosConfig: {                 // Optional: additional axios config
    // Any axios configuration options
  }
});
```

> [!TIP]
>
> You can create API keys in the Mixpeek dashboard under Organization Settings.

## Core Operations

### Collections

```typescript theme={null}
// Create a collection
const collection = await client.collections.createCollection({
  alias: 'my-collection',
  description: 'Store multimodal documents',
  metadata: { project: 'demo' }
});

// Get a collection
const retrieved = await client.collections.getCollection({
  collectionIdentifier: 'my-collection'
});

// List all collections
const allCollections = await client.collections.listCollections();

// Delete a collection
await client.collections.deleteCollection({
  collectionIdentifier: 'my-collection'
});
```

### Retrievers

```typescript theme={null}
// Create a retriever
const retriever = await client.retrievers.createRetriever({
  retrieverName: 'semantic-search',
  description: 'Search across all documents',
  collectionIdentifiers: ['my-collection'],
  stages: [
    {
      type: 'embed',
      model: 'openai-text-embedding-3-small'
    },
    {
      type: 'vector_search',
      top_k: 10
    }
  ]
});

// Execute a retriever
const results = await client.retrievers.executeRetriever({
  retrieverId: retriever.retrieverId,
  query: 'find relevant documents about AI'
});

// List retrievers
const retrievers = await client.retrievers.listRetrievers();
```

### Documents

```typescript theme={null}
// Upload documents to a collection
const documents = await client.documents.uploadDocuments({
  collectionId: 'col_abc123',
  documents: [
    {
      url: 's3://bucket/video.mp4',
      metadata: { title: 'Demo Video' }
    },
    {
      url: 's3://bucket/image.jpg',
      metadata: { title: 'Demo Image' }
    }
  ]
});

// Search documents
const searchResults = await client.documents.searchDocuments({
  collectionId: 'col_abc123',
  query: 'search query',
  limit: 20
});
```

### Buckets (Object Storage)

```typescript theme={null}
// Create a bucket
const bucket = await client.buckets.createBucket({
  alias: 'my-bucket',
  provider: 's3',
  credentials: {
    accessKeyId: 'YOUR_ACCESS_KEY',
    secretAccessKey: 'YOUR_SECRET_KEY',
    region: 'us-east-1'
  }
});

// List buckets
const buckets = await client.buckets.listBuckets();
```

## Error Handling

```typescript theme={null}
try {
  const collection = await client.collections.getCollection({
    collectionIdentifier: 'non-existent'
  });
} catch (error) {
  if (error.response) {
    // API error
    console.error('Status:', error.response.status);
    console.error('Message:', error.response.data?.error?.message);
  } else if (error.request) {
    // Network error
    console.error('Network error:', error.message);
  } else {
    // Other error
    console.error('Error:', error.message);
  }
}
```

## TypeScript Support

The SDK is built with TypeScript and provides full type definitions:

```typescript theme={null}
import { Mixpeek, MixpeekOptions } from 'mixpeek';
import type {
  Collection,
  Retriever,
  CreateCollectionRequest,
  CreateRetrieverRequest
} from 'mixpeek';

// All types are fully typed
const options: MixpeekOptions = {
  apiKey: 'sk_...',
  namespace: 'default'
};

const client = new Mixpeek(options);

// TypeScript will autocomplete and type-check all methods
const collection: Collection = await client.collections.createCollection({
  alias: 'typed-collection'
  // TypeScript will suggest all available fields
});
```

## Advanced Usage

### Custom Axios Configuration

```typescript theme={null}
const client = new Mixpeek({
  apiKey: 'sk_...',
  axiosConfig: {
    timeout: 60000,
    headers: {
      'X-Custom-Header': 'value'
    },
    proxy: {
      host: 'proxy.example.com',
      port: 8080
    }
  }
});
```

### Updating Configuration

```typescript theme={null}
const client = new Mixpeek({ apiKey: 'sk_old' });

// Update API key
client.setApiKey('sk_new');

// Update namespace
client.setNamespace('new-namespace');

// Get current configuration
const config = client.getConfig();
console.log(config);
// { apiKey: 'sk_new...', baseUrl: '...', namespace: 'new-namespace' }
```

## Framework Examples

### Next.js

```typescript theme={null}
// app/api/search/route.ts
import { Mixpeek } from 'mixpeek';
import { NextResponse } from 'next/server';

const client = new Mixpeek({
  apiKey: process.env.MIXPEEK_API_KEY!,
  namespace: process.env.MIXPEEK_NAMESPACE
});

export async function POST(request: Request) {
  const { query } = await request.json();

  try {
    const results = await client.retrievers.executeRetriever({
      retrieverId: 'ret_abc123',
      query
    });

    return NextResponse.json(results);
  } catch (error) {
    return NextResponse.json(
      { error: 'Search failed' },
      { status: 500 }
    );
  }
}
```

### Express

```typescript theme={null}
import express from 'express';
import { Mixpeek } from 'mixpeek';

const app = express();
const client = new Mixpeek({
  apiKey: process.env.MIXPEEK_API_KEY,
  namespace: process.env.MIXPEEK_NAMESPACE
});

app.post('/api/search', async (req, res) => {
  try {
    const results = await client.retrievers.executeRetriever({
      retrieverId: 'ret_abc123',
      query: req.body.query
    });
    res.json(results);
  } catch (error) {
    res.status(500).json({ error: 'Search failed' });
  }
});

app.listen(3000);
```

## Resources

> **[NPM Package](https://www.npmjs.com/package/mixpeek)**
>
> View on NPM

> **[API Reference](/docs/api-reference)**
>
> Complete API documentation

> **[OpenAPI Spec](https://api.mixpeek.com/docs/openapi.json)**
>
> OpenAPI specification

## Next Steps

> **[Quickstart](/docs/overview/quickstart)**
>
> Get started with Mixpeek

> **[Python SDK](/docs/integrations/developer-tools/python-sdk)**
>
> Use the Python SDK

> **[MCP Server](/docs/integrations/developer-tools/mcp-server)**
>
> Use Mixpeek with AI assistants

> **[Examples](https://mixpeek.com/recipes)**
>
> View example implementations


# MCP Server
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/mcp-server

Connect Claude and AI assistants to Mixpeek via the Model Context Protocol

The Mixpeek MCP server lets AI assistants like Claude manage your entire Mixpeek workflow — creating namespaces, uploading files, building search pipelines, and querying results — all through natural language.

> [!NOTE]
>
> **What is MCP?** The [Model Context Protocol](https://modelcontextprotocol.io) is an open standard that lets AI assistants connect to external tools and data sources. Instead of copy-pasting API calls, you describe what you want and Claude handles the rest.

## Choose Your Server

MCP clients have context limits. Instead of loading all 67 tools, pick the scoped server that matches your workflow:

> **Ingestion Server**
>
> **27 tools**: buckets, collections, documents, and storage connections.
> Best for data pipelines and content upload workflows.
>
> `https://mcp.mixpeek.com/ingestion/mcp`

> **Retrieval Server**
>
> **19 tools**: retrievers, agents, marketplace, and search.
> Best for RAG applications, search UIs, and agent workflows.
>
> `https://mcp.mixpeek.com/retrieval/mcp`

> **Admin Server**
>
> **23 tools**: namespaces, taxonomies, clusters, and storage connections.
> Best for platform administration and enrichment.
>
> `https://mcp.mixpeek.com/admin/mcp`

> **Full Platform**
>
> **67 tools**: everything in one server.
> Best for power users who need all capabilities.
>
> `https://mcp.mixpeek.com/mcp`

> **App Server (OAuth)**
>
> **9 tools**: read-only browse and search.
> Sign in with your Mixpeek account instead of pasting an API key.
>
> `https://mcp.mixpeek.com/app/mcp`. See [Sign in with OAuth](/docs/agent-integrations/mcp#sign-in-with-oauth).

> [!TIP]
>
> Need just one retriever? Use the [Per-Retriever Server](#retriever-server) — it exposes a single typed `search` tool with parameters generated from your retriever's input schema.

***

## Setup

Add to your Claude Desktop or Claude Code config. Replace `YOUR_API_KEY` with your key from the [Mixpeek Studio](https://studio.mixpeek.com).

**Ingestion**

```json theme={null}
{
  "mcpServers": {
    "mixpeek-ingestion": {
      "url": "https://mcp.mixpeek.com/ingestion/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

**Retrieval**

```json theme={null}
{
  "mcpServers": {
    "mixpeek-retrieval": {
      "url": "https://mcp.mixpeek.com/retrieval/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

**Admin**

```json theme={null}
{
  "mcpServers": {
    "mixpeek-admin": {
      "url": "https://mcp.mixpeek.com/admin/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

**Full Platform**

```json theme={null}
{
  "mcpServers": {
    "mixpeek": {
      "url": "https://mcp.mixpeek.com/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
      }
    }
  }
}
```

### Stdio (local development)

Run the server locally with an optional `--scope` flag:

```bash theme={null}
# Full server (default)
python -m mixpeek_mcp.main --transport stdio

# Scoped servers
python -m mixpeek_mcp.main --transport stdio --scope ingestion
python -m mixpeek_mcp.main --transport stdio --scope retrieval
python -m mixpeek_mcp.main --transport stdio --scope admin
```

***

> [!NOTE]
>
> The groups below cover part of each server's tools. The complete list for every scope is in [Available Tools by Scope](/docs/agent-integrations/mcp#available-tools-by-scope).

## Ingestion Server: 27 tools

Manage buckets, collections, and documents. Use this server when building data ingestion pipelines.

> **Bucket Management (6 tools)**
>
> Buckets store your raw files (videos, images, documents) before processing.
>
> | Tool | Description |
> | - | - |
> | `create_bucket` | Create a new bucket for file storage and processing |
> | `list_buckets` | List all buckets in a namespace |
> | `get_bucket` | Get details of a specific bucket |
> | `update_bucket` | Update bucket configuration |
> | `delete_bucket` | Delete a bucket and all its objects |
> | `upload_object` | Upload an object (file) to a bucket from a URL |

> **Collection Management (7 tools)**
>
> Collections define how your data is processed — which feature extractor runs, what embeddings are generated. Each collection has exactly **one** feature extractor.
>
> | Tool | Description |
> | - | - |
> | `create_collection` | Create a collection with a feature extractor, source config, and optional taxonomy/cluster/alert applications |
> | `list_collections` | List all collections in a namespace |
> | `get_collection` | Get collection details |
> | `update_collection` | Update collection configuration |
> | `clone_collection` | Clone an existing collection with optional overrides |
> | `trigger_collection` | Trigger the processing pipeline on bucket objects |
> | `delete_collection` | Delete a collection and all its documents |
>
> **Available feature extractors:**
>
> | Extractor | Description |
> | - | - |
> | `text_extractor` | Text embeddings (multilingual E5) |
> | `image_extractor` | Image embeddings (CLIP, SigLIP) |
> | `multimodal_extractor` | Text + image joint embeddings |
> | `face_identity_extractor` | Face detection and recognition |
> | `document_graph_extractor` | Document structure extraction |
> | `sentiment_classifier` | Sentiment analysis |
> | `web_scraper` | Web page content extraction |
> | `course_content_extractor` | Video/course content processing |

> **Document Management (5 tools)**
>
> Documents are the processed records stored in your namespace with extracted features and embeddings.
>
> | Tool | Description |
> | - | - |
> | `create_document` | Create a new document in a collection |
> | `list_documents` | List documents in a collection with filters |
> | `get_document` | Get a specific document by ID |
> | `update_document` | Update a document's data |
> | `delete_document` | Delete a document from a collection |

**Example prompts:**

```
"Create a bucket called marketing-videos, then create a collection with the
multimodal extractor and trigger processing"

"Upload this CSV to my data-imports bucket: https://example.com/products.csv"
```

***

## Retrieval Server: 19 tools

Search and query your data. Use this server for RAG applications, search UIs, and agent workflows.

> **Retriever Management (7 tools)**
>
> Retrievers are multi-stage search pipelines. Chain stages together to search, filter, rerank, and enrich results.
>
> | Tool | Description |
> | - | - |
> | `create_retriever` | Create a multi-stage search pipeline |
> | `list_retrievers` | List all retrievers in a namespace |
> | `get_retriever` | Get retriever configuration and all stages |
> | `update_retriever` | Update retriever metadata (name, description, tags) |
> | `clone_retriever` | Clone an existing retriever with optional modifications |
> | `execute_retriever` | Execute a retriever with inputs and get search results |
> | `delete_retriever` | Delete a retriever |
>
> **29+ available stages** across 5 categories:
>
> | Category | Stage IDs |
> | - | - |
> | **Filter** | `feature_search`, `attribute_filter`, `llm_filter`, `query_expand`, `agent_search` |
> | **Sort** | `sort_relevance`, `sort_attribute`, `rerank`, `mmr`, `score_normalize` |
> | **Reduce** | `limit`, `group_by`, `aggregate`, `summarize`, `sample`, `deduplicate`, `cluster` |
> | **Apply** | `json_transform`, `api_call`, `web_search`, `sql_lookup`, `cross_compare`, `unwind`, `rag_prepare` |
> | **Enrich** | `llm_enrich`, `document_enrich`, `taxonomy_enrich`, `code_execution`, `web_scrape` |
>
> Stages support template variables: `{{INPUT.field}}`, `{{DOC.field}}`, `{{STAGE.field}}`, `{{CONTEXT.field}}`.

> **Agent & Conversation (4 tools)**
>
> Conversational AI sessions with retriever-backed responses. When the agent wants to run a write, `send_agent_message` stops with a `confirmation_required` event; answer it with `respond_to_agent_confirmation`.
>
> | Tool | Description |
> | - | - |
> | `create_agent_session` | Create a new conversational agent session |
> | `send_agent_message` | Send a message to an agent session and get response |
> | `get_agent_history` | Get conversation history for an agent session |
> | `respond_to_agent_confirmation` | Approve or reject the write the session paused on, and get the rest of the response |

> **Search (1 tool)**
>
> | Tool | Description |
> | - | - |
> | `search_namespace` | Search across all resources in a namespace (buckets, collections, retrievers, etc.) |

**Example prompts:**

```
"Create a retriever that does a feature search on my product-demos collection,
reranks the top 50 down to 10 with Cohere, and adds a 2-sentence summary"

"Execute my product-search retriever with query 'red running shoes under $100'"
```

***

## Admin Server: 23 tools

Manage namespaces, taxonomies, and clusters. Use this server for platform administration and data enrichment.

> **Namespace Management (5 tools)**
>
> Namespaces are isolated workspaces. Each namespace maps to its own vector namespace in [MVS](https://mixpeek.com/mvs).
>
> | Tool | Description |
> | - | - |
> | `create_namespace` | Create a new workspace for organizing collections and resources |
> | `list_namespaces` | List all namespaces in your organization |
> | `get_namespace` | Get namespace details by ID or name |
> | `update_namespace` | Update namespace configuration |
> | `delete_namespace` | Delete a namespace and all its resources |

> **Taxonomy Management (5 tools)**
>
> Taxonomies are hierarchical classification systems you can apply to documents.
>
> | Tool | Description |
> | - | - |
> | `create_taxonomy` | Create a hierarchical classification taxonomy |
> | `list_taxonomies` | List all taxonomies |
> | `get_taxonomy` | Get taxonomy details |
> | `execute_taxonomy` | Apply taxonomy classification to document data |
> | `delete_taxonomy` | Delete a taxonomy |

> **Cluster Management (4 tools)**
>
> Clusters group similar documents together for discovery and organization.
>
> | Tool | Description |
> | - | - |
> | `create_cluster` | Create a document clustering configuration |
> | `list_clusters` | List all clusters |
> | `execute_cluster` | Execute clustering algorithm on collection |
> | `delete_cluster` | Delete a cluster configuration |

**Example prompts:**

```
"Create a new namespace called production-catalog"

"Create an IAB taxonomy for content classification, then run it on my articles collection"

"Cluster the documents in my product-images collection into 10 groups"
```

***

## Retriever Server

The Retriever MCP server is a lightweight server scoped to a **single retriever**. It reads your retriever's `input_schema` at startup and generates a typed `search` tool whose parameters match exactly — so the AI assistant knows what inputs are available without any guesswork.

### Tools

| Tool | Description |
| - | - |
| `search` | Execute the retriever. Parameters are generated from the retriever's `input_schema` — including the correct field names, types, required flags, enums, and descriptions. Pagination parameters (`page`, `page_size`) are added automatically. |
| `describe` | Returns structured metadata: retriever ID, name, collections, input fields, and stage configuration. |
| `explain` | Returns a human-readable explanation of the pipeline: what each stage does, in what order. |

### How Dynamic Schema Works

When the server starts, it fetches your retriever's configuration and converts its `input_schema` into a JSON Schema for the `search` tool. For example, if your retriever has:

```json theme={null}
{
  "input_schema": {
    "query": { "type": "text", "required": true, "description": "Search query" },
    "category": {
      "type": "string",
      "required": false,
      "enum": ["electronics", "clothing", "home"],
      "description": "Filter by category"
    }
  }
}
```

The `search` tool will expose `query` (required string) and `category` (optional enum) as typed parameters — plus `page` and `page_size` for pagination.

> [!NOTE]
>
> If your retriever's `input_schema` has a field named `page` or `page_size`, the pagination parameters are automatically renamed to `_pagination_page` and `_pagination_page_size` to avoid conflicts.

### Setup

**Claude Desktop (stdio)**

```json theme={null}
{
  "mcpServers": {
    "my-search": {
      "command": "mixpeek-mcp-retriever",
      "args": [
        "--retriever-id", "ret_abc123",
        "--namespace-id", "ns_xyz789",
        "--api-key", "YOUR_API_KEY"
      ]
    }
  }
}
```

Requires the `mixpeek-mcp-retriever` CLI installed:

```bash theme={null}
pip install mixpeek  # includes the CLI
```

**HTTP (deployed)**

Run the server as an HTTP endpoint for remote access:

```bash theme={null}
mixpeek-mcp-retriever \
  --retriever-id ret_abc123 \
  --namespace-id ns_xyz789 \
  --api-key YOUR_API_KEY \
  --transport http \
  --port 8081
```

Then connect your MCP client to `http://your-host:8081`.

**HTTP Endpoints:**

| Method | Path | Description |
| - | - | - |
| `GET` | `/health` | Health check with retriever info |
| `GET` | `/tools` | List all 3 tools with schemas |
| `POST` | `/tools/call` | Execute a tool (requires `Authorization` header) |

**Environment Variables**

All settings can be configured via environment variables (prefixed with `RETRIEVER_MCP_`):

```bash theme={null}
export RETRIEVER_MCP_RETRIEVER_ID=ret_abc123
export RETRIEVER_MCP_NAMESPACE_ID=ns_xyz789
export RETRIEVER_MCP_API_KEY=YOUR_API_KEY
export RETRIEVER_MCP_TRANSPORT=stdio   # or "http"
export RETRIEVER_MCP_HTTP_PORT=8081
```

Then just run `mixpeek-mcp-retriever` with no arguments.

### Example Conversation

Once connected, you can interact naturally:

```
You: "What does this retriever do?"
Claude: [calls describe] → "This is 'Product Search' — it searches your
products collection by text query with an optional category filter,
using feature search → attribute filter → reranking."

You: "Search for wireless headphones under electronics"
Claude: [calls search with query="wireless headphones", category="electronics"]
→ Returns top 10 matching products with scores and metadata.

You: "Explain the pipeline stages"
Claude: [calls explain] → "1. feature_search: embeds your query and finds
the top 50 matches. 2. attribute_filter: filters by category field.
3. rerank: reranks with Cohere down to the top 10."
```

***

## Authentication & Security

All MCP servers use your existing Mixpeek API key with the same permissions as the REST API.

* **HTTP transport:** Pass the API key in the `Authorization: Bearer` header. The server extracts it and injects it into every tool call.
* **Stdio transport:** Set the `MIXPEEK_API_KEY` environment variable or pass `api_key` in tool arguments.
* Same RBAC permissions as the REST API
* Rate limiting per organization
* Audit logging for all operations
* TLS encryption on the hosted server

> [!WARNING]
>
> **Keep your API key secure.** Never commit keys to version control. For the Retriever Server, prefer environment variables over CLI arguments in production.

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                   Claude / AI App                       │
└──────────────┬──────────────────────────────────────────┘
               │ MCP Protocol (Streamable HTTP / Stdio)
               ▼
┌─────────────────────────────────────────────────────────┐
│              Mixpeek MCP Server                         │
│  ┌──────────┐ ┌───────────┐ ┌─────────┐ ┌──────────┐  │
│  │   Full   │ │ Ingestion │ │Retrieval│ │  Admin   │  │
│  │  / (43)  │ │ /ing (18) │ │/ret (11)│ │/adm (14) │  │
│  └────┬─────┘ └─────┬─────┘ └────┬────┘ └────┬─────┘  │
│       └──────────────┴────────────┴───────────┘        │
│                    Tool Handlers                        │
└──────────────┬──────────────────────────────────────────┘
               │                    ┌──────────────────┐
               │                    │ Retriever Server  │
               │                    │ (per-retriever)   │
               │                    │ 3 tools           │
               │                    └────────┬─────────┘
               └──────────┬─────────────────┘
                          │ Direct service calls
                          ▼
               ┌──────────────────────┐
               │   Mixpeek Services   │
               │ MongoDB · MVS        │
               │ Redis · S3 · Ray     │
               └──────────────────────┘
```

The scoped servers share the same codebase and tool handlers — scoping controls which tools are registered, not how they execute. Each scoped sub-app is mounted at its path prefix (`/ingestion`, `/retrieval`, `/admin`) while the full server handles root-level requests.

## Troubleshooting

> **Claude can't connect to the MCP server**
>
> * Verify the URL is correct (e.g. `https://mcp.mixpeek.com/ingestion/mcp`)
> * Check that the `Authorization` header format is `Bearer YOUR_API_KEY`
> * Restart Claude Desktop or Claude Code after changing config

> **Tools return 'Unauthorized' or 'Invalid API key'**
>
> * Verify your API key at [studio.mixpeek.com](https://studio.mixpeek.com)
> * Check that the key has permissions for the namespace you're accessing
> * Make sure there are no extra spaces in the key

> **Tool not found (404)**
>
> * You may be calling a tool on the wrong scoped server (e.g. `execute_retriever` on `/ingestion`)
> * Check `GET /tools` on the scoped endpoint to see available tools
> * Use the full server (`/`) if you need all tools

> **Retriever Server fails to start**
>
> * Ensure `--retriever-id` and `--namespace-id` are correct
> * Verify the API key has access to that namespace
> * Check that the retriever exists: `GET /v1/retrievers/{id}`

> **Search returns empty results**
>
> * Confirm your collection has processed documents (not just uploaded files)
> * Check that the retriever's `feature_uri` matches your collection's extractor
> * Try a broader query or remove optional filters

> **Slow response times**
>
> * Large file uploads depend on file size and network
> * Multi-stage retrievers with LLM enrichment or reranking take more time
> * Check [status.mixpeek.com](https://status.mixpeek.com) for service issues

## Next Steps

> **[Core Concepts](/docs/overview/concepts)**
>
> Understand namespaces, collections, and documents

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> Choose the right extractor for your data

> **[Retriever Stages](/docs/retrieval/stages/overview)**
>
> Learn what each pipeline stage does

> **[API Reference](/docs/api-reference)**
>
> Full REST API documentation


# Mixpeek CLI
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/mixpeek-cli

Command-line interface for building, testing, and deploying custom extractors

## Installation

```bash theme={null}
pip install mixpeek
mixpeek --version
```

## Quick Start

```bash theme={null}
# 1. Create plugin
mixpeek plugin init my_extractor --category text

# 2. Edit processors/core.py with your logic

# 3. Test locally
cd my_extractor && mixpeek plugin test

# 4. Publish
mixpeek plugin publish --namespace ns_xxx
```

See [Custom Extractors](/docs/processing/custom-extractors) for full extractor development guide.

## Configuration

```bash theme={null}
export MIXPEEK_API_KEY="mxp_sk_your_api_key"
export MIXPEEK_NAMESPACE="ns_your_namespace"
```

| Option | Environment Variable | Description |
| - | - | - |
| `--api-key` | `MIXPEEK_API_KEY` | Your Mixpeek API key |
| `--base-url` | `MIXPEEK_BASE_URL` | API base URL (default: [https://api.mixpeek.com](https://api.mixpeek.com)) |

## Commands

### `mixpeek plugin init`

Create a new plugin from template.

```bash theme={null}
mixpeek plugin init <name> [options]
```

| Option | Description |
| - | - |
| `--category` | `text`, `image`, `video`, `audio`, `document`, `multimodal` |
| `--description` | Plugin description |
| `--author` | Author name |
| `--output` | Output directory |

```bash theme={null}
# Examples
mixpeek plugin init sentiment_analyzer --category text
mixpeek plugin init face_detector --category image --description "Detect faces"
```

### `mixpeek plugin test`

Validate and test plugin locally.

```bash theme={null}
mixpeek plugin test [options]
```

| Option | Description |
| - | - |
| `--path` | Plugin directory (default: `.`) |
| `--sample-data` | JSON/CSV file with test data |
| `--verbose` | Detailed output |

**Validates:**

* Structure (manifest.py, pipeline.py exist)
* Schemas (valid Pydantic models)
* Pipeline (`build_steps()` callable)
* Tests (runs pytest if tests/ exists)

```bash theme={null}
# Examples
mixpeek plugin test
mixpeek plugin test --path ./my_extractor --verbose
mixpeek plugin test --sample-data samples.json
```

### `mixpeek plugin publish`

Upload and deploy plugin to Mixpeek.

```bash theme={null}
mixpeek plugin publish [options]
```

| Option | Description |
| - | - |
| `--path` | Plugin directory |
| `--namespace` | Target namespace ID |
| `--dry-run` | Validate without uploading |

**What happens:**

1. Validates structure and schemas
2. Runs security scan
3. Creates .tar.gz archive
4. Uploads to S3 via presigned URL
5. Confirms and triggers deployment

```bash theme={null}
# Examples
mixpeek plugin publish
mixpeek plugin publish --namespace ns_abc123 --dry-run
```

### `mixpeek plugin list`

List plugins in namespace.

```bash theme={null}
mixpeek plugin list [options]
```

| Option | Description |
| - | - |
| `--namespace` | Namespace ID |
| `--source` | `all`, `builtin`, `custom`, `community` |

```bash theme={null}
mixpeek plugin list --source custom
```

## Plugin Structure

```
my_extractor/
├── manifest.py      # Metadata + schemas
├── pipeline.py      # Batch processing
├── realtime.py      # HTTP endpoint (optional, Enterprise)
└── processors/
    └── core.py      # Your logic
```

### manifest.py

```python theme={null}
from pydantic import BaseModel, Field
from typing import List

class MyInput(BaseModel):
    text: str

class MyOutput(BaseModel):
    embedding: List[float]

class MyParams(BaseModel):
    threshold: float = Field(default=0.5)

metadata = {
    "feature_extractor_name": "my_extractor",
    "version": "1.0.0",
    "description": "My extractor",
    "category": "text",
}

input_schema = MyInput
output_schema = MyOutput
parameter_schema = MyParams
supported_input_types = ["text"]

features = [
    {
        "feature_name": "my_embedding",
        "feature_type": "embedding",
        "embedding_dim": 384,
        "distance_metric": "cosine",
    },
]
```

### processors/core.py

```python theme={null}
from dataclasses import dataclass
import pandas as pd

@dataclass
class MyConfig:
    threshold: float = 0.5

class MyProcessor:
    def __init__(self, config: MyConfig, progress_actor=None):
        self.config = config
        self._model = None

    def _load_model(self):
        if self._model is None:
            from sentence_transformers import SentenceTransformer
            self._model = SentenceTransformer("all-MiniLM-L6-v2")

    def __call__(self, batch: pd.DataFrame) -> pd.DataFrame:
        self._load_model()
        texts = batch["text"].fillna("").tolist()
        batch["my_embedding"] = self._model.encode(texts).tolist()
        return batch
```

### pipeline.py

```python theme={null}
from typing import Any, Dict, Optional
from engine.plugins.extractors.pipeline import (
    PipelineDefinition, ResourceType, RowCondition, StepDefinition, build_pipeline_steps
)
from .manifest import MyParams, metadata
from .processors.core import MyConfig, MyProcessor

def build_steps(
    extractor_request: Any,
    container: Optional[Any] = None,
    base_steps: Optional[list] = None,
    **kwargs
) -> Dict[str, Any]:
    params = MyParams(**(extractor_request.extractor_config.parameters or {}))

    steps = [
        StepDefinition(
            service_class=MyProcessor,
            resource_type=ResourceType.CPU,
            config=MyConfig(threshold=params.threshold),
            condition=RowCondition.IS_TEXT,
        ),
    ]

    pipeline = PipelineDefinition(name=metadata["feature_extractor_name"], version=metadata["version"], steps=steps)
    return {"steps": (base_steps or []) + build_pipeline_steps(pipeline), "prepare": lambda ds: ds}
```

### realtime.py (Enterprise)

```python theme={null}
from typing import Any, Dict

class RealtimeHandler:
    def __init__(self):
        self._model = None

    def predict(self, request: Dict[str, Any]) -> Dict[str, Any]:
        if self._model is None:
            from sentence_transformers import SentenceTransformer
            self._model = SentenceTransformer("all-MiniLM-L6-v2")

        text = request.get("text", "")
        embedding = self._model.encode([text])[0].tolist()
        return {"embedding": embedding}
```

## Resource Types

| Type | Use For |
| - | - |
| `ResourceType.CPU` | Text embeddings, classification |
| `ResourceType.GPU` | Local models (Whisper, CLIP) |
| `ResourceType.API` | External APIs (OpenAI, Vertex) |

## Row Conditions

```python theme={null}
RowCondition.IS_TEXT       # text/* MIME types
RowCondition.IS_IMAGE      # image/* MIME types
RowCondition.IS_VIDEO      # video/* MIME types
RowCondition.IS_AUDIO      # audio/* MIME types
RowCondition.IS_PDF        # application/pdf
RowCondition.ALWAYS        # All rows (default)
```

## Security Constraints

Plugins are scanned before deployment. **Forbidden:**

| Pattern | Reason |
| - | - |
| `subprocess`, `os.system` | Shell execution |
| `eval`, `exec` | Dynamic code |
| `socket` | Direct network |
| `ctypes` | Memory access |
| `__import__` | Dynamic imports |

## Using Your Plugin

After publishing:

```python theme={null}
client.collections.create(
    collection_name="my_collection",
    source={"type": "bucket", "bucket_ids": ["bkt_..."]},
    feature_extractor={
        "feature_extractor_name": "my_extractor",
        "version": "1.0.0",
        "parameters": {"threshold": 0.7}
    }
)
```

## API Reference

| Endpoint | Method | Description |
| - | - | - |
| `/v1/namespaces/{id}/plugins/uploads` | POST | Get presigned upload URL |
| `/v1/namespaces/{id}/plugins/uploads/{id}/confirm` | POST | Confirm upload |
| `/v1/namespaces/{id}/plugins` | GET | List plugins |
| `/v1/namespaces/{id}/plugins/{id}` | GET | Get plugin details |
| `/v1/namespaces/{id}/plugins/{id}` | DELETE | Delete plugin |
| `/v1/namespaces/{id}/plugins/{id}/deploy` | POST | Deploy for realtime (Enterprise) |
| `/v1/namespaces/{id}/plugins/{id}/status` | GET | Check deployment status |

## Troubleshooting

| Issue | Solution |
| - | - |
| Plugin not found | Check namespace, wait for deployment |
| Import errors | Ensure `__init__.py` files exist |
| Security scan fails | Remove forbidden patterns |
| Validation errors | Check manifest.py exports metadata/schemas |

Debug mode:

```bash theme={null}
mixpeek plugin test --verbose
mixpeek plugin publish --dry-run
```


# Python SDK
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/python-sdk

Official Python SDK for the Mixpeek API

> [!NOTE]
>
> The Mixpeek Python SDK is auto-generated from our OpenAPI specification and always stays in sync with the latest API features.

## Installation

Install via pip:

```bash theme={null}
pip install mixpeek
```

**Requirements:** Python 3.9+

## Quick Start

```python theme={null}
import mixpeek
from mixpeek.rest import ApiException

# Configure the client
configuration = mixpeek.Configuration(
    host="https://api.mixpeek.com"
)

# Create an API client
with mixpeek.ApiClient(configuration) as api_client:
    # Create an instance of the Collections API
    collections_api = mixpeek.CollectionsApi(api_client)

    try:
        # List collections
        collections = collections_api.list_collections(
            authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
            x_namespace="ns_xxxxxxxxxxxxx"
        )
        print("Collections:", collections)
    except ApiException as e:
        print(f"Exception: {e}")
```

## Authentication

All API requests require authentication using a Bearer token:

```python theme={null}
authorization = "Bearer mxp_sk_xxxxxxxxxxxxx"  # Your API key
x_namespace = "ns_xxxxxxxxxxxxx"  # Your namespace ID or custom name
```

> [!TIP]
>
> You can create API keys in the Mixpeek dashboard under Organization Settings.

## Core Operations

### Collections

```python theme={null}
import mixpeek
from mixpeek.rest import ApiException

configuration = mixpeek.Configuration(host="https://api.mixpeek.com")

with mixpeek.ApiClient(configuration) as api_client:
    collections_api = mixpeek.CollectionsApi(api_client)

    # Create a collection
    create_request = mixpeek.CreateCollectionRequest(
        alias="my-collection",
        description="Store multimodal documents"
    )

    collection = collections_api.create_collection(
        create_collection_request=create_request,
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
    print(f"Created collection: {collection.collection_id}")

    # Get a collection
    retrieved = collections_api.get_collection(
        collection_identifier="my-collection",
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )

    # List all collections
    all_collections = collections_api.list_collections(
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )

    # Delete a collection
    collections_api.delete_collection(
        collection_identifier="my-collection",
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
```

### Retrievers

```python theme={null}
import mixpeek

with mixpeek.ApiClient(configuration) as api_client:
    retrievers_api = mixpeek.RetrieversApi(api_client)

    # Create a retriever
    create_request = mixpeek.CreateRetrieverRequest(
        retriever_name="semantic-search",
        description="Search across all documents",
        collection_identifiers=["my-collection"],
        stages=[
            {
                "stage_name": "search",
                "stage_type": "filter",
                "config": {
                    "stage_id": "feature_search",
                    "parameters": {
                        "searches": [{
                            "feature_uri": "mixpeek://universal_extractor@v1/gemini-embedding-2",
                            "query": {"input_mode": "text", "value": "{{INPUT.query_text}}"},
                            "top_k": 10
                        }],
                        "final_top_k": 10
                    }
                }
            }
        ]
    )

    retriever = retrievers_api.create_retriever(
        create_retriever_request=create_request,
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )

    # Execute a retriever
    execute_request = mixpeek.ExecuteRetrieverRequest(
        query="find relevant documents about AI"
    )

    results = retrievers_api.execute_retriever(
        retriever_id=retriever.retriever_id,
        execute_retriever_request=execute_request,
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
    print(f"Found {len(results.documents)} results")
```

### Adhoc Retrievers

Execute retrievers on-the-fly without creating them first:

```python theme={null}
import mixpeek

with mixpeek.ApiClient(configuration) as api_client:
    adhoc_api = mixpeek.AdhocRetrieversApi(api_client)

    # Execute adhoc retriever
    adhoc_request = mixpeek.AdhocExecuteRequest(
        query="search query",
        collection_identifiers=["my-collection"],
        stages=[
            {
                "stage_name": "search",
                "stage_type": "filter",
                "config": {
                    "stage_id": "feature_search",
                    "parameters": {
                        "searches": [{
                            "feature_uri": "mixpeek://universal_extractor@v1/gemini-embedding-2",
                            "query": {"input_mode": "text", "value": "{{INPUT.query_text}}"},
                            "top_k": 10
                        }],
                        "final_top_k": 10
                    }
                }
            }
        ]
    )

    results = adhoc_api.execute_adhoc_retriever(
        adhoc_execute_request=adhoc_request,
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
```

### Documents

```python theme={null}
import mixpeek

with mixpeek.ApiClient(configuration) as api_client:
    documents_api = mixpeek.DocumentsApi(api_client)

    # Upload documents
    upload_request = mixpeek.UploadDocumentsRequest(
        collection_id="col_abc123",
        documents=[
            {
                "url": "s3://bucket/video.mp4",
                "metadata": {"title": "Demo Video"}
            },
            {
                "url": "s3://bucket/image.jpg",
                "metadata": {"title": "Demo Image"}
            }
        ]
    )

    documents = documents_api.upload_documents(
        upload_documents_request=upload_request,
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
```

## Standalone Namespaces (Bring Your Own Vectors)

Standalone namespaces let you store and query **precomputed vectors** directly —
no buckets, collections, or extractors required. This is the fastest path for
agents that already have embeddings. The high-level `Mixpeek` client exposes the
MVS primitives ergonomically; every call sends your namespace as the
`X-Namespace` header automatically.

```python theme={null}
from mixpeek import Mixpeek

mp = Mixpeek(api_key="mxp_sk_xxxxxxxxxxxxx", namespace="my-vectors")

# 1. Create a standalone namespace with one or more vector configs.
mp.namespaces.create(
    namespace_id="my-vectors",
    mode="standalone",
    vector_configs=[{"name": "text_8", "dimension": 8, "metric": "cosine"}],
)

# 2. Upsert documents with your own vectors (direct write, no pipeline).
mp.namespaces.documents.upsert(
    namespace_id="my-vectors",
    documents=[
        {
            "document_id": "doc-1",
            "vectors": {"text_8": [0.11, 0.32, -0.38, -0.41, 0.006, 0.34, -0.40, -0.52]},
            "payload": {"body": "hello world", "category": "docs"},
        }
    ],
    # Optional: request a write token for read-your-writes consistency.
    options={"write_token": True},
)

# 3. Query is unified on retrievers. Create a retriever with a feature_search
#    stage and execute it. The X-Namespace header is sent for you.
retriever = mp.retrievers.create(
    retriever_name="byov-search",
    stages=[
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "final_top_k": 5,
                    "searches": [
                        {
                            "feature_uri": "text_8",
                            "query": {"input_mode": "vector", "value": "{{INPUT.qv}}"},
                            "top_k": 5,
                        }
                    ],
                },
            },
        }
    ],
    input_schema={"qv": {"type": "array", "required": True}},
)

results = mp.retrievers.execute(
    retriever["retriever_id"],
    inputs={"qv": [0.11, 0.32, -0.38, -0.41, 0.006, 0.34, -0.40, -0.52]},
)
```

> [!TIP]
>
> The removed `POST /v1/search` endpoint has been replaced by this retriever
> create + execute flow. `Mixpeek.search(namespace_id=..., queries=[...])` is a
> shortcut that builds and runs an ephemeral feature\_search retriever for you.

### Read-your-writes after a direct upsert

A direct upsert is durable immediately, but retriever reads are eventually
consistent by default. To read your own just-written document, pass the
`write_token` from the upsert response back as the `X-Write-Token` header on
execute — it routes the read to the primary shard and bypasses caches:

```python theme={null}
res = mp.namespaces.documents.upsert(
    namespace_id="my-vectors",
    documents=[{"document_id": "doc-2", "vectors": {"text_8": [...]}, "payload": {}}],
    options={"write_token": True},
)
# res["write_token"] -> send as X-Write-Token on the next retriever execute.
```

### Promote to managed (auto-embedding)

Promote a standalone namespace to **managed** mode to map a vector index to an
inference service. After promotion the same `feature_search` stage accepts
`input_mode: "text"` and auto-embeds queries — no client-side embedding needed.

```python theme={null}
import urllib3, json

http = urllib3.PoolManager()
http.request(
    "POST",
    "https://api.mixpeek.com/v1/namespaces/my-vectors/promote",
    headers={"Authorization": "Bearer mxp_sk_xxxxxxxxxxxxx", "Content-Type": "application/json"},
    body=json.dumps({
        "vector_mappings": [
            {"existing_index": "text_8", "inference_service": "intfloat/multilingual-e5-large-instruct"}
        ]
    }).encode(),
)
```

> [!NOTE]
>
> Before promotion, a standalone vector index has no inference mapping, so
> `input_mode: "text"` / `"content"` queries return an actionable 400 telling you
> to promote the namespace or use `input_mode=vector` with a precomputed embedding.

## Error Handling

The SDK raises `ApiException` for every non-2xx response:

```python theme={null}
from mixpeek.rest import ApiException

try:
    collection = collections_api.get_collection(
        collection_identifier="non-existent",
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
except ApiException as e:
    print(f"Status code: {e.status}")
    print(f"Reason: {e.reason}")
    print(f"Body: {e.body}")

    # Handle specific error codes
    if e.status == 404:
        print("Collection not found")
    elif e.status == 401:
        print("Authentication failed - check your API key")
    elif e.status == 403:
        print("Access forbidden - check your namespace")
```

## Available APIs

The SDK includes these API classes:

* `AdhocRetrieversApi` - Execute retrievers without saving them
* `AgentSessionsApi` - Manage AI agent sessions
* `AlertsApi` - Configure and manage alerts
* `AnalyticsApi` - Access usage and performance analytics
* `BucketsApi` - Manage object storage buckets
* `CollectionsApi` - Manage document collections
* `DocumentsApi` - Upload and manage documents
* `NamespacesApi` - Manage multi-tenant namespaces
* `OrganizationsApi` - Organization management
* `PluginsApi` - Configure and manage plugins
* `RetrieversApi` - Create and execute retrievers
* `TaxonomiesApi` - Manage classification taxonomies
* `WebhooksApi` - Configure webhook integrations

## Configuration Options

### Custom Host

```python theme={null}
configuration = mixpeek.Configuration(
    host="https://custom.api.endpoint.com"
)
```

### Timeouts

```python theme={null}
configuration = mixpeek.Configuration(
    host="https://api.mixpeek.com"
)
configuration.timeout = 60  # Set timeout to 60 seconds
```

## Resources

> **[PyPI Package](https://pypi.org/project/mixpeek/)**
>
> View on PyPI

> **[GitHub Repository](https://github.com/mixpeek/python-sdk)**
>
> Source code and issues

> **[API Reference](/docs/api-reference)**
>
> Complete API documentation

> **[OpenAPI Spec](https://api.mixpeek.com/docs/openapi.json)**
>
> OpenAPI specification

## Next Steps

> **[Quickstart](/docs/overview/quickstart)**
>
> Get started with Mixpeek

> **[JavaScript SDK](/docs/integrations/developer-tools/javascript-sdk)**
>
> Use the JavaScript/TypeScript SDK

> **[MCP Server](/docs/integrations/developer-tools/mcp-server)**
>
> Use Mixpeek with AI assistants

> **[Examples](https://mixpeek.com/recipes)**
>
> View example implementations


# SDK Overview
Source: https://docs.mixpeek.com/docs/integrations/developer-tools/sdk-usage

Official SDKs for Python, JavaScript/TypeScript, and more

> [!NOTE]
>
> Mixpeek provides auto-generated SDKs that are always in sync with our latest API features.

## Official SDKs

Choose the SDK that matches your development environment:

> **[Python SDK](/docs/integrations/developer-tools/python-sdk)**
>
> Python SDK with type hints and typed exceptions

> **[JavaScript SDK](/docs/integrations/developer-tools/javascript-sdk)**
>
> TypeScript-first SDK for Node.js and browser environments

## Quick Comparison

| Feature | Python SDK | JavaScript SDK |
| - | - | - |
| Package Manager | pip | npm/yarn/pnpm |
| Type Safety | Type hints | Full TypeScript |
| Async Support | ✅ | ✅ (Promise-based) |
| Auto-generated | ✅ | ✅ |
| Runtime Validation | ✅ | ✅ (Zod schemas) |

## Installation

```bash Python theme={null}
pip install mixpeek
```

```bash JavaScript theme={null}
npm install mixpeek
```

## Quick Start Examples

### Python

```python theme={null}
import mixpeek
from mixpeek.rest import ApiException

configuration = mixpeek.Configuration(host="https://api.mixpeek.com")

with mixpeek.ApiClient(configuration) as api_client:
    collections_api = mixpeek.CollectionsApi(api_client)
    collections = collections_api.list_collections(
        authorization="Bearer mxp_sk_xxxxxxxxxxxxx",
        x_namespace="ns_xxxxxxxxxxxxx"
    )
```

[View full Python SDK documentation →](/docs/integrations/developer-tools/python-sdk)

### JavaScript/TypeScript

```typescript theme={null}
import { Mixpeek } from 'mixpeek';

const client = new Mixpeek({
  apiKey: process.env.MIXPEEK_API_KEY,
  namespace: 'my-namespace'
});

const collections = await client.collections.listCollections();
```

[View full JavaScript SDK documentation →](/docs/integrations/developer-tools/javascript-sdk)

## Using the REST API Directly

If you prefer to use the REST API without an SDK, you can make direct HTTP requests:

```python theme={null}
import requests

# Create a namespace
response = requests.post(
    "https://api.mixpeek.com/v1/namespaces",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
    },
    json={
        "namespace_name": "my-workspace",
        "description": "Production workspace"
    }
)

namespace = response.json()
print(f"Created namespace: {namespace['namespace_id']}")
```

### Example: Direct API with Python

```python theme={null}
import requests
from typing import Optional

class MixpeekClient:
    def __init__(self, api_key: str, base_url: str = "https://api.mixpeek.com"):
        self.api_key = api_key
        self.base_url = base_url
        self.headers = {
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json"
        }

    def create_collection(
        self,
        namespace_id: str,
        collection_name: str,
        feature_extractors: list
    ) -> dict:
        """Create a new collection with feature extractors."""
        response = requests.post(
            f"{self.base_url}/v1/collections",
            headers={**self.headers, "X-Namespace": namespace_id},
            json={
                "collection_name": collection_name,
                "feature_extractors": feature_extractors
            }
        )
        response.raise_for_status()
        return response.json()

    def execute_retriever(
        self,
        namespace_id: str,
        retriever_id: str,
        inputs: dict,
        limit: Optional[int] = None
    ) -> dict:
        """Execute a retriever to search documents."""
        params = {"limit": limit} if limit else {}
        response = requests.post(
            f"{self.base_url}/v1/retrievers/{retriever_id}/execute",
            headers={**self.headers, "X-Namespace": namespace_id},
            json={"inputs": inputs},
            params=params
        )
        response.raise_for_status()
        return response.json()

# Usage
client = MixpeekClient(api_key="sk_...")

# Create a collection
collection = client.create_collection(
    namespace_id="ns_abc123",
    collection_name="videos",
    feature_extractors=[{
        "feature_extractor_name": "multimodal_extractor",
        "version": "v1"
    }]
)

# Execute a search
results = client.execute_retriever(
    namespace_id="ns_abc123",
    retriever_id="ret_xyz789",
    inputs={"query": "machine learning"},
    limit=10
)
```

### Example: Direct API with JavaScript

```javascript theme={null}
class MixpeekClient {
  constructor(apiKey, baseUrl = "https://api.mixpeek.com") {
    this.apiKey = apiKey;
    this.baseUrl = baseUrl;
    this.headers = {
      "Authorization": `Bearer ${apiKey}`,
      "Content-Type": "application/json"
    };
  }

  async createCollection(namespaceId, collectionName, featureExtractors) {
    const response = await fetch(`${this.baseUrl}/v1/collections`, {
      method: "POST",
      headers: {
        ...this.headers,
        "X-Namespace": namespaceId
      },
      body: JSON.stringify({
        collection_name: collectionName,
        feature_extractors: featureExtractors
      })
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.statusText}`);
    }

    return response.json();
  }

  async executeRetriever(namespaceId, retrieverId, inputs, limit = null) {
    const url = new URL(`${this.baseUrl}/v1/retrievers/${retrieverId}/execute`);
    if (limit) url.searchParams.set("limit", limit);

    const response = await fetch(url, {
      method: "POST",
      headers: {
        ...this.headers,
        "X-Namespace": namespaceId
      },
      body: JSON.stringify({ inputs })
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.statusText}`);
    }

    return response.json();
  }
}

// Usage
const client = new MixpeekClient("sk_...");

// Create a collection
const collection = await client.createCollection(
  "ns_abc123",
  "videos",
  [{
    feature_extractor_name: "multimodal_extractor",
    version: "v1"
  }]
);

// Execute a search
const results = await client.executeRetriever(
  "ns_abc123",
  "ret_xyz789",
  { query: "machine learning" },
  10
);
```

## Next Steps

> **[Python SDK](/docs/integrations/developer-tools/python-sdk)**
>
> Full Python SDK documentation

> **[JavaScript SDK](/docs/integrations/developer-tools/javascript-sdk)**
>
> Full JavaScript SDK documentation

> **[API Reference](/docs/api-reference)**
>
> Explore all API endpoints

> **[MCP Server](/docs/integrations/developer-tools/mcp-server)**
>
> Use Mixpeek with AI assistants

> **[Quickstart](/docs/overview/quickstart)**
>
> Get started with Mixpeek

> **[Examples](https://mixpeek.com/recipes)**
>
> View example implementations


# Search Widget
Source: https://docs.mixpeek.com/docs/integrations/search-widget

Drop-in React component for AI-powered multimodal search — add search to any site in minutes

`@mixpeek/search-js` is a drop-in React component that connects to any published Mixpeek retriever. It provides a full search UI with keyboard shortcuts, filters, AI answers, and streaming results — no backend code required on your side.

```bash npm theme={null}
npm install @mixpeek/search-js
```

```bash yarn theme={null}
yarn add @mixpeek/search-js
```

```bash pnpm theme={null}
pnpm add @mixpeek/search-js
```

## Quick Start

**1. Set up your search backend**

Use the [docs search quickstart](#quickstart-docs-search) to provision a complete pipeline in one API call, or [create a retriever manually](/docs/retrieval/retrievers) and [publish it](#publishing-a-retriever).

**2. Install the widget**

```bash theme={null}
npm install @mixpeek/search-js
```

**3. Add to your app**

```tsx theme={null}
import { MixpeekSearch } from "@mixpeek/search-js";
import "@mixpeek/search-js/styles.css";

export default function App() {
  return (
    <MixpeekSearch
      projectKey="your-retriever-slug"
      placeholder="Search..."
      theme="auto"
      accentColor="#6366f1"
    />
  );
}
```

Users press `Cmd+K` (or `Ctrl+K`) to open the search modal. Results stream in from your retriever pipeline.

## Props

| Prop | Type | Default | Description |
| - | - | - | - |
| `projectKey` | `string` | **required** | Published retriever slug or `ret_sk_*` API key |
| `placeholder` | `string` | `"Search..."` | Input placeholder text |
| `maxResults` | `number` | `10` | Maximum results to show |
| `theme` | `"light" \| "dark" \| "auto"` | `"auto"` | Color theme |
| `accentColor` | `string` | `"#6366f1"` | Accent color (hex) |
| `position` | `"modal" \| "inline"` | `"modal"` | Modal overlay or inline embed |
| `keyboardShortcut` | `boolean` | `true` | Enable Cmd+K / Ctrl+K |
| `showPoweredBy` | `boolean` | `true` | Show "Search by Mixpeek" badge |
| `enableAIAnswer` | `boolean` | `false` | Show AI-generated answer with citations |
| `enableShareLinks` | `boolean` | `false` | Enable shareable search URLs |
| `defaultOpen` | `boolean` | `false` | Start with modal open |
| `defaultFilters` | `Record<string, unknown>` | - | Default filter values on mount |

### Callbacks

| Prop | Type | Description |
| - | - | - |
| `onSearch` | `(query: string) => void` | Fires when a search is performed |
| `onResultClick` | `(result, index) => void` | Fires when a result is clicked |
| `onZeroResults` | `(query: string) => void` | Fires when no results are found |
| `onFilterChange` | `(filterInputs) => void` | Fires when filters change |
| `transformResults` | `(results[]) => results[]` | Transform results before rendering |
| `renderResult` | `(result, index) => ReactNode` | Custom result renderer |

## CDN Usage (No Build Step)

For sites without a build system, load the widget via CDN:

```html theme={null}
<link rel="stylesheet" href="https://cdn.mixpeek.com/search/v1/mixpeek-search.css" />
<script
  src="https://cdn.mixpeek.com/search/v1/loader.js"
  data-project-key="your-retriever-slug"
  data-mount="search-container">
</script>

<div id="search-container"></div>
```

## Filters

The widget includes built-in filter components for facets, ranges, and LLM-powered smart filtering.

### Facet Filter

Single or multi-select dropdown:

```tsx theme={null}
<MixpeekSearch projectKey="my-search">
  <FacetFilter
    field="category"
    label="Category"
    options={[
      { label: "Electronics", value: "electronics" },
      { label: "Clothing", value: "clothing" },
    ]}
    multiple={true}
  />
</MixpeekSearch>
```

### Range Filter

Numeric min/max slider:

```tsx theme={null}
<RangeFilter
  field="price"
  label="Price"
  min={0}
  max={1000}
  step={10}
  unit="$"
/>
```

### Smart Filter (LLM-based)

Natural language filtering powered by the retriever's LLM filter stage:

```tsx theme={null}
<SmartFilter
  label="Describe what you want"
  placeholder="e.g. red shoes under $50"
/>
```

## AI Answers

Enable `enableAIAnswer` to show an LLM-generated answer with citations above search results. This requires an `agent_search` or `rag_prepare` stage in your retriever.

```tsx theme={null}
<MixpeekSearch
  projectKey="docs-search"
  enableAIAnswer={true}
/>
```

## Hooks

Access search state from any child component:

```tsx theme={null}
import { useSearchKit } from "@mixpeek/search-js";

function MyComponent() {
  const {
    query,
    results,
    isLoading,
    aiAnswer,
    isOpen,
    open,
    close,
    search,
    filterInputs,
    setFilter,
    clearFilters,
    hasActiveFilters,
  } = useSearchKit();

  return <div>{results.length} results for "{query}"</div>;
}
```

**Available hooks:** `useSearchKit`, `useSearch`, `useFilters`, `useKeyboardShortcut`, `useRecentSearches`.

## Setting Up the Backend

The widget needs a published retriever to connect to. There are two paths:

### Quickstart: Docs Search

Provision a complete search pipeline in one API call. This creates a namespace, bucket, collection (with web scraper + text embeddings), retriever, and published endpoint automatically:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/quickstart/docs-search \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "site_url": "https://docs.example.com",
    "site_name": "Example Docs"
  }'
```

The response includes a retriever slug and embed snippet ready to paste:

```json theme={null}
{
  "retriever_slug": "example-docs-search",
  "api_key": "ret_sk_...",
  "embed_snippet": "<MixpeekSearch projectKey=\"example-docs-search\" />"
}
```

**Options:** `enable_code_search` (default: true), `enable_image_search` (default: false), `max_pages` (default: 200), `max_depth` (default: 3).

### Bootstrap CLI

The `@mixpeek/react-searchkit` package includes a CLI to scaffold a retriever with search, filter, and RAG stages:

```bash theme={null}
npx mixpeek-bootstrap --api-key YOUR_API_KEY --slug my-site-search
```

This creates a retriever with `feature_search`, `attribute_filter`, and `rag_prepare` stages pre-configured.

### Manual Setup

For full control, create each resource yourself:

**1. Create a namespace and bucket**

Set up storage and enable the feature extractors you need ([guide](/docs/ingestion/namespaces)).

**2. Create a collection**

Configure a collection with a feature extractor (e.g., `text_extractor`, `web_scraper`, `multimodal_extractor`) and trigger processing on your data ([guide](/docs/ingestion/collections)).

**3. Create and publish a retriever**

Build a retriever with the stages you need, then publish it to get a slug for the widget ([guide](/docs/retrieval/retrievers)).

### Publishing a Retriever

Once you have a retriever, publish it to make it accessible to the widget:

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

result = client.retrievers.publish(
    retriever_id="ret_abc123",
    public_name="my-search",
)

print(result.public_url)  # https://mxp.co/r/my-search
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/ret_abc123/publish \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"public_name": "my-search"}'
```

The `public_name` becomes the `projectKey` you pass to the widget.

## Authentication

The widget supports two authentication modes:

| Mode | `projectKey` value | When to use |
| - | - | - |
| **Public slug** | `"my-retriever-slug"` | Public-facing search (no auth needed) |
| **Scoped API key** | `"ret_sk_..."` | Authenticated access with a `retrieverSlug` prop |

```tsx theme={null}
// Public slug (most common)
<MixpeekSearch projectKey="my-retriever-slug" />

// Scoped API key
<MixpeekSearch projectKey="ret_sk_..." retrieverSlug="my-retriever-slug" />
```

## Examples

The search widget is used on [mixpeek.com](https://mixpeek.com) as the site-wide search in the navigation bar, powered by `@mixpeek/search-js` with the `mixpeek-blog-search` retriever slug.

```tsx theme={null}
<MixpeekSearch
  projectKey="mixpeek-blog-search"
  placeholder="Search..."
  theme={isScrolled ? "light" : "dark"}
  accentColor="#6366f1"
  maxResults={10}
  showPoweredBy={true}
  keyboardShortcut={true}
  transformResults={(results) =>
    results.map((r) => ({
      id: r.document_id || r.id,
      title: r.title || r.page_title || "Mixpeek",
      content: r.content || r.text || "",
      page_url: r.page_url || r.url || null,
      image_url: r.image_url || null,
      score: r.score,
    }))
  }
/>
```

## Exported Components

For building fully custom search experiences, the package exports composable sub-components:

| Component | Description |
| - | - |
| `SearchButton` | Standalone search trigger button |
| `SearchModal` | Search modal container |
| `SearchInput` | Input field |
| `SearchResults` | Results list |
| `ResultCard` | Individual result card |
| `AIAnswer` | AI-generated answer panel |
| `FilterPanel` | Filter container |
| `FacetFilter` | Select/multi-select filter |
| `RangeFilter` | Min/max range slider |
| `SmartFilter` | LLM-powered natural language filter |
| `PoweredBy` | "Search by Mixpeek" badge |
| `ShareLink` | Shareable search URL generator |
| `ZeroResults` | Empty state placeholder |
| `IntentCTA` | Enterprise CTA capture |

## Related

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Build the retriever pipeline behind your widget

> **[Retriever Stages](/docs/retrieval/stages/overview)**
>
> Configure search, reranking, and enrichment stages

> **[Interactions](/docs/retrieval/interactions)**
>
> Use interaction data to improve search relevance

> **[JavaScript SDK](/docs/integrations/developer-tools/javascript-sdk)**
>
> Full SDK for programmatic access


# Analytics & Performance
Source: https://docs.mixpeek.com/docs/operations/analytics-overview

Interpret metrics, optimize pipelines, and tune retriever performance

Mixpeek's Analytics API provides granular visibility into retrieval performance, cache efficiency, feature extraction throughput, and inference latency. Use these metrics to identify bottlenecks, validate optimizations, and allocate budgets effectively.

## Analytics Categories

> **Retrieval Performance**
>
> Track latency, cache hit rates, and stage-level breakdowns for retrievers.

> **Infrastructure Metrics**
>
> Monitor API response times, Engine throughput, and inference service health.

> **Feature Extraction**
>
> Measure extractor execution time, failure rates, and batch processing efficiency.

> **User Signals**
>
> Analyze interaction patterns (clicks, long views, feedback) to tune relevance.

## Key Metrics Explained

### Retrieval Metrics

| Metric | API Endpoint | Use Case |
| - | - | - |
| **Retriever Performance** | `GET /v1/analytics/retrievers/{retriever_id}/performance` | Overall latency percentiles (p50, p95, p99), cache hit rate, error rate |
| **Stage Breakdown** | `GET /v1/analytics/retrievers/{retriever_id}/stages` | Per-stage execution time to identify slow stages (e.g., LLM generation vs vector search) |
| **Retriever Signals** | `GET /v1/analytics/retrievers/{retriever_id}/signals` | User interactions aggregated by result position, document, or session |
| **Slowest Queries** | `GET /v1/analytics/retrievers/{retriever_id}/slowest` | Identify outlier executions with full input payloads for debugging |

### Infrastructure Metrics

| Metric | API Endpoint | Use Case |
| - | - | - |
| **API Performance** | `GET /v1/analytics/api/performance` | Request latency by endpoint and status code |
| **Engine Performance** | `GET /v1/analytics/engine/performance` | Ray task execution time, queue depth, worker utilization |
| **Inference Performance** | `GET /v1/analytics/inference/performance` | Model latency (embedding, LLM, classification) and throughput |

### Extraction Metrics

| Metric | API Endpoint | Use Case |
| - | - | - |
| **Extractor Performance** | `GET /v1/analytics/extractors/{extractor_name}/performance` | Avg processing time per object, failure rate, retry count |
| **Batch Efficiency** | Derived from extractor + engine metrics | Compare batch sizes vs throughput to optimize batching strategy |

### Cache Metrics

| Metric | API Endpoint | Use Case |
| - | - | - |
| **Cache Performance** | `GET /v1/analytics/cache/performance` | Hit rate, average TTL, eviction rate, memory usage |

### Usage & Cost Metrics

| Metric | API Endpoint | Use Case |
| - | - | - |
| **Usage Summary** | `GET /v1/analytics/usage/summary` | Usage (in dollars) by resource type (API calls, inference, storage) |
| **Org Usage** | `GET /v1/organizations/usage` | Per-org breakdown for billing and quota enforcement |
| **API Key Usage** | `GET /v1/organizations/usage/api-keys/{key_id}` | Attribute costs to specific services or teams |

## Optimization Workflows

### 1. Diagnose Slow Retrievers

**Symptoms:** High p95/p99 latency, user complaints about wait times

**Steps:**

1. Call `/analytics/retrievers/{retriever_id}/performance` to get overall latency distribution
2. Check `cache_hit_rate` – if low, consider increasing `cache_config.ttl_seconds` or caching more stages
3. Call `/analytics/retrievers/{retriever_id}/stages` to identify the bottleneck stage
4. Common culprits:
   * **LLM generation stages** – consider smaller models, prompt caching, or async processing
   * **Large limit values** – reduce `limit` in early stages and use rerankers
   * **Complex filters** – move structured filters before expensive search stages
5. Call `/analytics/retrievers/{retriever_id}/slowest` to inspect outlier queries with full inputs

### 2. Improve Cache Hit Rates

**Target:** >70% cache hit rate for common queries

**Strategies:**

* Enable caching at the retriever level: `cache_config.enabled = true`
* Increase TTL for stable queries: `cache_config.ttl_seconds = 3600`
* Use `cache_stage_names` to selectively cache expensive stages (e.g., web search, LLM)
* Normalize inputs (lowercase, trim whitespace) before hashing for cache keys
* Monitor `/analytics/cache/performance` to track hit rate trends

### 3. Optimize Feature Extraction

**Symptoms:** Batch processing takes hours, high extractor failure rate

**Steps:**

1. Check `/analytics/extractors/{extractor_name}/performance` for avg execution time
2. Compare across extractors – identify if one is significantly slower
3. Strategies:
   * **Batch sizing** – 100-1000 objects optimal; adjust based on object size
   * **Parallelism** – increase Ray workers or use GPU instances for vision/LLM extractors
   * **Retry logic** – review `__missing_features` in documents to identify flaky extractors
   * **Model selection** – swap heavy models for distilled versions (e.g., `distilbert` vs `bert-large`)
4. Monitor `/analytics/engine/performance` for worker saturation

### 4. Tune Inference Budgets

**Goal:** Balance cost and latency

**Steps:**

1. Review `/analytics/usage/summary` to see inference spend by model
2. Identify high-volume, low-value queries (e.g., exploratory searches)
3. Apply budget limits at the retriever level:
   ```json theme={null}
   {
     "budget_limits": {
       "max_inference_calls": 10,
       "max_llm_tokens": 2000,
       "max_execution_time_ms": 5000
     }
   }
   ```
4. Use cheaper models for initial ranking (e.g., `multilingual-e5-base`) and expensive rerankers only for top-K results

### 5. Use User Signals for Relevance Tuning

**Workflow:**

1. Instrument your app to send interactions via `/v1/retrievers/{retriever_id}/interactions`
   * `click`, `long_view`, `positive_feedback`, `negative_feedback`
2. Query `/analytics/retrievers/{retriever_id}/signals` to see:
   * Click-through rate by result position
   * Documents with high negative feedback
   * Queries with zero engagement
3. Use insights to:
   * Adjust stage ordering (e.g., move filters earlier to reduce noise)
   * Update taxonomy mappings or cluster definitions
   * Fine-tune reranker prompts or scoring functions
4. A/B test changes by creating retriever variants and comparing signal distributions

## Recommended Dashboards

### Production Health Dashboard

**Metrics to display:**

* API p95 latency (target: \<500ms)
* Cache hit rate (target: >70%)
* Error rate by endpoint (target: \<1%)
* Engine queue depth (alert if >100 pending tasks)
* Inference quota remaining (% of plan)

**Refresh interval:** 1 minute

### Retriever Performance Dashboard

**Metrics to display:**

* Per-retriever p95 latency (grouped by retriever\_id)
* Cache hit rate by retriever
* Top 5 slowest queries (with input previews)
* Stage breakdown for critical retrievers
* Click-through rate trends (weekly aggregation)

**Refresh interval:** 5 minutes

### Cost Optimization Dashboard

**Metrics to display:**

* Inference cost per retriever execution (derived: `inference_calls * model_cost`)
* Storage growth rate (documents + features + cache)
* API key usage breakdown (top consumers)
* Batch processing cost per object (extractor time \* worker cost)

**Refresh interval:** Daily

## Alerting Recommendations

> **Critical: API Error Rate >5%**
>
> **Trigger:** `/analytics/api/performance` shows error rate >5% over 5 min\
> **Action:** Check health endpoint `/v1/health`, inspect logs for MVS/Mongo connectivity issues

> **Warning: Retriever p95 Latency >2s**
>
> **Trigger:** Specific retriever p95 >2000ms for 10 min\
> **Action:** Review stage breakdown, disable non-critical stages temporarily, increase cache TTL

> **Warning: Cache Hit Rate \<50%**
>
> **Trigger:** Cache hit rate drops below 50% for 30 min\
> **Action:** Review cache config, check if query patterns changed (seasonality, A/B tests)

> **Info: High Inference Quota Usage**
>
> **Trigger:** 80% of monthly inference quota consumed\
> **Action:** Review top consumers via `/analytics/usage/summary`, throttle exploratory retrievers

> **Critical: Engine Queue Depth >500**
>
> **Trigger:** Ray queue depth exceeds 500 pending tasks\
> **Action:** Scale Ray workers, pause batch submissions, investigate slow extractors

## Analytics API Patterns

### Time-Windowed Queries

Most endpoints accept `start_time` and `end_time` filters (ISO 8601):

```bash theme={null}
GET /v1/analytics/retrievers/{retriever_id}/performance?start_time=2025-10-01T00:00:00Z&end_time=2025-10-31T23:59:59Z
```

### Aggregation Levels

Control granularity with `group_by`:

```bash theme={null}
# Group by day
GET /v1/analytics/api/performance?group_by=day

# Group by retriever_id
GET /v1/analytics/usage/summary?group_by=retriever_id
```

### Tuning Recommendations

The `/analytics/analyze-for-tuning` endpoint provides automated suggestions:

```bash theme={null}
POST /v1/analytics/retrievers/{retriever_id}/analyze
```

**Response includes:**

* Recommended stage order changes
* Cache config suggestions (TTL, stage-level caching)
* Budget limit recommendations
* Model swap suggestions (cost vs latency trade-offs)

## Best Practices

1. **Baseline before optimizing** – capture 7 days of metrics before making changes
2. **Change one variable at a time** – isolate the impact of each optimization
3. **Monitor post-deployment** – use execution IDs to compare before/after distributions
4. **Set SLOs early** – define p95 latency and cache hit targets per retriever tier
5. **Correlate signals with changes** – timestamp config updates and overlay on metric charts
6. **Automate reporting** – schedule weekly summaries via webhooks or export to your BI tool

## Integration with External Tools

### Export to Datadog / Grafana

Use the Analytics API to pull metrics into your existing observability stack:

```python theme={null}
import requests

resp = requests.get(
    "https://api.mixpeek.com/v1/analytics/retrievers/ret_123/performance",
    headers={"Authorization": "Bearer mxp_sk_...", "X-Namespace": "ns_prod"}
)
data = resp.json()

# Push to Datadog
statsd.gauge("mixpeek.retriever.p95_latency", data["p95_latency_ms"])
statsd.gauge("mixpeek.retriever.cache_hit_rate", data["cache_hit_rate"])
```

### Anomaly Alerts

Slow queries, failing extractors and cache degradation surface as alert
conditions. Webhooks carry object and collection lifecycle events, so route an
anomaly through an alert with a `system_condition` and give it a webhook channel:

```bash theme={null}
POST /v1/alerts
{
  "name": "retriever-latency-watch",
  "source": "system",
  "system_condition": {
    "type": "retriever_latency",
    "params": { "latency_multiplier": 2 }
  },
  "notification_config": {
    "channels": [
      { "channel_type": "webhook",
        "config": { "url": "https://your-app.com/webhooks/mixpeek" } }
    ]
  }
}
```

`type` takes one of twelve built-in conditions, including `retriever_latency`,
`batch_error_rate`, `batch_stalled`, `task_queue_degraded` and
`provider_quota_exhausted`. Every `params` key is optional and the evaluator
supplies a default. See [Alerts](/docs/enrichment/alerts).

## Next Steps

* Explore all analytics endpoints in the [API Reference](/docs/api-reference/analytics/get-retriever-performance)
* Learn how to record [Interactions](/docs/retrieval/interactions) for signal tracking
* Review [Caching Strategies](/docs/overview/caching) for hit rate optimization
* Set up [Webhooks](/docs/operations/webhooks) for automated alerting


# API Keys
Source: https://docs.mixpeek.com/docs/operations/api-keys

Create, scope, rotate, and monitor API keys — permissions, resource scopes, per-key usage, and end-user keys

An API key authenticates every request to Mixpeek. Keys belong to your
**organization** and are created **per user**. This page covers creating keys,
restricting what they can do (permissions and scopes), rotating and revoking
them, monitoring per-key usage, and the special key types that power end-user
multi-tenancy.

For the broader authentication and tenancy model (the `Authorization` and
`X-Namespace` headers, isolation guarantees), see
[Security & Tenancy](/docs/operations/security). For rate-limit tiers and usage
pools, see [Rate Limits & Quotas](/docs/operations/rate-limits-quotas).

> [!NOTE]
>
> **Managing keys requires an admin key.** Every endpoint on this page requires
> a key with the `admin` [permission](#permissions). Your organization ships
> with a protected `admin-key` you can use to mint your first scoped keys.

## How a key looks

When you create a key, the **plaintext secret is returned exactly once** — it is
hashed (SHA-256) at rest and can never be read back. Store it immediately in a
secrets manager or environment variable.

| Field | Example | What it is |
| - | - | - |
| `key` | `mxp_sk_9f3a…` (60 chars) | The plaintext secret. **Shown once, on create/rotate.** Send it as `Authorization: Bearer <key>`. |
| `key_prefix` | `mxp_sk_9f3...` | First 10 characters + `...`. A safe display hint so you can tell keys apart in a list — never the full secret. |
| `key_id` | `key_a1b2c3d4e5f6g7h` | Stable public identifier. Used for [usage lookups](#monitor-usage-per-key). |
| `name` | `backend-service` | Human label you choose. Used to reference the key in update/rotate/revoke calls. |

> [!WARNING]
>
> The plaintext `key` is **never retrievable after creation**. If you lose it or
> suspect exposure, [rotate](#rotate-expire-and-revoke) the key — don't try to
> recover it.

## Create a key

`POST /v1/organizations/users/{user_email}/api-keys` — create a key owned by the
user at `{user_email}` (use your own email to create keys for yourself).

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "backend-service",
    "description": "Service account for the ingestion pipeline",
    "permissions": ["read", "write"]
  }'
```

```python Python theme={null}
import os, requests

resp = requests.post(
    "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys",
    headers={"Authorization": f"Bearer {os.environ['MIXPEEK_ADMIN_KEY']}"},
    json={
        "name": "backend-service",
        "description": "Service account for the ingestion pipeline",
        "permissions": ["read", "write"],
    },
)
new_key = resp.json()["key"]   # plaintext — store it now, it won't be shown again
print(new_key)
```

```javascript JavaScript theme={null}
const resp = await fetch(
  "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.MIXPEEK_ADMIN_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      name: "backend-service",
      description: "Service account for the ingestion pipeline",
      permissions: ["read", "write"],
    }),
  },
);
const { key } = await resp.json(); // plaintext — store it now
console.log(key);
```

The response is the key's metadata plus the one-time plaintext `key`:

```json theme={null}
{
  "key": "mxp_sk_9f3a2c7b1e...KUwq",
  "key_id": "key_a1b2c3d4e5f6g7h",
  "key_prefix": "mxp_sk_9f3...",
  "name": "backend-service",
  "permissions": ["read", "write"],
  "scopes": [],
  "status": "active",
  "created_at": "2026-07-07T00:00:00Z",
  "created_by": "usr_a1b2c3d4e5f6g7"
}
```

### Request fields

**name** `string` *required*

Human-friendly label (1–100 chars). Used to reference the key in
update/rotate/revoke calls. Must be unique among your active keys.

**permissions** `string[]` (default: \[read, write, delete])

Permission levels granted to the key. **Defaults to full `read`+`write`+`delete`
access** — pass a narrower list explicitly when you want a restricted key. See
[Permissions](#permissions).

**scopes** `ResourceScope[]` (default: \[] (org-wide))

Resource-level restrictions. **Omitting scopes grants org-wide access.** See
[Restrict a key to specific resources](#restrict-a-key-to-specific-resources).

**rate\_limit\_override** `integer`

Per-key requests-per-minute ceiling. Defaults to your plan limit when absent.
See [Per-key rate limits](#per-key-rate-limits).

**expires\_at** `string (ISO 8601)`

UTC timestamp when the key auto-expires. Omit for a non-expiring key.

**description** `string`

Optional note (≤500 chars) explaining the key's purpose.

**principal\_id** `string`

End-user identifier. Setting this makes the key **user-scoped** for
document-level ACL — see [End-user keys](#end-user-keys-for-multi-tenant-apps).

**allowed\_origins** `string[]`

CORS allowlist of web origins (exact or wildcard-subdomain). When set, browser
requests must send a matching `Origin` header. See [Restrict a key to browser
origins](#restrict-a-key-to-browser-origins).

## Verify a key

`GET /v1/me` answers "which tenant is this key, and what can it do". It works for every valid key, so it is the check to run the moment you mint one, especially for a key going into CI.

```bash cURL theme={null}
curl https://api.mixpeek.com/v1/me \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY"
```

```json theme={null}
{
  "organization_id": "org_a1b2c3d4e5f6",
  "organization_name": "Acme",
  "account_type": "BUILD",
  "user_email": "you@example.com",
  "key": {
    "name": "ci-ingest",
    "key_type": "SECRET",
    "permissions": ["READ", "WRITE"],
    "expires_at": null
  },
  "rate_limits": {},
  "principal_scoped": false
}
```

| Field | Tells you |
| - | - |
| `organization_id` | Which tenant the key belongs to. Safe to log |
| `organization_name` | That tenant's display name |
| `account_type` | The tier, for example `FREE`, `BUILD`, `SCALE` |
| `user_email` | Which user owns the key |
| `key` | The key's own name, type, permissions, and expiry |
| `rate_limits` | Effective per-minute limits by category |
| `principal_scoped` | `true` when the key is end-user scoped, so document ACL filtering applies |

> [!TIP]
>
> Pointing a job at the wrong tenant fails silently: the calls succeed and the data lands somewhere you did not intend. One `GET /v1/me` at the top of a CI run turns that into an assertion on `organization_id`.

## Permissions

Every key carries one or more of four permission levels. They form a strict
hierarchy — a higher level **implies** all lower ones, so you never list more
than the strongest you need.

| Permission | Implies | Use it for |
| - | - | - |
| `read` | — | Dashboards, analytics, read-only search clients |
| `write` | `read` | Ingestion and service accounts that create documents |
| `delete` | `write`, `read` | Full data management (create + remove) |
| `admin` | `delete`, `write`, `read` | Org administration — **managing keys and users requires `admin`** |

```
admin  ⊃  delete  ⊃  write  ⊃  read
```

A route declares the minimum permission it needs, and a key satisfies it if the
key's permission is at least that level. To issue a **read-only key** for an
internal dashboard, create it with `"permissions": ["read"]`:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "metrics-dashboard", "permissions": ["read"] }'
```

## Restrict a key to specific resources

Permissions control *what actions* a key can take; **scopes** control *which
resources* it can take them on. A scope is a `ResourceScope`:

```json theme={null}
{
  "resource_type": "namespace",
  "resource_id": "ns_customer_acme",
  "operations": ["read_data", "execute_retriever"]
}
```

**resource\_type** `enum` *required*

What kind of resource the scope governs: `namespace`, `collection`, `bucket`,
`retriever`, `cluster`, `taxonomy` (and other [resource
types](#resource-types-and-operations)).

**resource\_id** `string` *required*

A literal ID (`ns_production`) **or a wildcard** — `*` for all, or a prefix
pattern like `ns_customer_*` to match every namespace for a set of tenants.

**operations** `string[]`

A subset of [namespace operations](#resource-types-and-operations) the key may
perform within the scope. Omit (or `null`) to allow any operation its
[permissions](#permissions) already grant.

Example — a key that can only **read and search** within one customer's
namespace, and nothing else:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "acme-readonly",
    "permissions": ["read"],
    "scopes": [{
      "resource_type": "namespace",
      "resource_id": "ns_customer_acme",
      "operations": ["read_data", "execute_retriever"]
    }]
  }'
```

> [!WARNING]
>
> **An empty `scopes` list means org-wide access** — this is the legacy default,
> so a key created without scopes can reach every namespace and resource in your
> organization. To confine a key, you **must** provide at least one scope.
> Setting `scopes` to `[]` on an update *widens* the key back to org-wide.

### Resource types and operations

> **resource\_type values**
>
> `organization`, `user`, `api_key`, `namespace`, `collection`, `bucket`,
> `retriever`, `cluster`, `taxonomy`, `storage_connection`, `alert`,
> `annotation`, `secret`, `webhook`.

> **operations values (within a namespace)**
>
> **Data:** `read_data`, `write_data`, `delete_data` ·
> **Retrieval:** `execute_retriever`, `create_retriever`, `delete_retriever` ·
> **Jobs:** `execute_job`, `cancel_job` ·
> **Clusters:** `create_cluster`, `delete_cluster`, `modify_cluster` ·
> **Infrastructure:** `modify_infrastructure`, `manage_permissions`.
>
> Common bundles: read-only `["read_data", "execute_retriever"]`;
> data-engineer `["read_data", "write_data", "execute_job"]`.

## Per-key rate limits

Set `rate_limit_override` to cap a single key's requests per minute — useful for
throttling a third-party integration below your plan's default:

```bash theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys/backend-service" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "rate_limit_override": 120 }'
```

The override interacts with your organization's plan tiers and usage pools,
which are documented in [Rate Limits & Quotas](/docs/operations/rate-limits-quotas).

## Restrict a key to browser origins

Set `allowed_origins` to a CORS allowlist of web origins. When set, any
**browser** request using the key must send an `Origin` header that matches the
list — an exact origin (`https://app.example.com`) or a wildcard subdomain
(`https://*.example.com`) — otherwise the request is rejected with a `403`.
Requests that send no `Origin` header (server-side curl and the SDKs) are
unaffected. This is enforced org-wide, on every route, for any key that carries
an allowlist.

Set it on create, or change it later with `PATCH`. **An empty list clears the
restriction:**

```bash Set origins theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys/web-widget" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "allowed_origins": ["https://app.example.com", "https://*.example.com"] }'
```

```bash Clear origins theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys/web-widget" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "allowed_origins": [] }'
```

The allowlist carries across [rotation](#rotate-expire-and-revoke) — a rotated
key keeps its origins.

> [!WARNING]
>
> `allowed_origins` is **defense-in-depth, not a standalone security boundary.**
> It only constrains requests that send an `Origin` header (i.e. browsers); a
> stolen key used from a non-browser client can omit `Origin` and bypass it.
> Pair it with least-privilege [permissions](#permissions) and
> [scopes](#restrict-a-key-to-specific-resources), and treat any key shipped to
> a browser as public.

## Rotate, expire, and revoke

**1. Rotate — issue a new secret, invalidate the old one**

`POST /v1/organizations/users/{user_email}/api-keys/{key_name}/rotate`
returns a **new plaintext `key`** and immediately revokes the previous
secret. The `key_id` and settings carry over; only the secret changes.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys/backend-service/rotate" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY"
```

**2. Expire — set an automatic cutoff**

Set `expires_at` on create or via `PATCH`. Once passed, the key's status
flips to `expired` and it stops authenticating. Expired keys cannot be
reactivated — create or rotate instead.

**3. Revoke — kill a key immediately**

`DELETE /v1/organizations/users/{user_email}/api-keys/{key_name}` sets the
key's status to `revoked`. It's a soft revoke (the record is retained for
the audit trail), and it cannot be undone.

```bash theme={null}
curl -X DELETE "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys/backend-service" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY"
```

The call is **refused** when a live published app still embeds the key, or
when the key authenticated in the last 7 days, and the refusal names what it
found. `?force=true` proceeds anyway.

> [!WARNING]
>
> **Before you `force`, find the consumers the API cannot see.** The refusal
> can only name published apps and recent use. A key held in a Kubernetes
> Secret, a CI secret or a deployment config leaves no record in Mixpeek's
> data, so `referencing_apps` comes back empty for it and the refusal looks
> like a false alarm.
>
> That is not hypothetical. A revoke forced past an empty `referencing_apps`
> took a public widget offline for 43 minutes; an earlier one took two
> published apps down for seven weeks before anyone noticed. In both cases
> the holders were found afterwards by scanning the cluster.
>
> Search those places yourself first: every Kubernetes Secret, CI secret
> and deployment config that could hold the key. Compare SHA-256 digests
> so the key itself never prints.

**4. After the revoke, confirm every holder re-keyed**

Search the same places once more when the rotation is done, this time for
the digest of the **old** key. A Secret that still holds it is on the revoked
key and is failing now.

Make this the last step of every rotation. A holder left on the dead key stays
broken for as long as nothing pages on its behalf. After one forced revoke the
consumer with a human-facing alert was repaired in 43 minutes, while a scheduled
job holding the same key ran dark for 26 hours, its failures accumulating only in
`failedJobsHistory` where nobody was watching.

When the holder is a scheduled job, read `lastSuccessfulTime` on the CronJob
rather than the pod list. `failedJobsHistoryLimit` retains only the last few runs,
which dated that 26-hour outage at 67 minutes.

A key's `status` is always one of `active`, `revoked`, or `expired`, and
`last_used_at` records the timestamp of its most recent successful request — a
quick way to find stale keys worth revoking. List a user's keys (add
`?include_revoked=true` to see revoked ones) with:

```bash theme={null}
curl "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY"
# -> { "results": [ { "key_id": "...", "name": "...", "status": "active", ... } ], "total": 3 }
```

> [!NOTE]
>
> The organization's primary **`admin-key` is protected** — it cannot be
> updated, rotated, or deleted (the Studio UI depends on it). Create additional
> scoped keys for specific use cases, and if the `admin-key` is ever
> compromised, [contact support](https://mixpeek.com/contact).

### Audit trail

Every key lifecycle event is written to your organization's audit log with the
acting user and a timestamp. Keys record `created_by` and `revoked_by`
(with `revoked_at`) directly, and the audit log captures the actions
`api_key_created`, `api_key_rotated`, `api_key_revoked`, and
`api_key_scope_updated` — each with the actor, resource, and what changed.

## Monitor usage per key

Attribute traffic and spend to individual keys. Metrics come from the analytics
store and default to the **last 7 days** unless you pass `start`
and `end` (ISO 8601).

```bash theme={null}
curl "https://api.mixpeek.com/v1/organizations/api-keys/key_a1b2c3d4e5f6g7h/usage" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY"
```

```json theme={null}
{
  "key_id": "key_a1b2c3d4e5f6g7h",
  "start": "2026-06-30T00:00:00Z",
  "end": "2026-07-07T00:00:00Z",
  "total_requests": 14820,
  "total_credits": 512,
  "unique_endpoints": 6,
  "avg_latency_ms": 43.7,
  "p95_latency_ms": 118.0
}
```

For a per-endpoint breakdown, call
`GET /v1/organizations/api-keys/{key_id}/usage/endpoints`.

> [!NOTE]
>
> Usage endpoints are keyed by **`key_id`** (e.g. `key_a1b2…`), not the key's
> `name`. Grab the `key_id` from the list response above. Your billing usage
> records also carry the attributing `key_id`, so credit spend reconciles to the
> exact key that drove it.

## Key types

Most keys you create are **standard** keys, but Mixpeek issues a few specialized
types for different jobs:

| Type | Prefix | Purpose |
| - | - | - |
| Standard | `mxp_sk_` | Regular organization key. Everything above. |
| User-scoped | `usr_sk_` | Carries a `principal_id` for [document-level ACL](#end-user-keys-for-multi-tenant-apps). |
| Retriever-scoped | `ret_sk_` | Locked to executing one [published retriever](#retriever-scoped-keys). |
| Marketplace | `sk_marketplace_` | Grants access to a subscribed marketplace retriever. |
| Session | — | Short-lived key Studio mints on login to back the UI. Hidden from key lists; not something you manage. |

### End-user keys for multi-tenant apps

If you're building an app where your own end-users each see only *their* data,
create a key with a `principal_id`. Mixpeek issues a **`usr_sk_`** user-scoped
key, and every document read made with it is automatically filtered to documents
that principal owns or has been granted access to — enforced server-side, so a
tampered client can't widen its own view.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/users/you@acme.com/api-keys" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "enduser-jane", "permissions": ["read"], "principal_id": "user_jane_123" }'
```

> [!NOTE]
>
> `principal_id` is an identifier for an **end-user in your application**, not a
> Mixpeek organization user. This is the building block for row-level security —
> see [Document-Level ACL](/docs/operations/document-acl) for the full ownership and
> grant model, and [Permissions](/docs/platform/permissions) for how it composes with
> external OpenFGA authorization.

### Retriever-scoped keys

To embed a single retriever in a browser app or hand it to a customer, mint a
**`ret_sk_`** key scoped to just that retriever — it can execute the retriever
and nothing else, and its scope/permissions are fixed at creation. Only the
retriever's owner can create one.

`POST /v1/retrievers/{retriever_id}/api-keys`:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/ret_abc123/api-keys" \
  -H "Authorization: Bearer $MIXPEEK_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "public-video-search",
    "expires_at": "2026-12-31T23:59:59Z",
    "allowed_origins": ["https://app.example.com", "https://*.example.com"]
  }'
```

`allowed_origins` works exactly as it does on
[standard keys](#restrict-a-key-to-browser-origins) — a CORS allowlist enforced
for browser requests, and the same defense-in-depth caveat applies. Because a
`ret_sk_` key is designed to be embedded in a browser, always pair it with a
short `expires_at` and treat the key as public.

## Manage keys from Studio

Everything on this page is also available without the API: in
[Studio](https://studio.mixpeek.com), go to **Settings → API Keys** to create,
scope, rotate, revoke, and view per-key usage.

## Related

> **[Security & Tenancy](/docs/operations/security)**
>
> The auth model, namespace isolation, and secrets vault.

> **[Rate Limits & Quotas](/docs/operations/rate-limits-quotas)**
>
> Plan tiers, usage pools, and how `rate_limit_override` fits in.

> **[Document-Level ACL](/docs/operations/document-acl)**
>
> Row-level security driven by user-scoped `principal_id` keys.

> **[Permissions](/docs/platform/permissions)**
>
> Retrieval-time authorization and OpenFGA integration.

> **[API reference — Create key](/docs/api-reference/organization-api-keys/create-api-key)**
>
> Full request/response schema for every key endpoint.

> **[API reference — Per-key usage](/docs/api-reference/organization-usage/get-api-key-usage)**
>
> Usage and endpoint-breakdown response schemas.


# Batch ingestion at scale
Source: https://docs.mixpeek.com/docs/operations/batch-ingestion-at-scale

How to size, submit, and monitor large batch ingestions — chunking, concurrency, automatic recovery, and the limits that apply at each plan tier.

Processing a large bucket (100k–5M objects) is a batching decision, not one API call. This page gives you the sizing model the platform itself uses, so you can pick a chunk size and batch count with numbers instead of guesses.

## Two submission paths

**`POST /v1/collections/{collection_id}/trigger`** creates **one batch containing every object** in the source bucket. There is no server-side splitting. Use it for buckets up to a few thousand objects.

**`POST /v1/buckets/{bucket_id}/batches/bulk-submit`** splits the bucket into batches of `chunk_size` objects (default 1,000, maximum 50,000) and submits each one. Use it for everything larger.

> [!WARNING]
>
> Do not trigger a collection over a very large bucket. The single batch it creates
> runs as one job with a fixed worker ceiling and a 24-hour execution deadline, so
> it under-parallelizes and can time out. Bulk-submit is the large-corpus path.

```python Python theme={null}
from mixpeek import Mixpeek

mp = Mixpeek(api_key="YOUR_API_KEY")

result = mp.buckets.batches.bulk_submit(
    bucket_id="bkt_123",
    collection_ids=["col_abc"],
    chunk_size=20000,
)
print(result.batch_ids)
```

```javascript JavaScript theme={null}
import { Mixpeek } from "@mixpeek/sdk";

const mp = new Mixpeek({ apiKey: "YOUR_API_KEY" });

const result = await mp.buckets.batches.bulkSubmit({
  bucketId: "bkt_123",
  collectionIds: ["col_abc"],
  chunkSize: 20000,
});
console.log(result.batchIds);
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/bkt_123/batches/bulk-submit" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace" \
  -H "Content-Type: application/json" \
  -d '{"collection_ids": ["col_abc"], "chunk_size": 20000}'
```

## Picking a chunk size

Each batch runs as its own job. A job scales its workers with object count — roughly one CPU worker per 500 objects — up to a per-job worker ceiling. A batch of 16,000–20,000 objects saturates one job's parallelism. Larger batches do not run faster; they only raise the cost of a mid-run failure.

Going too small hurts the other way. Every batch pays a cluster cold-start, and only a fixed number of batches run at once (see the tier table). 1,200 batches of 1,000 objects serialize into deep queue waves and spend a large share of wall-clock on startup.

| Corpus size | Suggested `chunk_size` | Resulting batches |
| - | - | - |
| under 5k | one trigger call | 1 |
| 5k – 100k | 5,000–10,000 | 1–20 |
| 100k – 1M | 20,000 | 5–50 |
| 1M+ | 20,000–50,000 | 20–100+ |

Calibrate before you commit: run one small batch (1,000 objects) and one at your target size, compare objects/hour and credits from the [Billing usage page](/docs/platform/billing), then extrapolate. Media mix changes cost more than batch size does.

## Concurrency and queueing

Submission is accept-and-queue. Batches past your concurrency limit park as `QUEUED` and promote automatically, first-in-first-out, as running batches finish. You do not pace submissions yourself.

| Plan | Max objects per batch | Concurrent running batches |
| - | - | - |
| Free | 10,000 | 1 |
| Pro | 50,000 | 5 |
| Team | 500,000 | 10 |
| Enterprise | 5,000,000 | 50 |

Submission returns `429` only when a queue is truly full. Each running job also has a 24-hour execution deadline.

## Automatic recovery

The platform heals transient failures without your involvement:

* Failed tasks retry 3 times with exponential backoff.
* A worker that dies mid-task has its work redelivered to another worker.
* Reapers sweep every 1–10 minutes for orphaned, stuck, or stalled batches. They re-drive recoverable ones (bounded retries) and mark the rest failed.
* Failures tagged safe-to-resubmit are resubmitted automatically every 30 minutes.
* Batches that failed on transient errors retry hourly, up to 3 times, with 1/2/4-hour backoff.

Progress is durable per object, not per batch. Each object is recorded in a processing ledger as it completes, so resubmitting a partially-failed batch skips finished objects and reprocesses only the remainder. You do not pay for extraction twice.

Some failures never self-heal by design: authentication errors, schema validation errors, and hard provider quota exhaustion (for example, an exhausted OpenAI balance). These end the affected batches in `FAILED` or `COMPLETED_WITH_ERRORS` and wait for you to fix the cause, then [resubmit](/docs/api-reference/bucket-batches/submit-batch-for-processing).

## Monitoring a run

Watch progress through the API, not by polling logs:

* `GET /v1/buckets/{bucket_id}/batches/{batch_id}` — status, per-tier progress, heartbeat freshness, `documents_written`, `failed_object_count`, and `error_summary`.
* `GET .../batches/{batch_id}/failed-documents` — every failed object with its error, for targeted retries.
* [Batch diagnostics](/docs/troubleshoot/batch-diagnostics) — deeper triage when a batch misbehaves.

Enable the batch [system alerts](/docs/api-reference/alerts) — `batch_failed`, `batch_stalled`, and `batch_error_rate` — before a large run. They evaluate every 5 minutes and notify in-app plus any webhook, Slack, or email channel you configure. Without them, a failed batch is visible only if you go looking for it.

> [!NOTE]
>
> A batch can finish as `COMPLETED_WITH_ERRORS`. Treat that as a work list, not a
> verdict: read `failed-documents`, fix the cause, resubmit, and the ledger skips
> everything already done.


# Change Feed
Source: https://docs.mixpeek.com/docs/operations/change-feed

Pull an ordered, resumable log of everything that changed in your organization, with a 90-day retention window

`GET /v1/events` returns your organization's changes in order. You hold the
position, so a consumer that dies mid-stream resumes exactly where it stopped.

[Webhooks](/docs/operations/webhooks) push the same events to you. The change feed
lets you pull them, which is what you want for a consumer that can be offline,
or one that needs to replay.

## Read the feed

```bash theme={null}
curl -sS "$MP_API_URL/v1/events?limit=100" \
  -H "Authorization: Bearer $MP_API_KEY"
```

```json Response theme={null}
{
  "results": [
    {
      "seq": 84213,
      "cursor": "b3BhcXVlLXN0cmluZw==",
      "namespace_id": "ns_0fda3a08ba",
      "event_type": "collection.documents.written",
      "resource_type": "collection",
      "operation": "insert",
      "payload": { "collection_id": "col_products", "document_ids": ["doc_1"] },
      "created_at": "2026-08-22T19:04:11Z"
    }
  ],
  "next_cursor": "b3BhcXVlLW5leHQ=",
  "has_more": true
}
```

Pass `next_cursor` back as `cursor` to get the next page. Omit `cursor` to start
at the oldest event still inside the retention window.

| Parameter | Meaning |
| - | - |
| `cursor` | Opaque position from a prior `next_cursor`. |
| `namespace_id` | Restrict to one namespace. |
| `event_type` | Restrict to one event type, from the same vocabulary as [webhook subscriptions](/docs/operations/webhooks). |
| `limit` | 1 to 1,000, default 100. |

`results` and `has_more` are always present. An empty page returns the cursor you
sent, unchanged, so a poller can keep the same value and call again.

## The three things that will bite you

> [!WARNING]
>
> **A cursor is only valid for the filter set that produced it.** It encodes a
> position in your organization's overall sequence, not a position within a
> `namespace_id` or `event_type` filter.
>
> Change either filter while reusing an old cursor and the feed silently skips
> whatever the previous combination would have matched in between. There is no
> error. Start from a fresh cursor, or none, whenever the filters change.

> [!WARNING]
>
> **Retention is 90 days and expiry is permanent.** Events older than that are
> deleted, not archived, so there is nothing to resume from.
>
> A consumer down for less than 90 days resumes from its stored `next_cursor`
> and misses nothing. A consumer down longer has to resync current state
> instead. Budget for that path rather than assuming the cursor always works.

> [!WARNING]
>
> **`payload` is not a stable contract.** Its shape is per-event-type and
> mirrors what the originating call site passes to the webhook delivery, so it
> can change when that call site changes.
>
> Read `event_type`, `resource_type`, `operation` and the ids you need, and
> treat everything inside `payload` as advisory. Fetch the resource if you need
> its current state.

## Fields on an event

| Field | Meaning |
| - | - |
| `seq` | Monotonic position within your organization's feed. |
| `cursor` | Opaque cursor pointing at this event. |
| `namespace_id` | Namespace scope, when the event has one. |
| `event_type` | Same vocabulary as webhook subscriptions. |
| `resource_type` | First segment of `event_type`. |
| `operation` | `insert`, `update`, `delete`, or `other`. |
| `payload` | Per-event-type body. See the warning above. |
| `created_at` | When the change happened. |

`seq` is monotonic per organization, so it orders events across namespaces. Use
`cursor` to resume; use `seq` when you need to compare two events' order.

## Batch ingest is covered, and how to trust it

Documents created by batch processing appear in the feed the same way direct
API writes do. After the pipeline confirms a flush of documents into storage,
one `collection.documents.written` event lands per flushed chunk, carrying
`namespace_id`, `collection_id`, `batch_id`, `document_ids` and `count`. A
large batch produces hundreds of chunk events rather than one event per
document, so project by fetching the listed ids.

When the batch finishes, the feed carries one terminal
`collection.documents.batch_completed` event per collection the batch wrote,
with a `total_documents` count taken from the store itself. If the store count
fails, `total_documents` is `null`. It is never a guess.

```json A terminal marker theme={null}
{
  "event_type": "collection.documents.batch_completed",
  "payload": {
    "batch_id": "btch_x",
    "collection_id": "col_products",
    "total_documents": 1204,
    "status": "COMPLETED"
  }
}
```

The marker exists so a consumer can verify its own completeness. Take the
union of `document_ids` across the batch's chunk events and compare its size
against `total_documents`. A match means you saw every write. A mismatch means
some chunk events were lost, and the fix is to list the collection's documents
filtered by that `batch_id` and reconcile.

> [!WARNING]
>
> **Delivery semantics are at-least-once, with one narrow loss window.** A
> chunk event is emitted only after its documents are confirmed in storage, so
> the feed never reports phantom writes. A chunk's event can be lost if the
> writing process dies between the storage confirm and the emit. The terminal
> marker is how you detect that: its store-counted total does not depend on
> the chunk events having survived. Reconcile against it rather than assuming
> the chunk stream is complete.

## Related

* [Webhooks](/docs/operations/webhooks) for push delivery of the same events
* [Observability](/docs/operations/observability)


# Using Mixpeek on corporate networks
Source: https://docs.mixpeek.com/docs/operations/corporate-networks

If Mixpeek won't load behind a corporate proxy or firewall, allowlist these domains — a one-page checklist for IT administrators

If Mixpeek Studio loads a blank page, hangs on sign-in, or "goes dead" a few
seconds after opening on your corporate network, the cause is almost always a
**proxy, firewall, or content-security policy blocking outbound requests** the app
needs to make. Mixpeek's own security policy already permits every domain below —
the block is on the network side, so the fix is a network-side allowlist.

This page is written for an IT administrator to action in one read.

> [!NOTE]
>
> **The symptom:** the browser loads `studio.mixpeek.com`, but calls to the API are
> silently dropped, so the app renders its shell and then does nothing — no data, no
> sign-in, no error. That is a blocked `api.mixpeek.com`, not a Mixpeek outage.

## Minimum allowlist — the app loads and works

Allowlist these for **outbound HTTPS (443)** from the browser. This is the smallest
set that makes Studio and the API fully functional:

| Domain | Purpose |
| - | - |
| `*.mixpeek.com` | Core API (`api.mixpeek.com`), authentication (`auth.mixpeek.com`), and the app itself (`studio.mixpeek.com`) |
| `*.mxp.co` | Mixpeek content-delivery domain the app calls at runtime |
| `js.stripe.com`, `*.stripe.com` | Billing and checkout (Stripe) |
| `*.amazonaws.com` | Media and file delivery — Studio fetches content over signed Amazon S3 URLs |
| `*.googleapis.com` | Media and file delivery — Studio fetches content over signed Google Cloud Storage URLs |

> [!WARNING]
>
> Allowlisting only `*.mixpeek.com` is a common partial fix: the app then **loads
> and signs in, but thumbnails, video, and file previews stay blank** because media
> is served over signed `*.amazonaws.com` / `*.googleapis.com` URLs. Include the
> storage domains, or evaluators will conclude the product is broken when it is only
> blocked.

### Scoped storage hosts for strict environments

If your policy does not permit whole-provider wildcards like `*.amazonaws.com`,
replace the two storage entries above with these narrower hosts. They cover all
media delivery and nothing else — no other AWS or Google service:

| Instead of | Allowlist |
| - | - |
| `*.googleapis.com` | `storage.googleapis.com` — the single host all signed media resolves to |
| `*.amazonaws.com` | `s3.*.amazonaws.com`, `*.s3.amazonaws.com`, `*.s3.*.amazonaws.com` — Amazon S3 only (path-style and virtual-hosted, all regions), e.g. `your-bucket.s3.us-east-2.amazonaws.com` |

> [!NOTE]
>
> Two `googleapis.com` hosts are needed **only if** you sync from Google Drive
> (`drive.googleapis.com` and `www.googleapis.com`, for the connector's OAuth scope).
> They are **not** required to view media — leave them out of a viewing-only
> evaluation. If you bring your own object storage, media resolves to *your* bucket
> host instead; allowlist that host.

## Optional — enhances the experience, not required

The app is fully usable without these. Allowlist them to enable in-product analytics,
support chat, and error reporting:

| Domain | Purpose |
| - | - |
| `*.posthog.com` (`us.i.posthog.com`, `us-assets.i.posthog.com`) | Product analytics |
| `*.intercom.io`, `*.intercomcdn.com` | In-app support chat |
| `*.ingest.us.sentry.io` | Client-side error reporting |
| `*.cloudflareinsights.com` | Performance telemetry |

## Protocols and ports

| Requirement | Value |
| - | - |
| Protocol | HTTPS |
| Port | 443 |
| WebSockets | Allow `wss://` to the domains above (Studio uses live connections for some features) |
| TLS interception | If your proxy does TLS/SSL inspection, it must present a certificate the browser trusts **and** preserve the allowlist above; deep-inspection proxies frequently strip the calls that break the app |

## How to confirm it's a network block (not Mixpeek)

Have the affected user, on the corporate network:

**1. Open the browser developer console**

In Studio, open DevTools (F12 or ⌘⌥I) → **Console** and **Network** tabs, then reload.

**2. Look for blocked requests**

Requests to `api.mixpeek.com` (or the domains above) showing `blocked`,
`net::ERR_BLOCKED_BY_CLIENT`, a CSP violation, or a proxy error page — rather than
a normal `200`/`4xx` from Mixpeek — confirm the network is dropping them.

**3. Compare off-network**

The same account on a non-corporate network (or personal hotspot) loading normally
is definitive: the account and the service are fine; the corporate network is the
variable.

> [!TIP]
>
> Fastest path for an evaluation: allowlist the **Minimum** table above and reload.
> If sign-in works but media is blank, you're missing the two storage domains.

## Still blocked?

If the domains are allowlisted and Studio still won't load, contact
[support@mixpeek.com](mailto:support@mixpeek.com) with the browser console's Network
tab (screenshot or HAR export) — the blocked request URLs tell us exactly which rule
is still catching them.


# Deployment
Source: https://docs.mixpeek.com/docs/operations/deployment

Run Mixpeek in Kubernetes or on managed Ray

Mixpeek is split into two deployable components:

* **API Layer** – FastAPI and a Redis connection (HTTP endpoints, task orchestration, webhooks). Background work is published to the work runtime, whose ledger is in MongoDB and whose units run on Ray.
* **Engine Layer** – Ray cluster + Ray Serve (extractors, inference, clustering, taxonomy runs).

Shared dependencies: MongoDB, [MVS](https://mixpeek.com/mvs), Redis, and S3-compatible object storage.

## Production Topology (Kubernetes)

```
Namespace: mixpeek-api
  └─ api-deployment (ReplicaSet + HPA; serves HTTP and publishes background work)

Namespace: kuberay-system
  └─ kuberay-operator (one per CLUSTER, watches every namespace)

Namespace: mixpeek-engine
  └─ RayService/mixpeek-engine-svc   (you apply this; KubeRay renders the rest)
       ├─ head pod          rayStartParams num-cpus: 0, schedules only
       ├─ cpu-workers       autoscaled group
       ├─ batch-workers     autoscaled group
       └─ gpu-workers       autoscaled group

Namespace: mixpeek-data
  ├─ mongodb (StatefulSet, replica set)
  ├─ mvs (StatefulSet or distributed cluster)
  └─ redis (Deployment or Redis Cluster)
```

Recommended node pools:

* **API nodes** – general purpose (e.g., `t3.xlarge`), scale the API pods horizontally.
* **CPU workers** – compute-optimized (e.g., `c5.4xlarge`) for text extraction, clustering.
* **GPU workers** – GPU instances (e.g., `p3.2xlarge`) for embeddings, rerankers, video processing.

Expose the API via an ingress or load balancer; keep Ray Serve internal unless exposing custom inference endpoints.

## Ray on Kubernetes (KubeRay)

> [!NOTE]
>
> This page describes Mixpeek operating a Ray cluster it deploys. If you already
> run Ray or KubeRay and want Mixpeek to use **yours** instead, see
> [Bring Your Own Ray](/docs/customer-hosted/overview) — it covers the three contracts, [every
> permission we ask for](/docs/customer-hosted/security-rbac), and [what we check before
> deploying anything](/docs/customer-hosted/qualification).

The Engine layer runs as a **RayService**, a custom resource the KubeRay operator
reconciles. You apply one YAML; KubeRay creates the head pod, the worker groups, and the
services in front of them, and replaces them on config change.

You do not create Ray pods yourself, and there is nothing to scale by hand. Each worker
group carries its own `minReplicas` and `maxReplicas`, and Ray's autoscaler moves within
that range on pending-task pressure.

The API layer reaches the cluster over `ENGINE_API_URL`, a plain in-cluster HTTP address
such as `http://mixpeek-engine-svc-head-svc.mixpeek-engine:8265`.

> [!WARNING]
>
> Job submission carries **no credential**. The API builds a bare `http://host:port` to
> the Ray dashboard and submits against it, so reachability is the whole access control.
> Keep the Ray dashboard port off any public ingress and restrict it with a
> NetworkPolicy. Anything that can reach the port can submit a job.

### Namespace-scoped or cluster-wide

Two different scopes, and only one of them is per-deployment:

| Component | Scope | Installed |
| - | - | - |
| KubeRay operator | Cluster | Once, in `kuberay-system`. It watches all namespaces. |
| RayService | Namespace | Once per deployment, in that deployment's namespace. |

So a second Mixpeek deployment on the same cluster means a second namespace with its own
RayService, not a second operator. The operator needs cluster-level permission to create
pods and services in the namespaces it watches, which is the one privilege you cannot
scope away.

Isolate deployments by namespace, one per tenant, with a `LimitRange` and a
`NetworkPolicy` on each. Separate node pools with taints keep GPU work off shared
capacity, since namespaces alone do not partition nodes.

## Core Environment Variables

| Service | Key | Description |
| - | - | - |
| API | `MONGO_URI`, `MVS_URL`, `REDIS_URL`, `S3_BUCKET`, `ENGINE_API_URL` | Connectivity |
| Engine | `MONGO_URI`, `MVS_URL`, `S3_BUCKET`, `RAY_memory`, `RAY_num_gpus` | Runtime config |
| Shared | `ENABLE_ANALYTICS`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc. | Optional providers |

Secrets should be injected via Kubernetes secrets, environment managers, or cloud secret stores.

## Health & Verification

* **Endpoint**: `GET /v1/health/deep` – checks Redis, MongoDB, MVS, object storage, the task queue and the engine. `GET /v1/health` is a liveness constant.
* **Smoke test**: create namespace → bucket → collection → upload object → submit batch → execute retriever.
* **Tasks**: `GET /v1/health/beat` confirms the work scheduler fired within the last 5 minutes, so webhook events, cache invalidations and maintenance tasks run.

## Scaling Guidelines

| Component | Scaling Strategy | Notes |
| - | - | - |
| FastAPI | Horizontal autoscale on CPU/utilization | Stateless, use HPA |
| Work runtime | Units run on the Ray worker pools; the ledger is in MongoDB | A unit outlives the pod that published it. Cancel a task with `POST /v1/tasks/{task_id}/kill` |
| Ray workers | Autoscale CPU/GPU pools | Use Ray autoscaler or managed Ray policies |
| MVS | Scales automatically with object storage backend | Monitor vector count and query latency |
| MongoDB | Use managed cluster (Atlas, DocumentDB) | Ensure indexes on `namespace_id`, `internal_id` |
| Redis | Scale vertically or cluster | Used for task queue + caching |

Monitor Ray dashboard (port 8265) for job status, resource utilization, and Serve deployments.

## Deployment Checklist

1. Provision MongoDB, MVS, Redis, and S3/GCS buckets (with IAM roles).
2. Deploy Ray cluster (head + workers) and confirm job submission works.
3. Deploy the API; configure environment variables to point to Ray + data stores.
4. Configure ingress/HTTPS, secrets, and network policies.
5. Run health checks and quickstart workflow to verify end-to-end functionality.
6. Set up observability (logs, metrics, webhooks) and configure backups for MongoDB.

## References

* [Architecture](/docs/overview/architecture) – full system design
* [Observability](/docs/operations/observability) – metrics, logs, dashboards
* [Security](/docs/operations/security) – tenancy, auth, secret management
* [Webhooks](/docs/operations/webhooks) – event processing pipeline


# Document-Level ACL
Source: https://docs.mixpeek.com/docs/operations/document-acl

Row-level security for multi-user applications with automatic access control on documents

## Overview

Document-level ACL (Access Control Lists) lets you build multi-user applications on top of Mixpeek where each end-user only sees the documents they are authorized to access. Instead of managing separate namespaces per user, you store all documents in a single namespace and let Mixpeek enforce read/write permissions automatically.

Key capabilities:

* **Automatic filter injection** — user-scoped API keys transparently filter all reads so users only see their own (or shared) documents
* **Per-document ownership** — every document has an `_acl` object tracking who can read, write, or own it
* **Public documents** — mark a document as `public: true` to make it visible to all users
* **Zero application-side filtering** — your app does not need to add user filters to queries; Mixpeek handles it server-side

## Concepts

### API Key Types

Mixpeek supports two types of API keys:

| Key Type | Prefix | ACL Behavior |
| - | - | - |
| **Org-scoped** | `mxp_sk_` | Full access — bypasses ACL filters entirely |
| **User-scoped** | `usr_sk_` | Restricted — ACL filters injected on every read and write |

Org-scoped keys are for your backend services and admin operations. User-scoped keys are for end-user sessions in your application.

### The `_acl` Object

Every document stores an `_acl` sub-object inside its `_internal` payload:

```json theme={null}
{
  "_internal": {
    "_acl": {
      "owner": "user_123",
      "read": ["user_123", "user_456"],
      "write": ["user_123"],
      "public": false
    }
  }
}
```

**\_acl.owner** `string` *required*

The `principal_id` of the user who created the document. Only the owner (or an org-scoped key) can modify ACL settings.

**\_acl.read** `string[]` *required*

List of `principal_id` values that can read this document.

**\_acl.write** `string[]` *required*

List of `principal_id` values that can update this document.

**\_acl.public** `boolean` *required*

When `true`, the document is visible to all user-scoped keys regardless of the `read` list.

## Creating User-Scoped API Keys

Generate a user-scoped key by providing a `principal_id` that represents the end-user in your application (e.g., your internal user ID, email, or UUID).

```python Python theme={null}
from mixpeek import Mixpeek

# Use your org-scoped key to create a user-scoped key
client = Mixpeek(api_key="mxp_sk_your-org-key")

user_key = client.organizations.api_keys.create(
    principal_id="user_123",  # setting principal_id makes the key user-scoped
    description="Key for user 123"
)

print(user_key.api_key)  # usr_sk_...
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk'

const client = new Mixpeek({ apiKey: 'mxp_sk_your-org-key' })

const userKey = await client.organizations.apiKeys.create({
  principalId: 'user_123', // setting principalId makes the key user-scoped
  description: 'Key for user 123'
})

console.log(userKey.apiKey) // usr_sk_...
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/organizations/users/{user_email}/api-keys \
  -H "Authorization: Bearer mxp_sk_your-org-key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "user-123-key",
    "principal_id": "user_123",
    "description": "Key for user 123"
  }'
```

> [!NOTE]
>
> Store the returned `usr_sk_` key securely. Pass it to your frontend or mobile app so the end-user's requests are automatically scoped.

## How ACL Works

### Automatic ACL on Document Creation

When a user-scoped key creates a document, Mixpeek automatically sets the `_acl`:

* `owner` = the key's `principal_id`
* `read` = `[principal_id]`
* `write` = `[principal_id]`
* `public` = `false`

You do not need to set `_acl` manually — it is populated from the API key's identity.

### Automatic Filter Injection on Reads

Every read operation made with a user-scoped key — including retriever executions, document listings, and searches — has an ACL filter injected server-side. The filter ensures the user only sees documents where:

1. Their `principal_id` is in the `_acl.read` list, **OR**
2. The document has `_acl.public: true`

This happens transparently. Your application code does not need to add any user-specific filters.

### Write Protection

Update and delete operations made with a user-scoped key check the `_acl.write` list. If the user's `principal_id` is not in the write list, the operation returns a `403 Forbidden` error.

### Org-Scoped Key Bypass

Org-scoped keys (`mxp_sk_` prefix) bypass all ACL checks. They can read, write, and modify any document regardless of its `_acl` settings. Use org-scoped keys for backend services, admin dashboards, and data pipelines.

## Managing ACL

Update a document's ACL using the dedicated endpoint. Only the document owner or an org-scoped key can modify ACL settings.

```python Python theme={null}
# Share a document with another user (add read access)
client = Mixpeek(api_key="usr_sk_owner-key")

client.collections.documents.update_acl(
    collection_id="col_abc123",
    document_id="doc_xyz789",
    read=["user_123", "user_456"],  # Add user_456 to readers
)
```

```javascript JavaScript theme={null}
const client = new Mixpeek({ apiKey: 'usr_sk_owner-key' })

await client.collections.documents.updateAcl({
  collectionId: 'col_abc123',
  documentId: 'doc_xyz789',
  read: ['user_123', 'user_456'] // Add user_456 to readers
})
```

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/collections/col_abc123/documents/doc_xyz789/acl \
  -H "Authorization: Bearer usr_sk_owner-key" \
  -H "Content-Type: application/json" \
  -d '{
    "read": ["user_123", "user_456"],
    "write": ["user_123"],
    "public": false
  }'
```

### Making a Document Public

Set `public: true` to make a document visible to all user-scoped keys in the namespace:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/collections/col_abc123/documents/doc_xyz789/acl \
  -H "Authorization: Bearer usr_sk_owner-key" \
  -H "Content-Type: application/json" \
  -d '{
    "public": true
  }'
```

### Revoking Access

Remove a user from the `read` or `write` list by sending the updated list without their `principal_id`:

```bash theme={null}
# Remove user_456 from readers — only user_123 retains access
curl -X PATCH https://api.mixpeek.com/v1/collections/col_abc123/documents/doc_xyz789/acl \
  -H "Authorization: Bearer usr_sk_owner-key" \
  -H "Content-Type: application/json" \
  -d '{
    "read": ["user_123"],
    "write": ["user_123"]
  }'
```

## Examples

### Multi-User Document Search

A SaaS app where each user uploads and searches their own documents, with optional sharing.

**1. Create user-scoped keys for each user**

When a user signs up, generate a `usr_sk_` key with their user ID as the `principal_id`.

**2. Users ingest documents with their key**

Each document is automatically tagged with the user's ACL. User A cannot see User B's documents.

**3. Users search normally**

Retriever executions with a user-scoped key only return documents the user can access — no extra filters needed.

**4. Share a document**

User A shares a document with User B by adding User B's `principal_id` to the `read` list via the ACL endpoint.

### Public Knowledge Base with Private Uploads

Combine public reference documents with private user uploads in the same namespace:

```python Python theme={null}
# Admin creates public reference docs using org-scoped key
admin_client = Mixpeek(api_key="mxp_sk_org-key")

# Create a document and make it public
doc = admin_client.collections.documents.create(
    collection_id="col_kb",
    data={"title": "Getting Started Guide", "content": "..."}
)

admin_client.collections.documents.update_acl(
    collection_id="col_kb",
    document_id=doc.document_id,
    public=True,
)

# End-user creates private notes using their user-scoped key
user_client = Mixpeek(api_key="usr_sk_user-key")

user_client.collections.documents.create(
    collection_id="col_kb",
    data={"title": "My private notes", "content": "..."}
)
# _acl automatically set: owner=user, read=[user], write=[user], public=false
```

When the end-user searches, they see both the public reference docs and their own private notes — but not other users' private documents.

## Backwards Compatibility

Documents created before ACL was enabled do not have an `_acl` field. These documents follow these rules:

| Key Type | Behavior for Documents Without `_acl` |
| - | - |
| **Org-scoped** | Full access — documents are visible and writable as before |
| **User-scoped** | **Not visible** — documents without `_acl` are excluded from reads |

> [!WARNING]
>
> If you adopt user-scoped keys on an existing namespace, pre-existing documents will be invisible to end-users until you set their `_acl` (either individually or via a bulk update). Org-scoped keys continue to work normally.

To backfill ACL on existing documents, use a bulk update with an org-scoped key. Omitting
`filters` applies the update to every document in the collection:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/collections/col_abc123/documents/bulk \
  -H "Authorization: Bearer mxp_sk_org-key" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "update_data": {
      "_internal._acl": {
        "owner": "system",
        "read": [],
        "write": [],
        "public": true
      }
    }
  }'
```

> [!WARNING]
>
> **This returns before the backfill has run.** The response carries `task_id` and
> `status: "PENDING"`, and `updated_count` is `null`. Poll `GET /v1/tasks/{task_id}` until
> it reports a terminal status, and only then treat the documents as visible to
> user-scoped keys.

To scope the backfill instead of applying it to everything, send `filters` in the
`{field, operator, value}` shape:

```json theme={null}
{
  "filters": {"AND": [{"field": "metadata.status", "operator": "eq", "value": "pending"}]},
  "update_data": {"_internal._acl": {"owner": "system", "read": [], "write": [], "public": true}}
}
```

## Best Practices

> [!TIP]
>
> Use your own stable user identifiers (UUIDs, database IDs) as `principal_id` values. Avoid using email addresses, which can change.

> [!TIP]
>
> Keep org-scoped keys on your backend only. Never expose `mxp_sk_` keys to client-side code. User-scoped `usr_sk_` keys are safe to use in frontend and mobile apps.

> [!WARNING]
>
> ACL is enforced at the Mixpeek API layer. If you have direct access to the underlying MVS or MongoDB instances, ACL filters are not applied. Always route end-user traffic through the Mixpeek API.

## Related

* [Security & Tenancy](/docs/operations/security) — org-level authentication and namespace isolation
* [Filters](/docs/retrieval/filters) — manual query filters (ACL filters are injected automatically on top of these)
* [Namespaces](/docs/ingestion/namespaces) — data isolation boundaries


# Environment Branching
Source: https://docs.mixpeek.com/docs/operations/environment-branching

Clone namespaces, collections, retrievers, and taxonomies to create isolated staging environments and run experiments without re-processing data

## Overview

Branching an AI data environment used to mean one of two things: re-processing your entire corpus (slow, expensive) or experimenting directly in production (dangerous). Mixpeek solves this with a **clone-based branching model** that operates at every layer of the pipeline — namespace, collection, retriever, and taxonomy — so you can create fully isolated environments instantly and promote changes deliberately.

This matters most when:

* You want to test a new retrieval pipeline on production data without affecting live traffic
* You need a staging namespace that mirrors production for QA without re-ingesting everything
* You're evaluating a new embedding model and want side-by-side comparison on the same corpus
* A taxonomy schema is about to change and you need a safe place to validate it first

## Branching Primitives

Mixpeek resources are **immutable by design** — you can't change a collection's feature extractor or a retriever's pipeline stages via PATCH. This preserves execution history, dependent results, and audit trails. The branching mechanism is a first-class `clone` operation available on every resource type.

### Namespace Clone (full environment branch)

The broadest primitive. A namespace clone deep-copies the entire environment: collections (including MVS vectors), retrievers, buckets, and optionally taxonomies — remapping all internal IDs so nothing points back to production.

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

clone = client.namespaces.clone(
    "ns_prod",
    namespace_name="listings_staging",
    include_resources={
        "collections": True,   # copies MVS vectors — no reprocessing
        "retrievers": True,    # remaps all collection refs to staging copies
        "taxonomies": False    # optional: include taxonomy configs
    }
)

# The response returns immediately with the new namespace and a clone status.
new_namespace_id = clone["namespace"]["namespace_id"]
print(clone.status)            # "cloning" -> completes async to "ready"/"failed"
print(clone.cloned_resources)  # what was copied (source_id -> cloned_id per resource)
```

```javascript JavaScript theme={null}
import { Mixpeek } from 'mixpeek-sdk';

const client = new Mixpeek({ apiKey: 'your-api-key' });

const clone = await client.namespaces.clone('ns_prod', {
  namespace_name: 'listings_staging',
  include_resources: {
    collections: true,
    retrievers: true,
    taxonomies: false,
  },
});

const newNamespaceId = clone.namespace.namespace_id;
console.log(clone.status);           // "cloning" -> completes async to "ready"
console.log(clone.cloned_resources); // source_id -> cloned_id per resource
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/namespaces/ns_prod/clone \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: ns_prod" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_name": "listings_staging",
    "include_resources": {
      "collections": true,
      "retrievers": true,
      "taxonomies": false
    }
  }'
```

> [!NOTE]
>
> Namespace clone copies MVS vectors directly — your data is **not re-processed**. The clone is isolated: changes to collections, retrievers, or documents in staging have zero effect on production.

**What gets copied:**

| Resource | What's cloned | Notes |
| - | - | - |
| Collections | Metadata + MVS vectors | Vectors copied, not recomputed |
| Retrievers | Full pipeline stage config | All collection refs remapped to staging |
| Buckets | Metadata only | S3 objects are not duplicated |
| Taxonomies | Config only (optional) | Retriever refs remapped |

### Collection Clone (swap extractor or source)

Clone a single collection and optionally override its feature extractor or source. This is the entry point for **embedding model experimentation** — run two extractors on the same corpus and compare retrieval quality.

```python Python theme={null}
# Clone with a different embedding model
new_col = client.collections.clone(
    "col_properties",
    collection_name="properties_siglip_v2",
    feature_extractor={
        "feature_extractor_name": "image_extractor",
        "version": "v2",
        "parameters": { "model": "google_siglip_base_v1" }
    }
)

# Trigger reprocessing — required when changing the extractor
client.collections.trigger(new_col["collection_id"])
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/collections/col_properties/clone \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "properties_siglip_v2",
    "feature_extractor": {
      "feature_extractor_name": "image_extractor",
      "version": "v2",
      "parameters": { "model": "google_siglip_base_v1" }
    }
  }'
```

> [!NOTE]
>
> When you clone a collection without changing the feature extractor, vectors are reused. When you **change the extractor**, you must trigger reprocessing — vectors are model-specific and cannot be ported across embedding spaces.

### Retriever Clone (pipeline experiment)

Immutable retriever stages mean the safe way to test a new ranking strategy, add a rerank stage, or adjust fusion weights is to clone the retriever with overrides.

```python Python theme={null}
# Clone retriever and add an MMR rerank stage
new_ret = client.retrievers.clone(
    "ret_ad_relevance",
    body={
        "retriever_name": "ad_relevance_mmr_v2",
        "stages": [
            {
                "stage_name": "search",
                "stage_type": "filter",
                "config": {
                    "stage_id": "feature_search",
                    "parameters": {
                        "feature_uri": "mixpeek://text_extractor@v1/e5_large",
                        "query": "{{INPUT.query}}",
                        "final_top_k": 50
                    }
                }
            },
            {
                "stage_name": "diversify",
                "stage_type": "sort",
                "config": {
                    "stage_id": "mmr",
                    "parameters": { "lambda": 0.7, "top_k": 20 }
                }
            }
        ]
    }
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers/ret_ad_relevance/clone \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "ad_relevance_mmr_v2",
    "stages": [...]
  }'
```

### Taxonomy Clone (schema version branch)

Taxonomies are immutable in their core config (`retriever_id`, `input_mappings`, `enrichment_fields`). Clone to branch a schema — swap the backing retriever, adjust the hierarchy, or test a new classification model.

```python Python theme={null}
# Branch a taxonomy to test a new classification retriever
new_tax = client.taxonomies.clone(
    taxonomy_identifier="tax_icd_codes",
    body={
        "taxonomy_name": "icd_codes_llama_v2",
        "config": {"retriever_id": "ret_llama_classifier"}  # only changed field
    }
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/taxonomies/tax_icd_codes/clone \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "taxonomy_name": "icd_codes_llama_v2",
    "config": { "retriever_id": "ret_llama_classifier" }
  }'
```

***

## Common Patterns

### Pattern 1: Staging environment for a production namespace

The most common use case: a full mirror of production where QA teams can validate changes before go-live.

```
prod namespace (ns_prod)
├── col_content_v1       (CLIP embeddings, 2M docs)
├── ret_content_search   (semantic + rerank pipeline)
└── tax_iab_v3           (IAB 3.0 taxonomy)

→ clone → staging namespace (ns_staging)
├── col_content_v1_copy  (vectors copied, no reprocessing)
├── ret_content_search_copy  (points to staging collection)
└── [taxonomies excluded]
```

After cloning, engineers can modify the staging retriever pipeline, run evaluations, and only promote to prod once quality gates pass.

### Pattern 2: Embedding model A/B test

Run two embedding models on the same corpus, then compare retrieval quality with Mixpeek's [Evaluations](/docs/retrieval/evaluations) before committing.

```python theme={null}
# Collection A — existing model (no reprocessing needed)
col_a = "col_listings_clip"   # already exists

# Collection B — new model (clone + reprocess)
col_b = client.collections.clone("col_listings_clip", {
    "collection_name": "col_listings_siglip",
    "feature_extractor": {
        "feature_extractor_name": "image_extractor",
        "version": "v1",
        "parameters": { "model": "google_siglip_base_v1" }
    }
})
client.collections.trigger(col_b["collection_id"])

# Retriever A — current model
ret_a = "ret_property_search"

# Retriever B — points to new collection
ret_b = client.retrievers.clone("ret_property_search", {
    "retriever_name": "ret_property_search_siglip",
    "collection_identifiers": [col_b["collection_id"]]
})

# Run evaluations side by side
eval_a = client.retrievers.run_evaluation("ret_property_search", dataset_id="eval_ds_001")
eval_b = client.retrievers.run_evaluation(ret_b["retriever_id"], dataset_id="eval_ds_001")
```

### Pattern 3: Retriever pipeline experiment

Test a new retrieval stage (reranker, MMR, query expansion) without touching the live retriever.

```python theme={null}
# Current: bare semantic search
# Experiment: add query expansion + rerank

exp_retriever = client.retrievers.clone("ret_content_search", {
    "retriever_name": "ret_content_search_exp_rerank",
    "stages": [
        { "stage_type": "filter", "config": { "stage_id": "query_expand", "parameters": {...} } },   # new
        { "stage_type": "filter", "config": { "stage_id": "feature_search", "parameters": {...} } },
        { "stage_type": "sort", "config": { "stage_id": "rerank", "parameters": {...} } }             # new
    ]
})

# Shadow-test: run both retrievers on the same queries, compare metrics
```

### Pattern 4: Taxonomy version checkpoint

Before migrating an IAB or ICD taxonomy schema, snapshot the current version so you can validate in parallel.

```python theme={null}
# Snapshot before migration
checkpoint = client.taxonomies.clone("tax_iab_content", {
    "taxonomy_name": "tax_iab_content_v3_snapshot"
})

# Apply to existing docs in the new taxonomy to validate
client.taxonomies.apply(
    taxonomy_identifier=checkpoint.taxonomy_id,
    collection_id="col_content_sample_100"
)
```

***

## Vertical Examples

> **Adtech — IAB taxonomy migration**
>
> **Problem:** Your IAB 2.2 taxonomy needs to be upgraded to IAB 3.0. You can't migrate production mid-campaign without validating classification quality first.
>
> **Solution:**
>
> 1. Clone the production namespace → `ns_adtech_staging`
> 2. Clone `tax_iab_v2` with the new retriever trained on IAB 3.0 → `tax_iab_v3_candidate`
> 3. Apply `tax_iab_v3_candidate` to a sample of staging documents
> 4. Validate label quality against ground truth
> 5. Promote: update production taxonomy config once quality gates pass
>
> No live campaigns are affected. Staging vectors are reused from production (no reprocessing).

> **Healthcare — clinical classification model upgrade**
>
> **Problem:** You're switching the ICD-10 classification retriever from GPT-4o to a fine-tuned clinical model. Patient record classification in production cannot be interrupted.
>
> **Solution:**
>
> 1. Clone `tax_icd10_prod` with the new retriever → `tax_icd10_clinical_v2`
> 2. Apply to a test collection of de-identified records
> 3. Compare classification accuracy against the production taxonomy's output on the same records
> 4. Only swap production once the new model meets or exceeds current accuracy
>
> Both taxonomies run in parallel. There's no downtime and no disruption to the production classification pipeline.

> **Media — search ranking experiment**
>
> **Problem:** Editorial wants to test a diversity-aware ranking algorithm (MMR) on the content search endpoint before rolling out to all users.
>
> **Solution:**
>
> 1. Clone `ret_content_search` → `ret_content_search_mmr`
> 2. Override stages to add an MMR sort step after semantic search
> 3. Route 10% of internal QA traffic to the experimental retriever (traffic splitting handled in your application layer)
> 4. Compare CTR, dwell time, and result diversity metrics
> 5. Promote by cloning the staging retriever config back to production
>
> The collection (and all its vectors) is shared between both retrievers — no extra storage cost.

> **CRE — new embedding model for property images**
>
> **Problem:** A new image embedding model shows better performance on architectural/interior photos. You want to validate before migrating 5M property images.
>
> **Solution:**
>
> 1. Clone `col_property_images` with the new extractor → `col_property_images_v2`
> 2. Trigger reprocessing (required for model changes — vectors are model-specific)
> 3. Create `ret_property_search_v2` pointing to the new collection
> 4. Run offline evaluation: compare top-5 retrieval precision on a labeled query set
> 5. If metrics improve, migrate production: update `ret_property_search` to point to the new collection
>
> The old collection stays active during migration as a fallback. Roll back is instant — just repoint the retriever.

***

## Promotion Workflow

Branching is only useful if you have a clear path back to production. The recommended flow:

```
dev branch  →  staging clone  →  eval gate  →  production
     ↑                                              |
     └──────────── rollback (repoint retriever) ───┘
```

**Promoting a retriever experiment to production:**

1. Run [Evaluations](/docs/retrieval/evaluations) on the experimental retriever
2. If metrics pass, delete the old production retriever (after confirming no dependent published pages)
3. Rename the experimental retriever to the production name via PATCH (name is mutable)
4. Or: update your application to point to the new retriever ID directly

**Rolling back** is always safe — because the old retriever and collection still exist unchanged, you can revert by updating the retriever ID in your application config.

***

## Best Practices

> [!TIP]
>
> **One namespace per environment** (dev, staging, prod). Clone from prod to create staging rather than maintaining them separately — this guarantees staging always reflects current production data and config.

* **Clone, don't modify.** Resist the urge to patch production resources for "quick experiments." A clone takes seconds and preserves the ability to roll back.
* **Retriever clones are free** — they share the underlying collection (and all its vectors). You only pay for additional MVS storage when collection vectors diverge.
* **Trigger reprocessing only when the extractor changes.** Cloning a collection with the same extractor reuses existing vectors — no GPU time consumed.
* **Use evaluations before promoting.** The [Evaluations API](/docs/retrieval/evaluations) lets you run offline quality checks on any retriever before it touches production traffic.
* **Name branches consistently.** A naming convention like `{resource}_staging`, `{resource}_exp_{date}`, or `{resource}_v{n}` makes it easy to identify which resources are active experiments vs. production.
* **Clean up stale branches.** Delete experimental collections and retrievers after promotion or abandonment. MVS vectors from branched collections consume storage until deleted.

***

## Related

* [Namespaces](/docs/ingestion/namespaces) — isolation boundaries and multi-tenancy
* [Collections](/docs/ingestion/collections) — processing pipelines and lifecycle states
* [Retrievers](/docs/retrieval/retrievers) — pipeline stages and configuration
* [Evaluations](/docs/retrieval/evaluations) — offline quality testing before promotion


# Manifests
Source: https://docs.mixpeek.com/docs/operations/manifests

Declarative resource configuration with YAML manifests

Manifests let you define Mixpeek resources in YAML files and apply them in a single operation. This enables version-controlled, reproducible infrastructure across environments.

## Quick Start

```yaml theme={null}
# mixpeek.yaml — a complete pipeline: ingest → index → search
version: "1.0"
metadata:
  name: "video-search-env"

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1

buckets:
  - name: raw_videos
    namespace: video_search
    schema:
      properties:
        video: { type: video }

collections:
  - name: video_index
    namespace: video_search
    source:
      type: bucket
      bucket: raw_videos
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v1

retrievers:
  - name: video_search_tool
    namespace: video_search
    collections: [video_index]
    input_schema:
      query:
        type: text
        required: true
    stages:
      - stage_name: search
        stage_type: filter
        config:
          stage_id: feature_search
          parameters:
            searches:
              - collection_identifiers: [video_index]
                query:
                  input_mode: text
                  text: "{{INPUT.query}}"
                feature_uri: "mixpeek://multimodal_extractor@v1/embedding"
                top_k: 10
            final_top_k: 10
```

Every collection needs a `source` and a `feature_extractor`; every retriever needs `namespace`, `collections`, and its `stages` pipeline. `validate` names any missing field, warns on unknown top-level keys (a typo like `retreivers:` won't silently drop your search pipeline), and checks that references (`namespace`, `bucket`, collection names) resolve within the manifest.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

### Applying over resources that already exist

`apply` takes a `mode` query parameter. It defaults to `create_only`, which **fails if any
resource in the manifest already exists**, because the manifest is treated as a closed world:
a reference to a bucket that exists but is not declared is a validation error.

Use `create_missing` to add to a namespace that already has resources. It creates what is
missing, leaves what exists alone, and resolves references against the resources already in
the namespace.

```bash theme={null}
# Add these collections over buckets that are already there
curl -X POST "https://api.mixpeek.com/v1/manifest/apply?mode=create_missing" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@collections.yaml"
```

`apply` plans before it creates anything. It runs the same checks as `validate`, and any
error refuses the whole manifest: nothing is created, and each error names the resource, the
field and the fix. The plan refuses:

* an extractor that no longer exists or is turned off on this deployment
* a retriever stage that is not registered, such as a retired stage type
* a feature URI that no collection in the manifest writes, such as one naming a renamed model
* two collections in one namespace that write the same vector at different sizes, such as one
  `multimodal_extractor` v2 collection with `output_dimensionality: 1536` and another at the
  default 3072. A namespace holds one index per vector, so give both the same size:

  ```yaml theme={null}
  feature_extractor:
    name: multimodal_extractor
    version: v2
    parameters:
      output_dimensionality: 1536   # every collection writing gemini-embedding-2 in this namespace
  ```
* an alert application that does not map an input its alert's retriever requires
* a resource its create request refuses: the namespace, bucket, collection, retriever, cluster, taxonomy, trigger, webhook, alert and storage connection are each built into the same request a real create sends, so a cluster with a non-boolean `hierarchical_grouping` or a taxonomy with an unknown `taxonomy_type` is refused here and reads `cluster:<name>: <field>: <reason>`
* a bucket sync whose `sync_mode` is not `initial_only` or `continuous`
* a `seed` that names a sample key the Mixpeek sample bucket does not hold (a read-only listing; if the bucket cannot be listed, `validate` adds a warning and refuses nothing)

`POST /v1/manifest/validate` runs every check above, so a manifest it passes is one `apply` accepts. `validate`, `diff` and `lint` each take the `parameters` form field that `apply` takes.

A namespace `budget` above your organization's budget is set to the organization's budget, and
apply reports the change in `warnings`.

`dry_run=true` returns that plan and creates nothing. If a resource fails partway through,
`apply` rolls back the ones it already created, and if the API is stopped in the middle of an
apply, it rolls that apply back when it starts again.

## Core Operations

| Operation | Description |
| - | - |
| **Apply** | Create all resources defined in the manifest. `mode=create_only` (default) fails if anything exists; `mode=create_missing` fills the gaps |
| **Validate** | Check syntax, schema, and references without changes |
| **Export** | Generate a manifest from existing resources |
| **Diff** | Compare manifest against current state |

## Resource Types

Manifests support these resource types, applied in dependency order:

1. **Namespaces** - Isolation boundaries with feature extractors and a `budget`
2. **Extractors** - The extractors collections run, declared under `extractors:`
3. **Storage connections** and **buckets** - Object storage with schemas
4. **Bucket syncs** - Scheduled pulls from a connection
5. **Collections** - Document stores with indexing
6. **Retrievers** - Search pipelines
7. **Alerts** - Retrievers or system conditions that notify you
8. **Clusters** and **taxonomies** - Grouping and classification
9. **Triggers** and **webhooks** - Automation

## Extractors

An `extractors:` section declares the extractors the manifest's collections run. Each entry has a `name`, a `namespace`, a `version` (default `v1`), a `source` and an optional `compute_profile`. Collections reference an entry by `name` in `feature_extractor`, and namespaces in `feature_extractors`. Once a manifest has an `extractors:` section, every extractor a collection or namespace names must be declared in it. Extractors apply before collections, so a collection never names an extractor that does not exist yet.

`source` is one of:

* `marketplace`: a feature extractor name from the registry, for example `text_extractor`. This is the default when `source` is omitted (`{marketplace: <name>}`).
* `model`: a model your organization uploaded through `POST /v1/models/uploads`, as `org:<model_id>`. Apply validates the model if it has not been validated, then serves it as an extractor at zero replicas until the first call.
* `huggingface`: a model repository on the Hugging Face Hub, as `{repo, revision, framework, token_secret}`. Apply imports the snapshot into your organization's model registry through the same checks an upload and confirm run, then serves it exactly like `model`. No manual upload is needed.

```yaml theme={null}
extractors:
  - name: text_extractor            # marketplace (the default when source is omitted)
    namespace: video_search
  - name: brand_classifier          # a model you uploaded
    namespace: video_search
    source:
      model: org:mdl_7f3a9c
  - name: minilm                    # a model from the Hugging Face Hub
    namespace: video_search
    source:
      huggingface:
        repo: sentence-transformers/all-MiniLM-L6-v2
        revision: "<40-character commit sha, or a tag>"
        framework: sentence-transformers   # the default; transformers is also accepted
        token_secret: HF_TOKEN             # only for a gated or private repo
```

A `huggingface` source is checked against the Hub at validate (read-only: repo info, refs and the file list), and apply refuses the same things:

* `revision` is required and must be a full 40-character commit sha or a tag. A branch such as `main` is refused because it moves, and the same manifest has to import the same weights on every apply.
* Weights must be safetensors. The import takes `.safetensors`, `.json`, `.txt` and `.model` files and skips pickle weights (`pytorch_model.bin`, `.pt`, `.ckpt`), because loading a pickle file can run code. A repo with only pickle weights is refused.
* The files the framework needs must be at the repo root: `config.json` and `modules.json` for `sentence-transformers`, `config.json` for `transformers`. An extractor can serve `sentence-transformers` models today, so `framework: transformers` is refused on an extractor.
* The snapshot must fit the model upload limit (5 GB). The limit is checked from the file list and again while downloading.
* A gated or private repo needs `token_secret`, the name of an organization secret holding a Hugging Face access token (`HF_TOKEN` or `${{ secrets.HF_TOKEN }}`). A public repo needs none.

The import records the repo, revision, resolved commit and the model card's license on the registry entry (`GET /v1/models/{model_id}` shows `license`). Applying the same repo and revision again in the same organization reuses that entry with no Hub call and no download, and a tag that points at an already imported commit reuses it too. While the import runs, the apply's task shows the extractor as `importing_model`; after that it moves through `validating_model`, `deploying_model` and `warming` or `ready`, the same states as a `model` source.

`compute_profile` carries the fields of a compute profile (`resource_type`, `batch_size`, `max_concurrency`, memory). Apply stores them as the namespace's batch override for that extractor, merged over the extractor's built-in profile when a batch is built. Two nested blocks are checked rather than stored:

* `batch.enabled`: only `true` is accepted. A collection that names the extractor runs it in batch.
* `serve`: the real-time serving block (`enabled`, `min_replicas`, `max_replicas`, `min_reason`, `idle_timeout_s`). Validate and apply check it; realtime still runs under the extractor's built-in serve settings, and `warnings` says so. A `min_replicas` above 0 is refused with the monthly cost of the warm replicas named, because a template adds no always-on floor. Leave it at 0 and the first call after idle waits for a replica to start.

```yaml theme={null}
extractors:
  - name: text_extractor
    namespace: video_search
    version: v1
    compute_profile:
      resource_type: cpu
      batch_size: 32
  - name: brand_classifier
    namespace: video_search
    source:
      model: org:mdl_7f3a9c

collections:
  - name: transcripts
    namespace: video_search
    source: {type: bucket, bucket: raw_videos}
    feature_extractor:
      feature_extractor_name: text_extractor
      version: v1
```

A collection naming an extractor the section does not declare is refused at plan with the name. Applying the same manifest again is a no-op for the extractors it declares.

## Parameters

A `parameters:` section declares the values a manifest needs from whoever applies it, so a template ships without a hand edit or a secret inside it. Reference a parameter anywhere in the document as `${{ params.<name> }}`. A reference that is the whole value takes the parameter's type; one inside a longer string is formatted into it as text. An `array` parameter is a list and can only be a whole value.

| Field | Type | Default | What it does |
| - | - | - | - |
| `name` | string | required | The name you reference as `${{ params.<name> }}` |
| `type` | `string`, `integer`, `number`, `boolean`, `array` | `string` | The type a supplied value must match |
| `items` | `string`, `integer`, `number`, `boolean` | `string` | The type of each item when `type` is `array` |
| `enum` | list of values | none | The allowed values. For an `array`, every item must be one of them |
| `required` | boolean | `true` | Whether a value must be supplied. Ignored when `default` is set |
| `default` | matches `type` | none | Used when no value is supplied. A list for an `array` |
| `description` | string | none | Shown next to the input in Studio |

```yaml theme={null}
parameters:
  - name: camera_url
    type: string
    description: Your camera's rtsp:// url
  - name: polling_seconds
    type: integer
    default: 300

buckets:
  - name: camera_feed
    namespace: site_cameras
    sync:
      source_url: ${{ params.camera_url }}
      interval_seconds: ${{ params.polling_seconds }}
```

Pass the values on validate and apply as the `parameters` form field, a JSON object of name to value:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@camera.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'
```

An array parameter with allowed values:

```yaml theme={null}
parameters:
  - name: search_collections
    type: array
    items: string
    enum: [products, reviews, faqs]
    default: [products, reviews]
  - name: tier
    enum: [free, pro]
    default: free

retrievers:
  - name: site_search
    namespace: shop
    description: "${{ params.tier }} plan search"   # a scalar can sit inside a string
    collections: ${{ params.search_collections }}   # an array must be the whole value
    stages: [...]
```

Supply an array as JSON: `-F 'parameters={"search_collections": ["faqs"], "tier": "pro"}'`. A value outside `enum` is refused with the allowed values listed, and an array written inside a longer string (`"types: ${{ params.search_collections }}"`) is refused.

A missing required parameter, a value of the wrong type, a reference to an undeclared parameter, or a name declared twice is refused before any resource is created, naming `parameters.<name>`.

## Budget

A namespace entry takes a `budget` with `daily_usd` and `monthly_usd`, the same fields the namespaces API takes as `spend_budget`. Omit it to use your organization's namespace default. A value above your organization's budget is set to the organization's budget, and apply reports the clamp in `warnings`.

```yaml theme={null}
namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1
    budget:
      daily_usd: 5
      monthly_usd: 100
```

See [Budgets](/docs/platform/budgets) for what happens when work reaches the limit.

## Alerts

An `alerts:` section creates alerts beside the retrievers they watch. An alert mirrors the create request of the alerts API, plus `namespace` and `retriever` by name.

| Field | Type | Default | What it does |
| - | - | - | - |
| `name` | string | required | Unique alert name |
| `namespace` | string | required | Namespace by name |
| `source` | `retriever` or `system` | `retriever` | Run a retriever, or watch a built-in condition |
| `retriever` | string | none | A retriever declared in this manifest, by name (`source: retriever`) |
| `retriever_id` | string | none | An existing retriever's id instead of `retriever`; set one of the two |
| `trigger_on` | `results` or `no_results` | `results` | When a retriever alert fires |
| `system_condition` | object | none | The built-in condition to watch (`source: system`) |
| `notification_config` | object | required | How and where to notify |
| `enabled` | boolean | `true` | Whether the alert runs |

```yaml theme={null}
alerts:
  - name: no_results_for_brand_queries
    namespace: video_search
    retriever: video_search_tool
    trigger_on: no_results
    notification_config:
      channels:
        - channel_type: email
          config:
            to_addresses: [ops@example.com]
```

Plan refuses an alert that does not map an input its retriever requires. The alert fields are described on [Alerts](/docs/enrichment/alerts).

## Track an apply

`apply` returns a `task_id` (absent on a dry run). `GET /v1/tasks/{task_id}` returns every resource of the plan with its states in order (`planned`, `creating`, `created`, `exists`, `updated`, or `failed` with the finding; a collection also passes `warming` and `ready`), each with the time it was reached. The same events arrive on the [change feed](/docs/operations/change-feed) and on a [webhook](/docs/operations/webhooks).

```bash theme={null}
curl https://api.mixpeek.com/v1/tasks/$TASK_ID \
  -H "Authorization: Bearer $API_KEY"
```

The apply response also carries `readiness`: for each collection whose extractor needs a real-time serving app, whether that app is running. An app scaled to zero is running; its first call answers 503 with `Retry-After` while it warms.

## Secret References

Use `${{ secrets.NAME }}` to reference organization secrets:

```yaml theme={null}
buckets:
  - name: external_data
    namespace: my_namespace
    sync:
      connection_id: ${{ secrets.S3_CONNECTION_ID }}
```

Secrets must exist before applying. Use the validate endpoint to check for missing secrets.

## Seed Data

A `seed:` section gives a namespace sample objects to answer its first query. After the resources are created, apply ingests them into the named bucket with one batch, bounded by `max_objects` (default 50, at most 500). Applying the manifest again finds that batch and ingests nothing more.

Each object names a source for each blob property under `blobs`. The blob type comes from the bucket schema property. A source takes one of three forms.

A Mixpeek sample key, from the sample set `samples/starter/<file>`:

```yaml theme={null}
seed:
  - bucket: ad-creatives
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}
```

A public `https://` URL (or `http://`, which object create also accepts). Mixpeek fetches it the way `POST /v1/buckets/{id}/objects` fetches a URL blob, with the same checks: the host must resolve to a public address, redirects are re-checked, and the response must be reachable and match the property's type.

```yaml theme={null}
seed:
  - bucket: ad-creatives
    objects:
      - blobs: {creative_asset_url: "https://cdn.example.com/ads/launch.mp4"}
```

One object in your own storage, given as `{connection, path}`. `connection` names a storage connection declared under `storage_connections` in the same manifest, or one that already exists in the organization. `path` is the object's key, written the way a bucket sync's `source_path` is. Apply reads the object through the connection with the same provider client a bucket sync uses and copies it into the bucket's storage. The connection's provider must be `s3`, `tigris`, `gcs`, `azure_blob`, `backblaze` or `supabase`; Google Drive and SharePoint name files by id and Snowflake yields rows, so those connections are refused here and belong in `bucket_syncs`.

```yaml theme={null}
storage_connections:
  - name: brand_s3
    provider: s3
    config:
      bucket_name: brand-assets
      region: us-east-1
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

seed:
  - bucket: ad-creatives
    objects:
      - blobs:
          creative_asset_url: {connection: brand_s3, path: ads/launch.mp4}
```

Validate and apply refuse the same seeds before anything is created: a connection that is not declared and does not exist, a connection whose provider cannot read one object by path (the error names the provider), a blob property the bucket schema does not declare, a URL that object create would refuse (no host, a private or internal address, unreachable, or the wrong content type), and any other `s3://`, `gs://` or `file://` string, since object storage is read only through a connection.

A seed's batch is billed to your organization when it is submitted, like any batch you create, whichever form its blobs take.

When the bucket declares other properties, set them on the object beside `blobs`, the way the object create API takes them:

```yaml theme={null}
buckets:
  - name: ad-creatives
    namespace: my_namespace
    schema:
      properties:
        creative_asset_url: {type: video}
        ad_id: {type: string, required: true}
        brand_slug: {type: string, required: true}

seed:
  - bucket: ad-creatives
    max_objects: 2
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}
        ad_id: sample-norelco
        brand_slug: norelco
```

Validate and apply check every object against the bucket's schema before anything is created. A missing required property, a property the bucket does not declare, a value of the wrong type, or a blob property given outside `blobs` is refused, and the error names the bucket, the object and the property.

### Seed or bucket sync

Use `seed` for a curated handful of objects, up to 500, that the first query should find. Use `bucket_syncs` with `sync_mode: initial_only` to import everything under a prefix of a storage connection once. The sync lists the prefix, applies `file_filters` and `schema_mapping`, and ingests the files in pages, with retries and a dead-letter queue for files that fail. `sync_mode: continuous` keeps polling for new files.

```yaml theme={null}
bucket_syncs:
  - name: brand-archive
    bucket: ad-creatives
    connection: brand_s3
    source_path: ads/2025/
    sync_mode: initial_only
    file_filters:
      include_patterns: ["*.mp4"]
```

`sync_mode` takes exactly `initial_only` or `continuous`; any other value is refused at validate.

## Taxonomy Labels

A flat taxonomy tags each document with its closest label. Write the labels inline on the taxonomy and name the feature to match on:

```yaml theme={null}
taxonomies:
  - name: creative-format
    namespace: ads
    type: flat
    match_on: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
    target_field: creative_format
    labels:
      - {label: testimonial, description: "A customer speaks to camera about results they saw."}
      - {label: tutorial, description: "A step by step routine that teaches while it sells."}

collections:
  - name: ad-scenes
    namespace: ads
    source: {type: bucket, bucket: ads}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {video: video}
      parameters: {run_multimodal_embedding: true, output_dimensionality: 1536}
    taxonomy_applications:
      - {taxonomy: creative-format, execution_mode: materialize}
```

| Field | Type | Default | What it does |
| - | - | - | - |
| `labels` | list | none | Each entry has `label` and an optional `description`. Any other key (a string, number or boolean) is stored on the label and written to the document as `<target_field>_<key>` |
| `match_on` | feature URI | inferred | The feature the classified collections write. The labels are embedded into the same feature. Optional when every collection applying the taxonomy writes exactly one feature that can embed text |
| `target_field` | string | taxonomy name, `-` as `_` | The document field the matched label is written to |

The manifest generates the resources the taxonomy runs on, and validate, the dry run and apply treat them like declared ones:

* a bucket `<taxonomy>-labels` with `content` (text) and `label` (string) properties
* a collection `<taxonomy>-labels` that embeds each label's text, `<label>: <description>`, with the extractor and feature `match_on` names. It copies `output_dimensionality` from the collections applying the taxonomy, so labels and documents share one vector space
* a retriever `<taxonomy>-lookup` that returns the single closest label for a document's vector
* a `seed:` entry for the label bucket, one object per label
* the taxonomy's `config`, wiring the retriever and the label collection, with `label` written to `target_field`

Each generated resource's description starts with `Generated for taxonomy '<name>'`, and each row of the dry run and apply response for one carries `generated_for: <taxonomy>`. A resource you declare under a generated name is refused; rename yours.

Validate refuses:

* labels on a taxonomy that is not flat
* a label given twice, or an empty label list
* labels together with `config.source_collection` or `config.retriever`
* a `match_on` whose extractor cannot compute that feature from text, such as a transcription embedding, which is computed from audio
* a collection applying the taxonomy that does not write the `match_on` feature

The older `type: flat` with `nodes` reads the same way: each node's `name` becomes a label and its `description` the label's description. It needs `match_on` unless it can be inferred as above, and is refused until it has one.

The seed runs once per label bucket, like any seed. To change the labels of a namespace that is already applied, add or edit objects in the `<taxonomy>-labels` bucket directly. An export writes the generated resources and the taxonomy config out in full, since the label text lives in the bucket's objects; the exported manifest applies as it is.

## Acceptance

A manifest can carry an `acceptance:` block: the steps that show its use case works and what each must return. Apply creates nothing from it. Apply runs the steps when you ask with `run_acceptance=true`, and Mixpeek's template acceptance harness runs them after every template apply.

| Field | Type | What it does |
| - | - | - |
| `steps` | list, at least one | The steps, run in order. Step names must be unique |
| `steps[].name` | string | Shown when the step fails |
| `steps[].kind` | `query`, `documents`, `alert` or `realtime` | What the step does |
| `steps[].retriever` | string | `query`: a retriever declared in this manifest |
| `steps[].collection` | string | `documents`: a collection declared in this manifest |
| `steps[].alert` | string | `alert`: an alert declared in this manifest |
| `steps[].feature_uri` | string | `realtime`: the `mixpeek://` feature URI to call |
| `steps[].inputs` | object | `query` and `realtime`: the inputs sent |
| `steps[].expect` | object | The outcome the step must produce, below |
| `max_cost_usd` | number | The most one acceptance run may cost, read from the meter |

Each kind reads its own `expect` fields:

| Kind | What runs | `expect` fields |
| - | - | - |
| `query` | the retriever, executed with `inputs` | `min_results`; `top_field` (a dotted path into the top result, such as `metadata.brand_slug`) with `top_equals` or `top_contains` |
| `documents` | the collection's document count | `min_documents` |
| `alert` | the alert's executions that triggered | `min_fires` |
| `realtime` | `POST /v1/inference` with `feature_uri` and `inputs` | `dims`, the length of the returned vector |

One step of each kind:

```yaml theme={null}
acceptance:
  max_cost_usd: 0.5
  steps:
    - name: seeded documents written
      kind: documents
      collection: transcripts
      expect: {min_documents: 2}
    - name: golden query
      kind: query
      retriever: video_search_tool
      inputs: {query: "electric shaver"}
      expect: {min_results: 1, top_field: metadata.brand_slug, top_equals: norelco}
    - name: shaver alert fires
      kind: alert
      alert: shaver_spotted
      expect: {min_fires: 1}
    - name: text embedding answers
      kind: realtime
      feature_uri: mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1
      inputs: {text: "electric shaver"}
      expect: {dims: 1024}
```

Validate refuses a step whose retriever, collection or alert this manifest does not declare, an `expect` field the step's kind does not read, and a step with no outcome, each naming `acceptance.steps.<name>`.

### Wait for the seed and run the steps on apply

Two query flags on `apply` make it wait until the namespace is usable and then prove it:

| Flag | What it does |
| - | - |
| `wait_for_seed=true` | After the resources exist, waits until every `seed:` batch has finished. The response lists each in `seed_batches` with its `status` and `documents_written`. A batch that failed, or did not finish in time, is also named in `errors` |
| `run_acceptance=true` | Implies `wait_for_seed`, and also waits for the serving apps listed in `readiness`. Then runs each step in order and returns the outcome in `acceptance`. A manifest with no `acceptance:` block is refused with 400 |

Both wait within `deadline_seconds`, or 900 seconds when you omit it. One clock covers the serving apps, the seed and the polling of `documents` and `alert` steps, which re-read until their count is reached or the time runs out.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/manifest/apply?run_acceptance=true&deadline_seconds=900" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

```json theme={null}
{
  "success": true,
  "seed_batches": [
    {"bucket": "creative_assets", "batch_id": "btch_123", "status": "COMPLETED",
     "terminal": true, "usable": true, "documents_written": 4}
  ],
  "acceptance": {
    "passed": false,
    "steps": [
      {"name": "seeded documents written", "kind": "documents", "passed": true,
       "expected": {"min_documents": 2}, "observed": {"documents": 4}, "error": null},
      {"name": "golden query", "kind": "query", "passed": false,
       "expected": {"min_results": 1, "top_field": "metadata.brand_slug", "top_equals": "norelco"},
       "observed": {"results": 3, "top_value": "philips"},
       "error": "top metadata.brand_slug is 'philips', expected 'norelco'"},
      {"name": "max_cost_usd", "kind": "cost", "passed": true,
       "expected": {"max_cost_usd": 0.5}, "observed": {"spent_usd": 0.012, "metered_from": "namespace creation"}, "error": null}
    ]
  },
  "errors": ["acceptance failed: golden query (top metadata.brand_slug is 'philips', expected 'norelco')"]
}
```

How to read the result:

* `success` reports resource creation only, exactly as without the flags. Read `acceptance.passed` for the use case.
* A step passes only when it ran and every expected outcome held. A step that could not run is `passed: false` and its `error` starts with `not run:` and says why: a seed batch failed or was still processing at the deadline, a serving app was not running, the apply rolled back, or the cost cap was reached.
* A failed run is also named in `errors`, so a caller that only reads `errors` sees it.
* On a dry run both flags do nothing, and `warnings` says so.

**Billing.** Acceptance steps on `apply` are normal executions in your organization: each query and realtime call is billed like the same call made directly. `max_cost_usd` is read from your usage meter (the per-namespace credits in `GET /v1/organizations/billing/usage/breakdown`). Before each step after the first, a run that has reached the cap stops, and the remaining steps report `not run`. A closing `max_cost_usd` row compares the spend with the cap. When this apply created the namespace, the spend counts from its creation, seed ingest included. On a namespace that already existed it counts from the first step. Metering of very small usage can lag a few minutes, so the reading is a lower bound right after the run.

**Re-running the steps.** Re-apply the same manifest with `mode=create_missing&run_acceptance=true`. Every resource reads `exists`, nothing is created, the seed is not ingested again (its earlier batch is reused), and the steps run against the existing resources. Under the default `create_only` a re-apply is refused because the namespace exists, and the steps report `not run`.

## Dependency Resolution

Resources are created in topological order. The manifest engine:

* Resolves cross-resource references automatically
* Detects circular dependencies
* Rolls back all changes if any creation fails

## Authoring manifests with an LLM

An LLM can write a complete manifest on the first or second try if it reads what the API publishes and lets `validate` check the result. The loop has six steps.

1. **Read the schema.** `GET /v1/discovery/schema` returns:

   * `schemas`: a JSON schema for every section (`parameters`, `namespace`, `extractor`, `storage_connection`, `bucket`, `bucket_sync`, `collection`, `retriever`, `alert`, `cluster`, `taxonomy`, `trigger`, `webhook`, `seed`, `acceptance`) and for the whole `manifest`.
   * `dependency_graph`: for each type, the types whose names it references. A bucket sync needs a bucket and a storage connection, a seed needs a bucket, an acceptance step needs a retriever, collection or alert.
   * `examples`: a short YAML block for each type.
   * `example_manifest`: the complete manifest shown below.
   * `extractor_catalog`: every extractor and version this plane serves, with its input types, the feature URIs it writes and its parameters with defaults.
   * `stage_catalog`: every retriever `stage_id` with its category and parameter names.
   * `authoring_loop`: these steps in short form.

   Use `GET /v1/discovery/extractors` and `GET /v1/discovery/stages` for the full parameter schemas and stage examples.

2. **Write the manifest.** Reference resources by name. Copy feature URIs from `extractor_catalog`. A collection writes a feature URI only when the extractor parameters that produce it are on, for example `run_transcription_embedding: true` for the transcript embedding.

3. **Validate.** `POST /v1/manifest/validate` returns `valid`, `errors`, `warnings`, `missing_secrets`, `resources` (a count per type) and the declared `parameters`. Validate refuses what apply would refuse, so fix every entry in `errors` and rerun. Send parameter values in the `parameters` form field as a JSON object.

4. **Lint.** `POST /v1/manifest/lint` returns suggestions that each carry a code, a location, a suggestion and a `fix_example`. Three rules catch wiring that validates but cannot work: `UNUSED_BUCKET` (no collection reads the bucket), `TAXONOMY_INPUT_NOT_WRITTEN` (the taxonomy reads a feature that the collection it is applied to does not write) and `RETRIEVER_FEATURE_NOT_IN_COLLECTIONS` (the retriever searches a feature that none of its own collections write).

5. **Dry run.** `POST /v1/manifest/apply` with `dry_run=true` returns the plan: what would be created, in dependency order, and any refusal. Nothing is created.

6. **Apply.** `POST /v1/manifest/apply` creates the resources and returns the result, or the task to poll for progress.

To have the API write the first draft, call `POST /v1/manifest/generate` with a `description`. The model is given the live extractor and stage catalogs and the example below. The answer goes through the same checks as `validate`. If they find errors, the model is shown them and asked to fix its answer, up to two times. The response keeps `manifest`, `format`, `manifest_name` and `description`, and adds `validation` (`valid`, `errors`, `warnings`) and `repair_rounds`. When `validation.valid` is false the errors are still listed, so you can fix them yourself and continue from step 3.

### Complete example

This manifest syncs ad videos from S3 once, seeds a few sample objects, indexes the ads in two collections, searches them with a hybrid retriever, tags each scene with a creative format, groups scenes by look and states what a working setup returns. It validates once `ads_s3_bucket` is supplied and the secrets `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` exist.

```yaml theme={null}
version: "1.0"
metadata:
  name: ad-creative-search
  description: Search video ads by moment, tag each scene with a creative format, group scenes by look

parameters:
  - name: ads_s3_bucket
    type: string
    required: true
    description: Name of the S3 bucket that holds your ad videos
  - name: ads_prefix
    type: string
    default: ads/
    description: Folder inside that bucket to sync

namespaces:
  - name: ad-creative
    description: Ad creative search
    feature_extractors:
      - name: multimodal_extractor
        version: v2
    payload_indexes:
      - {field_name: metadata.brand, type: keyword}

storage_connections:
  - name: ads-s3
    provider: s3
    description: Read access to the bucket that holds the ad videos
    config:
      bucket: ${{ params.ads_s3_bucket }}
      region: us-east-1
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

buckets:
  - name: ads
    namespace: ad-creative
    description: Video ads, plus the creative format labels (kind=label)
    schema:
      properties:
        video: {type: video}
        metadata: {type: object}
        content: {type: text}
        label: {type: string}
        kind: {type: string}

bucket_syncs:
  - name: ads-from-s3
    bucket: ads
    connection: ads-s3
    source_path: ${{ params.ads_prefix }}
    sync_mode: initial_only
    file_filters:
      include_patterns: ["*.mp4", "*.mov"]

seed:
  - bucket: ads
    max_objects: 5
    objects:
      - blobs: {video: samples/starter/videos/norelco-speed.mp4}
        kind: ad
        metadata: {brand: norelco}
      - texts: {content: "A real customer speaks to camera about the results she saw."}
        kind: label
        label: testimonial
      - texts: {content: "Skin shown before use and after use, side by side."}
        kind: label
        label: before-and-after

collections:
  - name: creative-formats
    namespace: ad-creative
    description: Creative format labels, embedded in the same space as the ad scenes
    source:
      type: bucket
      bucket: ads
      filters: {field: kind, operator: eq, value: label}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {text: content}
      field_passthrough:
        - {source_path: label}
      parameters:
        run_multimodal_embedding: true
  - name: ad-scenes
    namespace: ad-creative
    description: Every ad split into scenes, with a visual embedding and a transcript embedding
    source:
      type: bucket
      bucket: ads
      filters: {field: kind, operator: ne, value: label}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {video: video}
      field_passthrough:
        - {source_path: metadata.brand}
      parameters:
        split_method: scene
        run_multimodal_embedding: true
        run_transcription: true
        run_transcription_embedding: true
    taxonomy_applications:
      - {taxonomy: creative-format, execution_mode: materialize}

retrievers:
  - name: format-lookup
    namespace: ad-creative
    description: Finds the closest creative format label for a scene. Used by the creative-format taxonomy.
    collections: [creative-formats]
    input_schema:
      query_embedding: {type: text, required: true}
    stages:
      - stage_name: match
        stage_type: filter
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
              query: {input_mode: vector, vector: "{{INPUT.query_embedding}}"}
              top_k: 1
          final_top_k: 1
  - name: ad-moment-search
    namespace: ad-creative
    description: Describe a moment and get the matching ad scenes
    collections: [ad-scenes]
    input_schema:
      query:
        type: text
        required: true
        description: Describe the moment, the claim or the format
    stages:
      - stage_name: recall
        stage_type: filter
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
              query: {input_mode: text, value: "{{INPUT.query}}"}
              top_k: 50
              weight: 1.0
            - feature_uri: "mixpeek://multimodal_extractor@v2/multilingual_e5_large_instruct_v1"
              query: {input_mode: text, value: "{{INPUT.query}}"}
              top_k: 50
              weight: 0.6
          final_top_k: 20

taxonomies:
  - name: creative-format
    namespace: ad-creative
    description: Tags every scene with its creative format
    type: flat
    config:
      taxonomy_type: flat
      retriever: format-lookup
      input_mappings:
        - {input_key: query_embedding, source_type: payload, path: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"}
      source_collection:
        collection: creative-formats
        enrichment_fields:
          - {field_path: label, target_field: creative_format, merge_mode: replace}

clusters:
  - name: visual-styles
    namespace: ad-creative
    description: Groups scenes by how they look
    source_collections: [ad-scenes]
    cluster_type: vector
    vector_config:
      feature_uris: ["mixpeek://multimodal_extractor@v2/gemini-embedding-2"]
      clustering_method: kmeans
      algorithm_params: {n_clusters: 6}
    llm_labeling:
      provider: google
      model_name: gemini-2.5-flash
      labeling_inputs:
        input_mappings:
          - {input_key: text, source_type: payload, path: transcription}
    enrich_source_collection: true

acceptance:
  max_cost_usd: 5
  steps:
    - name: scenes are indexed
      kind: documents
      collection: ad-scenes
      expect: {min_documents: 1}
    - name: texture query finds a scene
      kind: query
      retriever: ad-moment-search
      inputs: {query: close-up of foundation being blended on skin}
      expect: {min_results: 1}
```

## Workflows

### Environment Replication

```bash theme={null}
# Export from production
curl https://api.mixpeek.com/v1/manifest/export \
  -H "Authorization: Bearer $PROD_KEY" \
  -o prod-manifest.yaml

# Apply to staging
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $STAGING_KEY" \
  -F "manifest_file=@prod-manifest.yaml"
```

### Describe what is in a namespace

Add `detail=stats` to an export to see the data as well as the configuration. The YAML gets a read-only `status` section with each collection's document count, field types and most common field values, and each cluster's groups with their labels and sizes. Apply ignores `status`, so the file still applies as-is. `detail=stats` covers one namespace, so pass `namespace_id`.

```bash theme={null}
curl "https://api.mixpeek.com/v1/manifest/export?namespace_id=ns_abc123&detail=stats" \
  -H "Authorization: Bearer $API_KEY" \
  -o namespace-with-stats.yaml
```

Each stats read has a 2 second limit. A field or cluster whose read fails or times out shows `unavailable` with the reason, never an empty list.

### Pre-deployment Validation

```bash theme={null}
# Validate without applying
curl -X POST https://api.mixpeek.com/v1/manifest/validate \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'   # only if the manifest declares parameters

# Check what would change
curl -X POST https://api.mixpeek.com/v1/manifest/diff \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

### CI/CD Integration

```yaml theme={null}
# GitHub Actions example
- name: Deploy Mixpeek Resources
  run: |
    curl -X POST https://api.mixpeek.com/v1/manifest/apply \
      -H "Authorization: Bearer ${{ secrets.MIXPEEK_API_KEY }}" \
      -F "manifest_file=@mixpeek.yaml"
```

## References

* [Apply Manifest](/docs/api-reference/manifest/apply-manifest)
* [Validate Manifest](/docs/api-reference/manifest/validate-manifest)
* [Export Manifest](/docs/api-reference/manifest/export-manifest)
* [Diff Manifest](/docs/api-reference/manifest/diff-manifest)


# Observability
Source: https://docs.mixpeek.com/docs/operations/observability

Monitor API, Engine, storage, and asynchronous jobs

Mixpeek provides multiple observability surfaces: health endpoints, task metadata, Ray dashboards, analytics APIs, and webhook histories. Combine them to detect regressions early and debug production issues quickly.

## Health & Status

Four health routes, and only one of them looks at your services.

| Route | Auth | What it answers |
| - | - | - |
| `GET /v1/health` | none | The process is alive and the event loop responds. Under 10ms, no external calls. |
| `GET /v1/health/liveness` | none | Same idea, shaped for container orchestration probes. |
| `GET /v1/health/beat` | none | The work scheduler fired within the last 5 minutes. |
| `GET /v1/health/deep` | **org API key** | Pings every dependent service. Cache, metadata, vector store, object storage, task queue and inference, with per-component errors. |

> [!WARNING]
>
> **`GET /v1/health` is a liveness constant. It does not check any service.** It
> returns the same `ok` whether or not MongoDB, [MVS](https://mixpeek.com/mvs),
> Redis, the work scheduler or the engine are reachable, by design and in under 10ms.
>
> **`GET /v1/health?deep=true` does nothing.** That route declares no query
> parameters, so the flag is dropped and you get the same constant back. A
> monitor built on it reports green through a total outage. The deep check is a
> different path: `GET /v1/health/deep`.
>
> If you built a dashboard on `?deep=true`, it is measuring the event loop rather
> than the services behind it. Point it at `GET /v1/health/deep`.

> [!NOTE]
>
> **`/v1/health/deep` needs an organization API key.** It is the only health
> route with auth. An unauthenticated call gets 403, and the platform private
> token gets 401 with "API key not found", so neither an anonymous probe nor the
> private token can read component health. A 401 or 403 here is the auth
> requirement, not a broken route.
>
> On that path, `?deep=true` is a real parameter: it adds data-plane
> verification on top of the service pings, reading from collections, probing
> MVS namespaces and inspecting the task queue.

* **Tasks API** – `/v1/tasks/{task_id}` and `/v1/tasks/list` expose status for batches, clustering jobs, taxonomy materialization, and migrations. All tasks use `TaskStatusEnum`.

## Who owns this alert

`GET /v1/ops/ownership` maps every alert class to the team that owns it and the
procedure for fixing it. Org-scoped, needs a bearer token.

```bash theme={null}
# the whole registry
curl -sS "$MP_API_URL/v1/ops/ownership" -H "Authorization: Bearer $MP_API_KEY"

# just the subsystem that owns one alert class
curl -sS "$MP_API_URL/v1/ops/ownership?alert_class=MvsShardDown" \
  -H "Authorization: Bearer $MP_API_KEY"
```

Filter by `alert_class` to resolve one alert, or by `subsystem` to pull one
entry. The response is `{subsystems: [...], total}`.

| Field | Meaning |
| - | - |
| `id` | Stable subsystem slug. |
| `owner` | The team that owns this subsystem. |
| `escalation` | Who to contact when the owner does not respond. |
| `runbook` | URL or path of the procedure doc. |
| `runbook_text` | Inline minimal procedure, when no doc exists yet. |
| `alert_classes` | The alert names this subsystem covers. |
| `alert_escalations` | Per-alert overrides of `escalation`, keyed by alert class. |
| `owner_confirmed` | Present when the owner agreed to this entry, with `by` and `at`, plus optional `ref` and `note`. |

Resolving the responder for one alert: use `alert_escalations[alert_class]` when
that key is present, otherwise `escalation`. Reach for an override when a single
alert inside a subsystem pulls in someone other than the subsystem default. A key
that names an alert the subsystem does not own is rejected at load time, because
an override on an unowned alert never fires and reads like routing that works.

`owner_confirmed` records that the owner agreed to the entry, and where they said
so. Absent means nobody has confirmed it. An unconfirmed entry can still be
correct, so do not read the absence as a dispute.

> [!NOTE]
>
> **Every entry names an owner and a procedure.** The loader rejects an entry
> carrying one without the other, so `runbook` or `runbook_text` is always
> populated even when the written doc does not exist yet. Read both: either can
> be the one that is filled.
>
> CI pins that every declared alert class is covered by exactly one subsystem,
> so an alert cannot be owned twice or by nobody.

> [!WARNING]
>
> **An unknown `alert_class` returns 404, and that is the useful answer.** It
> means the alert has no owner in the registry — a gap to fill, not an empty
> result to shrug at.
>
> Class names are the CamelCase names from the alerting rules, so
> `MvsShardDown`, not `mvs_shard_down`. A snake\_case name 404s the same way, and
> the error names the registry file so you can go and look.

* **Webhooks** – webhook events recorded in MongoDB provide a durable log of ingestion and enrichment milestones (`collection.documents.written`, etc.).

## Engine Monitoring

* **Ray Dashboard (port 8265)** – view worker health, task timelines, Serve deployments, resource utilization, and logs.
* **Ray logs** – pod logs (Kubernetes) or Ray CLI provide detailed extractor and clustering output (`ray logs <job_id>`).
* **Serve metrics** – per-model latency and request counts; scrape via Prometheus or Ray metrics endpoint.

## Analytics APIs

Enable analytics (`ENABLE_ANALYTICS=true`) to capture the events these endpoints read. They are kept in MongoDB time-series collections:

| Endpoint | Insight |
| - | - |
| `/v1/analytics/retrievers/{id}/performance` | Query volume, latency percentiles |
| `/v1/analytics/retrievers/{id}/stages` | Stage-level timing and candidate counts |
| `/v1/analytics/retrievers/{id}/signals` | Cache hits, rerank scores, filter reductions |
| `/v1/analytics/retrievers/{id}/cache-performance` | Hit/miss rates and latency delta |
| `/v1/analytics/retrievers/{id}/slow-queries` | Top slow queries with execution context |
| `/v1/analytics/usage/summary` | Usage and spend (billing support) |

Use these APIs to populate dashboards or feed alerting systems.

## Logging & Tracing

* **API layer** – structured JSON logs include request IDs, namespace, HTTP status, error codes, and downstream latency.
* **Work runtime** – logs task execution, retries, and webhook dispatch results.
* **Ray workers** – include extractor metrics, batch IDs, and queue stats; aggregate logs centrally for long-term retention.
* **Correlation** – propagate `x-request-id` from API to Engine jobs via `additional_data.request_id` to stitch traces together.

## Metrics to Track

| Component | Key Metrics |
| - | - |
| API | Request rate, p95 latency, error rate, rate-limit hits |
| Work runtime | Queue depth, task execution time, retry count |
| Ray | Worker utilization (CPU/GPU), job duration, Serve requests in flight |
| MongoDB | Operation latency, primary health, replication lag |
| MVS | Storage usage, search latency, vector count per namespace |
| Redis | Connection count, command latency, cache hit ratio |

Integrate with Prometheus, Datadog, or your preferred metrics stack via existing exporters or custom scrapers.

## Alerting Playbook

1. **Latency spike** → check retriever analytics, stage statistics, and Ray Serve load.
2. **Task backlog** → inspect the task queue depth, Redis health, and Ray worker availability.
3. **Failed enrichment** → query `/v1/tasks/list` for `FAILED`, inspect `error_message`, review webhook events.
4. **Storage saturation** → monitor MVS storage usage and MongoDB disk consumption; scale storage or shard by namespace.
5. **Cache regression** → view cache hit-rate endpoint; adjust TTLs or stage cache configuration.

## Dashboards to Build

* **API dashboard** – health endpoint status, request latency, error breakdown, rate-limit counters.
* **Engine dashboard** – Ray worker utilization, job runtime percentiles, extractor throughput, Serve queue depth.
* **Retrieval performance** – retriever analytics charts (latency, cache hits, slow queries).
* **Storage dashboard** – MongoDB/Redis/MVS metrics for capacity planning.
* **Task tracker** – open tasks by status, median processing times, failure rates.

## Incident Response Tips

* Keep runbooks for common failures (e.g., extractor timeouts, MVS restarts).
* Use webhook history to confirm whether ingestion completed or stalled.
* Capture Ray job IDs from task metadata to replay logs quickly.
* Snapshot retriever and collection configurations when debugging to ensure you’re reproducing the same pipeline.

With health checks, task metadata, analytics APIs, and Ray observability, you can confidently operate Mixpeek in production and catch issues before users notice.


# Rate Limits & Quotas
Source: https://docs.mixpeek.com/docs/operations/rate-limits-quotas

Understand API rate limits, usage pools, and strategies for scaling under constraints

Mixpeek enforces rate limits and quotas to ensure fair resource allocation and system stability. Limits vary by tier (free, pro, enterprise) and are applied per organization, API key, and endpoint.

## Rate Limiting Model

Mixpeek uses **token bucket** rate limiting with per-minute and per-second windows:

> **Request Rate Limits**
>
> Maximum requests per second (RPS) or per minute (RPM) per API key.

> **Usage & Pools**
>
> Monthly dollar usage pool included with each tier; overage bills at the rate card.

> **Concurrent Requests**
>
> Maximum simultaneous in-flight requests per organization.

> **Resource Quotas**
>
> Limits on collections, documents, feature extractors, and batch sizes.

## Rate Limit Tiers

| Tier | Requests/Min (RPM) | Requests/Sec (RPS) | Concurrent | Burst Allowance |
| - | - | - | - | - |
| **Free** | 60 | 10 | 5 | 20 requests |
| **Pro** | 600 | 100 | 50 | 200 requests |
| **Enterprise** | Custom | Custom | Custom | Custom |

**Burst Allowance:** Short spikes above the sustained rate are permitted using banked tokens (refill at steady rate).

## Rate Limit Headers

Every API response includes rate limit metadata:

```http theme={null}
X-RateLimit-Limit: 600          # Max requests per window
X-RateLimit-Remaining: 542       # Requests left in current window
X-RateLimit-Reset: 1698765432    # Unix timestamp when limit resets
X-RateLimit-Window: 60           # Window duration in seconds
```

**When rate limited:**

```http theme={null}
HTTP/1.1 429 Too Many Requests
Retry-After: 15                  # Seconds until retry is safe
X-RateLimit-Remaining: 0

{
  "success": false,
  "status": 429,
  "error": {
    "message": "Rate limit exceeded",
    "type": "TooManyRequestsError",
    "details": {
      "limit": 600,
      "window": "1m",
      "retry_after": 15
    }
  }
}
```

## Usage Pools & Metering

Usage is metered in **dollars**: each feature you enable is priced per modality unit (images, video minutes, document pages, text tokens, crawled web pages). Every tier includes a monthly dollar usage pool — \$10 on Build, \$100 on Scale — and once the pool is spent, additional usage bills at the rate card. See [Billing](/docs/platform/billing) for the full rate card, fetch it live via `GET /v1/billing/pricing` (no auth required), or quote a workload before running it with `POST /v1/organizations/billing/estimate`.

What's metered:

| What | How it bills |
| - | - |
| **Ingestion** | Per modality unit × features enabled — see the [rate card](/docs/platform/billing) |
| **Storage** | Per GB-month stored, measured daily; your invoice shows the rate |
| **Reads / queries** | Included per tier; \$2 per 1M queries beyond included |

### Monitoring Usage

Check current spend via the Usage API:

```bash theme={null}
GET /v1/organizations/usage
```

The response reports your spend in dollars for the current billing period, broken down by category (ingestion, search, storage), along with your remaining pool balance and the pool reset date. The Studio **Billing** page shows the same numbers with charts and trends.

**Set alerts:**

* 80% of pool spent → warning
* 95% of pool spent → critical
* Pool exhausted → usage continues; overage bills at the [rate card](/docs/platform/billing)

## Resource Quotas

### Per-Organization Limits

| Resource | Free Tier | Pro Tier | Enterprise |
| - | - | - | - |
| **Namespaces** | 1 | 10 | Unlimited |
| **Collections** | 5 | 50 | Unlimited |
| **Buckets** | 5 | 50 | Unlimited |
| **Documents** | 10,000 | 1,000,000 | Unlimited |
| **Retrievers** | 3 | 50 | Unlimited |
| **Taxonomies** | 2 | 20 | Unlimited |
| **Clusters** | 1 | 10 | Unlimited |
| **API Keys** | 2 | 10 | Unlimited |
| **Batch Size** | 100 objects | 10,000 objects | Custom |

### Enforcement

When a quota is exceeded:

```http theme={null}
HTTP/1.1 403 Forbidden

{
  "success": false,
  "status": 403,
  "error": {
    "message": "Collection quota exceeded",
    "type": "QuotaExceededError",
    "details": {
      "resource": "collections",
      "current": 5,
      "limit": 5,
      "tier": "free"
    }
  }
}
```

## Scaling Strategies

### 1. Optimize Request Patterns

**Problem:** Hitting RPM limits during peak traffic

**Solutions:**

* **Batch operations** – use `/batch` endpoints to group objects/documents
* **Cache aggressively** – enable `cache_config` on retrievers to reduce redundant searches
* **Async processing** – submit batches asynchronously, poll task status instead of blocking
* **Load shedding** – deprioritize non-critical operations during peak hours

### 2. Distribute Load Across API Keys

**Problem:** Single API key hitting concurrency limit

**Solutions:**

* Issue separate API keys per service/team
* Use key rotation for different application environments (staging, prod)
* Monitor per-key usage: `GET /v1/organizations/usage/api-keys/{key_id}`

### 3. Reduce Spend

**Problem:** Burning through your monthly usage pool

**Solutions:**

| High-Cost Operation | Optimization |
| - | - |
| **LLM generation stages** | Use smaller models (GPT-3.5 Turbo vs GPT-4), reduce `max_tokens` |
| **Frequent reprocessing** | Implement incremental updates instead of full reindexing |
| **Large batch ingestion** | Deduplicate objects before processing, filter out low-value content |
| **Exploratory searches** | Apply pre-filters to reduce search scope, lower `limit` values |
| **Web search stages** | Cache results with long TTL, fallback to internal collections |

### 4. Upgrade Tier

**When to upgrade:**

* Consistently hitting rate limits (>3 429 errors per hour)
* Usage pool >90% spent with 10+ days left in billing cycle
* Need for higher concurrency or batch sizes
* Require custom SLAs or dedicated infrastructure

Contact sales via "Talk to Engineers" CTA for enterprise pricing.

## Handling Rate Limit Errors

### Exponential Backoff

Implement retry logic with exponential backoff:

```python theme={null}
import time
import requests

def api_call_with_retry(url, headers, max_retries=5):
    for attempt in range(max_retries):
        resp = requests.post(url, headers=headers, json=payload)
        
        if resp.status_code == 429:
            retry_after = int(resp.headers.get("Retry-After", 2 ** attempt))
            print(f"Rate limited. Retrying in {retry_after}s...")
            time.sleep(retry_after)
            continue
        
        return resp
    
    raise Exception("Max retries exceeded")
```

### Circuit Breaker Pattern

Prevent cascading failures when rate limits are sustained:

```python theme={null}
class CircuitBreaker:
    def __init__(self, failure_threshold=5, timeout=60):
        self.failure_count = 0
        self.failure_threshold = failure_threshold
        self.timeout = timeout
        self.last_failure_time = None
        self.state = "CLOSED"  # CLOSED, OPEN, HALF_OPEN
    
    def call(self, func, *args, **kwargs):
        if self.state == "OPEN":
            if time.time() - self.last_failure_time > self.timeout:
                self.state = "HALF_OPEN"
            else:
                raise Exception("Circuit breaker is OPEN")
        
        try:
            result = func(*args, **kwargs)
            if self.state == "HALF_OPEN":
                self.state = "CLOSED"
                self.failure_count = 0
            return result
        except RateLimitError:
            self.failure_count += 1
            self.last_failure_time = time.time()
            if self.failure_count >= self.failure_threshold:
                self.state = "OPEN"
            raise
```

### Graceful Degradation

When rate limited, fall back to cached or reduced-quality results:

```python theme={null}
def search_with_fallback(query):
    try:
        return mixpeek.retrievers.execute(retriever_id, inputs={"query": query})
    except RateLimitError:
        # Fallback to cached results or simpler search
        return cached_search(query) or simplified_search(query)
```

## Endpoint-Specific Limits

Some endpoints have additional constraints:

| Endpoint | Special Limit | Reason |
| - | - | - |
| **Batch Submit** | 1 submission per batch every 60s | Prevents duplicate processing |
| **Cluster Execution** | 1 concurrent execution per cluster | Resource-intensive operation |
| **Web Search Stages** | 10 queries per minute (external API limit) | Third-party rate limit passthrough |
| **LLM Generation** | 100K tokens per minute | Model provider constraint |
| **Document List** | Max 10,000 results per query | Pagination required for large collections |

## Monitoring & Alerting

### Proactive Monitoring

Track these metrics to avoid surprises:

1. **Rate limit utilization** – alert at 80% of RPM limit
2. **Spend rate** – project end-of-month spend based on current trend
3. **Concurrent request count** – alert when approaching tier limit
4. **429 error frequency** – spike indicates need for optimization or upgrade

### Recommended Alerts

```yaml theme={null}
- name: "Rate Limit Warning"
  condition: rate_limit_remaining < 20% of limit
  action: Log warning, consider caching/batching

- name: "Usage Pool Critical"
  condition: usage_pool_remaining < 5% AND days_left > 5
  action: Upgrade tier or optimize high-cost operations

- name: "Sustained Rate Limiting"
  condition: 429_errors > 10 in 5 minutes
  action: Activate circuit breaker, alert on-call engineer

- name: "Quota Breach"
  condition: Resource creation fails with QuotaExceededError
  action: Archive unused resources or upgrade tier
```

## Best Practices

> **Implement client-side rate limiting**
>
> Don't rely solely on server enforcement. Implement token bucket or leaky bucket algorithms in your client to smooth request distribution and avoid bursts.

> **Cache responses aggressively**
>
> Enable retriever-level caching with appropriate TTLs. For exploratory queries, cache for 5-15 minutes. For stable queries (e.g., product search), cache for hours.

> **Use batch operations**
>
> Single-object operations consume rate limit budget faster. Batch 10-100 operations per request when possible.

> **Monitor per-API-key usage**
>
> Isolate noisy services by assigning separate API keys. Throttle or upgrade only the high-volume keys instead of affecting the entire org.

> **Set budget limits on retrievers**
>
> Configure `budget_limits` to prevent runaway costs from exploratory or LLM-heavy pipelines.

> **Paginate large result sets**
>
> Use `offset` and `limit` parameters instead of requesting thousands of documents at once. This reduces latency and spend.

## Enterprise Options

For organizations with sustained high volume:

* **Custom rate limits** – negotiate RPM/RPS based on traffic patterns
* **Reserved capacity** – pre-allocate Engine workers and inference quota
* **Dedicated infrastructure** – isolated MVS cluster, Redis, and Ray head nodes
* **Pool sharing** – share a usage pool across multiple sub-organizations
* **SLA guarantees** – contractual uptime and p99 latency commitments

Contact sales for custom pricing and limits.

## FAQ

> **Do rate limits apply per API key or per organization?**
>
> Rate limits are enforced at the **API key level**, but concurrent request limits apply at the **organization level**. This allows you to distribute load across multiple keys while respecting org-wide concurrency caps.

> **Are retries counted against rate limits?**
>
> Yes. Every request, including retries, counts toward your RPM/RPS limits. Implement exponential backoff to avoid wasting quota on rapid retries.

> **Can I request a temporary quota increase?**
>
> Yes. Contact support with your use case (e.g., annual reindexing, event-driven spike). We can provision temporary pool boosts or rate limit exemptions.

> **What happens if I exceed storage quota?**
>
> New document creation fails with a `QuotaExceededError`. Existing documents remain queryable. Delete unused documents or upgrade tier to restore write access.

> **Do cached retriever responses bill usage?**
>
> No. Cache hits are free and don't bill against your usage pool. Maximize cache hit rate to reduce spend.

## Next Steps

* Monitor usage via [Organization Usage API](/docs/api-reference/organization-usage/get-org-usage)
* Review [Analytics Overview](/docs/operations/analytics-overview) for cost optimization strategies
* Configure [Webhooks](/docs/operations/webhooks) to alert on quota thresholds
* Optimize retriever performance with [Caching Strategies](/docs/overview/caching)


# Security & Tenancy
Source: https://docs.mixpeek.com/docs/operations/security

Authentication, authorization, isolation, and operational safeguards

Mixpeek enforces organization-level authentication, namespace isolation, and stage-level validation across the entire stack. This page summarizes the security model and operational protections you should configure in production.

## Authentication

* **Header**: `Authorization: Bearer <api_key>`
* API keys belong to an organization; keys can be rotated, revoked, or scoped per environment.
* Sensitive operations (e.g., creating namespaces, rotating keys) require elevated permissions.

See [API Keys](/docs/operations/api-keys) for the full lifecycle: creating keys, setting `read`/`write`/`delete`/`admin` permissions and resource scopes, rotating and revoking, per-key rate limits, usage attribution, and end-user (`principal_id`) keys.

## Namespace Isolation

* **Header**: `X-Namespace: <namespace_id or namespace_name>`
* Every MongoDB query filters on `namespace_id`; indexes ensure isolation at scale.
* [MVS](https://mixpeek.com/mvs) uses one namespace per namespace (`ns_<namespace_id>`); payload filters ensure cross-namespace safety.
* Redis cache keys and Ray job metadata include namespace identifiers.

### Dual Identifier Model

| Identifier | Visible? | Purpose |
| - | - | - |
| `organization_id` | Yes | User-facing identifier in API responses |
| `internal_id` | No | Primary key for service-to-service lookups |
| `namespace_id` | Yes | Isolation boundary for data and compute |

Keep `internal_id` secret; it is intentionally absent from public APIs.

## Authorization & Rate Limits

* Routes declare required permission levels (`read`, `write`, `delete`, `admin`).
* Rate limits enforced via Redis middleware; set per-plan and per-route to protect backends.
* Tasks and retriever executions are metered in dollars; analytics endpoints expose usage metrics for billing reconciliation.

## Secrets & Credentials

Mixpeek provides an encrypted secrets vault for storing sensitive credentials like API keys. Secrets are encrypted at rest using Fernet symmetric encryption and are never exposed in API responses.

### Organization Secrets Vault

Store and manage secrets via the API:

```bash Create a Secret theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

```bash List Secrets theme={null}
# Returns secret names only - values are never exposed
curl "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

```bash Update a Secret theme={null}
curl -X PUT "https://api.mixpeek.com/v1/organizations/secrets/openai_api_key" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_value": "sk-proj-new-key..."
  }'
```

```bash Delete a Secret theme={null}
curl -X DELETE "https://api.mixpeek.com/v1/organizations/secrets/openai_api_key" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys. This gives you:

| Benefit | Description |
| - | - |
| **Cost Control** | Use your own LLM provider account and billing |
| **Rate Limits** | Use your own rate limits instead of shared |
| **Compliance** | Keep API calls under your own account |
| **Key Rotation** | Rotate keys without modifying retrievers |

#### Supported Providers

| Provider | Secret Name Example | Models |
| - | - | - |
| OpenAI | `openai_api_key` | gpt-4o, gpt-4o-mini |
| Anthropic | `anthropic_api_key` | claude-3-haiku, claude-3-sonnet, claude-3-opus |
| Google | `google_api_key` | gemini-3.1-flash-lite, gemini-2.5-pro |

#### Apply a Key to All LLM Operations

The fastest way to use your own key is to set it as the **organization-wide default**. Once configured, it automatically applies to every LLM operation — extractors, retrievers, clustering, taxonomy inference, and manifest generation — with no per-stage configuration needed.

**1. Store your key in the secrets vault**

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

```python Python theme={null}
from mixpeek import Mixpeek

mx = Mixpeek(api_key="YOUR_API_KEY")
mx.organizations.secrets.create(
    secret_name="openai_api_key",
    secret_value="sk-proj-abc123..."
)
```

**2. Set it as the org-wide default for that provider**

```bash cURL theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/organizations" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "default_llm_credentials": {
      "openai": "openai_api_key"
    }
  }'
```

```python Python theme={null}
mx.organizations.update(
    default_llm_credentials={
        "openai": "openai_api_key"
    }
)
```

You can configure defaults for multiple providers at once:

```json theme={null}
{
  "default_llm_credentials": {
    "openai": "my_openai_key",
    "anthropic": "my_anthropic_key",
    "google": "my_gemini_key"
  }
}
```

> [!TIP]
>
> You can also do this from Studio: when creating a secret, toggle **"Use as default LLM key"** and it will automatically be applied org-wide for the detected provider.

> [!NOTE]
>
> Values in `default_llm_credentials` are **secret names** (not raw API keys). The actual keys are stored encrypted in your secrets vault and resolved at runtime.

#### Override a Key on a Specific Stage

If you need a different key for a particular retriever stage, extractor, or cluster config, set `api_key` directly using the `{{secrets.name}}` template syntax. This overrides the org-wide default for that operation only.

```json theme={null}
{
  "stages": [
    {
      "stage_name": "llm_enrich",
      "stage_type": "enrich",
      "config": {
        "stage_id": "llm_enrich",
        "parameters": {
          "provider": "openai",
          "model_name": "gpt-4o-mini",
          "prompt": "Summarize this document.",
          "output_field": "summary",
          "api_key": "{{secrets.openai_api_key}}"
        }
      }
    }
  ]
}
```

#### Credential Resolution Order

Mixpeek resolves LLM credentials in this order (highest priority first):

1. **Per-resource** `api_key` / `{{secrets.name}}` — explicit key on a specific stage or extractor
2. **Organization default** — set once via `default_llm_credentials`, applied everywhere
3. **Mixpeek platform keys** — used when no custom key is configured (usage charged to your Mixpeek account)

#### Security

* **Encryption at rest**: All secrets are encrypted using Fernet symmetric encryption with a dedicated encryption key
* **Zero exposure**: Secret values are never returned in API responses — only secret names are visible
* **Per-request resolution**: Credentials are decrypted on-demand for each LLM call, not cached globally
* **No cross-tenant leakage**: Each organization's credentials are isolated; provider instances with custom keys are never shared between organizations
* **Audit trail**: Secret access and organization configuration changes are logged for compliance

### Security Best Practices

* **Rotate credentials regularly** – Update secrets via the API without changing retriever configurations
* **Use IAM roles** – For S3/GCS access, prefer IAM roles over long-lived access keys
* **Audit access logs** – Monitor secret access patterns for anomalies
* **Scope API keys** – Issue environment-specific Mixpeek API keys (dev, staging, prod)

## Audit Trail

Every audited write records who changed what, and optionally why. Read the trail
with `GET /v1/organizations/audit/logs` (filter by `resource_id`), one entry with
`GET /v1/organizations/audit/logs/{audit_id}`, or the whole thing as NDJSON via
`GET /v1/organizations/audit/logs/export`.

> [!NOTE]
>
> **Everything on this page arrived in one release, so confirm it against the
> host you call before depending on it.** Deployments move backward as well as
> forward: a rollback returns a whole release at once, so a capability that
> answered yesterday can stop answering without anything failing.
>
> The cheapest check is a write. Send a change with `X-Change-Reason`, read the
> audit entry back, and look at `reason` and at the shape of `changes`. A
> populated `reason` and `{old, new}` values mean the release is present. An
> absent `reason` means it is not, and nothing will have told you.

### Record why, not just what

Send `X-Change-Reason` on any audited write and the reason comes back on the
audit event.

```bash theme={null}
curl -X PATCH "$MP_API_URL/v1/retrievers/$RETRIEVER_ID" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Change-Reason: widening recall for the Q3 catalogue refresh" \
  -H "Content-Type: application/json" \
  -d '{ "description": "..." }'
```

The header is captured for the whole request, so it applies to every audited
write the call performs. The value is trimmed, has control characters stripped,
and is capped at 1,000 characters. An explicit `reason=` passed by an endpoint
that takes one wins over the header.

On a deployment that predates this, the header is **ignored** rather than
rejected: the write succeeds and `reason` comes back empty. That is the check in
the note above, and it is the only signal you get.

> [!NOTE]
>
> **`X-Change-Reason` is not in the OpenAPI spec.** It is read request-wide
> rather than declared per operation, so it appears on no endpoint's parameter
> list and the generated API reference cannot show it. This page is where it is
> documented.
>
> You can still confirm it is working: the `reason` field on `AuditEventResponse`
> names it, and a write sent with the header comes back with `reason` populated.

### Reading `changes`

`changes` is a JSON object, and datetime values inside it are ISO 8601 strings.

```json theme={null}
{
  "audit_id": "aud_...",
  "resource_type": "retriever",
  "action": "update",
  "reason": "widening recall for the Q3 catalogue refresh",
  "changes": {
    "description": { "old": "...", "new": "..." },
    "updated_at": { "old": "2026-08-01T09:00:00Z", "new": "2026-08-23T16:04:11Z" }
  }
}
```

How much detail `changes` carries depends on the resource:

| Resource | `changes` contains |
| - | - |
| Retriever, namespace, cluster, taxonomy | Value-level `{field: {old, new}}` |
| Collection common PATCH, bucket updates | Value-level `{field: {old, new}}` |
| Alerts update | Changed field names only |
| Object and document updates | Changed field names only |
| Collection `feature_extractor` repair path | Changed field names only |

> [!WARNING]
>
> **Names-only is deliberate where the table says so.** `changes` tells you which fields
> moved, and not what they moved from or to.
>
> **Verify the shape against a real entry before you build a compliance report on it.**
> Patch a collection, read the audit entry, and check whether its `changes` values are
> `{old, new}` objects or bare field names.
>
> **Check the host you actually call, and check it more than once.** A dedicated tenant
> deployment rolls separately from the shared API, so a capability can be present on one
> host and absent on the other. A deploy also replaces servers gradually, so two identical
> writes minutes apart can land on different sides. Treat a capability as live once
> repeated checks agree.

## Data Protection

* **Storage**: rely on encryption at rest provided by MongoDB Atlas, [MVS](https://mixpeek.com/mvs), or your infrastructure.
* **Transit**: require TLS for API endpoints and Ray Serve; use mTLS or network policies for cross-service traffic when available.
* **Backups**: configure automated backups for MongoDB and MVS; version S3 buckets with lifecycle policies.

## Operational Safeguards

* Enable `/v1/health` probes in load balancers to route around unhealthy instances.
* Use webhooks to detect ingestion completion; failed webhook deliveries remain retriable in MongoDB.
* Monitor rate-limit counters and task failure rates to spot abusive or buggy clients.
* Log request IDs and namespace IDs to correlate incidents quickly.

## Hardening Checklist

1. **Network** – restrict API access to trusted origins, configure CORS, and use private networking for backend services.
2. **Auth** – issue scoped API keys, expire unused keys, enable audit logging.
3. **Secrets** – manage via Vault, AWS Secrets Manager, GCP Secret Manager, or Kubernetes secrets with rotation.
4. **Tenancy** – adopt one namespace per environment/tenant; enforce `X-Namespace` always.
5. **Monitoring** – alert on health endpoint status, rate-limit breaches, or repeated 401/403 responses.

## References

* [API Keys](/docs/operations/api-keys)
* [Namespaces](/docs/ingestion/namespaces)
* [Observability](/docs/operations/observability)
* [Webhooks](/docs/operations/webhooks)
* [Tasks](/docs/processing/tasks)
* [Health Check](/docs/api-reference/health/healthcheck)


# Triggers vs Alerts
Source: https://docs.mixpeek.com/docs/operations/triggers-vs-alerts

Triggers run work on a schedule. Alerts watch for a condition and tell you. Which one you want, and where the seam is.

A **trigger** runs work. An **alert** watches and tells you.

Both are configured per namespace and both can end in a Slack message, which is why they get confused.

## Which one do I want

| You want to | Use |
| - | - |
| Re-cluster a collection every night | Trigger |
| Re-run enrichment after a taxonomy changes | Trigger |
| Re-process a batch that failed | Trigger |
| Know when a search starts returning nothing | Alert |
| Know when a search starts returning too much | Alert |
| Get a Slack message when a data-plane condition is met | Alert |

The short test: if the outcome is **work that happened**, you want a trigger. If the outcome is **someone finding out**, you want an alert.

## Triggers

A trigger is a schedule plus an action.

| `trigger_type` | Fires |
| - | - |
| `cron` | On a cron expression |
| `interval` | Every N seconds |
| `event` | When a named event occurs |
| `conditional` | When a condition evaluates true |

| `action_type` | Runs |
| - | - |
| `cluster` | Clustering over a collection |
| `taxonomy_enrichment` | Taxonomy enrichment |
| `batch_rerun` | A batch again |
| `collection_trigger` | A collection's processing |
| `retriever_execution` | A saved retriever with fixed inputs, judged against acceptance criteria |
| `alert_evaluation` | One retriever alert, on the trigger's schedule |

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/triggers \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "nightly recluster",
    "trigger_type": "cron",
    "schedule_config": {"cron_expression": "0 3 * * *", "timezone": "UTC"},
    "action_type": "cluster",
    "action_config": {"cluster_id": "clust_your_cluster_id"}
  }'
```

A trigger's output is its run history. Read it to see what ran and what failed.

## Alerts

An alert is a source plus a condition plus somewhere to send the news.

| `source` | Watches |
| - | - |
| `retriever` | A retriever's results. Set `trigger_on` to `results` or `no_results` |
| `system` | A built-in data-plane condition, via `system_condition` |

`notification_config.channels` takes one or more of `webhook`, `slack`, and `email`.

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/alerts \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "search returning nothing",
    "source": "retriever",
    "retriever_id": "ret_your_retriever_id",
    "trigger_on": "no_results",
    "notification_config": {
      "channels": [
        {"channel_type": "slack", "config": {"webhook_url": "https://hooks.slack.com/..."}}
      ]
    }
  }'
```

## When a retriever alert is evaluated

`execution_mode` takes `on_ingest`, `scheduled`, or `on_demand`.

`on_ingest` is what most alerts want. It rides the ingestion pipeline, so the alert is evaluated against data at the moment that data lands. Every retriever alert that is not `on_demand` is also evaluated every 5 minutes, which is how an alert on data written directly through the API fires.

To evaluate an alert on a schedule of your own, create a trigger whose action is `alert_evaluation`. While that trigger is active, the alert is evaluated on its schedule in the trigger's namespace and the 5-minute evaluation leaves it alone there. Pause or delete the trigger to hand it back.

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/triggers \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id" \
  -H "Content-Type: application/json" \
  -d '{
    "trigger_type": "cron",
    "schedule_config": {"cron_expression": "0 * * * *", "timezone": "UTC"},
    "action_type": "alert_evaluation",
    "action_config": {"alert_id": "alt_your_alert_id"}
  }'
```

A scheduled evaluation behaves like any other: the alert notifies when its condition starts to hold, stays quiet while it keeps holding unless `renotify_interval_seconds` is set, and respects `cooldown_seconds`. Each run appears in the trigger's run history.

`on_demand` alerts are never evaluated automatically; call them yourself.

## Related

> **[Webhooks](/docs/operations/webhooks)**
>
> Where an alert's `webhook` channel delivers.

> **[Change feed](/docs/operations/change-feed)**
>
> Reading what changed, rather than being told.


# Webhooks
Source: https://docs.mixpeek.com/docs/operations/webhooks

Respond to Mixpeek events without polling

Webhooks notify your systems when ingestion, enrichment, or retrieval events occur. The Engine writes events to MongoDB, the work scheduler dispatches them, and Mixpeek delivers HTTP POST requests (or channel-specific messages) until acknowledged.

## Event Flow

```
Engine → MongoDB (webhook_events) → work scheduler → delivery task → HTTP POST → Your endpoint
```

* Events persist until the worker receives a 2xx response.
* Retries use exponential backoff; failures remain in MongoDB for inspection.
* Delivery includes cache scopes so you can invalidate selectively.

## Common Event Types

| Event | Trigger | Payload Highlights |
| - | - | - |
| `collection.documents.written` | Engine finishes writing documents to [MVS](https://mixpeek.com/mvs) | `collection_id`, `document_ids`, `index_signature` |
| `object.created` | Object registered in a bucket | `bucket_id`, `object_id`, metadata snapshot |
| `documents.updated.batch` | A batch of documents finished updating | `collection_id`, `document_ids` |
| `cluster.execution.completed` | Clustering run finished | `cluster_id`, `run_id`, artifact URIs |
| `taxonomy.updated` | Taxonomy definition changed | `taxonomy_id` |
| `resource.executed` | Any primitive finished a run: a retriever, taxonomy, cluster, collection, storage connection, alert or trigger | `resource_type`, `resource_id`, `execution_id`, `duration_ms`, `cost_credits`, `output_count`, `feature_uris`, `model_versions`, `dropped_documents`, `pinned_revision` |
| `resource.execution_failed` | The same run failed | the same fields, with `error_class` set |

Use `/api-reference/webhooks/list-webhooks` to see the full catalog.

To receive runs of some primitives only, add `resource_types` to the webhook,
for example `"resource_types": ["retriever"]` with
`"event_types": ["resource.executed", "resource.execution_failed"]`. The filter
applies to every event type: for the per-kind types the resource type is the
first part of the name (`cluster` in `cluster.execution.completed`).

A batch is how a collection runs, so a finished batch sends one
`resource.executed` event per collection it wrote to, with `resource_type`
`collection`, the collection's id as `resource_id` and the batch id as
`execution_id`. The batch's credits are split evenly across those events, so
they add up to the batch's charge. Earlier events named the
batch itself (`resource_type` `batch`, the batch id as `resource_id`); to
receive batch runs now, filter on `"resource_types": ["collection"]`.
A sync run also sends an event for the storage connection it read through
(`resource_type` `storage_connection`).

A retriever's `resource.executed` payload carries `pinned_revision.config`: the
exact `stages`, `collection_ids` and `input_schema` that ran. Sending them to
`POST /v1/retrievers/execute` (with `collection_ids` as `collection_identifiers`)
and the same inputs reruns that version of the retriever.

> [!NOTE]
>
> The same events are readable as an ordered, resumable log via the
> [Change Feed](/docs/operations/change-feed). Webhooks push; the change feed lets
> you pull, which is what a consumer that can be offline needs.

## Create a Webhook

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/organizations/webhooks" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_name": "prod-notifications",
    "event_types": [
      "collection.documents.written",
      "cluster.execution.completed"
    ],
    "channels": [
      {
        "channel": "webhook",
        "configs": {
          "url": "https://hooks.example.com/mixpeek",
          "headers": { "X-Mixpeek-Secret": "super-secret"},
          "signing_secret": "a-long-random-string",
          "timeout": 10
        }
      }
    ]
  }'
```

Mixpeek also supports Slack, email, and SMS channels via channel-specific configs.

## Payload Structure

```json theme={null}
{
  "event_id": "evt_123",
  "event_type": "collection.documents.written",
  "occurred_at": "2025-10-28T10:03:22Z",
  "namespace_id": "ns_prod",
  "subject": {
    "type": "collection",
    "id": "col_products"
  },
  "metadata": {
    "document_count": 100,
    "collection_id": "col_products",
    "index_signature": "sig_xyz789"
  },
  "cache_scope": {
    "scope": "collection",
    "collection_id": "col_products"
  }
}
```

Use `event_id` for deduplication and store payloads for auditing.

## Security & Reliability

* Require HTTPS endpoints; reject plaintext URLs.
* Every delivery names its event in two headers: `X-Mixpeek-Event` (the event type, such as `alert.triggered`) and `X-Mixpeek-Event-Id` (the event's id, the same on a retry, so you can drop duplicates). The body is the event payload. A header you set on the webhook with the same name replaces ours.
* Set `signing_secret` and verify the `X-Mixpeek-Signature` header on every delivery (below). A static header such as `X-Mixpeek-Secret` also works, but it can be replayed.
* Respond quickly (\<10s). Offload heavy work to background jobs and return `200`.
* Use idempotent handlers; Mixpeek may retry on failure or timeout.
* Monitor webhook delivery with your logging pipeline; correlate by `event_id`.

### Verify a delivery

When a webhook channel has a `signing_secret`, every delivery carries:

```
X-Mixpeek-Signature: t=<unix seconds>,v1=<hex HMAC-SHA256>
```

The HMAC is computed over the string `<t>.` followed by the raw request body, keyed by your secret.
Recompute it over the raw bytes you received (before any JSON parsing), compare in constant time, and
reject a timestamp more than 300 seconds from your clock so a captured delivery cannot be replayed.

```python theme={null}
import hashlib, hmac, time

def verify(raw_body: bytes, header: str, secret: str, tolerance_s: int = 300) -> bool:
    parts = dict(p.split("=", 1) for p in header.split(",") if "=" in p)
    t, sig = parts.get("t", ""), parts.get("v1", "")
    if not t.isdigit() or abs(time.time() - int(t)) > tolerance_s:
        return False
    expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, sig)
```

The secret is stored encrypted and is never returned: API reads show `redacted` in its place.

## Operational Tips

1. Subscribe only to the events you need to reduce noise.
2. Combine webhook notifications with the Tasks API for full status context.
3. Use cache scopes to invalidate retriever caches efficiently (`collection`, `namespace`, or `key`).
4. Store webhook definitions in infrastructure-as-code so environments stay consistent.
5. Alert on sustained non-2xx responses and fix the endpoint quickly. After 30 failed deliveries in a row spanning at least two hours, Mixpeek pauses the webhook (`is_active: false`, with `paused_reason`) and alerts your organization; any successful delivery resets the count. Update the webhook to turn it back on.
6. Use a real endpoint. Create and update refuse URLs whose host can never receive a delivery, such as `example.com` and `.test` or `.invalid` names.

## Inbound Webhooks (from upstream providers)

In addition to **outbound** webhooks (Mixpeek → your endpoint), Mixpeek can receive **inbound** webhooks from external systems to trigger ingestion and lifecycle events.

### Sync lifecycle webhooks

Some sync integrations support inbound webhooks for **lifecycle events** (delete, update) on previously synced assets:

| Provider | Endpoint | Events handled | Use case |
| - | - | - | - |
| **Mux** | `POST /v1/webhooks/mux/{connection_id}` | `video.asset.deleted`, `video.asset.updated` | Cascade-delete bucket objects when a Mux asset is deleted. Re-evaluate metadata filters when asset metadata changes. See [Mux integration → Webhooks](/docs/integrations/object-storage/mux#webhooks-delete-and-update). |

These webhooks share common properties:

* Scoped to a single connection by URL path (no Authorization header needed).
* Signatures verified with `hmac.compare_digest` and provider-specific HMAC schemes.
* Unhandled events acknowledged with `200` and `handled: false` so retries stop cleanly.

## References

* [Create Webhook](/docs/api-reference/webhooks/create-webhook)
* [List Webhooks](/docs/api-reference/webhooks/list-webhooks)
* [Delete Webhook](/docs/api-reference/webhooks/delete-webhook)
* [Mux cascade-delete webhook](/docs/integrations/object-storage/mux#cascade-delete-via-webhooks)
* [Tasks](/docs/processing/tasks) – track the jobs that trigger webhook events
* [Security](/docs/operations/security) – authentication, tenancy, and secret management


# Caching & Signatures
Source: https://docs.mixpeek.com/docs/overview/caching

Keep retrieval fast without serving stale data

Mixpeek layers several caches to deliver low-latency responses while guaranteeing consistency. Every layer relies on deterministic signatures so you never serve results from an outdated index.

## Cache Layers

| Layer | Scope | Backing Store | TTL | Purpose |
| - | - | - | - | - |
| Retriever response | Full execution output | Redis | 1 hour (configurable) | Return entire execution payload instantly on repeated queries |
| Stage output | Individual stages (`feature_search`, `rerank`) | Redis | 1 hour (configurable per stage) | Reuse expensive stages across similar queries |
| Inference | Embeddings & rerankers | Redis | \~1 hour | Avoid recomputing identical model inferences |
| Document features | Stored vectors/payloads | [MVS](https://mixpeek.com/mvs) | Permanent | Reuse ingestion-time features for future queries |

## How It Works

```mermaid theme={null}
graph LR
  Q[Query] --> RC{Retriever Cache?}
  RC -->|HIT| R1[Return Cached Response]
  RC -->|MISS| P[Execute Pipeline]
  P --> SC{Stage Cache?}
  SC -->|HIT| SK[Skip Stage]
  SC -->|MISS| EX[Run Stage]
  EX --> SS[Store Stage Result]
  SK --> N[Next Stage]
  SS --> N
  N --> ST[Store Full Response]
  ST --> R2[Return Fresh Response]
```

On cache hit at the **retriever level**, the entire pipeline is skipped — response includes a `cached_at` timestamp so you can verify freshness. On cache hit at the **stage level**, only that stage is skipped and the rest of the pipeline continues.

## Index Signatures

Each collection stores an `index_signature` in MongoDB. The signature hashes:

* Collection configuration (feature extractor, passthrough fields)
* Document count and vector dimensions
* Timestamp of last ingestion event (with debounce logic)

Retriever cache keys include `index_signature`, so whenever ingestion updates the collection the signature changes and cached query responses automatically miss.

```text theme={null}
cache:retriever:quickstart-search:
  hash(
    inputs,
    filters,
    pagination,
    collection_signature="xyz789"
  )
```

## Response Cache Metadata

Retriever execution responses include cache information:

```json theme={null}
{
  "execution_id": "exec_abc123",
  "status": "completed",
  "cached_at": 1714150000.5,
  "documents": [...],
  "stage_statistics": {
    "stages": {
      "text_search": {
        "cache_hit": true,
        "cached_at": 1714150000.2,
        "duration_ms": 0.5
      },
      "rerank": {
        "cache_hit": false,
        "cached_at": null,
        "duration_ms": 45.3
      }
    }
  }
}
```

* **`cached_at`** (top-level) — Unix timestamp when the full response was cached. Present only on retriever-level cache hits. Compute freshness: `time.time() - cached_at`.
* **`cache_hit`** (per stage) — Whether this stage's result came from stage cache.
* **`cached_at`** (per stage) — Unix timestamp when this stage result was cached.

## Bypassing Cache

Force a fresh execution with `skip_cache`:

```bash theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/<id>/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H 'Content-Type: application/json' \
  -d '{ "inputs": { "query": "smart speaker" }, "skip_cache": true }'
```

## Stage-Level Controls

Control caching per stage via `cache_behavior` and `cache_ttl_seconds`:

```json theme={null}
{
  "stages": [
    {
      "stage_name": "text_search",
      "config": {
        "parameters": {
          "cache_behavior": "auto",
          "cache_ttl_seconds": 600
        }
      }
    },
    {
      "stage_name": "rerank",
      "config": {
        "parameters": {
          "cache_behavior": "disabled"
        }
      }
    }
  ]
}
```

**`cache_behavior` options:**

* `auto` (default) — Cache deterministic operations automatically
* `disabled` — Skip caching entirely for this stage
* `aggressive` — Cache even non-deterministic operations (use with caution)

## Inference Cache

The Engine caches model calls using a hashed payload of `(model_name, inputs, parameters)`. Use it to:

* Reuse embeddings for identical prompts or documents
* Skip recomputing reranking scores for popular queries
* Short-circuit repeated LLM-based filters with static criteria

## Cache Invalidation

Caches are invalidated automatically on:

| Event | Scope |
| - | - |
| Document ingestion completes | Collection-level (via index signature change) |
| Retriever deleted | All keys for that retriever |
| Collection deleted/updated | All keys for that collection |
| Namespace deleted | All keys in namespace |

Manual invalidation is also available:

```bash theme={null}
DELETE /v1/retrievers/{retriever_id}/cache
```

## Monitoring Cache Performance

* Use **`GET /v1/analytics/retrievers/{id}/cache-performance`** for hit/miss ratios and latency deltas.
* `stage_statistics` inside retriever responses flag `cache_hit` per stage.
* Redis namespaces per feature (e.g., `cache:retriever:...`) make it easy to inspect keys if needed.

## Best Practices

* Caching is on by default with `cache_behavior: "auto"` — no setup needed.
* Use `skip_cache: true` for debugging or when you need guaranteed-fresh results.
* Disable stage caching for stages with time-sensitive inputs (`now()`, `random()`).
* Use stage caching when reranking or feature search is the bottleneck.
* Use inference caching for expensive LLM or GPU workloads — even small hit rates pay off.


# Billing & Pricing
Source: https://docs.mixpeek.com/docs/platform/billing

Three questions set your price: what kind of files, how much content, and what you want to search by — tiers include a monthly usage pool

## How pricing works

Your price is the answer to three questions, in this order:

1. **What kind of files?** Video, image, audio, PDF/document, text, or web pages — the modality.
2. **How much content?** Each file type counts in its honest natural unit: images per file, video and audio per minute, PDFs per page, text per token, web per crawled page.
3. **What do you want to search by?** Visual similarity, faces, on-screen text, document layout, transcripts, ... — these plain-language capabilities are **[features](/docs/processing/features)**.

For each file type: **price = amount × (base rate + the search-by features you enable)**. Each tier is a flat monthly fee that includes a usage pool; anything beyond the pool bills at the same rate card.

**Worked example** — 100 videos averaging 2 minutes each, searchable by what's on screen (base video search) plus who appears (faces):

1. What kind of files: video → bills per minute
2. How much: 100 videos × 2 min = **200 minutes**
3. Search by: video search (base, $0.05/min) + faces (+$0.10/min)

200 min × ($0.05 + $0.10) = \*\*$30**. On Scale, the $100 monthly pool covers it entirely; on Build, $10 comes out of the pool and $20 bills as overage.

Everything is in dollars — rates, usage, invoices, and quotes are dollar-denominated end to end. Quote any planned ingestion before you run it with the [estimate endpoint](#estimate-before-you-run); it uses the same rating engine that bills you.

For current numbers, see the [pricing page](https://mixpeek.com/pricing). Manage your plan and payment method in [Studio Billing](https://studio.mixpeek.com/billing).

> [!NOTE]
>
> Under the hood, each feature maps to a feature extractor — that deeper layer (which extractor implements a feature, extractor options, bring-your-own extractors) is documented in [Features](/docs/processing/features#how-this-relates-to-extractors). You never need it to understand your bill.

***

## Plans

> [!NOTE]
>
> **Plan gating:** `GET /v1/organizations` returns `requires_plan`. When `true`,
> the workspace must pick a plan before creating namespaces or running billable
> work — gated endpoints return `403 PlanRequiredError` with a link to
> [choose a plan](https://studio.mixpeek.com/signup/plan). There is no separate
> "tier" endpoint: `account_type`, `credit_count`, and `requires_plan` on the
> organization are the capability surface.

Each tier is a **flat monthly fee that includes a usage pool** — a dollar allowance you can spend on any mix of modalities and features. Only usage beyond the pool bills as overage, at the same rate card.

| | Build | Scale | Enterprise |
| - | - | - | - |
| **Monthly price** | \$25 | \$250 | Custom |
| **Included usage pool** | \$10 | \$100 | Custom |
| **Pool buys, e.g.** | \~6,600 images **or** 200 video minutes **or** \~6,600 pages | \~66,000 images **or** 2,000 video minutes **or** \~66,000 pages | — |
| **Overage** | Rate card | Rate card | Contract |
| **Custom extractors** | Included | Included | Included |
| **Support** | Email | Priority + private Slack | Dedicated |

The pool is **fungible** — it isn't a per-modality cap. The natural-unit equivalents above are just the pool divided by each base rate, so you can compare tiers at a glance. Spend it on whatever mix you actually ingest.

> [!NOTE]
>
> Enterprise runs on dedicated single-tenant infrastructure with custom contracts — see [Single Tenant](/docs/resources/single-tenant) or [contact sales](https://mixpeek.com/contact).

***

## The rate card

**The API is the source of truth for rates.** `GET /v1/billing/pricing` is public (no auth) and returns the live rate card — the same payload that studio, the homepage, and the billing engine read, so displayed numbers can never disagree with billed numbers:

```bash theme={null}
curl "https://api.mixpeek.com/v1/billing/pricing"
```

```json theme={null}
{
  "pricing_model": "v2",
  "currency": "usd",
  "placeholder_rates": false,
  "modalities": [
    {
      "modality": "video",
      "unit": "minute",
      "unit_display": "per minute",
      "base": { "key": "video_search", "name": "Video search (scene embeddings)", "rate_usd": 0.05, "per": 1 },
      "addons": [
        { "key": "faces", "name": "Face detection + identity", "kind": "addon", "rate_usd": 0.10, "per": 1 },
        { "key": "onscreen_text", "name": "On-screen text (video OCR)", "kind": "addon", "rate_usd": 0.10, "per": 1 },
        { "key": "audio_fingerprint", "name": "Audio fingerprint (in video)", "kind": "addon", "rate_usd": 0.01, "per": 1 },
        { "key": "multimodal_understanding", "kind": "external", "pricing_note": "usage-based (external inference)" },
        { "key": "clustering", "kind": "included", "pricing_note": "included — no additional charge" },
        { "key": "taxonomy_enrichment", "kind": "included", "pricing_note": "included — no additional charge" }
      ]
    }
  ],
  "usage_pools": {
    "build": {
      "flat_monthly_usd": 25.0,
      "usage_pool_usd": 10.0,
      "pool_equivalents": { "images": 6667, "video_minutes": 200, "pages": 6667, "text_tokens": 5000000 }
    }
  },
  "mvs_usage_rates": {
    "vectors_stored_per_million_month_usd": 1.56,
    "storage_per_gb_month_usd": 0.023,
    "queries_per_million_usd": 2.0,
    "writes_per_million_usd": 2.0
  },
  "reads": { "overage_usd_per_1m_queries": 2.0, "image_query_embedding_usd": 0.001 },
  "full_res_multiplier": 2.0
}
```

Response abbreviated — the live payload covers all six modalities and every tier. How to read it:

* **`modalities[].base`** — the per-unit rate for making that modality searchable. Creating any collection for a modality gets you its base feature.
* **`modalities[].addons`** — features you opt into, priced per the same unit. `kind: "external"` rows are backed by external LLM inference and priced as usage-based passthrough (never flat-rated). `kind: "included"` rows (clustering, taxonomy enrichment) cost nothing extra.
* **`usage_pools`** — each tier's flat fee and included dollar pool, with derived natural-unit equivalents.
* **`placeholder_rates`** — `false` means the rates are live and billable. (The nested `v2` object in the payload is a deprecated alias of these same top-level keys and will be removed.)

> [!TIP]
>
> Don't hardcode rates into your own tooling — read them from `GET /v1/billing/pricing`, or quote concrete workloads with the [estimate endpoint](#estimate-before-you-run).

### Price a workload before you sign up

The rate card needs no account, so you can cost a workload from code before choosing a plan. This prices 6,000 video minutes a month with face search and on-screen text:

```python theme={null}
import requests

card = requests.get("https://api.mixpeek.com/v1/billing/pricing").json()
video = next(m for m in card["modalities"] if m["modality"] == "video")
rates = {video["base"]["key"]: video["base"]}
rates.update({a["key"]: a for a in video["addons"]})

minutes = 6000
wanted = [video["base"]["key"], "faces", "onscreen_text"]
usd = sum(minutes / rates[k]["per"] * rates[k]["rate_usd"] for k in wanted if k in rates)
print(f"~${usd:,.2f} per month before your plan's included usage")
```

Add-on keys differ per modality; print `video["addons"]` to see them. The [estimate endpoint](#estimate-before-you-run) returns the same numbers itemized against your plan's remaining pool, and it needs an API key on an active plan.

### How a batch is priced

When you submit a batch, each object bills as: **modality units x (base rate + enabled add-on rates)**.

Example at the rates shown above: a 10-minute video in a collection with `faces` and `onscreen_text` enabled bills 10 min x ($0.05 + $0.10 + $0.10) = **$2.50\*\*.

* **Measured units** — video/audio minutes come from measured duration (not file size), pages from actual page counts. Anything estimated at submit is trued up at batch completion, in both directions.
* **Batch minimum** — batches bill at least **\$0.01**.
* **Normalization included** — content is normalized once at ingest (video to 720p mezzanine, images capped, audio to 16kHz mono). Originals stay untouched in your bucket. Collections that opt out with `full_res: true` bill at a **2x multiplier**. See [full resolution opt-out](/docs/processing/features#full-resolution-opt-out).
* **Deduplication** — re-ingesting content Mixpeek has already processed (same content hash) skips extraction rather than re-billing full processing.

### Storage

Storage bills at two rates from `mvs_usage_rates`: **$1.56 per 1M vectors stored per month** (`vectors_stored_per_million_month_usd`) plus **$0.023 per GB-month** (`storage_per_gb_month_usd`).

### Reads (queries)

Reads bill at **\$2 per 1M queries** (`reads.overage_usd_per_1m_queries`).

A saved retriever run with an image as its query also embeds that image once, billed as one embedding at **\$0.001** (`reads.image_query_embedding_usd`) and listed as a `retriever_query_embedding` line in your usage. A query that sends a precomputed vector embeds nothing and is not billed.

***

## Estimate before you run

`POST /v1/organizations/billing/estimate` quotes planned ingestion — same rating engine, same rate card, so the quote matches the eventual charge:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/billing/estimate" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "items": [{ "mime_type": "video/mp4", "minutes": 42 }],
    "features": ["faces"]
  }'
```

The response itemizes dollars per (modality, feature), reports how much falls inside your remaining usage pool, and what would bill as overage. Full request/response details in [Features → Estimate](/docs/processing/features#estimate-before-you-run).

***

## Tracking usage

Current-period usage is available in [Studio Billing](https://studio.mixpeek.com/billing) and via the [usage API](/docs/api-reference/organization-billing/get-current-usage). Usage reports break down dollars by modality and feature:

* units consumed per modality (minutes, images, pages, tokens) and per feature
* storage GB-month and reads
* pool consumption, overage, and the period's total in dollars

You can set spending caps to limit overage through [Studio Billing](https://studio.mixpeek.com/billing) or the [spending caps API](/docs/api-reference/organization-billing/get-spending-caps).

***

## How billing works

**1. Sign up and choose a plan**

Create an account at [studio.mixpeek.com](https://studio.mixpeek.com). Upgrade to Build or Scale from the billing page.

**2. Add a payment method**

Paid plans require a card on file via Stripe. Add or update your card in [Studio Billing](https://studio.mixpeek.com/billing) or through the [payment method API](/docs/api-reference/organization-billing/setup-payment-method).

**3. Use Mixpeek**

Your usage pool is available immediately and resets each cycle. Usage is metered continuously in dollars and visible in the dashboard.

**4. Receive your invoice**

Stripe generates a monthly invoice: your flat plan fee plus any overage, itemized by modality and feature. Invoices are in [Studio Billing](https://studio.mixpeek.com/billing) and via the [invoices API](/docs/api-reference/organization-billing/list-invoices).

***

## MVS Standalone

Using Mixpeek's vector store with your own embeddings (no extraction)? MVS bills separately. You pay the plan's flat monthly fee, plus $1.56 per 1M vectors stored per month, $0.023 per GB-month, and \$2 per 1M queries or writes. See the live `mvs_plans` and `mvs_usage_rates` in `GET /v1/billing/pricing` and the [MVS overview](/docs/vector-store/overview).

***

## FAQ

> **What happened to credits?**
>
> Credits are gone as a customer-facing concept — every surface (rates, usage, invoices, quotes) is dollar-denominated. If you see a legacy `credits` field in an older API response, it's an internal ledger unit worth \$0.001 (a milli-dollar); the dollar fields alongside it are canonical.

> **Can the quote and the charge disagree?**
>
> No. The estimate endpoint and the metering pipeline call the same rating engine against the same rate card. Estimated units (e.g. duration unknown at submit) are trued up from measured values at batch completion — in both directions.

> **Is the usage pool a per-modality cap?**
>
> No. It's a fungible dollar pool — the natural-unit equivalents shown on tier cards are illustrations (pool ÷ base rate), not limits. Spend it on any mix of modalities and features.

> **Do unused pool dollars roll over?**
>
> No. The usage pool resets at the start of each billing cycle.

> **Can I switch plans mid-cycle?**
>
> Yes. Upgrades take effect immediately and are prorated. Downgrades take effect at the start of the next billing cycle.

> **How are LLM-backed features priced?**
>
> Features marked `kind: "external"` (e.g. `multimodal_understanding`) are backed by external LLM inference and priced as usage-based passthrough — their upstream cost is unbounded, so they're never flat-rated. The pricing payload flags them with a `pricing_note`.

> **How are custom extractors priced?**
>
> A `custom:<plugin>` feature derives its per-unit rate from the compute profile the plugin declares — the same machinery that prices native features. Quote it like anything else via the estimate endpoint.

> **What happens when I exceed caps?**
>
> Requests that exceed hard resource caps return `403` with type `QuotaExceededError` and an `upgrade_plan` hint. Overage within your plan simply bills at the rate card — no interruption.

> **What payment methods does Mixpeek accept?**
>
> All major credit and debit cards via Stripe. Enterprise customers can pay by invoice with net-30 terms.

> **Is there an annual billing option?**
>
> Annual billing is available for Scale and Enterprise. Contact sales for details.


# Budgets
Source: https://docs.mixpeek.com/docs/platform/budgets

Every organization and namespace has a daily and monthly spend budget. See it, change it, and know what happens when work reaches it.

Every organization and every namespace has a spend budget from the moment it exists. A budget has a daily and a monthly period, in US dollars of metered usage. It counts all metered work in its scope: batch processing, extraction, and search-query inference.

You change budgets yourself, in [Studio](https://studio.mixpeek.com) or through the API. Two bounds apply:

* A namespace budget fits inside its organization's budget.
* An organization budget stays at or below your plan's safety ceiling. To go above the ceiling, contact Mixpeek support.

Every change records who made it, when, and the value it replaced.

## Defaults

A new organization starts with the default budget for its plan. Each namespace starts with its organization's namespace default. If your organization already set a monthly spending cap under [Billing](/docs/platform/billing#tracking-usage), that cap is its monthly budget until you set one here.

| Plan | Daily budget | Monthly budget |
| - | - | - |
| Free | \$5 | \$25 |
| Build | \$50 | \$1,000 |
| Scale | \$200 | \$4,000 |
| Enterprise | \$500 | \$10,000 |

`GET /v1/organizations` returns the budget in effect as `spend_limits`, with spend to date and the headroom left. `GET /v1/namespaces/{namespace_id}` returns the namespace's as `budget`. Each field names its source, so you can tell a default from a value someone set.

## Set the organization budget

An organization admin sets the organization budget and the default every namespace gets. Send `null` for a field to return it to the plan default.

```python Python theme={null}
import requests

resp = requests.patch(
    "https://api.mixpeek.com/v1/organizations",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "spend_limits": {
            "daily_usd": 40,
            "monthly_usd": 800,
            "namespace_default_daily_usd": 10,
        }
    },
)
print(resp.json()["spend_limits"])
```

```javascript JavaScript theme={null}
const resp = await fetch("https://api.mixpeek.com/v1/organizations", {
  method: "PATCH",
  headers: {
    Authorization: "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    spend_limits: { daily_usd: 40, monthly_usd: 800, namespace_default_daily_usd: 10 },
  }),
});
console.log((await resp.json()).spend_limits);
```

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/organizations \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"spend_limits": {"daily_usd": 40, "monthly_usd": 800, "namespace_default_daily_usd": 10}}'
```

A value above your plan's ceiling returns `400` with the ceiling in the message.

## Set a namespace budget

```python Python theme={null}
import requests

resp = requests.patch(
    "https://api.mixpeek.com/v1/namespaces/ns_your_namespace",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"spend_budget": {"daily_usd": 5, "monthly_usd": 100}},
)
print(resp.json()["budget"])
```

```javascript JavaScript theme={null}
const resp = await fetch("https://api.mixpeek.com/v1/namespaces/ns_your_namespace", {
  method: "PATCH",
  headers: {
    Authorization: "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ spend_budget: { daily_usd: 5, monthly_usd: 100 } }),
});
console.log((await resp.json()).budget);
```

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/namespaces/ns_your_namespace \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"spend_budget": {"daily_usd": 5, "monthly_usd": 100}}'
```

A namespace budget above its organization's budget returns `400`. Raise the organization budget first.

## At the limit

Work in a scope that has reached its budget behaves by kind:

* **New batches are refused.** The batch submit returns `403` with `type: "QuotaExceededError"`. The message names the budget and the period, and says how to raise it. `details.how_to_set_budget` holds the request that raises it.
* **Search keeps working.** Retrievers and query inference continue under a conservative request quota.
* **Scheduled triggers wait.** A trigger due while its scope is at a limit holds that firing and runs once the scope has room again. A held firing does not count as a failure.

A daily budget resets at 00:00 UTC. A monthly budget resets on the 1st at 00:00 UTC.

## Automatic safety limits

Mixpeek also applies limits you do not set, which protect against runaway spend:

* **Spend velocity.** A namespace or organization whose hourly spend jumps far above its own recent pattern is paused for new paid work. One trial run is admitted after 30 minutes. If spend stays normal, the pause lifts on its own.
* **Retries.** A failed batch is retried automatically at most 5 times. All attempts share one spend budget. A batch that spent on a paid model and wrote no documents is retried at most once.
* **Failing schedules.** A scheduled trigger pauses after 3 failed runs in a row. It makes one trial run per hour, and a successful run restores it.
* **Free plan automation.** A free organization without a payment method can hold 1 active scheduled trigger and 1 active bucket sync. Add a payment method in [Studio Billing](https://studio.mixpeek.com/billing) to run more.
* **Mixpeek's monthly spending stop.** If Mixpeek's own monthly spending stop is reached, new paid work is paused for every organization. A batch submit returns `402` with `type: "PaymentRequiredError"` and `details.limit_kind: "hard_stop"`. No budget you set lifts it: Mixpeek lifts it, or it lifts when the next billing period starts (`details.lifts_on`). Search and existing results keep working.
* **Inactive accounts.** On an account with no sign-in or API activity for 14 days, scheduled triggers and syncs pause. The next API call or sign-in resumes them. Paid accounts are exempt.


# Classify Content
Source: https://docs.mixpeek.com/docs/platform/enrichment

Auto-label documents with taxonomies, retriever enrichments, and annotations

![Content classification: documents are enriched via taxonomies (flat or hierarchical similarity matching) or retriever enrichments (pipeline execution with field write-back)](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-enrichment.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=87c8771bf4023383ab93a1b794a9e9eb)

> [!NOTE]
>
> For full configuration details, parameters, and advanced options, see the [Taxonomies reference](/docs/enrichment/taxonomies).

## Taxonomies

Auto-classify documents by matching them against reference collections. Two types:

**Flat** — match each document against a single reference collection. When similarity exceeds the threshold, enrichment fields (SKU, category, label) are attached.

**Hierarchical** — parent/child nodes with inheritance. Documents traverse levels of refinement (brand → category → subcategory) using different features at each level.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/taxonomies" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "taxonomy_name": "product-categories",
    "config": {
      "taxonomy_type": "flat",
      "retriever_id": "'$REF_RETRIEVER_ID'",
      "input_mappings": [
        { "input_key": "query", "source_type": "payload", "path": "title" }
      ],
      "source_collection": {
        "collection_id": "'$REF_COLLECTION_ID'",
        "enrichment_fields": [
          { "field_path": "category", "merge_mode": "replace" },
          { "field_path": "subcategory", "merge_mode": "replace" },
          { "field_path": "brand", "merge_mode": "replace" }
        ]
      }
    }
  }'
```

### When to Run

| Mode | Runs | Configured via | Use case |
| - | - | - | - |
| On-demand | At query time | A `taxonomy_enrich` stage on a retriever | Dynamic classification, A/B testing |
| Materialized | After extraction, persists to collection | `taxonomy_applications[].execution_mode: "materialize"` on the collection | Stable labels, fast queries |
| Retroactive | Reapplies when taxonomy updates | `POST /v1/collections/{id}/apply-taxonomy` | Backfill when reference data improves |

`taxonomy_applications[].execution_mode` only accepts `materialize`; on-demand and retroactive run through the other two surfaces above, each with its own request shape (see [Taxonomies](/docs/enrichment/taxonomies)).

[Taxonomy API →](/docs/api-reference/taxonomies/create-taxonomy)

## Retriever Enrichments

Attach a retriever pipeline to a collection so it runs on every new document. The retriever executes, and selected result fields are written back to the document.

```bash theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/collections/$COLLECTION_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_enrichments": [{
      "retriever_id": "'$RETRIEVER_ID'",
      "input_mappings": { "query_text": { "source": "payload", "path": "description" } },
      "write_back_fields": { "category": { "mode": "first", "path": "results[0].metadata.category" } }
    }]
  }'
```

Use cases: auto-classify via LLM, cross-collection joins, label propagation from seed documents.

[Collection update API →](/docs/api-reference/collections/update-collection)

## Annotations

Explicit human decisions with full provenance — the ground truth layer for compliance, review workflows, and improving retrieval quality over time.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/annotations" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_abc",
    "collection_id": "col_xyz",
    "retriever_id": "ret_123",
    "execution_id": "exec_789",
    "stage_name": "feature_search",
    "label": "approved",
    "confidence": 0.95,
    "reasoning": "Matches reference product exactly",
    "payload": { "sku": "SKU-001", "action": "keep", "actor_id": "user_456", "actor_type": "human" }
  }'
```

### What Each Annotation Captures

| Field | Purpose |
| - | - |
| `document_id`, `collection_id` | What was reviewed |
| `retriever_id`, `execution_id`, `stage_name` | How it was surfaced |
| `label`, `confidence`, `reasoning` | The decision |
| `payload` | Structured workflow-specific data (SKU, action, notes). Who decided goes here too: there are no top-level `actor_id` / `actor_type` fields |

Annotations are stored independently from documents — they never modify the source data. Use them to build review queues, audit trails, and curated ground truth datasets.

### Bulk Operations

Process review queues at scale with the bulk API:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/annotations/bulk" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "create": [
      { "document_id": "doc_1", "collection_id": "col_xyz", "label": "approved" },
      { "document_id": "doc_2", "collection_id": "col_xyz", "label": "rejected", "reasoning": "Low quality match" }
    ],
    "update": [],
    "delete": []
  }'
```

### The Feedback Loop

Annotations feed directly into the platform's learning cycle:

1. **Annotations** provide explicit ground truth for edge cases
2. **Learned fusion** uses annotations to auto-tune retriever stage weights
3. **Approved annotations** can be piped into reference collections, expanding your taxonomy's coverage
4. **Retroactive taxonomy application** reclassifies existing documents when annotations improve the reference set

[Annotation API →](/docs/api-reference/annotations/create-annotation) · [Bulk API →](/docs/api-reference/annotations/bulk-annotations)

## Choosing an Approach

| Goal | Use |
| - | - |
| Auto-label with a reference catalog | Flat taxonomy (materialize mode) |
| Hierarchical classification (brand → category → SKU) | Hierarchical taxonomy |
| Auto-classify via LLM at ingest | Retriever enrichment with `llm_enrich` stage |
| Cross-collection joins (enrich from another dataset) | Retriever enrichment with `document_enrich` stage |
| Human review with audit trail | Annotations |
| Backfill when labels improve | Retroactive taxonomy application |


# Improve Relevance
Source: https://docs.mixpeek.com/docs/platform/improve-relevance

Make search better over time with interactions, fusion strategies, and evaluations

![Improve relevance: search results generate user interaction signals, which feed fusion strategies and evaluations in a feedback loop](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-improve-relevance.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=45d5b8c4e5e572a1ffe44216cefd90d5)

## Interaction Signals

Capture implicit user behavior — clicks, views, dwell time, purchases — to feed into retrieval optimization.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/interactions" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "execution_id": "exec_abc",
    "feature_id": "doc_xyz",
    "interaction_type": ["click"],
    "position": 3,
    "metadata": { "duration_ms": 4500 }
  }'
```

`interaction_type` is always a JSON **array** (co-occurring signals of one action, e.g. `["click", "long_view"]`), and the document is `feature_id`. Track `position` for every signal — it is recorded for analytics and evaluation (NDCG and other rank-aware metrics).

### Signal Strength

| Signal | Weight | When to track |
| - | - | - |
| `click` | Medium | User clicked a result |
| `long_view` | High | Sustained engagement (pass `duration_ms`) |
| `add_to_cart` | High | Intent / funnel step |
| `purchase` | Highest | User completed a goal action |
| `negative_feedback` | Penalty | User disliked / hid the result |

See [Interaction Signals](/docs/retrieval/interactions#signal-strategy) for the full signal matrix and per-use-case patterns.

[Interaction API →](/docs/api-reference/retriever-interactions/create-interaction)

## Auto-Tune (Learned Fusion)

Auto-Tune automatically adapts fusion weights per user based on their interaction history. Instead of manually choosing weights, the system uses Thompson Sampling to learn the optimal blend of features for each user. It learns one weight per feature and keeps no per-document state. To change how individual documents rank, add a [Rerank stage](/docs/retrieval/stages/rerank).

> **[How It Works](/docs/retrieval/auto-tune)**
>
> Concept page — Thompson Sampling, context levels, reward signals

> **[Reward Signals](/docs/retrieval/reward-signals)**
>
> Configure which interactions drive learning and how much

> **[Rollout Guide](/docs/retrieval/auto-tune-rollout)**
>
> Traffic splitting, shadow mode, kill switch, per-user opt-out

**Quick setup:**

```python Python SDK theme={null}
# 1. Create a retriever with learned fusion
retriever = client.retrievers.create(
    retriever_name="personalized-search",
    stages=[{
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [
                    {
                        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                        "query": "{{INPUT.query}}",
                        "top_k": 100
                    },
                    {
                        "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                        "query": "{{INPUT.query}}",
                        "top_k": 100
                    }
                ],
                "fusion": "learned",
                "learning_config": {
                    "context_features": ["INPUT.user_id"],
                    "reward_map": {"click": 1.0, "purchase": 3.0}
                },
                "final_top_k": 25
            }
        }
    }]
)

# 2. Send interactions — learning happens automatically
client.retrievers.create_interaction(
    feature_id="doc_123",
    interaction_type=["click"],
    position=2,
    user_id="user_abc"
)
```

```bash cURL theme={null}
# 1. Create a retriever with learned fusion
curl -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "personalized-search",
    "stages": [{
      "stage_name": "feature_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": "{{INPUT.query}}",
              "top_k": 100
            },
            {
              "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
              "query": "{{INPUT.query}}",
              "top_k": 100
            }
          ],
          "fusion": "learned",
          "learning_config": {
            "context_features": ["INPUT.user_id"],
            "reward_map": {"click": 1.0, "purchase": 3.0}
          },
          "final_top_k": 25
        }
      }
    }]
  }'

# 2. Send interactions — learning happens automatically
curl -X POST "$MP_API_URL/v1/retrievers/interactions" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "feature_id": "doc_123",
    "interaction_type": ["click"],
    "position": 2,
    "user_id": "user_abc"
  }'
```

For a step-by-step walkthrough, see the [Build a Feedback Loop](/docs/tutorials/feedback-loop) tutorial.

## Fusion Strategies

When a retriever has multiple search stages, fusion strategies determine how scores combine into the final ranking.

| Strategy | How it works | Best for |
| - | - | - |
| **RRF** (Reciprocal Rank Fusion) | Combines ranks, not scores. `1/(k + rank)` | Default — works well with no tuning |
| **DBSF** (Distribution-Based Score Fusion) | Normalizes score distributions then averages | When scores have different scales |
| **Weighted** | Manual weights per stage | When you know which stage matters more |
| **Max** | Takes the highest score across stages | When any match is sufficient |
| **Learned** | Auto-tunes weights from interaction signals | When you have 500+ interactions |

Set `fusion` inside the `feature_search` stage parameters (alongside `searches` and `final_top_k`):

```json theme={null}
{
  "fusion": "rrf"
}
```

For `"fusion": "learned"`, add a `learning_config` (see the Auto-Tune example above). Learned fusion uses Thompson Sampling to shift weight toward stages whose results users engage with; with zero interactions it behaves like `rrf` and transitions as signals accumulate.

## Evaluations

Measure retriever quality against ground truth datasets with standard IR metrics.

```bash theme={null}
# Create a ground truth dataset
curl -X POST "https://api.mixpeek.com/v1/retrievers/evaluations/datasets" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_name": "product-search-eval",
    "queries": [
      {
        "query_id": "q1",
        "query_input": {"query_text": "wireless headphones"},
        "relevant_documents": ["doc_1", "doc_2"]
      }
    ]
  }'

# Run evaluation
curl -X POST "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/evaluations" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_name": "product-search-eval",
    "evaluation_config": { "k_values": [1, 5, 10, 20] }
  }'
```

The run is asynchronous. Poll `GET /v1/retrievers/{retriever_id}/evaluations/{evaluation_id}` for Precision, Recall, F1, NDCG, MAP, and MRR at your configured cutoffs. Use evaluations to compare retriever configurations before deploying changes.

[Evaluation API →](/docs/api-reference/retriever-evaluations/run-evaluation)

## Analytics

Monitor retriever performance in production:

* **Stage latency breakdown** — identify which stages are slow
* **Cache hit rates** — verify caching is effective
* **Score distributions** — detect relevance drift
* **Query patterns** — understand what users search for

See [Analytics & Performance](/docs/operations/analytics-overview) for the analytics endpoints to build dashboards or trigger alerts on degradation.

## The Feedback Loop

```
Search → Results → User interacts → Signal captured → Fusion learns → Better results
```

1. Users search via retrievers
2. Interaction signals capture what they engage with
3. Learned fusion adjusts stage weights automatically
4. Annotations provide explicit ground truth for edge cases
5. Evaluations measure improvement quantitatively
6. The cycle repeats — retrieval improves with usage


# Operate
Source: https://docs.mixpeek.com/docs/platform/operations

Run Mixpeek in production — security, webhooks, manifests, and infrastructure

![Operations: authentication with API keys and namespaces, document ACL for row-level security, webhooks for event subscriptions, manifests for config-as-code](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-operations.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=31d24799fc6308a1ff120fc4b3f2bf88)

## Authentication

Every request requires a Bearer token and namespace header:

```
Authorization: Bearer mxp_sk_...
X-Namespace: ns_production
```

API keys are scoped to an organization. Namespaces provide the authorization boundary — use separate namespaces for dev, staging, and production.

See [API Keys](/docs/operations/api-keys) to create scoped keys, set permissions, rotate and revoke them, and monitor per-key usage.

## Secrets & LLM Keys

Store third-party API keys in an encrypted secrets vault. Secrets are encrypted at rest using Fernet symmetric encryption and are never exposed in API responses.

### Manage Secrets

```bash Create a Secret theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

```bash List Secrets theme={null}
# Returns secret names only — values are never exposed
curl "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

```bash Update a Secret theme={null}
curl -X PUT "https://api.mixpeek.com/v1/organizations/secrets/openai_api_key" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "secret_value": "sk-proj-new-key..." }'
```

```bash Delete a Secret theme={null}
curl -X DELETE "https://api.mixpeek.com/v1/organizations/secrets/openai_api_key" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys for cost control, higher rate limits, compliance, and independent key rotation.

**Supported providers:** OpenAI, Anthropic, Google

#### Apply a Key to All LLM Operations

Set a key as the **organization-wide default** and it automatically applies to every LLM operation — extractors, retrievers, clustering, taxonomy inference, and manifest generation — with no per-stage configuration.

**1. Store your key in the secrets vault**

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

```python Python theme={null}
from mixpeek import Mixpeek

mx = Mixpeek(api_key="YOUR_API_KEY")
mx.organizations.secrets.create(
    secret_name="openai_api_key",
    secret_value="sk-proj-abc123..."
)
```

**2. Set it as the org-wide default for that provider**

```bash cURL theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/organizations" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "default_llm_credentials": {
      "openai": "openai_api_key"
    }
  }'
```

```python Python theme={null}
mx.organizations.update(
    default_llm_credentials={
        "openai": "openai_api_key"
    }
)
```

Configure defaults for multiple providers at once:

```json theme={null}
{
  "default_llm_credentials": {
    "openai": "my_openai_key",
    "anthropic": "my_anthropic_key",
    "google": "my_gemini_key"
  }
}
```

> [!TIP]
>
> In Studio, toggle **"Use as default LLM key"** when creating a secret to automatically apply it org-wide for the detected provider.

#### Override a Key on a Specific Stage

Set `api_key` directly on a retriever stage, extractor, or cluster config using `{{secrets.name}}` template syntax to override the org-wide default for that operation only:

```json theme={null}
{
  "parameters": {
    "provider": "openai",
    "model": "gpt-4o-mini",
    "api_key": "{{secrets.openai_api_key}}"
  }
}
```

#### Credential Resolution Order

1. **Per-resource** `api_key` / `{{secrets.name}}` — explicit key on a specific stage or extractor
2. **Organization default** — set once via `default_llm_credentials`, applied everywhere
3. **Mixpeek platform keys** — used when no custom key is configured (usage charged to your Mixpeek account)

> [!NOTE]
>
> Values in `default_llm_credentials` are **secret names**, not raw API keys. Keys are encrypted at rest, never returned in API responses, decrypted on-demand per LLM call, and isolated per organization with no cross-tenant leakage.

## Document Access Control

Apply row-level security to documents with ACL rules. Filter results by user roles or attributes at query time without changing retriever logic. User-scoped keys (`usr_sk_`) transparently filter every read so each end-user sees only their documents — no app-side filtering.

For the full model (the `_acl` object, key types, public documents, sharing), see **[Document-Level ACL](/docs/operations/document-acl)**. For external policy engines, see [Permissions (OpenFGA)](/docs/platform/permissions). Broader auth/tenancy: [Security & Tenancy](/docs/operations/security).

## Webhooks

Subscribe to events like `collection.documents.written`, `document.created`, or `alert.triggered`:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/webhooks" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_name": "ingest-notify",
    "event_types": ["collection.documents.written", "documents.updated.batch"],
    "channels": [
      { "channel": "webhook", "configs": { "url": "https://example.com/webhook" } }
    ]
  }'
```

Webhooks are organization-scoped, so this route takes no `X-Namespace` header.

[Webhook API →](/docs/api-reference/webhooks/create-webhook) · [Full webhooks guide →](/docs/operations/webhooks)

## Manifests

Declare your entire namespace configuration as code — buckets, collections, retrievers, taxonomies — and apply it in one request:

```bash theme={null}
# Export current state
curl "https://api.mixpeek.com/v1/manifest/export" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"

# Apply a manifest
curl -X POST "https://api.mixpeek.com/v1/manifest/apply" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d @manifest.json
```

Use `POST /v1/manifest/diff` to preview changes before applying.

[Manifest API →](/docs/api-reference/manifest/apply-manifest)

## Lineage & Audit Traces

Every document links back to its source, and every retriever execution produces an auditable trace.

### Document Lineage

Track how a document was created — which object it came from, which collection processed it, and what features were extracted:

```bash theme={null}
curl "https://api.mixpeek.com/v1/documents/$DOCUMENT_ID/lineage" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

Returns the full decomposition tree: source object → batch → extracted documents. Use this to trace any search result back to the original file.

```bash theme={null}
# Get all documents derived from a source object
curl "https://api.mixpeek.com/v1/objects/$OBJECT_ID/derived" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

[Lineage API →](/docs/api-reference/document-lineage/get-document-lineage)

### Retriever Execution Traces

Every retriever execution captures a full trace — which stages ran, what scores were produced, which documents were dropped and why:

```bash theme={null}
curl "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/executions/$EXECUTION_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

A trace includes:

* Which retriever version and config were used
* Each stage's input parameters, output set, scores, and latency
* Which feature URIs and collections were consulted
* Which filters matched and which documents were eliminated
* The final result set with per-document provenance

Traces are replayable — if a model version or taxonomy changes, you can compare results against historical executions.

[Execution API →](/docs/api-reference/retrievers/get-execution) · [Explain API →](/docs/api-reference/retrievers/explain-retriever-execution-plan)

## Environment Branching

Clone namespaces to create isolated environments for testing:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/$NS_ID/clone" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "namespace_name": "staging-branch" }'
```

Branched namespaces share no state with the source — safe for experimentation.

[Clone API →](/docs/api-reference/namespace-clone/clone-namespace)


# Pagination
Source: https://docs.mixpeek.com/docs/platform/pagination

How to walk a list endpoint to the end, which field to follow, and the two that are always null.

Follow `next_cursor`. Send it back as `cursor` on the next call, and stop when the response omits it.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections/list?limit=100" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id"

# then, for each subsequent page:
curl -X POST "https://api.mixpeek.com/v1/collections/list?limit=100&cursor=CURSOR_FROM_LAST_RESPONSE" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "X-Namespace: ns_your_namespace_id"
```

> [!WARNING]
>
> **`next_page` and `previous_page` are always `null`.** They are deprecated and no endpoint populates them.
>
> A loop that treats `next_page: null` as "last page" stops after the first page and reads as a completed walk. That is the failure this page exists to prevent: a namespace with 130 collections returning its first 100 and nothing signalling that 30 were missed.

## The response object

| Field | Follow it? |
| - | - |
| `next_cursor` | **Yes.** Send it back as `cursor`. Absent means you have reached the end |
| `total` | Only present when you ask for it with `include_total=true` |
| `page`, `page_size`, `total_pages` | Present alongside `total`, on the same condition |
| `next_page`, `previous_page` | Never. Always `null` |

## Request parameters

Every list endpoint accepts all of these as query parameters. Pick one style and keep to it.

| Parameter | Means |
| - | - |
| `cursor` | Where to resume. The value comes from the previous response's `next_cursor` |
| `next_cursor`, `after` | Aliases for `cursor`. If you send `cursor` too, `cursor` wins |
| `limit` | How many items to return |
| `page_size` | Alias for `limit` |
| `offset` | Skip this many items |
| `page` | Alias for `offset`, in pages rather than items |
| `include_total` | Defaults to `false`. Setting it adds a count, and 50 to 200ms |

> [!TIP]
>
> `next_cursor` and `after` are accepted as aliases specifically because the response calls the field `next_cursor`, so sending it straight back is the natural thing to do. An undeclared query parameter is dropped silently before any handler sees it, which is what made an earlier version of this loop run forever.

## Walking a list to the end

```python Python theme={null}
cursor = None
items = []

while True:
    params = {"limit": 100}
    if cursor:
        params["cursor"] = cursor
    page = client.collections.list(**params)

    items.extend(page["results"])
    cursor = page.get("pagination", {}).get("next_cursor")
    if not cursor:
        break
```

The termination condition is the absence of `next_cursor`, not `next_page`, and not an empty result set. A page can be short without being last.

## Related

> **[Filters](/docs/retrieval/filters)**
>
> Narrowing a list before you page through it.

> **[Rate limits](/docs/operations/rate-limits-quotas)**
>
> What a tight pagination loop counts against.


# Permissions
Source: https://docs.mixpeek.com/docs/platform/permissions

Authorize retrieval per end-user — with Mixpeek's built-in document ACL or your own external authorization system (OpenFGA).

## Overview

Permissions control which documents an end-user can see when they search. Mixpeek
enforces them **server-side at retrieval time**, so a user's query only ever
returns documents they are authorized to access — your application never has to
add per-user filters, and there is no way for a client to bypass them.

There are two ways to drive permissions, and they share the same enforcement
path:

> **[Built-in document ACL](/docs/platform/operations#document-access-control)**
>
> Mixpeek stores and enforces an `_acl` (owner / read / write / public) on each
> document, keyed on user-scoped API keys. Best when Mixpeek is your source of
> truth for who-can-see-what.

> **[External authorization (OpenFGA)](#external-authorization-openfga)**
>
> Keep your existing permissions in your own [OpenFGA](https://openfga.dev)
> deployment; Mixpeek queries it at retrieval time and filters results to match.
> Best when an external system already owns authorization.

Both are **opt-in** and **fail-closed**: a document with no resolvable grant is
excluded, never leaked. Namespaces that don't opt in behave exactly as before.

## Which should I use?

| | Built-in document ACL | External authorization (OpenFGA) |
| - | - | - |
| **Source of truth** | Mixpeek (`_acl` on each document) | Your OpenFGA store |
| **Grant model** | `read` list + `public` flag | Your full OpenFGA model (groups, roles, folder inheritance, …) |
| **Best for** | Apps where Mixpeek owns permissions | Apps where an external system already owns permissions |
| **Setup** | Create user-scoped keys; ACL set automatically | Run OpenFGA; opt the namespace in; map subjects |
| **Consistency** | Strong (enforced inline) | Strong (pull) or eventual (push) — you choose |

If you already manage authorization in OpenFGA (or a Zanzibar-style system), use
external authorization so you don't duplicate your permission model. Otherwise
the [built-in document ACL](/docs/platform/operations#document-access-control) is the simplest path.

## External authorization (OpenFGA)

When a namespace opts in, Mixpeek acts as a **relying party** on your OpenFGA: at
retrieval time it asks OpenFGA what the acting user can see and filters the
results accordingly. Your OpenFGA remains the single source of truth.

### How it works

1. **The acting subject** is the `principal_id` of the user-scoped API key
   executing the query, mapped to an OpenFGA subject — `user:<principal_id>` by
   default.
2. **Mixpeek queries OpenFGA** for that subject using one of the strategies below.
3. **Results are filtered** — unauthorized documents are removed before the
   response is returned, for both ad-hoc and saved retrievers.

Your OpenFGA model must represent each Mixpeek document as an object whose id is
the Mixpeek `document_id`. By default Mixpeek checks the relation `viewer` on the
object type `document`:

```
document:doc_abc123 # viewer @ user:alice      # alice can retrieve doc_abc123
document:doc_abc123 # viewer @ user:*          # the document is public
```

Group, role, and parent-folder inheritance all work — OpenFGA resolves them
server-side; Mixpeek only relays the decision.

### Enforcement modes

Different strategies trade consistency against scale. Pick one with `mode`:

| `mode` | Strategy | Use when |
| - | - | - |
| `auto` *(default)* | `ListObjects` → pre-filter when the accessible set is small (≤ `list_objects_max`), otherwise `BatchCheck` post-filter | You want sensible behavior at any size |
| `pull_list_objects` | Always `ListObjects` → `document_id` pre-filter | The accessible set per user is small (≤ \~1000) |
| `pull_batch_check` | Run the search, then `BatchCheck` the candidates and drop unauthorized | Users can access many documents |
| `push` | Sync OpenFGA grants into an indexed field and filter in-index | Lowest query latency; eventual consistency is acceptable |

> [!NOTE]
>
> Pre-filtering via `ListObjects` only scales to a small accessible set
> (\~1000 objects). For larger corpora, `BatchCheck` post-filtering **over-fetches
> by `over_fetch_factor` (default 2×)** from the vector store so the page stays
> full after unauthorized documents are dropped. `auto` switches between the two
> for you.

### Configuration

Opt a namespace in by setting `infrastructure.authorization`:

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/namespaces/ns_abc123 \
  -H "Authorization: Bearer mxp_sk_your-org-key" \
  -H "Content-Type: application/json" \
  -d '{
    "infrastructure": {
      "authorization": {
        "enabled": true,
        "provider": "openfga",
        "api_url": "https://openfga.your-company.com",
        "store_id": "01J0XEXAMPLESTORE",
        "relation": "viewer",
        "mode": "auto"
      }
    }
  }'
```

**enabled** `boolean` *required*

Master switch. `false` (default) means the namespace is unchanged — no
enforcement and no calls to OpenFGA.

**provider** `string` (default: openfga)

Authorization backend. `openfga` is the supported provider.

**api\_url** `string` *required*

Base URL of your OpenFGA HTTP API, reachable from Mixpeek.

**store\_id** `string` *required*

The OpenFGA store id holding your relationship tuples.

**model\_id** `string`

Authorization-model id. When omitted, the store's latest model is used.

**api\_token\_secret\_ref** `string`

Name of an [organization secret](/docs/operations/security) holding the bearer
token for your OpenFGA API. The token is resolved from the encrypted secrets
vault at request time and is never stored in your namespace config. This is
the required way to supply a token for any hosted (`https://`) OpenFGA
deployment — store the token once via the organization secrets API, then
reference it here by name.

**api\_token** `string` *deprecated*

Deprecated plaintext bearer token, accepted only for local development
(`http://` or `localhost`) where OpenFGA runs without auth. Rejected for a
hosted OpenFGA deployment — use `api_token_secret_ref` instead so the token
is never sent or stored in plaintext.

**object\_type** `string` (default: document)

The OpenFGA object type representing a Mixpeek document.

**relation** `string` (default: viewer)

The relation that grants read/retrieve access.

**mode** `string` (default: auto)

Enforcement strategy: `auto`, `pull_list_objects`, `pull_batch_check`, or `push`.

**over\_fetch\_factor** `integer` (default: 2)

Post-filter over-fetch multiplier (≥ 2 recommended) so a page stays full after
unauthorized documents are dropped.

### Setup

**1. Run OpenFGA and write your tuples**

Stand up OpenFGA (or use your existing deployment) and write `viewer` tuples
whose object ids are Mixpeek `document_id`s, e.g.
`document:doc_abc123 # viewer @ user:alice`. Use `user:*` for public documents.

**2. Create user-scoped keys per end-user**

Generate a `usr_sk_` key whose `principal_id` matches the OpenFGA user id
(e.g. `alice`). Create it with your org-scoped key via
[`POST /v1/organizations/users/{user_email}/api-keys`](/docs/api-reference/organization-api-keys/create-api-key).
There is no `key_type` field — setting `principal_id` on the request is what
makes the key user-scoped (the response key carries the `usr_sk_` prefix).

**3. Opt the namespace in**

PATCH the namespace with the `infrastructure.authorization` block above.

**4. Query normally**

Execute retrievers with each user's `usr_sk_` key — results are filtered to
what that user can see in OpenFGA. No query changes needed.

### Keeping permissions current (push mode)

In `push` mode, Mixpeek subscribes to your OpenFGA changelog and projects grants
onto an indexed field on each document, then filters in-index for the lowest
query latency. Grant and revocation changes propagate within the sync interval
(eventually consistent). For strict point-in-time consistency, use `auto` or
`pull_batch_check`, which evaluate against OpenFGA live on every query.

## Behavior and guarantees

* **Fail-closed.** If OpenFGA is unreachable, queries return the safe subset
  (fewer results), never unauthorized documents.
* **No caching leaks.** When authorization is active, the per-retriever result
  cache is bypassed so one user's authorized page can never be served to another.
* **Both retrieval paths.** Ad-hoc and saved retrievers enforce identically.
* **Opt-out is unchanged.** A namespace with `enabled: false` (or no
  `authorization` block) behaves exactly as before.

> [!WARNING]
>
> Permissions are enforced at the Mixpeek API layer. If you have direct access to
> the underlying vector store or database, filters are not applied. Always route
> end-user traffic through the Mixpeek API.

## Related

* [Document Access Control](/docs/platform/operations#document-access-control) — Mixpeek's built-in `_acl` permissions
* [Operate](/docs/platform/operations) — authentication, secrets, and namespace isolation
* [Filters](/docs/retrieval/filters) — manual query filters (permission filters are injected on top)
* [OpenFGA documentation](https://openfga.dev/docs) — modeling, tuples, and the Check / ListObjects / BatchCheck APIs


# Triggers
Source: https://docs.mixpeek.com/docs/platform/triggers

Schedule clustering, taxonomy enrichment, and batch reruns on cron or intervals

Triggers run platform actions automatically — re-cluster a collection nightly, re-enrich documents as new data lands, or rerun a batch on a schedule. Instead of calling an endpoint by hand, you define *what* to run and *when*, and Mixpeek executes it.

## Anatomy of a trigger

Every trigger combines an **action** (what to run) with a **schedule** (when to run it):

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/triggers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "action_type": "cluster",
    "action_config": { "cluster_id": "clust_abc123" },
    "trigger_type": "cron",
    "schedule_config": { "cron_expression": "0 2 * * *", "timezone": "UTC" },
    "description": "Daily clustering at 2am"
  }'
```

| Field | Description |
| - | - |
| `action_type` | What to run: `cluster`, `taxonomy_enrichment`, `batch_rerun`, `collection_trigger`, `retriever_execution`, `alert_evaluation`. |
| `action_config` | Action-specific config (e.g. `{ "cluster_id": "..." }`). |
| `trigger_type` | When to run: `cron`, `interval`, `conditional`. (`event` is accepted by the schema but currently rejected at create; see below.) |
| `schedule_config` | Schedule-specific config (cron expression, interval seconds, etc.). |
| `description` | Human-readable label. |
| `status` | `active` (default) or `paused`. |

## Schedule types

**Cron**

Run on a cron expression in a given timezone — best for fixed times ("every night at 2am").

```json theme={null}
{
  "trigger_type": "cron",
  "schedule_config": { "cron_expression": "0 2 * * *", "timezone": "UTC" }
}
```

The expression is standard 5-field cron (`minute hour day-of-month month day-of-week`).

**Interval**

Run every N seconds — best for "keep it fresh" loops. Set `start_immediately` to run once on creation.

```json theme={null}
{
  "trigger_type": "interval",
  "schedule_config": { "interval_seconds": 3600, "start_immediately": false }
}
```

**Conditional**

Run when a condition is met (e.g. data drift or volume past a threshold). Provide the condition details in `schedule_config`. Use cron or interval for time-based schedules.

Event-based triggers (fire in response to new documents or a completed batch, rather than a schedule) are not available yet: `trigger_type: "event"` is rejected at creation. Poll for the condition you need in the meantime, or use a short interval.

## Actions

### Cluster

Re-run a clustering definition on a schedule so groupings stay current as data grows.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/triggers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "action_type": "cluster",
    "action_config": {
      "cluster_id": "clust_abc123",
      "filters": { "status": "active" },
      "output_collection_ids": ["col_daily_clusters"]
    },
    "trigger_type": "cron",
    "schedule_config": { "cron_expression": "0 2 * * *", "timezone": "UTC" },
    "description": "Daily clustering of active items"
  }'
```

### Taxonomy enrichment

Re-classify a collection against a taxonomy. Use `incremental: true` to enrich only new documents — far cheaper than re-running the whole collection.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/triggers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "action_type": "taxonomy_enrichment",
    "action_config": {
      "collection_id": "col_inventory",
      "taxonomy_id": "tax_products",
      "incremental": true
    },
    "trigger_type": "interval",
    "schedule_config": { "interval_seconds": 3600 },
    "description": "Hourly incremental enrichment (only new docs)"
  }'
```

Enrich into a separate target collection on a nightly cron:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/triggers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "action_type": "taxonomy_enrichment",
    "action_config": {
      "collection_id": "col_inventory",
      "taxonomy_id": "tax_products",
      "target_collection_id": "col_enriched",
      "incremental": true,
      "batch_size": 500,
      "parallelism": 8
    },
    "trigger_type": "cron",
    "schedule_config": { "cron_expression": "0 3 * * *", "timezone": "UTC" },
    "description": "Nightly incremental enrichment to target collection"
  }'
```

> [!TIP]
>
> `incremental: true` only processes documents added since the last run, so it won't re-pay extraction/enrichment cost on documents already processed. Prefer it for recurring enrichment.

### Batch rerun & collection trigger

`batch_rerun` re-runs processing for a collection's documents; `collection_trigger` fires a collection's configured processing. Both take a `collection_id` in `action_config`.

### Alert evaluation

`alert_evaluation` evaluates one retriever alert on the trigger's cron or interval schedule. It takes an `alert_id` in `action_config`. While the trigger is active, the periodic alert evaluation skips that alert in this namespace, so it runs on this schedule only. See [Triggers vs Alerts](/docs/operations/triggers-vs-alerts).

## Manage triggers

| Operation | Endpoint |
| - | - |
| List | `POST /v1/triggers/list` |
| Get | `GET /v1/triggers/{trigger_id}` |
| Update | `PATCH /v1/triggers/{trigger_id}` |
| Run now | `POST /v1/triggers/{trigger_id}/execute` |
| Pause / resume | `POST /v1/triggers/{trigger_id}/pause` · `POST .../resume` |
| Run history | `GET /v1/triggers/{trigger_id}/history` |
| Delete | `DELETE /v1/triggers/{trigger_id}` |

Run a trigger on demand (outside its schedule) to test it:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/triggers/{trigger_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

Check recent runs and their outcomes:

```bash theme={null}
curl -sS "$MP_API_URL/v1/triggers/{trigger_id}/history" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

## Related

* [Clusters](/docs/enrichment/clusters) — define the clustering a `cluster` trigger reruns
* [Taxonomies](/docs/enrichment/taxonomies) — define the taxonomy a `taxonomy_enrichment` trigger applies
* [Alerts](/docs/enrichment/alerts) — get notified when content matches a condition
* [Storage syncs](/docs/tutorials/ingest-video-from-s3) — continuously ingest new files from object storage


# Tasks
Source: https://docs.mixpeek.com/docs/processing/tasks

Track asynchronous jobs across Mixpeek

Tasks provide a uniform way to monitor long-running operations (batch processing, clustering, taxonomy materialization, namespace migrations, etc.). Every task exposes a status from the shared `TaskStatusEnum`.

## TaskStatusEnum

```
PENDING → PROCESSING → COMPLETED
            ↘           ↘
            FAILED      COMPLETED_WITH_ERRORS
```

Additional values include `IN_PROGRESS`, `CANCELED`, `SKIPPED`, `UNKNOWN`, `DRAFT`, `ACTIVE`, `ARCHIVED`, and `SUSPENDED`. All async resources in Mixpeek adopt this enum, so your polling logic works everywhere.

> [!WARNING]
>
> **Terminal statuses are `COMPLETED`, `COMPLETED_WITH_ERRORS`, `FAILED`, and `CANCELED`.** A poller that waits only for `COMPLETED` will hang forever on a batch that finished with some failed items (`COMPLETED_WITH_ERRORS` — partial success). Always stop on any terminal status.

## Anatomy of a Task

```json theme={null}
{
  "task_id": "tsk_processing_123",
  "task_type": "api_buckets_batches_process",
  "status": "PROCESSING",
  "inputs": ["batch_xyz789"],
  "outputs": null,
  "additional_data": {
    "batch_id": "batch_xyz789",
    "bucket_id": "bkt_products",
    "job_id": "ray_job_123"
  },
  "error_message": null
}
```

* Cached in Redis for \~24 hours (fast lookup).
* Persisted in MongoDB for historical auditing.
* `additional_data` stores resource-specific details (e.g., Ray job IDs).

## Polling Strategy

**1. Poll the task**

Query `/v1/tasks/{task_id}` with exponential backoff (start at 1s, cap at 30s).

**2. Handle 404 gracefully**

After Redis TTL expires you may receive `404`; fall back to the underlying resource (batch, cluster, etc.).

**3. Switch to resource polling**

Use `/v1/buckets/{bucket_id}/batches/{batch_id}`, `/v1/clusters/{cluster_id}`, etc., for long-running operations.

Example hybrid poller:

```python theme={null}
while True:
    try:
        task = get_task(task_id)
    except NotFound:
        task = get_batch(bucket_id, batch_id)

    if task.status in ("COMPLETED", "COMPLETED_WITH_ERRORS"):
        break  # both are terminal; COMPLETED_WITH_ERRORS = partial success
    if task.status in ("FAILED", "CANCELED"):
        raise RuntimeError(task.error_message)
    time.sleep(delay)
    delay = min(delay * 1.5, 30)
```

## Webhooks & Notifications

* Engine emits webhook events (e.g., `collection.documents.written`) when tasks complete relevant work.
* The work scheduler dispatches those events to invalidate caches, update schemas, and notify external systems.
* Prefer webhooks for near-real-time updates instead of aggressive polling.

## Managing Tasks

* `GET /v1/tasks/{task_id}` – fetch the latest status.
* `POST /v1/tasks/list` – filter by type, status, namespace, or creation time.
* `POST /v1/tasks/{task_id}/kill` – request cancellation. Idempotent: it succeeds when the task is already finished or does not exist.

## Best Practices

1. **Store task IDs** returned by submit endpoints.
2. **Use exponential backoff** to avoid hammering the API.
3. **Respect terminal states** (`COMPLETED`, `COMPLETED_WITH_ERRORS`, `FAILED`, `CANCELED`) and surface errors to operators.
4. **Use webhooks** for side-effects like cache invalidation or notifications.
5. **Instrument monitoring**—task history in MongoDB plus webhook logs provide a full audit trail.

Tasks keep the asynchronous parts of Mixpeek manageable—treat them as durable receipts for every long-running job.


# Analytics
Source: https://docs.mixpeek.com/docs/relevance/analytics

Monitor retriever performance, identify slow queries, and get AI-powered tuning recommendations

Analytics gives you visibility into how your retrievers perform in production. Track latency, stage-level bottlenecks, cache efficiency, and get automated recommendations for improvement.

## Endpoints Overview

| Endpoint | Method | Returns |
| - | - | - |
| `/analytics/retrievers/{id}/performance` | GET | Latency percentiles (P50/P95/P99), query counts, trends |
| `/analytics/retrievers/{id}/stages` | GET | Per-stage execution times and document flow |
| `/analytics/retrievers/{id}/signals` | GET | Operational signals (cache hits, rerank scores, filter reduction) |
| `/analytics/retrievers/{id}/cache-performance` | GET | Cache hit/miss rates, latency savings |
| `/analytics/retrievers/{id}/slow-queries` | GET | Slowest queries with stage-level breakdown |
| `/analytics/retrievers/{id}/analyze-tuning` | POST | AI-powered parameter tuning recommendations |

## Performance Metrics

Get latency percentiles and query volume over time:

```bash cURL theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/performance?group_by=hour&start_date=2025-01-01T00:00:00Z" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

performance = client.analytics.retrievers.performance(
    retriever_id="ret_abc123",
    group_by="hour",
    start_date="2025-01-01T00:00:00Z"
)
```

**Query parameters:**

| Parameter | Type | Default | Description |
| - | - | - | - |
| `start_date` | datetime | — | Start of time range (UTC) |
| `end_date` | datetime | — | End of time range (UTC) |
| `group_by` | string | `"hour"` | Time grouping: `hour`, `day`, `week` |

**Response includes:** P50, P95, and P99 latency, query counts, result counts, and trends over the time range.

## Stage Breakdown

Understand which stages consume the most time:

```bash cURL theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/stages?hours=24" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
stages = client.analytics.retrievers.stages(
    retriever_id="ret_abc123",
    hours=24
)
```

**Query parameters:**

| Parameter | Type | Default | Description |
| - | - | - | - |
| `hours` | integer | `24` | Hours of history (1–720) |

**Response includes:** Per-stage execution time, document count entering/exiting each stage, and stage-level latency distributions. Use this to identify bottlenecks — a rerank stage processing 500 documents is slower than one processing 50.

## Slow Queries

Find the queries that take the longest to execute:

```bash cURL theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/slow-queries?limit=10&hours=24" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
slow = client.analytics.retrievers.slow_queries(
    retriever_id="ret_abc123",
    limit=10,
    hours=24
)
```

**Query parameters:**

| Parameter | Type | Default | Description |
| - | - | - | - |
| `limit` | integer | `10` | Number of slow queries to return |
| `hours` | integer | `24` | Hours of history (1–720) |

**Response includes:** Query text, total execution time, result count, and stage-by-stage breakdown for each slow query. Use this to find pathological queries that need optimization.

## Cache Performance

Monitor how effectively caching reduces latency:

```bash cURL theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/cache-performance?hours=24" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
cache = client.analytics.retrievers.cache_performance(
    retriever_id="ret_abc123",
    hours=24
)
```

**Response includes:** Hit/miss rates, average latency for cache hits vs full execution, and hourly trends. A low hit rate may indicate your queries are too diverse for caching, or that cache TTL needs adjustment.

## Retriever Signals

Get raw operational signals for debugging:

```bash cURL theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/signals?signal_type=rerank_scores&limit=100&hours=24" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
signals = client.analytics.retrievers.signals(
    retriever_id="ret_abc123",
    signal_type="rerank_scores",
    limit=100,
    hours=24
)
```

**Signal types:**

| Signal | Description |
| - | - |
| `cache_hit` | Query served from cache |
| `cache_miss` | Cache miss, full execution |
| `rerank_scores` | Score distribution from rerank stage |
| `filter_reduction` | How much the filter stage reduced the candidate set |
| `expansion_results` | Query expansion output |

## AI-Powered Tuning

Get automated recommendations for improving your retriever:

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/analyze-tuning?days=7" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
tuning = client.analytics.retrievers.analyze_tuning(
    retriever_id="ret_abc123",
    days=7
)
```

**Query parameters:**

| Parameter | Type | Default | Description |
| - | - | - | - |
| `days` | integer | `7` | Days of history to analyze (1–90) |

**Response includes:** Parameter suggestions (e.g., "reduce top\_k from 500 to 200"), cache optimization tips, and performance improvement estimates based on observed patterns.

## Identifying Relevance Issues

Use analytics to spot relevance problems:

| Symptom | Analytics Signal | Likely Cause | Action |
| - | - | - | - |
| High latency, normal results | Stage breakdown shows slow rerank | Too many candidates entering rerank | Reduce `top_k` or add a limit stage before rerank |
| Low click-through rate | Interaction signals show high skip rate | Poor ranking or irrelevant results | Check fusion strategy, consider learned fusion |
| Cache hit rate dropping | Cache performance shows increasing misses | Query diversity increasing or TTL too short | Adjust cache strategy, review in [caching best practices](/docs/best-practices/caching-strategies) |
| Inconsistent latency | Slow queries show specific patterns | Certain query types trigger expensive paths | Add pre-filters or query-specific optimization |

## Monitoring Cadence

| Frequency | Check | Tools |
| - | - | - |
| **Daily** | Slow queries, P95 latency | `/slow-queries`, `/performance` |
| **Weekly** | Stage breakdown, cache efficiency, interaction trends | `/stages`, `/cache-performance`, `/signals` |
| **Monthly** | AI tuning analysis, full evaluation run | `/analyze-tuning`, [Evaluations](/docs/retrieval/evaluations) |

## Related

* [Evaluations](/docs/retrieval/evaluations) — offline quality measurement
* [Benchmarks](/docs/retrieval/benchmarks) — historical session replay
* [Caching Strategies](/docs/best-practices/caching-strategies) — optimizing cache performance
* [Interaction Signals](/docs/retrieval/interactions) — capturing user behavior data


# Annotations
Source: https://docs.mixpeek.com/docs/relevance/annotations

Record human decisions on documents for review workflows, compliance, and model improvement

Annotations capture explicit human judgments on documents — approve, reject, defer, or any domain-specific label. Unlike [interaction signals](/docs/retrieval/interactions) which track implicit behavior (clicks, views, dwell time), annotations record deliberate decisions with optional confidence scores, reasoning, and structured payloads. They are the foundation for human-in-the-loop workflows where retriever results need expert review before action.

## When to Use Annotations

Annotations solve the problem of turning retriever output into verified decisions. Any workflow where a person reviews documents and records a judgment benefits from annotations:

| Use Case | Labels | Payload Example |
| - | - | - |
| Medical coding review | `approved`, `rejected`, `deferred` | `{"codes_approved": ["E11.40"], "raf_impact": 0.302}` |
| Brand infringement triage | `infringement`, `safe`, `needs_review` | `{"confidence_model": 0.91, "match_type": "logo"}` |
| Duplicate detection | `confirmed_dupe`, `false_positive` | `{"canonical_id": "doc_abc", "similarity": 0.97}` |
| Content moderation | `approved`, `flagged`, `removed` | `{"policy_violation": "copyright", "severity": "high"}` |
| Document classification | `correct`, `incorrect`, `ambiguous` | `{"predicted_class": "invoice", "true_class": "receipt"}` |

Annotations are domain-agnostic — labels are free-form strings, and the `payload` field accepts any structured JSON your workflow needs.

## Annotation Lifecycle

```
Retriever executes → Results returned → Human reviews → Annotation recorded → Audit trail preserved
```

Each annotation links back to a document and optionally to the retriever execution that surfaced it:

* **`document_id`** and **`collection_id`** — what was reviewed
* **`retriever_id`**, **`execution_id`**, **`stage_name`** — how the document was found (provenance)
* **`label`**, **`confidence`**, **`reasoning`** — the human decision
* **`payload`** — structured data specific to the workflow
* **`actor_id`** and **`actor_type`** — who made the decision (user, API key, or system)

All mutations emit [webhooks](/docs/operations/webhooks) (`annotation.created`, `annotation.updated`, `annotation.deleted`) and log to the [audit trail](/docs/api-reference/organization-audit/list-audit-logs).

## Create an Annotation

Record a decision after reviewing a document:

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/annotations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_2e7650fa254b",
    "collection_id": "col_clinical_notes",
    "label": "approved",
    "confidence": 0.95,
    "reasoning": "Note clearly documents peripheral neuropathy with supporting lab values.",
    "payload": {
      "codes_approved": ["E11.40", "E11.65"],
      "raf_impact": 0.420,
      "annual_revenue": 2522
    },
    "retriever_id": "ret_hcc_review",
    "execution_id": "exec_abc123"
  }'
```

```python Python theme={null}
from mixpeek import Mixpeek

mp = Mixpeek(api_key="API_KEY")

annotation = mp.annotations.create(
    document_id="doc_2e7650fa254b",
    collection_id="col_clinical_notes",
    label="approved",
    confidence=0.95,
    reasoning="Note clearly documents peripheral neuropathy with supporting lab values.",
    payload={
        "codes_approved": ["E11.40", "E11.65"],
        "raf_impact": 0.420,
        "annual_revenue": 2522,
    },
    retriever_id="ret_hcc_review",
    execution_id="exec_abc123",
    namespace="ns_vitae",
)
```

```javascript JavaScript theme={null}
import Mixpeek from "mixpeek";

const mp = new Mixpeek({ apiKey: "API_KEY" });

const annotation = await mp.annotations.create({
  documentId: "doc_2e7650fa254b",
  collectionId: "col_clinical_notes",
  label: "approved",
  confidence: 0.95,
  reasoning:
    "Note clearly documents peripheral neuropathy with supporting lab values.",
  payload: {
    codes_approved: ["E11.40", "E11.65"],
    raf_impact: 0.42,
    annual_revenue: 2522,
  },
  retrieverId: "ret_hcc_review",
  executionId: "exec_abc123",
  namespace: "ns_vitae",
});
```

### `payload` and `metadata`

Two free-form fields, with different jobs.

`payload` holds the structured content of the judgment: the codes approved, the score, whatever the workflow decides. `metadata` holds anything the caller wants carried alongside it, stored and returned as-is.

```json theme={null}
{
  "document_id": "doc_2e7650fa254b",
  "label": "approved",
  "payload": { "codes_approved": ["E11.40"], "raf_impact": 0.420 },
  "metadata": { "reviewer_team": "hcc-west", "queue": "priority" }
}
```

On create and on patch, `metadata` is mirrored onto the annotation's row in the [`mxp_document_annotations`](/docs/vector-store/system-collections) system collection, under `metadata` on that document. The mirror is best-effort: if it fails, the annotation write still succeeds, so treat the annotation record as the source of truth.

`PATCH /v1/annotations/{annotation_id}` replaces `metadata` whole rather than merging keys. Send the full object you want to keep. Omitting the field leaves the stored value untouched, and so does sending `null`; send `{}` to clear it.

## Query Annotations

List annotations with filters to build review queues or dashboards:

```bash cURL theme={null}
# All rejected annotations for a collection
curl -sS -X POST "$MP_API_URL/v1/annotations/list" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_id": "col_clinical_notes",
    "label": "rejected"
  }'

# All annotations on a specific document
curl -sS -X POST "$MP_API_URL/v1/annotations/list" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_2e7650fa254b"
  }'
```

```python Python theme={null}
# All rejected annotations
rejected = mp.annotations.list(
    collection_id="col_clinical_notes",
    label="rejected",
    namespace="ns_vitae",
)

# All annotations on a specific document
doc_annotations = mp.annotations.list(
    document_id="doc_2e7650fa254b",
    namespace="ns_vitae",
)
```

```javascript JavaScript theme={null}
// All rejected annotations
const rejected = await mp.annotations.list({
  collectionId: "col_clinical_notes",
  label: "rejected",
  namespace: "ns_vitae",
});

// All annotations on a specific document
const docAnnotations = await mp.annotations.list({
  documentId: "doc_2e7650fa254b",
  namespace: "ns_vitae",
});
```

Available filters: `document_id`, `collection_id`, `label`, `actor_id`, `retriever_id`, `as_of`. All filters are optional and can be combined.

### Read the Judgment Set at a Past Instant

Pass `as_of` to get the annotations as they stood at one moment. Annotations created
after that instant are excluded, and every annotation still in the set is rewound to
its state at that time using its revision chain.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/annotations/list" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_id": "col_clinical_notes",
    "as_of": "2026-08-01T00:00:00Z"
  }'
```

```python Python theme={null}
frozen = mp.annotations.list(
    collection_id="col_clinical_notes",
    as_of="2026-08-01T00:00:00Z",
    namespace="ns_vitae",
)
```

```javascript JavaScript theme={null}
const frozen = await mp.annotations.list({
  collectionId: "col_clinical_notes",
  asOf: "2026-08-01T00:00:00Z",
  namespace: "ns_vitae",
});
```

This is the handle for reproducible evaluation. An evaluation pins one `as_of` and
re-reads the identical judgment set weeks later, after reviewers have added and
revised decisions.

> [!WARNING]
>
> **The boundary is inclusive on both sides.** An annotation created at exactly
> `as_of` is returned, and an edit applied at exactly `as_of` is kept rather than
> rewound. To exclude a record created at a known timestamp, pass an `as_of` strictly
> earlier than it.

A rewound annotation reports the `version` it held at that instant, and its
`revisions` array contains only the revisions below that version. Comparing `version`
across two `as_of` reads tells you which decisions moved between them.

## Aggregate Stats

Get label distribution counts for dashboards and progress tracking:

```bash cURL theme={null}
# Stats across all annotations
curl -sS "$MP_API_URL/v1/annotations/stats" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"

# Stats for a specific collection
curl -sS "$MP_API_URL/v1/annotations/stats?collection_id=col_clinical_notes" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
stats = mp.annotations.stats(namespace="ns_vitae")
# {"total": 142, "by_label": {"approved": 89, "rejected": 31, "deferred": 22}}
```

```javascript JavaScript theme={null}
const stats = await mp.annotations.stats({ namespace: "ns_vitae" });
// {total: 142, byLabel: {approved: 89, rejected: 31, deferred: 22}}
```

## Update a Decision

When a review is revisited — for example, a deferred case gets a clinical consult and can now be approved:

```bash cURL theme={null}
curl -sS -X PATCH "$MP_API_URL/v1/annotations/ann_3cefcdaf7536a19a" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "label": "approved",
    "confidence": 0.88,
    "reasoning": "Clinical review completed — peripheral neuropathy confirmed."
  }'
```

```python Python theme={null}
updated = mp.annotations.update(
    annotation_id="ann_3cefcdaf7536a19a",
    label="approved",
    confidence=0.88,
    reasoning="Clinical review completed — peripheral neuropathy confirmed.",
    namespace="ns_vitae",
)
```

```javascript JavaScript theme={null}
const updated = await mp.annotations.update({
  annotationId: "ann_3cefcdaf7536a19a",
  label: "approved",
  confidence: 0.88,
  reasoning: "Clinical review completed — peripheral neuropathy confirmed.",
  namespace: "ns_vitae",
});
```

The audit trail records both the original and updated values, preserving the full decision history.

## Bulk Operations

Process up to 1000 creates, updates, and deletes in a single call. Each operation is independent — a failure in one does not roll back the others.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/annotations/bulk" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "create": [
      {"document_id": "doc_001", "collection_id": "col_notes", "label": "approved", "confidence": 0.95},
      {"document_id": "doc_002", "collection_id": "col_notes", "label": "approved", "confidence": 0.91},
      {"document_id": "doc_003", "collection_id": "col_notes", "label": "rejected", "reasoning": "Insufficient documentation"}
    ],
    "update": [
      {"annotation_id": "ann_abc123", "label": "approved", "confidence": 0.88}
    ],
    "delete": ["ann_def456", "ann_ghi789"]
  }'
```

```python Python theme={null}
result = mp.annotations.bulk(
    create=[
        {"document_id": "doc_001", "collection_id": "col_notes", "label": "approved", "confidence": 0.95},
        {"document_id": "doc_002", "collection_id": "col_notes", "label": "approved", "confidence": 0.91},
        {"document_id": "doc_003", "collection_id": "col_notes", "label": "rejected", "reasoning": "Insufficient documentation"},
    ],
    update=[
        {"annotation_id": "ann_abc123", "label": "approved", "confidence": 0.88},
    ],
    delete=["ann_def456", "ann_ghi789"],
    namespace="ns_vitae",
)
# result.created_count=3, result.updated_count=1, result.deleted_count=2
```

```javascript JavaScript theme={null}
const result = await mp.annotations.bulk({
  create: [
    { documentId: "doc_001", collectionId: "col_notes", label: "approved", confidence: 0.95 },
    { documentId: "doc_002", collectionId: "col_notes", label: "approved", confidence: 0.91 },
    { documentId: "doc_003", collectionId: "col_notes", label: "rejected", reasoning: "Insufficient documentation" },
  ],
  update: [
    { annotationId: "ann_abc123", label: "approved", confidence: 0.88 },
  ],
  delete: ["ann_def456", "ann_ghi789"],
  namespace: "ns_vitae",
});
// result.createdCount=3, result.updatedCount=1, result.deletedCount=2
```

The response includes per-operation results so you can identify and retry individual failures.

## Example: Medical Coding Review Workflow

A healthcare organization uses Mixpeek to surface HCC suspect conditions from clinical notes. Coders review each result and record their decision:

1. **Retriever** runs `agent_search` across clinical notes, returning suspect HCC conditions with supporting evidence.
2. **Review queue** — the application calls `POST /v1/annotations/list?label=deferred` to show unresolved cases.
3. **Coder annotates** — for each document, the coder selects a label and the app calls `POST /v1/annotations` with the decision, ICD-10 codes, and RAF impact.
4. **Dashboard** — `GET /v1/annotations/stats?collection_id=col_notes` powers a progress bar showing approved/rejected/deferred counts.
5. **Audit** — compliance officers query annotations by `actor_id` to review individual coder decisions. The `reasoning` field provides the justification trail required for CMS audits.

> [!NOTE]
>
> Annotations are stored independently from documents — they don't modify the underlying document data. This separation ensures that the original clinical record remains untouched while the review layer captures all human decisions.

## Best Practices

* **Use consistent labels** within a workflow. Pick a label vocabulary (e.g., `approved`, `rejected`, `deferred`) and stick with it — the stats endpoint groups by exact string match.
* **Include reasoning** for audit-sensitive workflows. The `reasoning` field is indexed and retrievable, making it valuable for compliance reviews and dispute resolution.
* **Link provenance** when annotating retriever results. Setting `retriever_id`, `execution_id`, and `stage_name` lets you trace exactly how the document was surfaced, which is critical for evaluating retriever quality.
* **Use payload for structured data** rather than encoding it in the label. Labels should be human-readable categories; domain-specific fields (codes, scores, amounts) belong in `payload`.
* **Listen to webhooks** for real-time updates. Subscribe to `annotation.created` and `annotation.updated` events to trigger downstream workflows (e.g., auto-submit approved records, escalate rejected ones).

## References

* [Create Annotation](/docs/api-reference/annotations/create-annotation)
* [List Annotations](/docs/api-reference/annotations/list-annotations)
* [Annotation Stats](/docs/api-reference/annotations/annotation-stats)
* [Update Annotation](/docs/api-reference/annotations/update-annotation)
* [Delete Annotation](/docs/api-reference/annotations/delete-annotation)
* [Bulk Annotations](/docs/api-reference/annotations/bulk-annotations)
* [Interaction Signals](/docs/retrieval/interactions) — implicit behavioral signals (complementary to annotations)
* [Webhooks](/docs/operations/webhooks) — subscribe to annotation lifecycle events


# Architecture
Source: https://docs.mixpeek.com/docs/relevance/architecture

How Mixpeek's two-tower design decouples ingestion from retrieval and learns from usage

Mixpeek follows a **two-tower architecture** — a well-known pattern in recommendation systems adapted for multimodal search.

![Two towers: document tower writes N representation spaces at ingest, query tower reads and fuses them at search time, interaction signals close the feedback loop](https://mintcdn.com/mixpeek/o18CfVBlvyb5Tvo5/assets/relevance/two-towers.svg?fit=max\&auto=format\&n=o18CfVBlvyb5Tvo5\&q=85\&s=d4d7244ad08b3393ae7511c8fab8130d)

## Document Tower (Ingestion)

Source files enter through a [bucket](/docs/platform/data-model#buckets), trigger one or more [collections](/docs/platform/data-model#collections), and pass through the Ray engine for feature extraction. Each collection produces a different representation — text embeddings, multimodal embeddings, metadata, taxonomy labels — all stored as named vectors on a single point in [MVS](/docs/vector-store/overview), the Mixpeek Vector Store.

Documents are encoded **once at ingest time**. Adding a new extractor or updating a taxonomy triggers a re-process on the bucket — documents get new representations without changing the ingestion path.

## Query Tower (Retrieval)

A query arrives, gets encoded, and passes through a [multi-stage retriever](/docs/retrieval/retrievers). The key stage is **feature search**, which runs a separate vector query per embedding space and fuses the results.

The fusion strategy determines how per-feature scores combine into a final ranking:

| Strategy | Behavior |
| - | - |
| `rrf` | Rank-based, no tuning needed |
| `weighted` | Manual weights you set |
| `learned` | Weights sampled from Beta distributions, updated by user behavior |

See [Fusion Strategies](/docs/relevance/fusion-strategies) for the full comparison.

## Closing the Loop

With [learned fusion](/docs/relevance/learned-fusion), the two towers aren't static — they're connected by a feedback loop:

1. **Results** are shown to users
2. **Interactions** (clicks, purchases, skips) are stored in the namespace's `mxp_retriever_interactions` collection and, when analytics is enabled, in the analytics store
3. **Thompson Sampling** aggregates interactions into Beta(α, β) distributions per feature — α counts positive signals, β counts non-engagement
4. **Sampled weights** are drawn from those distributions on each query, naturally balancing exploration and exploitation
5. Weights converge toward the optimal blend as interactions accumulate

The system handles cold start through [hierarchical fallback](/docs/relevance/learned-fusion#hierarchical-fallback): personal weights → demographic segment → global → uniform prior. With zero interactions, learned fusion behaves identically to RRF.

## What Makes This Different

Standard two-tower systems learn a single embedding space. Mixpeek's document tower fans out into **N representation spaces** (visual, audio, text, multimodal, metadata), and the query tower traverses them in sequence through multi-stage retrieval. Learning happens at the **fusion layer** — which spaces to weight — not inside the embeddings themselves.

This means you can add a new extractor, re-process your data, and the bandit will automatically discover whether the new feature improves results — without retraining any model.

## Related

* [Feedback Loop Tutorial](/docs/tutorials/feedback-loop) — step-by-step setup guide
* [Learned Fusion](/docs/relevance/learned-fusion) — Thompson Sampling algorithm details
* [Interaction Signals](/docs/retrieval/interactions) — which signals to capture and when
* [Fusion Strategies](/docs/relevance/fusion-strategies) — all 5 strategies compared


# Fusion Strategies
Source: https://docs.mixpeek.com/docs/relevance/fusion-strategies

How multiple search results are combined into a single ranked list using RRF, DBSF, Weighted, Max, or Learned fusion

When a [feature search](/docs/retrieval/stages/feature-search) stage queries multiple embedding indexes (e.g., text + image), it produces separate ranked lists that need to be merged. Fusion strategies determine how those lists become one.

![Comparison of RRF, Weighted, and Learned fusion strategies](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/relevance/fusion-strategies-comparison.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=0d47c4b60054bfb57ef4dc48f7ea7169)

## Strategy Reference

| Strategy | Formula | Configuration | Best For |
| - | - | - | - |
| `rrf` | `w / (k + rank)` | Optional `weight` per search (k=60) | General purpose, no tuning needed |
| `dbsf` | `w₁·norm₁ + w₂·norm₂` | Optional `weight` per search | Different score distributions, or one search should lead |
| `weighted` | `w₁·score₁ + w₂·score₂` | `weight` per search | Known feature importance |
| `max` | `max(score₁, score₂)` | None | Any single match is sufficient |
| `learned` | Thompson Sampling | Interaction data | Personalized, adaptive weights |

## Reciprocal Rank Fusion (RRF)

The default strategy. RRF ignores raw similarity scores and uses only rank position. This makes it robust when different features produce scores on different scales. Each search's `weight` (default 1.0) multiplies its rank contribution. With k=60 a weight shifts the ranking only slightly; use DBSF when one search should lead.

**Formula:**

```
score(doc) = Σ  1 / (k + rank_i(doc))
```

Where `k = 60` (constant that prevents top-ranked items from dominating) and `rank_i` is the document's position in the i-th feature's result list.

**Why it works:** A document ranked #1 by text search and #3 by image search gets a higher fused score than a document ranked #2 by both. The rank-based approach means you don't need to calibrate score ranges across features.

> [!WARNING]
>
> **RRF scores are small on purpose — read order, not magnitude.** Because the score
> is `1 / (k + rank)` with `k = 60`, the best possible result scores about
> `1 / (60 + 1) ≈ 0.0164` per search (a little higher when several searches rank the
> same document). A top score of `~0.017` is **expected and healthy** — it is a fused
> rank position, **not** a similarity or a quality percentage, so a `0.0167` top hit
> does not mean the search matched poorly. Rank documents by their **relative** order.
> If you need an absolute similarity value (e.g. `0.0`–`1.0`), use a single search
> without fusion, or a `weighted`/`max` strategy that preserves raw scores.

```json Configuration theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": "{{INPUT.query}}",
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 100
        }
      ],
      "fusion": "rrf",
      "final_top_k": 25
    }
  }
}
```

> [!TIP]
>
> RRF is the best default. Use it unless you have a specific reason to choose another strategy.

## Distribution-Based Score Fusion (DBSF)

DBSF normalizes scores from each feature into a common distribution before combining them. This handles cases where one feature produces scores in \[0.8, 0.99] and another in \[0.1, 0.6].

**How it works:**

1. For each feature, take the minimum and maximum score it returned
2. Normalize each score to \[0, 1]: `normalized = (score - min) / (max - min)`
3. Multiply by the search's `weight` (default 1.0) and sum across features

DBSF is useful when features have different score distributions, but the raw scores themselves carry meaningful information (unlike RRF which ignores scores entirely).

Because each search is normalized on its own, DBSF works across different embedding models. Set `weight` on each search to make one of them lead. For example, a transcript search at `1.0` and a frame search at `0.2` puts the best transcript match first, even though the two models score on different scales. `weighted` fusion is refused across embedding models because it sums raw scores.

```json Configuration theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": "{{INPUT.query}}",
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.query}}",
          "top_k": 100
        }
      ],
      "fusion": "dbsf",
      "final_top_k": 25
    }
  }
}
```

## Weighted Fusion

You manually assign a weight to each search feature. Scores are multiplied by their weight and summed. Use this when you know from domain expertise that one feature is more important than another.

**Formula:**

```
score(doc) = w₁ · score₁(doc) + w₂ · score₂(doc) + ...
```

Weights don't need to sum to 1 — they're relative importance indicators.

```json Configuration theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": "{{INPUT.query}}",
          "top_k": 100,
          "weight": 0.7
        },
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.image_url}}",
          "top_k": 100,
          "weight": 0.3
        }
      ],
      "fusion": "weighted",
      "final_top_k": 25
    }
  }
}
```

> [!WARNING]
>
> Weighted fusion is sensitive to score scale differences. If text scores are in \[0.8, 1.0] and image scores are in \[0.1, 0.5], a 50/50 weight split will still favor text. Consider DBSF or RRF if score ranges differ significantly.

## Max Fusion

Takes the maximum score across all features for each document. A document only needs to be a strong match on **one** feature to rank highly.

**Formula:**

```
score(doc) = max(score₁(doc), score₂(doc), ...)
```

Use this when any single strong match is sufficient — for example, a product that matches either the text description or the visual similarity should rank high.

```json Configuration theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": "{{INPUT.query}}",
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
          "query": "{{INPUT.image_url}}",
          "top_k": 100
        }
      ],
      "fusion": "max",
      "final_top_k": 25
    }
  }
}
```

## Learned Fusion

Learned fusion replaces static weights with weights that adapt automatically from user [interaction](/docs/retrieval/interactions) data. Under the hood, it uses Thompson Sampling with Beta distributions to balance exploration (trying different weight combinations) with exploitation (using what's known to work).

See [Learned Fusion](/docs/relevance/learned-fusion) for the full deep dive.

```json Configuration theme={null}
{
  "stage_name": "feature_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": "{{INPUT.query}}",
          "top_k": 100
        },
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": "{{INPUT.image_url}}",
          "top_k": 100
        }
      ],
      "fusion": "learned",
      "final_top_k": 25
    }
  }
}
```

## Choosing a Strategy

```
Do you have multiple search features?
├── No  → No fusion needed (single feature search)
└── Yes
    ├── Do you have interaction data (100+ signals)?
    │   ├── Yes → Use "learned" (adapts automatically)
    │   └── No
    │       ├── Do you know which features matter more?
    │       │   ├── Yes → Use "weighted"
    │       │   └── No
    │       │       ├── Do features have different score scales?
    │       │       │   ├── Yes → Use "dbsf"
    │       │       │   └── No → Use "rrf" (default)
    │       └── Is any single match sufficient?
    │           └── Yes → Use "max"
```

## Performance Comparison

| Strategy | Latency Overhead | Configuration Effort | Adapts Over Time | Best Quality Ceiling |
| - | :-: | :-: | :-: | :-: |
| RRF | \< 5ms | None | No | Good |
| DBSF | \< 5ms | None | No | Good |
| Weighted | \< 5ms | Manual tuning | No | Good (if well-tuned) |
| Max | \< 5ms | None | No | Moderate |
| Learned | \< 10ms | Interaction tracking | Yes | Best |

All strategies add minimal latency. The real difference is in quality: learned fusion converges toward optimal weights for your specific users and content, while static strategies require manual tuning or accept a one-size-fits-all approach.

## Related

* [Feature Search stage](/docs/retrieval/stages/feature-search) — where fusion is configured
* [Learned Fusion deep dive](/docs/relevance/learned-fusion) — Thompson Sampling explained
* [Interaction Signals](/docs/retrieval/interactions) — capturing the data that powers learned fusion
* [Evaluations](/docs/retrieval/evaluations) — measuring the impact of different fusion strategies


# Learned Fusion
Source: https://docs.mixpeek.com/docs/relevance/learned-fusion

How Thompson Sampling adapts fusion weights from user interactions to personalize search results

Learned fusion automatically discovers the optimal blend of embedding features for your users. Instead of manually setting weights (`text: 0.7, image: 0.3`), the system learns from interaction data which features produce results users engage with.

![Thompson Sampling: Beta distributions evolve from uniform to peaked as interactions accumulate](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/relevance/thompson-sampling-cycle.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=ac47429ed459437033adb07d0ec59c02)

## How It Works

Learned fusion uses **Thompson Sampling**, a well-studied algorithm for the multi-armed bandit problem. Here's how it applies to search fusion:

**1. Initialize with uniform priors**

Each search feature (e.g., text embeddings, image embeddings) starts with a Beta(1, 1) distribution — a flat line that assigns equal probability to all weight values. This means zero assumptions about which feature is better.

**2. Sample weights at query time**

When a query arrives, the system draws a random weight from each feature's Beta distribution and normalizes them to sum to 1. Early on, samples are highly variable (exploration). As data accumulates, they stabilize (exploitation).

**3. Execute search with sampled weights**

The feature search stage runs each embedding search and fuses results using the sampled weights — functionally identical to weighted fusion, but with dynamically chosen weights.

**4. Capture user interactions**

Users interact with results: clicks, purchases, skips. Each interaction is recorded with the document ID, position, and the context key that identifies which weight sample was used.

**5. Update Beta distributions**

Positive interactions (clicks, purchases) increment the `alpha` parameter: `alpha = 1 + clicks`. Non-engagement increments `beta`: `beta = 1 + (impressions - clicks)`. This shifts the distribution toward weights that produce engaging results.

**6. Repeat with better weights**

Next query: the updated distributions produce weight samples closer to what works. After hundreds of interactions, the system converges on near-optimal weights while still occasionally exploring alternatives.

## What Learned Fusion Optimizes

Learned fusion learns one weight per feature. Each feature URI is an arm in the bandit, and its state is a Beta distribution for that feature. The system keeps no state per document.

A click updates feature weights, and every document is then scored from its rank in each feature under the new weights. A document that ranks low in every feature cannot be lifted by clicks.

To change how individual documents rank, add a [Rerank stage](/docs/retrieval/stages/rerank), which re-scores each document against the query.

## Thompson Sampling Explained

Think of it like flipping weighted coins. Each feature has its own coin:

* **At the start**, both coins are fair — you have no idea which feature is better, so you flip both and take whatever comes up.
* **After 50 interactions**, the text feature's coin lands "heads" 65% of the time (users click on text-matched results more). You naturally start weighting text higher, but still try image sometimes.
* **After 1000 interactions**, the text coin lands heads 72% of the time with very little variance. You're confident in the weights and rarely deviate.

The mathematical version: each "coin" is a Beta(alpha, beta) distribution where alpha counts successes (clicks) and beta counts non-successes (impressions without clicks). Sampling from this distribution gives you a weight that naturally balances exploration and exploitation.

## Hierarchical Fallback

Not every user has enough interaction history for personalized weights. The system uses a four-level fallback:

| Level | Context | Min Interactions | When Used |
| - | - | :-: | - |
| **Personal** | Individual user | 5 | User has clicked/purchased enough for reliable weights |
| **Demographic** | User segment | 1 | User is new, but their segment has data |
| **Global** | All users | 1 | No segment data; uses aggregate behavior |
| **Prior** | Uniform | 0 | No interactions at all; falls back to equal weights |

The `user_id` in your interaction signals enables personal-level learning. The `demographic_features` config (e.g., `["INPUT.user_segment"]`) enables demographic-level learning.

## End-to-End Walkthrough

### 1. Create a retriever with learned fusion

```json Retriever Configuration theme={null}
{
  "retriever_name": "product-search-learned",
  "stages": [
    {
      "stage_name": "feature_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": "{{INPUT.query}}",
              "top_k": 100
            },
            {
              "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
              "query": "{{INPUT.query}}",
              "top_k": 100
            }
          ],
          "fusion": "learned",
          "final_top_k": 25
        }
      }
    }
  ]
}
```

### 2. Execute a search

```bash theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "inputs": {
      "query": "wireless earbuds noise canceling",
      "user_id": "user_456"
    }
  }'
```

With zero interactions, this behaves like RRF (uniform weights). The response includes an `execution_id` you'll use for interaction tracking.

### 3. Capture interactions

```bash theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/interactions" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "feature_id": "doc_product_789",
    "interaction_type": ["click", "purchase"],
    "position": 2,
    "metadata": {
      "query": "wireless earbuds noise canceling"
    },
    "user_id": "user_456",
    "session_id": "sess_abc"
  }'
```

### 4. Improved results over time

After 100+ interactions, the same search for `user_456` returns results with personalized fusion weights. If this user consistently engages with text-matched results over image-matched ones, the text feature weight increases for their queries.

### 5. Verify convergence

Use [analytics](/docs/relevance/analytics) to check how weights are evolving:

```bash theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/signals?signal_type=learned_weights&hours=168" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

## Response Metadata

When learned fusion is active, the execution response includes a `__learned_fusion__` object in each result's metadata. Use it to verify the system is working and debug weight evolution:

```json theme={null}
{
  "__learned_fusion__": {
    "context_level": "personal",
    "context_key": "a3f8b2c1d4e5f607",
    "sampled_weights": {
      "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1": 0.72,
      "mixpeek://image_extractor@v1/google_siglip_base_v1": 0.28
    },
    "feature_uris": [
      "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
      "mixpeek://image_extractor@v1/google_siglip_base_v1"
    ],
    "effective_exploration": 0.37,
    "circuit_breaker_triggered": false,
    "weight_resolution_ms": 12.5
  }
}
```

| Field | Description |
| - | - |
| `context_level` | Which fallback level was used: `personal`, `demographic`, `global`, or `none` (circuit breaker triggered, using uniform weights) |
| `context_key` | A truncated SHA-256 hash of the context values used to look up interaction history (e.g., `a3f8b2c1d4e5f607`) |
| `sampled_weights` | The actual weights used for this query, keyed by feature URI |
| `effective_exploration` | Current exploration multiplier after decay — lower means more exploitation |
| `circuit_breaker_triggered` | `true` if the weight lookup timed out and fell back to uniform weights |
| `weight_resolution_ms` | How long the weight lookup took in milliseconds |

> [!TIP]
>
> Check `context_level` to verify personalization is active. If you see `"none"` consistently, the user may not have enough interactions (check `min_interactions`) or the circuit breaker may be triggering because weight resolution is slow.

## Configuration Reference

**fusion** `string` *required*

Set to `"learned"` to enable Thompson Sampling fusion.

**searches\[].feature\_uri** `string` *required*

Each feature URI defines an "arm" in the bandit. The system learns a separate weight for each.

**user\_id** `string`

Passed at execution time. Enables personal-level weight learning. Without this, the system uses global weights only.

The Thompson Sampler uses these parameters, configurable in `learning_config`:

| Parameter | Default | Range | Description |
| - | - | - | - |
| `prior_alpha` | `1.0` | `>= 0.1` | Beta distribution alpha prior. Higher = initial belief that features are effective |
| `prior_beta` | `1.0` | `>= 0.1` | Beta distribution beta prior. Higher = initial belief that features are ineffective |
| `exploration_bonus` | `1.0` | `0.1–10.0` | Multiplier for distribution variance; >1 increases exploration |
| `min_interactions` | `5` | — | Minimum interactions before using personal context |

## When to Use Learned vs Static

| Scenario | Recommendation | Why |
| - | - | - |
| New product, no interaction data | `rrf` | No data to learn from; RRF is a strong default |
| Domain expert knows feature importance | `weighted` | Manual weights capture expert knowledge immediately |
| Diverse user base with different preferences | `learned` | Different users may benefit from different feature weights |
| A/B testing fusion approaches | `rrf` → `learned` | Start with baseline, measure improvement with evaluations |
| Single search feature | None needed | Fusion only applies when combining multiple features |

## Session-Level Adaptation

Learned fusion computes weight state from stored interactions. With analytics enabled it reads them from the analytics store. Without it, it reads the namespace's `mxp_retriever_interactions` collection and caches the aggregate for 60 seconds, so a new click can take up to a minute to reach the weights. For within-session adaptation -- where a user's first few clicks should influence their next search immediately -- the system uses a Redis session cache.

When a user interacts with a result, the interaction is written to both the interactions collection (durable) and a Redis session cache (ephemeral, 1-hour TTL). On the next search in the same session, the bandit merges the session cache entries into the stored Beta distributions before sampling:

```
1. Read base α/β from the interactions collection (persistent history)
2. Read session interactions from Redis (current session)
3. Merge: α += session_successes, β += session_failures
4. Sample weights from the merged posterior
```

This gives sub-50ms feedback within a session while the interactions collection handles long-term persistence. Pass `session_id` on both search and interaction requests to enable this:

```python theme={null}
results = client.retrievers.execute(
    retriever_id,
    inputs={
        "query": "running shoes",
        "user_id": "user_456",
        "session_id": "sess_abc",
    },
)
```

> [!NOTE]
>
> Without `session_id`, the system still learns from interactions -- it just won't reflect them until the next weight refresh. Session-level adaptation is optional but recommended for real-time UX.

## Temporal Decay

User preferences change over time. The system applies exponential decay to older interactions so recent behavior matters more:

```
decayed_reward = reward * (decay_factor ^ days_ago)
```

With the default `decay_factor: 0.995`, the decay curve looks like:

| Age | Retained Weight | Effect |
| - | :-: | - |
| 1 day | 99.5% | Essentially full strength |
| 30 days | 86% (`0.995^30`) | Still strong |
| 90 days | 64% (`0.995^90`) | Noticeably faded |
| 180 days | 41% (`0.995^180`) | Weak influence |
| 365 days | 16% (`0.995^365`) | Nearly gone |

Configure decay in the `learning_config`:

```json theme={null}
{
  "learning_config": {
    "decay_factor": 0.995,
    "decay_window_days": 365
  }
}
```

* **`decay_factor`** (default `0.995`) -- per-day multiplier. Set to `1.0` to disable decay entirely.
* **`decay_window_days`** (default `365`) -- interactions older than this are ignored completely, reducing query cost.

> [!WARNING]
>
> Setting `decay_factor` too low (e.g., `0.95`) causes rapid forgetting -- a week-old interaction retains only 70% of its weight. Use values between `0.99` and `0.999` for most use cases.

## Weight Clamping

Thompson Sampling can produce extreme weights that effectively silence a feature (e.g., `text: 0.99, image: 0.01`). Weight clamping prevents this by enforcing minimum and maximum bounds:

```json theme={null}
{
  "learning_config": {
    "min_weight": 0.05,
    "max_weight": 0.95
  }
}
```

After sampling from the Beta posteriors and normalizing, each weight is clamped to `[min_weight, max_weight]` and then re-normalized. This guarantees that every feature contributes at least `min_weight` to the final fusion, even for users with heavily skewed interaction histories.

**Why this matters:** Without clamping, a user who clicks only text results could end up with `image: 0.01` -- effectively removing image search from their experience. If their preferences shift later, recovery is slow because the silenced feature produces almost no impressions to learn from.

## Exploration Decay

The `exploration_bonus` parameter controls how much the bandit explores (tries different weight combinations) vs. exploits (uses what it has learned). With a static bonus, the bandit never fully settles on the best weights.

Exploration decay reduces the bonus as interactions accumulate:

```
effective_exploration = max(exploration_floor, exploration_bonus * exploration_decay ^ total_interactions)
```

Configure it in `learning_config`:

```json theme={null}
{
  "learning_config": {
    "exploration_bonus": 1.0,
    "exploration_decay": 0.99,
    "exploration_floor": 0.1
  }
}
```

* **`exploration_bonus`** (default `1.0`) -- initial exploration multiplier. Higher values mean more random early sampling.
* **`exploration_decay`** (default `0.99`) -- per-interaction decay rate.
* **`exploration_floor`** (default `0.1`) -- minimum exploration. The bandit never fully stops exploring -- this prevents it from getting permanently stuck on suboptimal weights if preferences change.

After 100 interactions: `1.0 * 0.99^100 = 0.37`. After 500: `1.0 * 0.99^500 = 0.007` (floored to `0.1`). The system converges toward exploitation while maintaining a baseline level of exploration.

## Multi-Signal Rewards

By default, learned fusion treats `click` as the only learning signal. The `reward_map` lets you assign different reward magnitudes to different interaction types:

```json theme={null}
{
  "learning_config": {
    "reward_map": {
      "click": 1.0,
      "purchase": 3.0,
      "add_to_cart": 2.0,
      "bookmark": 1.5,
      "positive_feedback": 2.0,
      "negative_feedback": -2.0,
      "skip": -1.0
    }
  }
}
```

Positive values increase the `alpha` parameter for the associated feature (making it more likely to be weighted higher). Negative values increase the `beta` parameter (penalizing the feature). A purchase at `3.0` shifts weights three times as much as a click at `1.0`.

Per-interaction rewards are also capped at `max_reward_per_interaction` (default `5.0`) to prevent a single buggy or malicious interaction batch from dominating the learned weights.

See the [Reward Signals reference](/docs/retrieval/reward-signals) for all 17 supported interaction types and guidance on choosing reward values.

## Related

* [Auto-Tune overview](/docs/retrieval/auto-tune) -- the top-level guide to the full feedback loop
* [Reward Signals](/docs/retrieval/reward-signals) -- configuring which interactions drive learning
* [Rollout Guide](/docs/retrieval/auto-tune-rollout) -- traffic splitting, shadow mode, kill switch
* [Fusion Strategies](/docs/relevance/fusion-strategies) -- comparison of all 5 strategies
* [Interaction Signals](/docs/retrieval/interactions) -- capturing the data that powers learning
* [Evaluations](/docs/retrieval/evaluations) -- measuring learned fusion quality
* [Feature Search stage](/docs/retrieval/stages/feature-search) -- where fusion is configured


# Best Practices
Source: https://docs.mixpeek.com/docs/resources/best-practices

Schema design, feature selection, caching, and cost optimization

## Schema Design

* **One bucket per data domain** (products, support tickets, footage). Keep schemas coarse — collections slice data differently downstream.
* **Separate mutable from immutable fields.** Put stable data (file URL, created date) in the bucket schema. Put changing data (status, tags) in metadata that can be patched.
* **Use `key_prefix`** in objects for organization (e.g., `/2025/04/`).
* **Enable document schema validation** on collections (`validation_mode: "strict"`) to catch malformed data early.

## Feature Selection

| Data Type | Recommended Extractor | Output |
| - | - | - |
| Text search | `text_extractor` | E5-Large 1024D embeddings |
| Image search | `multimodal_extractor` | Vertex AI 1408D embeddings |
| Face matching | `face_identity_extractor` | ArcFace 512D embeddings |
| Video scenes | `multimodal_extractor` | Scene embeddings + transcripts + keyframes |
| Documents/PDF | `universal_extractor` | Text chunks + OCR + embeddings |
| Audio | `multimodal_extractor` | Transcripts + audio embeddings |

* **Start with one extractor per collection.** Add more collections for additional features rather than overloading one.
* **Match extractors to your query patterns.** If users search by text, prioritize text embeddings. If they search by image, prioritize visual embeddings.
* **Test with a small batch first** before processing your full corpus.

## Caching

Mixpeek caches at two levels:

* **Retriever-level** — cache the full pipeline result. Set `cache_config.ttl_seconds` on the retriever.
* **Stage-level** — cache expensive stages (KNN search, reranking) independently. Useful when early stages are stable but later stages change.

```json theme={null}
{
  "cache_config": {
    "enabled": true,
    "ttl_seconds": 300
  }
}
```

* Collection index signatures auto-invalidate caches when documents change.
* Use shorter TTLs (60-300s) for frequently updated collections, longer (3600s+) for stable corpora.
* Monitor cache hit rates via the [analytics API](/docs/api-reference/retriever-evaluations/list-evaluations).

## Cost Optimization

Mixpeek prices in dollars: each feature you enable is billed per modality unit (images, video minutes, document pages, text tokens, crawled web pages). Each tier includes a monthly dollar usage pool, and overage bills at the rate card — see [Billing](/docs/platform/billing). Quote a workload before running it with `POST /v1/organizations/billing/estimate`. Key cost drivers:

| Operation | Cost Level | Optimization |
| - | - | - |
| Feature extraction | High | Choose the right extractor — don't over-extract |
| LLM enrichment stages | High | Set `max_tokens` limits, cache results |
| Vector search | Low | Reads are included with each tier (\$2 per 1M queries beyond included) — cheap; the heavy cost is enrichment/LLM stages. See [Rate Limits & Quotas](/docs/operations/rate-limits-quotas) |
| Storage | Low | Auto-tiered — hot/warm/cold based on access patterns |

**Top optimizations:**

1. **Deduplicate before ingesting** — skip objects already processed
2. **Use field passthrough** for metadata that doesn't need extraction
3. **Batch process** rather than single-object ingestion
4. **Cache retriever results** for repeated query patterns
5. **Set reranking limits** (`top_k`) to avoid scoring too many candidates


# Deployment Models
Source: https://docs.mixpeek.com/docs/resources/deployment-models

Three ways to run Mixpeek, and exactly what each party owns in every one

Mixpeek runs in three shapes. They differ in one dimension: how much of the stack
you own. The API, the SDKs and the primitives are identical across all three, and
code written against one runs unchanged against the others.

This page names who owns what, so a security review can answer its questions
without a call.

## The ownership ladder

| | Cloud account | Kubernetes | Nodes & storage | Mixpeek software plane | Who operates | You pay for |
| - | - | - | - | - | - | - |
| **Mixpeek Cloud** | Mixpeek | Mixpeek | Mixpeek | Mixpeek | Mixpeek | Usage |
| **Mixpeek Cloud, dedicated** | Mixpeek | Mixpeek | Mixpeek, yours alone | Mixpeek | Mixpeek | Usage + dedicated capacity |
| **Customer-hosted** | **You** | **You** | **You** | Mixpeek, in a namespace you grant | You operate the substrate, Mixpeek operates its software | Your cloud bill + licence |

There are two products behind those three rows. The third row is the one with a boundary worth
drawing, and it sits in exactly one place:

**You provide the substrate. Mixpeek provides and operates the software.**

| You own | Mixpeek owns |
| - | - |
| Cloud account or datacenter | MVS |
| Kubernetes | Ray |
| CPU and GPU nodes | Execution logic |
| Disks and storage classes | Python and CUDA environment |
| Object storage | Models |
| Networking | Serve, indexing, retrieval |
| IAM | The software lifecycle |
| Quotas and policies | |

**Both planes move together.** In customer-hosted Mixpeek the data plane and the
compute plane are in your environment: MVS shards on your disks, Ray on your
nodes, your object storage. That is the whole point, and it is why we do not offer
a shape that moves only one of them.

> [!NOTE]
>
> The test this architecture is built around: **can Mixpeek upgrade itself without
> asking your platform team to upgrade a dependency first?** Because we ship our own
> Ray, Python, CUDA and MVS as containers, mostly yes. Your Kubernetes version,
> storage capabilities and GPU drivers remain a compatibility boundary, but that is
> a much smaller and slower-moving interface than one where our release cadence
> depends on your runtime.

### The vector store, in every row

Every row above stores vectors in **Mixpeek Vector Store on object storage**.
That is what we run, in all four models. Your vectors live in object storage in
the row's data plane, not in a separate database you have to operate.

In customer-hosted Mixpeek, MVS runs **in your Kubernetes, on your disks**. It is
Mixpeek software, operated by Mixpeek, running on substrate you provide. Your
vectors do not leave your environment.

MVS is a separate service rather than a Ray workload, deliberately: owning the
storage layer end to end is what makes the learned index possible. That is why it
moves with the rest of the plane rather than being something you host separately.

Mixpeek Vector Store is the vector store in all three shapes. Bringing your own
vector database is not a deployment we offer, because owning the storage layer end
to end is what makes the learned index possible.

## What changes, and what does not

On both Mixpeek Cloud rows the control plane stays with Mixpeek: the API gateway,
auth, billing, and the routing that sends your requests to the right data plane.
What moves is where the work runs and whose hardware it runs on.

Customer-hosted is the row where that is a choice. The managed shape keeps the
control plane with Mixpeek and your namespace egresses to it; a fully self-hosted
install keeps it in your namespace and needs no outbound to us. The
[boundary section of the data-flow page](/docs/customer-hosted/data-flow#the-boundary-first)
gives the six collections involved and what the self-hosted shape gives up.

> [!NOTE]
>
> Your integration does not change between models. The same buckets, collections,
> extractors, retrievers, taxonomies and clusters, the same API keys, the same
> SDKs. Moving down the ladder changes your infrastructure obligations, not your
> code.

## BYO Ray, in more detail

BYO Ray is the row where the most is yours, so it is the row with the most to
say. You already run Ray or KubeRay. Mixpeek runs its indexing and inference
workloads on it rather than asking you to stand up a second one.

> [!NOTE]
>
> This section is a summary. [Bring Your Own Ray](/docs/customer-hosted/overview) has the full
> contract: [every permission we ask for](/docs/customer-hosted/security-rbac), [what the
> qualification suite checks](/docs/customer-hosted/qualification), and [what each failure
> means](/docs/customer-hosted/troubleshooting).

**What you grant.** One Kubernetes namespace, and two namespace-scoped Roles
inside it. Nothing of ours is cluster-scoped: no `ClusterRole`, no CRDs of ours,
no ability to create namespaces.

**The one exception, stated rather than buried.** KubeRay is cluster-scoped: it
ships 3 CustomResourceDefinitions and 5 ClusterRoles, and there is no
namespace-only way to install it. Most customers already run it, and where they
do not, their platform team installs it once with a Helm command we publish. If
you would rather we owned that too, the grant is
`customresourcedefinitions` + `clusterroles` + `clusterrolebindings`, and the
[qualification suite](/docs/customer-hosted/qualification) reports which side of that line you
are on. Both are supported; the difference is whether a Mixpeek release needing a
newer KubeRay waits on your change window.

**What stays yours.** Your node pools, quotas, taints, autoscaler policy and
network policy. Mixpeek's workloads schedule under your rules; they do not
change them.

**What we deploy.** Our engine image into that namespace, and ephemeral batch
jobs into the same namespace. A Ray cluster runs one image, so where we bring
our own cluster your Ray, Python and CUDA versions are not inputs to whether it
works.

**We bring our own Ray.** Mixpeek deploys a `RayService` into your namespace with
our image, our Serve applications and our worker groups, and submits batch as
`RayJob`s into the same namespace. Your Ray, Python and CUDA versions are not
inputs to whether it works, because a Ray cluster runs one image and that image is
ours.

> [!NOTE]
>
> **If you already run Ray**, we still bring our own rather than joining yours. That
> is deliberate. Running Mixpeek inside your Ray runtime would make your Ray, Python
> and CUDA versions a dependency of our release cadence, and would mean qualifying a
> new build per customer environment.
>
> Running Mixpeek on an existing customer `RayCluster` is possible and we have
> measured what it involves, but it is a compatibility exception rather than the
> supported shape. Ask if you need it.

### Serve isolation on a shared cluster

If you run Mode 2, Mixpeek's Serve applications sit beside yours on a cluster
you own. How we deploy them depends on how *yours* were deployed, and the
difference is worth understanding before you grant access.

Ray Serve's configuration API is declarative over the whole application set: a
config that does not name an application removes it. Whether that reaches your
applications depends on whether Ray Serve holds a config for them.

| How your app was deployed | Does Serve hold its config | A whole-config deploy |
| - | - | - |
| `serve.run(...)` in Python (imperative) | No | **Leaves it alone** |
| A config PUT / `serveConfigV2` (declarative) | Yes | **Removes it** |

You can tell which you have: an application with an empty `deployed_app_config`
in `GET /api/serve/applications/` was deployed imperatively, and sits outside the
lifecycle that config governs.

Mixpeek does not use the whole-config path on a cluster it does not own. We add,
scale and remove **one named application at a time**, which cannot reach another
application regardless of how yours were deployed. For customers whose Serve
apps are declaratively deployed, that is protection against a real and measured
hazard rather than a precaution.

> [!NOTE]
>
> Measured on Ray 2.53.0 with KubeRay 1.5.1, on a single-head cluster with one
> application of each kind. It is enough to state the asymmetry above. It is not a
> promise that a whole-config deploy is safe on your cluster, and we do not make
> one.

### Qualifying your cluster

Before anything is deployed, Mixpeek runs a qualification suite against your Ray
cluster and hands you the report. It checks reachability, the namespace-scoped
permissions, connectivity to your object storage and control-plane endpoints,
whether your node pools, taints and quotas remain authoritative when we schedule,
and whether our workloads leave your existing Ray applications, jobs and actors
untouched.

The report is a plain-text table of rows and verdicts. Rows that could not be
run say so rather than passing quietly, and a row that cannot establish something
reports what it could not determine rather than reporting an absence.

## Choosing a model

* **Mixpeek Cloud** if you want none of this. Most customers.
* **Mixpeek Cloud, dedicated** if you need isolated capacity but not custody.
* **Customer-hosted** if your data must stay in your environment, or your
  security posture requires that the software run on infrastructure you control.

Already running Ray or KubeRay is not by itself a reason to choose customer-hosted.
The reason is custody of the data and the substrate.

> [!NOTE]
>
> Moving between models is a deployment change, not a migration. Your namespaces,
> collections and retrievers are defined through the API and move with you.


# Single Tenant
Source: https://docs.mixpeek.com/docs/resources/single-tenant

Dedicated infrastructure with isolated compute, storage, and data — deploy on any cloud, in any region

Mixpeek's single-tenant deployment gives enterprise customers a fully isolated data plane: dedicated database, compute cluster, cache, object storage, and job queues. The shared control plane (API gateway, auth, billing) routes requests to your data plane transparently — your API keys and SDKs work the same way.

## Architecture

![Single-tenant architecture: shared control plane routing to isolated enterprise data plane and shared standard data plane](https://mintcdn.com/mixpeek/CEJla4pymd_l6V28/assets/mixpeek-single-tenant.svg?fit=max\&auto=format\&n=CEJla4pymd_l6V28\&q=85\&s=3afcd79606a676228d44607fd19b761e)

### What's isolated

| Resource | Isolation Level | Details |
| - | - | - |
| **Database** | Dedicated database | Separate MongoDB instance or database per tenant |
| **Compute** | Dedicated Ray cluster | Head node + autoscaling worker pools (CPU, batch, GPU) |
| **Cache** | Dedicated Redis | Separate instance with independent memory and connection pools |
| **Storage** | Dedicated bucket | Separate GCS/S3 bucket per tenant |
| **Job queues** | Dedicated queues | Work queues prefixed per tenant, so jobs never compete with other customers |
| **Vector store** | Dedicated shard | Isolated MVS shard with tenant-specific GCS-backed snapshots |

### What's shared

The control plane is stateless — it routes requests but holds no customer data:

* **API gateway** — resolves your API key to your data plane endpoints
* **Studio UI** — connects to the API, holds no data
* **Auth and API key management**
* **Billing and usage metering**
* **Container image registry** — same code, separate compute

## Cloud & Region Deployment

Mixpeek's single-tenant architecture supports deployment across cloud providers and regions. Each tenant's data plane is self-contained — all customer data stays in the region you choose.

### In-region co-location

Your data plane runs **within your cloud provider and region**. All traffic between your application and Mixpeek stays in-region — no cross-region or cross-cloud networking overhead. The only out-of-region hop is the initial API request through the control plane for auth and routing (\~1 RTT, no customer data persisted).

### Supported clouds and regions

| Cloud | Region | Location | Status |
| - | - | - | - |
| **GCP** | `us-east1` | South Carolina | Available |
| **GCP** | `eu-west1` | Belgium | On-demand |
| **AWS** | `us-east-1` | N. Virginia | On-demand |
| **AWS** | `eu-west-1` | Ireland | On-demand |
| **AWS** | `eu-west-3` | Paris | On-demand |
| **AWS** | `ap-south-1` | Mumbai | On-demand |

"On-demand" regions are provisioned when a customer commits. Lead time is approximately one week for the first tenant in a new region. Additional tenants in the same region deploy in hours.

Need a region not listed? Contact us — we can deploy to any GCP or AWS region.

### How it works

The control plane runs centrally and routes requests to your data plane via URL-based tenant configuration. Your `engine_url`, `mongo_uri`, `redis_url`, and `storage_bucket` all point to infrastructure in your chosen cloud and region.

```
┌─────────────────────────────┐
│   Control Plane (shared)     │
│   api.mixpeek.com            │
│   Auth · Routing · Billing   │
└──────┬──────────┬────────────┘
       │          │
  ┌────▼───┐  ┌──▼──────┐
  │ GCP    │  │ AWS     │
  │ GKE    │  │ EKS     │
  │ GCS    │  │ S3      │
  └────────┘  └─────────┘
```

When you onboard, you select a cloud provider and region. Mixpeek provisions your isolated data plane there — dedicated compute, storage, database, and cache. The control plane reaches your data plane over private networking (VPC peering or internal load balancers), never over the public internet.

> [!NOTE]
>
> Switching between cloud providers or regions after initial deployment requires a data migration. Choose your target cloud and region during onboarding.

### Node & resource selection

Your tenant's workloads can run on dedicated node pools with tenant-specific taints and labels. This gives you:

* **Hardware selection** — choose machine types per worker group (CPU-optimized, memory-optimized, GPU)
* **Spot/preemptible nodes** — reduce cost for batch-tolerant workloads
* **GPU acceleration** — dedicated GPU nodes (NVIDIA L4, A100) for video processing and large model inference
* **Isolation guarantees** — tenant taints ensure no other workloads land on your nodes

Node pool configuration is defined in your tenant overrides file:

```yaml theme={null}
node_pools:
  cpu-workers:
    machine_type: n2-highmem-8    # or r6i.2xlarge on AWS
    min_nodes: 1
    max_nodes: 5
    spot: true
  gpu-workers:
    machine_type: g2-standard-8   # or g5.2xlarge on AWS
    min_nodes: 0
    max_nodes: 4
    accelerator:
      type: nvidia-l4
      count: 1
```

## Tenant Routing

Every API request goes through tenant resolution:

1. Your API key authenticates against the shared auth layer
2. The API resolves your organization to a tenant configuration
3. The tenant config specifies your data plane endpoints (database, cache, compute, storage)
4. The request executes entirely within your isolated infrastructure

> [!NOTE]
>
> Tenant routing is transparent. Your API keys, SDKs, and integrations work identically to the shared platform — no code changes required.

## Compute Cluster

Your Ray cluster runs in a dedicated Kubernetes namespace with independent scaling.

### Worker groups

| Group | Default Range | Use Case |
| - | - | - |
| **CPU workers** | 1–4 nodes | Text embeddings, reranking, classification, image embeddings |
| **Batch workers** | 0–30 nodes | Large ingestion jobs (scale from zero on demand) |
| **GPU workers** | 0–8 nodes (NVIDIA L4) | Video processing, large model inference |

Each group autoscales independently. Batch and GPU workers can default to zero replicas and scale up when jobs arrive — you only pay for compute when it's active.

### Extractor scaling

Individual extractors (embedding models, classifiers, etc.) scale independently within your cluster:

* **`min_replicas`** — minimum always-running instances (0 = scale to zero when idle)
* **`max_replicas`** — maximum instances under load
* **`target_ongoing_requests`** — requests per replica before scaling up
* **`downscale_delay_s`** — cooldown before scaling down (prevents flapping)

> [!TIP]
>
> Set `min_replicas: 1` for latency-sensitive extractors (e.g., your primary embedding model for search). Use `min_replicas: 0` for batch-only extractors to save cost.

### Disabling extractors

If you don't use certain capabilities (e.g., audio embeddings, face recognition, web scraping), disable the corresponding extractors. This frees compute resources for the extractors you do use and reduces your always-on footprint.

## Self-Service Configuration

Enterprise tenants manage their cluster configuration via a YAML overrides file. On each platform deploy, Mixpeek merges your overrides with the latest extractor registry — new extractors appear automatically, disabled extractors stay disabled.

### What you can configure

| Section | Controls |
| - | - |
| **`auto_deploy`** | When `true`, platform updates on main auto-deploy to your cluster |
| **`disabled`** | List of extractors to exclude from your cluster |
| **`overrides`** | Per-extractor scaling (min/max replicas, resources, concurrency) |
| **`cluster`** | Worker group sizing (replicas, min/max nodes) |
| **`head`** | Head node resources (CPU, memory) |
| **`redis`** | Cache memory limits, persistence policy |
| **`mvs`** | Vector store shard config (WAL, snapshots, index parameters) |
| **`node_pools`** | Dedicated node pool machine types, autoscaling ranges, GPU config |
| **`env`** | Environment variable overrides |
| **`spec`** | Health check thresholds |

### Example overrides

```yaml theme={null}
# Auto-deploy platform updates
auto_deploy: true

# Disable extractors you don't need
disabled:
  - mixpeek__playwright          # no web scraping
  - laion__clap_htsat_tiny       # no audio embeddings
  - insightface__arcface         # no face recognition

# Scale extractors for your workload
overrides:
  intfloat__multilingual_e5_large_instruct:
    autoscaling_config:
      min_replicas: 2            # always warm for search
      max_replicas: 6            # burst for batch ingestion

# Size your worker groups
cluster:
  cpu-workers:
    minReplicas: 1
    maxReplicas: 4
  batch-workers:
    minReplicas: 0
    maxReplicas: 5               # more batch capacity
  gpu-workers:
    minReplicas: 0
    maxReplicas: 3
```

Changes take effect on the next deploy. When `auto_deploy: true`, every push to main automatically rebuilds and deploys to your cluster.

## Kubernetes Access

Enterprise tenants get operator-level access to their namespace:

| Action | Access |
| - | - |
| View pods, logs, events | Yes |
| Scale worker groups | Yes |
| Restart stuck pods | Yes |
| Port-forward to Ray dashboard | Yes |
| View Ray cluster status | Yes |
| Access secrets or RBAC | No |
| Modify other namespaces | No |

### Quick scaling

For immediate scaling (e.g., before a large batch job):

```bash theme={null}
# Scale batch workers to 3
kubectl -n <your-namespace> scale raycluster <your-cluster> \
  --replicas=3 --resource-name=batch-workers

# Scale GPU workers to 1 for video processing
kubectl -n <your-namespace> scale raycluster <your-cluster> \
  --replicas=1 --resource-name=gpu-workers

# Scale back down after the job
kubectl -n <your-namespace> scale raycluster <your-cluster> \
  --replicas=0 --resource-name=batch-workers
```

> [!NOTE]
>
> Manual scaling is temporary. The next deploy resets to the values in your overrides file.

## Monitoring

### Ray Dashboard

Port-forward to access your Ray dashboard locally:

```bash theme={null}
kubectl -n <your-namespace> port-forward svc/<your-head-svc> 8265:8265
# Open http://localhost:8265
```

The dashboard shows active deployments, replica counts, request queues, worker resource usage, and cluster utilization.

### Grafana

Each tenant gets scoped Grafana dashboards:

* **Queue Health** — depth and age of your job queues
* **Batch Status** — ingestion progress, success/failure rates
* **API Latency** — p50/p95/p99 for your requests
* **Cost & Usage** — compute hours, resource utilization
* **Error Rate** — 5xx errors scoped to your tenant

### kubectl

```bash theme={null}
# Pod resource usage
kubectl -n <your-namespace> top pods

# Recent events (scheduling failures, OOM, probe failures)
kubectl -n <your-namespace> get events --sort-by=.lastTimestamp | tail -20

# Ray Serve status (which extractors are running)
kubectl -n <your-namespace> exec -it deploy/<your-head> -- serve status
```

## Billing

Single-tenant billing has two components:

1. **Platform fee** — fixed monthly fee for access to the Mixpeek platform, API, Studio, and support
2. **Compute passthrough** — actual cloud infrastructure cost (nodes, storage, networking) passed through at cost plus a management markup

There are no per-operation usage charges on the single-tenant plan. You pay for the underlying cloud resources your cluster consumes, and Mixpeek handles provisioning, monitoring, upgrades, and support.

> [!TIP]
>
> Keep batch and GPU workers at `minReplicas: 0` — they scale from zero on demand. You only pay for compute when it's active. Disable unused extractors to reduce your always-on footprint.

## Troubleshooting

### Pods stuck in Pending

Check events for the pending pod:

```bash theme={null}
kubectl -n <your-namespace> describe pod <pod-name>
```

Common causes:

* **Insufficient resources** — cluster autoscaler is provisioning a new node (2-3 minutes)
* **GPU unavailable** — GPUs may be temporarily exhausted in the region
* **Resource limits** — reduce `maxReplicas` on other worker groups to free capacity

### OOMKilled pods

A pod exceeded its memory limit. Increase memory for the affected worker group in your overrides file:

```yaml theme={null}
cluster:
  cpu-workers:
    resources:
      limits:
        memory: "48Gi"
```

### Extractor returning 503

The extractor has no running replicas (scaled to zero) or all replicas are saturated:

* First request after idle takes 5-10 seconds for cold start
* Set `min_replicas: 1` for latency-sensitive extractors
* Increase `max_replicas` if you're seeing sustained 503s under load

### Batch jobs stuck

1. Check if batch workers are running: `kubectl -n <your-namespace> get pods -l ray.io/group=batch-workers`
2. If no batch workers, manually scale up: `kubectl -n <your-namespace> scale raycluster <your-cluster> --replicas=1 --resource-name=batch-workers`
3. Check queue depth in Grafana — a backlog is normal for large batches

## Getting Started

To provision a single-tenant data plane:

1. Contact your Mixpeek account manager or email [sales@mixpeek.com](mailto:sales@mixpeek.com)
2. Choose your cloud provider (GCP or AWS) and target region
3. We provision your isolated infrastructure (database, compute, cache, storage)
4. You receive kubectl access to your namespace and Grafana dashboards
5. Your existing API keys are routed to your dedicated data plane — no code changes

> [!NOTE]
>
> Migration from the shared platform to single-tenant needs no change on your side. Your data is copied to the isolated database, the tenant config is updated, and routing switches instantly. Rollback is equally fast.


# Troubleshooting
Source: https://docs.mixpeek.com/docs/resources/troubleshooting

Common errors, rate limits, and debugging tips

## API Error Format

All non-validation errors return a consistent envelope. The machine-readable
field is `type` (stable PascalCase), not a SCREAMING\_SNAKE `code`:

```json theme={null}
{
  "success": false,
  "status": 404,
  "error": {
    "message": "bucket not found",
    "type": "NotFoundError",
    "code": "optional_string",
    "details": {}
  }
}
```

Request-validation errors (`422`) use the same envelope. `details.validation_errors`
lists every field that failed, and the message names the first one:

```json theme={null}
{
  "success": false,
  "status": 422,
  "error": {
    "message": "Validation error: querry: 'querry' is not a field here; accepted: options, query",
    "type": "ValidationError",
    "details": {
      "validation_errors": [
        {
          "type": "extra_forbidden",
          "location": ["body", "querry"],
          "message": "Extra inputs are not permitted",
          "accepted_keys": ["options", "query"]
        }
      ]
    }
  }
}
```

## Unknown keys in a request body

Until 2026-11-02, a key that a request body does not declare is accepted and
ignored, and the response tells you about it. A JSON object response gets a
`warnings` entry per key, and every response carries the `X-Mixpeek-Unknown-Keys`
header listing them:

```json theme={null}
{
  "warnings": [
    {
      "resource": "request_body",
      "id": "querry",
      "reason": "'querry' is not a field of this request and was ignored. From 2026-11-02 a request carrying it is refused with 422.",
      "next_step": "Remove it. Accepted here: options, query"
    }
  ]
}
```

From 2026-11-02 the same request is refused with the `422` above, with
`"type": "extra_forbidden"`. The error's `accepted_keys` lists what the body
takes where you sent the key, so remove the key or rename it to one of those.
The official SDKs send only declared keys, so this affects hand-written requests.
Each request body in the [API reference](https://api.mixpeek.com/docs) carries
the date under `x-unknown-keys`.

## Common Errors

| Type | Status | Cause | Fix |
| - | - | - | - |
| `ValidationError` | 422 | Missing or invalid fields | Check required fields in the [API reference](https://api.mixpeek.com/docs) |
| `AuthenticationError` | 401 | Invalid, missing, or revoked API key | Verify the `Authorization: Bearer <key>` header |
| `UnauthorizedError` / `ForbiddenError` | 403 | Missing/wrong `X-Namespace`, or insufficient permissions | Check the `X-Namespace` header matches the resource you're calling |
| `PlanRequiredError` | 403 | Workspace has not picked a plan yet, so billable work is gated | [Choose a plan](https://studio.mixpeek.com/signup/plan). `GET /v1/organizations` returns `requires_plan: true` while gated — see [Billing](/docs/platform/billing) |
| `QuotaExceededError` | 403 | Plan usage/quota exceeded | See [Rate limits & quotas](/docs/operations/rate-limits-quotas) |
| `PaymentRequiredError` | 402 | Mixpeek's monthly spending stop is holding new paid work for every organization | Nothing to change on your side: it lifts when Mixpeek lifts it or on `details.lifts_on`. Search keeps working. See [Budgets](/docs/platform/budgets) |
| `NotFoundError` | 404 | Resource doesn't exist | Verify the ID and namespace |
| `TooManyRequestsError` | 429 | Too many requests | Back off and retry with exponential delay (respect `Retry-After`) |
| `DependencyUnavailable` | 503 | A store the request needed (database or cache) could not be reached. Nothing was changed. `details.resource` names it | Retry after the `Retry-After` header's seconds |
| `ProcessingError` | 500 | Processing error in engine | Check task details for the specific failure reason |

> [!NOTE]
>
> **401 vs 403 — they mean different things.** A **401 `AuthenticationError`** means the API key itself is bad (missing, malformed, or revoked). A **403** means the key is valid but the request isn't allowed: either the `X-Namespace` header is missing/doesn't match the resource (`UnauthorizedError`/`ForbiddenError`), you've exceeded a plan quota (`QuotaExceededError`), or the workspace has not chosen a plan yet (`PlanRequiredError`). A new workspace hits `PlanRequiredError` first: it blocks namespace creation and any billable work until you pick a plan. Match the `type` field in the response body — not just the status code — to the fix.

## Rate Limits

| Tier | Requests/min | Concurrent tasks |
| - | - | - |
| Free | 60 | 5 |
| Pro | 600 | 50 |
| Enterprise | Custom | Custom |

When you hit a 429, the response includes `Retry-After` header with seconds to wait.

## Debugging Checklist

### Objects not processing

1. Check batch status: `GET /v1/buckets/{bucket_id}/batches/{batch_id}`
2. Check task status: `GET /v1/tasks/{task_id}`
3. Verify the collection's `feature_extractor` matches the bucket schema's blob types
4. Check for failed documents: `GET /v1/buckets/{bucket_id}/batches/{batch_id}/failed-documents`

### A list returns an empty page

`GET /v1/collections`, `GET /v1/buckets`, and `GET /v1/retrievers` return `200` with an empty page when `X-Namespace` does not resolve for your key. So do the matching `POST .../list` routes. A new workspace's first call is not an error.

1. Read the `warnings` array in the response body. An entry with `"resource": "namespace"` and `"reason": "unresolved"` means the namespace did not resolve. Its `id` is the value you sent and its `next_step` says what to do.
2. The same condition sets the response header `X-Namespace-Unresolved: true`, for callers that read headers.
3. The entry and the header appear when the namespace does not exist and when it belongs to another organization. The two cases look the same.
4. List your namespaces with `GET /v1/namespaces`, then send one of those ids in `X-Namespace`.
5. An empty `warnings` array and no header mean the namespace resolved. The empty page is the true answer.

```json theme={null}
{
  "results": [],
  "warnings": [
    {
      "resource": "namespace",
      "id": "ns_does_not_exist",
      "reason": "unresolved",
      "next_step": "This namespace does not exist for your API key, so the empty list is not its contents. List your namespaces with GET /v1/namespaces and send one of their ids in X-Namespace."
    }
  ]
}
```

### Retriever returning zero results

Zero results almost always trace to one of these — check in order:

| Likely cause | How to confirm | Fix |
| - | - | - |
| **Documents aren't indexed yet** | `GET /v1/tasks/{task_id}` — did the batch reach `COMPLETED`? `POST /v1/collections/{id}/documents/list` — are there any docs? | Wait for the batch to reach `COMPLETED` / `COMPLETED_WITH_ERRORS` before querying |
| **Wrong `feature_uri`** (most common) | A mismatched URI matches nothing and returns 0 **silently** — feature\_search attaches a note to the response `warnings` array instead of erroring | `GET /v1/collections/{id}` → copy `vector_indexes[].feature_uri` exactly; see [Find your feature\_uri](/docs/processing/features#find-your-feature_uri-to-search) |
| **Filter or threshold too strict** | Temporarily remove `attribute_filter` stages and any `score_threshold` / `min_score`, then re-run | Loosen the filter or lower the threshold, then re-tighten |
| **Wrong `collection_identifiers`** | The retriever points at a collection that has no matching documents | Set `collection_identifiers` to the collection you actually ingested into |
| **Empty query input** | An empty `inputs` value yields an empty query embedding → empty results | Pass a non-empty query in `inputs` |

Then use the [explain endpoint](/docs/api-reference/retrievers/explain-retriever-execution-plan) to see the execution plan and per-stage candidate counts.

### Poor retrieval quality

1. Check if the right extractor is being used for your query type (text query → text embedding, image query → visual embedding)
2. Add a reranking stage to improve precision
3. Review the execution trace for score distributions
4. Consider adding more retriever stages (filters, MMR for diversity)

### Slow processing

1. Video processing time scales with duration — 1 min video ≈ 1-2 min processing
2. Use batch processing for bulk imports instead of single-object ingestion
3. Check for resource contention: `GET /v1/tasks?status=PROCESSING`

## FAQ

**Can I use multiple feature extractors on the same data?**
Yes — create multiple collections pointing to the same bucket, each with a different extractor.

**How do I re-process documents after changing a collection's extractor?**
Create a new batch with the same objects and submit it. New documents replace old ones.

**What file formats are supported?**
Video (MP4, MOV, AVI, WebM), Images (JPG, PNG, WebP, GIF), Audio (MP3, WAV, M4A, FLAC), Documents (PDF, DOCX, TXT, HTML).

**How do I delete all data in a namespace?**
Delete the namespace: `DELETE /v1/namespaces/{namespace_id}`. This removes all buckets, collections, retrievers, and documents.

**Is there a size limit for uploads?**
Default: 500MB per file. Enterprise plans support larger files. Use URL references for files already in cloud storage.

[Contact support →](https://mixpeek.com/contact)


# Auto-Tune
Source: https://docs.mixpeek.com/docs/retrieval/auto-tune

Self-improving relevance that adapts per user

Retriever fusion weights automatically adapt based on how each user interacts with search results. Instead of manually tuning `text: 0.7, image: 0.3`, the system learns from clicks, purchases, and feedback to discover the optimal blend for every user — using Thompson Sampling (a multi-armed bandit algorithm) with hierarchical fallback from personal to demographic to global priors.

Auto-Tune closes the gap between "search works" and "search works *for this user*" without building a separate recommendation system.

## How It Works

**1. User searches**

A query arrives with a `user_id`. The system looks up that user's learned fusion weights — or falls back to segment-level or global weights if the user is new.

**2. Results ranked by fusion weights**

The feature search stage runs each embedding search and fuses results using the personalized weights. Early on, weights are exploratory (high variance). As data accumulates, they stabilize around what works for this user.

**3. User interacts**

The user clicks, purchases, skips, or provides feedback. Each interaction is recorded with the document ID, position, and the feature URI that produced the match.

**4. Weights updated**

Positive interactions (clicks, purchases) increase the weight of the feature that surfaced the result. Negative signals (skips, negative feedback) decrease it. Different interaction types carry different reward magnitudes — a purchase is a stronger signal than a click.

**5. Next search reflects preferences**

The next query from this user samples from the updated weight distributions. After dozens of interactions, the system converges on near-optimal weights while still occasionally exploring alternatives.

![Thompson Sampling: Beta distributions evolve from uniform to peaked as interactions accumulate](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/relevance/thompson-sampling-cycle.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=ac47429ed459437033adb07d0ec59c02)

## Quick Start

### 1. Create a retriever with learned fusion

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product-search-personalized",
    "collection_identifiers": ["col_products"],
    "input_schema": {
      "query": { "type": "text", "description": "Search query", "required": true },
      "user_id": { "type": "text", "description": "User identifier", "required": true }
    },
    "stages": [
      {
        "stage_name": "Personalized Search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100
              },
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100
              }
            ],
            "fusion": "learned",
            "learning_config": {
              "context_features": ["INPUT.user_id"],
              "reward_map": {
                "click": 1.0,
                "purchase": 3.0,
                "add_to_cart": 2.0,
                "negative_feedback": -2.0
              },
              "min_interactions": 5,
              "exploration_bonus": 1.0,
              "decay_factor": 0.995
            },
            "final_top_k": 25
          }
        }
      }
    ]
  }'
```

```python Python SDK theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

retriever = client.retrievers.create(
    retriever_name="product-search-personalized",
    collection_identifiers=["col_products"],
    input_schema={
        "query": {"type": "text", "description": "Search query", "required": True},
        "user_id": {"type": "text", "description": "User identifier", "required": True},
    },
    stages=[{
        "stage_name": "Personalized Search",
        "stage_type": "filter",
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [
                    {
                        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                        "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
                        "top_k": 100,
                    },
                    {
                        "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                        "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
                        "top_k": 100,
                    },
                ],
                "fusion": "learned",
                "learning_config": {
                    "context_features": ["INPUT.user_id"],
                    "reward_map": {
                        "click": 1.0,
                        "purchase": 3.0,
                        "add_to_cart": 2.0,
                        "negative_feedback": -2.0,
                    },
                    "min_interactions": 5,
                    "exploration_bonus": 1.0,
                    "decay_factor": 0.995,
                },
                "final_top_k": 25,
            },
        },
    }],
)
retriever_id = retriever["retriever_id"]
```

### 2. Execute with a user ID

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "query": "wireless earbuds noise canceling",
      "user_id": "user_456"
    }
  }'
```

With zero interactions, this behaves like RRF (uniform weights). The response includes an `execution_id` and each result contains a `feature_id` and `feature_uri` — you'll use these in Step 3.

### 3. Post interactions — results automatically improve

When a user clicks, purchases, or otherwise engages with a result, post an interaction using the `feature_id` and `execution_id` from the search response:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/interactions" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_id": "'$RETRIEVER_ID'",
    "feature_id": "doc_product_789",
    "interaction_type": ["click", "purchase"],
    "position": 2,
    "user_id": "user_456",
    "session_id": "sess_abc",
    "execution_id": "exec_from_step_2",
    "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
  }'
```

| Field | Required | Description |
| - | - | - |
| `feature_id` | Yes | The `feature_id` from the search result the user interacted with. |
| `interaction_type` | Yes | Array of interaction types for this event (e.g. `["click"]` or `["click", "purchase"]`). Multiple types in one call record a compound action. |
| `position` | Yes | 0-indexed position of the result in the ranked list. Recorded for analytics and rank-aware metrics (NDCG). |
| `execution_id` | No | The `execution_id` from the search response. Links the interaction to a specific search. |
| `session_id` | No | Client-generated session identifier. Groups interactions within a browsing session. |
| `feature_uri` | No | The `feature_uri` that produced the result. When provided, the system learns which embedding features the user prefers. |

**Python SDK shortcut** — `create_interaction_from_result()` extracts all the IDs for you:

```python theme={null}
results = client.retrievers.execute(retriever_id, inputs={"query": "earbuds", "user_id": "user_456"})
client.retrievers.create_interaction_from_result(results, position=2, interaction_type=["purchase"], user_id="user_456")
```

After enough interactions, the same search for `user_456` returns results personalized to their feature preferences. If this user consistently engages with text-matched results over image-matched ones, the text feature weight increases for their queries.

## Key Concepts

> **[Reward Signals](/docs/retrieval/reward-signals)**
>
> Configure how different interaction types (clicks, purchases, feedback) influence learned fusion weights. Customize the reward map, handle negative signals, and tune temporal decay.

> **[Rollout & Safety](/docs/retrieval/auto-tune-rollout)**
>
> Safely deploy learned fusion with traffic splitting, shadow mode, kill switches, per-user opt-out, and weight bounds. Includes a recommended rollout plan.

> **[Interactions](/docs/retrieval/interactions)**
>
> How to capture clicks, purchases, and feedback that power the learning loop.

> **[Evaluations](/docs/retrieval/evaluations)**
>
> Measure whether learned fusion actually improves retrieval quality. Compare NDCG, precision, and recall against static fusion baselines.

## Configuration Reference

The `learning_config` object is set inside the feature search stage parameters alongside `fusion: "learned"`:

| Field | Type | Default | Description |
| - | - | - | - |
| `context_features` | `string[]` | `["INPUT.user_id"]` | Input fields used for personal-level weight learning. Each value references an `INPUT.*` field from the retriever's `input_schema`. |
| `demographic_features` | `string[]` | `[]` | Input fields for segment-level fallback (e.g., `"INPUT.user_segment"`). Used when a user has insufficient personal history. |
| `reward_signal` | `string` | `"click"` | *Deprecated.* Single interaction type used as the learning signal. Use `reward_map` instead. If both are provided, `reward_map` takes precedence and `reward_signal` is ignored. |
| `reward_map` | `object` | See [Reward Signals](/docs/retrieval/reward-signals) | Maps interaction types to reward magnitudes. Positive values reinforce; negative values penalize. |
| `min_interactions` | `integer` | `5` | Minimum interactions before using personal-level weights. Below this threshold, the system falls back to demographic or global weights. |
| `exploration_bonus` | `float` | `1.0` | Initial multiplier for distribution variance. Higher values increase exploration (more weight variability). |
| `exploration_decay` | `float` | `0.99` | Per-interaction decay applied to the exploration bonus. Gradually shifts from exploration to exploitation. |
| `exploration_floor` | `float` | `0.1` | Minimum exploration bonus. Prevents the system from fully exploiting — there is always some chance of trying alternative weights. |
| `decay_factor` | `float` | `0.995` | Per-day exponential decay applied to older interactions. `1.0` disables decay (interactions never fade). |
| `decay_window_days` | `integer` | `365` | Interactions older than this are ignored entirely. |
| `min_weight` | `float` | `0.05` | Minimum weight any feature can receive after sampling. Prevents a feature from being silenced. |
| `max_weight` | `float` | `0.95` | Maximum weight any feature can receive after sampling. Prevents one feature from completely dominating. |
| `rollout_pct` | `float` | `100.0` | Percentage of requests (0-100) that use learned weights. The rest use static fusion. Uses deterministic bucketing so a user does not flip-flop between treatments. |
| `shadow_mode` | `boolean` | `false` | When `true`, learned weights are computed and logged but static fusion results are served. Use this to evaluate learned fusion before going live. |
| `algorithm` | `string` | `"thompson_sampling"` | Learning algorithm. Currently only `"thompson_sampling"` is supported. |
| `prior_alpha` | `float` | `1.0` | Alpha parameter for the Beta prior distribution. Higher values bias toward exploitation. |
| `prior_beta` | `float` | `1.0` | Beta parameter for the Beta prior distribution. `alpha=1, beta=1` is a uniform prior. |
| `max_reward_per_interaction` | `float` | `5.0` | Maximum absolute reward value per interaction. Rewards beyond this are clamped. |
| `circuit_breaker_timeout_ms` | `integer` | `1000` | Timeout in milliseconds for learned weight resolution. If exceeded, falls back to static fusion. Range: 1–30000. |
| `fallback_strategy` | `string` | `"hierarchical"` | How to resolve weights when personal data is insufficient. `"hierarchical"` uses the four-level fallback below; `"global"` skips demographic and goes straight to global. |

> [!NOTE]
>
> **Every `learning_config` field is optional** — each has the default shown above. The only field you almost always set is `context_features` (so the system knows which `INPUT.*` field identifies a user). The minimal config in the [quickstart](#1-create-a-retriever-with-learned-fusion) (`context_features` + `reward_map` + `min_interactions` + `exploration_bonus`) is enough to get started; the rest are tuning knobs.

### Complete `learning_config` example

Every documented field, set to its default. Drop this into the feature search stage's `parameters` (alongside `fusion: "learned"`) and adjust only what you need — omitted fields fall back to these defaults:

```json theme={null}
"learning_config": {
  "context_features": ["INPUT.user_id"],
  "demographic_features": ["INPUT.user_segment"],
  "reward_map": {
    "click": 1.0,
    "add_to_cart": 2.0,
    "purchase": 3.0,
    "negative_feedback": -2.0
  },
  "min_interactions": 5,
  "exploration_bonus": 1.0,
  "exploration_decay": 0.99,
  "exploration_floor": 0.1,
  "decay_factor": 0.995,
  "decay_window_days": 365,
  "min_weight": 0.05,
  "max_weight": 0.95,
  "rollout_pct": 100.0,
  "shadow_mode": false,
  "algorithm": "thompson_sampling",
  "prior_alpha": 1.0,
  "prior_beta": 1.0,
  "max_reward_per_interaction": 5.0,
  "circuit_breaker_timeout_ms": 1000,
  "fallback_strategy": "hierarchical"
}
```

> [!WARNING]
>
> `min_weight` must be strictly less than `max_weight` (the API rejects the retriever otherwise), and `reward_map` keys must be valid [interaction types](/docs/retrieval/interactions#signal-types).

## Hierarchical Fallback

Not every user has enough interaction history for personalized weights. The system uses a four-level fallback:

| Level | Context | Min Interactions | When Used |
| - | - | :-: | - |
| **Personal** | Individual user | Configurable (default 5) | User has enough interactions for reliable personal weights |
| **Demographic** | User segment | 1 | User is new, but their segment (e.g., "enterprise", "consumer") has data |
| **Global** | All users | 1 | No segment data available; uses aggregate behavior across all users |
| **Prior** | Uniform | 0 | No interactions at all; falls back to equal weights (equivalent to RRF) |

The `context_features` field controls personal-level resolution (typically `["INPUT.user_id"]`). The `demographic_features` field enables segment-level fallback (e.g., `["INPUT.plan_tier"]`).

> [!TIP]
>
> A new user starts at the Global or Prior level and automatically graduates to Personal as they interact with results — no configuration changes needed. The transition happens transparently at query time.

## Related

* [Interactions](/docs/retrieval/interactions) — capturing the user behavior that powers learning
* [Feature Search stage](/docs/retrieval/stages/feature-search) — where fusion and `learning_config` are configured
* [Fusion Strategies](/docs/relevance/fusion-strategies) — comparison of all 5 fusion strategies


# Rollout & Safety
Source: https://docs.mixpeek.com/docs/retrieval/auto-tune-rollout

Safely deploy learned fusion with traffic splitting and shadow mode, and turn it off with one config change

Learned fusion changes how search results are ranked, per user and per query. That power requires operational controls. This guide covers how to roll out Auto-Tune safely: start with shadow mode, ramp traffic gradually, monitor, and know how to turn it off if something goes wrong.

## Traffic Splitting

The `rollout_pct` field controls what percentage of requests use learned fusion weights. The rest fall back to static fusion (RRF by default).

```json theme={null}
{
  "learning_config": {
    "rollout_pct": 10.0
  }
}
```

Bucketing is **deterministic by user ID**: the same user always gets the same treatment (learned or static) on consecutive requests. This prevents a user from seeing different ranking behavior on every search.

```
rollout_pct: 0    → all requests use static fusion
rollout_pct: 10   → ~10% of users get learned fusion
rollout_pct: 50   → ~50% of users get learned fusion
rollout_pct: 100  → all requests use learned fusion (default)
```

> [!NOTE]
>
> Bucketing uses a hash of the `user_id`. If no `user_id` is provided in the query, the request always uses static fusion regardless of `rollout_pct`.

### Changing the rollout

`rollout_pct` lives in the feature search stage's `learning_config`, so you change it by updating the retriever. Send the retriever's `stages` with that stage's `learning_config` edited; the next request uses the new value, with no redeploy.

```bash theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "stages": [
      {
        "stage_name": "search",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "fusion": "learned",
            "learning_config": { "rollout_pct": 25 }
          }
        }
      }
    ]
  }'
```

Include the stage's other parameters (its searches, limits and the rest of `learning_config`) as they are today: the stage is replaced by what you send. In the Studio, the **Learning tab** rollout slider makes this update for you.

## Shadow Mode

Shadow mode computes learned fusion weights and logs what the results *would* have been, but serves static fusion results to the user. This lets you evaluate the quality of learned fusion before it affects real users.

```json theme={null}
{
  "learning_config": {
    "shadow_mode": true,
    "rollout_pct": 100.0
  }
}
```

In shadow mode:

1. Both fusion strategies execute in parallel
2. Static fusion results are served to the user
3. Learned fusion weights and re-ranked results are logged as `shadow_execution` metadata
4. The [evaluation system](/docs/retrieval/evaluations) can compare shadow vs. served results offline

The execution response includes a `learned_fusion_context` field (when shadow mode is active) so you can inspect what weights *would* have been used:

```json theme={null}
{
  "results": [ ... ],
  "learned_fusion_context": {
    "shadow_mode": true,
    "served_fusion": "rrf",
    "sampled_weights": {
      "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1": 0.68,
      "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding": 0.32
    },
    "context_level": "personal",
    "effective_exploration": 0.42,
    "feature_uris": [
      "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
      "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
    ]
  }
}
```

> [!TIP]
>
> Run shadow mode for at least one week before enabling live traffic. Compare NDCG and click-through rates between shadow and served results using [evaluations](/docs/retrieval/evaluations).

## Turning Learned Fusion Off

Set `rollout_pct` to `0` with the same update. Every request then uses static fusion, and the rest of `learning_config` and the accumulated interaction history stay as they are, so setting the percentage back resumes where it left off.

> [!WARNING]
>
> The `/v1/retrievers/{retriever_id}/learned-fusion/*` endpoints (enable, disable, rollout, per-user opt-out, opt-in and reset, weights, stats, activity) have been removed and answer `410 Gone` with the replacement in the error body. Rollout and shadow mode are set in `learning_config`; what served each search is in the execution history (below).

## Seeing Which Weights Served a Search

Each execution records the learned-fusion arm that served it as `learned_fusion_context`: the gate it took (`serve`, `shadow` or `skip`), the context level (`personal`, `demographic`, `global`), the sampled weights and the user. It is in the execute response and stored on the execution's profile:

```bash theme={null}
curl "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/executions/$EXECUTION_ID/profile" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"
```

List recent executions with `POST /v1/retrievers/{retriever_id}/executions/list` and read each one's profile to see weights per feature over time, or every execution served to one `user_id`.

## Weight Bounds

Even with strong personalization data, individual feature weights are clamped to prevent degenerate rankings:

```json theme={null}
{
  "learning_config": {
    "min_weight": 0.05,
    "max_weight": 0.95
  }
}
```

| Setting | Default | Effect |
| - | :-: | - |
| `min_weight` | `0.05` | No feature drops below 5% weight — it always contributes something to results |
| `max_weight` | `0.95` | No feature exceeds 95% weight — a single feature cannot completely dominate |

After Thompson Sampling produces raw weights, they are clamped to `[min_weight, max_weight]` and re-normalized to sum to 1.0. This means even a user with extreme interaction patterns will always see results influenced by all configured features.

## Circuit Breaker

If learned fusion weight resolution takes too long (slow signal reads, Redis issues, or high load), the system automatically falls back to static fusion for that individual request:

```
learned fusion resolution > circuit_breaker_timeout_ms (default 1000ms) → fall back to static fusion
```

The timeout is configurable via the `circuit_breaker_timeout_ms` field in `learning_config` (range 1–30000ms).

The circuit breaker is per-request, not per-retriever. A single slow request does not disable learned fusion for everyone. The response's `learned_fusion_context` metadata indicates when a fallback occurred:

```json theme={null}
{
  "learned_fusion_context": {
    "context_level": "none",
    "sampled_weights": { ... },
    "effective_exploration": 1.0,
    "circuit_breaker_triggered": true,
    "weight_resolution_ms": 1023.4
  }
}
```

A `context_level` of `"none"` with `circuit_breaker_triggered: true` indicates the circuit breaker fired and uniform priors were used. The `weight_resolution_ms` field shows the actual resolution time. Monitor the `mxp_learned_fusion_circuit_breaker_total` Prometheus metric to detect systemic latency issues.

## Recommended Rollout Plan

**1. Shadow mode (1-2 weeks)**

Enable `shadow_mode: true` with `rollout_pct: 100`. All users get static results, but learned fusion runs in parallel and logs what it would have returned.

**Check:** Run evaluations comparing shadow vs. served results. Learned fusion NDCG should be equal to or better than static.

**2. 1% canary (1 week)**

Set `shadow_mode: false`, `rollout_pct: 1.0`. A small fraction of users get learned fusion.

**Check:** Monitor click-through rates and circuit breaker triggers. No degradation vs. control group.

**3. 10% ramp (1 week)**

Set `rollout_pct: 10.0`. Enough traffic to produce statistically significant comparisons.

**Check:** Run a [benchmark](/docs/retrieval/evaluations) comparing learned vs. static. Expect learned fusion to show improvement for users with sufficient interaction history.

**4. 50% ramp (1 week)**

Set `rollout_pct: 50.0`. Half of users get personalized results.

**Check:** Monitor weight distributions across users. Look for convergence patterns (weights stabilizing) rather than oscillation.

**5. 100% GA**

Set `rollout_pct: 100.0`. All users get personalized fusion weights, with hierarchical fallback for new users.

**Check:** Keep evaluations running on a schedule to catch regressions. Set up alerts for high global fallback rates and circuit breaker storms.

At any step, if you see regression:

1. Set `rollout_pct: 0` to serve static fusion to everyone
2. Investigate a specific user through the `learned_fusion_context` on their executions' profiles
3. Check whether the issue is global (bad reward map) or user-specific (outlier behavior)
4. Adjust the `learning_config` and restart from the previous step

## Related

* [Auto-Tune](/docs/retrieval/auto-tune) — overview of the full feedback loop
* [Reward Signals](/docs/retrieval/reward-signals) — tuning how interactions influence weights
* [Evaluations](/docs/retrieval/evaluations) — measuring retrieval quality


# Evaluations
Source: https://docs.mixpeek.com/docs/retrieval/evaluations

Measure and compare retriever quality with ground truth datasets and standard IR metrics

Evaluations run your retriever against a curated set of queries with known-relevant documents, then compute standard information retrieval metrics at multiple cutoff points. Use them to quantify retriever quality, compare configurations, and catch regressions before they reach production.

## Quickstart

**1. Create a ground truth dataset**

Each query pairs an input with the document IDs that should be returned.

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/evaluations/datasets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_name": "product-search-golden",
    "queries": [
      {
        "query_id": "q1",
        "query_input": {"query": "wireless earbuds"},
        "relevant_documents": ["doc_a1", "doc_a2", "doc_a3"],
        "relevance_scores": {"doc_a1": 5, "doc_a2": 3, "doc_a3": 2}
      },
      {
        "query_id": "q2",
        "query_input": {"query": "noise canceling headphones"},
        "relevant_documents": ["doc_b1", "doc_b4", "doc_b7"]
      }
    ]
  }'
```

```python Python theme={null}
dataset = client.retrievers.evaluations.create_dataset(
    dataset_name="product-search-golden",
    queries=[
        {
            "query_id": "q1",
            "query_input": {"query": "wireless earbuds"},
            "relevant_documents": ["doc_a1", "doc_a2", "doc_a3"],
            "relevance_scores": {"doc_a1": 5, "doc_a2": 3, "doc_a3": 2},
        },
        {
            "query_id": "q2",
            "query_input": {"query": "noise canceling headphones"},
            "relevant_documents": ["doc_b1", "doc_b4", "doc_b7"],
        },
    ],
)
```

```javascript JavaScript theme={null}
const dataset = await client.retrievers.evaluations.createDataset({
  datasetName: "product-search-golden",
  queries: [
    {
      queryId: "q1",
      queryInput: { query: "wireless earbuds" },
      relevantDocuments: ["doc_a1", "doc_a2", "doc_a3"],
      relevanceScores: { doc_a1: 5, doc_a2: 3, doc_a3: 2 },
    },
    {
      queryId: "q2",
      queryInput: { query: "noise canceling headphones" },
      relevantDocuments: ["doc_b1", "doc_b4", "doc_b7"],
    },
  ],
});
```

**2. Run the evaluation**

Execute your retriever against every query in the dataset. The evaluation runs asynchronously and returns a `task_id` for progress tracking.

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/evaluations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_name": "product-search-golden",
    "evaluation_config": {
      "k_values": [1, 5, 10, 20],
      "metrics": ["precision", "recall", "f1", "f2", "ndcg", "map", "mrr"]
    }
  }'
```

```python Python theme={null}
evaluation = client.retrievers.evaluations.run(
    retriever_id="ret_abc123",
    dataset_name="product-search-golden",
    evaluation_config={
        "k_values": [1, 5, 10, 20],
        "metrics": ["precision", "recall", "f1", "f2", "ndcg", "map", "mrr"],
    },
)
# evaluation["task_id"], evaluation["evaluation_id"]
```

```javascript JavaScript theme={null}
const evaluation = await client.retrievers.evaluations.run({
  retrieverId: "ret_abc123",
  datasetName: "product-search-golden",
  evaluationConfig: {
    kValues: [1, 5, 10, 20],
    metrics: ["precision", "recall", "f1", "f2", "ndcg", "map", "mrr"],
  },
});
// evaluation.taskId, evaluation.evaluationId
```

**3. Get results**

Poll the evaluation until `status` is `completed`.

```bash theme={null}
curl "$MP_API_URL/v1/retrievers/{retriever_id}/evaluations/{evaluation_id}" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```json Example response theme={null}
{
  "evaluation_id": "eval_abc123",
  "status": "completed",
  "query_count": 50,
  "overall_metrics": {
    "precision_at_5": 0.85,
    "recall_at_5": 0.72,
    "f1_at_5": 0.78,
    "f2_at_5": 0.74,
    "ndcg_at_5": 0.81,
    "map": 0.71,
    "mrr": 0.93
  },
  "metrics_by_k": {
    "1":  {"precision": 0.90, "recall": 0.18, "f1": 0.30, "f2": 0.21, "ndcg": 0.90},
    "5":  {"precision": 0.85, "recall": 0.72, "f1": 0.78, "f2": 0.74, "ndcg": 0.81},
    "10": {"precision": 0.75, "recall": 0.88, "f1": 0.81, "f2": 0.85, "ndcg": 0.89},
    "20": {"precision": 0.62, "recall": 0.95, "f1": 0.75, "f2": 0.86, "ndcg": 0.91}
  }
}
```

## Metrics

Every metric is computed at each K value you specify. The defaults cover most use cases:

| Metric | What It Measures | Formula |
| - | - | - |
| **Precision\@K** | Accuracy of top results | relevant in top K ÷ K |
| **Recall\@K** | Coverage of relevant documents | relevant in top K ÷ total relevant |
| **F1\@K** | Balanced precision/recall | harmonic mean of P and R |
| **F2\@K** | Recall-weighted balance | weighted harmonic mean (β=2), penalizes missed docs 4× more than false positives |
| **MAP** | Ranking quality across all queries | average of precision at each relevant doc's position |
| **MRR** | How quickly users find a relevant result | 1 ÷ rank of first relevant document |
| **NDCG\@K** | Ranking quality with graded relevance | normalized discounted cumulative gain |

> [!NOTE]
>
> **F2 vs F1:** Use F2 when missing a relevant document is worse than showing an irrelevant one — the common case in search, recommendations, and discovery. F1 treats both errors equally.

### Reading Your Scores

| Score | What It Tells You |
| - | - |
| **NDCG\@10 = 0.89** | Your top-10 ranking captures 89% of the ideal ordering. Relevant docs appear near the top. |
| **Precision\@5 = 0.85** | 4–5 of every 5 results are relevant. Users see high-quality results. |
| **Recall\@20 = 0.95** | You surface 95% of all relevant documents within the top 20. Strong coverage. |
| **F2\@10 = 0.85** | Recall-weighted balance is strong — few relevant documents are being missed. |
| **MRR = 0.93** | The first relevant result typically appears at position 1 or 2. |
| **MAP = 0.71** | Overall ranking quality is solid but there's room to improve ordering. |

### Graded Relevance

When you provide `relevance_scores` in your dataset, NDCG uses graded relevance instead of binary. This distinguishes "exactly right" from "somewhat relevant":

```json theme={null}
{
  "query_id": "q1",
  "query_input": {"query": "wireless earbuds"},
  "relevant_documents": ["doc_a1", "doc_a2", "doc_a3"],
  "relevance_scores": {
    "doc_a1": 5,
    "doc_a2": 3,
    "doc_a3": 1
  }
}
```

| Score | Meaning |
| - | - |
| 5 | Perfect match |
| 3–4 | Highly relevant |
| 1–2 | Marginally relevant |
| 0 | Not relevant |

Without `relevance_scores`, all metrics use binary relevance (relevant or not).

## Comparing Retrievers

Run the same dataset against different retriever configurations to find the best pipeline:

```bash theme={null}
# Evaluate baseline (vector search only)
curl -X POST "$MP_API_URL/v1/retrievers/ret_baseline/evaluations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{"dataset_name": "product-search-golden"}'

# Evaluate candidate (vector search + reranker)
curl -X POST "$MP_API_URL/v1/retrievers/ret_reranked/evaluations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{"dataset_name": "product-search-golden"}'
```

Compare the `metrics_by_k` side by side:

| Metric | Baseline | + Reranker | Delta |
| - | - | - | - |
| NDCG\@10 | 0.78 | 0.89 | +14% |
| Precision\@5 | 0.72 | 0.85 | +18% |
| F2\@10 | 0.76 | 0.85 | +12% |
| MRR | 0.81 | 0.93 | +15% |

> [!TIP]
>
> Run the same dataset after every pipeline change — adding stages, swapping models, adjusting fusion weights — to quantify the impact before deploying.

## Ground Truth Datasets

### Dataset Requirements

* At least 1 query (aim for 50+ for statistically meaningful results)
* Each query must have at least 1 relevant document
* `query_input` must use the keys your retriever's `input_schema` declares
* `relevance_scores`, if provided, must cover all `relevant_documents`

A dataset needs `dataset_name` and `queries`, and each query needs `query_id`,
`query_input` and `relevant_documents`. `relevance_scores` is optional and takes
0 to 5 per document, which is what NDCG reads.

A run keys on the dataset **name**, not its id, and returns a `task_id`
immediately while the evaluation proceeds asynchronously.

### When every query skips

Input validation is strict. A key in `query_input` that the retriever does not
declare is rejected rather than ignored, and the message names what it does
accept:

```
Unknown input field(s) 'top_k'. This retriever accepts: ['query', ...]
```

That query skips. If every key is wrong, every query skips.

> [!NOTE]
>
> **An all-skipped run finishes as `failed`, not as zeros.** `EvaluationStatus`
> is `pending`, `in_progress`, `completed` or `failed`, and a run that evaluated
> nothing takes the failed branch rather than reporting 0.0 across the board.
>
> Read `query_results` on the evaluation record to see which branch each query
> took and the actual error for each skip. `total_queries`, `evaluated_queries`
> and `skipped_queries` give you the same story as counts. Diagnosing a skipped
> run needs neither worker logs nor a re-run.

> [!WARNING]
>
> The runner injects `top_k` only when the retriever's `input_schema` declares it, and a
> value in your own dataset wins over the injected one.
>
> If you are holding an older all-skip result, re-run it before treating it as a dataset
> problem. Earlier runners injected `top_k` unconditionally, which failed validation
> against any retriever that does not declare a `top_k` input.

### Managing Datasets

```bash theme={null}
# List all datasets
curl "$MP_API_URL/v1/retrievers/evaluations/datasets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"

# Get a specific dataset
curl "$MP_API_URL/v1/retrievers/evaluations/datasets/product-search-golden" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

### Building Good Datasets

> **Include query diversity**
>
> Cover head queries (popular), torso (moderate), and tail (rare/specific). Don't just test easy cases.

> **Use graded relevance**
>
> Binary relevant/not-relevant misses nuance. Score documents 0–5 so NDCG can distinguish good rankings from great ones.

> **Match real traffic**
>
> Sample queries from production logs. Synthetic queries test what you think users ask, not what they actually ask.

> **Version your datasets**
>
> Keep datasets stable across evaluations so you can track metric trends over time. Create new versions for schema changes.

## Configuration Reference

**dataset\_name** `string` *required*

Name of the ground truth dataset to evaluate against.

**evaluation\_config.k\_values** `integer[]` (default: \[1, 5, 10, 20])

Cutoff positions for @K metrics. Include the K values that match your UI — if you show 10 results per page, include `10`.

**evaluation\_config.metrics** `string[]` (default: \["precision", "recall", "f1", "f2", "map", "ndcg", "mrr"])

Metrics to compute. Available: `precision`, `recall`, `f1`, `f2`, `map`, `ndcg`, `mrr`.

## Related

* [Retriever Benchmarks](/docs/api-reference/retriever-benchmarks/create-benchmark) — replay live sessions against candidate retrievers
* [Improve Relevance](/docs/platform/improve-relevance) — interaction signals, fusion strategies, and the feedback loop
* [API Reference: Run Evaluation](/docs/api-reference/retriever-evaluations/run-evaluation) — full endpoint specification
* [API Reference: Create Dataset](/docs/api-reference/retriever-evaluations/create-evaluation-dataset) — dataset creation endpoint


# Filters
Source: https://docs.mixpeek.com/docs/retrieval/filters

Compose filter conditions with logical operators

![Filter composition: AND, OR, and NOT operators nest to build complex filter logic on document payloads](https://mintcdn.com/mixpeek/pDBzbsnRaRIThJZv/assets/mixpeek-filters.svg?fit=max\&auto=format\&n=pDBzbsnRaRIThJZv\&q=85\&s=1ab3a765413b2e1644fa88ce899e201d)

Filters narrow results using logical operators to combine conditions. They operate on document payloads (metadata, enrichments, passthrough fields) and can be applied in retriever execution or as dedicated `filter@v1` stages.

## Payload Indexes

> [!WARNING]
>
> Filters require **payload indexes** on the fields you filter by. Without an index, the vector store performs a full scan — which is slow on large collections and may return incomplete results.

Create indexes on your namespace before using filters:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/namespaces/{namespace_id} \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "payload_indexes": [
      {"field_name": "metadata.category", "type": "keyword"},
      {"field_name": "metadata.price", "type": "integer"},
      {"field_name": "metadata.status", "type": "keyword"}
    ]
  }'
```

Supported index types:

| Type | Use for |
| - | - |
| `keyword` | Exact-match strings (categories, IDs, tags) |
| `integer` | Whole numbers |
| `float` | Decimal numbers |
| `bool` | Boolean fields |
| `datetime` | Timestamps |
| `text` | Case-insensitive substring match, the same as `contains` |
| `geo` | Geospatial queries |

If you filter on an unindexed field, the response includes a `warnings` array telling you which fields need indexes:

```json theme={null}
{
  "warnings": [
    "Filter field 'brand' has no payload index — filtering may be slow or unreliable on large collections. Create an index via PATCH /v1/namespaces/{namespace_id} with payload_indexes: [{\"field_name\": \"brand\", \"type\": \"keyword\"}]"
  ]
}
```

> [!NOTE]
>
> System fields (`collection_id`, `bucket_id`, `object_id`, `batch_id`) and `_internal.*` fields are indexed automatically — you only need to create indexes for your own fields.

## Logical Operators

Mixpeek filters support three logical operators for composing conditions:

| Operator | Description | Usage |
| - | - | - |
| `AND` | All conditions must be true | Combine multiple required constraints |
| `OR` | At least one condition must be true | Match any of several alternatives |
| `NOT` | Inverts the condition | Exclude matching documents |

### AND Operator

Requires all nested conditions to match:

```json theme={null}
{
  "AND": [
    { "field": "metadata.status", "operator": "eq", "value": "published" },
    { "field": "metadata.price", "operator": "lte", "value": 100 }
  ]
}
```

### OR Operator

Matches if any nested condition is true:

```json theme={null}
{
  "OR": [
    { "field": "metadata.category", "operator": "eq", "value": "video" },
    { "field": "metadata.category", "operator": "eq", "value": "audio" }
  ]
}
```

### NOT Operator

Excludes documents matching the condition:

```json theme={null}
{
  "NOT": {
    "field": "metadata.status", "operator": "eq", "value": "draft"
  }
}
```

## Nesting Operators

Logical operators can be nested to create complex filter logic:

```json theme={null}
{
  "AND": [
    { "field": "metadata.status", "operator": "eq", "value": "published" },
    {
      "OR": [
        { "field": "metadata.category", "operator": "eq", "value": "video" },
        { "field": "metadata.category", "operator": "eq", "value": "audio" }
      ]
    },
    {
      "NOT": {
        "field": "metadata.restricted", "operator": "eq", "value": true
      }
    }
  ]
}
```

This filter matches documents that are:

* Published **AND**
* Either video or audio **AND**
* Not restricted

## Comparison Operators

Use these operators within conditions:

| Operator | Description |
| - | - |
| `eq` | Equals |
| `ne` | Not equals |
| `gt` | Greater than |
| `gte` | Greater than or equal |
| `lt` | Less than |
| `lte` | Less than or equal |
| `in` | Value in list |
| `nin` | Value not in list |
| `exists` | Field exists |
| `is_null` | Field is null |
| `contains` | String contains substring. On an array field, exact element equality instead (see the note below) |
| `starts_with` | String starts with prefix |
| `ends_with` | String ends with suffix |
| `regex` | Matches regular expression |
| `text` | Case-insensitive substring match, the same as `contains` (see the note below) |
| `words` | Every word of the value appears in the field as a whole word, matched on its English stem ("drinking" finds "drinks") |
| `phrase` | Exact phrase — matches the words **in order**, word-boundary safe |

> [!NOTE]
>
> `text` matches the value as a substring on retriever filters and document
> lists. It does not split the value into words or rank. Use `words` for
> whole-word matching, and `phrase` when word
> order matters — e.g. find a transcript where someone says an exact quote.
> `{ "field": "transcription", "operator": "phrase", "value": "make america great again" }`
> matches "...make america great again..." but not "america will be great again".

> [!NOTE]
>
> Changed 2026-10-14: on an array field, `contains` and `text` now require an
> exact match against one of the array's elements, not a substring of one.
> `{ "field": "tags", "operator": "contains", "value": "car" }` matches a
> document tagged `["car", "engine"]` but not one tagged only
> `["car_interior"]`. Before this change it matched both. A scalar string
> field is unaffected: `contains` and `text` there are still a substring
> match. There is currently no operator for a substring match against one
> element of an array; file a request if you need one.

## Geospatial Operators

Geospatial operators filter documents by a **location field**, a payload value
holding a geographic point. A point may be either an object `{ "lat": <num>, "lon": <num> }`
or a GeoJSON-style `[lon, lat]` array. A field holding a **list** of points matches
if **any** point satisfies the predicate. See
[Geospatial filtering](/docs/retrieval/stages/attribute-filter#geospatial-filtering) for
how the same operators behave inside the `attribute_filter` stage.

| Operator | Matches documents whose location… |
| - | - |
| `geo_radius` | falls within `radius` **meters** of a center point (haversine distance) |
| `geo_bounding_box` | falls inside an axis-aligned box defined by two corners |
| `geo_polygon` | falls inside an arbitrary polygon (ray-casting, point-in-polygon) |

Each operator takes a structured `value`:

```json theme={null}
// geo_radius — within 5 km of the Eiffel Tower
{ "field": "location", "operator": "geo_radius",
  "value": { "center": { "lat": 48.8584, "lon": 2.2945 }, "radius": 5000 } }

// geo_bounding_box — top_left (NW) and bottom_right (SE) corners
{ "field": "location", "operator": "geo_bounding_box",
  "value": { "top_left":     { "lat": 49.0, "lon": 2.0 },
             "bottom_right": { "lat": 48.0, "lon": 3.0 } } }

// geo_polygon — exterior ring of >= 3 points (auto-closed)
{ "field": "location", "operator": "geo_polygon",
  "value": { "exterior": { "points": [
    { "lat": 48.0, "lon": 2.0 }, { "lat": 49.0, "lon": 2.0 },
    { "lat": 49.0, "lon": 3.0 }, { "lat": 48.0, "lon": 3.0 } ] } } }
```

> [!NOTE]
>
> Distances use the haversine formula on a spherical earth (R = 6,371,000 m).
> Bounding boxes handle the **antimeridian**: when `top_left.lon > bottom_right.lon`
> the box is treated as wrapping across ±180°. Malformed geometry (out-of-range
> `lat`/`lon`, a missing corner, or fewer than 3 polygon points) is **rejected at
> request time** with a descriptive error; a document whose location field is
> missing or unparseable is a non-match (it is never an error).

## Lineage Shortcuts

Every Mixpeek document carries a `_internal.lineage` block recording where it
came from. To filter by lineage you don't have to use the underscore-prefixed
paths — use the friendly aliases below in any `field` position.

| Alias | Resolves to | Use for |
| - | - | - |
| `from_object` | `_internal.lineage.root_object_id` | "Everything derived from this bucket object" |
| `from_bucket` | `_internal.lineage.root_bucket_id` | "Everything derived from this bucket" |
| `from_document` | `_internal.lineage.source_document_id` | Direct children of one upstream document |
| `from_collection` | `_internal.lineage.source_collection_id` | Documents whose immediate parent was in this collection |

```json theme={null}
{
  "AND": [
    { "field": "from_object", "operator": "eq", "value": "obj_video_123" }
  ]
}
```

You can mix lineage aliases with regular fields and templates:

```json theme={null}
{
  "AND": [
    { "field": "from_object", "operator": "eq", "value": "{{INPUT.object_id}}" },
    { "field": "metadata.scene_score", "operator": "gte", "value": 0.8 }
  ]
}
```

The aliases are also accepted by document list endpoints and retriever filter
stages — the same vocabulary works everywhere `field` is used.

## The `_internal` Envelope

Every document carries an `_internal` envelope that Mixpeek writes and owns. The
rule for what lives there: **content at the root, record-keeping in `_internal`.**
Content is what the asset *is* — anything you filter on, rank by, or show a user,
including everything Mixpeek derived for you (embeddings, transcripts, taxonomy
labels). `_internal` is what the *row* is — how it came to exist, who may read it,
and the identity and timestamps of the record. You can filter on these fields
anywhere `field` is used; system-owned ones are rejected if you try to set them on
write.

Range filtering works on these timestamps and on any Datetime index you define.
Pass RFC 3339 strings as the bounds:

```json theme={null}
{
  "filters": {
    "AND": [
      { "field": "_internal.created_at", "operator": "gte", "value": "2026-01-01T00:00:00Z" },
      { "field": "_internal.created_at", "operator": "lte", "value": "2027-01-01T00:00:00Z" }
    ]
  }
}
```

### Every `_internal` field

This table is generated from the server's field registry, so it is the complete, authoritative set — if a name is not here, Mixpeek does not write it under `_internal`. Filter on the **field / alias** column; the storage path is shown so you can address a nested value explicitly.

**Identity**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `document_id` | `_internal.document_id` | `eq`, `in` — system-owned, rejected if you send it on write |
| `collection_id` | `_internal.collection_id` | `eq`, `in` — system-owned, rejected if you send it on write |

**Timestamps**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `created_at` | `_internal.created_at` | `eq` and range operators (`gte`/`lte`/`gt`/`lt`). Pass RFC 3339 strings as bounds |
| `updated_at` | `_internal.updated_at` | `eq` and range operators (`gte`/`lte`/`gt`/`lt`). Pass RFC 3339 strings as bounds |

**Lineage**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `root_object_id` · aliases `from_object` | `_internal.lineage.root_object_id` | `eq`, `in` |
| `root_bucket_id` · aliases `from_bucket` | `_internal.lineage.root_bucket_id` | `eq`, `in` |
| `source_type` | `_internal.lineage.source_type` | `eq`, `in` |
| `source_object_id` | `_internal.lineage.source_object_id` | `eq`, `in` |
| `source_document_id` · aliases `from_document` | `_internal.lineage.source_document_id` | `eq`, `in` |
| `source_collection_id` · aliases `from_collection` | `_internal.lineage.source_collection_id` | `eq`, `in` |
| `lineage_path` | `_internal.lineage.path` | `eq`, `in` |
| `lineage_chain` | `_internal.lineage.chain` | `eq`, `in` |
| `bucket_id` | `_internal.lineage.root_bucket_id` | `eq`, `in` |
| `object_id` | `_internal.lineage.source_object_id` | `eq`, `in` |
| `processing_tier` | — (not stored) | not stored anywhere — a filter on it matches nothing |
| `parent_document_id` | — (not stored) | not stored anywhere — a filter on it matches nothing |

**System metadata**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `modality` | `_internal.modality` | `eq`, `in` |
| `mime_type` | `_internal.mime_type` | `eq`, `in` |
| `size_bytes` | `_internal.size_bytes` | `eq`, `in` |
| `content_hash` | `_internal.content_hash` | `eq`, `in` — system-owned, rejected if you send it on write |
| `metadata` | `_internal.metadata` | `eq`, `in` |

**Blobs**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `source_blobs` | `_internal.source_blobs` | `eq`, `in` |
| `document_blobs` | `_internal.document_blobs` | `eq`, `in` |

**Provenance**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `source_details` | `_internal.source_details` | `eq`, `in` |
| `source_url` | `_internal.processing.source_url` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |
| `object_key_source` | `_internal.processing.object_key_source` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |
| `detected_mime_type` | `_internal.processing.detected_mime_type` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |
| `processing_history` | `_internal.processing.history` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |
| `taxonomy_lineage` | `_internal.processing.taxonomy_lineage` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |
| `last_health_check` | `_internal.processing.last_health_check` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |
| `include_processing_history` | `_internal.processing.include_history` | filter by the exact dotted path only (the bare name is a real customer field name, so it is not auto-resolved) |

**Tenancy**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `internal_id` | `_internal.internal_id` | `eq`, `in` — system-owned, rejected if you send it on write |
| `namespace_id` | `_internal.namespace_id` | `eq`, `in` — system-owned, rejected if you send it on write |

**Access control**

| Field / alias | Storage path | Filtering |
| - | - | - |
| `_acl` | `_internal._acl` | `eq`, `in` — system-owned, rejected if you send it on write |

## Using Templates

Reference request inputs or stage outputs in filter values:

```json theme={null}
{
  "AND": [
    { "field": "metadata.category", "operator": "eq", "value": "{{INPUT.category}}" },
    { "field": "metadata.price", "operator": "lte", "value": "{{INPUT.max_price}}" }
  ]
}
```

## Filter Stage Example

```json theme={null}
{
  "stage_name": "filter",
  "stage_type": "filter",
  "config": {
    "stage_id": "attribute_filter",
    "parameters": {
      "conditions": {
        "AND": [
          { "field": "metadata.category", "operator": "eq", "value": "audio" },
          { "field": "metadata.price", "operator": "lte", "value": "{{INPUT.max_price}}" }
        ]
      }
    }
  }
}
```

## Stage Pre-Filters and Post-Filters

Every stage accepts optional `pre_filters` and `post_filters` as siblings of
`parameters`. Use `pre_filters`. They narrow the candidate set **before** the
stage runs, pushed down into the vector store as native filters. Both take the
same logical-operator shape as any other filter.

> [!WARNING]
>
> **`post_filters` is accepted and never applied.** The field is declared on
> every stage and passes validation, and no stage applies it. A `post_filters`
> predicate returns the same documents as sending no filter at all, with HTTP
> 200 and no warning.
>
> Measured on `feature_search`: a nonsense value returns the full unfiltered
> set, identical to an unfiltered run. The identical predicate in `pre_filters`
> filters correctly.
>
> Put any predicate you rely on in `pre_filters`. Never use `post_filters` to
> restrict scope, because it does not restrict anything.

**Canonical shape** — wrap conditions in an explicit logical operator:

```json theme={null}
{
  "pre_filters": {
    "AND": [
      { "field": "archive_status", "operator": "ne", "value": "ARCHIVED" },
      { "field": "Keywords", "operator": "contains", "value": "skincare" }
    ]
  }
}
```

Always prefer the explicit `{ "AND": [ ... ] }` form — it is unambiguous and
nests cleanly with `OR`/`NOT`.

> [!NOTE]
>
> For convenience, two shorthand forms are coerced to an `AND` group:
>
> * a **single bare condition** — `{ "field": "...", "operator": "...", "value": "..." }` becomes `{ "AND": [ <condition> ] }`
> * a **list of conditions** — `[ { ... }, { ... } ]` becomes `{ "AND": [ ... ] }`
>
> Each condition must carry all three of `field`, `operator`, and `value`. An
> **incomplete** condition (for example, a missing `operator`) is rejected with a
> clear error rather than silently ignored. That guarantee covers the shape of a
> condition, not where you place it. A well-formed condition placed in
> `post_filters` still degrades into an unfiltered result, as described above.

## When a filter is refused

The API refuses a filter it cannot compile with HTTP 422, before it reads any document.
The error's `details` names the node by its path in your request and says why:

```json theme={null}
{
  "success": false,
  "status": 422,
  "error": {
    "message": "filter node AND[0].NOT[0]: an empty group: give it AND, OR or NOT with at least one condition",
    "details": {
      "node": "AND[0].NOT[0]",
      "reason": "an empty group: give it AND, OR or NOT with at least one condition"
    }
  }
}
```

The API refuses an empty group inside `AND`, `OR` or `NOT`, an unknown operator, and a key the filter grammar does not know.
An empty top-level filter (`{}` or `{"AND": []}`) means no filter.

## Options

| Option | Default | Description |
| - | - | - |
| `case_sensitive` | `false` | Case-sensitive matching for `contains`, `starts_with`, `ends_with`, `text`, `regex` and `phrase`. `eq`, `ne`, `in` and `nin` always compare exactly. |

```json theme={null}
{
  "field": "metadata.title",
  "operator": "contains",
  "value": "AI",
  "case_sensitive": true
}
```


# Interactions
Source: https://docs.mixpeek.com/docs/retrieval/interactions

Turn user behavior into better retrieval through automated feedback loops

Interactions capture how users engage with search results—clicks, purchases, skips, dwell time. This feedback automatically improves your retrievers through fine-tuning, personalization, and analytics.

![Mixpeek Interactions Flow](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-interactions.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=90d32424716608350755e20c6344ffab)

## Why Track Interactions

**Automatic Quality Improvement**: User behavior becomes training data for fine-tuning embedding models and ranking algorithms—no manual labeling required.

**Measure Real Performance**: Traditional IR metrics (precision, recall) use test sets. Interactions reveal actual user satisfaction through click-through rates, conversion, and engagement.

**Personalization at Scale**: Build user preference profiles from interaction history to deliver personalized results without managing separate recommendation systems.

**Identify Blind Spots**: Find queries with low engagement, results that users skip despite high rankings, and content gaps causing zero-result queries.

## What You Get

> **Training Data Generation**
>
> Interactions format into contrastive pairs (query, clicked\_doc) for embedding fine-tuning. Position is recorded for future bias correction.

> **Relevance Analytics**
>
> CTR by position, documents with high skip rates, queries needing tuning, time-to-first-click—all without custom infrastructure.

> **Result Deduplication**
>
> Exclude previously purchased, viewed, or consumed content automatically by filtering on past interactions.

> **Popularity Boosting**
>
> Rerank results based on recent interaction signals (clicks, conversions) to surface trending content.

## Capture Interactions

Record user behavior with a single API call:

```bash theme={null}
POST /v1/retrievers/interactions
{
  "feature_id": "doc_xyz789",        # Document ID from retriever results
  "interaction_type": ["click"],     # One or more signal types
  "position": 3,                      # 0-based position in results (critical for bias correction)
  "metadata": {
    "duration_ms": 4500,              # Optional: dwell time, device, query text
    "query": "wireless earbuds"
  },
  "user_id": "user_123",              # For personalization and session tracking
  "session_id": "sess_456"
}
```

> [!NOTE]
>
> **`interaction_type` is always a JSON array — even for a single signal** (`["click"]`, not `"click"`). The array captures the **co-occurring signals of one user action**, not several separate events: a single result that was shown, clicked, and dwelt on is `["view", "click", "long_view"]` in **one** request. Send a separate request per discrete user action (each click, each purchase) — don't batch unrelated events into one array.

### Signal Types

| Type | Strength | Use Case |
| - | - | - |
| `impression` | Neutral | Result was rendered on screen |
| `view` | Weak positive | User viewed a result |
| `click` | Moderate positive | User clicked result |
| `dwell` | Weak positive | User lingered on a result |
| `long_view` | Strong positive | Sustained engagement (track via `duration_ms`) |
| `purchase` | Strong positive | Conversion event |
| `add_to_cart` | Positive | Intent to purchase |
| `wishlist` | Positive | User added to wishlist |
| `positive_feedback` | Explicit positive | Thumbs up vote |
| `negative_feedback` | Explicit negative | Thumbs down vote |
| `share` | Strong positive | User shared the result |
| `bookmark` | Moderate positive | User saved for later |
| `query_refinement` | Neutral | User modified their search |
| `zero_results` | Neutral | Query yielded no results |
| `filter_toggle` | Neutral | User modified filters |
| `skip` | Weak negative | User ignored result |
| `return_to_results` | Moderate negative | User bounced back quickly |

> [!TIP]
>
> Combine multiple types per event: `["click", "long_view"]` when a user clicks *and* stays engaged.

## Where signals are stored

Every retriever execution and every interaction you record is written to
[system collections](/docs/vector-store/system-collections) in your namespace. Mixpeek
creates each one the first time it has something to store. They are normal,
visible collections: you can list and filter their documents, and you can point
retrievers at them. They hold your own search signals, which is what makes
training a relevance model on your own usage possible without exporting anything.

### What lands in them

| Collection | `signal_type` | Written when | Carries |
| - | - | - | - |
| `mxp_retriever_queries` | `retriever_query` | A retriever executes | `query_text`, `results_count`, `latency_ms`, `filter_fields`, the per-stage rows, and the query embedding |
| `mxp_retriever_executions` | `retriever_execution` | A retriever executes | `status`, `duration_ms`, `results_count`, `session_id` and the request's `inputs` |
| `mxp_retriever_interactions` | `retriever_interaction` | You `POST /v1/retrievers/interactions` | `target_document_id`, `interaction_type`, `position`, `session_id`, `user_id`, `retriever_id`, `execution_id` |

All three carry the run's `execution_id`, so an interaction joins to the query
and the execution it came from.

### Logging a query without a click

You do not log queries yourself. Every retriever execution writes its query to
`mxp_retriever_queries`, including a query that returned nothing and a query
nobody clicked on. To find zero-result queries, filter that collection on
`results_count = 0`.

An interaction records what a user did with one result, so every interaction
names that result (`feature_id`, or its alias `document_id`). That includes the
types that describe the search rather than a result (`zero_results`,
`query_refinement`, `filter_toggle`). A search with nothing to click needs no
interaction: its row in `mxp_retriever_queries` is the log. Send interactions
for results a user acted on, with the `execution_id` the search returned.

> [!TIP]
>
> **The query embedding is reused, not recomputed.** A `retriever_query` row
> stores the dense embedding the execution already produced, so vector search or
> clustering over your past queries costs no new inference.

### What already reads it

* [Learned fusion](/docs/relevance/learned-fusion) derives arm weights from
  interaction signals.
* Index recommendations use the query and execution record.
* [Evaluation](/docs/retrieval/evaluations) datasets can be mined from real queries.

> [!WARNING]
>
> **Per-tenant reranker and LTR training is demonstrated, not automated.** The
> signals needed to train one are captured and the path has been shown to work.
> There is no production loop training a model on them for you, so treat it as
> something you can build rather than something running.

## Signal Strategy

Which signals to capture, when, and why. Not all signals carry equal weight.

### Signal Strength Matrix

This table maps each interaction type to how it's used across the platform:

| Signal Type | Strength | Fusion Learning | Personalization | Analytics | Fine-Tuning |
| - | - | :-: | :-: | :-: | :-: |
| `click` | Moderate | Primary positive signal | Session history | CTR metrics | Positive pairs |
| `long_view` | Strong | High-weight positive | Preference indicator | Engagement depth | Strong positive |
| `purchase` | Strongest | Conversion signal | Purchase exclusion | Revenue attribution | Gold standard |
| `add_to_cart` | Strong | Intent signal | Cart-based recs | Funnel metrics | Near-positive |
| `positive_feedback` | Explicit | Direct reward | Profile building | Quality score | Ground truth |
| `negative_feedback` | Explicit | Direct penalty | Anti-preference | Issue detection | Hard negative |
| `skip` | Weak negative | Implicit penalty | — | Skip rate | Soft negative |
| `return_to_results` | Moderate negative | Bounce signal | — | Bounce rate | Negative pair |
| `share` | Strong | Social proof | Sharing patterns | Virality | Strong positive |
| `bookmark` | Moderate | Save intent | Saved items | Save rate | Positive |
| `dwell` | Variable | Duration-weighted | — | Engagement depth | Weighted |
| `query_refinement` | Neutral (0.0) | Not in default reward map | — | Query analysis | — |

### Use Case Patterns

**E-commerce**

**Primary signals:** `purchase`, `add_to_cart`, `click` — conversion is the strongest relevance indicator.

**Key patterns:**

* Track `add_to_cart` separately from `purchase` to measure funnel drop-off
* Use `position` for learning-to-rank bias correction
* Store `order_value` in metadata for revenue-weighted metrics

**Media & Entertainment**

**Primary signals:** `long_view`, `click`, `share` — engagement depth matters more than the click.

**Key patterns:**

* Combine `click` and `long_view` in one event when engagement is sustained
* Use `duration_ms` and `completion_pct` to distinguish genuine views from bounces
* Track `share` as a strong positive signal for content quality

**Enterprise Search**

**Primary signals:** `click`, `positive_feedback`, `bookmark` — clicks alone may indicate obligation, not satisfaction.

**Key patterns:**

* Add thumbs-up/down UI to capture explicit feedback
* Store `department` or `role` in metadata for demographic-level learned fusion
* Track `query_refinement` to find queries where results fall short

**Recommendations**

**Primary signals:** `click`, `purchase`, `negative_feedback` — you need negative signals to avoid recommending disliked content.

**Key patterns:**

* Capture "not interested" / "hide" as `negative_feedback` to suppress similar content
* Use purchase history to filter already-bought items from results
* Track position carefully — recommendation position bias is stronger than search

### Position Tracking

> [!WARNING]
>
> **Always capture `position`.** It is stored on every interaction for analytics and audit,
> and recording it from the start saves you a backfill.

Position bias is a well-known problem: users click higher-ranked results regardless of relevance. Mixpeek records `position` on every interaction, and reward computation does not weight by it, so a click at position 8 earns the same reward as a click at position 0. Correct for position bias in your own analysis if you need it.

### Cold Start

When you first deploy you have zero interactions. The system adapts automatically through the [hierarchical fallback](/docs/relevance/learned-fusion#hierarchical-fallback) in Thompson Sampling (personal → demographic → global → uniform prior):

| Interactions | Fusion Behavior | What To Do |
| - | - | - |
| 0 | Falls back to `rrf` (uniform weights) | Ship with `rrf`, start collecting signals |
| 1–50 | Global-level learned weights only | Monitor analytics for obvious issues |
| 50–500 | Demographic-level personalization begins | Verify evaluation metrics are trending up |
| 500+ | Per-user personalization kicks in | Run benchmarks to compare against baseline |

### Client-Side Capture

Track clicks and dwell time directly from the browser, using `sendBeacon` so dwell events survive page unload:

```javascript theme={null}
// Track clicks with position
document.querySelectorAll('.search-result').forEach((el, index) => {
  el.addEventListener('click', () => {
    fetch('/api/interactions', {
      method: 'POST',
      body: JSON.stringify({
        feature_id: el.dataset.documentId,
        interaction_type: ['click'],
        position: index,
        metadata: { query: currentQuery },
        user_id: userId, session_id: sessionId
      })
    });
  });
});

// Track dwell time on result detail pages
let startTime = Date.now();
window.addEventListener('beforeunload', () => {
  const dwellMs = Date.now() - startTime;
  navigator.sendBeacon('/api/interactions', JSON.stringify({
    feature_id: currentDocId,
    interaction_type: dwellMs > 5000 ? ['click', 'long_view'] : ['click'],
    position: resultPosition,
    metadata: { duration_ms: dwellMs, query: originalQuery },
    user_id: userId
  }));
});
```

## Outcomes & Use Cases

### 1. Fine-Tune Embedding Models

```python theme={null}
# Collect 30 days of interaction data as training pairs
training_data = analytics.get_query_document_pairs(
    date_range="last_30_days",
    min_interactions=2  # Only docs with real engagement
)

# System formats as contrastive pairs:
# Positive: (query, clicked_doc)
# Hard negative: (query, high_ranked_but_skipped_doc)
# Position bias automatically corrected

fine_tuned_model = training.fine_tune(
    base_model="text-embedding-3-small",
    training_data=training_data
)
```

**Outcome**: 10-30% improvement in relevance metrics from domain-specific fine-tuning using real user preferences.

### 2. Identify & Fix Ranking Issues

```python theme={null}
# Find queries where top results are skipped
signals = analytics.get_retriever_signals(retriever_id="ret_123")
# Returns:
# - Documents ranked high but skipped (ranking issues)
# - Queries with &lt;5% CTR (need tuning)
# - Average position of first click (relevance proxy)
```

**Outcome**: Data-driven decisions on which retrievers need adjustment, which taxonomy mappings to update, or which filters to refine.

### 3. Personalize Without Recommendation Infrastructure

```python theme={null}
# Exclude content user already consumed
previous_purchases = interactions.list(
    user_id="user_123",
    interaction_type=["purchase"],
    days=90
)

retriever.execute(
    query="new arrivals",
    filters={
        "document_id": {"$nin": [i.feature_id for i in previous_purchases]}
    }
)
```

**Outcome**: Improved user experience (no duplicate suggestions) without building separate collaborative filtering or recommendation systems.

### 4. Continuously Tune Ranking from Interactions

Set `fusion: "learned"` and click/purchase signals automatically re-weight your search over time — no manual reranking rules. The loop learns one weight per feature URI and keeps no per-document state (see [What Learned Fusion Optimizes](/docs/relevance/learned-fusion#what-learned-fusion-optimizes)). Weight higher-intent signals above clicks with the `reward_map`:

```json theme={null}
{
  "stages": [
    {
      "stage_name": "search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 },
            { "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
          ],
          "fusion": "learned",
          "learning_config": {
            "reward_map": { "click": 1.0, "add_to_cart": 2.0, "purchase": 3.0 },
            "decay_window_days": 30
          },
          "final_top_k": 25
        }
      }
    }
  ]
}
```

**Outcome**: click/purchase signals continuously re-tune feature weights via [auto-tune](/docs/retrieval/auto-tune) — higher-intent interactions carry more weight — with no manual reranking rules. See [Reward Signals](/docs/retrieval/reward-signals) for the full `reward_map`.

## Query & Export

Retrieve interactions for analysis or external ML pipelines:

```bash theme={null}
# Get all interactions for a document
POST /v1/retrievers/interactions/list
{
  "feature_id": "doc_xyz789"
}

# Get user interaction history. Pagination goes in the QUERY STRING.
POST /v1/retrievers/interactions/list?limit=50
{
  "user_id": "user_123"
}

# Export for training (bulk pagination)
POST /v1/retrievers/interactions/list?page=1&page_size=1000
{}
```

> [!WARNING]
>
> `limit`, `page`, `page_size`, `offset`, `cursor` and `after` are **query
> parameters** on this route. `ListInteractionsRequest` accepts `user_id`,
> `session_id`, `retriever_id`, `execution_id`, `feature_id`,
> `interaction_type`, `filters`, `search` and `sort`, and nothing else.
>
> A pagination key sent in the body is dropped rather than rejected, so the call
> returns 200 with the default page size. An export asking for `page_size: 1000`
> in the body silently gets the default instead, and the short result reads as
> "that is all the interactions there are."

## Reward Values and Auto-Tune

When a retriever uses [Auto-Tune](/docs/retrieval/auto-tune) (`fusion: "learned"`), every interaction automatically receives a computed `reward_value` stored in its metadata. This value is derived from the retriever's `reward_map` configuration at write time:

```json theme={null}
{
  "interaction_type": ["click", "purchase"],
  "metadata": {
    "reward_value": 3.0,
    "query": "wireless earbuds"
  }
}
```

In this example, the reward is determined by the interaction type with the largest absolute value: `purchase (3.0)` wins over `click (1.0)`, so `reward_value` is `3.0` (not summed). The reward value determines how much this interaction shifts the fusion weights for the associated feature.

**How it feeds into Auto-Tune:**

1. Each interaction records which feature URI surfaced the clicked result
2. The `reward_value` adjusts the Thompson Sampling Beta distribution for that feature
3. Positive rewards increase the feature's weight; negative rewards decrease it
4. The next search for this user reflects the updated weights

See [Reward Signals](/docs/retrieval/reward-signals) for the full default reward map, customization options, and examples for different use cases (e-commerce, content, internal search).

### Feature URI

The `feature_uri` field identifies which embedding model/feature produced the result the user interacted with. This is **required for Auto-Tune** — without it, the interaction cannot contribute to weight learning because the system doesn't know which feature to reward.

```json theme={null}
{
  "feature_id": "doc_product_789",
  "interaction_type": ["click"],
  "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
  "user_id": "user_456"
}
```

You can find the `feature_uri` for each result in the execution response's `learned_fusion_context.feature_uris` array, or extract it from the document's metadata. If `feature_uri` is set as a top-level field, it is automatically promoted into `metadata.feature_uri` for backward compatibility.

> [!TIP]
>
> The `session_id` field enables within-session adaptation. When provided, clicks in the current session influence the next search immediately via a session cache, without waiting for the next weight refresh.

## Privacy & Compliance

**GDPR Deletion**: Remove all interactions for a user to honor right-to-deletion requests.

```bash theme={null}
DELETE /v1/retrievers/interactions/<interaction_id>
```

**Anonymization**: Hash `user_id` before sending if you need consistent tracking without PII.

## Python SDK

```python Create Interaction theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="sk_...")

# After executing a retriever search
results = client.retrievers.execute(
    "ret_abc123",
    inputs={
        "query": "wireless headphones",
        "user_id": "user_456",
        "session_id": "sess_789",
    },
)

# Record a click on the 3rd result
client.retrievers.create_interaction(
    feature_id=results["documents"][2]["feature_id"],
    interaction_type=["click"],
    position=2,
    user_id="user_456",
    session_id="sess_789",
    retriever_id="ret_abc123"
)
```

```python Manage Interactions theme={null}
# List interactions for a retriever
interactions = client.retrievers.list_interactions("ret_abc123", page=1, page_size=50)

# Get a specific interaction
interaction = client.retrievers.get_interaction("int_abc123")

# Delete an interaction (GDPR right-to-deletion)
client.retrievers.delete_interaction("int_abc123")
```

## Bulk Backfill

Migrate existing interaction history into Mixpeek using the batch endpoint. Send up to 1,000 interactions per call with `occurred_at` timestamps so temporal decay weights them by their true age.

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/interactions/batch" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "interactions": [
      {
        "feature_id": "doc_001",
        "interaction_type": ["click"],
        "position": 0,
        "user_id": "user_123",
        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
        "occurred_at": "2026-01-15T10:30:00Z"
      },
      {
        "feature_id": "doc_002",
        "interaction_type": ["purchase"],
        "position": 2,
        "user_id": "user_123",
        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
        "occurred_at": "2026-01-16T14:00:00Z"
      }
    ]
  }'
```

```python Python SDK theme={null}
client.retrievers.backfill_interactions([
    {
        "feature_id": "doc_001",
        "interaction_type": ["click"],
        "position": 0,
        "user_id": "user_123",
        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
        "occurred_at": "2026-01-15T10:30:00Z",
    },
    {
        "feature_id": "doc_002",
        "interaction_type": ["purchase"],
        "position": 2,
        "user_id": "user_123",
        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
        "occurred_at": "2026-01-16T14:00:00Z",
    },
])
```

The response includes per-item results: `{created: 2, failed: 0, results: [{index: 0, interaction_id: "...", status: "created"}, ...]}`. Backfilled interactions bypass the real-time session cache, so they affect learned fusion weights only after the next weight refresh.

## Best Practices

1. **Always capture position** -- recorded for analytics and future position-aware modeling
2. **Store query text in metadata** -- enables query-document pair export for fine-tuning
3. **Track dwell time** -- separates genuine engagement (`long_view`) from accidental clicks
4. **Backfill with timestamps** -- use the [batch endpoint](#bulk-backfill) with `occurred_at` so temporal decay weights by true age
5. **Monitor weekly** -- set up dashboards to track CTR trends and catch relevance regressions early

> [!NOTE]
>
> See [Improve Relevance → Analytics](/docs/platform/improve-relevance#analytics) for dashboards and alerting on interaction-driven metrics like CTR, engagement rates, and query quality.


# Lineage Traversal
Source: https://docs.mixpeek.com/docs/retrieval/lineage-traversal

Walk a document's lineage chain — its source, provenance, and evidence trail from the original object through every transformation — efficiently, without N+1 round-trips.

Every Mixpeek document carries the full lineage chain that produced it,
from the original bucket object through every transformation. This guide
shows the four patterns for navigating that chain efficiently from the API.

> [!NOTE]
>
> Background on the lineage data model: see [Documents → Lineage](/docs/processing/extractors/document).
> The TL;DR is that each document has `_internal.lineage` with `root_object_id`,
> `root_bucket_id`, `source_document_id`, and a `chain` array recording every
> processing step.

## When to use what

| Pattern | Use case | Round-trips |
| - | - | - |
| `?expand=parent` | "Show me this scene and its source frame on one page" | 1 |
| `?expand=root_object` | "Show me this document with the original video metadata" | 1 |
| `?expand=ancestors` | "Show me the full pipeline that produced this document" | 1 |
| `?expand=children` | "Show me all the segments derived from this scene" | 1 |
| `GET /documents/{id}/ancestors` | Same as `expand=ancestors` but returns only the chain | 1 |
| `GET /documents/{id}/descendants` | Same as `expand=children` but returns only the children | 1 |
| `from_object` filter | "Search across everything derived from this video" | 1 (no GET first) |

The shared rule: never use a list response to grab IDs and then issue per-document
GETs. The `expand` parameter takes a comma-separated list, so a single request
fetches the document plus everything you need from its lineage tree.

## \$expand keywords

Lineage-aware `$expand` keywords resolve relative to a document's own
`_internal.lineage` block. They land under `_expanded.<keyword>` in the response,
matching the existing user-field expand shape.

**parent**

The single document referenced by `_internal.lineage.source_document_id`.

```bash cURL theme={null}
curl "$API/v1/collections/$COL/documents/$DOC?expand=parent" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NS"
```

```python Python theme={null}
client.documents.get(
    collection_identifier="col_scenes",
    document_id="doc_scene_42",
    expand="parent",
)
# Response includes:
# response._expanded.parent — the upstream frame document
```

For a tier-0 document (created directly from a bucket object), `parent` is
absent — there's no upstream document. Use `root_object` instead.

**root\_object**

The bucket object that started the lineage tree, read from the bucket's
object store.

```bash theme={null}
curl "$API/v1/collections/$COL/documents/$DOC?expand=root_object"
```

Useful for "show me this document with its source video filename and
upload metadata" without a separate `GET /v1/buckets/.../objects/{id}`.

**ancestors**

Every prior step in the chain, in order from the root through the
immediate parent. The document itself is excluded from the list.

```bash theme={null}
curl "$API/v1/collections/$COL/documents/$DOC?expand=ancestors"
```

For a document at tier 3, `ancestors` returns 3 elements: the tier-0,
tier-1, and tier-2 documents along the lineage path. Steps that have
no `document_id` (e.g., the bucket-object source step at tier 0) are
skipped — those don't correspond to fetchable documents.

**children**

Direct downstream documents (depth=1) — every document whose
`_internal.lineage.source_document_id` equals this document's ID.

Capped at **100 children per request**. If you need deeper traversal,
combine with the `from_object` filter (see below).

```bash theme={null}
curl "$API/v1/collections/$COL/documents/$DOC?expand=children"
```

You can request multiple keywords in one call by comma-separating them:

```bash theme={null}
curl "$API/v1/collections/$COL/documents/$DOC?expand=parent,root_object,children"
```

The same `expand` is accepted by `POST /documents/list` (in the request body)
and by retriever response shaping — the document GET endpoint is just the
simplest demonstration.

## Convenience endpoints

For SDKs and UIs that want only the lineage walk without fetching the
document itself, use the dedicated endpoints:

```bash theme={null}
# Returns the chain root → parent (excludes the document itself)
GET /v1/collections/{collection_identifier}/documents/{document_id}/ancestors

# Returns direct depth=1 children (max 100)
GET /v1/collections/{collection_identifier}/documents/{document_id}/descendants
```

Both endpoints return a `List<DocumentResponse>` with the same shape as
`GET /documents/{id}` per element.

## Filter aliases

When you want to *search* the lineage tree (find every document derived
from one root), use the filter aliases instead of expand. They work
in document list endpoints, retriever filter stages, and aggregations.

| Alias | Resolves to | Use for |
| - | - | - |
| `from_object` | `_internal.lineage.root_object_id` | Everything derived from this bucket object |
| `from_bucket` | `_internal.lineage.root_bucket_id` | Everything derived from this bucket |
| `from_document` | `_internal.lineage.source_document_id` | Direct children of one upstream document |
| `from_collection` | `_internal.lineage.source_collection_id` | Documents whose immediate parent was in this collection |

```json theme={null}
// "Show me all scene documents in col_scenes that came from this video"
{
  "AND": [
    { "field": "from_object", "operator": "eq", "value": "obj_video_123" }
  ]
}
```

```json theme={null}
// "Direct children of one specific frame document"
{
  "AND": [
    { "field": "from_document", "operator": "eq", "value": "doc_frame_42" }
  ]
}
```

These aliases are equivalent to the underscore-prefixed paths
(`_internal.lineage.*`) — they exist purely so you don't have to learn
the internal schema. Mix them freely with normal user fields:

```json theme={null}
{
  "AND": [
    { "field": "from_object", "operator": "eq", "value": "obj_video_123" },
    { "field": "metadata.scene_score", "operator": "gte", "value": 0.8 }
  ]
}
```

## End-to-end example: decomposition tree

To render a decomposition tree for one bucket object — every document at
every tier that descended from it — make one filtered list call per
collection in the namespace using `from_object`. The result is
already structured by collection, and each document's `_internal.lineage.chain`
tells you where to draw the edges.

```python theme={null}
def decomposition_tree(client, namespace, root_object_id):
    namespaces = {}
    for collection in client.collections.list(namespace=namespace):
        docs = client.documents.list(
            collection.collection_id,
            filters={
                "AND": [
                    {"field": "from_object", "operator": "eq", "value": root_object_id}
                ]
            },
        )
        if docs:
            namespaces[collection.collection_id] = docs
    return namespaces
```

For a deeper materialized view (the chain edges with parent/child resolved
inline), use the dedicated decomposition tree endpoint:

```bash theme={null}
GET /v1/buckets/{bucket_id}/objects/{object_id}/decomposition-tree
```

That endpoint pre-joins everything in one call and is what the Studio
namespace detail page uses to draw lineage diagrams.

## Limits & caveats

* **Maximum 50 unique user-field references per `expand` request** —
  doesn't apply to lineage keywords (those are bounded by chain length
  for `ancestors` and by the children cap for `children`).
* **`expand=children` is capped at 100 children per parent.** For deeper
  traversal or wider fan-out, fall back to a `from_document` filter.
* **Recursive expansion is not supported** — `expand=parent` resolves one
  level. To walk further, use `expand=ancestors` (full chain) or call
  `/ancestors` then re-`expand` from there.
* **Lineage is immutable provenance.** If an ancestor is deleted, its
  `document_id` reference in the chain remains. The `ancestors` expand
  silently skips unresolved references — never returns `null` slots — but
  client code should still be ready for shorter-than-expected chains.


# Multi-Stage Retrieval
Source: https://docs.mixpeek.com/docs/retrieval/multi-stage-deep-dive

The composable pipeline architecture that makes Mixpeek a warehouse, not a database

![Multi-stage retrieval: query flows through filter (WHERE), sort (ORDER BY), reduce (LIMIT), enrich (JOIN), apply (SELECT) stages to produce ranked results](https://mintcdn.com/mixpeek/Tp6Lmdmg2THegt2X/assets/mixpeek-multi-stage.svg?fit=max\&auto=format\&n=Tp6Lmdmg2THegt2X\&q=85\&s=d46d21cb4cb0c43489358435774cf437)

Single-query search returns a flat list of results ranked by one signal. That works for simple lookups. It falls apart the moment you need to combine signals, cross-reference collections, reshape output, or enforce business logic at query time. Multi-stage retrieval solves this by turning your search into a composable pipeline: a sequence of typed stages that filter, sort, reduce, enrich, and transform results in a single deterministic execution.

> [!NOTE]
>
> This is the definitive guide to multi-stage retrieval. For ready-to-copy pipeline configs, see the [Retrieval Cookbook](/docs/retrieval/cookbook). For the full stage catalog and parameter schemas, see [Retrievers](/docs/retrieval/retrievers).

> [!WARNING]
>
> Several pipelines on this page use a `score_linear` stage to blend multiple score
> fields with weights. That stage is not implemented. Sending a pipeline containing
> it returns HTTP 400.
>
> We left the examples in place because the pipelines around them are accurate and
> worth reading, and because swapping in a stage that merely looks plausible would
> be harder to spot. Treat every `score_linear` block as a sketch of intent.
>
> There is no weighted multi-field blend today, and no combination of existing
> stages produces one. The closest you can get is `score_normalize` to bring each
> score into a common range, then `sort_attribute` on the single field that matters
> most — you pick one signal, you do not blend them. The live stage list is at
> [Retrievers](/docs/retrieval/retrievers).

## Why Single-Query Search Isn't Enough

Traditional search systems give you one query, one index, one ranked list. This creates three problems that compound as your data grows:

**1. Signal collapse.** You want to find content that matches a face *and* contains a specific logo *and* has negative sentiment. A single vector query can only encode one of these signals. You end up running three separate queries and stitching results together in application code.

**2. N+1 enrichment.** After retrieving results, you need to join them with metadata from another collection, call an external API for licensing info, or classify each result against a taxonomy. Without pipeline-level enrichment, every result triggers a separate round-trip from your application.

**3. Brittle application logic.** Filtering, ranking, deduplication, and reshaping all live in your application layer. Every new use case means new glue code. Every change to ranking logic means a redeploy.

Multi-stage retrieval moves all of this into the retriever definition itself --- a declarative pipeline that the engine executes in a single pass.

## The SQL Analogy

If you know SQL, you already understand multi-stage retrieval. Each stage type maps to a SQL clause:

| Stage Type | SQL Equivalent | What It Does |
| - | - | - |
| **filter** | `WHERE` | Narrow the document set based on conditions --- semantic similarity, metadata predicates, feature thresholds |
| **sort** | `ORDER BY` | Reorder documents by score, attribute, or cross-encoder reranking |
| **reduce** | `LIMIT` / `GROUP BY` | Collapse results --- top-k sampling, deduplication, aggregation, summarization |
| **enrich** | `JOIN` | Add data from other collections, LLM-generated fields, or taxonomy classifications |
| **apply** | `SELECT` / `TRANSFORM` | Reshape output, call external APIs, execute custom code, run web searches |

A SQL query like:

```sql theme={null}
SELECT t.title, t.risk_score, r.license_type
FROM media_library t
JOIN rights_database r ON t.asset_id = r.asset_id
WHERE similarity(t.face_embedding, @query) > 0.72
  AND similarity(t.logo_embedding, @brand) > 0.6
ORDER BY t.risk_score DESC
LIMIT 10
```

Becomes a retriever pipeline:

```json theme={null}
{
  "stages": [
    {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.face_embedding}}", "top_k": 100, "min_score": 0.72}], "final_top_k": 100, "fusion": "rrf"}}},
    {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.brand_embedding}}", "top_k": 100, "min_score": 0.6}], "final_top_k": 100, "fusion": "rrf"}}},
    {"stage_name": "score_linear", "stage_type": "sort", "config": {"stage_id": "score_linear", "parameters": {"weights": {"risk_score": 1.0}}}},
    {"stage_name": "document_enrich", "stage_type": "enrich", "config": {"stage_id": "document_enrich", "parameters": {"target_collection_id": "col_rights", "source_field": "asset_id", "target_field": "asset_id"}}},
    {"stage_name": "sample", "stage_type": "reduce", "config": {"stage_id": "sample", "parameters": {"count": 10}}}
  ]
}
```

The difference: this pipeline works over multimodal embeddings, not just relational columns. You can filter on face vectors, sort by sentiment scores, and enrich with LLM-generated classifications --- all in one execution.

## The Five Stage Types

### Filter --- Narrow the Candidate Set

Filter stages reduce the number of documents flowing through the pipeline. They are the `WHERE` clause of your retrieval query. Every pipeline starts with at least one filter.

**Use filter stages to:**

* Run semantic similarity search against any extracted feature
* Apply metadata predicates (equality, range, set membership)
* Chain multiple filters for compound conditions (face match AND logo match AND date range)

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# Semantic search filter: find faces matching a reference
face_filter = {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
        "stage_id": "feature_search",
        "parameters": {
            "searches": [
                {
                    "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                    "query": "{{INPUT.face_embedding}}",
                    "top_k": 100,
                    "min_score": 0.72
                }
            ],
            "final_top_k": 100,
            "fusion": "rrf"
        }
    }
}

# Metadata filter: restrict to a date range
date_filter = {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
        "stage_id": "attribute_filter",
        "parameters": {
            "conditions": {
                "AND": [
                    {"field": "published_date", "operator": "gte", "value": "2025-01-01"},
                    {"field": "status", "operator": "eq", "value": "published"}
                ]
            }
        }
    }
}

# Chain them: both conditions must pass
retriever = client.retrievers.create(
    name="filtered-face-search",
    stages=[face_filter, date_filter]
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "filtered-face-search",
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": "{{INPUT.face_embedding}}",
                "top_k": 100,
                "min_score": 0.72
              }
            ],
            "final_top_k": 100,
            "fusion": "rrf"
          }
        }
      },
      {
        "stage_name": "attribute_filter",
        "stage_type": "filter",
        "config": {
          "stage_id": "attribute_filter",
          "parameters": {
            "conditions": {
              "AND": [
                {"field": "published_date", "operator": "gte", "value": "2025-01-01"},
                {"field": "status", "operator": "eq", "value": "published"}
              ]
            }
          }
        }
      }
    ]
  }'
```

> [!TIP]
>
> Chain multiple filter stages to express AND logic. Each successive filter operates on the output of the previous one, progressively narrowing the candidate set.

### Sort --- Control Ranking

Sort stages reorder the document set without adding or removing documents. They are the `ORDER BY` clause. Place them after filters to control which results appear first.

**Use sort stages to:**

* Apply weighted linear scoring across multiple signals
* Rerank results with a cross-encoder model for higher precision
* Sort by a metadata attribute (date, price, popularity)

```python Python theme={null}
# Weighted linear scoring across three signals
sort_stage = {
    "stage_name": "score_linear",
    "stage_type": "sort",
    "config": {
        "stage_id": "score_linear",
        "parameters": {
            "weights": {
                "audio.sentiment": 0.6,
                "recency": 0.3,
                "engagement": 0.1
            }
        }
    }
}

# Cross-encoder reranking for maximum precision
rerank_stage = {
    "stage_name": "cross_encoder_rerank",
    "stage_type": "sort",
    "config": {
        "stage_id": "rerank",
        "parameters": {
            "inference_name": "BAAI__bge_reranker_v2_m3",
            "query": "{{INPUT.query_text}}"
        }
    }
}
```

```bash cURL theme={null}
# Weighted linear scoring
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "scored-results",
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.query}}", "top_k": 100}], "final_top_k": 100, "fusion": "rrf"}
        }
      },
      {
        "stage_name": "score_linear",
        "stage_type": "sort",
        "config": {
          "stage_id": "score_linear",
          "parameters": {
            "weights": {
              "audio.sentiment": 0.6,
              "recency": 0.3,
              "engagement": 0.1
            }
          }
        }
      }
    ]
  }'
```

### Reduce --- Collapse and Limit

Reduce stages collapse the result set. They are the `LIMIT`, `GROUP BY`, and `DISTINCT` clauses. Use them to control result count, remove duplicates, or aggregate values.

**Use reduce stages to:**

* Sample the top-k results after sorting
* Deduplicate by a field (e.g., one result per source URL)
* Summarize results into an aggregated output

```python Python theme={null}
# Top-k sampling: keep the 10 highest-ranked results
sampling_stage = {
    "stage_name": "sample",
    "stage_type": "reduce",
    "config": {
        "stage_id": "sample",
        "parameters": {
            "count": 10
        }
    }
}

# Deduplication: one result per source domain
dedup_stage = {
    "stage_name": "dedup",
    "stage_type": "reduce",
    "config": {
        "stage_id": "deduplicate",
        "parameters": {
          "strategy": "field",
          "fields": ["metadata.source_url"]
        }
    }
}

# Combine: deduplicate first, then take top 10
retriever = client.retrievers.create(
    name="deduped-top-10",
    stages=[
        {"stage_name": "feature_search", "stage_type": "filter",
         "config": {"stage_id": "feature_search",
         "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.image}}", "top_k": 100}], "final_top_k": 100, "fusion": "rrf"}}},
        dedup_stage,
        sampling_stage
    ]
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "deduped-top-10",
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.image}}", "top_k": 100}], "final_top_k": 100, "fusion": "rrf"}
        }
      },
      {
        "stage_name": "dedup",
        "stage_type": "reduce",
        "config": {
          "stage_id": "deduplicate",
          "parameters": {"strategy": "field", "fields": ["metadata.source_url"]}
        }
      },
      {
        "stage_name": "sample",
        "stage_type": "reduce",
        "config": {
          "stage_id": "sample",
          "parameters": {"count": 10}
        }
      }
    ]
  }'
```

### Enrich --- Join External Knowledge

Enrich stages add data to each document without changing the result set size. They are the `JOIN` clause. Use them to attach metadata from other collections, generate LLM-powered annotations, or classify documents against taxonomies.

**Use enrich stages to:**

* Cross-collection joins (product data + catalog info + pricing)
* LLM enrichment (generate summaries, extract entities, assess risk)
* Taxonomy classification (label documents against a controlled vocabulary)

```python Python theme={null}
# Cross-collection join: attach rights/licensing data
rights_enrich = {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
        "stage_id": "document_enrich",
        "parameters": {
            "target_collection_id": "col_rights_database",
            "source_field": "metadata.asset_id",
            "target_field": "asset_id",
            "fields_to_merge": ["license_type", "expiry_date", "rights_holder"],
            "output_field": "rights_info"
        }
    }
}

# LLM enrichment: generate a risk assessment for each result
llm_enrich = {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
        "stage_id": "llm_enrich",
        "parameters": {
            "prompt": "Assess the IP risk level (low/medium/high) for this content based on the match confidence score {{DOC.score}} and rights status {{DOC.rights_info.license_type}}. Return a JSON object with 'risk_level' and 'reasoning' fields.",
            "output_field": "risk_assessment",
            "model": "gpt-4o-mini"
        }
    }
}

# Taxonomy classification: label by content category
taxonomy_enrich = {
    "stage_name": "taxonomy_enrich",
    "stage_type": "enrich",
    "config": {
        "stage_id": "taxonomy_enrich",
        "parameters": {
            "taxonomy_id": "tax_content_categories",
            "top_k": 5,
            "min_score": 0.5
        }
    }
}
```

```bash cURL theme={null}
# Cross-collection join
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "enriched-search",
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.audio}}", "top_k": 100, "min_score": 0.8}], "final_top_k": 100, "fusion": "rrf"}
        }
      },
      {
        "stage_name": "document_enrich",
        "stage_type": "enrich",
        "config": {
          "stage_id": "document_enrich",
          "parameters": {
            "target_collection_id": "col_rights_database",
            "source_field": "metadata.asset_id",
            "target_field": "asset_id",
            "fields_to_merge": ["license_type", "expiry_date", "rights_holder"],
            "output_field": "rights_info"
          }
        }
      }
    ]
  }'
```

> [!NOTE]
>
> Enrich stages execute per-document but are batched internally. A `document_enrich` join resolves all lookups in a single batch query to the target collection, not one query per document.

### Apply --- Transform and Reshape

Apply stages transform the structure or content of each document. They are the `SELECT` and function-call layer of your pipeline. Use them to reshape output for downstream consumers, call external APIs, execute custom code, or search the web.

**Use apply stages to:**

* Reshape JSON output with Jinja2 templates
* Call external APIs (Stripe, Salesforce, internal services)
* Execute custom Python/TypeScript/JavaScript in sandboxed environments
* Run web searches to augment results with external context

```python Python theme={null}
# JSON transform: reshape output for a frontend
json_transform = {
    "stage_name": "json_transform",
    "stage_type": "apply",
    "config": {
        "stage_id": "json_transform",
        "parameters": {
            "template": '{"id": "{{DOC.document_id}}", "title": "{{DOC.metadata.title}}", "risk": "{{DOC.risk_assessment.risk_level}}", "thumbnail": "{{DOC.metadata.thumbnail_url}}"}',
            "fail_on_error": False
        }
    }
}

# External API call: check licensing status
api_call = {
    "stage_name": "api_call",
    "stage_type": "apply",
    "config": {
        "stage_id": "api_call",
        "parameters": {
            "url": "https://licensing.internal/v1/check/{{DOC.metadata.asset_id}}",
            "method": "GET",
            "allowed_domains": ["licensing.internal"],
            "auth": {
                "type": "bearer",
                "secret_ref": "licensing_api_key"
            },
            "output_field": "metadata.license_check",
            "on_error": "skip"
        }
    }
}

# Custom code execution: compute a composite score
code_exec = {
    "stage_name": "code_execution",
    "stage_type": "apply",
    "config": {
        "stage_id": "code_execution",
        "parameters": {
            "language": "python",
            "code": "output = {'composite_score': doc['score'] * 0.7 + doc.get('metadata', {}).get('popularity', 0) * 0.3}",
            "output_field": "computed"
        }
    }
}
```

```bash cURL theme={null}
# JSON transform
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "transformed-output",
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.query}}", "top_k": 100}], "final_top_k": 100, "fusion": "rrf"}
        }
      },
      {
        "stage_name": "json_transform",
        "stage_type": "apply",
        "config": {
          "stage_id": "json_transform",
          "parameters": {
            "template": "{\"id\": \"{{DOC.document_id}}\", \"title\": \"{{DOC.metadata.title}}\"}",
            "fail_on_error": false
          }
        }
      }
    ]
  }'
```

## Building Multi-Stage Pipelines

The power of multi-stage retrieval is in composition. Here are three production pipelines that demonstrate how stages chain together to solve complex problems that no single query can address.

### Pipeline 1: Brand Safety Scanner

**Problem:** A media company needs to find scenes where their talent appears near competitor products in negative-sentiment content --- before the content goes live.

**Pipeline logic:** Find faces matching talent roster, then check for competitor logos in the same scenes, rank by sentiment risk, take the worst offenders, and attach brand safety context.

```python Python theme={null}
retriever = client.retrievers.create(
    name="brand-safety-scanner",
    namespace="media-library",
    stages=[
        # Stage 1: Find scenes containing talent faces
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "searches": [
                        {
                            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                            "query": "{{INPUT.talent_embedding}}",
                            "top_k": 500,
                            "min_score": 0.72
                        }
                    ],
                    "final_top_k": 500,
                    "fusion": "rrf"
                }
            }
        },
        # Stage 2: Narrow to scenes that also contain competitor logos
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "searches": [
                        {
                            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                            "query": "{{INPUT.competitor_logo_embedding}}",
                            "top_k": 500,
                            "min_score": 0.65
                        }
                    ],
                    "final_top_k": 500,
                    "fusion": "rrf"
                }
            }
        },
        # Stage 3: Rank by weighted risk (sentiment + recency + engagement)
        {
            "stage_name": "score_linear",
            "stage_type": "sort",
            "config": {
                "stage_id": "score_linear",
                "parameters": {
                    "weights": {
                        "audio.sentiment": 0.6,
                        "recency": 0.3,
                        "engagement": 0.1
                    }
                }
            }
        },
        # Stage 4: Take the 10 highest-risk scenes
        {
            "stage_name": "sample",
            "stage_type": "reduce",
            "config": {
                "stage_id": "sample",
                "parameters": {"count": 10}
            }
        },
        # Stage 5: Attach brand safety scores from reference collection
        {
            "stage_name": "document_enrich",
            "stage_type": "enrich",
            "config": {
                "stage_id": "document_enrich",
                "parameters": {
                    "target_collection_id": "col_brand_safety_scores",
                    "source_field": "metadata.content_id",
                    "target_field": "content_id",
                    "fields_to_merge": ["safety_rating", "advertiser_category", "risk_flags"],
                    "output_field": "brand_context"
                }
            }
        }
    ]
)

# Execute the pipeline
results = client.retrievers.execute(
    retriever_id=retriever.id,
    inputs={
        "talent_embedding": celebrity_face_vector,
        "competitor_logo_embedding": competitor_logo_vector
    }
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "brand-safety-scanner",
    "stages": [
      {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.talent_embedding}}", "top_k": 500, "min_score": 0.72}], "final_top_k": 500, "fusion": "rrf"}}},
      {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.competitor_logo_embedding}}", "top_k": 500, "min_score": 0.65}], "final_top_k": 500, "fusion": "rrf"}}},
      {"stage_name": "score_linear", "stage_type": "sort", "config": {"stage_id": "score_linear", "parameters": {"weights": {"audio.sentiment": 0.6, "recency": 0.3, "engagement": 0.1}}}},
      {"stage_name": "sample", "stage_type": "reduce", "config": {"stage_id": "sample", "parameters": {"count": 10}}},
      {"stage_name": "document_enrich", "stage_type": "enrich", "config": {"stage_id": "document_enrich", "parameters": {"target_collection_id": "col_brand_safety_scores", "source_field": "metadata.content_id", "target_field": "content_id", "fields_to_merge": ["safety_rating", "advertiser_category", "risk_flags"], "output_field": "brand_context"}}}
    ]
  }'
```

**Stage flow:** 500 face matches --> \~50 with competitor logos --> sorted by risk --> top 10 --> enriched with brand context

***

### Pipeline 2: IP Clearance Pipeline

**Problem:** Before publishing new content, a legal team needs to check it against a database of copyrighted material across audio fingerprints, visual similarity, and metadata --- then attach licensing information for review.

**Pipeline logic:** Match audio fingerprints, check visual similarity for the same assets, filter by rights status, sort by match confidence, and attach the full licensing record.

```python Python theme={null}
retriever = client.retrievers.create(
    name="ip-clearance-pipeline",
    namespace="rights-catalog",
    stages=[
        # Stage 1: Audio fingerprint matching against known works
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "searches": [
                        {
                            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                            "query": "{{INPUT.audio_fingerprint}}",
                            "top_k": 200,
                            "min_score": 0.8
                        }
                    ],
                    "final_top_k": 200,
                    "fusion": "rrf"
                }
            }
        },
        # Stage 2: Visual similarity check on the same content
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "searches": [
                        {
                            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                            "query": "{{INPUT.visual_frames}}",
                            "top_k": 200,
                            "min_score": 0.7
                        }
                    ],
                    "final_top_k": 200,
                    "fusion": "rrf"
                }
            }
        },
        # Stage 3: Exclude already-licensed content
        {
            "stage_name": "attribute_filter",
            "stage_type": "filter",
            "config": {
                "stage_id": "attribute_filter",
                "parameters": {
                    "conditions": {
                        "AND": [
                            {"field": "license_status", "operator": "ne", "value": "cleared"}
                        ]
                    }
                }
            }
        },
        # Stage 4: Rank by match confidence weighted with rights severity
        {
            "stage_name": "score_linear",
            "stage_type": "sort",
            "config": {
                "stage_id": "score_linear",
                "parameters": {
                    "weights": {
                        "match_confidence": 0.8,
                        "rights_severity": 0.2
                    }
                }
            }
        },
        # Stage 5: Attach full licensing records for legal review
        {
            "stage_name": "document_enrich",
            "stage_type": "enrich",
            "config": {
                "stage_id": "document_enrich",
                "parameters": {
                    "target_collection_id": "col_licensing_records",
                    "source_field": "metadata.rights_id",
                    "target_field": "rights_id",
                    "fields_to_merge": ["rights_holder", "license_type", "territory", "expiry_date", "contact_email"],
                    "output_field": "licensing"
                }
            }
        },
        # Stage 6: LLM-generated risk summary for each match
        {
            "stage_name": "llm_enrich",
            "stage_type": "enrich",
            "config": {
                "stage_id": "llm_enrich",
                "parameters": {
                    "prompt": "Based on the match confidence ({{DOC.score}}) and licensing status ({{DOC.licensing.license_type}}), provide a one-sentence risk assessment and recommended action (clear/review/block).",
                    "output_field": "legal_summary",
                    "model": "gpt-4o-mini"
                }
            }
        }
    ]
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "ip-clearance-pipeline",
    "stages": [
      {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.audio_fingerprint}}", "top_k": 200, "min_score": 0.8}], "final_top_k": 200, "fusion": "rrf"}}},
      {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.visual_frames}}", "top_k": 200, "min_score": 0.7}], "final_top_k": 200, "fusion": "rrf"}}},
      {"stage_name": "attribute_filter", "stage_type": "filter", "config": {"stage_id": "attribute_filter", "parameters": {"conditions": {"AND": [{"field": "license_status", "operator": "ne", "value": "cleared"}]}}}},
      {"stage_name": "score_linear", "stage_type": "sort", "config": {"stage_id": "score_linear", "parameters": {"weights": {"match_confidence": 0.8, "rights_severity": 0.2}}}},
      {"stage_name": "document_enrich", "stage_type": "enrich", "config": {"stage_id": "document_enrich", "parameters": {"target_collection_id": "col_licensing_records", "source_field": "metadata.rights_id", "target_field": "rights_id", "fields_to_merge": ["rights_holder", "license_type", "territory", "expiry_date", "contact_email"], "output_field": "licensing"}}},
      {"stage_name": "llm_enrich", "stage_type": "enrich", "config": {"stage_id": "llm_enrich", "parameters": {"prompt": "Based on the match confidence ({{DOC.score}}) and licensing status ({{DOC.licensing.license_type}}), provide a one-sentence risk assessment and recommended action (clear/review/block).", "output_field": "legal_summary", "model": "gpt-4o-mini"}}}
    ]
  }'
```

**Stage flow:** 200 audio matches --> \~30 with visual matches --> exclude cleared --> sorted by risk --> licensing data attached --> LLM risk summary generated

***

### Pipeline 3: Content Moderation

**Problem:** A platform needs to scan user-uploaded content across multiple safety dimensions (NSFW, violence, toxicity), aggregate risk scores, and route flagged content to a moderation queue.

**Pipeline logic:** Filter for NSFW content above threshold, check text toxicity, sort by combined risk, take the worst offenders, classify against a moderation taxonomy, and push to the review queue.

```python Python theme={null}
retriever = client.retrievers.create(
    name="content-moderation",
    namespace="user-uploads",
    stages=[
        # Stage 1: Flag visually unsafe content
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "searches": [
                        {
                            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                            "query": "{{INPUT.unsafe_reference}}",
                            "top_k": 1000,
                            "min_score": 0.6
                        }
                    ],
                    "final_top_k": 1000,
                    "fusion": "rrf"
                }
            }
        },
        # Stage 2: Check text-based toxicity in the same content
        {
            "stage_name": "feature_search",
            "stage_type": "filter",
            "config": {
                "stage_id": "feature_search",
                "parameters": {
                    "searches": [
                        {
                            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                            "query": {"input_mode": "text", "value": "{{INPUT.toxic_terms}}"},
                            "top_k": 1000,
                            "min_score": 0.5
                        }
                    ],
                    "final_top_k": 1000,
                    "fusion": "rrf"
                }
            }
        },
        # Stage 3: Aggregate risk signals into a combined score
        {
            "stage_name": "score_linear",
            "stage_type": "sort",
            "config": {
                "stage_id": "score_linear",
                "parameters": {
                    "weights": {
                        "nsfw_score": 0.4,
                        "violence_score": 0.3,
                        "toxicity_score": 0.3
                    }
                }
            }
        },
        # Stage 4: Take the top 50 highest-risk items
        {
            "stage_name": "sample",
            "stage_type": "reduce",
            "config": {
                "stage_id": "sample",
                "parameters": {"count": 50}
            }
        },
        # Stage 5: Classify against moderation taxonomy
        {
            "stage_name": "taxonomy_enrich",
            "stage_type": "enrich",
            "config": {
                "stage_id": "taxonomy_enrich",
                "parameters": {
                    "taxonomy_id": "tax_moderation_categories",
                    "top_k": 5,
                    "min_score": 0.5
                }
            }
        },
        # Stage 6: Push to external moderation queue
        {
            "stage_name": "api_call",
            "stage_type": "apply",
            "config": {
                "stage_id": "api_call",
                "parameters": {
                    "url": "https://moderation.internal/v1/review-queue",
                    "method": "POST",
                    "allowed_domains": ["moderation.internal"],
                    "auth": {
                        "type": "bearer",
                        "secret_ref": "moderation_api_key"
                    },
                    "output_field": "metadata.review_ticket",
                    "on_error": "skip"
                }
            }
        }
    ]
)
```

```bash cURL theme={null}
curl -X POST https://api.mixpeek.com/v1/retrievers \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "content-moderation",
    "stages": [
      {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding", "query": "{{INPUT.unsafe_reference}}", "top_k": 1000, "min_score": 0.6}], "final_top_k": 1000, "fusion": "rrf"}}},
      {"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.toxic_terms}}"}, "top_k": 1000, "min_score": 0.5}], "final_top_k": 1000, "fusion": "rrf"}}},
      {"stage_name": "score_linear", "stage_type": "sort", "config": {"stage_id": "score_linear", "parameters": {"weights": {"nsfw_score": 0.4, "violence_score": 0.3, "toxicity_score": 0.3}}}},
      {"stage_name": "sample", "stage_type": "reduce", "config": {"stage_id": "sample", "parameters": {"count": 50}}},
      {"stage_name": "taxonomy_enrich", "stage_type": "enrich", "config": {"stage_id": "taxonomy_enrich", "parameters": {"taxonomy_id": "tax_moderation_categories", "top_k": 5, "min_score": 0.5}}},
      {"stage_name": "api_call", "stage_type": "apply", "config": {"stage_id": "api_call", "parameters": {"url": "https://moderation.internal/v1/review-queue", "method": "POST", "allowed_domains": ["moderation.internal"], "auth": {"type": "bearer", "secret_ref": "moderation_api_key"}, "output_field": "metadata.review_ticket", "on_error": "skip"}}}
    ]
  }'
```

**Stage flow:** 1000 NSFW candidates --> \~200 also toxic --> sorted by combined risk --> top 50 --> taxonomy labels attached --> pushed to moderation queue

## Performance Characteristics

Multi-stage pipelines avoid the N+1 problem that plagues application-level orchestration. Here is how:

**1. Filter stages execute server-side against indexes.** A `feature_search` filter runs directly against the MVS vector index. No data leaves the engine until the candidate set is narrowed. Chaining two filter stages does not mean two round-trips from your application --- both execute within the engine in sequence.

**2. Enrich stages batch internally.** A `document_enrich` join across 50 results resolves in a single batch query to the target collection, not 50 separate lookups. LLM enrichment stages batch prompts where possible.

**3. Reduce stages shrink the working set early.** Place a `sample` or `deduplicate` stage as early as possible to minimize the number of documents flowing through expensive downstream stages (LLM enrichment, API calls).

**4. The pipeline streams, not materializes.** Documents flow through stages incrementally. A 6-stage pipeline does not create 6 intermediate copies of the full result set. Each stage processes and passes documents forward.

> [!WARNING]
>
> Stage ordering matters for performance. Place cheap, high-selectivity filters first (metadata filters, feature searches with high thresholds) and expensive stages last (LLM enrichment, external API calls). A pipeline that enriches 1000 documents and then filters to 10 is dramatically slower than one that filters to 10 and then enriches.

## When to Use Which Stage Type

Use this decision guide when designing your pipeline:

> **I need to narrow down results based on content or metadata**
>
> Use a **filter** stage. Start with `feature_search` for semantic/vector-based filtering, or `attribute_filter` for structured attribute filtering. Chain multiple filters for compound conditions.

> **I need to reorder results by relevance or business logic**
>
> Use a **sort** stage. Choose `rerank` for high-precision reranking with a cross-encoder model, `sort_relevance` to order by search relevance score, or `sort_attribute` for simple field-based ordering.

> **I need fewer results, or deduplicated results**
>
> Use a **reduce** stage. Choose `limit` for top-k limits, `sample` for random sampling, `deduplicate` for deduplication by field, or `summarize` for LLM-powered aggregation of results into a single summary.

> **I need to add data from another collection, an LLM, or a taxonomy**
>
> Use an **enrich** stage. Choose `document_enrich` for cross-collection joins, `llm_enrich` for AI-generated fields, or `taxonomy_enrich` for classification against a controlled vocabulary.

> **I need to reshape output, call an API, or run custom logic**
>
> Use an **apply** stage. Choose `json_transform` for output reshaping, `api_call` for external service integration, `code_execution` for custom Python/TypeScript/JavaScript, or `external_web_search` for web augmentation.

### Stage Ordering Rules of Thumb

1. **Filter first.** Every pipeline should start with one or more filter stages to narrow the candidate set.
2. **Sort second.** Apply ranking after filtering so you are sorting a smaller set.
3. **Reduce third.** Cut the result set to a manageable size before enrichment.
4. **Enrich fourth.** Add external data only to the documents that survived filtering, sorting, and reduction.
5. **Apply last.** Reshape output and trigger side effects at the end of the pipeline.

> [!NOTE]
>
> These are guidelines, not hard rules. Some pipelines benefit from enriching before sorting (e.g., sort by a field that only exists after enrichment). Design your pipeline around your data flow, not a rigid template.

## Related Resources

> **[Retrievers](/docs/retrieval/retrievers)**
>
> Full stage catalog, parameter schemas, and retriever configuration reference

> **[Retrieval Cookbook](/docs/retrieval/cookbook)**
>
> Ready-to-copy pipeline configurations for common use cases

> **[Stage Reference](/docs/retrieval/retrievers#stage-catalog)**
>
> Detailed documentation for every stage type and stage ID

> **[Caching](/docs/best-practices/caching-strategies)**
>
> Configure retriever-level caching for repeated queries


# Query Optimization & Explain
Source: https://docs.mixpeek.com/docs/retrieval/query-optimization

How Mixpeek automatically optimizes retriever pipelines, and how to inspect the execution plan with explain

Mixpeek optimizes every retriever before it runs — reordering, fusing, and pushing work down into the vector store — then lets you inspect exactly what it did with the **explain** endpoint. You write the pipeline you find readable; the optimizer makes it fast.

## Automatic optimizations

When you execute a retriever, the planner rewrites your stage list before execution. These transformations are automatic — you don't configure them:

| Optimization | What it does | Why it helps |
| - | - | - |
| **Filter push-down** | Moves attribute filters ahead of vector search | Shrinks the candidate set before the expensive embedding search runs |
| **Stage fusion** | Merges adjacent compatible stages into one | Fewer passes over the result set |
| **Grouping optimization** | Rewrites group/reduce stages to run database-side | Avoids materializing intermediate results |
| **Computation push-down** | Runs data-plane stages (`feature_search`, `attribute_filter`, `sort_attribute`, `aggregate`) **inside MVS** | Eliminates a network round-trip and lets the vector store filter/sort where the data lives |
| **Parallel sub-queries** | Runs independent operations (search + count, search + facet) concurrently | Lower wall-clock latency |
| **Over-fetch hints** | Fetches extra candidates when a later stage will filter them out | Preserves recall after post-filtering |

> [!TIP]
>
> Because the optimizer pushes filters down for you, **write filters wherever they read most clearly** — you don't need to hand-order stages for performance. Use `explain` to confirm what was pushed.

The retriever is also fetched and optimized **once** per request, then reused across a batch — so `POST /v1/retrievers/{id}/execute/batch` amortizes planning across all queries.

## Inspect the plan with explain

`POST /v1/retrievers/{retriever_id}/execute/explain` returns the **optimized** execution plan without running the query — per-stage cost and latency estimates, bottlenecks, and exactly which optimizations were applied. Pass hypothetical `inputs` to see how the plan changes with different parameters.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute/explain" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "inputs": { "query": "people discussing electric vehicles" } }'
```

```json Example response theme={null}
{
  "retriever_id": "ret_abc123",
  "execution_plan": [
    {
      "stage_index": 0,
      "stage_name": "attribute_filter",
      "stage_type": "filter",
      "estimated_input": 10000,
      "estimated_output": 5000,
      "estimated_efficiency": 0.5,
      "estimated_cost_credits": 0.01,
      "estimated_duration_ms": 20,
      "cache_likely": true,
      "optimization_notes": ["Pushed down from stage 2"],
      "warnings": []
    },
    {
      "stage_index": 1,
      "stage_name": "feature_search",
      "stage_type": "filter",
      "estimated_input": 5000,
      "estimated_output": 100,
      "estimated_efficiency": 0.02,
      "estimated_cost_credits": 0.5,
      "estimated_duration_ms": 200,
      "cache_likely": false,
      "optimization_notes": [],
      "warnings": ["High cost stage - consider reducing top_k"]
    }
  ],
  "estimated_cost": { "total_credits": 0.51, "total_duration_ms": 220 },
  "bottleneck_stages": ["feature_search"],
  "optimization_applied": true,
  "optimization_details": {
    "original_stage_count": 3,
    "optimized_stage_count": 2,
    "stage_reduction_pct": 33.3,
    "decisions": [
      {
        "rule_type": "push_down_filters",
        "applied": true,
        "reason": "Moved attribute_filter before feature_search to reduce search scope"
      }
    ]
  },
  "optimization_suggestions": [
    { "type": "reduce_limit", "stage": "feature_search", "message": "Consider reducing top_k to improve latency" }
  ]
}
```

> [!NOTE]
>
> `estimated_cost_credits` and `total_credits` are legacy fields expressed in the internal ledger unit (1 credit = \$0.001). Customer-facing pricing is in dollars — see [Billing](/docs/platform/billing).

### How to read it

| Field | Use it to… |
| - | - |
| `execution_plan[].estimated_input/output` | See how each stage narrows the set — a stage that barely reduces the set may be unnecessary |
| `estimated_efficiency` | Spot low-selectivity stages (close to 1.0 = passes almost everything through) |
| `estimated_cost_credits` / `estimated_duration_ms` | Budget in dollars before running (1 credit = \$0.001); find the expensive stage |
| `bottleneck_stages` | The stages dominating latency — optimize these first |
| `cache_likely` | Whether a stage will likely hit the [cache](/docs/resources/best-practices) |
| `optimization_details.decisions` | Exactly which automatic rewrites fired (and why) |
| `optimization_suggestions` | Concrete, actionable tuning hints |
| `warnings` | Per-stage red flags (e.g. high-cost stage, overly broad `top_k`) |

> [!NOTE]
>
> The `execution_plan` reflects the **optimized** pipeline, not your original stage list. Compare `optimization_details.original_stage_count` vs `optimized_stage_count` to see how much the planner collapsed.

### Profile a run that already happened

`explain` estimates a plan without running it. To see what an execution actually
did, ask for the profile.

Set `"explain": true` in an execute body and the response carries `explain_plan`
alongside the documents: per-stage timings with input and output counts, MVS
execution stats, and the optimizer summary for that run.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "inputs": { "query": "electric vehicles" }, "explain": true,
        "pagination": { "method": "offset", "page_size": 10, "page_number": 1 } }'
```

Read the same profile back later by execution id, without re-running anything:

```bash theme={null}
curl -sS "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/executions/$EXECUTION_ID/profile" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

| | Runs the query | Answers about |
| - | - | - |
| `POST /{id}/execute/explain` | No | what the optimizer *would* do |
| `execute` with `"explain": true` | Yes | what that run *did*, inline |
| `GET /{id}/executions/{eid}/profile` | No | what a past run did, by id |

> [!WARNING]
>
> The profile response is **untyped** in the OpenAPI spec, so a generated client
> gives you an open dict rather than named fields. Read keys defensively.
>
> It reads the `query_profile` persisted on the execution record, so an execution
> run **without** `explain` has no profile to return. Ask for `explain: true` at
> execute time if you intend to profile the run later.

## Typical workflow

**1. Explain before you ship**

Run `explain` with representative inputs to see estimated cost, bottlenecks, and applied optimizations.

**2. Act on bottlenecks & suggestions**

Reduce `top_k` on high-cost searches, add a selective `attribute_filter` (the optimizer pushes it down), or drop low-selectivity stages.

**3. Verify in production**

Use [retriever analytics](/docs/platform/operations) to confirm real latency and cache-hit rates match the estimate.

## Related

* [Multi-Stage Retrieval](/docs/retrieval/multi-stage-deep-dive) — how stages compose
* [Feature Search](/docs/retrieval/stages/feature-search) — the most common bottleneck stage
* [Evaluations](/docs/retrieval/evaluations) — measure quality alongside cost
* [Best Practices](/docs/resources/best-practices) — caching and cost optimization


# Deep Research Patterns
Source: https://docs.mixpeek.com/docs/retrieval/research

Compose multi-stage retrievers for investigations, literature reviews, and analysis

Deep research workflows orchestrate multiple retriever executions, enrichment passes, and synthesis steps to answer complex questions. Mixpeek's stage catalog—filter, sort, reduce, apply, enrich—gives you the primitives to build these flows without bespoke infrastructure.

## Building Blocks

| Stage | Examples | Use in Research |
| - | - | - |
| Search (filter) | `feature_search`, `query_expand` | Gather candidate documents across modalities; expand queries for recall |
| Narrow (filter) | `attribute_filter`, `llm_filter` | Restrict to relevant time ranges, entities, or sentiment |
| Web (apply) | `external_web_search`, `web_scrape` | Pull public sources from the open web |
| Enrich | `document_enrich`, `taxonomy_enrich`, `llm_enrich` | Attach related docs, taxonomy tags, or LLM-extracted facts |
| Synthesize (reduce) | `summarize` | Aggregate results into a single synthesized brief with citations |
| Compose (apply) | `api_call`, `cross_compare` | Call external services (e.g., fact-check APIs) or compare across collections |

## Common Patterns

### Literature Review

1. **Seed search** using `feature_search` to retrieve recent papers across text and figures.
2. **Narrow** by publication date and venue with `attribute_filter`.
3. **Classify** by research area with `taxonomy_enrich`.
4. **Synthesize** findings with citations using `summarize`.
5. Store summaries alongside `feature_id` references for auditability.

### Competitive Intelligence

1. Use `external_web_search` + `web_scrape` stages to pull public announcements.
2. Join with internal product docs via `document_enrich` to compare specs.
3. Apply an `attribute_filter` to spotlight price or feature gaps.
4. Generate a briefing memo with the `summarize` stage.

### Incident Investigation

1. Collect relevant runbooks/logs via `feature_search` over internal collections.
2. Use `attribute_filter` to isolate the incident window.
3. Enrich with taxonomy-based tags (`taxonomy_enrich`) for impacted systems.
4. Summarize timeline and root cause via `summarize`, keeping citations.

## Orchestrating Multi-Retriever Flows

Use the `document_enrich` stage to call a sub-retriever based on previous stage output:

```json theme={null}
{
  "stage_name": "retriever",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "retriever_id": "ret_internal_logs",
      "retriever_inputs": {
        "query_text": "{{INPUT.primary_question}}",
        "time_range": "{{STAGE.filter.time_range}}"
      },
      "strategy": "append"
    }
  }
}
```

This pattern lets you create macro retrievers that orchestrate domain-specific sub-searches, enabling modular reuse.

## Capturing Feedback

* Record user signals with the [Interactions API](/docs/retrieval/interactions) (`click`, `long_view`, `positive_feedback`, etc.).
* Feed interactions back into [auto-tuning](/docs/retrieval/auto-tune) or `attribute_filter` stages ("hide documents seen in this session").
* Use [Observability](/docs/operations/observability) to optimize parameter choices (e.g., increase a `feature_search` stage's `final_top_k` if users often tap beyond the top 10).

## Operational Tips

1. **Persist execution IDs** – each `execute` response includes an execution id; link it to your research session for audit trails.
2. **Monitor stage telemetry** – `stage_statistics` identifies bottlenecks (e.g., LLM stages dominating latency).
3. **Budget controls** – set `budget_limits` on retrievers to cap time or spend for exploratory workflows.
4. **Cache intermediate results** – cache expensive discovery steps, especially when analysts reiterate queries.
5. **Use tasks** – schedule enrichment batches (clusters, taxonomies) ahead of time so research pipelines stay low-latency.

## Suggested Architecture

```
Orchestration App
 ├─ Calls macro retriever (with document_enrich compose stages)
 ├─ Logs execution IDs + user prompts
 ├─ Stores generated summaries & citations
 └─ Sends interactions back to Mixpeek
```

Behind the scenes, Mixpeek handles stage execution, caching, and lineage tracking. You focus on stitching together the right stages and presenting the synthesized output.

## Next Steps

* Review [Retrievers](/docs/retrieval/retrievers) for stage configuration details.
* Learn how [Filters](/docs/retrieval/filters) and [Taxonomies](/docs/enrichment/taxonomies) contribute structure to exploratory pipelines.
* Use [Operations → Observability](/docs/operations/observability) to monitor research workloads in production.


# Reward Signals
Source: https://docs.mixpeek.com/docs/retrieval/reward-signals

Configure how different interaction types influence learned fusion weights

When a user interacts with a search result, that interaction carries a **reward value** that adjusts the learned fusion weights. A purchase is a stronger signal than a click; negative feedback is a penalty. The reward map controls these magnitudes.

## Default Reward Map

If you do not provide a custom `reward_map` in `learning_config`, the system uses these defaults:

| Interaction Type | Default Reward | Signal Strength | Description |
| - | :-: | - | - |
| `impression` | `0.0` | Neutral | Result was rendered on screen (passive signal) |
| `view` | `0.0` | Neutral | User viewed a result (not in default reward map) |
| `click` | `1.0` | Moderate positive | User clicked a result |
| `dwell` | `0.5` | Weak positive | User lingered on a result |
| `long_view` | `1.0` | Moderate positive | Sustained engagement (dwell time > 30s) |
| `purchase` | `3.0` | Strong positive | Conversion event |
| `add_to_cart` | `2.0` | Positive | Intent to purchase |
| `wishlist` | `0.0` | Neutral | User added to wishlist (not in default reward map) |
| `bookmark` | `1.5` | Positive | User saved for later |
| `share` | `1.5` | Positive | User shared the result |
| `positive_feedback` | `2.0` | Strong positive | Explicit thumbs up |
| `negative_feedback` | `-2.0` | Strong negative | Explicit thumbs down |
| `query_refinement` | `0.0` | Neutral | User modified their search (not in default reward map) |
| `zero_results` | `0.0` | Neutral | Query yielded no results (not in default reward map) |
| `filter_toggle` | `0.0` | Neutral | User modified filters (not in default reward map) |
| `skip` | `-0.5` | Weak negative | Result was shown but ignored |
| `return_to_results` | `-0.5` | Weak negative | User bounced back quickly |

Interaction types not listed in the reward map contribute a reward of `0.0` — they are recorded but do not influence fusion weights.

## Custom Reward Maps

Override the defaults by setting `reward_map` in `learning_config`:

```json theme={null}
{
  "fusion": "learned",
  "learning_config": {
    "context_features": ["INPUT.user_id"],
    "reward_map": {
      "click": 1.0,
      "purchase": 5.0,
      "add_to_cart": 2.5,
      "positive_feedback": 3.0,
      "negative_feedback": -3.0,
      "skip": -1.0
    }
  }
}
```

> [!NOTE]
>
> When you provide a custom `reward_map`, it **replaces** the defaults entirely. Only interaction types present in your map will influence fusion weights. Include every type you want to count.

The reward value is computed at interaction-write time and stored as `reward_value` in the interaction metadata. When an interaction has multiple types, the reward with the largest absolute value is used (not summed). For example, `['click', 'purchase']` yields `3.0` (the purchase reward), not `4.0`. This means changing the `reward_map` only affects future interactions — previously recorded interactions retain their original reward values.

## Negative Signals

Negative rewards (`negative_feedback`, `skip`, `return_to_results`) penalize the feature that surfaced the result. Mechanically, a negative reward increments the Beta distribution's `beta` parameter, making it less likely that the associated feature receives high weight in future queries:

```
positive reward → alpha += reward       → feature weight trends up
negative reward → beta  += abs(reward)  → feature weight trends down
```

> [!WARNING]
>
> Negative signals should generally have smaller absolute values than positive signals. A single `negative_feedback: -5.0` would outweigh five `click: 1.0` interactions, which can cause rapid weight swings. Start conservative and tune based on evaluation results.

## Position Bias

Results shown at position 0 get clicked more often than results at position 10, regardless of relevance. This is **position bias** — a well-known problem in learning-to-rank systems.

Auto-Tune records the `position` field on every interaction for analytics and audit. Reward computation does **not** weight interactions by position, so a click at position 8 earns the same reward value as a click at position 0.

> [!TIP]
>
> Include `position` when posting interactions. It is stored alongside each interaction and available to your own analytics, and recording it from the start saves you a backfill.

## Temporal Decay

User preferences change over time. Auto-Tune applies exponential decay to older interactions so that recent behavior matters more:

```
effective_reward = reward * (decay_factor ^ days_ago)
```

| `decay_factor` | After 30 days | After 90 days | After 180 days | After 365 days |
| :-: | :-: | :-: | :-: | :-: |
| `1.0` (no decay) | 100% | 100% | 100% | 100% |
| `0.999` | 97% | 91% | 84% | 69% |
| `0.995` | 86% | 64% | 41% | 16% |
| `0.990` | 74% | 41% | 17% | 3% |
| `0.980` | 55% | 16% | 3% | \~0% |

Configure via `learning_config`:

```json theme={null}
{
  "learning_config": {
    "decay_factor": 0.995,
    "decay_window_days": 365
  }
}
```

Interactions older than `decay_window_days` are ignored entirely (not just decayed to near-zero, but excluded from the aggregation query).

### Backfilling historical interactions

By default the server timestamps each interaction at the moment it's recorded. If you're **migrating existing click/purchase logs** into Mixpeek, pass `occurred_at` (ISO 8601) so temporal decay weights each interaction by its *true* age instead of treating everything as brand-new:

```json theme={null}
POST /v1/retrievers/interactions
{
  "feature_id": "doc_123",
  "interaction_type": ["purchase"],
  "position": 0,
  "user_id": "user_42",
  "feature_uri": "mixpeek://text_extractor@v1/...",
  "occurred_at": "2026-01-15T10:30:00Z"
}
```

```python Python SDK theme={null}
client.retrievers.create_interaction(
    feature_id="doc_123",
    interaction_type=["purchase"],
    user_id="user_42",
    feature_uri="mixpeek://text_extractor@v1/...",
    occurred_at="2026-01-15T10:30:00Z",  # backfill with the real timestamp
)
```

> [!NOTE]
>
> Omit `occurred_at` for live interactions — the server stamps "now". A naive datetime is interpreted as UTC, and a future value is clamped to now. Backfilled events (with `occurred_at`) bypass the real-time within-session cache so historical data can't pollute live, in-session adaptation.

For large histories, **send interactions in bulk** (1–1000 per call) instead of one request each:

```python Python SDK theme={null}
result = client.retrievers.backfill_interactions([
    {"feature_id": "d1", "interaction_type": ["purchase"], "user_id": "u1",
     "feature_uri": "mixpeek://text_extractor@v1/...",
     "occurred_at": "2026-01-15T10:30:00Z"},
    {"feature_id": "d2", "interaction_type": ["click"], "user_id": "u2",
     "feature_uri": "mixpeek://text_extractor@v1/...",
     "occurred_at": "2026-01-16T11:00:00Z"},
    # ... up to 1000 per call
])
# -> {"created": 2, "failed": 0, "errors": [], "results": [{"index": 0, "interaction_id": "int_abc123", "status": "created"}, ...]}
```

```bash REST theme={null}
POST /v1/retrievers/interactions/batch
{ "interactions": [ { "feature_id": "d1", "interaction_type": ["purchase"], "occurred_at": "2026-01-15T10:30:00Z" }, ... ] }
```

Each row is enriched (reward value, feature/context promotion) exactly like a single create; one bad row doesn't sink the batch (it's reported in `errors`). The response now also includes per-item `results` with assigned `interaction_id`s.

> [!NOTE]
>
> The private API uses `feature_id` while published (public) retrievers use `document_id` — they refer to the same field. Use whichever matches the endpoint you're calling.

## Examples

### E-commerce: purchases matter most

```json theme={null}
{
  "learning_config": {
    "context_features": ["INPUT.user_id"],
    "reward_map": {
      "purchase": 5.0,
      "add_to_cart": 2.0,
      "click": 1.0,
      "bookmark": 1.5,
      "negative_feedback": -2.0,
      "return_to_results": -1.0
    },
    "decay_factor": 0.995,
    "min_interactions": 3
  }
}
```

A user who purchases products found via text search will see their text feature weight increase faster than a user who only clicks.

### Content platform: engagement over clicks

```json theme={null}
{
  "learning_config": {
    "context_features": ["INPUT.user_id"],
    "reward_map": {
      "long_view": 2.0,
      "share": 3.0,
      "bookmark": 1.5,
      "click": 0.5,
      "skip": -1.5
    },
    "decay_factor": 0.990,
    "min_interactions": 5
  }
}
```

Clicks are downweighted relative to deep engagement (long views, shares). Skips are penalized more heavily — a result shown but ignored is a stronger negative signal than a mere absence of clicks.

### Internal search: explicit feedback only

```json theme={null}
{
  "learning_config": {
    "context_features": ["INPUT.user_id"],
    "reward_map": {
      "positive_feedback": 3.0,
      "negative_feedback": -3.0
    },
    "decay_factor": 1.0,
    "min_interactions": 10
  }
}
```

Only explicit thumbs up/down influence weights. Clicks and views are ignored. No temporal decay — in an internal tool, preferences tend to be stable. Higher `min_interactions` threshold because explicit feedback is sparse.

## Related

* [Auto-Tune](/docs/retrieval/auto-tune) — overview of the full feedback loop
* [Interactions](/docs/retrieval/interactions) — how to capture user behavior
* [Rollout & Safety](/docs/retrieval/auto-tune-rollout) — safely deploying learned fusion


# Agentic Enrich
Source: https://docs.mixpeek.com/docs/retrieval/stages/agentic-enrich

Classify documents using a multi-turn reasoning agent with tool access

![Agentic Enrich stage showing multi-turn reasoning agent with tool access for document classification](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/agentic-enrich.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=6122d2de543aa3fcfc7fc08003603674)

The Agentic Enrich stage uses a multi-turn reasoning agent (default: Claude) that can call tools — taxonomy lookup, example search, and content analysis — to produce high-quality structured classifications for each document.

> [!NOTE]
>
> **Stage Category**: ENRICH (Enriches documents)
>
> **Transformation**: N documents → N documents (with agent-produced classification added)

## When to Use

| Use Case | Description |
| - | - |
| **Complex classification** | Ambiguous categories requiring multi-step reasoning |
| **Multimodal analysis** | Video/image content needing perceptual analysis + reasoning |
| **Taxonomy-aware classification** | Agent looks up taxonomy definitions before deciding |
| **Few-shot classification** | Agent queries already-classified examples for reference |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple single-shot extraction | `llm_enrich` (faster, cheaper) |
| Vector-based taxonomy matching | `taxonomy_enrich` (no LLM cost) |
| High-throughput batch processing | `llm_enrich` with batch API |
| Deterministic field transformation | `json_transform` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `system_prompt` | string | *Required* | System prompt for the reasoning agent. Supports `{{INPUT.*}}`, `{{DOC.*}}`, `{{CONTEXT.*}}` templates |
| `output_schema` | object | *Required* | JSON schema for the structured output the agent must produce |
| `output_field` | string | `classification` | Dot-path where classification is stored on each document |
| `provider` | string | `anthropic` | LLM provider for the reasoning agent |
| `model_name` | string | `claude-sonnet-4-5-20250929` | Model for the reasoning agent |
| `api_key` | string | `null` | BYOK API key. Supports `{{secrets.*}}` |
| `taxonomy_id` | string | `null` | Taxonomy to load via `get_taxonomy_categories` tool |
| `example_collection_ids` | array | `null` | Collections to search for classified examples |
| `analysis_provider` | object | Google/Gemini | Secondary LLM config for `analyze_content` tool |
| `enabled_tools` | array | `null` | Explicit tool list. Auto-detected when null |
| `max_turns` | integer | `8` | Maximum agent reasoning turns (1-20) |
| `timeout_seconds` | float | `60.0` | Max wall-clock seconds per document (5-300) |
| `temperature` | float | `0.0` | Sampling temperature for the agent |
| `when` | object | `null` | Conditional filter — only enrich matching documents |
| `self_consistency` | object | `null` | Runs several parallel evaluations per document and merges them by majority vote. Fields: `n` (2 to 5, default 3), `temperatures`, `agreement_threshold` (default 0.70), `disagreement_max` (default 3). Multiplies LLM calls by `n` |
| `escalation` | object | `null` | Re-evaluates borderline results with a more capable model. Fields: `provider`, `model` (default `gemini-2.5-pro`), `api_key`, `trigger_agreement_below` (default 0.70), `trigger_disagreements_above` (default 3). Acts on `self_consistency` results |
| `score_threshold` | float | `null` | Documents whose `score` is below this value (0.0 to 1.0) skip the agent and get a `fail` verdict with `fast_rejected: true` in `output_field` |

The stage also reads `max_concurrency` (default `2`, 1 to 5), the number of agent loops that run in parallel. It is applied at run time and is not part of the stage's published parameter schema.

## Available Tools

The agent has access to three tools, auto-enabled based on configuration:

| Tool | Enabled When | Description |
| - | - | - |
| `get_taxonomy_categories` | `taxonomy_id` is set | Loads full taxonomy definition (categories, hierarchy, descriptions) from the database |
| `query_examples` | `example_collection_ids` is set | Vector search against already-classified collections, optionally filtered by category label |
| `analyze_content` | Always available | Delegates to a secondary LLM (default: Gemini) for specialized content analysis (video, image, audio) |

## Configuration Examples

```json Video Classification (Claude + Gemini) theme={null}
{
  "stage_name": "agentic_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "agentic_enrich",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-sonnet-4-5-20250929",
      "system_prompt": "You are an expert IAB content classifier. Use the available tools to: 1) Load the IAB taxonomy categories, 2) Analyze the video content with the analyze_content tool, 3) Query for similar already-classified examples. Make your classification decision with detailed reasoning.",
      "output_schema": {
        "type": "object",
        "properties": {
          "iab_tier1": {"type": "string"},
          "iab_tier2": {"type": "string"},
          "confidence": {"type": "number"},
          "reasoning": {"type": "string"}
        },
        "required": ["iab_tier1", "confidence", "reasoning"]
      },
      "output_field": "iab_classification",
      "taxonomy_id": "tax_iab_content",
      "example_collection_ids": ["col_classified_videos"],
      "analysis_provider": {
        "provider": "google",
        "model_name": "gemini-2.5-flash"
      },
      "max_turns": 10,
      "when": {"AND": [{"field": "_internal.modality", "operator": "eq", "value": "video"}]}
    }
  }
}
```

```json Simple Text Classification theme={null}
{
  "stage_name": "agentic_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "agentic_enrich",
    "parameters": {
      "system_prompt": "Analyze the document and classify it into a technology category with confidence score and reasoning.",
      "output_schema": {
        "type": "object",
        "properties": {
          "category": {"type": "string"},
          "confidence": {"type": "number"},
          "reasoning": {"type": "string"}
        },
        "required": ["category", "confidence", "reasoning"]
      },
      "output_field": "metadata.classification",
      "max_turns": 3,
      "timeout_seconds": 30
    }
  }
}
```

```json Taxonomy-Aware with Examples theme={null}
{
  "stage_name": "agentic_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "agentic_enrich",
    "parameters": {
      "system_prompt": "You are a product classifier. First load the taxonomy to see available categories, then search for similar already-classified products. Use those references to classify this product accurately.",
      "output_schema": {
        "type": "object",
        "properties": {
          "category_id": {"type": "string"},
          "category_name": {"type": "string"},
          "confidence": {"type": "number"},
          "similar_products": {"type": "array", "items": {"type": "string"}}
        },
        "required": ["category_id", "category_name", "confidence"]
      },
      "output_field": "product_classification",
      "taxonomy_id": "tax_product_categories",
      "example_collection_ids": ["col_classified_products"],
      "max_turns": 8
    }
  }
}
```

```json Conditional Enrichment theme={null}
{
  "stage_name": "agentic_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "agentic_enrich",
    "parameters": {
      "system_prompt": "Classify this image content by subject matter and artistic style.",
      "output_schema": {
        "type": "object",
        "properties": {
          "subject": {"type": "string"},
          "style": {"type": "string"},
          "confidence": {"type": "number"}
        }
      },
      "output_field": "image_classification",
      "max_turns": 5,
      "when": {"AND": [{"field": "_internal.modality", "operator": "eq", "value": "image"}]}
    }
  }
}
```

## How It Works

For each document, the stage runs a multi-turn agent loop:

1. **Initialize**: Agent receives the document content + system prompt + available tools
2. **Reason**: Agent analyzes the document and optionally calls tools (taxonomy lookup, example search, content analysis)
3. **Observe**: Tool results are fed back to the agent as context
4. **Iterate**: Loop continues until the agent produces a final answer (or `max_turns`/`timeout_seconds` is reached)
5. **Output**: The agent's structured JSON response is merged into the document at `output_field`

## Output Examples

### With Taxonomy + Examples

```json theme={null}
{
  "document_id": "doc_abc123",
  "content": "A product review video discussing...",
  "iab_classification": {
    "iab_tier1": "Technology & Computing",
    "iab_tier2": "Consumer Electronics",
    "confidence": 0.92,
    "reasoning": "The video discusses smartphone features and pricing. Taxonomy lookup confirmed 'Consumer Electronics' under 'Technology & Computing'. Similar classified videos (col_classified_videos) showed consistent T&C categorization for product review content."
  }
}
```

### Simple Classification

```json theme={null}
{
  "document_id": "doc_def456",
  "content": "Introduction to machine learning algorithms...",
  "metadata": {
    "classification": {
      "category": "Artificial Intelligence",
      "confidence": 0.88,
      "reasoning": "Document covers supervised and unsupervised learning methods, neural network architectures, and model evaluation."
    }
  }
}
```

### Conditional Skip (When Condition)

```json theme={null}
{
  "document_id": "doc_ghi789",
  "_internal": {"modality": "text"},
  "content": "Plain text document..."
}
```

Documents that don't match the `when` condition are passed through unchanged.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 2-30s per document (depends on turns and tools) |
| **LLM calls** | 3-15 per document |
| **Max documents** | 10 per execution |
| **Parallel** | Up to `max_concurrency` (default 2) |

> [!WARNING]
>
> Agentic enrichment makes multiple LLM calls per document. Use the `when` condition to limit which documents are processed, and keep `max_turns` low for simple tasks.

## Common Pipeline Patterns

### Search + Agentic Classify + Filter

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 20
          }
        ],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "agentic_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "agentic_enrich",
      "parameters": {
        "system_prompt": "Classify this content by IAB category using the taxonomy and examples.",
        "output_schema": {
          "type": "object",
          "properties": {
            "iab_category": {"type": "string"},
            "confidence": {"type": "number"}
          }
        },
        "output_field": "classification",
        "taxonomy_id": "tax_iab",
        "example_collection_ids": ["col_labeled"],
        "max_turns": 6
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "classification.confidence",
        "operator": "gte",
        "value": 0.8
      }
    }
  }
]
```

### Multimodal Analysis Pipeline

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 10
          }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "agentic_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "agentic_enrich",
      "parameters": {
        "system_prompt": "Use the analyze_content tool to examine this media, then classify by topic and sentiment.",
        "output_schema": {
          "type": "object",
          "properties": {
            "topic": {"type": "string"},
            "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
            "summary": {"type": "string"}
          }
        },
        "output_field": "media_analysis",
        "analysis_provider": {
          "provider": "google",
          "model_name": "gemini-2.5-flash"
        },
        "max_turns": 5
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  }
]
```

## Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys for both the reasoning agent and the analysis provider.

**1. Store your API keys as secrets**

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "anthropic_api_key",
    "secret_value": "sk-ant-..."
  }'
```

**2. Reference secrets in your stage config**

```json theme={null}
{
  "stage_name": "agentic_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "agentic_enrich",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-sonnet-4-5-20250929",
      "api_key": "{{secrets.anthropic_api_key}}",
      "system_prompt": "Classify this content.",
      "output_schema": {"type": "object", "properties": {"category": {"type": "string"}}},
      "analysis_provider": {
        "provider": "google",
        "model_name": "gemini-2.5-flash",
        "api_key": "{{secrets.google_api_key}}"
      }
    }
  }
}
```

> [!NOTE]
>
> When `api_key` is not specified, the stage uses Mixpeek's default API keys and usage is charged to your Mixpeek account.

## Stage Metadata

The stage returns execution metadata for observability:

| Field | Description |
| - | - |
| `documents_enriched` | Number of documents processed by the agent |
| `documents_skipped` | Number of documents skipped (when condition) |
| `documents_fast_rejected` | Number of documents rejected by `score_threshold` |
| `total_cost` | Total LLM API cost across all documents |
| `total_tokens_input` | Total input tokens consumed |
| `total_tokens_output` | Total output tokens generated |
| `reasoning_traces` | Per-document traces with tool calls and turn history |
| `conditional` | Whether a `when` condition was applied |

## Error Handling

| Error | Behavior |
| - | - |
| Agent timeout | Returns best result so far, or null |
| Max turns reached | Loop ends; latest structured output used |
| Schema validation fail | Raw text stored in `output_field` |
| Tool execution error | Error message returned to agent; it can retry or skip |
| Missing system\_prompt | Stage fails with validation error |
| Invalid taxonomy\_id | `get_taxonomy_categories` tool returns error to agent |
| Empty content | Agent receives empty document; classification based on metadata |
| Invalid API key | Error returned with auth failure |

## Cost Considerations

| Setting | Cost Impact |
| - | - |
| `max_turns: 3` | Low — simple direct classification |
| `max_turns: 10` | Medium — multi-tool research workflow |
| `max_concurrency: 1` | Sequential, slower but controlled cost |
| `when` condition | Skip documents that don't need classification |
| `timeout_seconds: 30` | Cap per-document spend |

> [!TIP]
>
> Start with `max_turns: 3` and increase only if the agent consistently needs more iterations. Most straightforward classifications finish in 2-4 turns.

## Related

* [LLM Enrich](/docs/retrieval/stages/llm-enrich) — Single-shot LLM enrichment (faster, cheaper)
* [Taxonomy Enrich](/docs/retrieval/stages/taxonomy-enrich) — Vector-based taxonomy matching (no LLM cost)
* [Agent Search](/docs/retrieval/stages/agent-search) — Multi-turn agent for search (different use case)


# Classify
Source: https://docs.mixpeek.com/docs/retrieval/stages/classify

Classify documents at query time using built-in tasks (like NSFW content safety) or a custom model deployed as a plugin

The Classify stage labels documents at query time. It supports two modes:

* **Built-in tasks** (via `task`) — managed classifiers Mixpeek hosts for you. The first available task is `nsfw`, a content-safety classifier for text, image, and video. No plugin to build or deploy.

* **Custom classifiers** (via `feature_uri`) — your own classifier model deployed as a [custom extractor](/docs/processing/custom-extractors). The stage sends document text to your extractor's inference endpoint and attaches predicted labels with confidence scores to each document.

* **Hosted classifiers with your labels** (via `inference_name` and `labels`). Set `inference_name: "typesafe__jev"` to classify with [Jev](/docs/processing/jev) into labels you list. Each document gets every label with its probability, highest first.

Set exactly one of `task`, `feature_uri` or `inference_name`.

> [!NOTE]
>
> **Stage Category**: APPLY (Enriches documents)
>
> **Transformation**: N documents → N documents (with classification results added). A built-in task can optionally drop documents (e.g. `drop_if_unsafe`), making it N → ≤N.

## When to Use

| Use Case | Description |
| - | - |
| **Custom classification** | Apply your own trained classifier to search results |
| **Content labeling** | Tag documents with domain-specific categories |
| **Compliance scoring** | Score documents against compliance criteria |
| **Intent detection** | Classify user queries or document intent |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Predefined taxonomy classification | [`taxonomy_enrich`](/docs/retrieval/stages/taxonomy-enrich) (no custom model needed) |
| LLM-based classification | [`llm_enrich`](/docs/retrieval/stages/llm-enrich) with `output_schema` |
| Simple keyword matching | [`attribute_filter`](/docs/retrieval/stages/attribute-filter) |

## Available Classification Tasks

Built-in tasks are managed classifiers Mixpeek hosts — set `task` and skip `feature_uri` entirely. This list grows over time as more built-in tasks ship.

| Task | What it does | Model | Modalities | Outputs |
| - | - | - | - | - |
| `nsfw` | Content-safety classification — scores how likely a document is unsafe (not safe for work) | `mixpeek/content-classifier-v1` | Text, image, video | `nsfw_score` (0–1), `label`, `is_nsfw` |

The `mixpeek/content-classifier-v1` model is a CPU-only multimodal classifier: DistilBERT for text, ViT for images, and frame sampling for video — so the `nsfw` task runs without a GPU. By default it annotates each document with the outputs above; set `drop_if_unsafe: true` to filter unsafe documents out of the result set instead.

> [!NOTE]
>
> The same `mixpeek/content-classifier-v1` model also powers a **tenant-level upload gate**: when `nsfw_check_enabled` is on for a shared-plane org, NSFW image, video, and text uploads are rejected at upload time. The Classify stage applies the same model at query time over search results. See [Uploads](/docs/ingestion/uploads).

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `task` | string (enum) | `null` | Built-in classification task to run (e.g. `"nsfw"`). Mutually exclusive with `feature_uri`. See [Available Classification Tasks](#available-classification-tasks). |
| `feature_uri` | string | *Required unless `task` or `inference_name` is set* | Feature URI of your custom classifier plugin. Mutually exclusive with `task` and `inference_name`. |
| `inference_name` | string | `null` | Hosted classifier to call by name, e.g. `typesafe__jev` or `mixpeek__decision`. Requires `labels`. |
| `decision_model` | string | `null` | Model that answers for `mixpeek__decision` or `typesafe__jev`, e.g. `jev-latest` or `gemini-2.5-flash-lite`. See [Swap the decision model](/docs/processing/jev#swap-the-decision-model). |
| `labels` | list or object | `null` | Labels to classify into: a list, or a map of label to a one-line description. 2 to 255 labels. |
| `instruction` | string | `null` | Optional question for the classifier, e.g. `"Which support team should handle this ticket?"` |
| `document_field` | string | `"content"` | Document field path containing text to classify |
| `image_field` | string | `null` | Document field path containing an image URL to classify instead of text (built-in tasks only) |
| `video_field` | string | `null` | Document field path containing a video URL to classify instead of text (built-in tasks only) |
| `output_field` | string | `"classification"` | Field path to store classification results |
| `nsfw_threshold` | float | `0.7` | Score at or above which a document is flagged `is_nsfw` (`nsfw` task only) |
| `drop_if_unsafe` | boolean | `false` | Drop documents flagged unsafe instead of annotating them (`nsfw` task only) |
| `max_document_chars` | integer | `5000` | Maximum characters sent for classification (100–50000) |
| `top_k_labels` | integer | `null` | Keep only the top-k labels by confidence (custom classifiers) |
| `min_confidence` | float | `null` | Minimum confidence threshold (0.0–1.0, custom classifiers) |
| `batch_size` | integer | `10` | Documents per inference call (1–100) |
| `max_concurrency` | integer | `5` | Maximum concurrent inference requests (1–20) |

## Plugin Contract

Your classifier plugin must accept `{text: str}` and return `{labels: [{label: str, confidence: float}]}`.

```python theme={null}
# In your plugin's realtime.py
class ClassifierService(BaseInferenceService):
    def _process_single(self, inputs: dict, parameters: dict) -> dict:
        text = inputs["text"]
        # Your classification logic here
        return {
            "labels": [
                {"label": "technology", "confidence": 0.92},
                {"label": "business", "confidence": 0.78},
                {"label": "science", "confidence": 0.45},
            ]
        }
```

> [!TIP]
>
> Set `inference_type: "classify"` in your plugin's manifest to declare compatibility with the classify stage.

## Configuration Examples

### Built-in NSFW Task

```json Annotate (flag NSFW) theme={null}
{
  "stage_name": "nsfw_check",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "task": "nsfw",
      "document_field": "content",
      "output_field": "safety",
      "nsfw_threshold": 0.7
    }
  }
}
```

```json Filter (drop unsafe docs) theme={null}
{
  "stage_name": "nsfw_filter",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "task": "nsfw",
      "document_field": "content",
      "drop_if_unsafe": true,
      "nsfw_threshold": 0.7
    }
  }
}
```

```json Classify a Media URL theme={null}
{
  "stage_name": "nsfw_image_check",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "task": "nsfw",
      "image_field": "metadata.thumbnail_url",
      "output_field": "safety"
    }
  }
}
```

The annotate example writes a result like `{"nsfw_score": 0.03, "label": "safe", "is_nsfw": false}` to `output_field` on every document. The filter example drops any document whose `nsfw_score` meets `nsfw_threshold` instead of annotating it.

### Classify with Jev

```json theme={null}
{
  "stage_name": "classify",
  "stage_type": "enrich",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "inference_name": "typesafe__jev",
      "labels": {
        "product_demo": "Someone shows how a product works",
        "testimonial": "A customer describes their experience",
        "tutorial": "Step-by-step instructions"
      },
      "document_field": "text",
      "top_k_labels": 1
    }
  }
}
```

Each document gets `classification: [{"label": "product_demo", "confidence": 0.76}]`. `min_confidence` and `top_k_labels` apply as they do for custom classifiers.

### Custom Classifier

```json Basic Classification theme={null}
{
  "stage_name": "my_classifier",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "feature_uri": "mixpeek://my_classifier@1.0.0/classify",
      "document_field": "content",
      "output_field": "classification"
    }
  }
}
```

```json With Confidence Filtering theme={null}
{
  "stage_name": "my_classifier",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "feature_uri": "mixpeek://my_classifier@1.0.0/classify",
      "document_field": "metadata.description",
      "output_field": "labels",
      "min_confidence": 0.5,
      "top_k_labels": 3
    }
  }
}
```

```json High-Throughput theme={null}
{
  "stage_name": "my_classifier",
  "config": {
    "stage_id": "classify",
    "parameters": {
      "feature_uri": "mixpeek://my_classifier@1.0.0/classify",
      "batch_size": 20,
      "max_concurrency": 10,
      "max_document_chars": 2000
    }
  }
}
```

## Output

For a **custom classifier**, each document gets the predicted labels added at `output_field`:

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Apple Inc. announced new AI features...",
  "classification": [
    {"label": "technology", "confidence": 0.95},
    {"label": "business", "confidence": 0.82}
  ]
}
```

For the built-in **`nsfw` task**, each document gets a safety result at `output_field` (unless `drop_if_unsafe` removed it):

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Apple Inc. announced new AI features...",
  "safety": {
    "nsfw_score": 0.03,
    "label": "safe",
    "is_nsfw": false
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | Depends on your plugin model |
| **Batch size** | 10 documents default |
| **Concurrency** | 5 parallel requests default |
| **Max document chars** | 5000 default |

## Common Pipeline Patterns

### Search + Classify + Filter by Label

```json theme={null}
[
  {
    "stage_name": "search",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{
          "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
          "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
          "top_k": 100
        }],
        "final_top_k": 25
      }
    }
  },
  {
    "stage_name": "classify",
    "config": {
      "stage_id": "classify",
      "parameters": {
        "feature_uri": "mixpeek://my_classifier@1.0.0/classify",
        "min_confidence": 0.7,
        "top_k_labels": 1
      }
    }
  },
  {
    "stage_name": "filter_by_label",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "classification.0.label",
        "operator": "eq",
        "value": "{{INPUT.target_category}}"
      }
    }
  }
]
```

## Related

* [Taxonomy Enrich](/docs/retrieval/stages/taxonomy-enrich) - Predefined taxonomy classification (no custom model)
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - LLM-based enrichment and classification
* [Custom Extractors](/docs/processing/custom-extractors) - Build and deploy custom inference models
* [Uploads](/docs/ingestion/uploads) - Tenant-level NSFW upload gate using the same `mixpeek/content-classifier-v1` model


# Code Execution
Source: https://docs.mixpeek.com/docs/retrieval/stages/code-execution

Execute custom code to transform, filter, or enrich documents

![Code Execution stage showing custom code transformations](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/code-execution.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=eb80f950593bfd772e987fcd72411f6a)

The Code Execution stage allows you to run custom Python code to transform, filter, or enrich documents. This provides maximum flexibility for complex logic that can't be expressed with other stages.

> [!NOTE]
>
> **Stage Category**: APPLY (Transforms documents)
>
> **Transformation**: N documents → M documents (custom logic)

## When to Use

| Use Case | Description |
| - | - |
| **Custom transformations** | Complex field calculations |
| **Business logic** | Domain-specific rules |
| **Data normalization** | Custom parsing/formatting |
| **Advanced filtering** | Logic beyond structured\_filter |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple field transforms | `json_transform` |
| LLM-based enrichment | `llm_enrich` |
| Standard filtering | `attribute_filter` |
| External API calls | `api_call` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `code` | string | *Required* | Code to execute |
| `language` | string | `python` | Execution language (`python`, `typescript`, `javascript`) |
| `output_field` | string | `computed` | Document field path where results are merged |
| `result_variable` | string | `result` | Variable name containing the output list |
| `timeout_ms` | integer | `5000` | Execution timeout in milliseconds (100-30000) |
| `max_output_size` | integer | `100000` | Max output size in bytes (1024-1000000) |
| `env` | object | `{}` | Environment variables (supports `INPUT`/`SECRET` templates) |
| `on_error` | string | `skip` | Error handling strategy (`skip`, `raise`) |

## Configuration Examples

```json Basic Transformation theme={null}
{
  "stage_name": "code_execution",
  "stage_type": "apply",
  "config": {
    "stage_id": "code_execution",
    "parameters": {
      "code": "def transform(doc):\n    doc['word_count'] = len(doc.get('content', '').split())\n    return doc"
    }
  }
}
```

```json Custom Scoring theme={null}
{
  "stage_name": "code_execution",
  "stage_type": "apply",
  "config": {
    "stage_id": "code_execution",
    "parameters": {
      "code": "def transform(doc):\n    base_score = doc.get('score', 0)\n    recency_boost = 1.0 if doc.get('metadata', {}).get('is_recent') else 0.8\n    doc['adjusted_score'] = base_score * recency_boost\n    return doc"
    }
  }
}
```

```json Filtering Logic theme={null}
{
  "stage_name": "code_execution",
  "stage_type": "apply",
  "config": {
    "stage_id": "code_execution",
    "parameters": {
      "code": "def transform(doc):\n    content = doc.get('content', '')\n    # Filter out short or low-quality content\n    if len(content) < 100:\n        return None\n    if content.count('http') > 5:\n        return None  # Too many links\n    return doc"
    }
  }
}
```

```json With External Packages theme={null}
{
  "stage_name": "code_execution",
  "stage_type": "apply",
  "config": {
    "stage_id": "code_execution",
    "parameters": {
      "code": "import dateutil.parser\n\ndef transform(doc):\n    date_str = doc.get('metadata', {}).get('date')\n    if date_str:\n        parsed = dateutil.parser.parse(date_str)\n        doc['metadata']['year'] = parsed.year\n        doc['metadata']['month'] = parsed.month\n    return doc"
    }
  }
}
```

```json Text Processing theme={null}
{
  "stage_name": "code_execution",
  "stage_type": "apply",
  "config": {
    "stage_id": "code_execution",
    "parameters": {
      "code": "import re\n\ndef transform(doc):\n    content = doc.get('content', '')\n    # Extract emails\n    emails = re.findall(r'[\\w.-]+@[\\w.-]+', content)\n    doc['extracted_emails'] = emails\n    # Clean content\n    doc['clean_content'] = re.sub(r'\\s+', ' ', content).strip()\n    return doc"
    }
  }
}
```

## Code Structure

Your code must define a `transform` function:

```python theme={null}
def transform(doc):
    """
    Transform a single document.

    Args:
        doc: Dictionary containing document fields

    Returns:
        - Modified doc dict to keep document
        - None to filter out document
    """
    # Your logic here
    return doc
```

### Available in Scope

| Variable | Type | Description |
| - | - | - |
| `doc` | dict | Current document |
| `INPUT` | dict | Pipeline input parameters |
| `CONTEXT` | dict | Pipeline context |

## Input Document Structure

```python theme={null}
doc = {
    "document_id": "doc_123",
    "content": "Document text content...",
    "score": 0.85,
    "metadata": {
        "title": "Document Title",
        "author": "John Doe",
        "date": "2024-01-15"
    }
}
```

## Output Options

| Return Value | Effect |
| - | - |
| `doc` (modified) | Keep document with changes |
| `doc` (unmodified) | Keep document as-is |
| `None` | Filter out document |

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-50ms per document |
| **Timeout** | Configurable (default 5s) |
| **Memory** | Configurable (default 128MB) |
| **Concurrency** | Parallel execution |

> [!WARNING]
>
> Code execution adds latency. Keep transformations simple and avoid heavy computation. For complex processing, consider pre-computing during ingestion.

## Common Pipeline Patterns

### Custom Scoring Pipeline

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "code_execution",
    "stage_type": "apply",
    "config": {
      "stage_id": "code_execution",
      "parameters": {
        "code": "def transform(doc):\n    score = doc.get('score', 0)\n    # Boost verified sources\n    if doc.get('metadata', {}).get('verified'):\n        score *= 1.2\n    # Penalize old content\n    if doc.get('metadata', {}).get('year', 2024) < 2020:\n        score *= 0.8\n    doc['custom_score'] = score\n    return doc"
      }
    }
  },
  {
    "stage_name": "sort_relevance",
    "stage_type": "sort",
    "config": {
      "stage_id": "sort_relevance",
      "parameters": {
        "score_field": "custom_score"
      }
    }
  }
]
```

### Data Normalization Pipeline

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 30 }
        ],
        "final_top_k": 30
      }
    }
  },
  {
    "stage_name": "code_execution",
    "stage_type": "apply",
    "config": {
      "stage_id": "code_execution",
      "parameters": {
        "code": "def transform(doc):\n    meta = doc.get('metadata', {})\n    # Normalize price to USD\n    price = meta.get('price', 0)\n    currency = meta.get('currency', 'USD')\n    rates = {'EUR': 1.1, 'GBP': 1.27, 'USD': 1.0}\n    doc['metadata']['price_usd'] = price * rates.get(currency, 1.0)\n    return doc"
      }
    }
  }
]
```

### Advanced Filtering

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "code_execution",
    "stage_type": "apply",
    "config": {
      "stage_id": "code_execution",
      "parameters": {
        "code": "def transform(doc):\n    content = doc.get('content', '')\n    # Complex filtering logic\n    word_count = len(content.split())\n    if word_count < 50:\n        return None\n    # Check for required sections\n    required = ['introduction', 'conclusion']\n    content_lower = content.lower()\n    if not all(section in content_lower for section in required):\n        return None\n    doc['metadata']['word_count'] = word_count\n    return doc"
      }
    }
  }
]
```

## Security

| Restriction | Description |
| - | - |
| Network | No outbound network access |
| Filesystem | No file system access |
| Imports | Limited to approved packages |
| Resources | Memory and CPU limits enforced |

## Allowed Packages

Built-in packages available:

* `json`, `re`, `math`, `datetime`, `collections`
* `itertools`, `functools`, `operator`

The sandbox provides a standard runtime environment for the selected `language`.

## Error Handling

| Error | Behavior |
| - | - |
| Syntax error | Stage fails |
| Runtime exception | Document skipped |
| Timeout | Document skipped |
| Memory exceeded | Stage fails |

> [!TIP]
>
> Always handle missing fields gracefully using `.get()` with defaults to avoid runtime errors.

## Debugging

Enable debug mode to see execution details:

```json theme={null}
{
  "stage_name": "code_execution",
  "stage_type": "apply",
  "config": {
    "stage_id": "code_execution",
    "parameters": {
      "code": "def transform(doc):\n    print(f'Processing: {doc.get(\"document_id\")}')\n    return doc"
    }
  }
}
```

## Related

* [JSON Transform](/docs/retrieval/stages/json-transform) - Template-based transforms
* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - AI-powered enrichment
* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Standard filtering


# Document Enrich
Source: https://docs.mixpeek.com/docs/retrieval/stages/document-enrich

Join documents across collections for cross-reference enrichment

![Document Enrich stage showing collection joins and cross-reference lookups](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/document-enrich.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=16b0bb36191f390695226d70e4622221)

The Document Enrich stage performs collection joins by looking up related documents from other Mixpeek collections. This enables cross-reference enrichment without external database calls.

> [!NOTE]
>
> **Stage Category**: ENRICH (Enriches documents)
>
> **Transformation**: N documents → N documents (with joined data added)

## When to Use

| Use Case | Description |
| - | - |
| **Cross-collection joins** | Link products to reviews, users to profiles |
| **Reference resolution** | Expand foreign keys to full documents |
| **Denormalization** | Flatten related data for display |
| **Multi-index search** | Combine results from different collections |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| External database joins | `sql_lookup` |
| Single collection search | No enrichment needed |
| Real-time external data | `api_call` |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `target_collection_id` | string | `{{COLLECTION_ID}}` | Collection to join from. Replace the placeholder with a real collection ID. With an inline retriever it also scopes the collections that retriever searches |
| `source_field` | string | `source_object_id` | Direct join: field in the current document whose value is the join key |
| `target_field` | string | `document_id` | Direct join: field in the target collection matched against the join key. The stage creates a keyword index on it when one is missing. Ignored for retriever joins |
| `retriever_id` | string | `null` | Retriever join: ID of an existing retriever to run for each document. Cannot be combined with `retriever_config` |
| `retriever_config` | object | `null` | Retriever join: inline retriever definition with a non-empty `stages` array. Cannot be combined with `retriever_id` |
| `retriever_inputs` | object | `null` | Retriever join: maps retriever input names to values, using `{{DOC.field_name}}` to read fields from the current document |
| `fields_to_merge` | array | `null` | Fields to copy from the matched document. `null` merges the entire matched payload |
| `output_field` | string | `null` | Field on the current document that receives the matched data. `null` merges the data into the document root |
| `strategy` | string | `enrich` | How the match is merged: `enrich`, `replace` or `append` (see below) |
| `when` | object | `null` | Condition that selects which documents are enriched. Documents that do not match pass through unchanged |
| `allow_missing` | boolean | `true` | `true` passes documents with no match through unchanged. `false` removes them from the results |
| `cache_behavior` | string | `auto` | Internal stage caching: `auto`, `disabled` or `aggressive` |
| `cache_ttl_seconds` | integer | `null` | Cache entry lifetime in seconds. `null` leaves eviction to the cache |

### Join modes

The stage runs in one of two modes, chosen by whether `retriever_id` or `retriever_config` is set.

**Direct join** (the default). The stage reads the value at `source_field` from each document and looks for a document in `target_collection_id` whose `target_field` has the same value. A document with no value at `source_field` is not looked up and passes through unchanged when `allow_missing` is `true`.

**Retriever join.** The stage evaluates `retriever_inputs` for each document and runs the retriever named by `retriever_id`, or the inline `retriever_config`, with those inputs. Documents that produce identical inputs share a single retriever run. In an inline `retriever_config`, string stage parameters that contain `{{DOC.field_name}}` are filled in from the current document. `source_field` and `target_field` are not used in this mode.

In both modes each document receives at most one match: the first document found, or the top result of the retriever. `fields_to_merge`, `output_field`, `strategy`, `when` and `allow_missing` apply the same way in both.

### Merge strategies

* `enrich` places the match under `output_field`. Without `output_field`, it merges the match into the document root and keeps existing fields unless the match has a field of the same name.
* `replace` merges the match into the document root and overwrites fields of the same name. `output_field` is not used.
* `append` adds the match to an array at `output_field`, or at `enrichments` when `output_field` is unset.

## Configuration Examples

```json Basic Collection Join theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "user_profiles",
      "target_field": "user_id",
      "source_field": "metadata.author_id",
      "output_field": "author"
    }
  }
}
```

```json Select Specific Fields theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "products",
      "target_field": "product_id",
      "source_field": "metadata.product_id",
      "output_field": "product_details",
      "fields_to_merge": ["name", "price", "category", "image_url"]
    }
  }
}
```

```json Conditional Enrichment theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "reviews",
      "target_field": "product_id",
      "source_field": "document_id",
      "output_field": "reviews"
    }
  }
}
```

```json Retriever Join theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "retriever_id": "ret_find_similar_products",
      "retriever_inputs": {
        "query": "{{DOC.description}}"
      },
      "output_field": "similar_product",
      "fields_to_merge": ["name", "price"]
    }
  }
}
```

```json Nested Field Lookup theme={null}
{
  "stage_name": "document_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "document_enrich",
    "parameters": {
      "target_collection_id": "categories",
      "target_field": "category_code",
      "source_field": "metadata.taxonomy.primary_category",
      "output_field": "category_info"
    }
  }
}
```

## Output Schema

### Single Document Join

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Product review content...",
  "metadata": {
    "product_id": "prod_456"
  },
  "product_details": {
    "name": "Wireless Headphones",
    "price": 199.99,
    "category": "Electronics",
    "image_url": "https://..."
  }
}
```

### Append Strategy

With `strategy: "append"`, `output_field` holds an array and the match is added to it.

```json theme={null}
{
  "document_id": "prod_456",
  "content": "Product description...",
  "reviews": [
    {
      "document_id": "rev_1",
      "rating": 5,
      "text": "Great product!"
    }
  ]
}
```

### No Match Found

With `allow_missing: true` (the default) the document passes through unchanged and no `output_field` is added. With `allow_missing: false` the document is removed from the results.

```json theme={null}
{
  "document_id": "doc_123",
  "content": "..."
}
```

When none of the checked documents match, the response also carries a warning that names the join. The stage metadata reports `documents_checked`, `documents_enriched` and `documents_skipped`.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-20ms per document |
| **Batch processing** | Automatic batching |
| **Index usage** | Creates a keyword index on `target_field` if missing |
| **Lookup strategy** | One exact-match lookup per key for up to 50 keys, then batched lookups of 100 keys |

> [!TIP]
>
> Use `fields_to_merge` to reduce payload size. Documents that share the same retriever inputs run the retriever once, so keep `retriever_inputs` to the fields that matter.

## Common Pipeline Patterns

### Search + Author Enrichment

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 20 }
        ],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "document_enrich",
      "parameters": {
        "target_collection_id": "users",
        "target_field": "user_id",
        "source_field": "metadata.author_id",
        "output_field": "author",
        "fields_to_merge": ["name", "avatar", "bio"]
      }
    }
  }
]
```

### Product Search with Reviews

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 10 }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "document_enrich",
      "parameters": {
        "target_collection_id": "reviews",
        "target_field": "product_id",
        "source_field": "document_id",
        "output_field": "recent_reviews",
        "fields_to_merge": ["rating", "text", "author_name", "created_at"]
      }
    }
  }
]
```

### Hierarchical Category Enrichment

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "document_enrich",
      "parameters": {
        "target_collection_id": "categories",
        "target_field": "category_id",
        "source_field": "metadata.category_id",
        "output_field": "category"
      }
    }
  },
  {
    "stage_name": "document_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "document_enrich",
      "parameters": {
        "target_collection_id": "categories",
        "target_field": "category_id",
        "source_field": "category.parent_id",
        "output_field": "parent_category"
      }
    }
  }
]
```

## Error Handling

| Condition | Behavior |
| - | - |
| No matching document, or target collection has no match | Document passes through unchanged (`allow_missing: true`) or is removed (`false`). A warning is added when nothing matched |
| Document has no value at `source_field` | Not looked up. Passes through unchanged when `allow_missing` is `true` |
| Retriever execution fails | Affected documents follow `allow_missing`, and the warning names the error |
| `retriever_id` not found in the namespace | Stage fails with a not-found error |
| Both `retriever_id` and `retriever_config`, or `retriever_config` without stages | Configuration is rejected as invalid |

## vs Other Enrichment Stages

| Feature | document\_enrich | sql\_lookup | api\_call |
| - | - | - | - |
| Data source | Mixpeek collections | SQL database | External API |
| Latency | 5-20ms | 10-100ms | 50-500ms |
| Best for | Cross-collection | External relational | REST APIs |
| Setup | None | Connection config | Endpoint config |

## Related

* [SQL Lookup](/docs/retrieval/stages/sql-lookup) - External database joins
* [API Call](/docs/retrieval/stages/api-call) - REST API enrichment
* [Taxonomy Enrich](/docs/retrieval/stages/taxonomy-enrich) - Classification enrichment


# LLM Enrich
Source: https://docs.mixpeek.com/docs/retrieval/stages/llm-enrich

Extract structured data from documents using language model analysis

![LLM Enrich stage showing structured data extraction with language models](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/llm-enrich.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=029ebd511f0d8a7b933a0168a6624a55)

The LLM Enrich stage uses language models to extract structured data from document content. It can identify entities, classify content, extract key information, and generate structured outputs.

> [!NOTE]
>
> **Stage Category**: ENRICH (Enriches documents)
>
> **Transformation**: N documents → N documents (with extracted data added)

## When to Use

| Use Case | Description |
| - | - |
| **Entity extraction** | Extract names, dates, amounts from text |
| **Content classification** | Categorize documents by topic/type |
| **Key information extraction** | Pull specific facts from unstructured text |
| **Structured output generation** | Convert prose to structured data |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple field transformation | `json_transform` |
| Predefined taxonomy classification | `taxonomy_enrich` |
| Large-scale processing | Pre-process during indexing |
| Real-time low-latency | Use cached extractions |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `provider` | string | `google` | LLM provider: `openai`, `google`, `anthropic`, or `typesafe` (auto-inferred from `model_name` if omitted) |
| `model_name` | string | provider default | Specific model, e.g. `gpt-4o-mini`, `gemini-2.5-flash-lite`, `claude-3-5-haiku` |
| `prompt` | string | `Summarize the following content in 2-3 sentences: {{DOC.content}}` | Extraction instructions (template, supports `{{DOC.field}}` / `{{INPUT.field}}`) |
| `output_field` | string | `summary` | Dot-path of the field that receives the result |
| `output_schema` | object | `null` | JSON schema for structured output (`schema` is accepted as an alias) |
| `batch_size` | integer | `5` | Documents per batch, 1 to 25. Each document gets its own LLM call, and the calls in a batch run concurrently |
| `temperature` | float | `0.2` | Sampling temperature, 0.0 to 1.0 |
| `max_tokens` | integer | `null` | Maximum output tokens, 100 to 16000. `null` uses the provider default |
| `when` | object | `null` | Condition that selects which documents are enriched. Documents that do not match pass through unchanged |
| `use_vcache` | boolean | `null` | Turns semantic caching of LLM calls on or off for this stage. `null` follows the global setting |

The stage also accepts `multimodal_inputs`, `api_key` and `feature_uri`, described in the sections below.

Each document is sent to the model as content, with embedding vectors, blobs and internal bookkeeping fields removed, and `prompt` as the instruction. To point the model at a specific field, reference it in the prompt with `{{DOC.field_name}}`.

> [!WARNING]
>
> The stage runs on at most 10 documents. A larger working set fails with an error, so reduce it upstream with a filter or reduce stage, or set a smaller `final_top_k` on the search.

## Available Models

| Model | Speed | Quality | Best For |
| - | - | - | - |
| `gpt-4o-mini` | Fast | Good | Simple extractions |
| `gpt-4o` | Medium | Excellent | Complex analysis |
| `claude-3-haiku` | Fast | Good | Quick processing |
| `claude-3-sonnet` | Medium | Excellent | Nuanced extraction |
| `jev-latest` | Fast | Calibrated | Labels, flags and ratings from a fixed set |

`jev-latest` is [Jev](/docs/processing/jev), an evaluation model from TypeSafe AI. It requires an `output_schema`, and every field needs a closed set of values:

| Schema field | Jev answers it as |
| - | - |
| `boolean` | Yes/no, true when the probability is at least 0.5 |
| `number` with `minimum: 0`, `maximum: 1` | The yes probability itself |
| `string` with `enum` (2 to 255 values) | A choice from the enum |
| `integer` with `minimum` and `maximum` (at most 10 levels) | A score on those levels |
| `array` of `enum` strings | Every value whose probability is at least 0.5 |

A string field named `reason`, `reasoning`, `explanation`, `rationale` or `justification` is filled with the probabilities behind the answer. Any other free-text field is rejected when you create the retriever, and the error names the field.

## Configuration Examples

```json Classify with Jev theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "model_name": "jev-latest",
      "prompt": "Classify this document: {{DOC.content}}",
      "output_field": "classification",
      "output_schema": {
        "type": "object",
        "properties": {
          "channel": {"type": "string", "enum": ["tv_ad", "how_to", "news", "other"]},
          "urgency": {"type": "integer", "minimum": 1, "maximum": 3}
        }
      }
    }
  }
}
```

```json Basic Entity Extraction theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract all company names and person names mentioned in this document.",
      "output_field": "entities"
    }
  }
}
```

```json Structured Output with Schema theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Extract the following information from this product review.",
      "output_field": "review_analysis",
      "output_schema": {
        "type": "object",
        "properties": {
          "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
          "rating_mentioned": {"type": "number", "minimum": 1, "maximum": 5},
          "pros": {"type": "array", "items": {"type": "string"}},
          "cons": {"type": "array", "items": {"type": "string"}},
          "would_recommend": {"type": "boolean"}
        }
      }
    }
  }
}
```

```json Key Facts Extraction theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract key facts from this news article: main event, date, location, people involved, and outcome.",
      "output_field": "key_facts",
      "output_schema": {
        "type": "object",
        "properties": {
          "main_event": {"type": "string"},
          "date": {"type": "string"},
          "location": {"type": "string"},
          "people": {"type": "array", "items": {"type": "string"}},
          "outcome": {"type": "string"}
        }
      }
    }
  }
}
```

```json Topic Classification theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-3-haiku",
      "prompt": "Classify this document into primary and secondary topics. Be specific.",
      "output_field": "topics",
      "output_schema": {
        "type": "object",
        "properties": {
          "primary_topic": {"type": "string"},
          "secondary_topics": {"type": "array", "items": {"type": "string"}},
          "confidence": {"type": "number", "minimum": 0, "maximum": 1}
        }
      }
    }
  }
}
```

```json Contact Information Extraction theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract all contact information (emails, phone numbers, addresses) from this document.",
      "output_field": "contacts",
      "output_schema": {
        "type": "object",
        "properties": {
          "emails": {"type": "array", "items": {"type": "string", "format": "email"}},
          "phones": {"type": "array", "items": {"type": "string"}},
          "addresses": {"type": "array", "items": {"type": "string"}}
        }
      }
    }
  }
}
```

## Output Schema

Define structured output using JSON Schema:

```json theme={null}
{
  "output_schema": {
    "type": "object",
    "properties": {
      "field_name": {"type": "string"},
      "numeric_field": {"type": "number"},
      "boolean_field": {"type": "boolean"},
      "array_field": {"type": "array", "items": {"type": "string"}},
      "enum_field": {"type": "string", "enum": ["option1", "option2", "option3"]}
    },
    "required": ["field_name"]
  }
}
```

### Supported Types

| Type | Description |
| - | - |
| `string` | Text values |
| `number` | Numeric values |
| `boolean` | True/false |
| `array` | Lists of items |
| `object` | Nested objects |

## Output Examples

### Without Schema

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Apple Inc. announced...",
  "entities": "Companies: Apple Inc., Microsoft\nPeople: Tim Cook, Satya Nadella"
}
```

### With Schema

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Great product, 5 stars!...",
  "review_analysis": {
    "sentiment": "positive",
    "rating_mentioned": 5,
    "pros": ["easy to use", "great value", "fast shipping"],
    "cons": ["packaging could be better"],
    "would_recommend": true
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 300-800ms per batch |
| **Batch size** | 5 documents default |
| **Token usage** | \~200 tokens per document |
| **Parallel** | Documents in a batch are sent concurrently, and batches run one after another |

> [!WARNING]
>
> LLM enrichment is expensive. Consider pre-computing extractions during indexing for frequently accessed data.

## Common Pipeline Patterns

### Search + Extract + Filter

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 10
          }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Extract the main topic and sentiment.",
        "output_field": "analysis",
        "output_schema": {
          "type": "object",
          "properties": {
            "topic": {"type": "string"},
            "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]}
          }
        }
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "analysis.sentiment",
        "operator": "eq",
        "value": "positive"
      }
    }
  }
]
```

### Entity Extraction Pipeline

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 10
          }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "prompt": "Extract all entities with their types and relationships.",
        "output_field": "entities",
        "output_schema": {
          "type": "object",
          "properties": {
            "people": {"type": "array", "items": {"type": "object", "properties": {"name": {"type": "string"}, "role": {"type": "string"}}}},
            "organizations": {"type": "array", "items": {"type": "string"}},
            "locations": {"type": "array", "items": {"type": "string"}},
            "dates": {"type": "array", "items": {"type": "string"}}
          }
        }
      }
    }
  }
]
```

## Writing Effective Prompts

### Good Prompts

```
✓ "Extract the product name, price, and key features from this product listing."
✓ "Identify all dates mentioned and their associated events."
✓ "Classify the sentiment as positive, neutral, or negative, and explain why."
```

### Poor Prompts

```
✗ "Analyze this document" (too vague)
✗ "Get the data" (not specific)
✗ "Tell me about it" (unclear output)
```

> [!TIP]
>
> Be specific about what to extract and in what format. When using `output_schema`, the LLM will conform to the structure.

## Multimodal Query Inputs

Pass images from your query inputs to the LLM alongside document content. This enables visual comparison, brand matching, and cross-modal analysis.

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `multimodal_inputs` | object | `null` | Map of INPUT field names to media types (`"image"`, `"video"`) |

### How It Works

1. Declare which INPUT fields carry multimodal content via `multimodal_inputs`
2. At runtime, the stage extracts URLs from `{{INPUT.field_name}}`
3. Images are sent to the LLM alongside the prompt and document content
4. Works with providers that support vision (Google Gemini, OpenAI GPT-4o, Anthropic Claude)

### Configuration Examples

```json Brand Logo Comparison theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-lite",
      "prompt": "Compare this brand logo to the document. Rate alignment 1-10: {{DOC.brand_name}}",
      "output_field": "brand_alignment",
      "multimodal_inputs": {
        "query_image": "image"
      }
    }
  }
}
```

```json Visual Product Match theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Does this product image match the description? {{DOC.description}}",
      "output_field": "visual_match",
      "output_schema": {
        "type": "object",
        "properties": {
          "matches": {"type": "boolean"},
          "confidence": {"type": "number"},
          "differences": {"type": "array", "items": {"type": "string"}}
        }
      },
      "multimodal_inputs": {
        "product_photo": "image"
      }
    }
  }
}
```

```json Multi-Image Reference theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-lite",
      "prompt": "Compare the reference image against this document's content: {{DOC.title}}",
      "output_field": "comparison",
      "multimodal_inputs": {
        "reference_image": "image",
        "style_guide": "image"
      }
    }
  }
}
```

### Calling with Multimodal Inputs

When using `multimodal_inputs`, pass the image URLs in the retriever's `inputs`:

```json theme={null}
{
  "inputs": {
    "query": "Find products similar to this",
    "query_image": "https://storage.example.com/brand-logo.png"
  },
  "stages": [
    {
      "stage_name": "feature_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": { "..." : "..." }
      }
    },
    {
      "stage_name": "llm_enrich",
      "stage_type": "enrich",
      "config": {
        "stage_id": "llm_enrich",
        "parameters": {
          "prompt": "Compare this image to: {{DOC.product_name}}",
          "output_field": "match_score",
          "multimodal_inputs": {"query_image": "image"}
        }
      }
    }
  ]
}
```

> [!NOTE]
>
> If a declared multimodal input field is missing from the query inputs at runtime, the stage proceeds without it (text-only mode). No error is raised.

***

## Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys. This gives you control over costs, rate limits, and API usage.

### Why Use BYOK?

| Benefit | Description |
| - | - |
| **Cost Control** | Use your own LLM provider account and billing |
| **Rate Limits** | Use your own rate limits instead of shared |
| **Compliance** | Keep API calls under your own account |
| **Key Rotation** | Rotate keys without changing retrievers |

### Setup

**1. Store your API key as a secret**

Store your LLM provider API key in the organization secrets vault:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

**2. Reference the secret in your stage**

Use the `api_key` parameter with template syntax:

```json theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract key entities from this document.",
      "output_field": "entities",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}
```

### BYOK Configuration Example

```json OpenAI BYOK theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Summarize this document in 2-3 sentences.",
      "output_field": "summary",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}
```

```json Anthropic BYOK theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "anthropic",
      "provider": "anthropic",
      "model_name": "claude-3-haiku-20240307",
      "prompt": "Extract the main topics from this content.",
      "output_field": "topics",
      "api_key": "{{secrets.anthropic_api_key}}"
    }
  }
}
```

```json Google BYOK theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "provider": "google",
      "model_name": "gemini-3.1-flash-lite",
      "prompt": "Classify the sentiment of this text.",
      "output_field": "sentiment",
      "api_key": "{{secrets.google_api_key}}"
    }
  }
}
```

### Supported Providers

| Provider | Secret Name Example | Models |
| - | - | - |
| OpenAI | `openai_api_key` | gpt-4o, gpt-4o-mini |
| Anthropic | `anthropic_api_key` | claude-3-haiku, claude-3-sonnet, claude-3-opus |
| Google | `google_api_key` | gemini-3.1-flash-lite, gemini-2.5-pro |

> [!NOTE]
>
> When `api_key` is not specified, the stage uses Mixpeek's default API keys and usage is charged to your Mixpeek account.

## Custom Enrichment Model (BYO Plugin)

Use your own enrichment model deployed as a [custom extractor](/docs/processing/custom-extractors) instead of a hosted LLM provider. Set `feature_uri` to route enrichment through your extractor's inference endpoint.

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | `null` | Feature URI of a custom enrichment plugin. Overrides `provider`/`model` when set. |

Your plugin must accept `{prompt: str, document: dict}` and return `{text: str}`.

### Configuration Example

```json theme={null}
{
  "stage_name": "my_enricher",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "feature_uri": "mixpeek://my_summarizer@1.0.0/summarize",
      "prompt": "Summarize: {{DOC.content}}",
      "output_field": "summary"
    }
  }
}
```

> [!TIP]
>
> Set `inference_type: "generate"` in your plugin's manifest to declare compatibility with LLM stages.

## Error Handling

| Error | Behavior |
| - | - |
| LLM call fails (timeout, rate limit, provider error) | Retried once when the error is retryable, then `output_field` holds an object with an `error` message |
| Response is not valid JSON | Raw text in `output_field` |
| More than 10 documents reach the stage | Stage fails; reduce the working set first |
| Invalid API key | `output_field` holds an `error` object with the provider's auth failure |

## Related

* [LLM Filter](/docs/retrieval/stages/llm-filter) - Filter using LLM evaluation
* [Taxonomy Enrich](/docs/retrieval/stages/taxonomy-enrich) - Predefined classification
* [JSON Transform](/docs/retrieval/stages/json-transform) - Template-based transformation


# Taxonomy Enrich
Source: https://docs.mixpeek.com/docs/retrieval/stages/taxonomy-enrich

Classify and tag documents using predefined taxonomies

![Taxonomy Enrich stage showing document classification with hierarchical taxonomies](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/taxonomy-enrich.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=64bc8288d3799384b8bd0097193b27e3)

The Taxonomy Enrich stage classifies documents against predefined taxonomies, adding structured category labels and hierarchical classifications to your search results.

> [!NOTE]
>
> **Stage Category**: ENRICH (Enriches documents with classifications)
>
> **Transformation**: N documents → N documents (with taxonomy labels added)

## When to Use

| Use Case | Description |
| - | - |
| **Content categorization** | Auto-classify documents into topics |
| **Faceted search** | Add filterable category facets |
| **Compliance tagging** | Apply regulatory classifications |
| **Product taxonomy** | Classify into product hierarchies |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Free-form tagging | `llm_enrich` |
| Pre-classified content | Skip this stage |
| Custom classification logic | `api_call` to custom service |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `taxonomy_id` | string | `{{TAXONOMY_ID}}` | ID of the taxonomy to use. Replace the placeholder with a real taxonomy ID. The taxonomy's `input_mappings` decide which vector field of each document is matched |
| `fields` | array | `[]` | Taxonomy node fields to merge into the document. Each item has `field_path`, an optional `target_field` to rename it, and `merge_mode` (`replace` or `append`). When empty, the fields configured on the taxonomy's source collection are used |
| `top_k` | integer | `3` | Maximum taxonomy assignments kept per document (1 to 50) |
| `min_score` | float | `0.0` | Minimum similarity score (0.0 to 1.0) an assignment needs to be kept |
| `when` | object | `null` | Condition that selects which documents are classified. Documents that do not match pass through unchanged |

Each document stays one document: `top_k` limits how many assignments are considered for it and does not duplicate the document.

## Configuration Examples

```json Basic Classification theme={null}
{
  "stage_name": "taxonomy_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "taxonomy_enrich",
    "parameters": {
      "taxonomy_id": "product_categories"
    }
  }
}
```

```json High-Confidence Only theme={null}
{
  "stage_name": "taxonomy_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "taxonomy_enrich",
    "parameters": {
      "taxonomy_id": "topics",
      "top_k": 1,
      "min_score": 0.8
    }
  }
}
```

```json Limit Matches Per Document theme={null}
{
  "stage_name": "taxonomy_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "taxonomy_enrich",
    "parameters": {
      "taxonomy_id": "industry_taxonomy",
      "top_k": 3
    }
  }
}
```

```json Multiple Classifications theme={null}
{
  "stage_name": "taxonomy_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "taxonomy_enrich",
    "parameters": {
      "taxonomy_id": "content_types",
      "top_k": 5,
      "min_score": 0.3
    }
  }
}
```

## Output Schema

The stage writes assignments as flat fields on the document. Field names start with a prefix taken from the taxonomy ID: a leading `tax_` is removed, the rest is lowercased, and runs of characters other than letters and digits become `_`. The taxonomy `tax_product_categories` gives the prefix `product_categories`.

| Field | Written when |
| - | - |
| `_taxonomy_{prefix}_taxonomy_id` | An assignment is kept |
| `_taxonomy_{prefix}_id` | The matched node has an ID |
| `_taxonomy_{prefix}_label` | The matched node has a label |
| `_taxonomy_{prefix}_score` | The assignment has a similarity score |
| `_taxonomy_{prefix}_matched_at` | The match has a timestamp |
| `_taxonomy_{prefix}_source_document_id` | The matched node's source document ID is known |
| `{prefix}_path` | The assignment carries a hierarchy path |
| `{prefix}_hierarchy_level` and `{prefix}_level` | The assignment carries a hierarchy level, or a path from which one is derived |

When several assignments are kept, these fields describe the highest-scoring one. Values selected by `fields` are merged for every kept assignment, from the highest score to the lowest.

### Basic Classification

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Latest smartphone with 5G connectivity...",
  "_taxonomy_product_categories_taxonomy_id": "tax_product_categories",
  "_taxonomy_product_categories_id": "col_smartphones",
  "_taxonomy_product_categories_label": "Smartphones",
  "_taxonomy_product_categories_score": 0.95
}
```

### Hierarchical Taxonomy

For a hierarchical taxonomy the path and level fields are added as well.

```json theme={null}
{
  "document_id": "doc_456",
  "content": "Investment banking services...",
  "_taxonomy_industry_taxonomy_id": "tax_industry",
  "_taxonomy_industry_label": "Investment Banking",
  "_taxonomy_industry_score": 0.91,
  "industry_path": ["finance", "banking", "investment"],
  "industry_hierarchy_level": 2,
  "industry_level": 2
}
```

### Merged Enrichment Fields

Fields listed in `fields` are copied from the matched node onto the document, under `target_field` when one is set. With `fields` set to `[{"field_path": "label", "target_field": "visual_style"}]`, the node's `label` lands in `visual_style`.

```json theme={null}
{
  "document_id": "doc_789",
  "content": "Sunset over the harbor...",
  "visual_style": "Golden hour"
}
```

### No Match

When no assignment reaches `min_score`, the document continues without any taxonomy fields added.

```json theme={null}
{
  "document_id": "doc_000",
  "content": "Random unrelated content..."
}
```

## Taxonomy Structure

Taxonomies are hierarchical classification systems:

```
Electronics
├── Mobile Devices
│   ├── Smartphones
│   ├── Tablets
│   └── Wearables
├── Computers
│   ├── Laptops
│   ├── Desktops
│   └── Components
└── Audio
    ├── Headphones
    └── Speakers
```

A match reports:

* **ID**: Identifier of the matched node, usually a collection ID (`_taxonomy_{prefix}_id`)
* **Label**: Human-readable name (`_taxonomy_{prefix}_label`)
* **Level**: Depth in the hierarchy, where 0 is the root (`{prefix}_hierarchy_level`)

## Performance

| Metric | Value |
| - | - |
| **Latency** | 10-50ms per document |
| **Batch processing** | Automatic |
| **Model type** | Embedding-based classification |
| **Engine calls** | One taxonomy join call per stage run, covering every document that passes `when` |

> [!TIP]
>
> Pre-compute taxonomy embeddings for faster classification. Use `top_k: 1` and higher `min_score` when you only need the best match.

## Common Pipeline Patterns

### Search + Classify + Filter

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "taxonomy_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "taxonomy_enrich",
      "parameters": {
        "taxonomy_id": "product_categories",
        "top_k": 1,
        "min_score": 0.7
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "field": "_taxonomy_product_categories_label",
          "operator": "starts_with",
          "value": "{{INPUT.category_filter}}"
        }
      }
    }
  }
]
```

### Multi-Taxonomy Classification

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 50 }
        ],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "taxonomy_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "taxonomy_enrich",
      "parameters": {
        "taxonomy_id": "topics"
      }
    }
  },
  {
    "stage_name": "taxonomy_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "taxonomy_enrich",
      "parameters": {
        "taxonomy_id": "sentiment"
      }
    }
  }
]
```

### Faceted Search Results

```json theme={null}
[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 100 }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "taxonomy_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "taxonomy_enrich",
      "parameters": {
        "taxonomy_id": "categories",
        "top_k": 3
      }
    }
  }
]
```

## Error Handling

| Condition | Behavior |
| - | - |
| Unknown `taxonomy_id` | Documents pass through unchanged and a warning names the taxonomy |
| No match found | Document continues with no taxonomy fields added |
| Taxonomy join call fails | Documents continue with no taxonomy fields added |
| Low confidence | Assignments below `min_score` are dropped |

## Related

* [LLM Enrich](/docs/retrieval/stages/llm-enrich) - Free-form extraction
* [Document Enrich](/docs/retrieval/stages/document-enrich) - Collection joins
* [Attribute Filter](/docs/retrieval/stages/attribute-filter) - Filter by categories


# Traverse Edge
Source: https://docs.mixpeek.com/docs/retrieval/stages/traverse-edge

Follow typed relationships (edges) from documents to their linked documents

The Traverse Edge stage follows **[object edges](/docs/platform/data-model)** — typed, directed, customer-owned relationships stored on a document — to fetch the documents they point to. Starting from the documents in the pipeline, it reads each document's root-level `edges`, filters by edge type and direction, and returns the linked documents (carrying the edge's attributes with each result).

> [!NOTE]
>
> **Stage Category**: APPLY (relationship traversal)
>
> **Transformation**: N documents → M linked documents (optionally including the originating documents)

Edges are saved onto documents at ingestion — for example, an assembled ad linked to every piece of footage it uses (see [Iconik project-file linkage](/docs/integrations/object-storage/iconik#project-file-linkage)). Traverse Edge is how you follow those saved relationships at query time instead of recomputing them.

> [!NOTE]
>
> **Traversal returns every linked document, across all pages of the underlying scan.**
> An older storage-layer defect sorted within a page rather than across the whole key
> range, so linked documents past the first page were never reached. The stage returned
> HTTP 200 with a short result and no warning, which is indistinguishable from a document
> that genuinely has fewer links.
>
> On a deployment that predates the fix, the symptom is a result that is short rather than
> wrong. Cross-check against the source document's own `edges`, which are always reliable,
> and treat a traversal result as a lower bound until a traversal you can predict returns
> everything you expect.

> [!NOTE]
>
> **Writing edges is unaffected.** Edges added by PATCH read back intact in both
> directions, so nothing about how you create edges needs to change.

## When to Use

| Use Case | Description |
| - | - |
| **Footage ↔ ad linkage** | From an ad, fetch every footage clip it uses (with clip order + start-ticks); from footage, find the ads that use it |
| **Bill-of-materials / composition** | Follow "part-of" relationships from an assembly to its components |
| **Reference expansion** | Pull the documents a result explicitly links to, without a similarity search |
| **Graph hops** | Walk saved relationships between entities you've modeled as edges |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Similarity / semantic matching | `feature_search` |
| Joining by a shared field value | `document_enrich` |
| Comparing content across collections | `cross_compare` |
| Filtering on an attribute | `attribute_filter` |

## How it works

1. For each document entering the stage, the stage reads its root-level `edges` list.
2. Edges are kept when their `type` matches `edge_type` and their `direction` matches the configured `direction`.
3. Each kept edge's `target_object_id` is resolved to the linked document(s); with `target_collection_id` set, only documents from that collection are returned.
4. Matched documents are returned with the originating edge's `attributes` attached (e.g. `clip_order`, `start_ticks_in`/`start_ticks_out`), so downstream stages can use them.

Because edges are stored **reciprocally** (an edge and its inverse can be written on both endpoints), you can traverse the same relationship from either side by choosing the matching `edge_type` and `direction`.

## Creating edges on objects

Edges are **customer-owned data** written at the **root** of an object when you
create it (never under `_internal`). They flow automatically from the object,
through your collections, into each document's search payload — so a stage can
follow them at query time without you re-computing anything.

Pass an `edges` array to `POST /v1/buckets/{bucket_id}/objects`:

```json theme={null}
{
  "key_prefix": "ads/ad1.txt",
  "blobs": [{"property": "content", "type": "text", "data": "final produced ad one"}],
  "edges": [
    {
      "type": "uses_footage",
      "target_object_id": "obj_1a2b3c...",
      "direction": "out",
      "attributes": {"clip_order": 3, "start_ticks_in": 123456789, "start_ticks_out": 987654321}
    }
  ]
}
```

| Edge field | Description |
| - | - |
| `type` | The edge type you'll match on in `traverse_edge` (e.g. `uses_footage`). |
| `target_object_id` | The object this edge points to. It resolves to that object's document(s) at traversal time. |
| `direction` | `out` or `in` — which way the edge points. Follow it with the stage's matching `direction`. |
| `attributes` | Arbitrary key/values carried on the edge (e.g. `clip_order`, `start_ticks_in`/`start_ticks_out`). They ride into results via `traversed_via`. |

> [!TIP]
>
> Write **reciprocal** edges to walk a relationship from either end: put
> `uses_footage`/`out` on the ad and `used_in_ad`/`out` on the footage, each
> carrying the same attributes. Connectors like [Iconik](/docs/integrations/object-storage/iconik#project-file-linkage)
> populate this shape automatically from editorial project files.

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `edge_type` | string \| string\[] | *Required* | Edge type(s) to follow, e.g. `"uses_footage"` or `["uses_footage", "used_in_ad"]` |
| `direction` | string | `any` | Which edge directions to follow: `out`, `in`, or `any` |
| `target_collection_id` | string | `null` | Only return traversed documents from this collection (else all collections) |
| `include_source` | boolean | `false` | Also keep the originating documents in the output |
| `max_per_source` | integer | `50` | Max edges to follow per source document (guards fan-out) |
| `limit` | integer | `200` | Max total traversed documents to fetch across all sources |

## Example

Starting from an ad document, fetch every piece of footage it uses:

```json theme={null}
{
  "stage_name": "footage_used",
  "config": {
    "stage_id": "traverse_edge",
    "parameters": {
      "edge_type": "uses_footage",
      "direction": "out"
    }
  }
}
```

`edge_type` also accepts a **list** to follow several relationship types in a single
stage — e.g. `"edge_type": ["uses_footage", "used_in_ad"]` returns documents reached
by either edge, each still labelled with its own `traversed_via`.

Each returned footage document carries the edge attributes from the ad that referenced it — `clip_order`, `start_ticks_in`, and `start_ticks_out` — so you know exactly where in the footage each clip was taken.

## The `traversed_via` response

Every document produced by traversal carries a `traversed_via` field describing
the edge (or edges) that reached it:

| Field | Description |
| - | - |
| `edge_type` | The edge type that was followed. |
| `direction` | The direction it was followed in (`out` / `in`). |
| `attributes` | The attributes stored on that edge (e.g. `clip_order`, `start_ticks_in`/`start_ticks_out`). |
| `source_document_id` | The document the traversal started from — the other end of the edge. |

> [!WARNING]
>
> `traversed_via` is an **object** when a single source reached the document, but
> a **list** when several sources traversed to the *same* document (a many-to-one
> fan-in — the document is returned once, with one entry per incoming edge). Handle
> both shapes: check whether `traversed_via` is a list before indexing it.

Single source (footage → ad, `used_in_ad`) — `traversed_via` is an object:

```json theme={null}
{
  "document_id": "doc_9bd3572ecf180e72b7042b96",
  "traversed_via": {
    "edge_type": "used_in_ad",
    "direction": "out",
    "attributes": {"clip_order": 3, "start_ticks_in": 123456789, "start_ticks_out": 987654321},
    "source_document_id": "doc_e32de934c79775b870047c62"
  }
}
```

Multiple sources fan in to one document (ad → footage, `uses_footage`, where two
ads reference the same footage) — `traversed_via` is a list:

```json theme={null}
{
  "document_id": "doc_0b84044df6cbd7e4f10bbcbc",
  "traversed_via": [
    {
      "edge_type": "uses_footage",
      "direction": "out",
      "attributes": {"clip_order": 2, "start_ticks_in": 2000000, "start_ticks_out": 4000000},
      "source_document_id": "doc_a56a844765839e4abfcdbca2"
    },
    {
      "edge_type": "uses_footage",
      "direction": "out",
      "attributes": {"clip_order": 4, "start_ticks_in": 4000000, "start_ticks_out": 8000000},
      "source_document_id": "doc_7547528e54cc390a0fb59b1e"
    }
  ]
}
```

Because reciprocal edges are stored on both endpoints, the same pair is walkable
from either side — traverse `uses_footage`/`out` from an ad to reach its footage,
or `used_in_ad`/`out` from that footage to reach the ads that use it — and the
`attributes` (clip order, ticks) are identical in both directions.

## Related

* [Objects](/docs/platform/data-model) — how edges are stored on objects and flow to documents
* [Iconik integration](/docs/integrations/object-storage/iconik#project-file-linkage) — how footage↔ad edges are captured at ingestion
* [Cross Compare](/docs/retrieval/stages/cross-compare) — match content across collections by similarity instead of saved edges


# Unwind
Source: https://docs.mixpeek.com/docs/retrieval/stages/unwind

Decompose array fields into separate documents for per-element processing

![Unwind stage showing array decomposition into separate documents](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/unwind.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=39019ad7c201222911fd58ac2cf787f9)

The Unwind stage decomposes array fields into separate documents, producing one output document per array element. This is the retriever pipeline equivalent of MongoDB's `$unwind`, Snowflake's `LATERAL FLATTEN`, and Spark's `explode()`.

> [!NOTE]
>
> **Stage Category**: APPLY (Expands documents)
>
> **Transformation**: N documents → M documents (where M ≥ N, one per array element)

## When to Use

| Use Case | Description |
| - | - |
| **Tag expansion** | Decompose multi-tag documents for per-tag analysis |
| **Segment decomposition** | Flatten video/audio segments into individual results |
| **Author attribution** | Expand author lists for per-author scoring |
| **Chunk flattening** | Convert grouped chunks back into individual documents |
| **Category expansion** | Expand multi-category items for faceted search |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Filtering documents | `attribute_filter` or `llm_filter` |
| Restructuring without expansion | `json_transform` |
| Sorting documents | `sort_attribute` or `sort_relevance` |
| Grouping documents | `group_by` (inverse operation) |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `field` | string | *required* | Dot-notation path to the array field to unwind |
| `preserve_null_and_empty` | boolean | `false` | Keep documents where array is null/missing/empty |
| `include_array_index` | string | `null` | Field name to store the element's array index |
| `output_field` | string | `null` | Place unwound element in this field instead of replacing |

## Configuration Examples

```json Basic Tag Unwind theme={null}
{
  "stage_name": "unwind",
  "stage_type": "apply",
  "config": {
    "stage_id": "unwind",
    "parameters": {
      "field": "metadata.tags"
    }
  }
}
```

```json With Array Index Tracking theme={null}
{
  "stage_name": "unwind",
  "stage_type": "apply",
  "config": {
    "stage_id": "unwind",
    "parameters": {
      "field": "content.segments",
      "include_array_index": "segment_index",
      "preserve_null_and_empty": true
    }
  }
}
```

```json Output to Separate Field theme={null}
{
  "stage_name": "unwind",
  "stage_type": "apply",
  "config": {
    "stage_id": "unwind",
    "parameters": {
      "field": "metadata.authors",
      "output_field": "current_author"
    }
  }
}
```

## How It Works

1. For each input document, extracts the array value at the specified `field` path
2. If the value is an array with K elements, produces K output documents
3. Each output document preserves all original fields, with the array field replaced by a single element
4. Documents with null/empty arrays are either dropped or preserved based on `preserve_null_and_empty`
5. Non-array values are passed through unchanged

> [!TIP]
>
> Use `include_array_index` when you need to reconstruct the original order later, such as when reassembling video segments after per-segment scoring.

## Performance

| Metric | Value |
| - | - |
| **Latency** | \< 5ms |
| **Memory** | Proportional to output count |
| **Cost** | Free |
| **Complexity** | O(total array elements) |

## Common Pipeline Patterns

### Per-Tag Scoring

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 50}],
        "final_top_k": 50
      }
    }
  },
  {
    "stage_name": "unwind",
    "stage_type": "apply",
    "config": {
      "stage_id": "unwind",
      "parameters": {
        "field": "metadata.tags",
        "include_array_index": "tag_index"
      }
    }
  },
  {
    "stage_name": "group_by",
    "stage_type": "apply",
    "config": {
      "stage_id": "group_by",
      "parameters": {
        "group_by_field": "metadata.tags"
      }
    }
  }
]
```

### Segment-Level Retrieval

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [{"feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": {"input_mode": "text", "value": "{{INPUT.query}}"}, "top_k": 20}],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "unwind",
    "stage_type": "apply",
    "config": {
      "stage_id": "unwind",
      "parameters": {
        "field": "content.segments",
        "output_field": "current_segment"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "current_segment"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Field path doesn't exist | Document dropped (or preserved if `preserve_null_and_empty=true`) |
| Field is not an array | Document passed through unchanged |
| Empty array | Document dropped (or preserved with null if `preserve_null_and_empty=true`) |
| Null field value | Same as empty array behavior |

## Related

* [JSON Transform](/docs/retrieval/stages/json-transform) - Restructure document fields without expansion
* [Group By](/docs/retrieval/stages/group-by) - Inverse operation: group documents by field value
* [Deduplicate](/docs/retrieval/stages/deduplicate) - Remove duplicates after expansion


# Web Scrape
Source: https://docs.mixpeek.com/docs/retrieval/stages/web-scrape

Fetch and extract page content from URLs in your documents

![Web Scrape stage showing content extraction from URLs](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/retrievers/web-scrape.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=e12278d3bcf69dd335260e72367de304)

The Web Scrape stage fetches content from web URLs using the Engine's Playwright service. It handles JavaScript rendering and main-content extraction to add web content to your retrieval pipeline.

> [!NOTE]
>
> **Stage Category**: APPLY (Enriches documents with scraped content)
>
> **Transformation**: N documents → N documents (with web content added)

## When to Use

| Use Case | Description |
| - | - |
| **URL enrichment** | Extract content from URLs in documents |
| **Reference expansion** | Scrape linked references for context |
| **Content aggregation** | Pull in external content sources |
| **Real-time content** | Access current webpage content |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Searching the web | `external_web_search` (Exa) |
| Static content already indexed | Use indexed content |
| High-volume scraping | Pre-index content in Mixpeek |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `url_field` | string | `metadata.url` | Document field containing the URL to scrape |
| `output_field` | string | `web_content` | Field that receives the extracted content |
| `strategy` | string | `auto` | `static`, `javascript`, or `auto` |
| `timeout` | integer | `10` | Request timeout in seconds (1 to 60) |
| `include_html` | boolean | `false` | Include the raw HTML in the output |
| `min_content_length` | integer | `500` | With `auto`, fall back to JavaScript when static parsing returns fewer characters |
| `when` | object | none | Only scrape documents that match this condition |
| `on_error` | string | `skip` | `skip`, `remove`, or `raise` |
| `batch_size` | integer | `5` | Concurrent fetch requests (1 to 20) |

## Configuration Examples

```json Basic URL Scraping theme={null}
{
  "stage_name": "web_scrape",
  "stage_type": "apply",
  "config": {
    "stage_id": "web_scrape",
    "parameters": {
      "url_field": "metadata.source_url",
      "output_field": "page_content"
    }
  }
}
```

```json With JavaScript Rendering theme={null}
{
  "stage_name": "web_scrape",
  "stage_type": "apply",
  "config": {
    "stage_id": "web_scrape",
    "parameters": {
      "url_field": "metadata.url",
      "output_field": "content",
      "strategy": "javascript",
      "timeout": 30
    }
  }
}
```

```json Selective Scraping theme={null}
{
  "stage_name": "web_scrape",
  "stage_type": "apply",
  "config": {
    "stage_id": "web_scrape",
    "parameters": {
      "url_field": "metadata.reference_url",
      "output_field": "reference",
      "when": {
        "AND": [
          {"field": "score", "operator": "gte", "value": 0.8}
        ]
      }
    }
  }
}
```

```json HTML Extraction theme={null}
{
  "stage_name": "web_scrape",
  "stage_type": "apply",
  "config": {
    "stage_id": "web_scrape",
    "parameters": {
      "url_field": "url",
      "output_field": "html_content",
      "include_html": true,
      "timeout": 15
    }
  }
}
```

## Output Schema

### Default Output

The extracted content is stored as an object at `output_field`.

```json theme={null}
{
  "document_id": "doc_123",
  "metadata": {
    "source_url": "https://example.com/article"
  },
  "page_content": {
    "url": "https://example.com/article",
    "final_url": "https://example.com/article",
    "title": "Article Title",
    "text": "Article content here...",
    "content_length": 1842,
    "metadata": {
      "author": "Jane Doe",
      "date": "2025-01-15",
      "description": "Meta description",
      "sitename": "Example",
      "language": "en"
    },
    "strategy_used": "static"
  }
}
```

### With Raw HTML

Set `include_html` to `true` to add the page HTML to the same object.

```json theme={null}
{
  "document_id": "doc_123",
  "html_content": {
    "url": "https://example.com/article",
    "final_url": "https://example.com/article",
    "title": "Article Title",
    "text": "Article content here...",
    "content_length": 1842,
    "metadata": {},
    "strategy_used": "static",
    "html": "<html>...</html>"
  }
}
```

### Failed Fetch

With the default `on_error: skip`, a document whose fetch fails passes through unchanged, without the output field.

## Scraping Strategies

| Strategy | Description |
| - | - |
| `static` | Fast HTML parsing without JavaScript. Best for articles and blogs |
| `javascript` | Full browser rendering. Slower, and required for single-page apps |
| `auto` | Tries `static` first and falls back to `javascript` when the extracted text is shorter than `min_content_length` |

> [!TIP]
>
> Use `strategy: "javascript"` when scraping JavaScript-heavy sites, and raise `timeout` to 20-30 seconds.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 1-10s (depends on page complexity) |
| **Concurrent requests** | `batch_size`, 5 by default and up to 20 |
| **Timeout default** | 10 seconds |

> [!WARNING]
>
> Web scraping adds significant latency. Use sparingly and consider pre-indexing frequently accessed content.

## Common Pipeline Patterns

### Enrich Documents with Referenced Content

```json theme={null}
[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 10 }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.has_url",
        "operator": "eq",
        "value": true
      }
    }
  },
  {
    "stage_name": "web_scrape",
    "stage_type": "apply",
    "config": {
      "stage_id": "web_scrape",
      "parameters": {
        "url_field": "metadata.source_url",
        "output_field": "source_content"
      }
    }
  }
]
```

### Scrape and Summarize

```json theme={null}
[
  {
    "stage_name": "web_scrape",
    "stage_type": "apply",
    "config": {
      "stage_id": "web_scrape",
      "parameters": {
        "url_field": "metadata.url",
        "output_field": "page_content"
      }
    }
  },
  {
    "stage_name": "json_transform",
    "stage_type": "apply",
    "config": {
      "stage_id": "json_transform",
      "parameters": {
        "template": {
          "content": "{{ DOC.page_content.text }}",
          "source": "{{ DOC.metadata.url }}"
        }
      }
    }
  },
  {
    "stage_name": "summarize",
    "stage_type": "reduce",
    "config": {
      "stage_id": "summarize",
      "parameters": {
        "provider": "google",
        "model_name": "gemini-2.5-flash-lite",
        "prompt": "Summarize the key points from these webpages:\n\n{{DOCUMENTS}}"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Missing or empty URL field | Follows `on_error` |
| Invalid URL, timeout, or HTTP error such as 404 or 403 | Follows `on_error` |

`on_error` accepts three values. `skip` (default) passes the document through unchanged. `remove` drops the document from the results. `raise` fails the stage.

## Rate Limits and Best Practices

1. **Batch wisely**: Limit to 5-10 URLs per pipeline run
2. **Cache results**: Consider storing scraped content
3. **Use timeouts**: Set an appropriate `timeout` (in seconds) for your use case

## Related

* [External Web Search](/docs/retrieval/stages/external-web-search) - Search the web (Exa)
* [API Call](/docs/retrieval/stages/api-call) - General HTTP enrichment
* [Document Enrich](/docs/retrieval/stages/document-enrich) - Collection joins


# Typeahead
Source: https://docs.mixpeek.com/docs/retrieval/typeahead

Prefix suggestions for a search box, drawn concurrently from field values, collection names, and the caller's own recent searches

`GET /v1/retrievers/typeahead` returns autocomplete suggestions for a prefix. It
queries three namespace-scoped sources concurrently and returns each separately,
plus one flat ranked list you can render directly.

```bash theme={null}
curl -sS "$MP_API_URL/v1/retrievers/typeahead?q=nik&fields=metadata.talent,metadata.production&limit=10" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $NS"
```

```json Response theme={null}
{
  "query": "nik",
  "suggestions": ["Nike", "Nikon", "nike-summer-2026"],
  "values": [
    { "field": "metadata.talent", "value": "Nike", "count": 412 },
    { "field": "metadata.production", "value": "Nikon", "count": 37 }
  ],
  "collections": [
    { "collection_id": "col_abc", "collection_name": "nike-summer-2026" }
  ],
  "recent_searches": ["nike hero cut"],
  "warnings": []
}
```

## Parameters

| Parameter | Meaning |
| - | - |
| `q` | Required. The typed prefix, 1 to 200 characters. |
| `fields` | Comma-separated metadata field paths to suggest values from, for example `metadata.talent,metadata.production`. Omit it to skip the field-value source entirely. |
| `limit` | 1 to 50, default 10. |

> [!WARNING]
>
> **`limit` is per source, not per response.** With all three sources answering,
> a `limit` of 10 can return up to 30 entries in `suggestions`. Size your UI
> against the total rather than the parameter.
>
> **At most 5 `fields` are consulted.** Passing more does not fail the request,
> so a sixth field simply never contributes suggestions and nothing says so.
> Each field also needs a keyword index to be searchable.

## Reading the response

| Field | Contents |
| - | - |
| `query` | The prefix that was matched. Always present. |
| `suggestions` | Flat ranked texts across all sources. |
| `values` | Field-value matches, each with `field`, `value` and a document `count`. |
| `collections` | Matching collections, each with `collection_id` and `collection_name`. |
| `recent_searches` | The caller's own recent queries, newest first. |
| `warnings` | Sources that could not be consulted on this request. |

> [!NOTE]
>
> **A source that cannot answer is dropped, not fatal.** The request still
> returns 200 with whatever the other sources produced, and the one that failed
> is named in `warnings`.
>
> So a thin result set has two different causes: few matches, or a source that
> did not run. Read `warnings` to tell them apart. This is the one place the API
> tells you which of the two happened, and it costs one field to check.

`recent_searches` is scoped to the calling identity, so two users typing the same
prefix get different suggestions from that source and identical suggestions from
the other two.

## Related

* [Search Widget](/docs/integrations/search-widget) for the prebuilt UI components
* [Retrievers](/docs/retrieval/retrievers) for running the search itself


# Batch Diagnostics & Troubleshooting
Source: https://docs.mixpeek.com/docs/troubleshoot/batch-diagnostics

Use the API to diagnose, troubleshoot, and fix batch processing issues without accessing infrastructure.

The Mixpeek API provides complete observability into batch processing jobs. You can diagnose issues, cancel stuck jobs, retry failed tiers with modified resources, and trigger self-healing — all through the API.

## Quick Diagnosis

Call the diagnose endpoint to get a complete picture of a batch's health:

```python Python theme={null}
import requests

response = requests.get(
    "https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/diagnose",
    headers={"Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace"}
)
diagnostic = response.json()

print(f"Status: {diagnostic['status']}")
print(f"Failure category: {diagnostic['failure_category']}")
print(f"Failed docs: {diagnostic['failed_document_count']}")
for rec in diagnostic['recommendations']:
    print(f"  → {rec}")
```

```javascript JavaScript theme={null}
const response = await fetch(
  `https://api.mixpeek.com/v1/buckets/${bucketId}/batches/${batchId}/diagnose`,
  { headers: { "Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace" } }
);
const diagnostic = await response.json();

console.log(`Status: ${diagnostic.status}`);
console.log(`Failure: ${diagnostic.failure_category}`);
diagnostic.recommendations.forEach(r => console.log(`  → ${r}`));
```

```bash cURL theme={null}
curl -X GET "https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/diagnose" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```

The response includes:

* **status** and **failure\_category** — programmatic failure classification (infrastructure, timeout, orphaned, pipeline)
* **infrastructure\_events** — OOM, preemption, node failures, Ray bugs with timestamps
* **per\_tier** — timing, submission params, and resource details per tier
* **failed\_documents\_sample** — first 10 failed documents with error details
* **recommendations** — actionable next steps based on the failure type

## Common Failure Scenarios

### Out of Memory (OOM)

The diagnose endpoint will show `failure_category: "infrastructure"` with an infrastructure event of type `oom`.

**Fix:** Retry the failed tier with more resources:

```python Python theme={null}
requests.post(
    f"https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/tiers/{tier_num}/retry",
    headers={"Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace"},
    json={"requires_gpu": True, "priority": 50}
)
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/tiers/{tier_num}/retry" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace" \
  -H "Content-Type: application/json" \
  -d '{"requires_gpu": true, "priority": 50}'
```

### Stuck Job

If a tier shows `IN_PROGRESS` but `last_activity_at` is stale (minutes old), the job may be stuck.

**Fix:** Run stuck detection, then cancel the stuck tier:

```python Python theme={null}
# Detect stuck jobs
requests.post(
    f"https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/tiers/{tier_num}/heal",
    headers={"Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace"},
    json={"action": "detect_stuck"}
)

# Cancel the stuck tier
requests.post(
    f"https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/tiers/{tier_num}/cancel",
    headers={"Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace"}
)
```

```bash cURL theme={null}
# Detect stuck
curl -X POST ".../tiers/{tier_num}/heal" -H "..." -d '{"action": "detect_stuck"}'
# Cancel
curl -X POST ".../tiers/{tier_num}/cancel" -H "..."
```

### Duplicate Jobs

If multiple Ray jobs are running for the same extractor in a tier:

```python Python theme={null}
requests.post(
    f"https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/tiers/{tier_num}/heal",
    headers={"Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace"},
    json={"action": "kill_duplicates"}
)
```

```bash cURL theme={null}
curl -X POST ".../tiers/{tier_num}/heal" -d '{"action": "kill_duplicates"}'
```

### Cancel a Single Job (Not the Whole Batch)

If one extractor job in a multi-extractor tier is failing but others are fine:

```python Python theme={null}
requests.post(
    f"https://api.mixpeek.com/v1/buckets/{bucket_id}/batches/{batch_id}/tiers/{tier_num}/jobs/{ray_job_id}/cancel",
    headers={"Authorization": "Bearer YOUR_API_KEY", "X-Namespace": "your-namespace"}
)
```

```bash cURL theme={null}
curl -X POST ".../tiers/{tier_num}/jobs/{ray_job_id}/cancel" -H "..."
```

## Submission Parameters

Every batch tier now persists its submission parameters — the resources, GPU setting, plugins, and entrypoint used when the job was submitted. Access them in the batch response:

```json theme={null}
{
  "tier_tasks": [{
    "submission_params": {
      "entrypoint": "python -m engine.pipelines.entrypoint",
      "deployment_mode": "gke",
      "requires_gpu": true,
      "num_cpus": 1,
      "memory_bytes": 8589934592,
      "priority": 100,
      "plugin_archives": null,
      "extractor_name": "universal_extractor_v1"
    }
  }]
}
```

## Stage Timing Breakdown

The batch progress now includes `stage_history` — a timing breakdown of each completed processing stage:

```json theme={null}
{
  "progress": {
    "stage_history": [
      {"name": "loading", "duration_seconds": 2.5},
      {"name": "processing", "duration_seconds": 45.3},
      {"name": "writing", "duration_seconds": 8.1}
    ]
  }
}
```

Use this to identify bottlenecks — if "processing" takes 90% of the time, the extractor itself is the bottleneck. If "writing" is slow, the vector store may be under pressure.


# Backup and Restore
Source: https://docs.mixpeek.com/docs/vector-store/backup-and-restore

Snapshot a namespace, check the snapshot, and restore it beside the source, to a point in time, or in place

A snapshot captures a namespace's configuration, documents, vectors and a manifest of its bucket objects. You restore a snapshot into a new namespace beside the source, or over the source itself.

Every call takes your API key in `Authorization: Bearer`. The path names the source namespace by id or name.

## Take a snapshot

```python Python theme={null}
import os, requests

BASE = "https://api.mixpeek.com/v1/namespaces/my-namespace"
H = {"Authorization": f"Bearer {os.environ['MIXPEEK_API_KEY']}"}

snap = requests.post(f"{BASE}/snapshots", headers=H, json={"copy_s3_objects": True}).json()
snapshot_id = snap["snapshot_id"]
```

```javascript JavaScript theme={null}
const BASE = "https://api.mixpeek.com/v1/namespaces/my-namespace";
const H = {
  Authorization: `Bearer ${process.env.MIXPEEK_API_KEY}`,
  "Content-Type": "application/json",
};

const snap = await fetch(`${BASE}/snapshots`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({ copy_s3_objects: true }),
}).then((r) => r.json());
const snapshotId = snap.snapshot_id;
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/my-namespace/snapshots" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"copy_s3_objects": true}'
```

`copy_s3_objects` copies the bucket objects into the snapshot. Without it, the snapshot lists the objects and a restore reads them from the bucket. Set it if you might delete the namespace and restore it later.

The snapshot runs in the background. `GET /v1/namespaces/{namespace}/snapshots/{snapshot_id}` shows its status.

## Check that it can restore

```python Python theme={null}
report = requests.get(f"{BASE}/snapshots/{snapshot_id}/verify", headers=H, params={"deep": "true"}).json()
```

```javascript JavaScript theme={null}
const report = await fetch(`${BASE}/snapshots/${snapshotId}/verify?deep=true`, { headers: H }).then((r) => r.json());
```

```bash cURL theme={null}
curl "https://api.mixpeek.com/v1/namespaces/my-namespace/snapshots/$SNAPSHOT_ID/verify?deep=true" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

Verify reads the snapshot and changes nothing. It lists what a restore would drop: a missing collection dump, unreachable vectors, or objects that are gone. `deep=true` also opens every collection dump and checks it is complete. After a restore from the snapshot, verify also returns that restore's report.

## Restore beside the source

Name a new namespace in `target_namespace_name`. The source keeps running and is not touched.

```python Python theme={null}
task = requests.post(
    f"{BASE}/snapshots/{snapshot_id}/restore",
    headers=H,
    json={"target_namespace_name": "my-namespace-restored"},
).json()
```

```javascript JavaScript theme={null}
const task = await fetch(`${BASE}/snapshots/${snapshotId}/restore`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({ target_namespace_name: "my-namespace-restored" }),
}).then((r) => r.json());
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/my-namespace/snapshots/$SNAPSHOT_ID/restore" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target_namespace_name": "my-namespace-restored"}'
```

The response returns a `task_id`. Poll `GET /v1/tasks/{task_id}` until the task completes.

The restored namespace gets new ids, so it never collides with the source. History and telemetry stay with the source: tasks, retriever executions and interactions are not copied.

## Restore to a point in time

`restore_point` restores the namespace as it was at an instant or at a WAL sequence number. It needs `target_namespace_name`.

```python Python theme={null}
requests.post(
    f"{BASE}/snapshots/{snapshot_id}/restore",
    headers=H,
    json={"target_namespace_name": "my-namespace-at-1800", "restore_point": "2026-10-01T18:00:00Z"},
)
```

```javascript JavaScript theme={null}
await fetch(`${BASE}/snapshots/${snapshotId}/restore`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({
    target_namespace_name: "my-namespace-at-1800",
    restore_point: "2026-10-01T18:00:00Z",
  }),
});
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/my-namespace/snapshots/$SNAPSHOT_ID/restore" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target_namespace_name": "my-namespace-at-1800", "restore_point": "2026-10-01T18:00:00Z"}'
```

An instant is an RFC 3339 timestamp. The snapshot supplies the configuration: collections, retrievers and buckets. So pick a snapshot taken at or before the point.

The vectors and payloads come from the write-ahead log, replayed to the point. A restore can reach any point from the vector store's most recent checkpoint up to its newest durable write. A point outside that range is refused, and the error names the oldest and newest instants you can restore to now.

## Restore onto an isolated shard

`"isolated": true` puts the restored copy on a temporary vector shard of its own. It needs `target_namespace_name`. Use it to check or measure a restore without load on the shared shard.

* The isolated shard stops after six hours. Deleting the copy's namespace stops it sooner.
* One isolated copy runs at a time. A second request is refused, and the error names the namespace that holds it.

## Restore in place

Leave out `target_namespace_name` and set `confirm_in_place`. The snapshot's data replaces the namespace's current data.

```python Python theme={null}
requests.post(f"{BASE}/snapshots/{snapshot_id}/restore", headers=H, json={"confirm_in_place": True})
```

```javascript JavaScript theme={null}
await fetch(`${BASE}/snapshots/${snapshotId}/restore`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({ confirm_in_place: true }),
});
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/my-namespace/snapshots/$SNAPSHOT_ID/restore" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"confirm_in_place": true}'
```

The current data is deleted. The namespace is read-only from when the restore is accepted until it finishes. A write in that window gets a 409 that asks you to retry later.

A request without `confirm_in_place` is refused. To switch over with no read-only window, restore beside the source and point your traffic at the new namespace.

## Related

* [Namespaces](/docs/vector-store/namespaces)
* [Restore From Snapshot (API reference)](/docs/api-reference/namespace-snapshots/restore-from-snapshot)
* [Verify Snapshot Restore Readiness (API reference)](/docs/api-reference/namespace-snapshots/verify-snapshot-restore-readiness)


# Collection Lifecycle
Source: https://docs.mixpeek.com/docs/vector-store/collection-lifecycle

Move collections between hot, cold, and archived tiers — pay for fast vector search only where you need it, keep everything else restorable

Not every collection needs to serve low-latency vector search all the time. Seasonal campaigns, completed projects, and long-tail archives can move to cheaper tiers — without losing the data, and without re-processing anything to bring them back.

## Lifecycle states

| State | Vector search | Storage | Restorable |
| - | - | - | - |
| `active` *(default)* | Fast, fully indexed | Hot vector index | — |
| `cold` | Evicted from the hot index; vectors preserved in object storage | Object-storage backup | Yes — rehydrate to `active` at any time |
| `archived` | Removed | Vectors permanently deleted; documents' metadata remains | No |

Allowed transitions: `active → cold`, `active → archived`, `cold → active`, `cold → archived`. Archival is permanent — the vectors are deleted, not parked.

> [!NOTE]
>
> Going `cold` does not delete documents or metadata — it evicts the collection's vectors from the hot index after backing them up. Rehydrating restores search without re-running any extraction or embedding.

## Check lifecycle status

```bash theme={null}
curl "https://api.mixpeek.com/v1/collections/{collection_id}/lifecycle" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: my-namespace"
```

Returns the current state plus counts from both the hot index and the object-storage backup, so you can verify a transition completed.

## Transition a collection

```bash theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/collections/{collection_id}/lifecycle" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: my-namespace" \
  -H "Content-Type: application/json" \
  -d '{"lifecycle_state": "cold"}'
```

* **`cold`** evicts the collection from the vector store (vectors preserved via object storage).
* **`active`** (from `cold`) rehydrates — the backup is restored into the hot index.
* **`archived`** permanently removes the vectors.

> [!WARNING]
>
> `archived` is irreversible for vectors. If there is any chance you'll search this collection again, use `cold` — restoring from cold is cheap; rebuilding from archive means re-processing the source objects.

## When to use each tier

* **`active`** — anything queried by production retrievers.
* **`cold`** — completed campaigns and historical projects you want *restorable on demand* (for example, an annual review): rehydrate, query, and send back to cold.
* **`archived`** — data you're retaining for record-keeping where the documents' metadata is enough and vector search will never be needed again.

## Related

* [Buckets & storage classes](/docs/platform/data-model) — tiering for the *source objects* (this page covers the *index* tier; the two are independent)
* [Manage data](/docs/vector-store/manage-data) — document-level operations
* [Deduplication & re-processing](/docs/processing/deduplication) — what happens if you do need to rebuild from source


# Documents & Search
Source: https://docs.mixpeek.com/docs/vector-store/documents

Upsert vectors, search with dense/sparse/BM25/hybrid, and manage documents

## Upsert Documents

Insert or update documents with your pre-computed vectors and JSON payload. Up to 1,000 documents per request.

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="mxp_sk_...")

result = client.namespaces.documents.upsert(
    namespace_id="product-search",
    documents=[
        {
            "document_id": "prod-001",
            "vectors": {"text_embedding": [0.12, -0.34, 0.56]},  # 1536 floats
            "payload": {"title": "Noise-Canceling Headphones", "price": 149.99, "in_stock": True},
        },
        {
            "document_id": "prod-002",
            "vectors": {"text_embedding": [0.08, 0.22, -0.41]},  # 1536 floats
            "payload": {"title": "Bluetooth Speaker", "price": 59.99, "in_stock": True},
        },
    ],
)
# → {"inserted": 2, "document_ids": ["prod-001", "prod-002"]}
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/product-search/documents/upsert" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "documents": [{
      "document_id": "prod-001",
      "vectors": {"text_embedding": [0.12, -0.34, 0.56]},
      "payload": {"title": "Noise-Canceling Headphones", "price": 149.99}
    }]
  }'
```

Vector dimensions are validated against namespace config. If a document with the same ID exists, it is overwritten.

Mixpeek writes its own `_internal` envelope alongside your payload. Some of those
fields are system-owned and are rejected if you send them on a write. See
[The `_internal` envelope](/docs/retrieval/filters#the-_internal-envelope) for the
complete list of what Mixpeek writes and which fields you may set.

### Reserved payload names

These payload keys are refused with a 422: `document_id`, `collection_id`,
`_internal`, `_mvs_namespace` and `_source_tier`. Set the document's id with the
top-level `document_id` on each document, not in `payload`. Mixpeek sets
`collection_id` itself; if your source data carries its own, store it under a
different key such as `source_collection_id` or `metadata.collection_id`. The 422
names the offending documents and keys, and says what to use instead.

### Large Imports

Repeat the upsert call in batches of up to 1,000 documents. There is no
separate bulk-import endpoint.

### Carry lineage on the way in

A document you upsert defaults to `source_type: "direct_upsert"` and no lineage.
Send the lineage fields instead and the document arrives already knowing where it
came from, which is what makes this route usable for importing derived documents
rather than only standalone ones.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/$NS/documents/upsert" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "documents": [{
      "document_id": "chunk-001",
      "vectors": {"text_embedding": [0.12, -0.34, 0.56]},
      "payload": {"text": "..."},
      "source_type": "collection",
      "source_document_id": "doc-parent-001",
      "source_collection_id": "col_transcripts",
      "root_object_id": "obj_meeting_2026_08_22",
      "root_bucket_id": "bkt_recordings",
      "lineage_path": "bkt_recordings/col_transcripts/col_chunks",
      "content_hash": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
    }]
  }'
```

| Field | Meaning |
| - | - |
| `source_type` | Immediate-parent type: `bucket`, `collection`, `direct_upsert` or `signal`. Defaults to `direct_upsert`. |
| `root_object_id` | Decomposition-tree root. Every document derived from the same source object shares it. |
| `root_bucket_id` | Bucket holding the root object. |
| `source_object_id` | Immediate parent object id, when `source_type` is `bucket`. |
| `source_document_id` | Immediate parent document id, when `source_type` is `collection`. |
| `source_collection_id` | Immediate parent collection id, when `source_type` is `collection`. |
| `lineage_path` | Materialized path, for example `bkt_123/col_456/col_789`. |
| `lineage_chain` | Ordered steps from root to this document. Each step carries `collection_id`, `feature_extractor_id`, `document_id` and `timestamp`. |
| `content_hash` | SHA256 of the source object, stored at `_internal.content_hash`. |

Every field is optional. Send none and you get the previous behaviour, so
existing callers are unaffected.

> [!TIP]
>
> `content_hash` is worth sending even when you do not care about lineage. A
> later heal or reprocess uses it to recognise a cached derivation and skip the
> recompute, so supplying it is the difference between re-paying for extraction
> and not.

> [!NOTE]
>
> This is the only route that imports N documents with your own vectors, full
> lineage and a content hash in one call, at up to 1,000 documents per request.
> Which parent id applies depends on `source_type`: `bucket` uses
> `source_object_id`, `collection` uses `source_document_id` plus
> `source_collection_id`.

### Write vectors into a managed collection

`POST /v1/collections/{collection_id}/documents` takes `vectors` for the document you
create. Each key names one of the collection's vector indexes. Read the names from
`vector_indexes` on `GET /v1/collections/{collection_id}`.

A key is either the last segment of the index `feature_uri` or the full `index.name`.
For a `text_extractor` collection:

| Where you read it | Value |
| - | - |
| `feature_uri` | `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1` |
| Key from the last segment | `multilingual_e5_large_instruct_v1` |
| Key from `index.name` | `text_extractor_v1_embedding` |

Both keys work, and the vector is stored under `index.name`. The array length must match
the index `dimensions`. A key the collection does not have returns a 422 that lists the
available index names and dimensions.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections/$COLLECTION_ID/documents" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {"title": "Noise-Canceling Headphones"},
    "vectors": {"multilingual_e5_large_instruct_v1": [0.12, -0.34, 0.56, "...1024 floats"]}
  }'
```

## Search

> [!NOTE]
>
> **Querying is unified through retrievers** — Mixpeek has one query path, so there is
> no direct `POST /v1/namespaces/{ns}/documents/search` REST endpoint. The `client.search(...)`
> helper below is SDK sugar that authors and runs a retriever for you. From raw HTTP/curl,
> create a one-stage `feature_search` retriever and execute it — the verified BYO example
> immediately below does exactly that in three calls.

### Query BYO vectors from raw HTTP (verified)

Bring your own vectors and query them with a raw query vector — no extractor, no re-processing.
The query is an **object** (`{"input_mode": "vector", "value": "{{INPUT.qv}}"}`), and the
retriever's `input_schema` declares the input variable your execute call fills in.

```bash cURL theme={null}
# 1. Create a feature_search retriever over your BYO vector index ("my_vec").
#    Note: stage_type is "filter"; the operation is stage_id "feature_search".
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $NS" -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "byo-knn",
    "input_schema": {"qv": {"type": "array", "required": true}},
    "stages": [{
      "stage_name": "knn", "stage_type": "filter",
      "config": {"stage_id": "feature_search", "parameters": {
        "searches": [{
          "feature_uri": "my_vec",
          "query": {"input_mode": "vector", "value": "{{INPUT.qv}}"}
        }],
        "final_top_k": 10
      }}
    }]
  }'
# → {"retriever_id": "ret_...", ...}

# 2. Execute it with your query vector. pagination.method is required.
curl -X POST "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $NS" -H "Content-Type: application/json" \
  -d '{"inputs": {"qv": [1,0,0,0,0,0,0,0]}, "pagination": {"method": "offset", "page_size": 10, "page_number": 1}}'
# → {"documents": [{"document_id": "d1", "score": 1.0, ...}, ...]}
```

The execute cache is invalidated by upserts — a query re-run after an upsert returns fresh
results (`cache_hit: false`), so the write-then-verify loop is safe.

### Dense (Vector) — SDK

```python theme={null}
results = client.search(
    namespace_id="product-search",
    queries=[{
        "vector_name": "text_embedding",
        "vector": query_embedding,
        "top_k": 10,
        "score_threshold": 0.7,
    }],
)
```

### BM25 (Keyword)

Requires a [text index](/docs/vector-store/namespaces#text-indexes-bm25) on the target field.

```python theme={null}
results = client.search(
    namespace_id="product-search",
    queries=[{"text": "wireless noise canceling", "top_k": 10}],
)
```

### Sparse

```python theme={null}
results = client.search(
    namespace_id="product-search",
    queries=[{"sparse_vector": {42: 0.8, 1337: 0.5, 9001: 0.3}, "top_k": 10}],
)
```

### Hybrid

Combine multiple query types with RRF or DBSF fusion.

```python theme={null}
results = client.search(
    namespace_id="product-search",
    queries=[
        {"vector_name": "text_embedding", "vector": query_embedding, "top_k": 20},
        {"text": "wireless headphones", "top_k": 20},
    ],
    fusion="rrf",
)
```

### Filtered

Add payload filters to any query type. Filters narrow results before scoring.

```python theme={null}
results = client.search(
    namespace_id="product-search",
    queries=[{
        "vector_name": "text_embedding",
        "vector": query_embedding,
        "top_k": 10,
        "filters": {
            "must": [
                {"key": "price", "range": {"lte": 100}},
                {"key": "in_stock", "match": {"value": True}}
            ]
        },
    }],
)
```

## Document Operations

| Operation | Method | Endpoint |
| - | - | - |
| Get by ID | `GET` | `/v1/namespaces/{ns}/documents/{doc_id}` |
| Read many by ID | `POST` | `/v1/documents/batch-get` |
| List and paginate | `POST` | `/v1/documents/list` |
| Update payload | `PATCH` | `/v1/documents/{doc_id}` |
| Delete | `DELETE` | `/v1/collections/{collection_id}/documents/{doc_id}` |

Only upsert and get-by-ID carry the namespace in the path. The rest read it from
the `X-Namespace` header. Delete is collection-scoped, so a document you reach
by namespace is deleted through its collection.

### Scope a list to one collection

`POST /v1/documents/list` is namespace-scoped. Narrow it with `collection_ids`, which
takes a list even for one collection, or with a `filters` condition:

```json theme={null}
{ "collection_ids": ["col_2e13178d4f"], "limit": 50 }
```

```json theme={null}
{ "filters": { "field": "collection_id", "operator": "eq", "value": "col_2e13178d4f" } }
```

> [!WARNING]
>
> **A singular `collection_id` in the body does nothing, and the call still returns
> `200`.** The request model declares `collection_ids`, and an unrecognized key is
> dropped rather than rejected. You get every document in the namespace, and
> `total_documents` counts that wider set, so the response looks like a valid answer
> to the question you thought you asked.
>
> Nothing in the response marks the difference. `unknown_collection_ids` and
> `warnings` report ids that do not resolve, not keys that were ignored.
>
> Check the plural before you trust a count. Retriever `inputs` behave the opposite
> way and reject an undeclared key with `400`.

### Read Many by ID

Resolve up to 1,000 documents in one call. The namespace comes from the
`X-Namespace` header here, not from the path.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/documents/batch-get" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $MIXPEEK_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"document_ids": ["prod-001", "prod-002", "prod-404"]}'
# → {"documents": [...], "not_found": ["prod-404"]}
```

| Field | Meaning |
| - | - |
| `document_ids` | 1 to 1,000 ids. Required. |
| `return_presigned_urls` | Defaults to `true` on this endpoint. Pass `false` to skip presigning and get raw storage URIs back. |
| `documents` | The documents that resolved. |
| `not_found` | Each id that did not resolve, listed individually. |

> [!NOTE]
>
> `documents` follows the order of `document_ids`. Ids that did not resolve are
> omitted from `documents` and listed in `not_found`, and a repeated id yields
> its document once, at the first position you asked for it.
>
> Both of those shift later array positions, so match by `document_id` rather
> than by index.
>
> An id that does not resolve does not fail the call. It lands in `not_found`
> and every other id still returns, so read `not_found` rather than comparing
> lengths.

### Media URLs

Stored media fields hold a raw storage URI: `gs://` on Mixpeek Cloud, `s3://` on
S3-backed deployments. Whether you get a fetchable `https://` URL back depends on
the endpoint you called, not on the field name.

| Surface | Presigns |
| - | - |
| Retriever execute | Always. `return_presigned_urls` is not read on this path. |
| `POST /v1/documents/batch-get` | By default. Pass `"return_presigned_urls": false` to skip it. |
| `POST /v1/documents/list` | Only with `"return_presigned_urls": true` in the body. |
| `GET /v1/documents/{doc_id}` | Only with the flag, and these URLs last 1 hour. |

> [!WARNING]
>
> `POST /v1/documents/list` reads that flag from the request body only. A
> `?return_presigned_urls=true` query string is ignored, and you get raw `gs://`
> values back, which most HTTP clients reject with `unknown url type: gs`.

Selection works on the value, not the name. Any field whose value starts with
`gs://` or `s3://` gets signed, whatever the field is called. These stay raw:

* anything under `metadata`
* values nested inside dicts or lists, on documents carrying a `collection_id`
* `original_url`, `source_blobs`, and `presigned_urls`
* any URI whose signing call fails

Signed URLs last 24 hours everywhere except the single-document `GET` above.

### Update Vectors

Re-upsert the document through `POST /v1/namespaces/{ns}/documents/upsert`.
Sending an existing `document_id` overwrites that document, vectors and payload
together, so send the payload you want to keep alongside the new vectors.

No endpoint replaces vectors while leaving the payload untouched.

## Related

* [Namespace Configuration](/docs/vector-store/namespaces)
* [Promote to Managed](/docs/vector-store/promote)


# Import an External Export
Source: https://docs.mixpeek.com/docs/vector-store/import

Reconstruct documents with vectors and lineage from another system's export, in three calls that validate before anything is written

Import rebuilds documents from one external export and writes them into a
collection through the same BYO write core as
[upsert](/docs/vector-store/documents#upsert-documents), so imported documents arrive
with their vectors and their [lineage](/docs/vector-store/documents#carry-lineage-on-the-way-in)
intact.

> [!NOTE]
>
> **The target can be a collection that has a feature extractor.** Import goes
> through the BYO write core and submits no extraction batch, so the vectors in
> your export are the vectors that get stored. Nothing re-embeds them on the way
> in, and bucket and object lineage on the imported rows is preserved.
>
> This says nothing about later runs. A trigger or a batch you start afterwards
> treats these documents like any others.

Three calls. The first two write nothing.

**1. Infer a mapping from a sample**

`POST /v1/collections/import/infer` reads a sample of your export and
returns the source fields it found with a suggested source-to-target
mapping.

```bash theme={null}
curl -X POST "$MP_API_URL/v1/collections/import/infer" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{ "sample_rows": [ { "id": "a1", "title": "...", "embedding": [0.1, 0.2] } ],
        "sample_size": 100 }'
```

Returns `fields` and `warnings`. It runs on the sample alone, so nothing is
written and nothing is reserved.

**2. Validate the finished mapping**

`POST /v1/collections/import/validate-mapping` checks the mapping you
settled on. `mapping` is required.

```bash theme={null}
curl -X POST "$MP_API_URL/v1/collections/import/validate-mapping" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{ "mapping": { "fields": [...], "source_type": "collection" } }'
```

Returns `ok` plus `errors`. It flags four things: a missing root-identity
mapping, source and target types that disagree, a vector dimension that does
not match a known index, and source fields it does not recognise.

**3. Import**

`POST /v1/collections/{collection_identifier}/import` reconstructs the
documents and writes them. `mapping` is required.

```bash theme={null}
curl -X POST "$MP_API_URL/v1/collections/$COLLECTION_ID/import" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $NS" \
  -H "Content-Type: application/json" \
  -d '{ "export": { ... }, "mapping": { ... }, "dry_run": true }'
```

Returns `reconstructed`, `imported`, `skipped`, `errors`, `warnings` and
`dry_run`.

## Reading the result

| Field | Meaning |
| - | - |
| `reconstructed` | Documents rebuilt from the export. |
| `imported` | Documents written. Always `0` when `dry_run` is true. |
| `skipped` | Rows that failed reconstruction. |
| `errors` | Per-row failures. |
| `warnings` | Non-fatal notes. |

> [!NOTE]
>
> **A bad row does not stop the batch.** `skipped` counts rows that failed
> reconstruction while `imported` counts the ones that were written, so a
> partially-bad export gives you both, not an all-or-nothing outcome.
>
> Compare `reconstructed` against `imported` and `skipped` rather than checking
> for an error status. A response with `errors` in it can still have written
> most of the batch.

> [!TIP]
>
> Run the third call with `"dry_run": true` first. It reconstructs and validates
> the full export, not a sample, and writes nothing — `imported` comes back `0`
> while `reconstructed` and `skipped` tell you what the real run would do.
>
> Step 2 validates the mapping. A dry run validates the mapping against every
> row of the actual export, which is a different and stronger check.

## Options

| Field | Default | Meaning |
| - | - | - |
| `export` | — | The export payload to import. |
| `mapping` | *required* | `fields`, plus optional `source_type` (`bucket` or `collection`) and `chain_template`. |
| `explode` | `null` | Split one export row into several documents. `explode_axis` names the doc-generating array, for example `scene_content`. |
| `dry_run` | `false` | Reconstruct and validate without writing. |
| `seed_ledger` | `false` | Seed the ledger alongside the import. |

## Related

* [Documents & Search](/docs/vector-store/documents) for the single-call BYO upsert
* [Collection Lifecycle](/docs/vector-store/collection-lifecycle)


# Update & Delete Data
Source: https://docs.mixpeek.com/docs/vector-store/manage-data

Update document metadata and delete documents, collections, and objects — including what cascades

How to change and remove data after ingestion, and exactly what each delete cascades to.

## Update document metadata

`PATCH` a document to change its fields. The body is a partial update — send **only the fields you want to change**; you never resend vectors. Any fields you pass are merged into the document.

```bash theme={null}
curl -sS -X PATCH \
  "$MP_API_URL/v1/collections/$COLLECTION_ID/documents/$DOCUMENT_ID" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "status": "reviewed", "priority": 3 }'
```

### Set and remove metadata keys

`PUT` and `PATCH` treat `metadata` the same way. Each key you send is set on the stored
metadata. Keys you omit stay as they are. A key set to `null` is removed.

```bash theme={null}
curl -sS -X PATCH \
  "$MP_API_URL/v1/collections/$COLLECTION_ID/documents/$DOCUMENT_ID" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "metadata": { "status": "reviewed", "stale_field": null } }'
```

This sets `metadata.status` and removes `metadata.stale_field`. Every other metadata key
is untouched. A `PUT` behaves the same way, so a key missing from a `PUT` body stays.

### Where each key lands

A key inside `metadata` updates `metadata.<key>`. A key at the top level of the body
updates the document's own top-level field of that name. They are separate fields: a
field your collection copies onto the document (a passthrough field such as `brand_id`)
sits at the top level, so change it by sending it at the top level:

```bash theme={null}
curl -sS -X PATCH \
  "$MP_API_URL/v1/collections/$COLLECTION_ID/documents/$DOCUMENT_ID" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "brand_id": "brand-b" }'
```

`{"metadata": {"brand_id": "brand-b"}}` would instead add `metadata.brand_id` and leave
the top-level `brand_id` as it was. The response is the whole document, so you can see
which field changed.
`"metadata": null` returns a 422. To remove keys, name each one with a `null` value.

To update many documents at once, use `PATCH /v1/collections/{collection_id}/documents/bulk`.
Send `update_data` with the field-value pairs to apply, and `filters` to choose which
documents receive them. **Omitting `filters` updates every document in the collection.**

> [!WARNING]
>
> **Bulk update is asynchronous, and the response tells you nothing about the result.**
> The call returns `task_id` and `status: "PENDING"`, and `updated_count` is `null` on that
> enqueue response. The terminal status and the real `updated_count` land on the task
> record, so poll `GET /v1/tasks/{task_id}` before treating the update as applied.
>
> Reading `updated_count` off the immediate response gives you `null`, not zero.

> [!NOTE]
>
> **Object fields come back with their keys in alphabetical order.** A field written as `{"SKUs": ..., "source": ..., "Keywords": ...}` reads back from `documents/list` and retriever results as `{"Keywords": ..., "SKUs": ..., "source": ...}`. The values are unchanged. If you show an object's fields in a set order, keep that order in your application.

> [!NOTE]
>
> Updating metadata does **not** re-run extraction or change vectors — it only edits the stored payload. To change the embedding model, see [Migrate Embedding Models](/docs/processing/model-migration).

To change **access control** (not metadata), use the dedicated ACL endpoint `PATCH /v1/collections/{collection_id}/documents/{document_id}/acl` — see [Permissions](/docs/platform/permissions).

> [!NOTE]
>
> **Object metadata vs document metadata are separate.** The `PATCH` above edits a **document** (a processed, searchable record in a collection). To edit the source **object** in the bucket, use [`PATCH /v1/buckets/{bucket_id}/objects/{object_id}`](/docs/api-reference/bucket-objects/partially-update-object) and send the object's custom fields flat at the root of the body, for example `{"campaign_id": "camp_002"}`. Only the fields you send change. Editing one does not change the other, and re-processing an object regenerates its documents from the object's current state.

## Delete data

> [!WARNING]
>
> Deletes are permanent and some **cascade**. Read the cascade column before deleting — removing a bucket object or a collection also destroys derived documents.

| Delete | Endpoint | Cascades to |
| - | - | - |
| A document | `DELETE /v1/collections/{collection_id}/documents/{document_id}` | Just that document |
| Many documents | `DELETE /v1/collections/{collection_id}/documents/batch` | The documents you list |
| A bucket object | `DELETE /v1/buckets/{bucket_id}/objects/{object_id}` | **Hard-deletes every collection document derived from that object** |
| A collection | `DELETE /v1/collections/{collection_id}` | **All documents in the collection** |
| A namespace | `DELETE /v1/namespaces/{namespace_id}` | **Everything in the namespace** (buckets, collections, documents, retrievers) |

```bash theme={null}
# Delete a single document
curl -sS -X DELETE \
  "$MP_API_URL/v1/collections/$COLLECTION_ID/documents/$DOCUMENT_ID" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"

# Delete a bucket object (also removes its derived documents)
curl -sS -X DELETE \
  "$MP_API_URL/v1/buckets/$BUCKET_ID/objects/$OBJECT_ID" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

> [!TIP]
>
> Deleting the **object** is the right move when you want the source asset *and* its derived documents gone. Deleting just the **document** leaves the source object in the bucket, so a re-process would recreate the document.

### Deleting many documents

`DELETE /v1/collections/{collection_id}/documents/batch` takes `document_ids` or
`filters`, and the two are mutually exclusive. Which one you send decides whether the
call is synchronous, so the response shape differs between them.

| Mode | Send | Response |
| - | - | - |
| Explicit IDs | `document_ids` | Synchronous. `deleted_count`, `failed_count` and per-document `results` are populated; `task_id` and `status` are `null` |
| Filter | `filters` | Asynchronous. `task_id` and `status: "PENDING"` are populated; `deleted_count` is `null` and `results` is an empty list |

> [!WARNING]
>
> **In filter mode, `deleted_count` is `null` rather than a count**, and `results` is empty
> because only counts are returned. Poll `GET /v1/tasks/{task_id}` for the terminal status
> and the final `deleted_count`. Code that reads `deleted_count` straight off the response
> sees `null` on a delete that is still running.

`reason` is optional on both modes and is recorded on the `DOCUMENT_BULK_SOFT_DELETED`
audit entry.

### Synced sources

If documents came from a [storage sync](/docs/platform/syncs), the sync's `reconcile.on_delete` is `true` by default, so deleting a file in the source (S3, Google Drive, etc.) automatically removes the corresponding object and its derived documents. Set `reconcile.on_delete` to `false` to keep Mixpeek objects when the source asset is deleted.

## Related

* [Documents](/docs/vector-store/documents) — document structure and payload
* [Syncs](/docs/platform/syncs) — source-deletion cascade (`on_delete`)
* [Ingest Data](/docs/platform/data-model) — objects, batches, collections


# Namespaces
Source: https://docs.mixpeek.com/docs/vector-store/namespaces

Create standalone namespaces with vector indexes, distance metrics, BM25, and payload indexes

A namespace is the isolation boundary for your vectors, documents, and indexes.

## Create a Namespace

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="mxp_sk_...")

# Minimal — MVS infers vector configs on first write
client.namespaces.create(
    namespace_id="product-search",
)

# Or pre-declare configs to set a specific distance metric
client.namespaces.create(
    namespace_id="product-search",
    vector_configs=[
        {"name": "text_embedding", "dimension": 1536, "metric": "cosine"},
        {"name": "image_embedding", "dimension": 512, "metric": "dot"},
    ],
)
```

```bash cURL theme={null}
# Minimal — MVS infers vector configs on first write
curl -X POST "https://api.mixpeek.com/v1/namespaces/standalone" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"namespace_id": "product-search"}'

# Or pre-declare configs to set a specific distance metric
curl -X POST "https://api.mixpeek.com/v1/namespaces/standalone" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_id": "product-search",
    "vector_configs": [
      {"name": "text_embedding", "dimension": 1536, "metric": "cosine"},
      {"name": "image_embedding", "dimension": 512, "metric": "dot"}
    ]
  }'
```

### Parameters

**namespace\_id** `string` *required*

Unique identifier. Lowercase with hyphens (e.g., `product-search`).

**mode** `string` *required*

Must be `"standalone"`.

**vector\_configs** `array`

Optional. Pre-declare named vector indexes with specific dimensions and distance metrics. If omitted, MVS uses [schema-on-write](#schema-on-write) — vector indexes are created automatically on first upsert with inferred dimensions and cosine metric.

| Field | Type | Description |
| - | - | - |
| `name` | string | Index name (e.g., `text_embedding`) |
| `dimension` | integer | Must match your model's output dimension |
| `metric` | string | `cosine` (default), `dot`, or `euclidean` |

Documents can include vectors for any subset of indexes — not every index needs a vector in every document. When you upsert a document with a new vector name that hasn't been seen before, MVS creates a new vector index automatically.

## Text Indexes (BM25)

Full-text keyword search runs on **payload indexes of type `text`**. You declare indexes with `payload_indexes`, then register them in the store with `ensure-indexes` — which also backfills existing documents with **no re-ingestion or re-embedding**.

**Step 1 — Declare a `text` index** (on an existing namespace, patch `payload_indexes`):

```python Python theme={null}
requests.patch(f"{BASE}/namespaces/product-search", headers=headers, json={
    "payload_indexes": [
        {"field_name": "title", "type": "text"},
        {"field_name": "description", "type": "text"}
    ]
})
```

```bash cURL theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/namespaces/product-search" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"payload_indexes": [
        {"field_name": "title", "type": "text"},
        {"field_name": "description", "type": "text"}
      ]}'
```

**Step 2 — Register and backfill** (idempotent, zero re-extraction):

```python Python theme={null}
requests.post(f"{BASE}/namespaces/product-search/ensure-indexes", headers=headers)
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/product-search/ensure-indexes" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

Pass `?force_rebuild=true` to drop and rebuild an index from the full store — use this to repair a `text`/BM25 index that registered empty (e.g. during a cold start).

> [!NOTE]
>
> BM25 matches across **all** `text`-indexed string fields, not a single field. Once a text index exists, run keyword search in a retriever with the [`lexical` option on Feature Search](/docs/retrieval/stages/feature-search#lexical-bm25-search) — and fuse it with a dense vector search under `rrf` for hybrid retrieval.

## Payload Indexes

Speed up filtered searches by indexing frequently queried fields. Payload indexes use the same `payload_indexes` declaration + `ensure-indexes` flow as text indexes above — just pick the matching `type`:

```python Python theme={null}
requests.patch(f"{BASE}/namespaces/product-search", headers=headers, json={
    "payload_indexes": [{"field_name": "category", "type": "keyword"}]
})
requests.post(f"{BASE}/namespaces/product-search/ensure-indexes", headers=headers)
```

```bash cURL theme={null}
curl -X PATCH "https://api.mixpeek.com/v1/namespaces/product-search" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"payload_indexes": [{"field_name": "category", "type": "keyword"}]}'

curl -X POST "https://api.mixpeek.com/v1/namespaces/product-search/ensure-indexes" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

Types: `keyword`, `integer`, `float`, `bool`, `datetime`, `geo`, `text`, `uuid`. Use dot notation for nested fields (e.g. `metadata.title`). Arrays are indexed **element-wise** — index a multi-value field with its element type (`objects: ["car", "person"]` → `keyword`) and a filter matches if any element matches.

> [!NOTE]
>
> **`PATCH` merges, `PUT` replaces.** A `PATCH` is additive: send only the fields
> you're adding and every existing index is kept. Use `PUT` when you want the body
> to be the complete list — any user index you omit there is **dropped**.
>
> You can also `GET` the namespace, append to `payload_indexes`, and `PATCH` the
> whole thing back. The protected system entries in that response are ignored on
> input, so you don't need to strip them first.

### Adaptive Payload Indexes

MVS monitors your query patterns and **automatically creates payload indexes** when a field appears in filters frequently enough. By default, any field used in >10 queries/hour gets auto-indexed — no manual action needed.

| Setting | Default | Description |
| - | - | - |
| `auto_create_payload_indexes` | `true` | Auto-create payload indexes when filter threshold is hit |
| `filter_threshold` | 10 queries/hour | How many filter queries before auto-indexing |
| `auto_create_text_indexes` | `false` | BM25 indexes are heavier — suggest-only by default |
| `check_interval_s` | 600 (10 min) | How often the advisor checks query patterns |

Control this with the `MVS_AUTO_INDEX` environment variable (`true`/`false`).

### Index Suggestions

For fields that don't yet meet the auto-create threshold, you can check what MVS recommends:

```bash theme={null}
curl "https://api.mixpeek.com/v1/namespaces/product-search/index-suggestions" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

Returns recommended indexes based on filter frequency, field cardinality, and existing coverage.

### Schema-on-Write

Unlike traditional vector databases that require declaring all vector indexes at creation time, MVS is **schema-on-write**. When you upsert a document with a new vector name, MVS automatically:

1. Infers the dimension from the first write
2. Creates a dense index with the detected dimension and cosine metric
3. Enforces schema on subsequent writes (mismatched dimensions raise an error)

This means you can add new embedding models to an existing namespace without reconfiguring anything — just start writing vectors with the new name.

## Usage Metrics

Get detailed performance and usage metrics for a namespace via `GET /v1/namespaces/{id}/usage`.

```bash theme={null}
curl "https://api.mixpeek.com/v1/namespaces/product-search/usage" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

```json title="Response" theme={null}
{
  "namespace_id": "product-search",
  "document_count": 125000,
  "vector_count": 250000,
  "storage_bytes": 1073741824,
  "index_count": 3,
  "queries_last_24h": 48200,
  "vectors": {"total": 250000, "by_name": {"text_embedding": 125000, "image_embedding": 125000}},
  "documents": {"total": 125000, "with_all_vectors": 120000, "partial": 5000},
  "storage": {
    "total_bytes": 1073741824,
    "by_tier": {"hot": 1073741824, "cold": 0, "archive": 0},
    "vector_bytes": 768000000,
    "payload_bytes": 32000000,
    "index_bytes": 16384000
  },
  "queries": {"last_24h": 48200, "last_7d": 312000, "last_30d": 1250000,
    "by_type": {"dense_search": 30000, "hybrid_search": 15000, "bm25_search": 3200}},
  "writes": {"last_24h": 5200, "last_7d": 35000, "last_30d": 142000},
  "shards": {"total": 4, "hot": 4, "cold": 0}
}
```

Key metrics:

| Metric | Description |
| - | - |
| `vectors.by_name` | Vector count per named index — useful for checking partial coverage |
| `queries.by_type` | Breakdown by search type — informs [cost optimization](/docs/best-practices/cost-optimization) |
| `shards.hot` / `shards.cold` | Shard distribution across hot (in-memory) and cold (object-storage) partitions |

These metrics feed into [usage-based billing](/docs/platform/billing). You can also monitor namespace health via the [observability dashboard](/docs/operations/observability).

## Other Operations

| Operation | Method | Endpoint |
| - | - | - |
| Get info | `GET` | `/v1/namespaces/{id}` |
| Delete | `DELETE` | `/v1/namespaces/{id}` |
| Clone | `POST` | `/v1/namespaces/{id}/clone` |
| Snapshot and restore | `POST` | `/v1/namespaces/{id}/snapshots`, then [Backup and Restore](/docs/vector-store/backup-and-restore) |
| Vector metadata | `PUT` | `/v1/namespaces/{id}/vector-metadata` |

## Related

* [Documents & Search](/docs/vector-store/documents)
* [Backup and Restore](/docs/vector-store/backup-and-restore)
* [Promote to Managed](/docs/vector-store/promote)
* [Billing & Usage](/docs/platform/billing)


# Vector Store
Source: https://docs.mixpeek.com/docs/vector-store/overview

Use Mixpeek as a standalone vector database — bring your own embeddings, search instantly, promote to managed when ready

Bring your own embeddings, upsert them directly, and query with dense, sparse, BM25, or hybrid search. No collections or extractors required.

> **[Create a vector namespace](https://studio.mixpeek.com/namespaces/create?mode=standalone\&utm_source=docs\&utm_medium=vector_store\&utm_campaign=mvs_onramp\&utm_content=vector_store_overview)**
>
> Upsert your own embeddings and run a search in minutes, with no extractors or schema to declare. Creating a namespace needs a plan; see [Billing](/docs/platform/billing) for what each one covers.

## Quickstart

**1. Create a namespace**

No schema needed — MVS infers vector dimensions on first write.

> [!WARNING]
>
> **Pick a vector name of your own**, like `my_text_v1`. A few short generic names
> (`text_embedding` among them) collide with legacy pre-namespace keys in the store,
> and a namespace that uses one accepts your writes with a `200` and then refuses
> every search with a `409` (a cross-namespace index refusal). Prefixing with
> your project or model version avoids it.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/standalone" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"namespace_id": "product-search"}'
```

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="mxp_sk_...")

client.namespaces.create(
    namespace_id="product-search",
    mode="standalone",
)
```

You can optionally pre-declare vector configs if you want to set a specific distance metric:

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/standalone" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace_id": "product-search",
    "vector_configs": [
      {"name": "my_text_v1", "dimension": 1536, "metric": "dot"}
    ]
  }'
```

```python Python theme={null}
client.namespaces.create(
    namespace_id="product-search",
    mode="standalone",
    vector_configs=[
        {"name": "my_text_v1", "dimension": 1536, "metric": "dot"}
    ],
)
```

**2. Upsert documents with your vectors**

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/product-search/documents/upsert" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "documents": [{
      "document_id": "prod-001",
      "vectors": {"my_text_v1": [0.12, -0.34, 0.56, "...1536 floats"]},
      "payload": {"title": "Wireless Headphones", "category": "audio", "price": 79.99}
    }]
  }'
```

```python Python theme={null}
client.namespaces.documents.upsert(
    namespace_id="product-search",
    documents=[{
        "document_id": "prod-001",
        "vectors": {"my_text_v1": [0.12, -0.34, 0.56]},  # 1536 floats
        "payload": {"title": "Wireless Headphones", "category": "audio", "price": 79.99},
    }],
)
```

**3. Create a retriever (one-time)**

Querying is unified on **retrievers** — one query concept whether you bring your own vectors or promote to managed embedding. Create a retriever once; for a standalone namespace it takes the query vector you computed (`input_mode: vector`). All requests are scoped to the namespace via the `X-Namespace` header.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: product-search" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product_search",
    "input_schema": {"query_vector": {"type": "array", "required": true}},
    "stages": [{
      "stage_name": "search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "pre_filters": {"AND": [{"field": "category", "operator": "eq", "value": "audio"}]},
        "parameters": {
          "searches": [{
            "feature_uri": "my_text_v1",
            "query": {"input_mode": "vector", "value": "{{INPUT.query_vector}}"},
            "top_k": 10
          }],
          "final_top_k": 10
        }
      }
    }]
  }'
```

```python Python theme={null}
retriever = client.retrievers.create(
    namespace_id="product-search",
    retriever_name="product_search",
    input_schema={"query_vector": {"type": "array", "required": True}},
    stages=[{
        "stage_name": "search",
        "stage_type": "filter",
        "pre_filters": {"AND": [{"field": "category", "operator": "eq", "value": "audio"}]},
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [{
                    "feature_uri": "my_text_v1",
                    "query": {"input_mode": "vector", "value": "{{INPUT.query_vector}}"},
                    "top_k": 10,
                }],
                "final_top_k": 10,
            },
        },
    }],
)
```

**4. Execute the retriever**

Run the retriever with your query embedding. Each hit exposes `document_id` and payload fields.

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: product-search" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {"query_vector": [0.15, -0.28, 0.44, "...query embedding"]}
  }'
```

```python Python theme={null}
results = client.retrievers.execute(
    namespace_id="product-search",
    retriever_id=retriever.retriever_id,
    inputs={"query_vector": [0.15, -0.28, 0.44]},  # query embedding
)
```

## Architecture

![Standalone vector store architecture — your pipeline feeds vectors into Mixpeek's sharded storage with dense, BM25, and payload indexes](https://mintcdn.com/mixpeek/PRYIJzrWYd-bekCW/assets/vector-store/standalone-architecture.svg?fit=max\&auto=format\&n=PRYIJzrWYd-bekCW\&q=85\&s=c0d82555dad0a11f73a11debdb4ccc5d)

## Scaling

The index **auto-partitions as you grow** — there's no capacity planning, resharding, or replica provisioning on your side, and no per-vector or per-namespace caps. The same namespace serves thousands or hundreds of millions of vectors.

At large scale (10M–100M+ vectors), two things matter:

* **Tiering.** Hot (in-memory) vectors serve at \~10ms; cold (object-storage) vectors serve at \~100ms via brute-force scan. The vector store keeps your actively-queried set resident automatically — there's no manual tier management on your side.
* **Shard visibility.** The `shards.hot` / `shards.cold` counts in [namespace usage metrics](/docs/vector-store/namespaces) show how the index has partitioned. Dedicated/enterprise deployments can tune shard and index parameters — see [Single-Tenant](/docs/resources/single-tenant).

> [!NOTE]
>
> Need concrete latency/recall targets for a specific corpus size and query rate? [Talk to engineers](https://mixpeek.com/contact) — sizing depends on dimensions, filter selectivity, and your hot/cold split.

## Standalone vs Managed

Every namespace runs in one of two modes. Start standalone and [promote](/docs/vector-store/promote) when you're ready — no reindexing.

| | Standalone | Managed |
| - | - | - |
| **Query latency** | Lower — no embedding at query time | +50-200ms for auto-embedding |
| **Embedding cost** | You pay your provider directly | Included in platform pricing |
| **Model flexibility** | Any model, any fine-tune | Bound to registered inference services |
| **Write path** | Direct upsert only | Collections auto-process + direct upsert |
| **Search input** | Pre-computed vectors, text (BM25), sparse | Also accepts raw text/URLs (auto-embedded) |
| **Feature URIs** | Plain feature names only (e.g. `my_text_v1`) — `mixpeek://…` extractor-backed URIs are **not** available | Full `mixpeek://<extractor>@<version>/<model>` URIs |
| **Best for** | Existing ML infra, low-latency, custom models | End-to-end processing, file pipelines |

> [!TIP]
>
> Start standalone if you already have embeddings. Promotion is additive — all existing data is preserved.

## Features

All features work identically in standalone and managed modes.

| Capability | Description |
| - | - |
| **Dense search** | Vector similarity with cosine, dot product, or euclidean distance |
| **Sparse search** | Sparse vector queries for learned sparse representations (SPLADE, etc.) |
| **BM25 keyword search** | Full-text search on payload fields via [text indexes](/docs/vector-store/namespaces#text-indexes-bm25) |
| **Hybrid search** | Combine dense + BM25 + sparse in one query with [RRF or DBSF fusion](/docs/vector-store/documents#hybrid) |
| **Metadata filtering** | Filter on any payload field — combine with any search type |
| **Payload indexes** | Manual or [adaptive](/docs/vector-store/namespaces#adaptive-payload-indexes) — auto-created based on query patterns |
| **Schema-on-write** | [Auto-create vector indexes](/docs/vector-store/namespaces#schema-on-write) on first upsert — no upfront declaration needed |
| **Usage metrics** | [Per-namespace breakdowns](/docs/vector-store/namespaces#usage-metrics) of vectors, queries, and writes |
| **Namespace cloning** | Clone namespaces for testing or environment branching |

## Billing

MVS pricing is **pure usage-based** — no per-vector caps, no namespace limits. Tiers gate support level, not features.

| Resource | Price |
| - | - |
| **Storage** | \$0.023 / GB / month |
| **Hot cache** | \$25 / GB / month |
| **Queries** | \$2 / 1M queries |
| **Writes** | \$2 / 1M writes |

Tier minimums and what each one includes live on one page: [Billing](/docs/platform/billing).

All search features (dense, sparse, BM25, hybrid, adaptive indexes) are available on every tier. See the [pricing calculator](https://mixpeek.com/mvs#pricing) for cost estimates at scale.

Track usage programmatically with the [vector-backend usage endpoint](/docs/api-reference/organization-billing/get-vector-backend-usage) (`GET /v1/organizations/billing/usage/vector-backend`) or view it in the Studio dashboard under **Billing**.

## Next Steps

> **[Namespaces](/docs/vector-store/namespaces)**
>
> Vector indexes, metrics, BM25

> **[Documents & Search](/docs/vector-store/documents)**
>
> Upsert, query, manage

> **[Promote](/docs/vector-store/promote)**
>
> Standalone → managed

***

> [!NOTE]
>
> **Ready to go beyond BYO vectors?** Promote your standalone namespace to **managed mode** and add automatic embedding, file processing pipelines, and enrichment — without reindexing. Your retrievers keep working unchanged — after promotion, the same retriever can auto-embed raw text instead of taking a pre-computed vector. [See the migration guide](/docs/vector-store/promote#querying-with-retrievers) for details. [Learn how to promote →](/docs/vector-store/promote)


# Add Objects to Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/add-objects-to-batch

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/objects
Add objects to an existing batch. The batch must be in 'draft' status.



# Create Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/create-batch

post /v1/buckets/{bucket_identifier}/batches
Create a new batch for grouping bucket objects.



# Get Batch Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/get-batch-configuration

get /v1/buckets/{bucket_identifier}/batches/{batch_id}
Retrieve batch configuration and historical data from MongoDB. Status is automatically synchronized from Task API. For real-time monitoring, use GET /v1/tasks/{task_id} (Redis, faster).



# Partially Update Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch

patch /v1/buckets/{bucket_identifier}/batches/{batch_id}
This endpoint partially updates a batch (PATCH operation).
    Only provided fields will be updated. At minimum, metadata can always be updated.
    Immutable fields like batch_id and timestamps cannot be modified.



# Aggregate Objects
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/aggregate-objects

post /v1/buckets/{bucket_identifier}/objects/aggregate
This endpoint performs aggregation operations on objects in a bucket.

    **Aggregation Framework**: Provides MongoDB-style aggregation operations:
    - GROUP BY: Group objects by one or more fields
    - Aggregations: COUNT, SUM, AVG, MIN, MAX, COUNT_DISTINCT, etc.
    - Date Operations: Truncate or extract date parts for time-series analysis
    - Filtering: Pre-aggregation filters (WHERE) and post-aggregation filters (HAVING)
    - Sorting & Limiting: Control result ordering and size

    **Use Cases**:
    - Count objects by status or category
    - Calculate daily/monthly upload statistics
    - Analyze content distribution and trends
    - Generate reports with multiple metrics

    **Note**: This endpoint works with both MongoDB objects and vector store documents
    using the same interface. The system automatically selects the appropriate
    aggregation provider.



# Bulk Soft-Delete Objects
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/bulk-soft-delete-objects

post /v1/buckets/{bucket_identifier}/objects/bulk-soft-delete
Soft-delete many objects by ID in one call. Records are
    retained and flipped to a non-active lifecycle state (so retrieval excludes
    them), with the lifecycle propagated to downstream documents. There is no
    self-service restore endpoint: clearing the marker is a support action, so
    treat this as a one-way call. Bucket stats
    are recomputed once. Designed for large purges (e.g. scoping a synced bucket
    to a subset) without the per-object webhook/stats storms of single delete.



# Create Object
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/create-object

post /v1/buckets/{bucket_identifier}/objects
This endpoint creates a new object in the specified bucket.
    The object must conform to the bucket's schema.

    **Processing**: By default, objects are created in PENDING status, which
    means not processed yet, and are processed when a batch that includes them
    is submitted. An object stays PENDING until that batch run finishes with it;
    the batch status shows whether it has been submitted. Set the `auto_process=true` query parameter
    to automatically create a batch and submit it for processing (zero-touch
    workflow). Sending `auto_process` in the JSON body returns 422, unless the
    bucket schema declares a field with that name.

    If the bucket has a unique_key configured, the insertion policy determines behavior:
    - insert: Create only. Fail with 409 Conflict if unique key exists.
    - update: Update only. Fail with 404 Not Found if unique key doesn't exist.
    - upsert: Create if new, update if exists (idempotent).

    Policy resolution:
    1. Use ?policy= query parameter if provided
    2. Fall back to bucket's default_policy if configured
    3. Error 400 if neither is specified



# Create Objects in Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/create-objects-in-batch

post /v1/buckets/{bucket_identifier}/objects/batch
This endpoint creates multiple new objects in the specified bucket as a batch.
    Each object must conform to the bucket's schema.

    **Processing**: By default, objects are created in PENDING status, which
    means not processed yet, and are processed when a batch that includes them
    is submitted. An object stays PENDING until that batch run finishes with it;
    the batch status shows whether it has been submitted. Set the `auto_process=true` query parameter
    to automatically create a processing batch and submit it (zero-touch
    workflow). Sending `auto_process` in the JSON body, at the top level or in
    an object, returns 422, unless the bucket schema declares a field with that
    name inside objects.

    **Partial Success**: This endpoint uses partial success - valid objects are created
    even if some fail validation. Failed objects are returned separately with error details,
    allowing you to fix and retry only the failed ones.

    **Response**: Returns both succeeded and failed objects. The batch succeeds (200 OK) as long
    as at least one object is created. Check the `failed` array for objects that need attention.
    If EVERY object in the batch failed, the response is 422 instead of 200 -- the body shape is
    identical (`succeeded: []`, `failed: [...]`), only the status code signals total failure.

    **unique_key buckets**: not supported here yet. If the bucket declares a `unique_key`
    policy, this endpoint rejects the whole request (422) instead of silently creating
    duplicates -- use `POST /v1/buckets/{bucket_id}/objects` (singular) per object instead,
    which enforces it.



# Delete Object
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/delete-object

delete /v1/buckets/{bucket_identifier}/objects/{object_identifier}
This endpoint deletes an object from the specified bucket.



# Get Object
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/get-object

get /v1/buckets/{bucket_identifier}/objects/{object_identifier}
This endpoint retrieves an object by its ID from the specified bucket.

    **Presigned URLs**: Set `return_presigned_urls=true` query parameter to generate fresh presigned download URLs
    for all blobs with S3 storage (default: false). URLs are added to each blob's properties as
    `presigned_url` and expire after 1 hour.

    **Document count**: `document_count` (how many documents this object produced, via vector-store
    lineage) is computed by default. It fans out to the vector partition, which on a cold/serverless
    partition can be slow — pass `include_document_count=false` to skip it for latency-sensitive,
    interactive views (e.g. an object-detail modal) that don't render the count. Even when requested,
    the count is best-effort and bounded by a deadline: it returns `null` rather than blocking the
    response if the partition is cold.



# List Objects
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/list-objects

post /v1/buckets/{bucket_identifier}/objects/list
This endpoint lists objects in a bucket with cursor-based pagination, filtering, and sorting.
    Filtering, sorting and field selection go in the JSON request body. Pagination goes in the
    query string. A body key this endpoint does not read is rejected with a 422.

    **Filtering**: `filters` in the body takes a logical operator tree. Use dot notation for
    metadata fields (for example `metadata.type`).

    **Sorting**: `sort` in the body is one object with a field and a direction
    - Example: {"sort": {"field": "created_at", "direction": "desc"}}
    - Direction: asc (ascending) or desc (descending), defaults to asc
    - Without `sort`, objects come back in creation order, oldest first

    **Pagination**: Cursor-based for efficient deep pagination
    - First page: ?limit=100 (omit cursor)
    - Next pages: ?limit=100&cursor={next_cursor}
    - Use next_cursor from response to navigate
    - No limit on pagination depth

    **Total Count**: Optional (expensive operation)
    - Use ?include_total=true to get total count
    - Adds 50-200ms to response time
    - Returns total, page, page_size, total_pages fields in pagination response



# List Objects (GET)
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/list-objects-get

get /v1/buckets/{bucket_identifier}/objects
List objects in a bucket via GET with cursor-based pagination. Use POST /list for filtering and sorting.



# Partially Update Object
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/partially-update-object

patch /v1/buckets/{bucket_identifier}/objects/{object_identifier}
This endpoint partially updates an existing object in the specified bucket (PATCH operation).
    Send the object's custom fields flat at the root of the body, for example {"campaign_id": "camp_002"}.
    Only the fields you send change. `key_prefix` and `skip_duplicates` are also accepted.
    This route has no `metadata` field. System names such as `status`, `object_id` and `blobs` are reserved and return a 422.
    It does not trigger processing.



# Update Object
Source: https://docs.mixpeek.com/docs/api-reference/bucket-objects/update-object

put /v1/buckets/{bucket_identifier}/objects/{object_identifier}
This endpoint updates an existing object in the specified bucket.
    It accepts `key_prefix`, `skip_duplicates` and `metadata`; each `metadata` key is set
    at the root of the object, and fields you leave out keep their values. It cannot
    replace blobs: to change an object's files, delete it and create it again.
    It does not trigger processing.



# Get Adapter Status
Source: https://docs.mixpeek.com/docs/api-reference/bucket-source-adapter/get-source-adapter-status

get /v1/buckets/{bucket_id}/source-adapter/status
Get source adapter status and statistics.



# Refresh Source Adapter
Source: https://docs.mixpeek.com/docs/api-reference/bucket-source-adapter/refresh-source-adapter

post /v1/buckets/{bucket_id}/source-adapter/refresh
Trigger a pull-based source refresh (collection bucket).

This is the explicit trigger that turns a source's contents into first-class
objects in this bucket: a ``collection`` source demotes the source collection's
documents, a ``bucket`` source fans one or more source buckets' objects in. Only
valid for an enabled ``adapter_type`` in {``collection``, ``bucket``}; webhook
adapters do not use it.



# Bulk Mark Dlq Permanently Failed
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/bulk-mark-dlq-permanently-failed

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/dlq/bulk-mark-failed
Mark DLQ entries as permanently failed by error type.

Stops automatic retries for entries matching the specified error types.
Use this for structurally unrecoverable errors (e.g. missing proxy URLs,
unsupported file types) that will never succeed on retry.



# Create Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/create-sync-configuration

post /v1/buckets/{bucket_id}/syncs
Create a sync configuration for automated storage ingestion.

Establishes automated synchronization between an external storage provider
and a Mixpeek bucket. The sync monitors the source path and ingests files
according to the specified mode and filters.

**Supported Providers:** google_drive, s3, snowflake, sharepoint, tigris

**Built-in Robustness:**
- Dead Letter Queue (DLQ): Failed objects tracked with 3 retries
- Idempotent ingestion: Deduplication prevents duplicate objects
- Distributed locking: Prevents concurrent sync execution
- Rate limit handling: Automatic backoff on 429 responses
- Metrics: Duration, files synced/failed, batches created

**Sync Modes** (the request enum accepts exactly these two):
- `initial_only`: Single bulk import, then sync stops
- `continuous`: Polling-based monitoring (``polling_interval_seconds``,
  max 900)

(Docstring previously advertised ``one_time``/``scheduled``, which the
enum rejects with a 422 — modes here must match ``SyncCreateRequest``.)



# Delete Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/delete-sync-configuration

delete /v1/buckets/{bucket_id}/syncs/{sync_config_id}
Delete a sync configuration.



# Force Unlock Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/force-unlock-sync-configuration

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/unlock
Force-release a distributed lock on a sync configuration.

Use this when a sync trigger returns 'skipped_already_running' but no sync
is actually executing (zombie lock from a crashed worker).



# Get Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/get-sync-configuration

get /v1/buckets/{bucket_id}/syncs/{sync_config_id}
Fetch a sync configuration.



# Get Sync Job
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/get-sync-job

get /v1/buckets/{bucket_id}/syncs/{sync_config_id}/jobs/{sync_job_id}
Get details for a specific sync job.

Returns the full job record including status, file counts,
start/completion times, and any error messages.



# Get Sync Metrics
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/get-sync-metrics

get /v1/buckets/{bucket_id}/syncs/{sync_config_id}/metrics
Operational metrics for a sync in one call.

Folds config totals + health/staleness + the DLQ failure breakdown (grouped
by normalized error reason) + the most-recent job's throughput into a single
response — so callers can see "is this sync healthy / is anything backlogged"
without stitching /jobs and /dlq or reading logs.



# List Dlq Entries
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/list-dlq-entries

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/dlq
List Dead Letter Queue entries for a sync configuration.

Returns objects that failed to sync after all retry attempts.
Each entry includes the source object ID, error details, retry count,
and timestamps. Use this to diagnose and resolve sync failures.



# List Sync Configurations
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/list-sync-configurations

post /v1/buckets/{bucket_id}/syncs/list
List all sync configurations for a bucket with optional filtering.

Returns paginated list of sync configurations with status, metrics, and settings.
Use filters to find syncs by connection, status, or active state.



# List Sync Jobs
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/list-sync-jobs

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/jobs
List sync job history for a sync configuration.

Returns paginated job records ordered by most recent first.
Each job includes status, file counts, timing, and error details.
Use the optional status filter to find failed or running jobs.



# Pause Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/pause-sync-configuration

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/pause
Pause a sync configuration.

Pauses the sync, preventing new sync jobs from executing.
Running jobs will complete but no new jobs will be scheduled.
The sync configuration and all settings are preserved.

**Note:** Use POST /resume to reactivate the sync.



# Requeue Dlq Entries
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/requeue-dlq-entries

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/dlq/requeue
Return exhausted DLQ entries to the retry path.

The inverse of bulk-mark-failed. Use after fixing the underlying cause (a
restored proxy URL, a storage permission, an upstream outage) to re-drive the
entries that gave up. Without this, the only way to re-drive a DLQ was to
lean on natural sync-diff and hope the objects were rediscovered.

Resets `max_retries_reached` and `retry_count`, stamps `requeued_at`, and
clears `disposal_reason` so the retry task does not immediately re-retire the
entry. Only entries that have EXHAUSTED their retries are touched; anything
still retrying is left alone rather than silently granted extra attempts.

Omit `error_types` to requeue everything exhausted for the sync config.



# Resume Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/resume-sync-configuration

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/resume
Resume a paused sync configuration.

Reactivates a paused sync, allowing new sync jobs to be scheduled.
For continuous syncs, polling will resume at the configured interval.
The next sync will be incremental (only files modified since last sync).



# Trigger Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/trigger-sync-configuration

post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/trigger
Manually trigger a sync execution.

Creates a sync job and immediately dispatches it for async execution.
The sync job processes files from the storage provider and creates objects in the bucket.

**Execution Flow:**
1. Acquires distributed lock (prevents concurrent runs)
2. Iterates files from storage provider with configured filters
3. Creates objects idempotently (skips duplicates)
4. Failed objects go to Dead Letter Queue (3 retries)
5. Creates batches for collection processing (100 objects/batch)
6. Emits metrics and releases lock

**Use Cases:**
- Test sync configuration after creation
- Force sync outside of scheduled intervals
- Re-sync after updating connection credentials
- Trigger incremental sync (only modified files)

**Returns:** `sync_job_id` to track progress via GET /syncs/{id}



# Update Sync Configuration
Source: https://docs.mixpeek.com/docs/api-reference/bucket-syncs/update-sync-configuration

patch /v1/buckets/{bucket_id}/syncs/{sync_config_id}
Update a sync configuration.



# Batch Confirm Uploads
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/batch-confirm-uploads

post /v1/buckets/{bucket_identifier}/uploads/confirm/batch
Confirm multiple uploads in a single request (processed asynchronously).

    Maximum 100 confirmations per batch.
    All uploads must belong to the same bucket.

    Returns a task_id to track progress via GET /v1/tasks/{task_id}.



# Batch Create Uploads
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/batch-create-uploads

post /v1/buckets/{bucket_identifier}/uploads/batch
Generate multiple presigned URLs in a single request.

    All uploads belong to the same bucket (from path parameter).
    Maximum 100 uploads per batch.

    Shared metadata is merged with individual upload metadata (individual takes precedence).



# Confirm Upload
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/confirm-upload

post /v1/buckets/{bucket_identifier}/uploads/{upload_id}/confirm
Verify S3 upload completion and create bucket object.

    After uploading to S3 using the presigned URL, call this endpoint to:
    1. Verify the file exists in S3
    2. Validate ETag and file size (if provided)
    3. Create bucket object (default, unless create_object_on_confirm=false)
    4. Update upload status to COMPLETED

    **Processing**: the created object is PENDING and is not processed by
    confirmation. Add it to a batch and submit the batch to start processing,
    or set `auto_submit_batch=true` on this request to create and submit a
    batch for the new object in the same call.

    **Sync vs Async**:
    - Files < 100MB: Processed synchronously (~100ms)
    - Files >= 100MB or async=true: Processed asynchronously (returns task_id)

    **Duplicate Detection**:
    - If file hash matches existing upload, marks as duplicate
    - References original object_id if available

    **Unique Keys**:
    - If the bucket has a unique_key, the object is created under an insertion
      policy: `?policy=insert|update|upsert` on this request, else the bucket's
      `unique_key.default_policy`. With neither, confirmation returns 422.



# Create Upload
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/create-upload

post /v1/buckets/{bucket_identifier}/uploads
Generate a presigned URL for direct S3 upload.

    This endpoint validates all requirements BEFORE generating the presigned URL,
    ensuring immediate feedback if something is wrong (bucket inactive, quota exceeded, etc.).

    **Duplicate Detection (Enabled by Default)**:
    - If `file_hash` provided and `skip_duplicates=true`: Checks for existing upload
    - If duplicate found: Returns existing upload (200 OK) with `is_duplicate=true`
    - If new file: Returns presigned URL (201 Created) with `is_duplicate=false`

    **Two-Step Flow**:
    1. Call this endpoint → Get presigned URL
    2. PUT file to presigned URL → Upload directly to S3
    3. Call confirm endpoint → Verify upload and create object



# Delete Upload
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/delete-upload

delete /v1/buckets/{bucket_identifier}/uploads/{upload_id}
Cancel an upload and optionally delete the S3 object.

    Cannot cancel uploads with status COMPLETED.
    Can cancel uploads with status: PENDING, IN_PROGRESS, FAILED.



# Get Upload
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/get-upload

get /v1/buckets/{bucket_identifier}/uploads/{upload_id}
Retrieve an upload by its ID.

    Use this to check upload status, get S3 key, or retrieve created object_id after confirmation.

    **Presigned URLs**: Set `return_presigned_urls=true` query parameter to generate fresh presigned download URLs (default: false).
    The presigned URLs expire after 1 hour and allow direct download from S3.



# List Uploads
Source: https://docs.mixpeek.com/docs/api-reference/bucket-uploads/list-uploads

post /v1/buckets/{bucket_identifier}/uploads/list
List uploads in a bucket with filtering, sorting, search, and pagination.

    **Filtering**: Use LogicalOperator with shorthand syntax
    - Simple: `{"status": "PENDING", "metadata.campaign": "summer_2024"}`
    - Complex: `{"AND": [{"field": "file_size_bytes", "operator": "gte", "value": 1000000}]}`

    **Sorting**: Specify field and direction
    - Example: `{"field": "created_at", "direction": "desc"}`

    **Search**: Full-text search across filename and metadata
    - Example: `"search": "video"`

    **Pagination**: Use limit and offset
    - Example: `"limit": 50, "offset": 100`



# Create Bucket
Source: https://docs.mixpeek.com/docs/api-reference/buckets/create-bucket

post /v1/buckets
This endpoint allows you to create a new bucket with a defined schema.
    A bucket is a collection of objects that conform to the schema.
    The schema defines the structure and validation rules for objects in the bucket.



# Delete Bucket
Source: https://docs.mixpeek.com/docs/api-reference/buckets/delete-bucket

delete /v1/buckets/{bucket_identifier}
This endpoint deletes a bucket and all its resources including:
    - S3 objects and blobs
    - Running Ray jobs (cancels active batch processing jobs)
    - Batch processing artifacts
    - Upload files
    - Unique key lookups
    - MongoDB metadata

    The deletion is performed **asynchronously** via a background task.
    Returns immediately with a task_id that can be polled via GET /v1/tasks/{task_id}.

    **Response**:
    - `task_id`: Use this to poll deletion status via GET /v1/tasks/{task_id}
    - `status`: Initial status (PENDING)
    - `bucket_id`: The bucket being deleted
    - `bucket_name`: Name of the bucket
    - `object_count`: Number of objects that will be deleted

    **Polling**:
    Poll GET /v1/tasks/{task_id} until status is COMPLETED or FAILED.
    Use exponential backoff (start 1s, max 30s).



# Get Bucket
Source: https://docs.mixpeek.com/docs/api-reference/buckets/get-bucket

get /v1/buckets/{bucket_identifier}
This endpoint retrieves a bucket by its ID.



# List Buckets
Source: https://docs.mixpeek.com/docs/api-reference/buckets/list-buckets

post /v1/buckets/list
This endpoint lists buckets with pagination, sorting, and filtering options.



# Partially Update Bucket
Source: https://docs.mixpeek.com/docs/api-reference/buckets/partially-update-bucket

patch /v1/buckets/{bucket_identifier}
This endpoint allows you to partially update an existing bucket (PATCH operation).
    Only provided fields will be updated. At minimum, metadata can always be updated.
    Immutable fields like bucket_id and timestamps cannot be modified.



# Update Bucket
Source: https://docs.mixpeek.com/docs/api-reference/buckets/update-bucket

put /v1/buckets/{bucket_identifier}
This endpoint allows you to update an existing bucket.
    You can update the bucket's name, description, and metadata.



# Create standalone namespace (BYO vectors)
Source: https://docs.mixpeek.com/docs/api-reference/byo-documents/create-standalone-namespace

post /v1/namespaces/standalone
Create a standalone namespace for BYO (Bring Your Own) vectors. No collection or feature extractor binding is required. Users upsert pre-computed vectors directly via the direct upsert endpoint.



# Dry-run validate a standalone → managed promotion
Source: https://docs.mixpeek.com/docs/api-reference/byo-documents/dry-run-validate-standalone-managed-promotion

post /v1/namespaces/{namespace_id}/promote/validate
Validate a proposed promotion without mutating the namespace. Returns per-index mapping suggestions (dimension-matched candidates, flagged low-confidence) and validates any supplied vector_mappings / add_vectors for unknown indexes, unknown inference services, and dimension mismatches. Returns HTTP 422 when supplied mappings are invalid so callers can fix them before the real promote call.



# Get a document by ID (path-scoped namespace)
Source: https://docs.mixpeek.com/docs/api-reference/byo-documents/get-document-id

get /v1/namespaces/{namespace_id}/documents/{document_id}
Fetch a single document by document_id from a namespace named in the PATH — the mirror of POST /v1/namespaces/{namespace_id}/documents/upsert, so directly-upserted (BYO/standalone) documents can be read back on the same URL convention they were written with. Equivalent to GET /v1/documents/{document_id} with the X-Namespace header.



# Promote standalone namespace to managed mode
Source: https://docs.mixpeek.com/docs/api-reference/byo-documents/promote-standalone-namespace-managed-mode

post /v1/namespaces/{namespace_id}/promote
Promote a standalone (BYO vectors) namespace to managed mode. Existing vectors and documents are preserved. Optionally maps existing vector indexes to inference services and adds new indexes.



# Upsert documents with BYO vectors
Source: https://docs.mixpeek.com/docs/api-reference/byo-documents/upsert-documents-byo-vectors

post /v1/namespaces/{namespace_id}/documents/upsert
Upsert documents with user-provided vectors directly into a namespace. This bypasses the collection/batch/extractor pipeline entirely. Documents go directly to the vector store. Maximum 1000 documents per call.



# Delete a model
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/delete-a-model

delete /v1/namespaces/{namespace_id}/models/{model_id}
Delete a custom model from the namespace.



# Deploy model to Ray object store
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/deploy-model-to-ray-object-store

post /v1/namespaces/{namespace_id}/models/{model_id}/deploy
Pre-load model weights into the Ray object store for fast access by plugins.

This operation:
1. Downloads the model archive from S3
2. Deserializes weights based on format (safetensors, pytorch, etc.)
3. Stores weights in Ray object store for zero-copy sharing

After deployment, plugins can load the model instantly using:
```python
from engine.models.loader import load_namespace_model
weights = load_namespace_model("model_id")
```

**Note:** This is optional - models are also loaded on-demand when plugins
first request them. Use this endpoint to pre-warm the cache.



# Disable org model for namespace
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/disable-org-model-for-namespace

post /v1/namespaces/{namespace_id}/models/org/{model_id}/disable
Disable an org-level model for this namespace.

This removes the namespace-specific deployment record. The model
remains available at the organization level and can be re-enabled.



# Enable org model for namespace
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/enable-org-model-for-namespace

post /v1/namespaces/{namespace_id}/models/org/{model_id}/enable
Enable an org-level model for this namespace.

This creates a namespace-specific deployment record for the model,
allowing it to be used within this namespace. Optionally deploys
the model to Ray immediately.

**Note:** The model must first be registered at the organization level
via POST /models/uploads.



# Get model details
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/get-model-details

get /v1/namespaces/{namespace_id}/models/{model_id}
Get detailed information about a specific model.



# List available org models
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/list-available-org-models

get /v1/namespaces/{namespace_id}/models/available
List org-level models available to enable in this namespace.

Shows all models registered at the organization level and whether
they are already enabled in the target namespace.



# List models in namespace
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/list-models-in-namespace

get /v1/namespaces/{namespace_id}/models
List all custom models uploaded to the namespace.



# Upload a custom model
Source: https://docs.mixpeek.com/docs/api-reference/custom-models/upload-a-custom-model

post /v1/namespaces/{namespace_id}/models
Upload custom model weights to the namespace.

**Requirements:**
- Organization must be on Enterprise tier

**Supported Model Formats:**
- `safetensors`: SafeTensors format (recommended for transformers)
- `onnx`: ONNX Runtime format
- `pytorch`: PyTorch state_dict or TorchScript
- `huggingface`: HuggingFace model directory

**Base Images (auto-selected based on format):**
- `mixpeek/serve-gpu:latest`: For safetensors, pytorch, huggingface (includes torch, transformers)
- `mixpeek/serve-minimal:latest`: For ONNX (includes onnxruntime only)

**Important:** Models run in fixed base images. You cannot install additional pip packages.
All required frameworks (torch, transformers, onnxruntime) are pre-installed.



# Clone Namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespace-clone/clone-namespace

post /v1/namespaces/{namespace_identifier}/clone
Clone a namespace with all its data.

    **What gets cloned:**
    - Namespace configuration (extractors, payload indexes)
    - Buckets (metadata, references same S3 files)
    - Collections (full copy of all vectors/embeddings)
    - Retrievers (pipeline configuration)

    **Use Cases:**
    - Create staging environment from production
    - Backup namespace with all data
    - Fork namespace for experimentation

    **For a config-only copy (no data):** export the namespace with
    POST /v1/manifest/export and apply the manifest with POST /v1/manifest/apply.



# Get extractor details
Source: https://docs.mixpeek.com/docs/api-reference/namespace-extractors/get-extractor-details

get /v1/namespaces/{namespace_id}/extractors/{extractor_id}
Get detailed information about a specific extractor.

Works for both builtin extractors and custom plugins.

**Parameters:**
- `extractor_id`: Extractor identifier (e.g., 'text_extractor_v1', 'my_custom_plugin_1_0_0')

**Response includes:**
- Full schema information (input, output, parameters)
- Vector index configuration
- For custom plugins: deployment status, validation status



# List all extractors available to namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespace-extractors/list-all-extractors-available-to-namespace

get /v1/namespaces/{namespace_id}/extractors
List all feature extractors available for use in this namespace.

This endpoint returns a **unified view** combining:
- **Builtin extractors**: Core extractors shipped with Mixpeek (text_extractor, image_extractor, etc.)
- **Custom plugins**: User-uploaded plugins at org or namespace level (Enterprise)

Each extractor includes:
- `input_schema`: JSON schema for input data validation
- `output_schema`: JSON schema for output document structure
- `parameter_schema`: JSON schema for configurable parameters
- `required_vector_indexes`: Vector indexes produced by this extractor
- `feature_uri`: URI to reference this extractor in collections

**Use Cases:**
- Discover available extractors when creating collections
- Get schema information for SDK code generation
- Check which custom plugins are deployed

**Filtering:**
- `source=builtin`: Only builtin extractors
- `source=custom`: Only custom plugins
- `source=all` (default): All extractors



# Cancel Migration
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/cancel-migration

post /v1/namespaces/migrations/{migration_id}/cancel
Cancel a running migration.

Args:
    request: FastAPI request
    migration_id: Migration ID
    cancel_request: Cancellation options

Returns:
    CancelMigrationResponse with updated status



# Create Migration
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/create-migration

post /v1/namespaces/migrations/
Create a new namespace migration.

This endpoint creates a migration and optionally validates it.
Use start_immediately=True to begin execution immediately.

Args:
    request: FastAPI request
    create_request: Migration configuration

Returns:
    CreateMigrationResponse with migration ID and status



# Delete Migration
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/delete-migration

delete /v1/namespaces/migrations/{migration_id}
Delete a migration record.

Only draft, completed, failed, or cancelled migrations can be deleted.

Args:
    request: FastAPI request
    migration_id: Migration ID



# Get Migration
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/get-migration

get /v1/namespaces/migrations/{migration_id}
Get migration details and status.

Args:
    request: FastAPI request
    migration_id: Migration ID

Returns:
    GetMigrationResponse with full migration details



# List Migrations
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/list-migrations

post /v1/namespaces/migrations/list
List migrations with optional filters.

Args:
    request: FastAPI request
    list_request: Filter and pagination parameters

Returns:
    ListMigrationsResponse with migrations list



# Start Migration
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/start-migration

post /v1/namespaces/migrations/{migration_id}/start
Start a migration execution.

Args:
    request: FastAPI request
    migration_id: Migration ID
    start_request: Start options

Returns:
    StartMigrationResponse with task ID



# Validate Migration
Source: https://docs.mixpeek.com/docs/api-reference/namespace-migrations/validate-migration

post /v1/namespaces/migrations/validate
Validate a migration configuration without creating it.

Use this endpoint to check if a migration configuration is valid
before actually creating and running it.

Args:
    request: FastAPI request
    validate_request: Configuration to validate

Returns:
    ValidateMigrationResponse with validation results



# Create Namespace Snapshot
Source: https://docs.mixpeek.com/docs/api-reference/namespace-snapshots/create-namespace-snapshot

post /v1/namespaces/{namespace_identifier}/snapshots
Create a point-in-time snapshot of the entire namespace. Captures all MongoDB documents and an S3 object manifest. Optionally copies S3 objects to the snapshot prefix for full data preservation (required if you plan to delete the namespace and restore later). With import_from, imports a snapshot taken on another plane instead: its artifacts are copied from an allowed backup bucket, each checked against its recorded sha256, and the snapshot is completed only if verify finds it restorable.



# Delete Snapshot
Source: https://docs.mixpeek.com/docs/api-reference/namespace-snapshots/delete-snapshot

delete /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}
Delete a snapshot and its associated S3 data.



# Get Snapshot
Source: https://docs.mixpeek.com/docs/api-reference/namespace-snapshots/get-snapshot

get /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}
Get details of a specific snapshot. With download=true the response also carries a signed URL and the sha256 for every artifact (configuration, Mongo dumps, vector export, object manifest), valid for one hour, so you can download the namespace.



# List Namespace Snapshots
Source: https://docs.mixpeek.com/docs/api-reference/namespace-snapshots/list-namespace-snapshots

post /v1/namespaces/{namespace_identifier}/snapshots/list
List all snapshots for a namespace, newest first.



# Restore From Snapshot
Source: https://docs.mixpeek.com/docs/api-reference/namespace-snapshots/restore-from-snapshot

post /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}/restore
Restore a namespace from a snapshot. If target_namespace_name is provided, creates a new namespace with the restored data. Otherwise, restores in-place (replaces current namespace data).



# Verify Snapshot Restore-Readiness
Source: https://docs.mixpeek.com/docs/api-reference/namespace-snapshots/verify-snapshot-restore-readiness

get /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}/verify
Read-only pre-flight check that a snapshot can actually be restored. Restore is lenient and silently skips missing artifacts, so this surfaces what a restore would drop (missing collection dumps, unreachable vectors, absent copied objects) BEFORE you depend on it in a recovery. Pass deep=true to also decompress each collection dump and detect truncation.



# Create Namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/create-namespace

post /v1/namespaces
Creates a new namespace with specified feature extractors and payload indexes.



# Delete Namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/delete-namespace

delete /v1/namespaces/{namespace_identifier}
This endpoint deletes a namespace and ALL its resources including:
    - All buckets (with S3 objects, batches, uploads)
    - All collections (with vector points, cache, webhooks)
    - All clusters (with Ray jobs, executions, triggers, S3 artifacts)
    - All retrievers (with executions, evaluations, interactions, cache)
    - Remaining MongoDB collections (tasks, uploads, taxonomies, API keys, etc.)
    - All S3 objects with namespace prefix
    - The namespace's vectors in the vector store
    - All namespace cache (across all scopes)
    - The namespace's analytics events
    - Namespace metadata

    The deletion is performed asynchronously. Returns a task_id that can be used
    to poll for deletion progress via GET /v1/tasks/{task_id}.

    ⚠️  WARNING: This operation is irreversible and will delete ALL data in the namespace!



# Ensure Namespace Payload Indexes
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/ensure-namespace-payload-indexes

post /v1/namespaces/{namespace_identifier}/ensure-indexes
(Re-)create the namespace's declared payload indexes in the vector store, including FULLTEXT/BM25, idempotently. ZERO-EXTRACTION: registers any missing indexes and backfills existing documents WITHOUT triggering any ingest or re-extraction. Use to repair a missing index (e.g. a FULLTEXT index that failed to register) or as a self-serve re-index tool — no re-ingest, no GPU.

READING THE RESPONSE: `ensured` lists indexes the call confirmed present — it does NOT mean they were newly built, and an already-registered index is a cheap no-op on the shard. `rebuilt` is populated only with `force_rebuild=true`, which is the case that does real work. `skipped` is normally empty because the store reports an idempotent no-op as success rather than an error, so an empty `skipped` does not imply the rest were created. `created` is a deprecated alias of `ensured` + `rebuilt`.

COST: a plain ensure is cheap. `force_rebuild=true` applies to EVERY declared index, drops and rebuilds each from the complete store, and for FULLTEXT fields first hydrates the full payload store — no GPU, but real shard CPU and memory proportional to the corpus. Size it before running on a large or memory-constrained namespace.



# Get Namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/get-namespace

get /v1/namespaces/{namespace_identifier}
Retrieve details of a specific namespace using either its name or ID



# Get Namespace Stats
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/get-namespace-stats

get /v1/namespaces/{namespace_identifier}/stats
Returns document, bucket, collection, and object counts for a single namespace. Use this instead of relying on counts in the list endpoint.



# Get Namespace Stats (Batch)
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/get-namespace-stats-batch

post /v1/namespaces/stats/batch
Returns document/bucket/collection/object counts for many namespaces in ONE call. Replaces per-row /stats fan-out on list views. Counts are aggregated in batch and live doc-counts gathered concurrently; namespaces the caller cannot see are omitted.



# List Namespaces
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/list-namespaces

post /v1/namespaces/list
List all namespaces for a user



# List Namespaces
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/list-namespaces-get

get /v1/namespaces
List all namespaces (GET alias for POST /list). Supports query param pagination: ?limit=N&offset=M



# Partially Update Namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/partially-update-namespace

patch /v1/namespaces/{namespace_identifier}
Partially updates an existing namespace (PATCH operation)



# Update Namespace
Source: https://docs.mixpeek.com/docs/api-reference/namespaces/update-namespace

put /v1/namespaces/{namespace_identifier}
Fully updates an existing namespace (all fields required)



# Self-Improving CV Pipeline
Source: https://docs.mixpeek.com/docs/tutorials/annotations-improve-features

Deploy a YOLO model, annotate detections, fine-tune from corrections, and compound accuracy over time using annotations, taxonomies, and clusters

> [!TIP]
>
> This tutorial builds a computer vision pipeline that gets smarter with use. You'll deploy YOLO as a custom extractor, review detections with annotations, export corrections as training data, and close the loop by uploading improved weights — all through Mixpeek primitives.

![Self-improving CV pipeline: YOLO extractor → human review → fine-tune → redeploy, with taxonomies and clusters feeding back into the loop](https://mintcdn.com/mixpeek/HQEhVpg-9tm3E9k-/assets/tutorials/self-improving-cv-pipeline.svg?fit=max\&auto=format\&n=HQEhVpg-9tm3E9k-\&q=85\&s=dbf12b0a569e26ee99a30b70b0f6dd6b)

## What You'll Build

A closed-loop object detection system that compounds accuracy over time:

**1. Detect**

Deploy YOLO as a custom extractor. Every image ingested produces bounding boxes, class labels, and detection embeddings.

**2. Review**

Surface low-confidence detections for human review. Annotate each detection as confirmed, corrected, false positive, or missed.

**3. Fine-Tune**

Export annotations as YOLO-format training data. Fine-tune externally and upload improved weights as a new extractor version.

**4. Compound**

Taxonomies auto-classify future detections against your curated ground truth. Clusters discover new categories. Retroactive reapplication improves old data.

**Prerequisites:** A Mixpeek namespace with an API key. Familiarity with [custom extractors](/docs/processing/custom-extractors) and the [model registry](/docs/processing/model-registry) helps but isn't required.

***

## 1. Deploy YOLO as a Custom Extractor

Package a YOLO-based detector as a custom extractor. The extractor reads images, runs inference, and outputs detection features — bounding boxes, class labels, and confidence scores.

> **manifest.py — extractor metadata and feature definitions**
>
> ```python theme={null}
> feature_extractor_name = "yolo_detector"
> version = "1.0.0"
> description = "YOLOv8 object detection with bounding boxes and class embeddings"
>
> dependencies = ["ultralytics==8.2.0", "torch>=2.0"]
>
> features = [
>     {
>         "feature_type": "json",
>         "feature_name": "detections",
>     },
>     {
>         "feature_type": "embedding",
>         "feature_name": "detection_embedding",
>         "embedding_dim": 512,
>         "distance_metric": "cosine",
>     },
> ]
>
> output_schema = {
>     "detections": {
>         "type": "array",
>         "items": {
>             "type": "object",
>             "properties": {
>                 "class": {"type": "string"},
>                 "confidence": {"type": "number"},
>                 "bbox": {
>                     "type": "object",
>                     "properties": {
>                         "x": {"type": "number"},
>                         "y": {"type": "number"},
>                         "w": {"type": "number"},
>                         "h": {"type": "number"},
>                     },
>                 },
>             },
>         },
>     },
>     "detection_embedding": {
>         "type": "array",
>         "items": {"type": "number"},
>         "description": "512-dim CLIP embedding of the highest-confidence crop",
>     },
> }
>
> input_mappings = {"image": "image"}
> tier = 1
> tier_label = "OBJECT_DETECTION"
> compute_profile = {"resource_type": "gpu"}
> ```
>
> > [!WARNING]
> >
> > Use the exact key names: `feature_type`, `feature_name`, `embedding_dim`, `distance_metric`. Using `name`/`type`/`dimensions`/`distance` will silently create zero vector indexes.

> **pipeline.py — YOLO inference with LazyModelMixin**
>
> ```python theme={null}
> import numpy as np
> import pandas as pd
> from engine.models.lazy import LazyModelMixin
> from engine.inference.services import BaseBatchInferenceService
> from engine.io import parallel_io
>
>
> class YOLODetector(LazyModelMixin, BaseBatchInferenceService):
>     model_id = "ultralytics/yolov8m"
>     model_source = "huggingface"
>
>     def _instantiate_model(self, cached_data):
>         from ultralytics import YOLO
>         model = YOLO("yolov8m.pt")
>         model.to(self._detect_device())
>         return model, None
>
>     def _process_batch(self, batch):
>         model, _ = self.get_model()
>
>         images = parallel_io(batch["data"].tolist())
>
>         results = model(images, conf=0.25)
>
>         all_detections = []
>         all_embeddings = []
>
>         for result in results:
>             detections = []
>             for box in result.boxes:
>                 detections.append({
>                     "class": result.names[int(box.cls)],
>                     "confidence": float(box.conf),
>                     "bbox": {
>                         "x": float(box.xywh[0][0]),
>                         "y": float(box.xywh[0][1]),
>                         "w": float(box.xywh[0][2]),
>                         "h": float(box.xywh[0][3]),
>                     },
>                 })
>             all_detections.append(detections)
>
>             if detections:
>                 best = max(detections, key=lambda d: d["confidence"])
>                 crop = result.orig_img[
>                     int(best["bbox"]["y"] - best["bbox"]["h"]/2):int(best["bbox"]["y"] + best["bbox"]["h"]/2),
>                     int(best["bbox"]["x"] - best["bbox"]["w"]/2):int(best["bbox"]["x"] + best["bbox"]["w"]/2),
>                 ]
>                 embedding = self._embed_crop(crop)
>             else:
>                 embedding = np.zeros(512).tolist()
>             all_embeddings.append(embedding)
>
>         batch["detections"] = all_detections
>         batch["detection_embedding"] = all_embeddings
>         return batch
>
>     def _embed_crop(self, crop):
>         # Replace with CLIP or similar for production
>         return np.random.randn(512).astype(np.float32).tolist()
>
>
> def build_steps(extractor_request=None, base_steps=None, **kwargs):
>     steps = list(base_steps or [])
>     steps.append(YOLODetector())
>     return {"steps": steps, "prepare": lambda ds: ds}
>
>
> def extract(extractor_request=None, base_steps=None, **kwargs):
>     result = build_steps(
>         extractor_request=extractor_request,
>         base_steps=base_steps, **kwargs
>     )
>     class PipelineResult:
>         def __init__(self, steps, prepare):
>             self.steps = steps
>             self.prepare = prepare
>     return PipelineResult(result["steps"], result["prepare"])
> ```

### Upload and Deploy

```bash theme={null}
# Package
zip -r yolo_detector.zip yolo_detector/

# Upload
UPLOAD=$(curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/plugins/uploads" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "yolo_detector", "version": "1.0.0", "file_size_bytes": 50000}')

UPLOAD_ID=$(echo $UPLOAD | jq -r '.upload_id')
PRESIGNED_URL=$(echo $UPLOAD | jq -r '.presigned_url')

curl -s -X PUT "$PRESIGNED_URL" \
  -H "Content-Type: application/zip" \
  --data-binary @yolo_detector.zip

curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/plugins/uploads/$UPLOAD_ID/confirm" \
  -H "Authorization: Bearer $MP_API_KEY"

# Deploy
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/plugins/yolo_detector_1_0_0/deploy?deployment_type=batch_only" \
  -H "Authorization: Bearer $MP_API_KEY"
```

> [!NOTE]
>
> Your extractor is now available at feature URI `mixpeek://yolo_detector@1.0.0/detection_embedding`. This URI is the stable contract — retrievers, taxonomies, and clusters all reference it, so you can swap model versions without breaking downstream consumers.

***

## 2. Create a Collection and Ingest Images

Bind the YOLO extractor to a bucket so every uploaded image gets processed automatically.

```bash cURL theme={null}
# Create bucket
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/buckets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "security_footage",
    "bucket_schema": {
      "properties": {
        "image": {"type": "image", "required": true},
        "camera_id": {"type": "text"},
        "timestamp": {"type": "text"}
      }
    }
  }'

# Create collection with YOLO extractor
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/collections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "detected_objects",
    "feature_extractor": {
      "feature_extractor_name": "yolo_detector",
      "version": "1.0.0"
    },
    "source": {
      "type": "bucket",
      "bucket_ids": ["bkt_security_footage"]
    }
  }'

# Upload images
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/buckets/$BUCKET_ID/objects" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {"camera_id": "cam_lobby_01", "timestamp": "2026-05-03T14:30:00Z"},
    "blobs": [{
      "property": "image",
      "type": "image",
      "data": {"url": "s3://my-bucket/footage/frame_001.jpg"}
    }]
  }'
```

```python Python SDK theme={null}
from mixpeek import Mixpeek

mp = Mixpeek(api_key="API_KEY")

bucket = mp.buckets.create(
    name="security_footage",
    bucket_schema={
        "properties": {
            "image": {"type": "image", "required": True},
            "camera_id": {"type": "text"},
            "timestamp": {"type": "text"},
        }
    },
)

collection = mp.collections.create(
    name="detected_objects",
    feature_extractor={
        "feature_extractor_name": "yolo_detector",
        "version": "1.0.0",
    },
    source={"type": "bucket", "bucket_ids": [bucket.bucket_id]},
)

mp.buckets.objects.create(
    bucket_id=bucket.bucket_id,
    metadata={"camera_id": "cam_lobby_01", "timestamp": "2026-05-03T14:30:00Z"},
    blobs=[{
        "property": "image",
        "type": "image",
        "data": {"url": "s3://my-bucket/footage/frame_001.jpg"},
    }],
)
```

Trigger batch processing to run YOLO across all uploaded images:

```bash theme={null}
# Buckets/batches are top-level, keyed by the X-Namespace header.
# A batch is created (objects + collections) then submitted.
H=(-H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $NS_ID" -H "Content-Type: application/json")

OBJIDS=$(curl -s "$MP_API_URL/v1/buckets/$BUCKET_ID/objects" "${H[@]}" | jq -c '[.results[].object_id]')
BATCH_ID=$(curl -s -X POST "$MP_API_URL/v1/buckets/$BUCKET_ID/batches" "${H[@]}" \
  -d "{\"batch_name\":\"yolo-run\",\"object_ids\":$OBJIDS,\"collection_ids\":[\"$COLLECTION_ID\"]}" | jq -r '.batch_id')
curl -s -X POST "$MP_API_URL/v1/buckets/$BUCKET_ID/batches/$BATCH_ID/submit" "${H[@]}" \
  -d "{\"collection_ids\":[\"$COLLECTION_ID\"]}"
```

***

## 3. Build a Retriever for Detection Review

Create a retriever that surfaces detections for human review. Filter by confidence to focus reviewers on borderline cases where the model is least sure.

```bash cURL theme={null}
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "detection_review",
    "stages": [{
      "stage_name": "low_confidence",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [{
            "feature_uri": "mixpeek://yolo_detector@1.0.0/detection_embedding",
            "query": "{{INPUT.query}}"
          }],
          "filters": {
            "AND": [{
              "field": "detections.0.confidence",
              "operator": "lt",
              "value": 0.7
            }]
          }
        }
      }
    }]
  }'
```

```python Python SDK theme={null}
retriever = mp.retrievers.create(
    name="detection_review",
    stages=[{
        "stage_name": "low_confidence",
        "stage_type": "filter",
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [{
                    "feature_uri": "mixpeek://yolo_detector@1.0.0/detection_embedding",
                    "query": "{{INPUT.query}}",
                }],
                "filters": {
                    "AND": [{
                        "field": "detections.0.confidence",
                        "operator": "lt",
                        "value": 0.7,
                    }]
                },
            },
        },
    }],
)
```

> [!TIP]
>
> **Focus reviewers on uncertainty.** Annotating high-confidence correct detections adds little value. Filtering for confidence \< 0.7 routes reviewers to the cases where YOLO is least sure — exactly the training signal you need for the next fine-tune.

***

## 4. Annotate Detections

Reviewers examine each detection and record their decision. The `payload` carries the corrected bounding boxes and class labels — this is what becomes training data.

### Label Vocabulary

Before annotating, establish a consistent label vocabulary. The stats endpoint groups by exact string match, so consistency matters.

> **confirmed**
>
> Detection is correct as-is. Becomes a **positive training sample** that reinforces the model.

> **corrected**
>
> Bounding box or class was adjusted. **Highest-value sample** — teaches the model its mistakes.

> **false\_positive**
>
> No real object at this location. Becomes a **hard negative** that reduces false alarms.

> **missed**
>
> Object exists but wasn't detected. Added as **new ground truth** for the next training run.

### Recording Decisions

**Individual Annotations**

```bash cURL theme={null}
# Correct detection — class was wrong
curl -X POST "$MP_API_URL/v1/annotations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_frame_001_det_3",
    "collection_id": "col_detected_objects",
    "label": "corrected",
    "confidence": 1.0,
    "reasoning": "Model predicted car, actual object is delivery van.",
    "payload": {
      "predicted_class": "car",
      "true_class": "delivery_van",
      "bbox": {"x": 340, "y": 220, "w": 180, "h": 120},
      "image_width": 1920,
      "image_height": 1080
    },
    "retriever_id": "ret_detection_review",
    "execution_id": "exec_review_batch_01"
  }'

# Confirm correct detection
curl -X POST "$MP_API_URL/v1/annotations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_frame_001_det_1",
    "collection_id": "col_detected_objects",
    "label": "confirmed",
    "confidence": 1.0,
    "payload": {
      "predicted_class": "person",
      "true_class": "person",
      "bbox": {"x": 640, "y": 300, "w": 90, "h": 200}
    },
    "retriever_id": "ret_detection_review",
    "execution_id": "exec_review_batch_01"
  }'

# Reject false positive
curl -X POST "$MP_API_URL/v1/annotations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_frame_001_det_5",
    "collection_id": "col_detected_objects",
    "label": "false_positive",
    "reasoning": "Shadow on wall, not an actual object.",
    "retriever_id": "ret_detection_review",
    "execution_id": "exec_review_batch_01"
  }'
```

```python Python SDK theme={null}
# Correct detection — class was wrong
mp.annotations.create(
    document_id="doc_frame_001_det_3",
    collection_id="col_detected_objects",
    label="corrected",
    confidence=1.0,
    reasoning="Model predicted car, actual object is delivery van.",
    payload={
        "predicted_class": "car",
        "true_class": "delivery_van",
        "bbox": {"x": 340, "y": 220, "w": 180, "h": 120},
        "image_width": 1920,
        "image_height": 1080,
    },
    retriever_id="ret_detection_review",
    execution_id="exec_review_batch_01",
)

# Confirm correct detection
mp.annotations.create(
    document_id="doc_frame_001_det_1",
    collection_id="col_detected_objects",
    label="confirmed",
    confidence=1.0,
    payload={
        "predicted_class": "person",
        "true_class": "person",
        "bbox": {"x": 640, "y": 300, "w": 90, "h": 200},
    },
    retriever_id="ret_detection_review",
    execution_id="exec_review_batch_01",
)

# Reject false positive
mp.annotations.create(
    document_id="doc_frame_001_det_5",
    collection_id="col_detected_objects",
    label="false_positive",
    reasoning="Shadow on wall, not an actual object.",
    retriever_id="ret_detection_review",
    execution_id="exec_review_batch_01",
)
```

**Bulk Review (up to 1,000)**

Process an entire review queue in a single call:

```bash theme={null}
curl -X POST "$MP_API_URL/v1/annotations/bulk" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "create": [
      {
        "document_id": "doc_det_001",
        "collection_id": "col_detected_objects",
        "label": "confirmed",
        "payload": {"true_class": "person", "bbox": {"x": 100, "y": 200, "w": 50, "h": 120}}
      },
      {
        "document_id": "doc_det_002",
        "collection_id": "col_detected_objects",
        "label": "corrected",
        "payload": {"predicted_class": "dog", "true_class": "cat", "bbox": {"x": 300, "y": 150, "w": 80, "h": 60}}
      },
      {
        "document_id": "doc_det_003",
        "collection_id": "col_detected_objects",
        "label": "false_positive"
      }
    ]
  }'
```

Each operation is independent — a failure in one does not roll back the others. The response includes per-operation results so you can retry individual failures.

> [!NOTE]
>
> Always include `retriever_id` and `execution_id` when annotating retriever results. This provenance link lets you measure which retrievers produce the most approved vs. rejected results — critical for evaluating retriever quality over time.

***

## 5. Track Model Quality with Stats

Monitor how your model is performing across review cycles. A rising `corrected` or `false_positive` rate signals the model needs retraining.

```bash cURL theme={null}
curl "$MP_API_URL/v1/annotations/stats?collection_id=col_detected_objects" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python SDK theme={null}
stats = mp.annotations.stats(collection_id="col_detected_objects")
# {"total": 500, "by_label": {"confirmed": 340, "corrected": 95, "false_positive": 45, "missed": 20}}
```

### Interpreting Stats for Retraining Decisions

| Metric | Healthy | Action Needed |
| - | - | - |
| Confirmed rate | > 80% | Model is performing well |
| Corrected rate | > 15% | Class confusion — retrain with corrected examples |
| False positive rate | > 10% | Confidence threshold too low, or hard negatives needed |
| Missed rate | > 5% | Model is missing objects — add missed annotations as positive training data |

> [!WARNING]
>
> Track stats **over time**, not just cumulatively. A model at 90% confirmed overall might be at 60% confirmed on last week's data if the deployment context changed (new camera angle, different lighting, seasonal changes).

***

## 6. Export Annotations as YOLO Training Data

Query your annotations and convert them to YOLO format. Every corrected bounding box and confirmed detection becomes a labeled training sample.

```python theme={null}
import os

confirmed = mp.annotations.list(
    collection_id="col_detected_objects",
    label="confirmed",
)
corrected = mp.annotations.list(
    collection_id="col_detected_objects",
    label="corrected",
)

os.makedirs("dataset/labels", exist_ok=True)

class_map = {}
class_counter = 0

for ann in confirmed.items + corrected.items:
    payload = ann.payload
    true_class = payload.get("true_class", payload.get("predicted_class"))
    bbox = payload.get("bbox", {})
    img_w = payload.get("image_width", 1920)
    img_h = payload.get("image_height", 1080)

    if not bbox or not true_class:
        continue

    if true_class not in class_map:
        class_map[true_class] = class_counter
        class_counter += 1

    # Convert to YOLO format: class x_center y_center width height (normalized)
    x_center = bbox["x"] / img_w
    y_center = bbox["y"] / img_h
    width = bbox["w"] / img_w
    height = bbox["h"] / img_h

    label_file = f"dataset/labels/{ann.document_id}.txt"
    with open(label_file, "a") as f:
        f.write(f"{class_map[true_class]} {x_center:.6f} {y_center:.6f} {width:.6f} {height:.6f}\n")

with open("dataset/classes.txt", "w") as f:
    for name, idx in sorted(class_map.items(), key=lambda x: x[1]):
        f.write(f"{name}\n")

print(f"Exported {len(confirmed.items) + len(corrected.items)} annotations across {len(class_map)} classes")
```

> [!NOTE]
>
> The YOLO format expects one `.txt` file per image with lines of `class x_center y_center width height`, all values normalized to `[0, 1]`. The export script handles this conversion from Mixpeek's pixel-coordinate annotation payloads.

***

## 7. Fine-Tune and Redeploy

Fine-tune YOLO externally with your exported annotations, then upload the improved weights as a new extractor version.

**Fine-Tune (Ultralytics)**

```python theme={null}
from ultralytics import YOLO

model = YOLO("yolov8m.pt")
model.train(
    data="dataset/data.yaml",
    epochs=50,
    imgsz=640,
    batch=16,
    name="yolo_v2_finetuned",
)

model.export(format="torchscript")
```

**Upload as New Extractor Version**

```bash theme={null}
# Update manifest.py version to "2.0.0" and package
zip -r yolo_detector_v2.zip yolo_detector/

UPLOAD=$(curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/plugins/uploads" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "yolo_detector", "version": "2.0.0", "file_size_bytes": 80000}')

UPLOAD_ID=$(echo $UPLOAD | jq -r '.upload_id')
PRESIGNED_URL=$(echo $UPLOAD | jq -r '.presigned_url')

curl -s -X PUT "$PRESIGNED_URL" \
  -H "Content-Type: application/zip" \
  --data-binary @yolo_detector_v2.zip

curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/plugins/uploads/$UPLOAD_ID/confirm" \
  -H "Authorization: Bearer $MP_API_KEY"

curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/plugins/yolo_detector_2_0_0/deploy?deployment_type=batch_only" \
  -H "Authorization: Bearer $MP_API_KEY"
```

**Alternative: Model Registry**

Upload fine-tuned weights to the [Model Registry](/docs/processing/model-registry) as a namespace model. This separates model weights from extractor code, so you can iterate on weights without repackaging the extractor.

```bash theme={null}
# Package weights
tar -czvf yolo_v2_weights.tar.gz ./runs/detect/yolo_v2_finetuned/weights/

# Upload to registry
curl -X POST "$MP_API_URL/v1/namespaces/$NS_ID/models" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -F "file=@yolo_v2_weights.tar.gz" \
  -F "name=yolo-detector" \
  -F "version=2.0.0" \
  -F "model_format=pytorch" \
  -F "task_type=detection" \
  -F "num_gpus=1"

# Deploy to Ray object store
curl -X POST "$MP_API_URL/v1/namespaces/$NS_ID/models/yolo-detector_2_0_0/deploy" \
  -H "Authorization: Bearer $MP_API_KEY"
```

Then reference in your extractor via `load_namespace_model("yolo-detector_2_0_0")`.

> [!TIP]
>
> The new version gets its own feature URI — `mixpeek://yolo_detector@2.0.0/detection_embedding` — so you can run both versions side by side and compare results before switching production traffic.

***

## 8. Auto-Classify Detections with Taxonomies

Once you have enough confirmed annotations, promote them to a reference collection. Then create a taxonomy that auto-classifies future detections by matching against your curated ground truth.

**1. Create the taxonomy**

```bash theme={null}
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/taxonomies" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"taxonomy_name\": \"object-catalog\",
    \"taxonomy_type\": \"flat\",
    \"retriever_id\": \"ret_object_catalog_search\",
    \"collection_id\": \"col_verified_objects\",
    \"input_mappings\": [{
      \"source\": \"detection_embedding\",
      \"target\": \"query\"
    }],
    \"enrichment_fields\": [
      {\"source\": \"true_class\", \"target\": \"verified_class\"},
      {\"source\": \"category\", \"target\": \"object_category\"}
    ],
    \"threshold\": 0.75,
    \"execution_mode\": \"materialize\"
  }"
```

**2. Apply to your detection collection**

Every new image auto-classifies at ingestion time:

```json theme={null}
{
  "taxonomy_applications": [
    {
      "taxonomy_id": "tax_object_catalog",
      "execution_mode": "materialize"
    }
  ]
}
```

**3. Backfill existing data when the reference improves**

When annotations accumulate and your reference collection gets better, trigger retroactive mode to reclassify all existing detections:

```bash theme={null}
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/taxonomies/tax_object_catalog/apply" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"execution_mode": "retroactive", "collection_id": "col_detected_objects"}'
```

> [!NOTE]
>
> Retroactive reapplication is a first-class operation, not a data migration. When your reference improves — more annotations, better coverage, new categories — old data automatically re-benefits.

***

## 9. Discover New Categories with Clusters

YOLO might detect "unknown" objects that don't fit existing classes. Use clustering to group similar unknowns and discover categories you haven't labeled yet.

```bash theme={null}
curl -s -X POST "$MP_API_URL/v1/namespaces/$NS_ID/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"cluster_name\": \"unknown-objects\",
    \"collection_ids\": [\"$COLLECTION_ID\"],
    \"cluster_type\": \"vector\",
    \"vector_config\": {
      \"feature_uris\": [\"mixpeek://yolo_detector@2.0.0/detection_embedding\"],
      \"clustering_method\": \"hdbscan\",
      \"hdbscan_parameters\": {\"min_cluster_size\": 5}
    },
    \"llm_labeling\": {
      \"enabled\": true,
      \"input_mappings\": [{
        \"source\": \"payload\",
        \"fields\": [\"detections\"]
      }]
    },
    \"dimension_reduction\": {\"method\": \"umap\", \"n_components\": 2}
  }"
```

Clusters reveal groups like "delivery trucks," "bicycles," or "strollers" — objects the base YOLO model might lump together or miss entirely.

**1. Review cluster labels**

The LLM-generated name gives you a starting point. Review the cluster members to confirm the grouping makes sense.

**2. Promote to taxonomy node**

The cluster becomes a reference for auto-classification. Future detections matching this cluster auto-label.

**3. Add to training data**

Confirmed cluster members become training samples for the next YOLO fine-tune — new classes discovered from your own data.

***

## 10. Automate the Loop with Webhooks

Wire up webhooks so the pipeline runs without manual intervention. Each annotation event can trigger downstream processing.

```bash theme={null}
curl -X POST "$MP_API_URL/v1/organizations/webhooks" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_name": "detection-review-events",
    "event_types": [
      "annotation.created",
      "annotation.updated",
      "collection.documents.written"
    ],
    "channels": [
      { "channel": "webhook",
        "configs": { "url": "https://your-app.com/webhooks/detections" } }
    ]
  }'
```

### Automation Patterns

| Event | Trigger | Action |
| - | - | - |
| `annotation.created` | Label is `confirmed` or `corrected` | Add to reference collection, append to training dataset |
| `annotation.created` | Label is `false_positive` | Log as hard negative for next training run |
| Annotation count | Crosses threshold (e.g., 500 new corrections) | Trigger fine-tuning job, export training data |
| `collection.documents.written` | New extractor version finishes processing | Run evaluation comparing v1 vs. v2 detection quality |

***

## The Compounding Flywheel

Each Mixpeek primitive contributes to a system that gets better with use:

> **Custom Extractor**
>
> Runs YOLO, produces detections with stable feature URIs. Versioned — v1 and v2 coexist.

> **Annotations**
>
> Captures human corrections — the highest-quality training signal. Bulk API for review queues.

> **Model Registry**
>
> Stores fine-tuned weights. Upload, deploy, version — without repackaging the extractor.

> **Taxonomies**
>
> Auto-classifies detections against curated ground truth. Retroactive mode backfills old data.

> **Clusters**
>
> Discovers object categories you haven't labeled yet. Promote stable clusters to taxonomy nodes.

> **Webhooks**
>
> Triggers downstream actions on every annotation event. No polling required.

> [!TIP]
>
> The key insight is that these primitives **compose**. Annotations curate the edges where the model was wrong. Those curated edges become training data *and* reference collection entries. The reference collection powers taxonomy auto-classification. Clusters discover what you haven't labeled yet. And every improvement backfills via retroactive taxonomy application — old data re-benefits from every new correction.

## Next Steps

> **[Custom Extractors](/docs/processing/custom-extractors)**
>
> Full guide to packaging and deploying custom feature extractors.

> **[Model Registry](/docs/processing/model-registry)**
>
> Upload fine-tuned weights, manage versions, and deploy to the inference cluster.

> **[Taxonomies](/docs/enrichment/taxonomies)**
>
> Build flat and hierarchical classification systems with retroactive reapplication.

> **[Clusters](/docs/enrichment/clusters)**
>
> Discover structure in your data with 8 algorithms and LLM-powered labeling.


# Auto-Labeling Datasets
Source: https://docs.mixpeek.com/docs/tutorials/bootstrap-labeled-dataset

Build a self-improving classification system using taxonomy auto-labeling

**Build a labeled dataset from scratch and auto-classify new data using taxonomy-based matching.**

> [!TIP]
>
> Auto-labeling uses the warehouse's enrichment layer (taxonomies) to classify documents at query time, the multimodal equivalent of a SQL JOIN.

This tutorial shows how to:

1. Start with unlabeled data
2. Use feature extraction to find relevant items
3. Manually label a small reference set
4. Automatically classify new items based on the reference set
5. Create a self-improving system that gets better over time

![Bootstrap Labeled Dataset Workflow](https://mintcdn.com/mixpeek/TmiAqiYj-LwmWL2a/assets/tutorials/bootstrap-labeled-dataset.svg?fit=max\&auto=format\&n=TmiAqiYj-LwmWL2a\&q=85\&s=0bb8096d894c78c46141b27f8538e8a9)

***

## Overview

This tutorial demonstrates two approaches to building an auto-labeling system:

* **Option A: Unified Approach** (Recommended) - Single bucket/collection that grows smarter over time
* **Option B: Separate Approach** - Dedicated reference set with production data separated

Both approaches follow the same core workflow:

1. Upload unlabeled data with feature extraction
2. Manually label a small reference set (10-20 examples per category)
3. Configure taxonomy to auto-label new items based on similarity
4. Review and label unknowns to continuously improve

***

## Use Cases

* **Product Recognition**: Label product images, auto-tag new inventory
* **People Identification**: Build a face recognition system from photos
* **Document Classification**: Categorize documents by type or topic
* **Object Detection**: Label objects in images for training data

***

## Option A: Unified Approach (Recommended)

The unified approach uses a single bucket and collection that references itself. As you label items, they immediately become part of the reference set for future matches.

### Step 1: Create Bucket and Collection

Create a bucket and collection with self-referencing taxonomy:

```bash theme={null}
# Create bucket
POST /v1/buckets
{
  "bucket_name": "products_unified",
  "bucket_schema": {
    "properties": {
      "product_label": { "type": "text" },
      "image_url": { "type": "text" }
    }
  }
}

# Create retriever (do this first, before collection)
POST /v1/retrievers
{
  "retriever_name": "products_unified_classifier",
  "collection_identifiers": ["products_unified"],
  "stages": [
    {
      "stage_name": "labeled_filter",
      "stage_type": "filter",
      "config": {
        "stage_id": "attribute_filter",
        "parameters": {
          "conditions": {
            "AND": [
              { "field": "product_label", "operator": "exists", "value": true }
            ]
          }
        }
      }
    },
    {
      "stage_name": "image_match",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
              "query": { "input_mode": "content", "value": "{{INPUT.query_image}}" },
              "top_k": 1,
              "min_score": 0.30
            }
          ]
        }
      }
    }
  ]
}

# Create collection that references itself
POST /v1/collections
{
  "collection_name": "products_unified",
  "source": {
    "type": "bucket",
    "bucket_ids": ["bkt_products_unified"]
  },
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": { "image": "image_url" },
    "field_passthrough": ["product_label"]
  },
  "retriever_enrichments": [
    {
      "retriever_id": "ret_products_unified_classifier",
      "input_mappings": [
        { "input_key": "query_image",
          "source": { "source_type": "document_field", "path": "image_url" } }
      ],
      "write_back_fields": [
        { "source_field": "product_label", "target_field": "product_label" }
      ]
    }
  ]
}
```

> [!NOTE]
>
> `input_key` must match a key in the retriever's `input_schema`, and
> `source_field` is the field copied off the matched labeled document. The
> 0.30 score floor lives on the retriever's `feature_search` stage as
> `min_score`, above, so the enrichment carries no threshold of its own.
>
> `retriever_enrichments` is an array, and each entry requires `retriever_id`,
> `input_mappings` and `write_back_fields`. `source_type` accepts
> `document_field`, `constant` or `source_blob`.

### Step 2: Upload Initial Unlabeled Data

```bash theme={null}
POST /v1/buckets/{bucket_id}/objects
{
  "key_prefix": "/bootstrap",
  "product_label": null,
  "blobs": [{
    "property": "image_url",
    "type": "image",
    "data": {
      "url": "s3://my-bucket/products/shoe-001.jpg"
    }
  }]
}
```

Upload 50-100 images. Feature extraction happens automatically, but no auto-labeling occurs yet (no labeled examples to match against).

### Step 3: Manually Label Reference Set

Query documents and label them:

```bash theme={null}
# Get documents
GET /v1/collections/{collection_id}/documents?return_presigned_urls=true

# Label via bucket (syncs to collection automatically)
PATCH /v1/buckets/{bucket_id}/objects/{object_id}
{
  "product_label": "Red Running Shoes"
}
```

**Labeling tips:**

* Label 10-20 examples per category minimum
* Include diverse examples (angles, lighting, backgrounds)
* Use consistent naming conventions

### Step 4: Upload New Items - Auto-Labeling Works!

Now that you have labeled examples, new uploads auto-label automatically:

```bash theme={null}
POST /v1/buckets/{bucket_id}/objects
{
  "key_prefix": "/new-arrivals",
  "blobs": [{
    "property": "image_url",
    "type": "image",
    "data": {
      "url": "s3://my-bucket/new-arrivals/shoe-new.jpg"
    }
  }]
}
```

**What happens automatically:**

1. Feature extraction runs on the new image
2. Taxonomy searches your labeled items for similar matches
3. If similarity > 0.30 → Auto-labels (e.g., `"Red Running Shoes"`)
4. If similarity \< 0.30 → Leaves as `null` for manual review

**Check the result:**

```bash theme={null}
GET /v1/collections/{collection_id}/documents/{document_id}
```

**Matched:**

```json theme={null}
{
  "metadata": {
    "product_label": "Red Running Shoes"
  },
  "taxonomy_match": {
    "matched": true,
    "confidence": 0.87,
    "source_document_id": "doc_xyz123"
  }
}
```

**Unknown (needs manual review):**

```json theme={null}
{
  "metadata": {
    "product_label": null
  },
  "taxonomy_match": {
    "matched": false,
    "confidence": 0.21
  }
}
```

### Step 5: Review and Label Unknowns

Find items that need manual labeling:

```bash theme={null}
GET /v1/collections/{collection_id}/documents?filters={
  "must": [
    {
      "key": "product_label",
      "match": { "operator": "eq", "value": null }
    }
  ]
}
```

Label them via bucket (automatically syncs to collection):

```bash theme={null}
PATCH /v1/buckets/{bucket_id}/objects/{object_id}
{
  "metadata": {
    "product_label": "Blue Basketball Shoes"
  }
}
```

**Self-improvement in action**: This newly labeled item becomes part of the reference set for future uploads!

***

## Option B: Separate Approach

For more control, keep reference data separate from production data:

* **Reference bucket/collection**: Curated, high-quality labeled examples
* **Production bucket/collection**: All data with auto-labels

**When to use:**

* Need strict quality control on reference set
* Want to prevent noisy auto-labels from affecting matching
* Prefer to manually review before promoting items to reference

### Step 1: Create Reference Bucket and Collection

```bash theme={null}
# Reference bucket
POST /v1/buckets
{
  "bucket_name": "product_reference",
  "bucket_schema": {
    "properties": {
      "product_label": { "type": "text" },
      "image_url": { "type": "text" }
    }
  }
}

# Reference collection (no taxonomy needed)
POST /v1/collections
{
  "collection_name": "product_reference",
  "source": {
    "type": "bucket",
    "bucket_ids": ["bkt_product_reference"]
  },
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": { "image": "image_url" },
    "field_passthrough": ["product_label"]
  }
}

# Create taxonomy retriever
POST /v1/retrievers
{
  "retriever_name": "product_classifier",
  "collection_identifiers": ["product_reference"],
  "stages": [
    {
      "stage_name": "labeled_filter",
      "stage_type": "filter",
      "config": {
        "stage_id": "attribute_filter",
        "parameters": {
          "conditions": {
            "AND": [
              { "field": "product_label", "operator": "exists", "value": true }
            ]
          }
        }
      }
    },
    {
      "stage_name": "image_match",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
              "query": { "input_mode": "content", "value": "{{INPUT.query_image}}" },
              "top_k": 1,
              "min_score": 0.30
            }
          ]
        }
      }
    }
  ]
}
```

### Step 2: Upload and Label Reference Set

Upload 50-100 curated images to the reference bucket and manually label them:

```bash theme={null}
# Upload to reference
POST /v1/buckets/bkt_product_reference/objects
{
  "blobs": [{ "property": "image_url", "type": "image", "data": "https://example.com/image.jpg" }]
}

# Label them
PATCH /v1/buckets/bkt_product_reference/objects/{object_id}
{
  "metadata": { "product_label": "Red Running Shoes" }
}
```

### Step 3: Create Production Bucket and Collection

```bash theme={null}
# Production bucket
POST /v1/buckets
{
  "bucket_name": "product_catalog",
  "bucket_schema": {
    "properties": {
      "product_label": { "type": "text" },
      "image_url": { "type": "text" }
    }
  }
}

# Production collection with taxonomy
POST /v1/collections
{
  "collection_name": "product_catalog",
  "source": {
    "type": "bucket",
    "bucket_ids": ["bkt_product_catalog"]
  },
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": { "image": "image_url" },
    "field_passthrough": ["product_label"]
  },
  "retriever_enrichments": [
    {
      "retriever_id": "ret_product_classifier",
      "input_mappings": [
        { "input_key": "query_image",
          "source": { "source_type": "document_field", "path": "image_url" } }
      ],
      "write_back_fields": [
        { "source_field": "product_label", "target_field": "product_label" }
      ]
    }
  ]
}
```

### Step 4: Upload Production Data

New uploads auto-label based on the reference set:

```bash theme={null}
POST /v1/buckets/bkt_product_catalog/objects
{
  "blobs": [{ "property": "image_url", "type": "image", "data": "https://example.com/image.jpg" }]
}
```

### Step 5: Promote High-Confidence Items to Reference

Periodically review production data and promote high-confidence matches:

```bash theme={null}
# Find high-confidence items
GET /v1/collections/product_catalog/documents?filters={
  "must": [
    {
      "key": "taxonomy_match.confidence",
      "match": { "operator": "gte", "value": 0.85 }
    }
  ]
}

# Copy to reference bucket
POST /v1/buckets/bkt_product_reference/objects
{
  "metadata": { "product_label": "..." },
  "blobs": [{ ... }]
}
```

***

## Real-World Examples

### Example 1: Face Recognition System

```bash theme={null}
# Create bucket for employee photos
POST /v1/buckets
{
  "bucket_name": "employee_photos",
  "bucket_schema": {
    "properties": {
      "person_name": { "type": "text" },
      "employee_id": { "type": "text" },
      "photo_url": { "type": "text" }
    }
  }
}

# Bootstrap collection with face extraction
POST /v1/collections
{
  "collection_name": "employee_faces",
  "source": {
    "type": "bucket",
    "bucket_ids": ["bkt_employee_photos"]
  },
  "feature_extractor": {
    "feature_extractor_name": "face_identity_extractor",
    "version": "v1",
    "input_mappings": { "image": "photo_url" },
    "field_passthrough": ["person_name", "employee_id"]
  }
}

# Upload 50 employee photos → manually label with names
# Create taxonomy retriever
# Security camera footage auto-identifies employees
```

### Example 2: Document Classification

```bash theme={null}
# Create bucket for documents
POST /v1/buckets
{
  "bucket_name": "company_documents",
  "bucket_schema": {
    "properties": {
      "document_type": { "type": "text" },
      "content": { "type": "text" }
    }
  }
}

# Bootstrap collection with text extraction
POST /v1/collections
{
  "collection_name": "document_types",
  "source": {
    "type": "bucket",
    "bucket_ids": ["bkt_company_documents"]
  },
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": { "text": "content" },
    "field_passthrough": ["document_type"]
  },
  "retriever_enrichments": [
    {
      "retriever_id": "ret_document_type_classifier",
      "input_mappings": [
        { "input_key": "query_text",
          "source": { "source_type": "document_field", "path": "content" } }
      ],
      "write_back_fields": [
        { "source_field": "document_type", "target_field": "document_type" }
      ]
    }
  ]
}

# Label 20 invoices, 20 contracts, 20 receipts
# New documents auto-classify by type
```

***

## Advanced Configuration

### Tuning Confidence Thresholds

The `confidence_threshold` determines how conservative auto-labeling is:

| Threshold | Behavior | Use Case |
| - | - | - |
| `0.20-0.25` | Aggressive | High recall, more false positives |
| `0.30-0.35` | Balanced | Good starting point |
| `0.40-0.50` | Conservative | High precision, fewer auto-labels |
| `0.60+` | Very strict | Only exact matches |

**Finding the right threshold:**

1. Start with `0.30`
2. Monitor false positive rate (wrong auto-labels)
3. Check coverage (% of items auto-labeled)
4. Adjust based on cost of errors:
   * **High cost of errors** (e.g., medical imaging) → Higher threshold
   * **Low cost of errors** (e.g., photo organization) → Lower threshold

### Monitoring & Analytics

Track performance with these queries:

```bash theme={null}
# Get distribution of labels
GET /v1/collections/{collection_id}/analytics/field-distribution?field=product_label

# Check match confidence distribution
GET /v1/collections/{collection_id}/documents?sort_by=taxonomy_match.confidence&limit=100

# Find low-confidence matches for review
GET /v1/collections/{collection_id}/documents?filters={
  "must": [
    {
      "key": "taxonomy_match.matched",
      "match": { "operator": "eq", "value": true }
    },
    {
      "key": "taxonomy_match.confidence",
      "match": { "operator": "lt", "value": 0.40 }
    }
  ]
}
```

**Key metrics:**

* **Auto-label coverage**: % of new items auto-labeled
* **Manual review queue**: # of items with `label: null`
* **Confidence distribution**: Are matches clustered around threshold?
* **False positive rate**: Sample and manually verify auto-labels

### Best Practices

**Reference set quality:**

* Include diverse examples (angles, lighting, backgrounds)
* Use consistent naming conventions
* Aim for balanced distribution across categories
* Maintain high-quality, unambiguous images

**Labeling guidelines:**

* Create a labeling style guide
* Consider hierarchical labels: `"Shoes > Running > Red"`
* Define rules for edge cases
* Version your taxonomy as it evolves

**Continuous improvement:**

* Review unknowns regularly
* Audit auto-labels periodically
* Add corrected examples when system makes mistakes
* Expand categories as needed

**Production deployment:**

* Start with conservative threshold (0.40+)
* Implement human-in-the-loop for critical applications
* Enable feedback mechanism for corrections
* A/B test threshold changes

***

## Troubleshooting

### Too many unlabeled items

**Causes**: Threshold too high, insufficient reference examples, new categories

**Solutions**:

* Lower `confidence_threshold` to 0.25-0.30
* Add 20+ examples per category to reference set
* Review and label new categories

### False positives (wrong labels)

**Causes**: Threshold too low, similar categories, poor quality references

**Solutions**:

* Raise `confidence_threshold` to 0.40+
* Add diverse examples to distinguish categories
* Clean up reference set

### System not self-improving

**Causes**: Labels not syncing, configuration issues

**Solutions**:

* Verify `field_passthrough` includes label field
* Check retriever filters for non-null labels
* Confirm bucket-to-collection sync is working

***

## Summary

**Workflow:**

1. Create bucket and collection with feature extraction
2. Upload unlabeled data (50-100 items)
3. Manually label reference set (10-20 per category)
4. Create taxonomy retriever pointing to labeled items
5. New uploads auto-label based on similarity
6. Review and label unknowns to improve system

**Key benefits:**

* Start with zero labels, build incrementally
* Automate repetitive labeling
* Self-improving with each manual correction
* Scales from dozens to millions

**Next steps:**

* Choose unified (simpler) or separate (more control) approach
* Start with 50-100 reference items
* Test different confidence thresholds (start at 0.30)
* Monitor auto-label quality and adjust

***

## Discover Clusters

Use clustering to find new categories before defining them manually:

```bash theme={null}
POST /v1/clusters
{
  "cluster_name": "product-discovery",
  "collection_ids": ["col_products_unified"],
  "cluster_type": "vector",
  "vector_config": {
    "feature_uris": ["mixpeek://image_extractor@v1/google_siglip_base_v1"],
    "clustering_method": "hdbscan",
    "algorithm_params": { "min_cluster_size": 5 },
    "preprocessing_steps": [
      { "method": "umap", "n_components": 2 }
    ]
  },
  "llm_labeling": {
    "enabled": true,
    "input_mappings": [{ "source": "payload", "fields": ["product_label"] }]
  }
}
```

Clusters reveal groups you haven't labeled yet — "sandals", "boots", "athletic wear" — without predefined categories. Once a cluster stabilizes, promote it to a taxonomy node so future items auto-classify into it. See [Clusters](/docs/enrichment/clusters).

## Set Up Alerts

Get notified when items fail to auto-label (unknown categories needing manual review). An alert **runs a retriever and fires on its results** — so first create a retriever that surfaces unlabeled items, then point an alert at it.

```bash theme={null}
# 1. Retriever that returns products with no label. attribute_filter as the
#    first stage fetches + filters straight from the collection (no search needed).
POST /v1/retrievers
{
  "retriever_name": "unlabeled-products",
  "collection_identifiers": ["col_products_unified"],
  "input_schema": {},
  "stages": [
    { "stage_name": "missing_label", "stage_type": "filter",
      "config": { "stage_id": "attribute_filter", "parameters": {
        "field": "product_label", "operator": "exists", "value": false } } }
  ]
}
# -> { "retriever_id": "ret_unlabeled" }

# 2. Fire a webhook whenever that retriever returns any results
POST /v1/alerts
{
  "name": "unknown-products",
  "source": "retriever",
  "retriever_id": "ret_unlabeled",
  "trigger_on": "results",
  "notification_config": {
    "channels": [
      { "channel_type": "webhook", "config": { "url": "https://example.com/webhook" } }
    ]
  }
}
```

See [Alerts](/docs/enrichment/alerts) for system-metric alerts and Slack/email channels.

## Set Up Webhooks

Forward ingestion and labeling events to your own systems:

```bash theme={null}
POST /v1/organizations/webhooks
{
  "webhook_name": "labeling-events",
  "event_types": ["object.created", "object.updated"],
  "channels": [
    { "channel": "webhook", "configs": { "url": "https://example.com/webhook", "method": "POST" } }
  ]
}
```

`event_types` accepts object/collection/cluster/taxonomy/alert lifecycle events (e.g. `object.created`, `object.updated`, `object.deleted`, `collection.created`). See [Webhooks](/docs/operations/webhooks) for the full event list and Slack/email channels.


# Visual Document Retrieval
Source: https://docs.mixpeek.com/docs/tutorials/colpali-visual-document-retrieval

Search PDFs, scanned pages, and figure-heavy reports by visual content using cross-modal embeddings — no OCR required

> [!TIP]
>
> Visual document retrieval treats each page as an image and embeds it with a cross-modal model, so a text query can find the right page by what it *looks like* — tables, charts, diagrams, scans — without relying on OCR.

## Why Visual Document Retrieval

Traditional document search pipelines drop information at every stage:

1. **OCR** mangles tables, equations, and low-contrast scans.
2. **Layout parsers** miss chart and diagram semantics.
3. **Text-only embeddings** never see the visual structure of the page.

Mixpeek's `multimodal_extractor` embeds page images directly with Google's Vertex multimodal model into a shared text-image space (`vertex_multimodal_embedding`). Because the space is cross-modal, a **text query retrieves visually-relevant pages** — the words "revenue breakdown by region" can match a page dominated by a financial table, even with no clean extractable text.

> [!NOTE]
>
> This is single-vector cross-modal retrieval (one embedding per page). Mixpeek does not currently offer ColPali-style multi-vector *late-interaction* (per-patch MaxSim) scoring. For born-digital, text-heavy PDFs where you want extracted text + OCR, use the [universal extractor](/docs/processing/extractors/universal) instead (see [Document Intelligence](/docs/tutorials/document-intelligence)).

## 1. Create a bucket

Hold the page images. Render each PDF page to an image (PNG/JPG) and store the page image URL — the cross-modal model embeds images.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "visual-documents",
    "bucket_schema": {
      "properties": {
        "page_image": { "type": "image" },
        "document_title": { "type": "text" },
        "doc_type": { "type": "string" },
        "page_number": { "type": "integer" }
      }
    }
  }'
```

> [!TIP]
>
> Render PDF pages to images client-side (e.g. `pdftoppm`, `pdf2image`) — one object per page — so each page becomes an independently retrievable result with its own `page_number`.

## 2. Create a collection

Embed each page image with `multimodal_extractor`. Map the extractor's `image` input to your `page_image` field.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "visual-doc-index",
    "source": { "type": "bucket", "bucket_ids": ["bkt_visual_documents"] },
    "feature_extractor": {
      "feature_extractor_name": "multimodal_extractor",
      "version": "v1",
      "input_mappings": { "image": "page_image" },
      "field_passthrough": [
        { "source_path": "document_title" },
        { "source_path": "doc_type" },
        { "source_path": "page_number" }
      ]
    }
  }'
```

## 3. Ingest pages

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_visual_documents/objects" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "blobs": [
      { "property": "page_image", "type": "image", "data": "s3://my-bucket/filings/acme-10k-2024/page-042.png" },
      { "property": "document_title", "type": "text", "data": "Acme Corp 2024 10-K" },
      { "property": "doc_type", "type": "text", "data": "financial" },
      { "property": "page_number", "type": "text", "data": "42" }
    ]
  }'
```

## 4. Process

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_visual_documents/batches" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "object_ids": ["obj_001"] }'

curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_visual_documents/batches/{batch_id}/submit" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

## 5. Create a retriever

Search the cross-modal page embeddings with a text query.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "visual-doc-retriever",
    "collection_identifiers": ["visual-doc-index"],
    "input_schema": { "query": { "type": "text", "required": true } },
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 50
              }
            ],
            "final_top_k": 10
          }
        }
      }
    ]
  }'
```

## 6. Query

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": { "query": "segment revenue breakdown by region with year-over-year growth" },
    "filters": { "field": "doc_type", "operator": "eq", "value": "financial" }
  }'
```

Each result is a page, ranked by cross-modal similarity, carrying its `document_title` and `page_number` so you can deep-link to the exact page.

## When to use visual vs text retrieval

> **Use visual retrieval when**
>
> * Pages are visually complex (tables, charts, infographics, equations)
> * OCR quality is unreliable (scans, handwriting, multi-column layouts)
> * Figures and diagrams carry meaning text alone cannot capture

> **Use text retrieval when**
>
> * Documents are born-digital and text-only (use [universal extractor](/docs/processing/extractors/universal))
> * You need extracted text, NER, or summaries (see [Document Intelligence](/docs/tutorials/document-intelligence))
> * You want exact-keyword matching (add a [lexical/BM25 search](/docs/retrieval/stages/feature-search#lexical-bm25-search))

## Next steps

> **[Classify by layout](/docs/enrichment/taxonomies)**
>
> Auto-classify pages (financial report, slide deck, research paper) with a taxonomy.

> **[Discover layout patterns](/docs/enrichment/clusters)**
>
> Cluster page embeddings to surface visual document patterns.

> **[Combine with text](/docs/tutorials/document-intelligence)**
>
> Pair visual retrieval with extracted-text search for hybrid document QA.

> **[Get notified](/docs/enrichment/alerts)**
>
> Alert when new documents of a given type are indexed.

## Further reading

* [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) — the late-interaction approach that inspired visual document retrieval
* [ViDoRe Benchmark](https://huggingface.co/spaces/vidore/vidore-leaderboard) — visual document retrieval leaderboard


# Custom Extractor Quickstart
Source: https://docs.mixpeek.com/docs/tutorials/custom-extractor-quickstart

Build, test, and query a custom text embedding extractor

> [!TIP]
>
> Custom extractors extend the warehouse's Decompose layer with your own feature extraction logic. For the full reference, see [Custom Extractors](/docs/processing/custom-extractors).

## What You'll Build

A custom text embedding extractor that:

1. Generates 128-dimensional embeddings from text (batch + real-time)
2. Validates locally with `lint` and `test` — no API key needed
3. Powers search through a retriever

> [!NOTE]
>
> **Where this runs.** Authoring + `lint` + `test` work everywhere. The **upload/deploy** steps require a [dedicated deployment](/docs/processing/custom-extractors#availability) — the shared `api.mixpeek.com` API does not expose custom-extractor uploads. On the shared API, swap in a built-in extractor (e.g. `text_extractor`) at Step 4 and skip Step 3, or ship your extractor via [Submissions](/docs/processing/extractor-marketplace). The ingest + search steps (4–5) run on any plan.

Set these first:

```bash theme={null}
export MIXPEEK_API_KEY="mxp_sk_..."
export MIXPEEK_NAMESPACE="ns_..."
export MIXPEEK_API_URL="https://api.mixpeek.com"   # dedicated deployments use your tenant URL
```

> [!NOTE]
>
> These are the same three variables the `plugins.py` CLI reads, so the CLI and the raw `curl` examples below use one consistent convention.

## Step 1: Create Extractor Files

Create a directory `text_embed/` with three files.

### manifest.py

```python theme={null}
feature_extractor_name = "text_embed"
version = "1.0.0"
description = "Text embedding extractor"

dependencies = []

features = [
    {
        "feature_type": "embedding",
        "feature_name": "text_embed_v1_embedding",
        "embedding_dim": 128,
        "distance_metric": "cosine",
    }
]

# Declares real-time inference capability (query embedding for feature_search)
inference_type = "embedding"

# Skip GPU — this extractor is CPU-only
compute_profile = {"resource_type": "cpu"}
```

> [!WARNING]
>
> Use the exact key names: `feature_type`, `feature_name`, `embedding_dim`, `distance_metric`. Using `name`/`type`/`dimensions`/`distance` will silently produce a collection with no vector indexes.

### pipeline.py

```python theme={null}
import hashlib
from typing import List

import numpy as np
import pandas as pd


def text_to_embedding(text: str, dim: int = 128) -> List[float]:
    """Generate a deterministic, L2-normalized embedding from text."""
    hash_bytes = hashlib.sha256(text.encode("utf-8")).digest()
    seed = int.from_bytes(hash_bytes[:4], byteorder="big")
    rng = np.random.default_rng(seed)
    embedding = rng.standard_normal(dim).astype(np.float32)
    norm = np.linalg.norm(embedding)
    if norm > 0:
        embedding = embedding / norm
    return embedding.tolist()


class TextEmbedBatchProcessor:
    def __init__(self, config=None, **kwargs):
        # Custom extractors receive blob content in the 'data' column
        self.text_column = "data"
        self.output_column = "text_embed_v1_embedding"
        self.embedding_dim = 128

    def __call__(self, batch: pd.DataFrame) -> pd.DataFrame:
        if batch.empty:
            return batch
        batch = batch.reset_index(drop=True)

        texts, valid = [], []
        for idx, v in enumerate(batch.get(self.text_column, [])):
            text = "" if v is None else str(v)
            if text.strip():
                texts.append(text)
                valid.append(idx)

        batch[self.output_column] = None
        if texts:
            embs = [text_to_embedding(t, self.embedding_dim) for t in texts]
            for i, orig_idx in enumerate(valid):
                batch.at[orig_idx, self.output_column] = embs[i]
        return batch


def build_steps(extractor_request=None, container=None,
                base_steps=None, dataset_size=None, content_flags=None):
    steps = list(base_steps or [])
    steps.append(TextEmbedBatchProcessor())
    return {"steps": steps, "prepare": lambda ds: ds}
```

> [!WARNING]
>
> Always read from `batch["data"]` — NOT `batch["text"]`. The `data` column holds raw text for text blobs and S3 URLs for binary blobs. The local `test` harness feeds the same `data` column, so a passing `test` matches production.

### realtime.py

Embeds the query at search time so it lands in the same space as your indexed vectors.

```python theme={null}
import hashlib
import numpy as np


class InferenceService:
    async def run_inference(self, inputs: dict, parameters: dict) -> dict:
        text = inputs.get("text", "")
        h = hashlib.sha256(text.encode("utf-8")).digest()
        rng = np.random.default_rng(int.from_bytes(h[:4], "big"))
        v = rng.standard_normal(128).astype(np.float32)
        n = np.linalg.norm(v)
        return {"embedding": (v / n if n else v).tolist()}
```

## Step 2: Validate Locally (no API key)

```bash theme={null}
python plugins.py lint text_embed
python plugins.py test text_embed
```

`lint` validates your manifest + runs the security scanner. `test` runs the pipeline through real Ray Data `map_batches` and confirms your output column is populated. Both are offline.

## Step 3: Deploy (dedicated infrastructure)

> [!NOTE]
>
> Upload/deploy is only available on a [dedicated deployment](/docs/processing/custom-extractors#availability) — full HTTP contract in the [Custom Extractor API](/docs/processing/custom-extractor-api) reference. On the shared API, skip to Step 4 with a built-in extractor, or use [Submissions](/docs/processing/extractor-marketplace).

The CLI does this for you (`plugins.py push` then `deploy`). Raw HTTP (custom extractors are addressed as `/plugins` on this surface):

```bash theme={null}
zip -r text_embed.zip text_embed/

UPLOAD=$(curl -s -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/plugins/uploads" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "Content-Type: application/json" \
  -d '{"name":"text_embed","version":"1.0.0","file_size_bytes":5000}')
UPLOAD_ID=$(echo "$UPLOAD" | jq -r '.upload_id')
PRESIGNED_URL=$(echo "$UPLOAD" | jq -r '.presigned_url')

curl -s -X PUT "$PRESIGNED_URL" -H "Content-Type: application/zip" --data-binary @text_embed.zip
curl -s -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/plugins/uploads/$UPLOAD_ID/confirm" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" -H "Content-Type: application/json" -d '{}'
curl -s -X POST "$MIXPEEK_API_URL/v1/namespaces/$MIXPEEK_NAMESPACE/plugins/text_embed_1_0_0/deploy" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
```

## Step 4: Ingest Data

> [!NOTE]
>
> Buckets, collections, retrievers, and batches are **top-level** resources keyed by the **`X-Namespace` header** — not nested under `/namespaces/{ns}/`.

```bash theme={null}
H=(-H "Authorization: Bearer $MIXPEEK_API_KEY" -H "X-Namespace: $MIXPEEK_NAMESPACE" -H "Content-Type: application/json")
B="https://api.mixpeek.com/v1"

# Bucket
BUCKET_ID=$(curl -s -X POST "$B/buckets" "${H[@]}" \
  -d '{"bucket_name":"articles","bucket_schema":{"properties":{"text":{"type":"text"}}}}' | jq -r '.bucket_id')

# Object
curl -s -X POST "$B/buckets/$BUCKET_ID/objects" "${H[@]}" \
  -d '{"blobs":[{"property":"text","type":"text","data":"Quantum computing uses qubits to perform calculations exponentially faster than classical computers."}]}'

# Collection — use "text_embed" if deployed, or a built-in like "text_extractor" on the shared API
COLLECTION_ID=$(curl -s -X POST "$B/collections" "${H[@]}" -d "{
  \"collection_name\":\"text_embed_articles\",
  \"source\":{\"type\":\"bucket\",\"bucket_ids\":[\"$BUCKET_ID\"]},
  \"feature_extractor\":{\"feature_extractor_name\":\"text_embed\",\"version\":\"1.0.0\",\"input_mappings\":{\"text\":\"text\"}}
}" | jq -r '.collection_id')

# Batch is two-step: create (objects + collections) then submit
OBJIDS=$(curl -s "$B/buckets/$BUCKET_ID/objects" "${H[@]}" | jq -c '[.results[].object_id]')
BATCH_ID=$(curl -s -X POST "$B/buckets/$BUCKET_ID/batches" "${H[@]}" \
  -d "{\"batch_name\":\"run-1\",\"object_ids\":$OBJIDS,\"collection_ids\":[\"$COLLECTION_ID\"]}" | jq -r '.batch_id')
curl -s -X POST "$B/buckets/$BUCKET_ID/batches/$BATCH_ID/submit" "${H[@]}" \
  -d "{\"collection_ids\":[\"$COLLECTION_ID\"]}"

# Poll until COMPLETED
while true; do
  S=$(curl -s "$B/buckets/$BUCKET_ID/batches/$BATCH_ID" "${H[@]}" | jq -r '.status')
  echo "batch: $S"; case "$S" in COMPLETED|COMPLETED_WITH_ERRORS|FAILED|CANCELED) break;; esac; sleep 8
done
```

## Step 5: Create a Retriever and Search

> [!WARNING]
>
> **First-run cold start.** On a cold engine the embedding model can take **several minutes** to load. The batch poll in Step 4 waits for the ingest side, but the **first `execute` immediately after may return 0 results** (with `status` `completed` *or* `degraded`) while the **query-side** model warms up. This is expected on the first call only — **retry after a few seconds** and subsequent searches return ranked results. A warm namespace responds immediately.

```bash theme={null}
RETRIEVER_ID=$(curl -s -X POST "$B/retrievers" "${H[@]}" -d "{
  \"retriever_name\":\"text_search\",
  \"collection_identifiers\":[\"$COLLECTION_ID\"],
  \"input_schema\":{\"query\":{\"type\":\"text\",\"required\":true}},
  \"stages\":[{\"stage_name\":\"semantic\",\"stage_type\":\"filter\",\"config\":{\"stage_id\":\"feature_search\",\"parameters\":{
    \"searches\":[{\"feature_uri\":\"mixpeek://text_embed@1.0.0/text_embed_v1_embedding\",\"query\":{\"input_mode\":\"text\",\"value\":\"{{INPUT.query}}\"},\"top_k\":10}],
    \"final_top_k\":10,\"fusion\":\"rrf\"}}}]
}" | jq -r '.retriever_id')

curl -s -X POST "$B/retrievers/$RETRIEVER_ID/execute" "${H[@]}" \
  -d '{"inputs":{"query":"quantum computing"}}' | jq '.documents[:3]'
```

> [!NOTE]
>
> The execute body is `{"inputs": {"query": "..."}}`. The `feature_uri` must match your extractor + feature name (for a built-in, fetch it from `GET /v1/namespaces/$MIXPEEK_NAMESPACE/extractors/{extractor_id}`).

## Next Steps

* [Custom Extractors](/docs/processing/custom-extractors) — full manifest, SDK, security, and platform-services reference
* [Model Registry](/docs/processing/model-registry) — load HuggingFace or your own fine-tuned weights
* [Taxonomies](/docs/enrichment/taxonomies) and [Clusters](/docs/enrichment/clusters) — auto-classify and group your embeddings
* [Alerts](/docs/enrichment/alerts) and [Webhooks](/docs/operations/webhooks) — monitor new content and processing events


# Document Intelligence
Source: https://docs.mixpeek.com/docs/tutorials/document-intelligence

Extract and query data from PDFs, images, and scanned documents

> [!TIP]
>
> Document intelligence uses the warehouse's decompose layer to parse text and layout from PDFs and scanned documents, then makes them queryable through retrieval. Every code block below uses the real API field names.

![Document Intelligence Pipeline](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-document-intelligence.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=a05e2d2077fb485096559763e0f87b7d)

## How It Works

When you ingest a document, the `universal_extractor` runs a multi-stage pipeline:

1. **Content extraction** — text is parsed from native PDFs, with OCR fallback for scanned pages.
2. **Chunking** — documents are split into searchable segments.
3. **Embedding** — each document is embedded for semantic search.
4. **Indexing** — segments are stored with metadata for filtered vector search.

At query time, the retriever runs semantic search over the document embeddings and can filter by metadata such as document type.

## Feature Extractors

| Extractor | Use For |
| - | - |
| `universal_extractor@v1` | Parse any file (PDF, image, scanned doc), OCR fallback, and produce a Gemini embedding for semantic search |
| `text_extractor@v1` | Text embeddings, NER, and summarization over already-extracted text |

> [!NOTE]
>
> Mixpeek does not have separate `pdf_extractor` / `table_extractor` extractors. The `universal_extractor` ingests arbitrary documents (PDFs and scans) and produces a searchable embedding.

## 1. Create a bucket

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "contracts",
    "bucket_schema": {
      "properties": {
        "document_url": { "type": "pdf" },
        "document_type": { "type": "string" },
        "contract_date": { "type": "datetime" }
      }
    }
  }'
```

## 2. Create a collection

Use `universal_extractor` to parse each document and produce a searchable embedding. Map the extractor's `content` input to your bucket's `document_url` field.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "contracts-text",
    "source": { "type": "bucket", "bucket_ids": ["bkt_contracts"] },
    "feature_extractor": {
      "feature_extractor_name": "universal_extractor",
      "version": "v1",
      "input_mappings": { "content": "document_url" },
      "field_passthrough": [
        { "source_path": "document_type" },
        { "source_path": "contract_date" }
      ]
    }
  }'
```

## 3. Ingest documents

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_contracts/objects" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "key_prefix": "/2025/agreements",
    "blobs": [
      { "property": "document_url", "type": "pdf", "data": "s3://my-bucket/contracts/vendor-001.pdf" },
      { "property": "document_type", "type": "text", "data": "vendor_agreement" }
    ]
  }'
```

## 4. Process

```bash theme={null}
# Create a batch, then submit it; poll the returned task until COMPLETED
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_contracts/batches" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "object_ids": ["obj_001", "obj_002"] }'

curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_contracts/batches/{batch_id}/submit" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

See [Monitoring ingestion](/docs/processing/tasks) for the task-polling loop.

## 5. Create a retriever

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "contract-search",
    "collection_identifiers": ["contracts-text"],
    "input_schema": {
      "query": { "type": "text", "required": true }
    },
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://universal_extractor@v1/gemini-embedding-2",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 50
              }
            ],
            "final_top_k": 20
          }
        }
      }
    ]
  }'
```

## 6. Query

Filter by document type at execution time with the `filters` field:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": { "query": "termination clauses with 30-day notice" },
    "filters": {
      "field": "document_type",
      "operator": "eq",
      "value": "vendor_agreement"
    }
  }'
```

## Multi-page assembly

Retrieve all segments from a source document using lineage:

```bash theme={null}
curl -sS "$MP_API_URL/v1/documents/{document_id}/lineage" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

## Next steps

> **[Classify documents](/docs/enrichment/taxonomies)**
>
> Auto-classify documents by type (contract, invoice, NDA) with a taxonomy.

> **[Discover themes](/docs/enrichment/clusters)**
>
> Cluster document embeddings to surface recurring contract patterns.

> **[Get notified](/docs/enrichment/alerts)**
>
> Trigger alerts when new documents match a query.

> **[Add NER & summaries](/docs/processing/extractors/text)**
>
> Pair with `text_extractor` for named-entity recognition and summaries.


# Build a Feedback Loop
Source: https://docs.mixpeek.com/docs/tutorials/feedback-loop

Start with static fusion weights, capture interaction signals, and let Thompson Sampling learn optimal weights automatically

> [!TIP]
>
> This tutorial walks through the full lifecycle: static weights → interaction capture → learned fusion → convergence monitoring. You don't need interaction data to start — the system gracefully degrades to uniform weights with zero signal.

## What You'll Build

A search retriever that starts with manually tuned weights and progressively learns the optimal blend of features from user behavior. By the end, your retriever adapts per-user (or per-segment) without manual tuning.

**Prerequisites:** A namespace with at least two collections producing different embedding types (e.g., text + multimodal). See [Semantic Search](/docs/tutorials/semantic-search) or [Video Understanding](/docs/tutorials/video-understanding) to set those up first.

## 1. Start with Static Weights

Begin with `weighted` fusion. This gives you a deterministic baseline to measure against later.

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "retriever_name": "product-search",
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": "{{INPUT.query}}",
                "top_k": 100,
                "weight": 0.6
              },
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": "{{INPUT.query}}",
                "top_k": 100,
                "weight": 0.4
              }
            ],
            "fusion": "weighted",
            "final_top_k": 25
          }
        }
      }
    ]
  }'
```

```python Python SDK theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

retriever = client.retrievers.create(
    retriever_name="product-search",
    stages=[{
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [
                    {
                        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                        "query": "{{INPUT.query}}",
                        "top_k": 100,
                        "weight": 0.6
                    },
                    {
                        "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                        "query": "{{INPUT.query}}",
                        "top_k": 100,
                        "weight": 0.4
                    }
                ],
                "fusion": "weighted",
                "final_top_k": 25
            }
        }
    }]
)
```

Run searches against this retriever and record the result quality. These static-weight results are your baseline.

## 2. Instrument Your Application with Interaction Signals

Before switching to learned fusion, you need to emit signals. Add interaction tracking wherever users engage with search results.

```javascript Client-Side (Browser) theme={null}
document.querySelectorAll('.search-result').forEach((el, index) => {
  el.addEventListener('click', () => {
    fetch(`${MP_API_URL}/v1/retrievers/interactions`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${MP_API_KEY}`,
        'X-Namespace': MP_NAMESPACE
      },
      body: JSON.stringify({
        feature_id: el.dataset.documentId,
        interaction_type: ['click'],
        position: index,
        metadata: { query: currentQuery },
        user_id: userId,
        session_id: sessionId
      })
    });
  });
});
```

```python Server-Side (Python) theme={null}
client.retrievers.create_interaction(
    feature_id="doc_product_789",
    interaction_type=["purchase"],
    position=2,
    metadata={
        "query": original_query,
        "order_value": 49.99
    },
    user_id=user_id,
    session_id=session_id
)
```

> [!TIP]
>
> **Shortcut: `create_interaction_from_result()`** — pass the full execute response and a position, and the SDK extracts `feature_id`, `execution_id`, `retriever_id`, and `feature_uri` automatically:
>
> ```python theme={null}
> results = client.retrievers.execute(retriever_id, inputs={"query": "earbuds", "user_id": "user_456"})
> # User clicked the first result
> client.retrievers.create_interaction_from_result(results, position=0, user_id="user_456")
> # User purchased the third result
> client.retrievers.create_interaction_from_result(results, position=2, interaction_type=["purchase"], user_id="user_456")
> ```

> [!WARNING]
>
> **Always include `position`.** It is recorded for analytics and evaluation — helping you understand which result positions drive engagement. Position is not currently used in reward computation, but is required for accurate NDCG and other rank-aware metrics.

**Which signals to capture depends on your domain:**

| Domain | Primary Signals | Why |
| - | - | - |
| E-commerce | `purchase`, `add_to_cart`, `click` | Conversion is the strongest relevance indicator |
| Media / Video | `long_view`, `click`, `share` | Watch completion > click for engagement |
| Enterprise Search | `positive_feedback`, `click`, `bookmark` | Clicks may be obligatory; explicit feedback is clearer |
| Content Matching | `click`, `positive_feedback`, `skip` | Editor accept/reject on matched content |

See the [Signal Strength Matrix](/docs/retrieval/interactions#signal-strength-matrix) for the full list of 17 signal types and how they're weighted.

## 3. Switch to Learned Fusion

Once interactions are flowing, update the retriever to use `learned` fusion. You can do this at any time — even with zero interactions (it falls back to uniform weights).

```bash cURL theme={null}
curl -X PATCH "$MP_API_URL/v1/retrievers/{retriever_id}" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "stages": [
      {
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": "{{INPUT.query}}",
                "top_k": 100
              },
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": "{{INPUT.query}}",
                "top_k": 100
              }
            ],
            "fusion": "learned",
            "final_top_k": 25
          }
        }
      }
    ]
  }'
```

```python Python SDK theme={null}
client.retrievers.update(
    retriever_id=retriever.retriever_id,
    stages=[{
        "stage_name": "feature_search",
        "stage_type": "filter",
        "config": {
            "stage_id": "feature_search",
            "parameters": {
                "searches": [
                    {
                        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
                        "query": "{{INPUT.query}}",
                        "top_k": 100
                    },
                    {
                        "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                        "query": "{{INPUT.query}}",
                        "top_k": 100
                    }
                ],
                "fusion": "learned",
                "final_top_k": 25
            }
        }
    }]
)
```

No `weights` field needed — the system samples weights from Beta distributions on every query.

## 4. Understand Cold Start Behavior

With learned fusion enabled, the system handles sparse data automatically through hierarchical fallback:

| User Interactions | What Happens | Effective Behavior |
| - | - | - |
| **0** | Beta(1,1) = uniform prior for all features | Equivalent to RRF |
| **\< min\_interactions** (default 5) | Falls back to demographic or global weights | Shared weights from segment or all users |
| **>= min\_interactions** | Personal weights from this user's history | Per-user individually tuned weights |

The threshold for trusting personal weights is the `min_interactions` parameter (default **5**). Below that, the system falls back up the hierarchy: personal → demographic → global → uniform prior.

> [!NOTE]
>
> If you pass `user_id` in your search requests, the system tracks personal-level weights automatically. Without `user_id`, you still get global-level learning — the system learns which features are better *overall*, just not per-user.

## 5. Execute Searches with User Context

Pass `user_id` on every search request so the bandit can build per-user weight profiles:

```bash cURL theme={null}
curl -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{
    "inputs": {
      "query": "wireless noise canceling earbuds",
      "user_id": "user_456"
    }
  }'
```

```python Python SDK theme={null}
results = client.retrievers.execute(
    retriever_id=retriever.retriever_id,
    inputs={"query": "wireless noise canceling earbuds", "user_id": "user_456"}
)
```

Behind the scenes, the Thompson Sampler:

1. Looks up `user_456`'s stored interaction history
2. Computes Beta(α, β) per feature: `α = 1 + clicks`, `β = 1 + (impressions - clicks)`
3. Samples a weight from each Beta distribution
4. Normalizes weights to sum to 1
5. Executes each feature search and fuses results using the sampled weights

If `user_456` has consistently clicked text-matched results over image-matched ones, the text feature's Beta distribution is peaked higher — so sampled weights skew toward text.

## 6. Monitor Convergence

Check whether the learned weights are stabilizing using the analytics endpoint:

```bash theme={null}
curl "$MP_API_URL/v1/analytics/retrievers/{retriever_id}/signals?signal_type=learned_weights&hours=168" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

**What to look for:**

* **Weights stabilizing** — variance decreasing over time means the system is converging
* **Feature dominance** — if one feature's weight approaches 1.0, the other features may not be contributing value
* **Per-segment differences** — different user segments learning different weights validates that personalization is working

## 7. Measure Improvement

Use [evaluations](/docs/retrieval/evaluations) to compare learned fusion against your static baseline. Create an evaluation with the same queries and judge whether learned fusion produces better-ranked results.

The key metrics to track:

| Metric | What It Tells You |
| - | - |
| **CTR at position 1-3** | Are top results more clickable? |
| **Mean Reciprocal Rank** | Is the first relevant result appearing earlier? |
| **Interaction rate** | Are users engaging more overall? |
| **Weight variance** | Is the system still exploring or has it converged? |

> [!TIP]
>
> **Recommended rollout:** Run learned fusion on 10% of traffic alongside your static baseline. Compare metrics over 1-2 weeks. If learned fusion wins or ties, ramp to 100%.

## When to Use Each Strategy

| Starting Point | Recommendation |
| - | - |
| No interaction data, launching today | Start with `rrf` — strong default, no tuning needed |
| Domain expert knows feature importance | Start with `weighted` — encode expert knowledge as initial weights |
| Have 100+ interactions flowing | Switch to `learned` — let the data decide |
| Multiple user segments with different needs | `learned` with `user_id` — per-segment and per-user personalization |
| Need deterministic, reproducible results | Stay with `weighted` — learned fusion is stochastic by design |

## 8. Monitor Personalization

Once learned fusion is running, read which weights served each search. Every execution records them as `learned_fusion_context`, in the execute response and on the execution's profile:

```bash cURL theme={null}
curl "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/executions/$EXECUTION_ID/profile" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python SDK theme={null}
results = client.retrievers.execute(
    retriever.retriever_id,
    inputs={"query": "wireless noise canceling earbuds", "user_id": "user_123"},
)
context = results["learned_fusion_context"]
print(context["context_level"])    # "personal", "demographic", or "global"
print(context["sampled_weights"])  # {"text@v1/embed": 0.72, "image@v1/embed": 0.28}
```

To follow one user over time, list recent executions with `POST /v1/retrievers/{retriever_id}/executions/list` and read each profile. The old per-user weights endpoint was removed and answers `410 Gone` (see [Seeing Which Weights Served a Search](/docs/retrieval/auto-tune-rollout#seeing-which-weights-served-a-search)).

A user with enough interactions shows `context_level: "personal"`. New users fall back to `"demographic"` or `"global"` until they cross the `min_interactions` threshold.

## 9. Configure Reward Signals

Not all interactions are equal. Customize the `reward_map` to weight different signals:

```json theme={null}
{
  "learning_config": {
    "context_features": ["INPUT.user_id"],
    "reward_map": {
      "click": 1.0,
      "purchase": 3.0,
      "add_to_cart": 2.0,
      "positive_feedback": 2.0,
      "negative_feedback": -2.0,
      "skip": -1.0
    }
  }
}
```

Negative values act as penalties -- `negative_feedback` actively pushes the weight *away* from the feature that produced the disliked result. See the [Reward Signals reference](/docs/retrieval/reward-signals) for all 17 signal types and their defaults.

## 10. Safe Rollout

Before rolling learned fusion to all traffic, use the built-in rollout controls:

1. **Shadow mode** -- compute learned weights but serve static results. Compare offline.
2. **Traffic splitting** -- route a percentage of users to learned fusion (`rollout_pct: 10`).
3. **Kill switch** -- instantly disable learned fusion if something goes wrong.
4. **Per-user opt-out** -- exclude internal test accounts or specific users.

```bash theme={null}
# Enable shadow mode (logs learned weights, serves static results)
curl -X PATCH "$MP_API_URL/v1/retrievers/$RETRIEVER_ID" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{"stages": [{"stage_name": "feature_search", "stage_type": "filter", "config": {"stage_id": "feature_search", "parameters": {"fusion": "learned", "learning_config": {"shadow_mode": true, "rollout_pct": 10.0}}}}]}'
```

See the full [Rollout Guide](/docs/retrieval/auto-tune-rollout) for step-by-step rollout instructions.

## 11. Evaluate Learned vs Static

Generate an evaluation dataset from real interactions and compare learned fusion against your static baseline:

```bash cURL theme={null}
# Generate eval dataset from interaction history
curl -X POST "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/evaluations/generate-from-interactions" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{"dataset_name": "learned-vs-static", "min_interactions": 5}'

# Start the evaluation — returns 202 with an evaluation_id
curl -X POST "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/evaluations" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -d '{"dataset_name": "learned-vs-static", "evaluation_config": {"k_values": [1, 5, 10, 20]}}'

# Poll until status is completed, then read NDCG, MRR, Precision
curl "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/evaluations/$EVALUATION_ID" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python SDK theme={null}
import httpx

# Generate eval dataset from interaction history
resp = httpx.post(
    f"{base_url}/v1/retrievers/{retriever_id}/evaluations/generate-from-interactions",
    headers={"Authorization": f"Bearer {api_key}", "X-Namespace": namespace},
    json={"dataset_name": "learned-vs-static", "min_interactions": 5},
)
dataset = resp.json()

# Start the evaluation — this returns 202, metrics are not ready yet
resp = httpx.post(
    f"{base_url}/v1/retrievers/{retriever_id}/evaluations",
    headers={"Authorization": f"Bearer {api_key}", "X-Namespace": namespace},
    json={
        "dataset_name": dataset["dataset_name"],
        "evaluation_config": {"k_values": [1, 5, 10, 20]},
    },
)
evaluation_id = resp.json()["evaluation_id"]

# Poll until status is completed
resp = httpx.get(
    f"{base_url}/v1/retrievers/{retriever_id}/evaluations/{evaluation_id}",
    headers={"Authorization": f"Bearer {api_key}", "X-Namespace": namespace},
)
result = resp.json()
print(result["overall_metrics"]["ndcg_at_10"])
```

> [!TIP]
>
> Run evaluations on a regular cadence (e.g., weekly or after every 1,000 interactions). If learned fusion regresses vs. static, the kill switch gives you an instant rollback path.

## Next Steps

> **[Auto-Tune Concept](/docs/retrieval/auto-tune)**
>
> Full overview of the auto-tune system — how it works, when to use it, and configuration options.

> **[Learned Fusion Deep Dive](/docs/relevance/learned-fusion)**
>
> Thompson Sampling internals, session adaptation, temporal decay, weight clamping, and exploration decay.

> **[Reward Signals](/docs/retrieval/reward-signals)**
>
> All 17 interaction types, reward weighting, negative signals, and position bias correction.

> **[Rollout Guide](/docs/retrieval/auto-tune-rollout)**
>
> Shadow mode, traffic splitting, kill switch, per-user opt-out, and preference reset.

> **[Fusion Strategies](/docs/relevance/fusion-strategies)**
>
> Compare all 5 fusion strategies: RRF, DBSF, Weighted, Max, and Learned.

> **[Evaluations](/docs/retrieval/evaluations)**
>
> Set up benchmarks to measure whether learned fusion is improving result quality.


# Hierarchical search over clusters
Source: https://docs.mixpeek.com/docs/tutorials/hierarchical-cluster-search

Cluster a collection into a labeled hierarchy, then answer queries by walking it with Jev

Vector search finds the documents nearest to a query. Hierarchical search first finds the right part of your corpus, then searches inside it. This guide clusters a collection into a two-level hierarchy, labels every cluster from a fixed vocabulary, and answers queries by walking the tree top-down with [Jev](/docs/processing/jev).

The walk uses the `cluster_navigation` strategy of [Agent Search](/docs/retrieval/stages/agent-search). At each level, Jev returns a probability for every child cluster. The stage keeps the best few paths and returns members of the best leaf clusters.

## 1. Cluster into a hierarchy and label it

Set `hierarchical: true` to split each top-level cluster into sub-clusters. Give `candidate_labels` the names you want at both levels.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/clusters" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_name": "content_tree",
    "collection_ids": ["col_content"],
    "cluster_type": "vector",
    "vector_config": {
      "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
      "clustering_method": "kmeans",
      "algorithm_params": {"n_clusters": 3},
      "hierarchical": true,
      "max_hierarchy_depth": 2
    },
    "llm_labeling": {
      "enabled": true,
      "provider": "typesafe",
      "model_name": "jev-latest",
      "candidate_labels": ["Product Ads", "Sports", "Cooking",
                           "Pharmaceutical Ads", "Car Ads",
                           "Skateboarding", "Surfing", "Baking", "Grilling"],
      "labeling_inputs": {
        "input_mappings": [{"input_key": "text", "source_type": "payload", "path": "content"}]
      }
    }
  }'

curl -sS -X POST "$MP_API_URL/v1/clusters/$CLUSTER_ID/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

```python Python theme={null}
import requests

H = {"Authorization": f"Bearer {MP_API_KEY}", "X-Namespace": MP_NAMESPACE}
cluster = requests.post(f"{MP_API_URL}/v1/clusters", headers=H, json={
    "cluster_name": "content_tree",
    "collection_ids": ["col_content"],
    "cluster_type": "vector",
    "vector_config": {
        "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
        "clustering_method": "kmeans",
        "algorithm_params": {"n_clusters": 3},
        "hierarchical": True,
        "max_hierarchy_depth": 2,
    },
    "llm_labeling": {
        "enabled": True,
        "provider": "typesafe",
        "model_name": "jev-latest",
        "candidate_labels": ["Product Ads", "Sports", "Cooking",
                             "Pharmaceutical Ads", "Car Ads",
                             "Skateboarding", "Surfing", "Baking", "Grilling"],
        "labeling_inputs": {"input_mappings": [
            {"input_key": "text", "source_type": "payload", "path": "content"}]},
    },
}).json()
requests.post(f"{MP_API_URL}/v1/clusters/{cluster['cluster_id']}/execute", headers=H)
```

```javascript JavaScript theme={null}
const H = {
  Authorization: `Bearer ${MP_API_KEY}`,
  "X-Namespace": MP_NAMESPACE,
  "Content-Type": "application/json",
};
const cluster = await (await fetch(`${MP_API_URL}/v1/clusters`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({
    cluster_name: "content_tree",
    collection_ids: ["col_content"],
    cluster_type: "vector",
    vector_config: {
      feature_uri: "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
      clustering_method: "kmeans",
      algorithm_params: { n_clusters: 3 },
      hierarchical: true,
      max_hierarchy_depth: 2,
    },
    llm_labeling: {
      enabled: true,
      provider: "typesafe",
      model_name: "jev-latest",
      candidate_labels: ["Product Ads", "Sports", "Cooking", "Pharmaceutical Ads",
        "Car Ads", "Skateboarding", "Surfing", "Baking", "Grilling"],
      labeling_inputs: { input_mappings: [
        { input_key: "text", source_type: "payload", path: "content" }] },
    },
  }),
})).json();
await fetch(`${MP_API_URL}/v1/clusters/${cluster.cluster_id}/execute`, { method: "POST", headers: H });
```

The run writes one centroid document per cluster into the cluster's output collection. Each centroid carries `label`, `label_confidence` and `parent_cluster_id`.

## 2. Create a retriever that walks the tree

Point the retriever at the cluster's output collection.

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "browse_content_tree",
    "collection_identifiers": ["'"$OUTPUT_COLLECTION_ID"'"],
    "input_schema": {"query": {"type": "text", "required": true}},
    "stages": [{
      "stage_name": "browse",
      "config": {
        "stage_id": "agent_search",
        "parameters": {
          "strategy": "cluster_navigation",
          "model_name": "jev-latest",
          "cluster_navigation": {"beam_width": 2, "max_leaf_clusters": 1}
        }
      }
    }]
  }'
```

```python Python theme={null}
retriever = requests.post(f"{MP_API_URL}/v1/retrievers", headers=H, json={
    "retriever_name": "browse_content_tree",
    "collection_identifiers": [OUTPUT_COLLECTION_ID],
    "input_schema": {"query": {"type": "text", "required": True}},
    "stages": [{"stage_name": "browse", "config": {
        "stage_id": "agent_search",
        "parameters": {
            "strategy": "cluster_navigation",
            "model_name": "jev-latest",
            "cluster_navigation": {"beam_width": 2, "max_leaf_clusters": 1},
        },
    }}],
}).json()
```

```javascript JavaScript theme={null}
const retriever = await (await fetch(`${MP_API_URL}/v1/retrievers`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({
    retriever_name: "browse_content_tree",
    collection_identifiers: [OUTPUT_COLLECTION_ID],
    input_schema: { query: { type: "text", required: true } },
    stages: [{ stage_name: "browse", config: {
      stage_id: "agent_search",
      parameters: {
        strategy: "cluster_navigation",
        model_name: "jev-latest",
        cluster_navigation: { beam_width: 2, max_leaf_clusters: 1 },
      },
    } }],
  }),
})).json();
```

## 3. Query it

```bash cURL theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/$RETRIEVER_ID/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"inputs": {"query": "drug commercials that disclose side effects"}}'
```

```python Python theme={null}
result = requests.post(
    f"{MP_API_URL}/v1/retrievers/{retriever['retriever_id']}/execute",
    headers=H,
    json={"inputs": {"query": "drug commercials that disclose side effects"}},
).json()
```

```javascript JavaScript theme={null}
const result = await (await fetch(
  `${MP_API_URL}/v1/retrievers/${retriever.retriever_id}/execute`,
  { method: "POST", headers: H,
    body: JSON.stringify({ inputs: { query: "drug commercials that disclose side effects" } }) },
)).json();
```

The stage metadata shows the path Jev took. `chosen_clusters` lists each leaf with its path and score, and `navigation_trace` lists the candidates and probabilities at every level.

```json theme={null}
{
  "chosen_clusters": [
    {"cluster_id": "cl_3", "label": "Gourmet Food & Floral Arrangements",
     "path": ["Gourmet Food & Floral Arrangements"], "score": 1.0}
  ]
}
```

That output is from a production run of "a hand-tied bouquet of flowers" against a
product catalog: the stage returned 10 members, all food or home products, in 415 ms.

## Combine with vector search

Put a `feature_search` stage before `agent_search` to search first and then keep only the hits inside the chosen clusters. The order from the search is preserved. Member documents need a cluster field, so enrich the source collection with cluster assignments (`enrich_source_collection: true`) and set `cluster_navigation.centroid_collection_ids` to the collection holding the centroids.

## Tuning

* `beam_width: 1` is greedy descent and makes one request per level. Wider beams let a later level correct an ambiguous early choice. All beams at one level share one request.
* Summaries improve routing. Jev sees each centroid's `summary` as the option's description.
* `max_leaf_clusters` above 1 returns neighbors of the best cluster, useful when a query spans topics.


# Ingest Video from S3
Source: https://docs.mixpeek.com/docs/tutorials/ingest-video-from-s3

Connect an S3 bucket, sync video files automatically, and make them searchable end-to-end

This guide walks the full path from an S3 bucket of videos to a working search index: create a storage connection, point a sync at your S3 prefix, and let Mixpeek ingest and process new files automatically.

> [!NOTE]
>
> This is the automated/continuous path. To upload a single file by URL instead, see [Video Understanding](/docs/tutorials/video-understanding). For the AWS IAM policy and role setup, see [AWS S3](/docs/integrations/object-storage/s3).

## Prerequisites

* An S3 bucket containing video files.
* AWS credentials — either an access key pair or an IAM role ARN (see [AWS S3](/docs/integrations/object-storage/s3) for the IAM policy).
* A Mixpeek API key and namespace.

## 1. Create a storage connection

A connection stores your S3 credentials once and can be reused across buckets. Credentials are validated before the connection is saved (`test_before_save` defaults to `true`).

```bash Access key theme={null}
curl -sS -X POST "$MP_API_URL/v1/organizations/connections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-video-s3",
    "provider_type": "s3",
    "provider_config": {
      "provider_type": "s3",
      "region": "us-east-1",
      "credentials": {
        "type": "access_key",
        "access_key_id": "AKIA...",
        "secret_access_key": "..."
      }
    }
  }'
```

```bash IAM role (recommended) theme={null}
curl -sS -X POST "$MP_API_URL/v1/organizations/connections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-video-s3",
    "provider_type": "s3",
    "provider_config": {
      "provider_type": "s3",
      "region": "us-east-1",
      "credentials": {
        "type": "iam_role",
        "role_arn": "arn:aws:iam::123456789012:role/mixpeek-read",
        "external_id": "your-external-id"
      }
    }
  }'
```

The response includes a `connection_id` (e.g. `conn_abc123`). Verify connectivity any time with:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/organizations/connections/conn_abc123/test" \
  -H "Authorization: Bearer $MP_API_KEY"
```

> [!TIP]
>
> Connections are org-level and not namespace-scoped — no `X-Namespace` header is needed for connection calls. Buckets, syncs, and collections below are namespace-scoped, so they require `X-Namespace`.

## 2. Create a bucket

The bucket declares the schema for each synced object. Use the Mixpeek type `video` for the file field.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "video-catalog",
    "bucket_schema": {
      "properties": {
        "video_url": { "type": "video" }
      }
    }
  }'
```

## 3. Create a collection

One collection with `multimodal_extractor` produces both visual (`vertex_multimodal_embedding`) and speech (`multilingual_e5_large_instruct_v1`) features per segment.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "video-moments",
    "source": { "type": "bucket", "bucket_ids": ["bkt_videos"] },
    "feature_extractor": {
      "feature_extractor_name": "multimodal_extractor",
      "version": "v1",
      "input_mappings": { "video": "video_url" }
    }
  }'
```

## 4. Sync the S3 prefix into the bucket

A sync watches an S3 path and ingests matching files. With `sync_mode: "continuous"`, Mixpeek polls for new files and ingests them automatically; `initial_only` runs a single backfill.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_videos/syncs" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "source_path": "s3://my-bucket/videos/",
    "sync_mode": "continuous",
    "polling_interval_seconds": 300,
    "batch_size": 50,
    "file_filters": { "extensions": [".mp4", ".mov"] },
    "schema_mapping": {
      "mappings": {
        "video_url": {
          "target_type": "blob",
          "source": { "type": "file" },
          "blob_type": "video",
          "blob_property": "video_url"
        }
      }
    }
  }'
```

| Field | Default | Description |
| - | - | - |
| `connection_id` | required | The storage connection from step 1. |
| `source_path` | required | S3 URI/prefix to watch (e.g. `s3://my-bucket/videos/`). |
| `sync_mode` | `continuous` | `continuous` polls for new files; `initial_only` backfills once. |
| `polling_interval_seconds` | `300` | How often to check for new files (30–900). |
| `batch_size` | `50` | Files processed per batch (1–100). |
| `file_filters` | — | Restrict by extension/pattern. |
| `schema_mapping` | — | Map each file to a bucket property. |
| `skip_duplicates` | `true` | Skip files already ingested (by source ID). |

The response includes a `sync_config_id` (e.g. `sync_xyz`).

## 5. Run and monitor the sync

`continuous` syncs start on their own, but you can trigger the first run immediately and watch its job:

```bash theme={null}
# Trigger a run now
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_videos/syncs/sync_xyz/trigger" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"

# Check sync metrics (files discovered, ingested, failed)
curl -sS "$MP_API_URL/v1/buckets/bkt_videos/syncs/sync_xyz/metrics" \
  -H "Authorization: Bearer $MP_API_KEY" -H "X-Namespace: $MP_NAMESPACE"
```

| Operation | Endpoint |
| - | - |
| Trigger now | `POST /v1/buckets/{bucket_id}/syncs/{sync_config_id}/trigger` |
| Pause / resume | `POST .../pause` · `POST .../resume` |
| Job status | `GET .../jobs/{sync_job_id}` |
| Failed files (DLQ) | `POST .../dlq` |

> [!NOTE]
>
> The sync ingests files as objects and (unless `skip_batch_submission` is set) submits them for processing automatically — so your `multimodal_extractor` collection populates without any extra step. Vectors are searchable within 10–30s of each batch completing.

## 6. Search

Create a retriever over the collection and execute it:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "video-search",
    "collection_identifiers": ["video-moments"],
    "input_schema": { "query": { "type": "text", "required": true } },
    "stages": [
      {
        "stage_name": "search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100, "weight": 0.6
              },
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100, "weight": 0.4
              }
            ],
            "final_top_k": 20,
            "fusion": "weighted"
          }
        }
      }
    ]
  }'
```

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "inputs": { "query": "people discussing electric vehicles" } }'
```

## Keep it fresh

Because the sync is `continuous`, new videos dropped into the S3 prefix are ingested and indexed automatically. To re-run clustering or enrichment on a schedule as new content lands, see [Triggers](/docs/platform/triggers).

## Other storage providers

The same connection → bucket → sync flow works for every supported provider — swap `provider_type` and `provider_config`:

> **[Google Cloud Storage](/docs/integrations/object-storage/gcs)**

> **[Azure Blob](/docs/integrations/object-storage/azure-blob)**

> **[Cloudflare R2](/docs/integrations/object-storage/r2)**


# Add a New Understanding to Existing Content
Source: https://docs.mixpeek.com/docs/tutorials/reprocess-existing-content

Run a new extractor over an already-ingested corpus — scoped, cost-safe, and priced before you run — without re-uploading or re-paying for existing work

You've ingested a corpus and built search over it. Now you want to add another
understanding — a new extractor, a new model version, a classifier — over the
**same content**, without re-uploading anything and without re-paying for the
extraction you already ran.

This is a first-class workflow. Three pieces make it cost-safe by default:

1. **Batch scoping** — `collection_ids` on a batch runs *only* that collection's
   extractor over the bucket.
2. **Dedup protection** — `dedup_strategy: "skip"` (the default) never re-runs
   extraction for content a collection has already processed, even if you
   forget to scope.
3. **Pre-flight pricing** — `POST /batches/{id}/estimate-cost` is a true dry
   run: it tells you exactly what would run and what it costs *before* you
   commit, including an `already_extracted_count` showing the work you will
   NOT be re-billed for.

> [!NOTE]
>
> Everything below operates on content already in a bucket. Buckets hold your
> source objects once; collections are independent processing pipelines over
> them. That separation is what makes iterating on models cheap.

## 1. Create a second collection over the same bucket

Your existing collection keeps serving queries untouched. The new
understanding gets its own collection pointed at the same bucket:

```python Python theme={null}
new_col = client.collections.create(
    collection_name="scripts-v2-understanding",
    source={"type": "bucket", "bucket_id": bucket_id},
    features=["text_search"],  # or a feature key from GET /v1/collections/features
)
```

```javascript JavaScript theme={null}
const newCol = await client.collections.create({
  collection_name: "scripts-v2-understanding",
  source: { type: "bucket", bucket_id: bucketId },
  features: ["text_search"],
});
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/collections" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "scripts-v2-understanding",
    "source": {"type": "bucket", "bucket_id": "'$BUCKET_ID'"},
    "features": ["text_search"]
  }'
```

If the extractor behind your chosen feature isn't registered on the namespace
yet, the platform registers it for you at create time. (On older deployments a
422 tells you the exact `PATCH /v1/namespaces` body to run first.)

## 2. Create a batch scoped to the new collection

Scope the batch with `collection_ids` so only the new collection's pipeline
runs:

```python Python theme={null}
batch = client.batches.create(
    bucket_id=bucket_id,
    object_ids=object_ids,           # or filters selecting the slice you want
    collection_ids=[new_col.collection_id],
)
```

```javascript JavaScript theme={null}
const batch = await client.batches.create({
  bucket_id: bucketId,
  object_ids: objectIds,
  collection_ids: [newCol.collection_id],
});
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/batches" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{"object_ids": ["obj_..."], "collection_ids": ["'$NEW_COLLECTION_ID'"]}'
```

## 3. Price it before you run it

`estimate-cost` is a dry run — nothing processes, nothing is charged. (For quoting *planned* ingestion in dollars by modality and feature before you even build the batch, use [`POST /v1/organizations/billing/estimate`](/docs/platform/billing#estimate-before-you-run) — same rating engine that bills you.)

```python Python theme={null}
estimate = client.batches.estimate_cost(bucket_id=bucket_id, batch_id=batch.batch_id)
print(estimate)
# {
#   "object_count": 8,
#   "already_extracted_count": 0,   # nothing skipped — all new work
#   "estimated_credits": 10,        # legacy field: internal ledger unit (1 credit = $0.001 → $0.01)
#   "extractors": ["text_extractor"]
# }
```

```javascript JavaScript theme={null}
const estimate = await client.batches.estimateCost(bucketId, batch.batch_id);
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/buckets/$BUCKET_ID/batches/$BATCH_ID/estimate-cost" \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE"
```

**The trap probe worth knowing:** if you estimate an *unscoped* batch over a
bucket that an existing collection already processed, the estimate shows that
prior work under `already_extracted_count` and prices it at zero — the default
`dedup_strategy: "skip"` refuses to re-run (and re-bill) extraction whose
inputs haven't changed. Your existing collection's GPU spend is protected even
when you forget to scope.

## 4. Submit, then verify only new work ran

Submit the batch and let it complete. Two checks confirm the cost boundary
held:

* The **new** collection has documents; the **old** collection's documents show
  no changed `updated_at`.
* The batch record carries its own cost:
  `batch.cost → {"credits_consumed": 10, "cost_usd": 0.01}` — `cost_usd` is the
  number that matters (the `credits_consumed` field is the internal ledger unit,
  1 credit = $0.001). Batches bill at least the $0.01 minimum.

For a multi-batch rollout, sum the `cost_usd` of the rollout's batches —
each batch is its own attribution record.

## 5. Compare the two understandings side by side

Both collections index the same source objects, so retrieval comparisons need
no second corpus copy: run the same query against each collection (or one
retriever spanning both) and compare. For systematic comparison, build a small
[evaluation](/docs/retrieval/evaluations) dataset once and run it against both.

When a new *version* of an extractor ships (versions coexist in the catalog —
e.g. v1 and v2 side by side), the same pattern applies: pin the new version in
a new collection, roll a scoped batch, compare, and cut over when the numbers
say so. For whole-namespace model swaps, use
[model migration](/docs/processing/model-migration) instead.

## 6. Re-run only failures

If some objects fail, you don't resubmit the batch:

* `GET /v1/buckets/{bucket_id}/batches/{batch_id}/failed-documents` lists
  failures with per-tier detail and what's retryable.
* `POST /v1/buckets/{bucket_id}/batches/{batch_id}/retry` re-runs just those.

## Next steps

> **[Feature Extractors](/docs/processing/feature-extractors)**
>
> How a single extractor is configured — inputs, outputs, inference cache.

> **[Model Migration](/docs/processing/model-migration)**
>
> Swap a namespace's embedding model wholesale with validation and dry-run.

> **[Multi-Tier Extraction](/docs/processing/multi-tier-extractors)**
>
> Chain collections into a DAG — transcribe, then embed, then classify.

> **[Custom Extractor Quickstart](/docs/tutorials/custom-extractor-quickstart)**
>
> Build and test your own extractor end-to-end.


# Reverse Media Search
Source: https://docs.mixpeek.com/docs/tutorials/reverse-search

Find visually similar content using images or videos as your search query

> **[Search your own images and video](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=docs_reverse_search\&utm_campaign=managed_onramp)**
>
> Create a namespace, upload a few assets, and run a visual similarity search in minutes — no API key setup required.

> [!TIP]
>
> Reverse media search uses the warehouse's Decompose layer to extract visual features, then queries them in the Reassemble layer. See [Multi-Stage Retrieval](/docs/retrieval/multi-stage-deep-dive) for composing pipelines.

## Input Methods

| Method | Example | Speed |
| - | - | - |
| Pre-computed embedding | `{"embedding": [0.1, 0.2, ...]}` | Fastest |
| Image URL | `{"url": "https://example.com/img.jpg"}` | Fast |
| Video URL | `{"url": "s3://bucket/video.mp4"}` | Medium |
| Base64 | `{"base64": "data:image/jpeg;base64,..."}` | Fast |

## 1. Create a Bucket

```bash theme={null}
POST /v1/buckets
{
  "bucket_name": "visual-assets",
  "bucket_schema": {
    "properties": {
      "asset_url": { "type": "image", "required": true },
      "brand": { "type": "text" },
      "campaign_id": { "type": "text" }
    }
  }
}
```

## 2. Create a Collection

**For images:**

```bash theme={null}
POST /v1/collections
{
  "collection_name": "product-images",
  "source": { "type": "bucket", "bucket_ids": ["bkt_visual_assets"] },
  "feature_extractor": {
    "feature_extractor_name": "image_extractor",
    "version": "v1",
    "input_mappings": { "image_url": "asset_url" },
    "parameters": { "model": "clip-vit-large-patch14" },
    "field_passthrough": [
      { "source_path": "brand" },
      { "source_path": "campaign_id" }
    ]
  }
}
```

**For videos:**

```bash theme={null}
POST /v1/collections
{
  "collection_name": "video-segments",
  "source": { "type": "bucket", "bucket_ids": ["bkt_visual_assets"] },
  "feature_extractor": {
    "feature_extractor_name": "multimodal_extractor",
    "version": "v1",
    "input_mappings": { "video": "asset_url" },
    "parameters": {
      "scene_detection_threshold": 0.3,
      "extract_keyframes": true
    }
  }
}
```

## 3. Ingest Assets

```bash theme={null}
POST /v1/buckets/{bucket_id}/objects
{
  "key_prefix": "/products/shoes",
  "blobs": [
    { "property": "asset_url", "type": "image", "data": "s3://my-bucket/products/sneaker-001.jpg" }
  ],
  "brand": "Nike",
  "campaign_id": "fall-2025"
}
```

## 4. Process

```bash theme={null}
POST /v1/buckets/{bucket_id}/batches
{ "object_ids": ["obj_001", "obj_002"] }

POST /v1/buckets/{bucket_id}/batches/{batch_id}/submit
```

## 5. Create a Retriever

```bash theme={null}
POST /v1/retrievers
{
  "retriever_name": "reverse-image-search",
  "collection_identifiers": ["col_product_images"],
  "input_schema": {
    "query_image": { "type": "image", "required": true },
    "min_similarity": { "type": "number", "default": 0.7 }
  },
  "stages": [
    {
      "stage_name": "visual_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
              "query": {
                "input_mode": "content",
                "value": "{{INPUT.query_image}}"
              },
              "top_k": 50
            }
          ]
        }
      }
    },
    {
      "stage_name": "filter",
      "stage_type": "reduce",
      "config": {
        "stage_id": "score_threshold",
        "parameters": {
          "min_score": "{{INPUT.min_similarity}}"
        }
      }
    }
  ]
}
```

## 6. Search

**With image URL:**

```bash theme={null}
POST /v1/retrievers/{retriever_id}/execute
{
  "inputs": {
    "query_image": "https://example.com/reference.jpg",
    "min_similarity": 0.75
  }
}
```

**With pre-computed embedding (faster):**

```bash theme={null}
POST /v1/retrievers/{retriever_id}/execute
{
  "inputs": {
    "embedding": [0.1, 0.2, ...]
  }
}
```

**With base64:**

```python theme={null}
import base64

with open("image.jpg", "rb") as f:
    image_data = base64.b64encode(f.read()).decode()

response = requests.post(
    "https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute",
    json={
        "inputs": {
            "query_image": f"data:image/jpeg;base64,{image_data}"
        }
    }
)
```

## Cross-Modal Search (Image → Video)

Search videos using a reference image:

```bash theme={null}
{
  "stage_name": "cross_modal_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "feature_search",
    "parameters": {
      "searches": [
        {
          "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
          "query": {
            "input_mode": "content",
            "value": "{{INPUT.query_image}}"
          },
          "top_k": 50
        }
      ]
    }
  }
}
```

## Multi-Collection Search

Search images and videos together:

```bash theme={null}
{
  "retriever_name": "visual-federated-search",
  "collection_identifiers": ["col_images", "col_videos"],
  "stages": [
    {
      "stage_name": "federated_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://image_extractor@v1/google_siglip_base_v1",
              "collections": ["col_images"],
              "weight": 0.5,
              "top_k": 25
            },
            {
              "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
              "collections": ["col_videos"],
              "weight": 0.5,
              "top_k": 25
            }
          ],
          "fusion": "rrf"
        }
      }
    }
  ]
}
```

## Similarity Thresholds

| Score | Meaning |
| - | - |
| 0.95+ | Near-duplicate |
| 0.85-0.94 | Very similar |
| 0.70-0.84 | Related |
| Below 0.70 | Weak match |

## Classify with Taxonomies

Auto-tag assets by matching against a reference collection of known brands or product types:

```bash theme={null}
POST /v1/taxonomies
{
  "taxonomy_name": "brand-classifier",
  "config": {
    "taxonomy_type": "flat",
    "retriever_id": "ret_reverse_image_search",
    "input_mappings": [
      { "input_key": "query_image", "source_type": "payload", "path": "image_url" }
    ],
    "source_collection": {
      "collection_id": "col_product_images",
      "enrichment_fields": [
        { "field_path": "metadata.brand", "merge_mode": "enrich" }
      ]
    }
  }
}
```

New assets automatically get `metadata.brand` enriched when they visually match a known reference. See [Taxonomies](/docs/enrichment/taxonomies) for hierarchical taxonomies.

## Discover Clusters

Find visual themes across your asset library:

```bash theme={null}
POST /v1/clusters
{
  "cluster_name": "visual-themes",
  "collection_ids": ["col_product_images"],
  "cluster_type": "vector",
  "vector_config": {
    "feature_uris": ["mixpeek://image_extractor@v1/google_siglip_base_v1"],
    "clustering_method": "hdbscan",
    "algorithm_params": { "min_cluster_size": 10 },
    "preprocessing_steps": [
      { "method": "umap", "n_components": 2 }
    ]
  },
  "llm_labeling": {
    "provider": "openai",
    "model_name": "gpt-4o-mini"
  }
}
```

Clusters reveal groupings like "product close-ups", "lifestyle shots", and "packaging" without predefined categories. Promote stable clusters to taxonomy nodes. See [Clusters](/docs/enrichment/clusters) for all algorithms.

## Set Up Alerts

Get notified when new assets closely match existing ones (counterfeit detection, duplicate detection):

```bash theme={null}
POST /v1/alerts
{
  "name": "duplicate-detection",
  "retriever_id": "ret_reverse_image_search",
  "trigger_on": "results",
  "notification_config": {
    "channels": [
      { "channel_type": "webhook",
        "config": { "url": "https://example.com/webhook" } }
    ]
  }
}
```

A retriever-source alert fires on whatever its retriever returns, so scope the
match inside the retriever rather than on the alert. Set `trigger_on` to
`no_results` to fire when nothing matches. See [Alerts](/docs/enrichment/alerts).

## Set Up Webhooks

Track batch processing for large asset uploads:

```bash theme={null}
POST /v1/organizations/webhooks
{
  "webhook_name": "asset-processing",
  "event_types": ["objects.created.batch", "collection.documents.written"],
  "channels": [
    { "channel": "webhook", "configs": { "url": "https://example.com/webhook" } }
  ]
}
```


# Semantic Search
Source: https://docs.mixpeek.com/docs/tutorials/semantic-search

Build search with vector embeddings and hybrid ranking

> **[Create your free workspace](https://studio.mixpeek.com/namespaces/create?mode=managed\&utm_source=docs\&utm_medium=tutorial\&utm_campaign=managed_onramp\&utm_content=semantic_search_tutorial)**
>
> Follow this tutorial on your own data — create a free workspace, upload your documents, and run your first semantic search in minutes.

> [!TIP]
>
> Semantic search is one stage in the warehouse's Reassemble layer. This tutorial covers the basics; see [Multi-Stage Retrieval](/docs/retrieval/multi-stage-deep-dive) for composing it with other stages.

## 1. Create a Bucket

```bash theme={null}
POST /v1/buckets
{
  "bucket_name": "knowledge-base",
  "bucket_schema": {
    "properties": {
      "title": { "type": "text", "required": true },
      "content": { "type": "text", "required": true },
      "category": { "type": "text" },
      "tags": { "type": "array" }
    }
  }
}
```

## 2. Create a Collection

```bash theme={null}
POST /v1/collections
{
  "collection_name": "docs-search",
  "source": { "type": "bucket", "bucket_ids": ["bkt_kb"] },
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": { "text": "content" },
    "parameters": {
      "model": "multilingual-e5-large-instruct",
      "chunk_strategy": "sentence",
      "chunk_size": 512,
      "chunk_overlap": 50
    },
    "field_passthrough": [
      { "source_path": "title" },
      { "source_path": "category" },
      { "source_path": "tags" }
    ]
  }
}
```

**Chunking strategies:**

* `sentence` – Best for Q\&A
* `paragraph` – Best for long-form content
* `fixed` – Predictable token windows

## 3. Ingest Documents

```bash theme={null}
POST /v1/buckets/{bucket_id}/objects
{
  "key_prefix": "/docs/api",
  "blobs": [
    { "property": "title", "type": "text", "data": "Authentication Guide" },
    { "property": "content", "type": "text", "data": "Mixpeek uses Bearer token authentication..." },
    { "property": "category", "type": "text", "data": "getting-started" }
  ]
}
```

## 4. Create a Retriever

```bash theme={null}
POST /v1/retrievers
{
  "retriever_name": "docs-search",
  "collection_identifiers": ["col_docs"],
  "input_schema": {
    "query": { "type": "text", "required": true }
  },
  "stages": [
    {
      "stage_name": "knn_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
              "top_k": 50
            }
          ],
          "final_top_k": 50
        }
      }
    }
  ]
}
```

## 5. Search

```bash theme={null}
POST /v1/retrievers/{retriever_id}/execute
{
  "inputs": { "query": "how do I authenticate API requests?" }
}
```

## Hybrid Search (Vector + BM25)

Combine semantic (vector) and keyword (BM25) matching in one stage. BM25 is **not** a separate feature — set `lexical: true` on a search to match the query against the namespace's full-text index instead of embedding it. Use `rrf` fusion so the score-scale mismatch between cosine similarity and BM25 doesn't matter.

```bash theme={null}
{
  "stages": [
    {
      "stage_name": "hybrid_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
              "top_k": 100
            },
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
              "lexical": true,
              "top_k": 100
            }
          ],
          "fusion": "rrf",
          "final_top_k": 50
        }
      }
    }
  ]
}
```

> [!NOTE]
>
> Lexical search requires a `text` payload index on the field. See [Text Indexes (BM25)](/docs/vector-store/namespaces#text-indexes-bm25) and the [Feature Search](/docs/retrieval/stages/feature-search#lexical-bm25-search) reference.

## Pre-Filter by Metadata

Filter before vector search for efficiency:

```bash theme={null}
{
  "stages": [
    {
      "stage_name": "category_filter",
      "stage_type": "filter",
      "config": {
        "stage_id": "attribute_filter",
        "parameters": {
          "field": "category",
          "operator": "eq",
          "value": "getting-started"
        }
      }
    },
    {
      "stage_name": "knn_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
              "top_k": 50
            }
          ],
          "final_top_k": 50
        }
      }
    }
  ]
}
```

## Reranking

Use a cross-encoder for better accuracy:

```bash theme={null}
{
  "stages": [
    {
      "stage_name": "knn_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [
            {
              "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
              "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
              "top_k": 100
            }
          ],
          "final_top_k": 100
        }
      }
    },
    {
      "stage_name": "rerank",
      "stage_type": "sort",
      "config": {
        "stage_id": "rerank",
        "parameters": {
          "inference_name": "BAAI__bge_reranker_v2_m3",
          "top_k": 20
        }
      }
    }
  ]
}
```

See the [Rerank stage](/docs/retrieval/stages/rerank) reference for available models and parameters.

## Model Options

| Model | Speed | Use Case |
| - | - | - |
| `multilingual-e5-base` | Fast | High-volume |
| `multilingual-e5-large-instruct` | Medium | General-purpose |
| `bge-large-en-v1.5` | Medium | English-only |
| `openai/text-embedding-3-large` | Slow | Premium |

## Next steps

> **[Classify documents](/docs/enrichment/taxonomies)**
>
> Auto-categorize documents against a taxonomy of reference categories.

> **[Discover topics](/docs/enrichment/clusters)**
>
> Cluster document embeddings to surface topic groups without predefined categories.

> **[Get notified](/docs/enrichment/alerts)**
>
> Trigger alerts when new documents match a condition.

> **[Schedule jobs](/docs/platform/triggers)**
>
> Re-cluster or re-enrich on a cron or interval as new content lands.


# Video Understanding
Source: https://docs.mixpeek.com/docs/tutorials/video-understanding

Ingest video, extract visual + speech embeddings, and search for moments by what's shown or said

> [!TIP]
>
> This is the full warehouse flow end-to-end: decompose video into searchable segments, store visual and speech embeddings, then reassemble answers through a retriever. Every code block below is copy-pasteable and uses the real API field names.

![Video Understanding Pipeline](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/mixpeek-video-understanding.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=8370fdfc5820afd33811585e4eacf302)

## How It Works

When you ingest a video, the `multimodal_extractor` runs a multi-stage pipeline:

1. **Chunking** — the video is split into segments (scene detection / fixed intervals).
2. **Visual embeddings** — each segment's keyframes are embedded as `vertex_multimodal_embedding`.
3. **Speech embeddings** — speech is transcribed (Whisper) and embedded (feature URI `mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1`).
4. **Multi-vector indexing** — both embeddings are indexed per segment, so one collection supports hybrid visual + transcript search.

At query time, a retriever searches across both embeddings and fuses the results — finding moments by what's *shown* or what's *said*.

> [!NOTE]
>
> A single `multimodal_extractor@v1` produces **both** the visual feature (URI suffix `vertex_multimodal_embedding`) and the speech/transcript feature (URI suffix `multilingual_e5_large_instruct_v1`). You do **not** need separate video/audio extractors.

## 1. Create a bucket

The bucket schema declares the fields each video object carries. Use the Mixpeek type `video` for the file field.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "bucket_name": "video-catalog",
    "bucket_schema": {
      "properties": {
        "video_url": { "type": "video" },
        "title": { "type": "text" },
        "category": { "type": "string" }
      }
    }
  }'
```

## 2. Create a collection

One collection with the `multimodal_extractor` gives you both visual and speech search. Map the extractor's `video` input to your bucket's `video_url` field.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/collections" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_name": "video-moments",
    "source": { "type": "bucket", "bucket_ids": ["bkt_videos"] },
    "feature_extractor": {
      "feature_extractor_name": "multimodal_extractor",
      "version": "v1",
      "input_mappings": { "video": "video_url" },
      "field_passthrough": [
        { "source_path": "title" },
        { "source_path": "category" }
      ]
    }
  }'
```

> [!NOTE]
>
> A collection takes exactly **one** `feature_extractor`. To run additional extractors over the same objects, create more collections from the same bucket.

## 3. Ingest a video

Register a video object. The blob's content goes in the `data` field (a URL, `s3://` path, base64, or raw content).

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_videos/objects" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "key_prefix": "/marketing/demos",
    "blobs": [
      {
        "property": "video_url",
        "type": "video",
        "data": "s3://my-bucket/demos/product-launch.mp4"
      },
      { "property": "title", "type": "text", "data": "Product Launch Q4 2025" },
      { "property": "category", "type": "text", "data": "marketing" }
    ]
  }'
```

## 4. Process

Create a batch from your object(s), submit it, then poll the returned task until it completes.

```bash theme={null}
# Create a batch (returns batch_id)
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_videos/batches" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{ "object_ids": ["obj_video_001"] }'

# Submit it for processing (returns task_id)
curl -sS -X POST "$MP_API_URL/v1/buckets/bkt_videos/batches/{batch_id}/submit" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"

# Poll until status is terminal: COMPLETED or COMPLETED_WITH_ERRORS
curl -sS "$MP_API_URL/v1/tasks/{task_id}" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE"
```

> [!TIP]
>
> Prefer the SDK? `client.tasks.get(task_id=...)` returns the same status. See [Monitoring ingestion](/docs/processing/tasks) for a complete polling loop.

## 5. Create a hybrid retriever

The retriever fuses two feature searches — visual and speech — over the same collection. Note the canonical field names: `collection_identifiers`, a flat `input_schema`, and the stage `config` wrapping `stage_id` + `parameters`.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "video-search",
    "collection_identifiers": ["video-moments"],
    "input_schema": {
      "query": { "type": "text", "required": true }
    },
    "stages": [
      {
        "stage_name": "hybrid_search",
        "stage_type": "filter",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "searches": [
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100,
                "weight": 0.6
              },
              {
                "feature_uri": "mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1",
                "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
                "top_k": 100,
                "weight": 0.4
              }
            ],
            "final_top_k": 20,
            "fusion": "weighted"
          }
        }
      }
    ]
  }'
```

| Field | Meaning |
| - | - |
| `collection_identifiers` | Collections to query (names or IDs). |
| `input_schema` | Flat map of input field → type. Reference inputs in stages as `{{INPUT.<field>}}`. |
| `searches[].feature_uri` | Which embedding index to search. Here: visual + speech. |
| `searches[].weight` | Per-search weight used by `weighted` fusion. |
| `final_top_k` | Results returned after fusion. |
| `fusion` | `rrf` (default) or `weighted`. |

## 6. Search

Execute the retriever. The `inputs` keys must match your `input_schema`.

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": { "query": "people discussing electric vehicles" }
  }'
```

Each result is a video segment with its timestamps and keyframe, ranked by combined visual + transcript relevance.

### Filter by category

Apply ad-hoc filters at execution time without changing the retriever:

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": { "query": "product demonstration" },
    "filters": {
      "field": "category",
      "operator": "eq",
      "value": "marketing"
    }
  }'
```

### Search by a reference image

Because `vertex_multimodal_embedding` is cross-modal, you can query the visual index with an image to find similar scenes. Add an image input to `input_schema` and set the visual search's `query` to `{ "input_mode": "content", "value": "{{INPUT.query_image}}" }`. See [Feature Search](/docs/retrieval/stages/feature-search) for image/URL query modes.

## Moment-level search

Segments carry start/end timestamps, so you can scope results to part of a video with a filter (field names depend on your collection's output schema):

```bash theme={null}
curl -sS -X POST "$MP_API_URL/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "X-Namespace: $MP_NAMESPACE" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": { "query": "pricing discussion" },
    "filters": { "field": "start_time", "operator": "gte", "value": 60.0 }
  }'
```

## Search the same video other ways

The same segments are searchable across every modality the `multimodal_extractor` produced — combine or swap these into your retriever:

> **[Search faces](/docs/processing/extractors/face-identity)**
>
> Add the `face_identity_extractor` to find every clip featuring a specific person (ArcFace 1:N).

> **[Search transcripts](/docs/retrieval/stages/feature-search)**
>
> Query `mixpeek://multimodal_extractor@v1/multilingual_e5_large_instruct_v1` to find moments by what's *said*.

> **[Group into moments](/docs/retrieval/stages/moment-group)**
>
> Merge matching frames into continuous time-ranges with the `moment_group` stage.

> **[Rerank for precision](/docs/retrieval/stages/rerank)**
>
> Add a cross-encoder `rerank` stage to sharpen the top results.

> **[Filter unsafe content](/docs/retrieval/stages/classify)**
>
> Drop or flag NSFW segments at query time with the `classify` stage.

> **[Hybrid keyword search](/docs/retrieval/stages/feature-search#lexical-bm25-search)**
>
> Add a `lexical: true` search to catch exact terms in transcripts (BM25).

### Search by action or activity

There is no dedicated action-recognition extractor — instead, search **actions as natural language** against the visual embedding. A text query like `"person running"` or `"two people shaking hands"` matches the `vertex_multimodal_embedding` because it understands actions depicted in frames:

```json theme={null}
{
  "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
  "query": { "input_mode": "text", "value": "person running on a treadmill" },
  "top_k": 50
}
```

For higher precision on a fixed set of actions, enable `run_video_description` / `response_shape` on the extractor to emit structured action labels you can then [`attribute_filter`](/docs/retrieval/stages/attribute-filter) or [`classify`](/docs/retrieval/stages/classify).

## Next steps

> **[Auto-tag segments](/docs/enrichment/taxonomies)**
>
> Classify segments ("product demo", "interview") with a taxonomy.

> **[Discover themes](/docs/enrichment/clusters)**
>
> Cluster segment embeddings to surface recurring visual themes.

> **[Get notified](/docs/enrichment/alerts)**
>
> Trigger alerts when new content matches a query.

> **[Tune feature search](/docs/retrieval/stages/feature-search)**
>
> Full reference for searches, fusion, and query modes.


# Bring Your Own Object Storage
Source: https://docs.mixpeek.com/docs/vector-store/byo-object-storage

Back the Mixpeek Vector Store with your own S3-compatible or GCS bucket on dedicated and self-hosted deployments

The Mixpeek Vector Store (MVS) persists everything it owns — the write-ahead log, snapshots, and the cold tier — to object storage through a provider-agnostic layer. On **dedicated and self-hosted deployments** you can point that layer at your own bucket, on any supported provider, so your vector data lives in storage you control.

> [!NOTE]
>
> This is about where MVS **stores its own data**, not where you ingest from. To connect a bucket as an ingestion *source*, see [Object Storage Integrations](/docs/integrations/object-storage/overview). The managed shared plane uses Mixpeek-operated storage; BYO object storage applies to [single-tenant](/docs/resources/single-tenant) and self-hosted deployments and is configured at the deployment level (not a per-namespace setting).

## Supported providers

MVS speaks the S3 API (via boto3) and the native GCS API. Any S3-compatible store works through the same code path — the only per-provider differences are the endpoint, region, and path-style addressing.

| Provider | `MVS_OBJECT_STORE_PROVIDER` | Docs |
| - | - | - |
| Amazon S3 | `s3` | [S3](/docs/integrations/object-storage/s3) |
| Google Cloud Storage | `gcs` | [GCS](/docs/integrations/object-storage/gcs) |
| Cloudflare R2 | `r2` | [R2](/docs/integrations/object-storage/r2) |
| Tigris | `tigris` | [Tigris](/docs/integrations/object-storage/tigris) |
| Wasabi | `wasabi` | [Wasabi](/docs/integrations/object-storage/wasabi) |
| Backblaze B2 | `b2` | [Backblaze](/docs/integrations/object-storage/backblaze) |
| MinIO | `minio` | S3-compatible |
| DigitalOcean Spaces | `spaces` | S3-compatible |
| Ceph | `ceph` | S3-compatible |
| LocalStack (dev) | `localstack` | S3-compatible |

See all providers in the [Object Storage overview](/docs/integrations/object-storage/overview).

Local and parallel filesystems, including Lustre, are in [private beta](#local-and-parallel-filesystems-private-beta).

## Configuration

The API, coordinator, and Ray worker pods read the object-store backend from these environment variables. The shard reads its own set, listed under [Shard settings](#shard-settings).

| Variable | Default | Description |
| - | - | - |
| `MVS_OBJECT_STORE_PROVIDER` | `gcs` | Provider key from the table above. |
| `MVS_BUCKET` | — | Bucket that holds all MVS data. (Legacy alias: `MVS_GCS_BUCKET` — both must name the same bucket.) |
| `MVS_OBJECT_STORE_ENDPOINT` | — | Custom S3 endpoint. **Required** for R2, Tigris, and Wasabi; optional for MinIO/Ceph. |
| `MVS_REGION` | — | Region for S3-compatible providers. |
| `MVS_PREFIX` | `mvs/` | Global key prefix for all objects. |
| `MVS_ENCRYPTION_KEY` | — | Optional base64-encoded 32-byte key to encrypt all data at rest (see below). |

> [!WARNING]
>
> The coordinator and the Rust shard must read the **same** bucket. Set `MVS_BUCKET` (or `MVS_GCS_BUCKET`) consistently across both — a split here causes cross-tenant namespace discovery. MVS fails fast if the two env vars disagree.

### Credentials

Credentials are never stored in config — they're resolved from the deployment environment:

* **S3-compatible:** the standard AWS credential chain. Prefer an **IAM role** attached to the pods; otherwise set `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`. R2/Tigris/Wasabi/B2 issue S3 access keys you use the same way.
* **GCS:** Application Default Credentials (workload identity / `GOOGLE_APPLICATION_CREDENTIALS`), or a service-account JSON supplied as a secret.

### Endpoint and addressing notes

* **Endpoint required:** R2 (`https://<account_id>.r2.cloudflarestorage.com`), Tigris, and Wasabi have no usable default — set `MVS_OBJECT_STORE_ENDPOINT` or MVS fails fast with a clear message rather than silently hitting AWS.
* **Path-style addressing** is applied automatically for B2, MinIO, Ceph, and LocalStack (they only serve path-style); AWS/R2/Tigris/Wasabi use virtual-hosted style.

### Shard settings

Set these on the MVS shard, and point them at the same bucket as the variables above.

**With the Helm chart**, set `mvs.objectStore` in your values file. The chart writes the shard variables for you:

```yaml theme={null}
mvs:
  objectStore:
    backend: s3                          # "s3" or "gcs"
    bucket: my-mvs-data
    endpoint: https://minio.internal:9000  # S3-compatible stores only
    region: us-east-1                    # optional
    secretRef: mvs-object-store-creds    # Secret with the access keys
```

The chart accepts only `s3` and `gcs`. Use `s3` for MinIO, R2, and every other S3-compatible store. Keep credentials out of the values file. Put them in the Secret that `secretRef` names, as `MVS_S3_ACCESS_KEY_ID` and `MVS_S3_SECRET_ACCESS_KEY` (or `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`).

**Without the chart**, set these on each shard container:

| Variable | Backend | Default | Description |
| - | - | - | - |
| `MVS_STORE_BACKEND` | all | `none` | `s3` or `gcs`. With `none`, the shard starts empty and persists nothing. |
| `MVS_S3_BUCKET` | `s3` | `mvs-snapshots` | Bucket name. |
| `MVS_S3_ENDPOINT` | `s3` | (unset) | Endpoint for S3-compatible stores. |
| `MVS_S3_REGION` | `s3` | `us-east-1` | Region. |
| `MVS_S3_PREFIX` | `s3` | `mvs` | Key prefix. |
| `MVS_S3_ACCESS_KEY_ID` / `MVS_S3_SECRET_ACCESS_KEY` | `s3` | (unset) | Access keys. Falls back to `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`. |
| `MVS_GCS_BUCKET` | `gcs` | `mvs-snapshots` | Bucket name. Credentials come from Application Default Credentials. |

> [!WARNING]
>
> Set `MVS_STORE_BACKEND` explicitly. If you leave it unset, the shard boots without an error, serves an empty index, and writes no snapshots.

## Local and parallel filesystems (private beta)

> [!NOTE]
>
> MVS can store its data on a POSIX filesystem, such as local NVMe, NFS, or Lustre. This backend is in private beta and opt-in. To enable it for your deployment, [contact us](https://mixpeek.com/contact).

## Encryption at rest

Set `MVS_ENCRYPTION_KEY` (base64-encoded 32 bytes) to transparently encrypt every object with **AES-256-GCM**. The object's storage key is used as additional authenticated data (AAD), binding each ciphertext to its path. Migration needs no backfill: existing unencrypted objects stay readable while new writes are encrypted, so you can enable it on a populated bucket without a backfill.

## Examples

```bash Amazon S3 theme={null}
MVS_OBJECT_STORE_PROVIDER=s3
MVS_BUCKET=my-mvs-data
MVS_REGION=us-east-1
# credentials via attached IAM role (recommended) or AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY
```

```bash Cloudflare R2 theme={null}
MVS_OBJECT_STORE_PROVIDER=r2
MVS_BUCKET=my-mvs-data
MVS_OBJECT_STORE_ENDPOINT=https://<account_id>.r2.cloudflarestorage.com
AWS_ACCESS_KEY_ID=<r2_access_key>
AWS_SECRET_ACCESS_KEY=<r2_secret_key>
```

```bash Google Cloud Storage theme={null}
MVS_OBJECT_STORE_PROVIDER=gcs
MVS_BUCKET=my-mvs-data
# credentials via workload identity / GOOGLE_APPLICATION_CREDENTIALS
```

```bash MinIO (self-hosted) theme={null}
MVS_OBJECT_STORE_PROVIDER=minio
MVS_BUCKET=my-mvs-data
MVS_OBJECT_STORE_ENDPOINT=https://minio.internal:9000
AWS_ACCESS_KEY_ID=<minio_access_key>
AWS_SECRET_ACCESS_KEY=<minio_secret_key>
```

## Write extraction outputs to your bucket

A bucket can name a `destination`: a storage connection you own with write access enabled. Extraction outputs and derived assets (segments, thumbnails, transcodes, renders) are then written under your prefix, each object carrying its lineage as object metadata, and Mixpeek's storage keeps no copy. This works on every deployment, including the shared plane.

| Field | Type | What it does |
| - | - | - |
| `destination.connection_id` | string | A storage connection you own (S3-compatible, including Cloudflare R2 through its `endpoint_url`, or GCS) with `write_enabled: true` |
| `destination.prefix` | string | Key prefix inside the connection's bucket, for example `mixpeek-outputs/`. Empty writes at the root; a trailing slash is added if you omit it |

Enable writes on the connection first:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/organizations/connections/$CONNECTION_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"write_enabled": true}'
```

Then bind it on the bucket, on create or update:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/buckets/$BUCKET_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{"destination": {"connection_id": "'$CONNECTION_ID'", "prefix": "mixpeek-outputs/"}}'
```

A connection without write access is refused naming the connection.

## Related

* [Object Storage Integrations](/docs/integrations/object-storage/overview) — connect a bucket as an ingestion source
* [Single-Tenant Deployments](/docs/resources/single-tenant) — fully isolated data plane with a dedicated bucket
* [Vector Store Overview](/docs/vector-store/overview) — using MVS as a standalone vector store


# Promote to Managed
Source: https://docs.mixpeek.com/docs/vector-store/promote

Transition from standalone vectors to auto-embedding and collections — no reindexing required

Promotion converts a standalone namespace to managed mode. Existing vectors and documents stay in place. Mixpeek adds auto-embedding for queries and collection-driven processing for new content.

![Promotion flow — standalone namespace with vectors becomes managed with inference mappings, no reindexing](https://mintcdn.com/mixpeek/PRYIJzrWYd-bekCW/assets/vector-store/promote-flow.svg?fit=max\&auto=format\&n=PRYIJzrWYd-bekCW\&q=85\&s=64ddb66b70acdf8b747aba018d43fd95)

## Promote

Map existing vector indexes to inference services and optionally add new ones.

```python Python theme={null}
from mixpeek import Mixpeek

client = Mixpeek(api_key="mxp_sk_...")

client.namespaces.promote(
    namespace_id="product-search",
    vector_mappings=[
        {"existing_index": "text_embedding", "inference_service": "openai_text_3_small"}
    ],
    add_vectors=[
        {"name": "image_embedding", "dimension": 512, "metric": "cosine", "inference_service": "clip_vit_b32"}
    ],
)
```

```bash cURL theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/product-search/promote" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "vector_mappings": [
      {"existing_index": "text_embedding", "inference_service": "openai_text_3_small"}
    ],
    "add_vectors": [
      {"name": "image_embedding", "dimension": 512, "metric": "cosine", "inference_service": "clip_vit_b32"}
    ]
  }'
```

### Parameters

**vector\_mappings** `array`

Map existing indexes to inference services for auto-embedding queries.

| Field | Description |
| - | - |
| `existing_index` | Name of an existing vector index |
| `inference_service` | Model to auto-embed queries (e.g., `openai_text_3_small`) |

**add\_vectors** `array`

New vector indexes to create during promotion.

| Field | Description |
| - | - |
| `name` | New index name (must not conflict with existing) |
| `dimension` | Embedding dimension |
| `metric` | `cosine`, `dot`, or `euclidean` |
| `inference_service` | Model for auto-embedding (optional) |

### Response

```json theme={null}
{
  "namespace_id": "product-search",
  "previous_mode": "standalone",
  "mode": "managed",
  "status": "active",
  "vector_configs": [
    {"name": "text_embedding", "dimension": 1536, "metric": "cosine"},
    {"name": "image_embedding", "dimension": 512, "metric": "cosine"}
  ],
  "vector_inference_map": {
    "text_embedding": "openai_text_3_small",
    "image_embedding": "clip_vit_b32"
  }
}
```

## What Changes

| | Before (standalone) | After (managed) |
| - | - | - |
| **Search input** | Must provide pre-computed vectors | Also accepts raw text/URLs — auto-embedded |
| **New content** | Direct upsert only | Collections + extractors auto-process |
| **Existing data** | — | Preserved, no reindexing |
| **Direct upsert** | — | Still works alongside collections |

## Rules

> [!WARNING]
>
> Promotion is one-way. Managed namespaces cannot be demoted.

* Only `standalone` namespaces can be promoted
* `existing_index` must reference an index that exists — errors list available indexes
* `name` in `add_vectors` must not conflict with existing indexes
* Promotion is atomic — if validation fails, nothing changes
* Both `vector_mappings` and `add_vectors` are optional (promote with just one or neither)

## Full Workflow

**1. Start standalone**

Create a namespace and upsert your existing embeddings.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/standalone" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"namespace_id": "legal-docs", "vector_configs": [{"name": "text_embedding", "dimension": 1536, "metric": "cosine"}]}'
```

**2. Load and validate**

Upsert documents and verify retrieval quality before promoting.

```bash theme={null}
# Upsert
curl -X POST "https://api.mixpeek.com/v1/namespaces/legal-docs/documents/upsert" ...

# Execute a retriever to validate (input_mode: vector — you pass the query embedding)
curl -X POST "https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute" \
  -H "X-Namespace: legal-docs" \
  -d '{"inputs": {"query_vector": [...]}}'
```

**3. Promote**

Map your embedding to the matching inference service.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/namespaces/legal-docs/promote" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"vector_mappings": [{"existing_index": "text_embedding", "inference_service": "openai_text_3_small"}]}'
```

**4. Switch queries to text input**

Your retrievers already run before and after promotion — no API change. Update the query stage from `input_mode: vector` (you pass the embedding) to `input_mode: text` so Mixpeek auto-embeds. See [Querying with Retrievers](#querying-with-retrievers) below.

**5. Use managed features**

Create collections with extractors for new content. Existing documents coexist with extractor-processed documents.

## Querying with Retrievers

Querying is unified on **retrievers** in both standalone and managed modes — you learn one query concept regardless of whether you bring your own vectors or let Mixpeek embed for you. Promotion doesn't change *how* you query; it only changes *what you pass in*. A standalone retriever takes the query vector you computed (`input_mode: vector`); after promotion the same retriever can take raw text and auto-embed it (`input_mode: text`).

### Before promotion: pass your own vector

The query stage runs in `input_mode: vector` — you compute the embedding and pass it in `inputs`.

```bash theme={null}
# Step 1 — Create a retriever (one-time)
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: product-search" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product_search",
    "input_schema": {"query_vector": {"type": "array", "required": true}},
    "stages": [{
      "stage_name": "search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [{
            "feature_uri": "text_embedding",
            "query": {"input_mode": "vector", "value": "{{INPUT.query_vector}}"},
            "filters": {"must": [{"key": "category", "match": {"value": "audio"}}]},
            "top_k": 10
          }],
          "final_top_k": 10
        }
      }
    }]
  }'

# Step 2 — Execute with your query embedding
curl -X POST "https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: product-search" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {"query_vector": [0.12, -0.34, 0.56, "...1536 floats"]}
  }'
```

### After promotion: pass raw text

Once `text_embedding` is mapped to an inference service, switch the query stage to `input_mode: text`. Mixpeek embeds the text for you — no vectors needed.

```bash theme={null}
# Step 1 — Create (or update) the retriever to take text
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: product-search" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "product_search",
    "input_schema": {"q": {"type": "string", "required": true}},
    "stages": [{
      "stage_name": "search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": {
          "searches": [{
            "feature_uri": "text_embedding",
            "query": {"input_mode": "text", "value": "{{INPUT.q}}"},
            "filters": {"must": [{"key": "category", "match": {"value": "audio"}}]},
            "top_k": 10
          }],
          "final_top_k": 10
        }
      }
    }]
  }'

# Step 2 — Execute with raw text (auto-embedded)
curl -X POST "https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: product-search" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {"q": "wireless headphones"}
  }'
```

Each hit in the response exposes `document_id` plus the document's payload fields.

### What Changes

| | Standalone (`input_mode: vector`) | Managed (`input_mode: text`) |
| - | - | - |
| **Query input** | Raw vectors you computed (also text/BM25, sparse) | Raw text, URLs, filters — auto-embedded |
| **Embedding** | You compute and pass vectors | Retriever stages auto-embed via inference services |
| **Query path** | Retriever create + execute | Retriever create + execute — same API |
| **Pipeline** | Single search stage + optional fusion | Multi-stage: search → filter → enrich → rerank → transform |
| **Hybrid search** | Multiple `feature_search` searches + fusion | Multiple `feature_search` stages + sort/reduce stages |

> [!TIP]
>
> Promotion is additive — your retrievers keep working. Flip the query stage to `input_mode: text` whenever you're ready to let Mixpeek handle embedding, and layer in multi-stage pipelines as your needs grow.

## Related

* [Vector Store Overview](/docs/vector-store/overview)
* [Namespace Configuration](/docs/vector-store/namespaces)
* [Collections](/docs/ingestion/collections) — set up auto-processing after promotion
* [Feature Extractors](/docs/processing/feature-extractors) — available extractors


# Storage format (.mvs)
Source: https://docs.mixpeek.com/docs/vector-store/storage-format

How Mixpeek stores your vectors and documents in your object storage bucket, in a documented format you can read without us.

Mixpeek stores each namespace as a `.mvs` dataset: a documented tree of objects in an object-storage bucket. There is no proprietary storage service in the path. The same layout runs on GCS, S3, MinIO, or your own hardware.

A `.mvs` dataset is a prefix, not a single file. Ranged reads, parallel restore, and lazy per-partition loading all depend on that shape.

## The layout

```
<bucket>/mvs/snapshots/<namespace>/<shard>/latest/
    meta.json               snapshot descriptor
    payloads.bin            document payload store
    keyword_indexes.bin     filter index image
    namespace_index.bin     namespace routing index
    indexes/<name>/
      manifest.bin          declares every partition by filename
      partitions/           one blob per index partition
<bucket>/mvs/wal/<namespace>/<shard>/
    <seq>.wal               write-ahead log segments
    highwater.json          highest shipped sequence
<bucket>/mvs/leader/<namespace>/<shard>/leader.json
```

`meta.json` carries a format identifier (`lire_rs_v1` today). Readers refuse identifiers they do not recognize, so a future format change cannot silently misread your data.

## What the format guarantees

Eight invariants define a valid dataset. The load-bearing ones:

* **Pairing.** Every indexed point has both an id and a vector entry. A point cannot be countable but unsearchable.
* **Parts sum to whole.** Partition row counts add up to the manifest's total, and the manifest's total reconciles with the payload store.
* **Manifest honesty.** Every partition the manifest declares exists as an object, by name. Vanished blobs surface as findings, not as silently missing results.
* **Durability frontier.** The snapshot's replay floor never exceeds the shipped write-ahead log. Recovery can always replay forward from a snapshot to the last shipped write.
* **Checksums.** The snapshot image carries an inline SHA-256 trailer. Restore refuses a mismatch.

Mixpeek validates namespaces against these invariants with an internal checker. Each check exists because a real production incident motivated it.

## Why this matters to you

**No lock-in.** Your data sits in your bucket in a documented layout. A tool that implements this page can read it without Mixpeek in the path.

**Auditability.** Regulated teams can verify what is stored, where, and whether it is internally consistent, from the objects alone.

**Portability.** Copying a namespace is an object copy of three trees. The format validates identically at the source and the destination, on any object store.

**Tiering without translation.** Warm and cold data use the same layout at different object-storage classes. Nothing is rewritten when data moves between tiers.

## Status

The format specification is at v1. The identifier `lire_rs_v1` ships in every production `meta.json` today, so v1 describes what is already stored, not a migration target. Enterprise customers can request the full byte-level specification.

> [!NOTE]
>
> The write path stays exclusive to Mixpeek: all ingestion flows through the API and processing pipeline. The format documents what we store so you can always read, verify, and move your own data.


# System Collections
Source: https://docs.mixpeek.com/docs/vector-store/system-collections

The mxp_ collections Mixpeek provisions per namespace, what each stores, and what you can do with it

Mixpeek maintains a set of system collections in every namespace, named with the `mxp_` prefix.
Each is created lazily. It appears the first time a matching event occurs, not at namespace setup.
You can filter, retrieve, cluster, and export them with the same API you use on any other collection.

The `_`, `mxp_` and `mvs_bucket_` prefixes are reserved. Creating a collection under any of them fails
with a 422 that names the reserved prefixes. `mvs_bucket_` belongs to the
[bucket-backed collections](#bucket-backed-collections) below.

***

## Bucket-backed collections

Every bucket has one more system collection, named after the bucket:
`mvs_bucket_<bucket_id>`. It holds one document per object in that bucket, so
the bucket's objects are searchable, filterable and alertable through the same
API as any other collection.

**You do not create it.** A bucket gets its collection when the bucket is
created, and a bucket that predates this gets it the first time a bucket id is
named where a collection id goes:

```bash theme={null}
curl -X POST "$API/v1/retrievers" \
  -H "Authorization: Bearer $KEY" -H "X-Namespace: $NS" \
  -d '{
    "retriever_name": "over-my-bucket",
    "collection_identifiers": ["bkt_dc47fce8"],
    "stages": [{"stage_name": "s1", "config": {
      "stage_id": "attribute_filter",
      "parameters": {"field": "object_id", "operator": "exists", "value": true}
    }}]
  }'
```

The response comes back with the resolved collection id, and the collection
appears in `collections/list` as `mvs_bucket_bkt_dc47fce8`. Studio hides these
collections behind their own filter chip, which you can clear. Referencing the
same bucket again reuses the collection. The `mvs_bucket_...` name works
anywhere the bucket id does.

**It is payload-only.** The collection declares no vector indexes, so it costs
no embedding and no GPU. `vector_indexes` on the collection is `[]`.

**It is written for you.** Creating, updating or deleting an object updates its
document in the same request. You cannot target the collection with a batch
action or create another collection under its name.

### What is on a backing document

| on the object | on its backing document |
| - | - |
| `object_id`, `bucket_id` | `object_id`, `bucket_id`, at the root |
| `key_prefix`, `blobs`, `source_details` | the same fields, at the same paths |
| `metadata.<your field>` | `metadata.<your field>`, filterable at the same path |
| `status`, `error`, `error_type` | `_internal.object.status`, `_internal.object.error`, `_internal.object.error_type` |
| (lineage) | `_internal.lineage.root_object_id` and `_internal.lineage.root_bucket_id` |

`object_id`, `key_prefix` and `_internal.object.status` carry keyword indexes,
so filters on them stay fast on large buckets.

> [!WARNING]
>
> **A filter on a field the document does not carry returns an empty set, not an error.**
> HTTP 200 with zero documents reads as "the bucket is empty" rather than
> "that field does not exist". Use the paths in the table above. To check a field
> before you filter on it, read one document with
> `POST /v1/collections/{id}/documents/list` and look at the key.

Each object produces exactly one document, so for a bucket B:

```
objects(B) = documents(mvs_bucket_<B>)
```

Read the object count from `GET /v1/buckets/{bucket_id}` (`object_count`) and
the document count from `documents/list` with `include_total=true`. A failed
object keeps its document, with the failure under `_internal.object.status` and
`_internal.object.error`. Filter on `_internal.object.status` equal to `FAILED`
to list them; the total matches the bucket's `failed_object_count`.

### Filtering and alerting on a bucket

Everything that works on a collection works on the bucket:

```bash theme={null}
# every object in the bucket whose metadata says brand = acme
curl -X POST "$API/v1/collections/mvs_bucket_bkt_dc47fce8/documents/list?include_total=true" \
  -H "Authorization: Bearer $KEY" -H "X-Namespace: $NS" \
  -d '{"filters": {"AND": [
        {"field": "metadata.brand", "operator": "eq", "value": "acme"}]}}'
```

Point a retriever at it, put an alert on that retriever, or build a tier-1
collection sourced from it.

> [!NOTE]
>
> Deleting a bucket object cascades: the backing document and every document
> derived from it are removed with it.

***

## What is writing today

Lazy creation means an absent collection is usually just an event you have not triggered. For some of these it means something else: nothing writes them yet.

> [!WARNING]
>
> **Three of these have no writer reaching them today**, so they will not appear however long you wait.
>
> | Collection | Why it is empty |
> | - | - |
> | `mxp_batch_failures` | Written from the engine, which has no write path to collections |
> | `mxp_infra_metrics` | The scrape task is in no schedule |
> | `mxp_healing_events` | No production caller |
>
> Read the sections below as what each collection is for, not as something you can query now.

> [!NOTE]
>
> **Two are named differently on namespaces that predate the rename.** If you have existing data and the documented name returns nothing, look under the old one.
>
> | Documented | Older namespaces |
> | - | - |
> | `mxp_retriever_evaluations` | `mxp_evaluations` |
> | `mxp_compute_utilization` | `mxp_resources` |
>
> There is no alias, so the two names hold separate data.

Writes are best-effort. A write that cannot reach the collection is dropped and logged rather than failing the request that triggered it, so a gap here never breaks the operation it describes.

***

## Retriever signal collections

These four collections record what your retrievers do. They back [learned fusion](/docs/relevance/learned-fusion)
and are the input to [evaluations](/docs/retrieval/evaluations).

### `mxp_retriever_interactions`

Stores every interaction a user reports on a search result: clicks, dwells, bookmarks, downloads.

**What you can do with it:**

* Study the signals the [Thompson-sampling bandit](/docs/relevance/learned-fusion) learns from, which
  shift retriever weights toward features your users engage with. With analytics enabled the bandit
  reads its state from the analytics store's copy of these interactions, so deleting rows here does
  not reset it. Without analytics it reads this collection, cached for 60 seconds, and deleting rows
  here changes what it learns from on the next refresh.
* Filter by `interaction_type` to measure click-through rate per retriever or per feature.
* Export to a reranker training set: cluster by `session_id`, then label by `interaction_type`.

> [!NOTE]
>
> This is the primary input to relevance tuning. A namespace with no interaction data
> falls back to equal-weight fusion across retriever stages.

### `mxp_retriever_queries`

Stores every query submitted to your retrievers, with the query embedding.

**What you can do with it:**

* Cluster by query embedding to discover the most common search intents in your namespace.
* Filter on `results_count = 0` to find queries that return nothing — then decide whether to
  expand your extractor coverage or tune your retriever stages.
* Join with `mxp_retriever_executions` on `execution_id` to correlate query shape with latency.

### `mxp_retriever_executions`

Stores one document per retriever run, with status, latency, result count, and execution mode.

**What you can do with it:**

* Monitor retriever health: filter on `status = degraded` or `status = timeout` to surface
  failures without reading server logs.
* Track latency over time: retrieve by `retriever_id` and aggregate `duration_ms` with a
  rollup stage.
* Alert on spikes: wire an alert against this collection with a `duration_ms` threshold.

### `mxp_retriever_rollups`

Stores pre-aggregated health windows per retriever and per namespace.

**What you can do with it:**

* Build dashboards without running aggregation queries on the raw execution log.
* Use `scope = retriever` for per-retriever panels; use `scope = namespace` for namespace-wide health.
* Compare `window_start_ts` ranges to detect degradation trends across deployments.

> [!NOTE]
>
> Rows appear after the rollup sweep runs, which is every 15 minutes.
>
> `baseline_*` and `*_delta` stay null until the namespace has 24 hours of history
> (`SIGNALS_ROLLUP_BASELINE_HOURS`). The degradation comparison above needs that
> history, so a new namespace gets rows with those fields empty.

***

## Evaluation and annotation collections

### `mxp_retriever_evaluations`

Stores the result of each evaluation run: nDCG, precision, recall, and status.

**What you can do with it:**

* Track retriever quality over time after tuning, configuration changes, or new extractors.
* Compare `evaluation_id` across runs on the same `dataset_id` to confirm an improvement is real.
* Fail a CI gate: query this collection after each deploy and assert nDCG did not drop.

### `mxp_document_annotations`

Stores every human label applied to a document, with the annotator and target document.

**What you can do with it:**

* Use as ground truth for [supervised relevance tuning](/docs/relevance/annotations): the evaluation
  pipeline reads this collection to score retriever output against your labels.
* Audit annotation coverage: filter by `retriever_id` to see which results have been rated.
* Train a reranker: export labels keyed on `target_document_id` as positive or negative signal.

***

## Platform event collections

### `mxp_platform_events`

Stores lifecycle events for resources in your namespace: buckets, collections, extractors created
or deleted.

**What you can do with it:**

* Trigger outbound webhooks on resource changes: any event written here fires your configured
  webhook URL automatically.
* Audit your namespace history: query by `event_type = collection_created` to see when pipelines
  were added.
* Detect unexpected deletes: alert when `event_type = bucket_deleted` appears.

### `mxp_change_events`

Retired in October 2026. Namespaces that already have this collection keep their existing rows,
and no new rows are added.

Read the ordered change feed from `GET /v1/events`. It covers your whole organization with one
`seq` cursor, takes an optional `namespace_id` filter, and keeps events after a namespace is
deleted. Poll it with the `next_cursor` from your last page to replay history or drive a sync to
an external system.

***

## Compute and infrastructure collections

### `mxp_compute_utilization`

Stores per-batch compute cost: documents processed, duration, and throughput.

**What you can do with it:**

* Get an email alert when throughput drops below 0.5 docs/sec or a batch runs longer than 1 hour.
  Mixpeek sends this automatically for anomalous batches.
* Filter by `measured_collection_id` to compare ingestion cost across your collections.
* Identify expensive extractors: join with `mxp_batch_failures` on `batch_id` to see whether
  high-cost batches also have high failure rates.

### `mxp_infra_metrics`

Stores infrastructure metric snapshots: CPU, memory, pod count, queue depth.

**What you can do with it:**

* Trigger scale-up alerts: Mixpeek fires an outbound webhook when `engine_pod_count` drops below
  your configured floor (`INFRA_MIN_ENGINE_PODS`).
* Retrieve historical metrics to understand whether a slow batch correlated with resource pressure.
* Filter by `source = prometheus` or `source = ray` to separate infrastructure metrics from
  local-dev estimates.

### `mxp_batch_failures`

Stores one document per object that failed extraction, with error type, extractor name, and retry
eligibility.

**What you can do with it:**

* Find which objects failed in a batch: filter by `batch_id` and read `error_type` and
  `error_message` to understand the cause without reading server logs.
* Identify systemic failures: filter by `collection_id` across batches to find extractors that
  fail consistently on a particular collection.
* Drive targeted retries: filter on `can_retry = true` and `retry_count < 3` to get the set
  worth re-submitting.

***

## Operational collections

### `mxp_webhook_deliveries`

Stores the delivery history of every webhook event your namespace generated. A row with
`status = pending` is written when the event is queued. Each delivery attempt then writes its
own row with the outcome: `completed`, `retry` (another attempt is scheduled) or `failed` (the
retry ladder is exhausted or the webhook is inactive), plus `retry_count` and `last_error`.

**What you can do with it:**

* Verify delivery: filter by `event_id` to confirm a specific webhook fired and succeeded.
* Debug failures: filter on `status = failed` and read `last_error` to see what the endpoint
  returned.
* Monitor retry depth: filter on `retry_count >= 3` to find events stuck in the retry ladder
  before they reach the dead-letter queue.

> [!NOTE]
>
> Rows written before attempt rows shipped in October 2026 carry `status = pending` only.

### `mxp_healing_events`

Stores every auto-repair action the platform took on your namespace: stalled job cancellations,
circuit breaker trips, batch repairs.

**What you can do with it:**

* Audit platform interventions: query by `affected_resource` to see what the platform touched
  and why.
* Correlate with batch failures: join on `batch_id` to see whether a batch failure was preceded
  by a healing action.
* Monitor recurrence: if `source = circuit_breaker` appears repeatedly for the same resource,
  that is a signal to investigate the upstream service.

### `mxp_alert_executions`

Stores one document per alert evaluation: which alert fired, the condition that was met, and
the severity.

**What you can do with it:**

* Review alert history: filter by `alert_id` to see every time an alert fired, without reading
  the alert configuration.
* Suppress noise: query for repeated `alert_id` occurrences in a short window to identify
  alerts that need threshold adjustment.
* Build alert analytics: cluster by `severity` and `retriever_id` to find which retrievers
  produce the most alerts.

> [!WARNING]
>
> **A row is written when an alert is created, not when it fires.** `condition_met` is set to `true` on that row regardless, and nothing writes on a later firing.
>
> So this collection currently counts alerts you configured. Treat it as a record of alert creation until firing is wired up.

***

## Reference table

| Collection | Created when | Signal type |
| - | - | - |
| `mxp_retriever_interactions` | First interaction reported | `retriever_interaction` |
| `mxp_retriever_queries` | First retriever query runs | `retriever_query` |
| `mxp_retriever_executions` | First retriever execution | `retriever_execution` |
| `mxp_retriever_rollups` | First rollup computed | `retriever_rollup` |
| `mxp_retriever_evaluations` | First evaluation run | `retriever_evaluation` |
| `mxp_document_annotations` | First annotation created | `document_annotation` |
| `mxp_platform_events` | First resource lifecycle event | `platform_event` |
| `mxp_change_events` | Retired October 2026, no new rows | `change_event` |
| `mxp_compute_utilization` | First batch completes | `compute` |
| `mxp_infra_metrics` | First infra metric scraped | `infra_metric` |
| `mxp_batch_failures` | First extraction failure | `batch_failure` |
| `mxp_webhook_deliveries` | First webhook fires | `webhook_delivery` |
| `mxp_healing_events` | First auto-heal action | `healing_event` |
| `mxp_alert_executions` | First alert fires | `alert_execution` |

Query any of these collections with `POST /v1/collections/{collection_name}/documents/list`,
filtering on `signal_type` and any payload field. All standard retriever stages work against them.

## `universal_extractor`

Provisioned at namespace creation, alongside the `default` namespace, an `uploads` bucket, and a
`search` retriever. When you upload files from Studio, Mixpeek runs the universal extractor over
them: multimodal embeddings at 3072 dimensions plus generated descriptions.

This collection carries no system prefix. You can rename, reconfigure, or delete it like any
collection you created yourself.


# Storage Tiering
Source: https://docs.mixpeek.com/docs/vector-store/tiering

Set the tiering override on a namespace or collection: turn it on or off, choose the idle windows, or exempt the data with a reason

Mixpeek demotes idle data to cheaper storage on a schedule. The platform declares the default windows per data class. A namespace can override them, and a collection can override its namespace. Any part you leave unset inherits from the level above.

## The `tiering` field

Namespaces and collections take the same `tiering` object on create and on update (`PATCH /v1/namespaces/{namespace_id}`, `PATCH /v1/collections/{collection_id}`).

| Field | Type | What it does |
| - | - | - |
| `enabled` | boolean | Turn tiering on or off here. Unset inherits from above. |
| `idle_days.cold` | integer, at least 1 | Days without a use before the cold tier. |
| `idle_days.frozen` | integer, more than `cold` | Days without a use before the frozen tier. |
| `exempt` | string | Opt out, with the reason (for example a legal hold). An exempt namespace or collection is never demoted. An empty reason is refused. |

## Examples

Keep a collection hot for 90 days and frozen after a year:

```bash cURL theme={null}
curl -X PATCH https://api.mixpeek.com/v1/collections/$COLLECTION_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{"tiering": {"idle_days": {"cold": 90, "frozen": 365}}}'
```

```python Python theme={null}
client.collections.update(
    collection_id=COLLECTION_ID,
    tiering={"idle_days": {"cold": 90, "frozen": 365}},
)
```

```javascript JavaScript theme={null}
await client.collections.update(COLLECTION_ID, {
  tiering: { idle_days: { cold: 90, frozen: 365 } },
});
```

Exempt a namespace under a legal hold:

```bash theme={null}
curl -X PATCH https://api.mixpeek.com/v1/namespaces/$NAMESPACE_ID \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tiering": {"exempt": "legal hold, matter 2026-014"}}'
```

The older collection field `tiering_rules` is retired. It is returned on reads so existing records still parse; a write that sets it is refused. Use `tiering`.

## Related

* [Collection Lifecycle](/docs/vector-store/collection-lifecycle) for moving a whole collection between index tiers by hand
* [Storage Format](/docs/vector-store/storage-format) for what each tier stores


# Bulk-submit a whole bucket as N auto-chunked, auto-queued batches
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/bulk-submit-whole-bucket-n-auto

post /v1/buckets/{bucket_identifier}/batches/bulk-submit
One call that streams the bucket's objects (paginated, de-duplicated), chunks them server-side into `chunk_size`-object batches, and submits each (accept-and-queue). Replaces the client-side chunk loop + cursor pacing — no 409 cursor-overlap, no 5k count-timeout, no manual 429 handling. Poll the returned batch ids / `batch_group_id` for QUEUED -> PROCESSING.



# Cancel Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/cancel-batch

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/cancel
Cancel a submitted/processing batch: cancels Ray job via engine and marks task/batch as CANCELLED.



# Cancel Job
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/cancel-job

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/jobs/{ray_job_id}/cancel
Cancel a single Ray job within a tier without affecting sibling jobs.



# Cancel Tier
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/cancel-tier

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/cancel
Cancel a single tier within a batch without affecting other tiers. Cancels all Ray jobs and queued work for this tier and marks it CANCELED.



# Delete Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/delete-batch

delete /v1/buckets/{bucket_identifier}/batches/{batch_id}
Delete a batch by its ID.



# Delete Documents Produced by Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/delete-documents-produced-by-batch

delete /v1/buckets/{bucket_identifier}/batches/{batch_id}/documents
Delete all documents that were produced by this batch. Useful for cleaning up documents from batches that produced bad data (e.g. null vectors). Only works for terminal batches.



# Diagnose Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/diagnose-batch

get /v1/buckets/{bucket_identifier}/batches/{batch_id}/diagnose
Aggregate diagnostic information for a batch: status, timing, errors, infrastructure events, progress, failed documents, and recommendations.



# Estimate Batch Cost (pre-flight, dry-run)
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/estimate-batch-cost

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/estimate-cost
Pre-flight estimate of the credits a DRAFT batch will consume on submit, plus how many of its objects were already extracted — so a force/replace re-run that re-pays GPU extraction surfaces its cost BEFORE submitting. Does not consume credits or change the batch's contents/status; it DOES refresh a DRAFT batch's activity timestamp so the draft-TTL reaper grants a fresh grace window while you confirm the cost.



# Get Failed Documents for Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/get-failed-documents-for-batch

get /v1/buckets/{bucket_identifier}/batches/{batch_id}/failed-documents
Retrieve failed documents for a batch, optionally filtered by tier or collection.



# Get Ray Job Logs for Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/get-ray-job-logs-for-batch

get /v1/buckets/{bucket_identifier}/batches/{batch_id}/logs
Retrieve Ray job submission logs for a batch's processing tiers. You can get logs for a specific tier or all tiers. User must have access to the batch to retrieve logs.



# Manual Self-Healing
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/heal-tier

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/heal
Trigger a self-healing action on a tier: kill duplicate jobs, detect stuck jobs, sync extractor status, or check the circuit breaker.



# List Batches
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/list-batches

post /v1/buckets/{bucket_identifier}/batches/list
List batches with pagination and filtering options.



# Resubmit Failed Batch
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/resubmit-failed-batch

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/resubmit
Re-run a terminal (FAILED/CANCELED/COMPLETED/…) batch: resets it to DRAFT and auto-submits by default, so the verb matches the behavior. Pass auto_submit=false to only park it as DRAFT — it must then be submitted manually before the draft TTL expires.



# Retry Failed Documents
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/retry-failed-documents

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/retry
Retry failed documents in a batch with intelligent filtering by error type and tier.



# Retry Tier with Resource Overrides
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/retry-tier

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/retry
Retry a failed or canceled tier, optionally with modified resource parameters. Useful for retrying OOM failures with more memory.



# Submit All Draft Batches
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/submit-all-draft-batches

post /v1/buckets/{bucket_identifier}/batches/submit-all
Submit all DRAFT batches in this bucket for processing in a single request. Bypasses per-request rate limiting. Each batch is submitted sequentially and subject to the same admission control as individual submit.



# Submit Batch for Processing
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/submit-batch-for-processing

post /v1/buckets/{bucket_identifier}/batches/{batch_id}/submit
Submit a draft batch for asynchronous processing. The batch must be in 'DRAFT' status and contain objects.



# Tier-completion invariant audit
Source: https://docs.mixpeek.com/docs/api-reference/bucket-batches/tier-completion-invariant-audit

get /v1/buckets/{bucket_identifier}/batches/{batch_id}/audit
Returns the per-tier completion accounting for a batch: how many objects were submitted, how many ended up in the per-object success ledger (processed_objects), how many in the per-object failure ledger (failed_documents), how many were skipped, and how many are 'lost' (submitted but absent from both per-object ledgers).

Phase 1 of INGESTION_RELIABILITY_PLAN.md. ``has_lost_objects=true`` is a smoking gun for the row-failure-swallow class of bug — the tier was marked COMPLETED but rows silently went missing. Compare against ``tier_tasks[].audit`` (persisted at completion) to see the snapshot at the time the callback fired.



# List Buckets
Source: https://docs.mixpeek.com/docs/api-reference/buckets/list-buckets-get

get /v1/buckets
List buckets (GET alias for POST /buckets/list), so buckets can be listed the same way as collections, retrievers, and namespaces.



# Aggregate Documents
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/aggregate-documents

post /v1/collections/{collection_identifier}/documents/aggregate
This endpoint performs aggregation operations on documents in a collection.

    **Aggregation Framework**: Provides MongoDB-style aggregation operations:
    - GROUP BY: Group documents by one or more fields
    - Aggregations: COUNT, SUM, AVG, MIN, MAX, COUNT_DISTINCT, etc.
    - Date Operations: Truncate or extract date parts for time-series analysis
    - Filtering: Pre-aggregation filters (WHERE) and post-aggregation filters (HAVING)
    - Sorting & Limiting: Control result ordering and size

    **Counting documents that match a filter**: omit group_by and send only count
    aggregations, for example
    `{"aggregations": [{"function": "count", "alias": "total"}], "filters": {...}}`.
    The response holds one row whose metrics carry the exact number of matching
    documents (an empty `group`). This is the same count documents/list returns in
    `pagination.total` when you send `"include_total": true`. Any other
    aggregation needs at least one group_by field.

    **Use Cases**:
    - Count documents by feature type or collection
    - Calculate daily/monthly processing statistics
    - Analyze feature distributions and confidence scores
    - Generate reports with multiple metrics

    **Group values**: a group_by value comes back as a JSON number or boolean when the
    namespace declares a payload index of type integer, float or bool on the field. Any
    other field, including an undeclared numeric one, groups as strings.

    **Note**: This endpoint works with both MongoDB and the vector store using the same interface.
    The system automatically selects the appropriate aggregation provider based on
    the underlying metadata store.



# Batch Delete Documents
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/batch-delete-documents

delete /v1/collections/{collection_identifier}/documents/batch
Batch delete multiple documents by explicit IDs or filters.

Supports TWO modes:
1. Explicit IDs mode: Provide 'document_ids' array
   - Deletes specific documents by ID
   - Returns detailed per-document results
   - Maximum 1000 documents per batch

2. Filter mode: Provide 'filters' to delete all matching documents
   - Deletes ALL documents matching the filters
   - Returns total count only
   - Use with caution - can delete many documents

Key Features:
- Per-document success/failure reporting in explicit mode
- Validates documents exist in the specified collection
- Automatic document count update for the collection
- Efficient bulk deletion

Examples:
    Explicit IDs mode:
    ```json
    {
        "document_ids": ["doc_123", "doc_456", "doc_789"]
    }
    ```

    Filter mode (logical AND/OR/NOT shape — NOT MVS-native must/key):
    ```json
    {
        "filters": {"AND": [{"field": "metadata.status", "operator": "eq", "value": "archived"}]}
    }
    ```



# Batch Update Documents
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/batch-update-documents

post /v1/collections/{collection_identifier}/documents/batch
Batch update multiple documents by explicit IDs or filters.

Supports TWO modes:
1. Explicit IDs mode: Provide 'updates' array with document_id + update_data for each document
   - Each document can have DIFFERENT update_data
   - Returns detailed per-document results

2. Filter mode: Provide 'filters' + 'update_data' to update all matching documents
   - All documents receive the SAME update_data
   - Returns total count only

Key Features:
- Update any document field except vectors (metadata, internal_metadata, source_blobs, etc.)
- Maximum 1000 documents per batch in explicit mode
- Per-document success/failure reporting in explicit mode
- Validates documents exist in the specified collection

Examples:
    Explicit IDs mode:
    ```json
    {
        "updates": [
            {"document_id": "doc_123", "update_data": {"metadata": {"status": "processed"}}},
            {"document_id": "doc_456", "update_data": {"metadata": {"status": "archived"}}}
        ]
    }
    ```

    Filter mode (logical AND/OR/NOT shape — NOT MVS-native must/key):
    ```json
    {
        "filters": {"AND": [{"field": "metadata.status", "operator": "eq", "value": "pending"}]},
        "update_data": {"metadata": {"status": "processed"}}
    }
    ```



# Bulk Update Documents
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/bulk-update-documents

patch /v1/collections/{collection_identifier}/documents/bulk
Bulk update documents matching filter conditions.

Partially updates all documents in the collection that match the provided filters.
If no filters are provided, updates all documents in the collection.

This endpoint applies the SAME update_data to ALL documents matching the filters.
For per-document updates with different values, use POST /batch endpoint instead.



# Count documents in a collection.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/count-documents-collection

get /v1/collections/{collection_identifier}/documents/count
Return the total number of documents in the collection. This is the obvious REST endpoint for a document count; it is declared before the `/{document_id}` route so the literal `count` segment is never matched as a document id (which previously produced a misleading 404).



# Create a document.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/create-a-document

post /v1/collections/{collection_identifier}/documents
Create a document in a collection addressed by ID or name.



# Delete a document by ID.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/delete-a-document-by-id

delete /v1/collections/{collection_identifier}/documents/{document_id}
Delete a document by ID.



# Download frame-accurate scene media for this document.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/download-document-media

get /v1/collections/{collection_identifier}/documents/{document_id}/download
Export this scene's media cut frame-accurately to its time range.

The document's ``video_segment_url`` serves the extraction-time clip,
which is stream-copied and therefore keyframe-bounded — it can overshoot
``[start_time, end_time]`` by up to a GOP (~1s). This endpoint re-cuts
from the ORIGINAL source with re-encode so the exported media matches the
declared range within one source frame, caches the
result beside the extraction clip, and returns a presigned download URL.
The first call for a given range pays the re-encode (seconds, bounded);
repeat calls serve the cache.



# Get a document by ID.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/get-a-document-by-id

get /v1/collections/{collection_identifier}/documents/{document_id}
Get a document by ID.



# Get direct downstream documents (depth=1 children).
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/get-direct-downstream-documents

get /v1/collections/{collection_identifier}/documents/{document_id}/descendants
Return direct downstream documents (depth=1) — every document whose
`_internal.lineage.source_document_id` equals this document's ID.

Capped at 100 children per request. Equivalent to
``GET /documents/{id}?expand=children`` but returns just the children.

Deeper traversal (descendants beyond depth=1) is not yet supported in
a single call — walk recursively client-side or filter the namespace by
``from_object`` and post-process.



# Get document ancestors (lineage chain root → parent).
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/get-document-ancestors

get /v1/collections/{collection_identifier}/documents/{document_id}/ancestors
Return every prior step in this document's lineage chain, in order from
root through immediate parent. Excludes the document itself.

Equivalent to ``GET /documents/{id}?expand=ancestors`` but returns just
the ancestor list — convenient for SDKs and the Studio decomposition tree.



# List documents.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/list-documents

post /v1/collections/{collection_identifier}/documents/list
List documents with optional grouping support.

Supports two modes:
1. Regular listing: Returns flat list of documents with pagination
2. Grouped listing: When group_by is specified, returns documents grouped by field value

When using group_by:
- Requires a payload index on the specified field in the vector store
- Pagination applies to groups, not individual documents
- Each group contains all documents sharing the same field value



# List documents.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/list-documents-get

get /v1/collections/{collection_identifier}/documents
List documents with optional grouping support.

Supports two modes:
1. Regular listing: Returns flat list of documents with pagination
2. Grouped listing: When group_by is specified, returns documents grouped by field value

When using group_by:
- Requires a payload index on the specified field in the vector store
- Pagination applies to groups, not individual documents
- Each group contains all documents sharing the same field value



# Patch Document
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/patch-document

patch /v1/collections/{collection_identifier}/documents/{document_id}
Partially update a document by ID (PATCH operation).

In `metadata`, each key you send is set, keys you omit stay, and a key set to
`null` is removed.



# Return distinct values for a single field.
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/return-distinct-values-for-a-single-field

post /v1/collections/{collection_identifier}/documents/distinct
Return the set of unique values for one document field across a collection, together with a per-value document count sorted desc. Thin wrapper over `/aggregate` for the common `SELECT DISTINCT <field>` case — use this when a UI needs a facet list (e.g. `brand_slug`) without loading every document. Null/missing values are excluded. A value comes back as a JSON number or boolean when the namespace declares a payload index of type integer, float or bool on the field. Any other field, including an undeclared numeric one, facets as strings, so check the namespace's `payload_indexes` to know which you will get. When a storage shard times out or returns only part of its documents, the list can be short: `query_info.truncated` is then true and `query_info.warning` says what to do.



# Update a document's access control list (ACL).
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/update-a-documents-access-control-list-acl

patch /v1/collections/{collection_identifier}/documents/{document_id}/acl
Update a document's ACL (access control list).

Only the document owner or an org-scoped key can modify ACL.
User-scoped keys that are not the owner will receive a 403.



# Update Document
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/update-document

put /v1/collections/{collection_identifier}/documents/{document_id}
Update a document by ID.

Fields you omit keep their stored values. In `metadata`, each key you send is
set, keys you omit stay, and a key set to `null` is removed.



# Validate document against collection schema
Source: https://docs.mixpeek.com/docs/api-reference/collection-documents/validate-document-schema

post /v1/collections/{collection_identifier}/documents/schema/validate
Dry-run validation: checks a document payload against the collection's document_schema without creating or modifying any data. Returns validation result with any violations found.



# Get Lifecycle Status
Source: https://docs.mixpeek.com/docs/api-reference/collection-lifecycle/get-lifecycle-status

get /v1/collections/{collection_identifier}/lifecycle
Get the storage lifecycle status of a collection including vector counts.



# Transition Lifecycle
Source: https://docs.mixpeek.com/docs/api-reference/collection-lifecycle/transition-lifecycle

patch /v1/collections/{collection_identifier}/lifecycle
Transition a collection's storage tier. 'cold' evicts from the vector store (vectors searchable via object storage). 'active' rehydrates back to the vector store. 'archived' permanently removes vectors.



# Sync Collection Schema
Source: https://docs.mixpeek.com/docs/api-reference/collection-schema/sync-collection-schema

post /v1/collections/{collection_id}/sync-schema
Sample documents from the vector store and automatically discover new fields to add to the collection's output_schema.

    This endpoint:
    - Samples N documents from the collection (default: 1000)
    - Discovers all fields present in actual documents
    - Merges discovered fields into the collection's output_schema (additive only)
    - Optionally cascades schema updates to downstream collections
    - Respects debounce window (max once per 5 minutes, unless force=true)

    The sync operation is additive only - it never removes or changes existing field types.

    Use this endpoint to:
    - Manually trigger schema discovery after data ingestion
    - Force an immediate schema sync (bypassing debounce)
    - Update schemas with new fields discovered in documents



# Apply Taxonomy to Existing Documents
Source: https://docs.mixpeek.com/docs/api-reference/collection-taxonomies/apply-taxonomy-to-existing-documents

post /v1/collections/{collection_identifier}/apply-taxonomy
Apply a taxonomy to all existing documents in a collection retroactively.

This endpoint triggers distributed Ray processing to enrich existing documents
with taxonomy data. Unlike automatic materialization (which happens during ingestion),
this endpoint allows you to:

1. **Backfill enrichment** for documents ingested before the taxonomy was created
2. **Re-apply taxonomy** after configuration changes
3. **Process specific subsets** using scroll_filters

⚙️ **Processing Details:**
- Uses Ray datasets with map_batches for parallel processing
- Scales horizontally across Ray cluster
- Non-blocking: Returns immediately with task_id
- Monitor progress via Tasks API

⚠️ **Prerequisites:**
- Taxonomy must exist and be valid
- Taxonomy must be in collection's taxonomy_applications list
- Collection must contain documents

📊 **Performance:**
- ~1000-5000 docs/second depending on cluster size
- Parallel processing across multiple Ray workers
- Batch size and parallelism configurable

🔍 **Use Cases:**
- Backfill: Apply new taxonomy to historical data
- Re-enrichment: Update after taxonomy changes
- Selective: Process filtered document subsets

See Collections API and Taxonomies API documentation for details.



# Clone Collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/clone-collection

post /v1/collections/{collection_identifier}/clone
Clone a collection with optional modifications.

    **Purpose:**
    Creates a NEW collection (with new ID) based on an existing one. This is the
    recommended way to iterate on collection designs when you need to modify core
    configuration that PATCH doesn't allow (source, feature_extractor, field_passthrough).

    **Clone vs PATCH vs Template:**
    - **PATCH**: Update settings in place (enabled, metadata, schedule, taxonomy_applications, tiering and the other mutable fields)
    - **Clone**: Copy and modify core configuration (source, feature_extractor). Everything you do not override is copied, including a retriever_transform, schedule, alert and cluster applications, and tiering
    - **Template**: Start from a pre-configured pattern (for new projects)

    **Common Use Cases:**
    - Change feature extractor configuration (model, parameters)
    - Modify field_passthrough to include/exclude fields
    - Switch to different source (bucket or collection)
    - Test modifications before replacing production collection
    - Create variants (e.g., different embedding models)

    **How it works:**
    1. Source collection is copied
    2. You provide a new name (REQUIRED)
    3. Optionally override any other fields
    4. A new collection is created with a new ID
    5. Original collection remains unchanged



# Create Collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/create-collection

post /v1/collections
Create a new processing collection linked to a namespace.

A collection defines the feature extraction pipeline that runs when objects are
uploaded to a bucket.  One feature extractor per collection.

**Custom plugin collections:**

When `feature_extractor_name` references a custom plugin, the collection's vector
indexes are read from the plugin's `manifest.py` `features` list.  The `features`
entries **must** use these exact key names — wrong keys silently produce a collection
with no vector indexes and 0 documents will be written:

```json
{
  "feature_type": "embedding",
  "feature_name": "my_embedding",
  "embedding_dim": 768,
  "distance_metric": "cosine"
}
```

Common wrong keys (will be ignored): `type`, `name`, `dimensions`, `distance`.

**Vector schema:** Custom plugin vectors are automatically added to the namespace's
vector store collection the first time a batch is processed, so there is no need to recreate
the namespace when adding a plugin with new embedding types.



# Delete Collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/delete-collection

delete /v1/collections/{collection_identifier}
This endpoint deletes a collection and all its resources including:
    - Vector store points (documents) with this collection_id
    - Cache entries
    - MongoDB collection metadata

    Note: Collections are payload IDs within the namespace's vector store collection,
    not separate vector store collections.

    The deletion is performed synchronously and returns when complete.



# Describe collection features
Source: https://docs.mixpeek.com/docs/api-reference/collections/describe-collection-features

get /v1/collections/{collection_identifier}/features
List feature addresses and metadata available in this collection



# Per-document support diagnostics (visibility, failures, history).
Source: https://docs.mixpeek.com/docs/api-reference/collections/document-diagnostics

get /v1/collections/{collection_identifier}/documents/{document_id}/diagnostics
Answer three support facets for ONE document that no other surface exposes: (1) visibility_reason - are this document's vectors present in MVS (searchable) and, if not, why (the Unsearchable class, made per-document answerable via the vector-presence machinery); (2) recent_failures - the source object's own status/error, with an honest note that structured batch errors are aggregate-only; (3) diagnostic_history - the document's processing history from its own _internal records. Every facet returns a real value, or an empty/None value with a reason naming what would carry it.



# Export Collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/export-collection

post /v1/collections/{collection_identifier}/export
Export collection documents to JSON, CSV, Parquet or WebDataset format.

    **Export Formats:**
    - **JSON**: Line-delimited JSON (JSONL) format. Good for streaming.
    - **CSV**: Comma-separated values. Best for spreadsheets.
    - **PARQUET**: Columnar format (default). Best for data pipelines.
    - **WEBDATASET**: Tar shards for training loaders. One sample per document,
      carrying `<key>.json` and, with `include_media=True`, the source object's
      bytes. The response returns every shard plus the brace pattern loaders take.

    **Vector Export:**
    Vectors are large and exported separately. When `include_vectors=True`,
    a separate file is created for vectors with document_id mapping.

    **Field Selection:**
    Use `select_fields` to export only specific fields, reducing file size.

    **Lineage:**
    Set `include_lineage=True` to add `lineage_chain`, `source_content_hash` and
    `document_created_at` to every row, so each exported document traces back to its
    source object and processing steps.

    **Filtering:**
    Apply filters to export a subset of documents.

    **Destination (your own storage):**
    Set `destination` to a storage connection you own plus a prefix, and the files and a
    `manifest.json` are written there as well as to the download copy. The connection
    needs `write_enabled` set, which is off by default, and its credentials need the
    write permission on your side (`s3:PutObject`, or `storage.objects.create` for GCS).
    A connection without it is refused with the permission named.

    **Response:**
    Returns presigned download URLs valid for 1 hour.

    **Limits:**
    - Large exports may take time. Consider using `sample_size` for testing.
    - Vector exports significantly increase processing time.



# Get Collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/get-collection

get /v1/collections/{collection_identifier}
This endpoint allows you to retrieve a collection by ID or name.



# Import external documents into a collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/import-collection-documents

post /v1/collections/{collection_identifier}/import
Reconstruct documents (with vectors + lineage) from one external export using a validated mapping, and write them into the collection via the BYO write core. Use /import/infer and /import/validate-mapping first; dry_run reconstructs and validates without writing.



# Infer import schema
Source: https://docs.mixpeek.com/docs/api-reference/collections/infer-import-schema

post /v1/collections/import/infer
Infer source fields + types from a sample of an external export and return a suggested source->target mapping for the collection-import wizard. Runs on the sample alone; no data is written.



# List Collections
Source: https://docs.mixpeek.com/docs/api-reference/collections/list-collections

post /v1/collections/list
This endpoint allows you to list collections.



# Reconcile objects vs documents
Source: https://docs.mixpeek.com/docs/api-reference/collections/reconcile-collection

get /v1/collections/{collection_identifier}/reconciliation
Objects-vs-documents conservation for a collection. Returns the distinct source-object count, the count that produced at least one document, the document count, and their delta, plus an itemized (bounded) list of source objects that produced no document with a machine-readable reason for each.



# Trigger Collection Processing
Source: https://docs.mixpeek.com/docs/api-reference/collections/trigger-collection-processing

post /v1/collections/{collection_identifier}/trigger
Process data through a collection - works for both bucket-sourced and collection-sourced collections.

    **For bucket-sourced collections:**
    Creates a batch and answers with its id and a task id, then reads the source
    bucket(s), splits a load over 1,000 objects into several batches, and submits
    them. Use `include_buckets` to limit which source buckets to process from.
    If the buckets hold no matching objects, or a submission is refused, the
    batch and the task read FAILED with the reason. The task lists every batch
    as `additional_data.trigger.batch_ids` once they are submitted.

    **For collection-sourced collections:**
    Processes existing documents from upstream collection(s).
    Use `include_collections` to limit which source collections to process from.

    **Filtering:**
    - `source_filters`: Field-level filters using LogicalOperator format
    - Example: `{"AND": [{"field": "status", "operator": "eq", "value": "pending"}]}`
    - For specific objects: `{"AND": [{"field": "object_id", "operator": "in", "value": ["obj_1", "obj_2"]}]}`

    **Returns:**
    - batch_id: Track progress via GET /batches/{batch_id}
    - task_id: Monitor via GET /tasks/{task_id}



# Update Collection
Source: https://docs.mixpeek.com/docs/api-reference/collections/update-collection

patch /v1/collections/{collection_identifier}
Update mutable collection fields (collection_name, description, taxonomy_applications, enabled)



# Validate an import mapping
Source: https://docs.mixpeek.com/docs/api-reference/collections/validate-import-mapping

post /v1/collections/import/validate-mapping
Validate a finalized source->target mapping before any import. Flags a missing root-identity mapping, source/target type mismatches, vector-dim mismatches against a known index, and unknown source fields.



# Get all documents derived from an object
Source: https://docs.mixpeek.com/docs/api-reference/document-lineage/get-all-documents-derived-from-an-object

get /v1/objects/{object_id}/documents
Get all documents your collections created from a specific root object. The object's own record in its bucket is not listed or counted, so an object no collection has processed yet returns total_documents 0. Useful for finding all processing outputs across multiple collections.



# Get decomposition tree visualization
Source: https://docs.mixpeek.com/docs/api-reference/document-lineage/get-decomposition-tree-visualization

get /v1/objects/{object_id}/decomposition-tree
Get a hierarchical tree structure showing all collections and documents derived from a root object. Shows the complete multi-stage processing pipeline.



# Get document lineage
Source: https://docs.mixpeek.com/docs/api-reference/document-lineage/get-document-lineage

get /v1/collections/{collection_id}/documents/{document_id}/lineage
Get the complete processing lineage for a document. Shows the full chain of transformations from the root bucket object through all collection processing stages.



# Batch get documents by IDs
Source: https://docs.mixpeek.com/docs/api-reference/documents/batch-get-documents-by-ids

post /v1/documents/batch-get
Batch retrieve multiple documents by their IDs.

Returns documents and a list of IDs that were not found.
Maximum 1000 document IDs per request.



# Get a document by ID (namespace-scoped).
Source: https://docs.mixpeek.com/docs/api-reference/documents/get-a-document-by-id-namespace-scoped

get /v1/documents/{document_id}
Get a document by ID without specifying a collection.

Searches across all collections in the namespace. Use the
collection-scoped GET /v1/collections/{collection_id}/documents/{document_id}
when you know the collection for a faster lookup.



# List documents across all collections (namespace-scoped).
Source: https://docs.mixpeek.com/docs/api-reference/documents/list-documents-across-all-collections-namespace-scoped

post /v1/documents/list
List documents across all collections in the namespace.

Use this when you don't know which collection a document belongs to,
or when you need to search across collections. Optionally filter to
specific collection_ids.

For collection-specific listing, use
POST /v1/collections/{collection_id}/documents/list instead.



# Partially update a document by ID (namespace-scoped).
Source: https://docs.mixpeek.com/docs/api-reference/documents/partially-update-a-document-by-id-namespace-scoped

patch /v1/documents/{document_id}
Partially update a document by ID without specifying a collection.

Only the fields provided are updated; all other fields are left unchanged.



# Run multiple list-documents queries concurrently.
Source: https://docs.mixpeek.com/docs/api-reference/documents/run-multiple-list-documents-queries-concurrently

post /v1/documents/list/batch
Execute up to 10 `POST /v1/documents/list` queries in parallel against the
same namespace and return their responses together.

Collapses dossier / multi-collection N+1 fan-outs into one round-trip. Each
sub-query is independent; a failure in one does not fail the batch — the
failing entry's `response` is null and `error` carries a short message.



# Update a document by ID (namespace-scoped).
Source: https://docs.mixpeek.com/docs/api-reference/documents/update-a-document-by-id-namespace-scoped

put /v1/documents/{document_id}
Replace a document's fields by ID without specifying a collection.



# Get published extractor status
Source: https://docs.mixpeek.com/docs/api-reference/extractor-publishing/get-published-extractor-status

get /v1/extractors/{extractor_id}/publish



# Install a public extractor
Source: https://docs.mixpeek.com/docs/api-reference/extractor-publishing/install-public-extractor

post /v1/extractors/install



# Publish an extractor to the marketplace
Source: https://docs.mixpeek.com/docs/api-reference/extractor-publishing/publish-extractor-marketplace

post /v1/extractors/{extractor_id}/publish



# Unpublish an extractor
Source: https://docs.mixpeek.com/docs/api-reference/extractor-publishing/unpublish-extractor

delete /v1/extractors/{extractor_id}/publish



# Update published extractor
Source: https://docs.mixpeek.com/docs/api-reference/extractor-publishing/update-published-extractor

patch /v1/extractors/{extractor_id}/publish



# Confirm extractor submission upload
Source: https://docs.mixpeek.com/docs/api-reference/extractor-submissions/confirm-extractor-submission-upload

post /v1/extractors/submissions/uploads/{submission_id}/confirm
Step 2: Confirm upload, triggers security scan and manifest validation.



# Get presigned URL for extractor submission
Source: https://docs.mixpeek.com/docs/api-reference/extractor-submissions/get-presigned-url-extractor-submission

post /v1/extractors/submissions/uploads
Step 1: Get a presigned URL to upload your extractor archive to S3.



# Get submission details
Source: https://docs.mixpeek.com/docs/api-reference/extractor-submissions/get-submission-details

get /v1/extractors/submissions/{submission_id}



# List all extractors available to organization
Source: https://docs.mixpeek.com/docs/api-reference/extractor-submissions/list-all-extractors-available-organization

get /v1/extractors
List all extractors available across the organization.

Returns a unified view combining:
- **Builtin extractors**: Core extractors shipped with Mixpeek
- **Custom extractors**: Org-level custom extractors (uploaded via the upload workflow)

Use `source` to filter by origin.



# List extractor submissions
Source: https://docs.mixpeek.com/docs/api-reference/extractor-submissions/list-extractor-submissions

get /v1/extractors/submissions
List all extractor submissions for this organization.



# Get Feature Extractor by Name
Source: https://docs.mixpeek.com/docs/api-reference/feature-extractors/get-feature-extractor-by-name

get /v1/collections/features/extractors/{feature_extractor_id}
Get detailed information about a specific feature extractor by its name



# List Feature Extractors
Source: https://docs.mixpeek.com/docs/api-reference/feature-extractors/list-feature-extractors

get /v1/collections/features/extractors
List all available feature extractors grouped by category



# Discover Features
Source: https://docs.mixpeek.com/docs/api-reference/features/discover-features

get /v1/collections/features
The menu of what you can search by: each file type (modality), its billing unit, and the search features available on it (with rates). Feature keys are the values accepted by collection config `features: [...]`. No authentication required.



# Get Public Extractor
Source: https://docs.mixpeek.com/docs/api-reference/public-extractor-marketplace/get-public-extractor

get /v1/public/extractors/{public_name}
Get full detail for a public extractor including manifest and readme.



# List Public Extractors
Source: https://docs.mixpeek.com/docs/api-reference/public-extractor-marketplace/list-public-extractors

get /v1/public/extractors/
Browse and search public extractors.

No authentication required. Returns active extractors sorted by install count.



# List evaluation datasets
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/list-evaluation-datasets

get /v1/retrievers/evaluations/datasets
List all evaluation datasets with pagination



# Create Interaction
Source: https://docs.mixpeek.com/docs/api-reference/retriever-interactions/create-interaction

post /v1/retrievers/interactions
Record a search interaction (view, click, feedback, etc.).

Automatically computes and injects ``metadata.reward_value`` based on
the interaction types and the retriever's reward_map configuration.
This pre-computed value is used by the learned-fusion bandit aggregation
query so it doesn't need to join against the retriever config at read time.

**Deduplication (Phase 4e):** If ``(execution_id, feature_id,
interaction_type)`` has already been recorded within the last 5 minutes
the duplicate is silently dropped.

**Session cache (Phase 1b):** When ``session_id`` is present the
interaction is also written to the Redis session cache so the Thompson
Sampling bandit can incorporate it in real time, before the stored
aggregate refreshes.



# Create Interactions Bulk
Source: https://docs.mixpeek.com/docs/api-reference/retriever-interactions/create-interactions-bulk

post /v1/retrievers/interactions/batch
Record up to 1000 interactions in one call — for backfilling history.

Each interaction is enriched (reward_value, feature_uri/context promotion)
exactly like the single-create path, then written. Reward config is resolved
ONCE per distinct retriever (not per interaction). Dedup and the real-time
session cache are skipped — bulk writes are for historical backfill, not
live in-session adaptation. Set each interaction's ``occurred_at`` to its
true timestamp so temporal decay weights it by real age.



# Delete Interaction
Source: https://docs.mixpeek.com/docs/api-reference/retriever-interactions/delete-interaction

delete /v1/retrievers/interactions/{interaction_id}
Delete a specific interaction (idempotent - succeeds even if already deleted).



# Get Interaction
Source: https://docs.mixpeek.com/docs/api-reference/retriever-interactions/get-interaction

get /v1/retrievers/interactions/{interaction_id}
Get a specific interaction.



# List Interactions
Source: https://docs.mixpeek.com/docs/api-reference/retriever-interactions/list-interactions

post /v1/retrievers/interactions/list
List interactions with optional filters and pagination.

Supports hybrid filtering: simple fields + advanced LogicalOperator.



# List Available Retriever Stages
Source: https://docs.mixpeek.com/docs/api-reference/retriever-stages/list-available-retriever-stages

get /v1/retrievers/stages
List all registered retriever stages with their configurations. Use this endpoint to discover available stages before creating retrievers. Each stage includes its ID, description, category, and full parameter schema. The parameter_schema field contains complete Pydantic JSON Schema with validation rules, descriptions, and examples for all stage parameters.



# Batch Execute Retriever
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/batch-execute-retriever

post /v1/retrievers/{retriever_id}/execute/batch
Execute a retriever against multiple queries in a single request. The retriever is fetched and optimized once, then executed concurrently against each query with bounded parallelism.

**Use case:** IP safety / copyright clearance — scan 20 media files against face, logo, or audio retrievers in one call instead of 60 sequential SSE requests.

**Limits:** 1-16 queries per batch (a 17th is refused with `limits.max_queries`), 1-20 concurrency.

Returns results keyed by query index with per-query documents and errors.



# Clone Retriever
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/clone-retriever

post /v1/retrievers/{retriever_id}/clone
Clone a retriever with optional modifications.

**Purpose:**
Creates a NEW retriever (with new ID) based on an existing one. This is the
recommended way to iterate on retriever designs when you need to modify core
logic that PATCH doesn't allow (stages, input_schema, collections).

**Clone vs PATCH vs Template:**
- **PATCH**: Update metadata only (name, description, tags, display_config)
- **Clone**: Copy and modify core logic (stages, input_schema, collections)
- **Template**: Start from a pre-configured pattern (for new projects)

**Common Use Cases:**
- Fix a typo in a stage name
- Add or remove stages
- Change target collections
- Create variants (e.g., "strict" vs "relaxed" versions)
- Test modifications before replacing production retriever

**How it works:**
1. Source retriever is copied
2. You provide a new name (REQUIRED)
3. Optionally override any other fields
4. A new retriever is created with a new ID
5. Original retriever remains unchanged

**All fields except retriever_name are OPTIONAL:**
- Omit a field to copy from source
- Provide a field to override the source value



# Create Retriever
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/create-retriever

post /v1/retrievers
Create a new retriever.

A retriever executes a series of stages to find and process documents
from one or more collections.



# Delete Retriever
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/delete-retriever

delete /v1/retrievers/{retriever_id}
Delete a retriever and all its resources comprehensively.

Deletes:
- Published retrievers
- Execution history
- Interactions (user feedback)
- Evaluations
- Cache entries
- Retriever metadata

The deletion is performed synchronously and returns when complete.



# Execute Retriever (Auto-Optimized)
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/execute-retriever-auto-optimized

post /v1/retrievers/{retriever_id}/execute
Execute a retriever and return matching documents. The pipeline is automatically optimized before execution for best performance.

**Automatic Optimization:**
Your pipeline stages are automatically transformed for optimal performance:
- Filters pushed down to reduce expensive operations
- Redundant stages merged or eliminated
- Grouping operations pushed to database layer (10-100x faster)
- Operations reordered for efficiency

**Streaming Support:**
Set stream=true in the request body to receive real-time stage updates via SSE:
- Response uses text/event-stream content type
- Each stage emits stage_start and stage_complete events
- Final event contains complete results and pagination
- Useful for progress tracking and debugging

**Response Includes (when stream=false):**
- documents: Final matching documents
- pagination: Pagination metadata
- stage_statistics: Per-stage execution metrics
- budget: Credit/time consumption
- limits: Every cap that clamped the request (setting, value asked for, value used, the cap's maximum, the stage), absent when nothing was capped. A capped run can return fewer documents than asked for.
- optimization_applied: Whether optimizations were applied
- optimization_summary: Details about transformations (when applied)

**Optimization Summary Example:**
```json
{
  "optimization_applied": true,
  "optimization_summary": {
    "original_stage_count": 5,
    "optimized_stage_count": 3,
    "optimization_time_ms": 8.2,
    "rules_applied": ["push_down_filters", "group_by_push_down"],
    "stage_reduction_pct": 40.0
  }
}
```

Use the /explain endpoint to see the optimized execution plan before running.



# Explain Retriever Execution Plan
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/explain-retriever-execution-plan

post /v1/retrievers/{retriever_id}/execute/explain
Get a detailed execution plan for a retriever without actually executing it. Similar to MongoDB's explain plan or SQL's EXPLAIN command, this endpoint helps you understand performance characteristics, identify bottlenecks, estimate costs, and troubleshoot retrieval issues before running expensive queries.

**What This Returns:**
- Stage-by-stage execution plan (AFTER automatic optimizations)
- Estimated costs (credits + time per stage)
- Document flow projections (input/output counts per stage)
- Efficiency metrics (selectivity ratios, cache likelihood)
- Bottleneck identification (slowest/most expensive stages)
- Optimization details (transformations applied by the optimizer)
- Performance warnings and improvement suggestions


**Key Features:**
- **Cost Estimation**: See how many credits and milliseconds each stage will consume
- **Bottleneck Detection**: Identify which stages dominate execution time
- **Optimization Transparency**: Understand how your pipeline was optimized
- **Cache Analysis**: See which stages are likely to hit cache
- **Accuracy Troubleshooting**: Analyze stage efficiency and document flow
- **Latency Analysis**: Break down estimated duration by stage


**Important:** The execution_plan shows OPTIMIZED stages (after automatic transformations like filter push-down, stage fusion, and grouping optimization). Check optimization_details to understand what changed from your original configuration.


**Use Cases:**
- Debug slow retrievers by identifying bottleneck stages
- Estimate costs before running expensive queries
- Understand how the optimizer transformed your pipeline
- Troubleshoot accuracy issues by analyzing stage selectivity
- Compare different retriever configurations
- Plan budget allocation for production workloads


**Example Response:**
```json
{
  "retriever_id": "ret_abc123",
  "retriever_name": "product_search",
  "execution_plan": [
    {
      "stage_index": 0,
      "stage_name": "attribute_filter",
      "stage_type": "filter",
      "estimated_input": 10000,
      "estimated_output": 5000,
      "estimated_efficiency": 0.5,
      "estimated_cost_credits": 0.01,
      "estimated_duration_ms": 20,
      "cache_likely": true,
      "optimization_notes": ["Pushed down from stage 2"],
      "warnings": []
    },
    {
      "stage_index": 1,
      "stage_name": "semantic_search",
      "stage_type": "filter",
      "estimated_input": 5000,
      "estimated_output": 100,
      "estimated_efficiency": 0.02,
      "estimated_cost_credits": 0.5,
      "estimated_duration_ms": 200,
      "cache_likely": false,
      "optimization_notes": [],
      "warnings": ["High cost stage - consider reducing limit"]
    }
  ],
  "estimated_cost": {
    "total_credits": 0.51,
    "total_duration_ms": 220
  },
  "bottleneck_stages": ["semantic_search"],
  "optimization_applied": true,
  "optimization_details": {
    "original_stage_count": 3,
    "optimized_stage_count": 2,
    "optimization_time_ms": 8.2,
    "stage_reduction_pct": 33.3,
    "decisions": [
      {
        "rule_type": "push_down_filters",
        "applied": true,
        "reason": "Moved attribute_filter before semantic_search to reduce search scope"
      }
    ]
  },
  "optimization_suggestions": [
    {
      "type": "reduce_limit",
      "stage": "semantic_search",
      "message": "Consider reducing limit to improve latency"
    }
  ]
}
```



# Get Execution
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/get-execution

get /v1/retrievers/{retriever_id}/executions/{execution_id}
Get execution details and statistics.



# Get Execution Query Profile (explain-on-execute read path)
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/get-execution-profile

get /v1/retrievers/{retriever_id}/executions/{execution_id}/profile
Return the query profile for a PAST execution, by execution_id — the same plan that `explain=true` returns inline at execute time, read back later. Contains per-stage timings + input/output counts, MVS execution stats, the optimizer summary, and (as it is wired) the shard ExecutionTrace (chosen legs, fusion, push-downs, nprobe, served/shadow planner). Reads the query_profile stored on the execution's record in the mxp_retriever_executions collection. Returns 404 if the execution has no captured profile. This is the queryable, Studio-friendly counterpart to inline explain.



# Get Live Execution Status
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/get-live-execution-status

get /v1/retrievers/{retriever_id}/executions/{execution_id}/status
Fetch live execution status from the durability journal. Returns real-time progress including current stage, completed stages, and heartbeat timestamp. If the execution is completed, returns the full results. Falls back to the execution history for past executions.



# Get Retriever
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/get-retriever

get /v1/retrievers/{retriever_id}
Get a retriever by ID or name.



# List Executions
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/list-executions

post /v1/retrievers/{retriever_id}/executions/list
List execution history for a retriever.



# List Retrievers
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/list-retrievers

post /v1/retrievers/list
List all retrievers in the namespace.



# Patch Retriever
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/patch-retriever

patch /v1/retrievers/{retriever_id}
Update a retriever's metadata.

Editable fields:
- name, description, tags, display_config: metadata
- collection_identifiers: re-points + re-validates stage feature URIs
- stages: edit stages in place on an UNPUBLISHED retriever
  (change a filter/operator/rerank without clone+repoint+delete); the full
  stage list is replaced + re-validated as on create. A PUBLISHED retriever's
  stages stay immutable — clone or unpublish to change them.
- input_schema: NON-BREAKING evolution only (add default/examples/description
  to a field, add a new optional field). Breaking changes — removing a field,
  changing a type, making a field newly required — are rejected with 422
  naming each one; clone the retriever for those.

budget_limits remains immutable; use POST /{retriever_id}/clone.



# Typeahead
Source: https://docs.mixpeek.com/docs/api-reference/retrievers/typeahead

get /v1/retrievers/typeahead
Prefix suggestions for a search box (autocomplete/typeahead).

Suggests from three namespace-scoped sources concurrently: values of the
requested metadata fields (index-backed, with per-value document counts),
collection names, and the caller's own recent search queries. A source
that cannot answer is dropped and named in ``warnings`` rather than
failing the request.



# Execute Adhoc Retriever
Source: https://docs.mixpeek.com/docs/api-reference/adhoc-retrievers/execute-adhoc-retriever

post /v1/retrievers/execute
Execute a retriever ad-hoc without persisting the configuration.

This endpoint allows you to execute a retriever without saving it to the database.
Useful for one-time queries, testing configurations, or temporary searches.

Caching: this route does not use the retriever-level execute cache, so
``cache_status`` is always ``disabled`` and there is no ``skip_cache``
parameter.

That does NOT mean every response is freshly computed. Individual stages
keep their own caches, which this route cannot switch off, so a stage may
serve a cached result and report ``cache_hit: true`` under
``stage_statistics.stages[*]`` on a response whose top-level
``cache_status`` reads ``disabled``. The two fields describe different
caches. If you need a guaranteed fresh execution, persist the retriever
(``POST /retrievers``) and call ``/retrievers/{id}/execute`` with
``skip_cache``, which this route lacks and which does reach the stages.

Streaming Execution (stream=True):
    Response uses Server-Sent Events (SSE) format with Content-Type: text/event-stream.
    Each stage emits events as it executes, formatted as: data: {json}\n\n

    Event Types (StreamEventType):
    - stage_start: Emitted when a stage begins (includes stage_name, stage_index, total_stages)
    - stage_complete: Emitted when a stage finishes (includes documents, statistics, budget_used)
    - stage_error: Emitted if a stage fails (includes error message)
    - execution_complete: Final event with complete results and pagination
    - execution_error: Emitted if entire execution fails

    StreamStageEvent Fields:
    - event_type: Type of event
    - execution_id: Unique execution identifier
    - stage_name/stage_index/total_stages: Stage progress info
    - documents: Intermediate results (stage_complete only)
    - statistics: Stage metrics (duration_ms, input_count, output_count, efficiency)
    - budget_used: Cumulative consumption (credits_used, time_elapsed_ms, tokens_used)

    Response Headers:
    - Content-Type: text/event-stream
    - Cache-Control: no-cache
    - Connection: keep-alive
    - X-Execution-Mode: adhoc

Standard Execution (stream=False, default):
    - Returns ExecuteRetrieverResponse after all stages complete
    - Includes X-Execution-Mode: adhoc header
    - execution_metadata.retriever_persisted = False

Use Cases:
    - One-time queries without saving retriever configuration
    - Testing stage configurations before persisting
    - Dynamic retrieval with varying parameters
    - Real-time progress tracking with streaming



# Get Adhoc Execution
Source: https://docs.mixpeek.com/docs/api-reference/adhoc-retrievers/get-adhoc-execution

get /v1/retrievers/executions/{execution_id}
Get detailed execution information for a specific ad-hoc retriever execution.

Returns comprehensive execution details including:
    - Execution metadata (status, duration, credits used)
    - Performance metrics (documents processed/returned, cache hit rate)
    - Input data and query summary
    - Stage completion information
    - Collections queried

Use Cases:
    - Debug specific ad-hoc executions
    - Analyze performance of a particular query
    - Retrieve execution inputs for reproduction
    - Audit ad-hoc retriever usage

Raises:
    404 NotFoundError: If execution not found or not an ad-hoc execution



# List Adhoc Executions
Source: https://docs.mixpeek.com/docs/api-reference/adhoc-retrievers/list-adhoc-executions

post /v1/retrievers/executions/list
List execution history for ad-hoc retrievers.

Returns execution history for all ad-hoc retriever executions in the namespace,
sorted by timestamp descending (most recent first).

Use Cases:
    - Track ad-hoc retriever usage across the namespace
    - Debug ad-hoc retriever executions
    - Analyze query patterns from ad-hoc searches
    - Monitor performance of ad-hoc executions

Filtering:
    - Filter by status (completed, failed, etc.)
    - Filter by time range (start_time, end_time)

Pagination:
    - Supports offset-based pagination via query parameters
    - Default limit: 20, max limit: 100
    - Use ?page_size=X&page_number=Y for pagination



# Annotation Stats
Source: https://docs.mixpeek.com/docs/api-reference/annotations/annotation-stats

get /v1/annotations/stats
Aggregate annotation counts grouped by label.



# Bulk Annotations
Source: https://docs.mixpeek.com/docs/api-reference/annotations/bulk-annotations

post /v1/annotations/bulk
Create, update, and/or delete annotations in a single call.

Each operation is independent — a failure in one does not roll back the others.
Maximum 1000 operations per type (creates, updates, deletes).



# Create Annotation
Source: https://docs.mixpeek.com/docs/api-reference/annotations/create-annotation

post /v1/annotations
Record a human decision on a document.



# Delete Annotation
Source: https://docs.mixpeek.com/docs/api-reference/annotations/delete-annotation

delete /v1/annotations/{annotation_id}
Delete an annotation.



# Get Annotation
Source: https://docs.mixpeek.com/docs/api-reference/annotations/get-annotation

get /v1/annotations/{annotation_id}
Get a single annotation by ID.



# Get Annotation At Version
Source: https://docs.mixpeek.com/docs/api-reference/annotations/get-annotation-at-version

get /v1/annotations/{annotation_id}/versions/{version}
The annotation exactly as it stood at a given version.

Reproducible evaluation: pin (annotation_id, version) and this route
returns the identical judgment later, however many edits followed.



# Get Annotation Versions
Source: https://docs.mixpeek.com/docs/api-reference/annotations/get-annotation-versions

get /v1/annotations/{annotation_id}/versions
Version history of an annotation.

Every value-changing edit mints a new version; each descriptor names who
made the edit, when, and which fields changed. Fetch a specific version's
full state at /versions/{version}.



# List Annotations
Source: https://docs.mixpeek.com/docs/api-reference/annotations/list-annotations

post /v1/annotations/list
Query annotations with optional filters.



# Patch Annotation
Source: https://docs.mixpeek.com/docs/api-reference/annotations/update-annotation

patch /v1/annotations/{annotation_id}
Update an existing annotation (label, confidence, reasoning, payload).



# Cancel Execution
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/cancel-execution

post /v1/clusters/{cluster_id}/executions/{run_id}/cancel
Cancel a pending or processing cluster execution.

    This cancels the Ray job (if running) and marks the execution as CANCELED.
    Useful for:
    - Cleaning up orphaned executions where the Ray job was never submitted
    - Stopping long-running clustering jobs
    - Handling executions stuck in pending/processing state

    Only works for executions in 'pending' or 'processing' status.
    Returns an error if the execution is already 'completed', 'failed', or 'canceled'.



# Export Cluster Run Data (CSV / Parquet)
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/export-cluster-run-data

get /v1/clusters/{cluster_id}/executions/{run_id}/export
One-click export of a clustering run's data — the exact rows behind the
    visualization (one row per member plus one per centroid: document id,
    cluster id and current label, x/y[/z] layout coordinates, per-cluster
    stats, and any custom LLM fields).

    Formats:
    - **`format=parquet`** (default): returns JSON with a short-lived
      presigned download URL for the run's `cluster_documents.parquet`
      artifact — the full-fidelity file (includes centroid vectors), any
      size. Download it promptly; the URL expires.
    - **`format=csv`**: streams a spreadsheet-friendly CSV conversion —
      stable column order, current (renamed) cluster labels, no raw vectors.
      Capped at 100,000 rows; bigger runs get a clear 413 pointing to
      parquet. Scope to one cluster with `cluster_label` (accepts the run's
      cluster id like `cl_3` OR its current label).

    Exports are strictly per-run: the run you pass is the run you get.
    Runs that completed before artifacts existed, failed before the export
    step, or whose artifacts have aged out of object storage return 404
    with a message saying exactly what's missing.



# Get Latest Cluster Execution
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/get-latest-cluster-execution

get /v1/clusters/{cluster_id}/executions
Get the most recent execution results for a cluster.

    The response is a single execution object, not a list, even though the path
    is plural. A cluster that has never been executed returns 200 with a null
    body. To list past executions, call
    POST /v1/clusters/{cluster_id}/executions/list, which returns them under
    `results`. To read one specific run, call
    GET /v1/clusters/{cluster_id}/executions/{run_id}.

    Returns execution metadata including:
    - Execution status (pending, processing, completed, failed)
    - Clustering metrics (silhouette score, Davies-Bouldin index, etc.)
    - Number of clusters found and documents processed
    - Centroid information with labels and summaries
    - Execution timestamps

    Useful for:
    - Displaying cluster statistics in dashboards
    - Showing cluster quality metrics to users
    - Rendering cluster labels and summaries in the UI
    - Tracking execution status and errors



# Get Specific Cluster Execution
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/get-specific-cluster-execution

get /v1/clusters/{cluster_id}/executions/{run_id}
Get a specific execution by run ID.

    Returns detailed execution information for a particular clustering run,
    allowing you to review historical executions and compare results over time.



# List Cluster Execution History
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/list-cluster-execution-history

post /v1/clusters/{cluster_id}/executions/list
List execution history for a cluster with pagination, filtering, sorting, and search.

    Returns all historical executions for the specified cluster, including:
    - Execution status (pending, processing, completed, failed)
    - Clustering metrics (silhouette score, Davies-Bouldin index, etc.)
    - Number of clusters found and documents processed
    - Execution timestamps and duration
    - Centroid information

    Supports:
    - **Filtering**: Filter by status, date range, metrics, etc.
    - **Sorting**: Sort by created_at, execution time, metrics
    - **Search**: Full-text search across execution metadata
    - **Pagination**: Limit and offset for large result sets

    Use cases:
    - View all past executions for a cluster
    - Compare metrics across runs
    - Track execution history over time
    - Debug failed executions
    - Analyze clustering performance trends



# Name an Execution Run
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/name-execution-run

patch /v1/clusters/{cluster_id}/executions/{run_id}/name
Set (or clear) a human-friendly name on a single execution run — no
    re-run required.

    A non-empty value sets/replaces the run's `run_name`; an empty string
    clears it, restoring the unnamed (run_id-only) state. Names are trimmed
    and capped at 120 characters. The name is persisted on the execution
    record and returned by execution GET/LIST responses (`run_name`,
    omitted entirely while unnamed), so run selectors and execution-history
    views can show 'July tuning baseline' instead of `run_a8e2709532...`.

    Sibling of PATCH `.../executions/{run_id}/labels`: each display field
    gets its own strict single-purpose PATCH rather than a general
    execution PATCH, because execution results themselves are immutable.

    Example body: `{"run_name": "July tuning baseline"}` (or
    `{"run_name": ""}` to clear).



# Rename Cluster Labels (Per-Execution)
Source: https://docs.mixpeek.com/docs/api-reference/cluster-executions/rename-cluster-labels

patch /v1/clusters/{cluster_id}/executions/{run_id}/labels
Rename cluster labels for a single execution — no re-run required.

    Entries merge into the execution record's `label_overrides` map
    (per-key upsert): a non-empty value sets/replaces the override for that
    cluster_id, an empty string deletes it (restoring the original LLM/auto
    label). Overrides are applied server-side wherever labels are read —
    execution GETs (`centroids[].label`) and the cluster visualization
    endpoint (`cluster_label` on centroid points) — so all clients agree.

    Overrides are per-run because cluster ids (`cl_0`, `cl_1`, ...) are
    assigned per-run: a rename on one execution never leaks onto another.

    Example body: `{"labels": {"cl_0": "Gadget Reviews", "cl_2": ""}}`
    (renames cl_0, deletes any override on cl_2).



# Create Cluster Trigger
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/create-cluster-trigger

post /v1/clusters/triggers
Create a new trigger for automated cluster execution.

    Supports multiple trigger types:
    - **cron**: Execute at specific times using cron expressions
    - **interval**: Execute at fixed intervals
    - **event**: Execute when specific events occur (e.g., documents added)
    - **conditional**: Execute when conditions are met (e.g., drift threshold)



# Delete Cluster Trigger
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/delete-cluster-trigger

delete /v1/clusters/triggers/{trigger_id}
Delete a cluster trigger (soft delete).



# Get Cluster Trigger
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/get-cluster-trigger

get /v1/clusters/triggers/{trigger_id}
Get a cluster trigger by ID.



# Get Trigger Execution History
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/get-trigger-execution-history

post /v1/clusters/triggers/{trigger_id}/history
Get execution history for a trigger with pagination.



# List Cluster Triggers
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/list-cluster-triggers

post /v1/clusters/triggers/list
List cluster triggers with filters and pagination.



# Pause Cluster Trigger
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/pause-cluster-trigger

post /v1/clusters/triggers/{trigger_id}/pause
Pause trigger execution. Paused triggers retain configuration but do not execute.



# Resume Cluster Trigger
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/resume-cluster-trigger

post /v1/clusters/triggers/{trigger_id}/resume
Resume paused trigger. Next execution time is recalculated from current time.



# Update Cluster Trigger
Source: https://docs.mixpeek.com/docs/api-reference/cluster-triggers/update-cluster-trigger

patch /v1/clusters/triggers/{trigger_id}
Update a cluster trigger.

    Allowed updates:
    - schedule_config: Modify trigger schedule
    - description: Update description
    - status: Change status (use pause/resume endpoints instead)

    Not allowed:
    - trigger_type: Must delete and recreate
    - cluster_id: Immutable
    - execution_config: Immutable



# Cluster Visualization Data
Source: https://docs.mixpeek.com/docs/api-reference/clusters/cluster-visualization-data

get /v1/clusters/{cluster_id}/visualization
Return sampled visualization coordinates for rendering a cluster scatter plot. Returns all centroids plus a random sample of members (default 10,000). Only the fields needed for rendering are returned, keeping the payload small.



# Create Cluster
Source: https://docs.mixpeek.com/docs/api-reference/clusters/create-cluster

post /v1/clusters
Create a new cluster configuration and output collection.

    This endpoint:
    1. Creates cluster metadata
    2. Creates output collection for cluster documents
    3. Returns cluster metadata with output_collection_id

    The cluster can then be executed via POST /v1/clusters/{id}/execute



# Delete Cluster
Source: https://docs.mixpeek.com/docs/api-reference/clusters/delete-cluster

delete /v1/clusters/{cluster_id}
This endpoint deletes a cluster and all its resources including:
    - Running Ray jobs (cancels active jobs)
    - Cluster triggers
    - Execution history (clustering_results)
    - S3 artifacts (parquet files, documents, members)
    - Related tasks
    - Clustering jobs
    - MongoDB cluster metadata
    - Output collections created by the cluster (by default)

    By default, output collections created by the cluster are cascade-deleted
    along with the cluster. Pass `cascade_output_collections=false` to preserve
    them (useful when the cluster output documents were enriched / promoted
    into user-managed data).

    The deletion is performed synchronously and returns when complete.



# Execute Clustering
Source: https://docs.mixpeek.com/docs/api-reference/clusters/execute-clustering

post /v1/clusters/{cluster_id}/execute
Execute clustering on a specific cluster.

    This endpoint:
    1. Validates the cluster exists
    2. Queues a background clustering job
    3. Returns task_id immediately (non-blocking)
    4. The background job prepares data and submits to Engine
    5. Monitor progress via GET /v1/tasks/{task_id}

    Flow:
    - API: Receives request
    - Background job: Fetches documents, creates parquet, uploads to S3
    - Engine: Runs Ray job on parquet data
    - Status: Automatically updates cluster when complete

    Use GET /v1/clusters/{id}/executions to retrieve results.

    Optional body (`ExecuteClusterByIdRequest`) accepts a `filters` override
    that applies to this execution only and is not persisted on the cluster.



# Get Cluster
Source: https://docs.mixpeek.com/docs/api-reference/clusters/get-cluster

get /v1/clusters/{cluster_identifier}
Retrieve a cluster by ID or name.

    Returns cluster metadata including:
    - Configuration (cluster_type, algorithm, parameters)
    - Output collection information (output_collection_ids, output_collection_names)
    - Execution results (num_clusters, num_documents_clustered, status)
    - Timestamps and metadata

    Per-document cluster assignments are not on this object. They are documents in
    the output collections listed in output_collection_ids: a member document
    (is_centroid=false) carries cluster_id, cluster_label and source_document_id.
    With enrich_source_collection=true they are fields on your source documents.



# List Cluster Groups
Source: https://docs.mixpeek.com/docs/api-reference/clusters/list-cluster-groups

get /v1/clusters/{cluster_id}/groups
Get all cluster groups with labels, summaries, and document counts from the latest execution.



# List Clusters
Source: https://docs.mixpeek.com/docs/api-reference/clusters/list-clusters

post /v1/clusters/list
This endpoint allows you to list clusters.



# List Clusters (GET)
Source: https://docs.mixpeek.com/docs/api-reference/clusters/list-clusters-get

get /v1/clusters
REST-convention listing endpoint. Equivalent to ``POST /v1/clusters/list``
    with no filters or sort. Use the POST variant when you need filters,
    sort, or search.



# Partially Update Cluster
Source: https://docs.mixpeek.com/docs/api-reference/clusters/partially-update-cluster

patch /v1/clusters/{cluster_identifier}
This endpoint partially updates a cluster (PATCH operation).
    Only provided fields will be updated. At minimum, metadata can always be updated.
    Immutable fields like cluster_id, status, and computed fields cannot be modified.



# Create Trigger
Source: https://docs.mixpeek.com/docs/api-reference/create-trigger

post /v1/triggers
Create a new trigger for scheduled job execution.

    **Action Types:**
    - `cluster`: Execute clustering on a cluster definition
    - `taxonomy_enrichment`: Apply taxonomy enrichment to a collection

    **Schedule Types:**
    - `cron`: Execute at specific times using cron expressions (e.g., "0 2 * * *" for daily at 2am)
    - `interval`: Execute at fixed intervals (e.g., every 6 hours)
    - `event`: Execute when specific events occur (e.g., after 100 documents added)
    - `conditional`: Execute when conditions are met (e.g., drift threshold exceeded)

    **Examples:**

    Cluster trigger (daily at 2am):
    ```json
    {
      "action_type": "cluster",
      "action_config": {"cluster_id": "clust_abc123"},
      "trigger_type": "cron",
      "schedule_config": {"cron_expression": "0 2 * * *", "timezone": "UTC"},
      "description": "Daily clustering at 2am"
    }
    ```

    Taxonomy enrichment trigger (every 6 hours):
    ```json
    {
      "action_type": "taxonomy_enrichment",
      "action_config": {
        "taxonomy_id": "tax_products",
        "collection_id": "col_inventory",
        "batch_size": 1000
      },
      "trigger_type": "interval",
      "schedule_config": {"interval_seconds": 21600},
      "description": "Re-enrich products every 6 hours"
    }
    ```



# Delete Trigger
Source: https://docs.mixpeek.com/docs/api-reference/delete-trigger

delete /v1/triggers/{trigger_id}
Delete a trigger (soft delete - sets status to disabled).



# Execute Trigger Now
Source: https://docs.mixpeek.com/docs/api-reference/execute-trigger-now

post /v1/triggers/{trigger_id}/execute
Manually execute a trigger immediately.

    This bypasses the schedule and executes the trigger's action right away.
    Useful for testing trigger configuration or forcing immediate execution.

    Returns a task response that can be used to monitor execution progress.



# Feature Search
Source: https://docs.mixpeek.com/docs/api-reference/feature-search/feature-search

post /v1/features/search
Search documents by vector similarity across collections. A convenience wrapper that builds an ad-hoc retriever with a single feature_search stage.



# Get Trigger
Source: https://docs.mixpeek.com/docs/api-reference/get-trigger

get /v1/triggers/{trigger_id}
Get a trigger by ID.



# Get Trigger Execution History
Source: https://docs.mixpeek.com/docs/api-reference/get-trigger-execution-history

get /v1/triggers/{trigger_id}/history
Get execution history for a trigger with pagination.



# List Triggers
Source: https://docs.mixpeek.com/docs/api-reference/list-triggers

post /v1/triggers/list
List triggers with filters and pagination.

    **Filters:**
    - `action_type`: Filter by action type (cluster, taxonomy_enrichment)
    - `trigger_type`: Filter by trigger type (cron, interval, event, conditional)
    - `status`: Filter by status (active, paused, disabled, failed)
    - `resource_id`: Filter by resource ID (cluster_id or taxonomy_id)



# Pause Trigger
Source: https://docs.mixpeek.com/docs/api-reference/pause-trigger

post /v1/triggers/{trigger_id}/pause
Pause trigger execution. Paused triggers retain configuration but do not execute.



# Execute Public Retriever
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/execute-public-retriever

post /v1/public/retrievers/{public_name}/execute
Execute a published retriever (public endpoint).

**Authentication:**
- API key is OPTIONAL for public retrievers
- Supports: no key, prk_ keys (deprecated), or ret_sk_ keys
- If password-protected, requires `X-Retriever-Password` header

**Rate Limiting:**
- Subject to per-retriever rate limits (per minute/hour/day)
- May also have IP-based rate limits

**Response:**
- Only returns fields specified in `exposed_fields` configuration
- Internal metadata is stripped from results
- Includes `execution_id` for interaction tracking
- Presigned URLs returned by default (return_presigned_urls=true) for media rendering

**Example (no API key - recommended for public access):**
```bash
curl -X POST "https://api.mixpeek.com/v1/public/retrievers/video-search/execute" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {"query": "red car"},
    "pagination": {"method": "offset", "page_number": 1, "page_size": 10}
  }'
```

**Example with ret_sk_ key (for SDK/programmatic access):**
```bash
curl -X POST "https://api.mixpeek.com/v1/public/retrievers/video-search/execute" \
  -H "X-Public-API-Key: ret_sk_abc123..." \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {"query": "red car"},
    "pagination": {"method": "offset", "page_number": 1, "page_size": 10}
  }'
```



# Get Public Retriever Config
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/get-public-retriever-config

get /v1/public/retrievers/{public_name}/config
Get display configuration for public page rendering.

⚠️ **DEPRECATED**: Use `/v1/marketplace/catalog/{public_name}/config` instead.
This endpoint is maintained for backwards compatibility but may be removed in the future.

Returns the UI configuration needed to render the public search interface.
Used by the frontend app at mxp.co to dynamically build the UI.

**Authentication:**
- NO authentication required - this endpoint is public
- Anyone can access the config if they know the public_name
- The config includes the public_api_key needed for execute/interact endpoints

**Response includes:**
- Display config (logo, theme, components, field rendering)
- Title and description
- Password protection status
- Public API key for subsequent authenticated requests

**Example (deprecated):**
```bash
curl -X GET "https://api.mixpeek.com/v1/public/retrievers/video-search/config"
```

**Example (recommended):**
```bash
curl -X GET "https://api.mixpeek.com/v1/marketplace/catalog/video-search/config"
```



# Get Public Retriever Template
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/get-public-retriever-template

get /v1/public/retrievers/{public_name}/template
Get retriever configuration as a reusable template.

Returns the published retriever's configuration in a format that can be
directly used to create your own retriever. This is perfect for discovering
patterns and adapting them to your own data.

**Authentication:**
- NO authentication required - this endpoint is completely public
- Anyone can get the template if they know the public_name

**Use Case:**
1. Browse public retrievers to find patterns you like
2. GET this endpoint to get the full configuration
3. Copy the config and modify for your needs (especially `collection_identifiers`)
4. POST to `/v1/retrievers` to create your own retriever
5. Optionally publish it with the same display_config

**What's included:**
- Retriever configuration (stages, input_schema, budget_limits)
- Display configuration (for publishing with similar UI)
- Original metadata for reference

**What you need to change:**
- `collection_identifiers`: Replace with your own collection IDs
- `retriever_name`: Give it a unique name
- Optionally modify stages, inputs, display_config as needed

**Example:**
```bash
# 1. Get the template
curl -X GET "https://api.mixpeek.com/v1/public/retrievers/video-search/template"

# 2. Modify the response and create your own retriever
curl -X POST "https://api.mixpeek.com/v1/retrievers" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_name": "my_video_search",
    "collection_identifiers": ["my_videos"],
    "stages": [...],  # From template
    "input_schema": {...},  # From template
    "budget_limits": {...},  # From template
    "display_config": {...}  # From template (optional)
  }'
```

**Response includes:**
- All retriever configuration fields
- Display config for publishing (optional to use)
- Source reference (where this template came from)



# List Public Retrievers
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/list-public-retrievers

get /v1/public/retrievers/
List all public retrievers with pagination and search.

This endpoint allows browsing and discovering all published retrievers
across all organizations. No authentication required.

**Authentication:**
- NO authentication required - completely public endpoint
- Discover retrievers created by all Mixpeek users

**Pagination:**
- Default: page=1, page_size=20
- Maximum page_size: 100
- Returns total count and total pages

**Search:**
- Search across retriever titles and descriptions
- Case-insensitive regex matching
- Combine with pagination

**Filtering:**
- By default, only active retrievers are shown
- Set `include_inactive=true` to see all retrievers

**Response includes:**
- List of public retrievers with basic info
- Pagination details (page, page_size, total_count, total_pages)
- Aggregate statistics (total active, password protected, open)

**What's NOT exposed:**
- API keys (except in individual config endpoint)
- Internal IDs or organization details
- Full retriever configuration (use template endpoint for that)
- Password values (only password_protected: true/false)

**Example:**
```bash
# List all public retrievers (first page)
curl -X GET "https://api.mixpeek.com/v1/public/retrievers/"

# Search for video-related retrievers
curl -X GET "https://api.mixpeek.com/v1/public/retrievers/?search=video&page_size=50"

# Get page 2 with custom page size
curl -X GET "https://api.mixpeek.com/v1/public/retrievers/?page=2&page_size=50"
```

**Use Cases:**
- Browse available public retrievers
- Discover search patterns and implementations
- Find retrievers to use as templates
- Explore what others have built



# Track Interaction
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/track-interaction

post /v1/public/retrievers/{public_name}/interactions
Track user interaction with search results.

Records user engagement (clicks, views, etc.) for analytics and
potential search optimization (Learning to Rank).

**Authentication:**
- API key is OPTIONAL (same as execute endpoint)
- Password NOT required (tracking should work even without auth)

**Recommended Headers:**
- `X-Session-ID`: Session identifier for tracking user journey

**Interaction Types:**
- `VIEW`: Result was visible in viewport
- `CLICK`: User clicked on result
- `POSITIVE_FEEDBACK`: User explicitly liked result
- `NEGATIVE_FEEDBACK`: User explicitly disliked result
- `PURCHASE`: User purchased/converted
- `ADD_TO_CART`: User added to cart
- `WISHLIST`: User added to wishlist
- `LONG_VIEW`: User spent significant time viewing
- `SHARE`: User shared result
- `BOOKMARK`: User bookmarked result

**Example:**
```bash
curl -X POST "https://api.mixpeek.com/v1/public/retrievers/video-search/interactions" \
  -H "X-Session-ID: sess_xyz..." \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "doc_123",
    "interaction_type": ["CLICK"],
    "position": 2,
    "execution_id": "exec_abc",
    "query_snapshot": {"query": "red car"}
  }'
```



# Track Interaction Batch
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/track-interaction-batch

post /v1/public/retrievers/{public_name}/interactions/batch
Track multiple interactions in a single request (batching).

More efficient than sending individual interaction requests.
Use this for batching viewport visibility, bulk actions, etc.

**Authentication:**
- API key is OPTIONAL (same as execute endpoint)
- Password NOT required (tracking should work even without auth)

**Recommended Headers:**
- `X-Session-ID`: Applied to all interactions in the batch

**Limits:**
- Maximum 100 interactions per batch

**Example:**
```bash
curl -X POST "https://api.mixpeek.com/v1/public/retrievers/video-search/interactions/batch" \
  -H "X-Session-ID: sess_xyz..." \
  -H "Content-Type: application/json" \
  -d '{
    "interactions": [
      {
        "document_id": "doc_123",
        "interaction_type": ["VIEW"],
        "position": 0,
        "execution_id": "exec_abc"
      },
      {
        "document_id": "doc_456",
        "interaction_type": ["VIEW"],
        "position": 1,
        "execution_id": "exec_abc"
      }
    ]
  }'
```



# Verify Password
Source: https://docs.mixpeek.com/docs/api-reference/public-retriever-api/verify-password

post /v1/public/retrievers/{public_name}/verify
Verify password for a password-protected retriever.

Allows the frontend to check if a password is valid before attempting to execute
a password-protected retriever. Returns the public API key if the password is valid.

**Authentication:**
- NO authentication required - this endpoint is public
- The password is verified against the retriever's configured password

**Use Case:**
1. Frontend detects that a retriever is password-protected (from /config endpoint)
2. User enters password in the UI
3. Frontend calls this endpoint to verify the password
4. If valid, frontend receives the public_api_key to use for subsequent requests

**Response:**
- `valid`: Whether the password is correct
- `public_api_key`: The API key to use for execute/interact endpoints (only if valid)

**Example:**
```bash
curl -X POST "https://api.mixpeek.com/v1/public/retrievers/private-search/verify" \
  -H "Content-Type: application/json" \
  -d '{"password": "secret123"}'
```

**Response if valid:**
```json
{
  "valid": true,
  "public_api_key": "prk_abc123..."
}
```

**Response if invalid:**
```json
{
  "valid": false,
  "public_api_key": null
}
```



# Check Name Availability
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/check-name-availability

get /v1/retrievers/{retriever_id}/publish/availability
Check if a public name is available.

Public names must be globally unique across all organizations.
Use this endpoint before publishing to ensure your desired name is available.



# Get Organization Publish Stats
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/get-organization-publish-stats

get /v1/retrievers/publish/stats
Get publish statistics for the namespace in the request.

The publish limit counts each namespace separately.



# Get Published Retriever
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/get-published-retriever

get /v1/retrievers/{retriever_id}/publish
Get published retriever details.

Returns configuration including public URL, display settings, and rate limits.
The public API key is not returned (it was shown only once during publishing).

A retriever that simply hasn't been published yet is a normal state (the
Studio Publish tab opens this on every visit), not an error — so this
returns ``200`` with a ``null`` body rather than ``404``, which previously
logged a console error for every unpublished retriever.



# List Published Retrievers
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/list-published-retrievers

get /v1/retrievers/published
List all published retrievers for the organization.



# Publish Retriever
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/publish-retriever

post /v1/retrievers/{retriever_id}/publish
Publish a retriever as a public search interface.

Creates a public API endpoint and branded page for the retriever.
Returns a public API key that should be stored securely (shown only once).

**Limits:**
- Maximum 10 published retrievers per organization
- Public name must be globally unique

**Security:**
- Public API key is required for all requests
- Optional password protection via organization secrets
- Configurable rate limits per retriever
- Field masking ensures only approved fields are exposed



# Reconcile Publish State
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/reconcile-publish-state

post /v1/retrievers/{retriever_id}/publish/reconcile
Repair published_retrievers <-> marketplace_listings divergence.

A retriever's public surfaces can drift apart — e.g. a marketplace listing
exists but its published_retrievers record is missing, so
/v1/public/retrievers/{name} 404s (half-published). This detects the state
and (unless ``dry_run``) repairs it idempotently:
- missing_published -> create the published record from the listing (reused id)
- missing_listing   -> create the listing from the published record
- consistent        -> ensure cross-link
- not_published     -> no-op



# Unpublish Retriever
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/unpublish-retriever

delete /v1/retrievers/{retriever_id}/publish
Unpublish a retriever.

Removes the public API endpoint and deletes the short URL redirect.
The public API key will immediately stop working.
This action cannot be undone (you'll need to republish to get a new API key).



# Update Published Retriever
Source: https://docs.mixpeek.com/docs/api-reference/published-retrievers/update-published-retriever

patch /v1/retrievers/{retriever_id}/publish
Update published retriever configuration.

Allows updating display config, rate limits, exposed fields, and password protection.
The public API key and public name cannot be changed (unpublish and republish instead).



# Resume Trigger
Source: https://docs.mixpeek.com/docs/api-reference/resume-trigger

post /v1/triggers/{trigger_id}/resume
Resume a paused trigger. Next execution time is recalculated from current time.



# Create Retriever API Key
Source: https://docs.mixpeek.com/docs/api-reference/retriever-api-keys/create-retriever-api-key

post /v1/retrievers/{retriever_id}/api-keys
Generate a scoped API key for executing this specific retriever.

    **Use Cases:**
    - Provide external services with execution-only access
    - Embed retriever calls in customer applications
    - Create separate keys for staging vs production
    - Implement per-customer access keys for SaaS products

    **Security:**
    - Keys grant EXECUTE_RETRIEVER permission only
    - Keys are scoped to single retriever (cannot access others)
    - Keys inherit org's rate limits
    - Keys can be revoked instantly
    - Key prefix (ret_sk_abc...) shown in UI for identification

    **Ownership:**
    - Only the organization that owns the retriever can create keys
    - Verified by matching internal_id + namespace_id

    **Key Format:**
    - Prefix: ret_sk_
    - Length: 60 characters
    - Example: ret_sk_abcdefghijklmnopqrstuvwxyz123456789...

    **Response:**
    - Plaintext key shown ONLY ONCE in response
    - Save the key immediately - it cannot be retrieved later
    - Key prefix stored for identification in UI



# List Retriever API Keys
Source: https://docs.mixpeek.com/docs/api-reference/retriever-api-keys/list-retriever-api-keys

get /v1/retrievers/{retriever_id}/api-keys
List all API keys for this retriever.

    **Fields:**
    - key_id: Public identifier for the key
    - key_prefix: First 10 characters + "..." for identification (e.g., "ret_sk_abc...")
    - name: Human-friendly label
    - created_at: When the key was created
    - last_used_at: When the key was last used (if ever)
    - status: ACTIVE, REVOKED, or EXPIRED
    - expires_at: Expiration timestamp (if set)

    **Note:**
    - Plaintext key is NEVER returned in list responses
    - Only shown once in creation response
    - Use key_prefix to identify keys in the UI



# Revoke Retriever API Key
Source: https://docs.mixpeek.com/docs/api-reference/retriever-api-keys/revoke-retriever-api-key

delete /v1/retrievers/{retriever_id}/api-keys/{key_id}
Revoke a retriever-scoped API key.

    **Effect:**
    - Key status is set to REVOKED
    - Key can no longer be used for authentication
    - Revocation is immediate (no grace period)
    - Auth cache is invalidated immediately
    - Cannot be undone (create a new key if needed)

    **Audit:**
    - Revocation is logged in the retriever's audit trail
    - Includes actor user ID and timestamp



# Create benchmark
Source: https://docs.mixpeek.com/docs/api-reference/retriever-benchmarks/create-benchmark

post /v1/retrievers/benchmarks
Create a new benchmark run to compare retriever pipelines. The benchmark will replay historical sessions and measure alignment with observed user behavior.



# Delete benchmark
Source: https://docs.mixpeek.com/docs/api-reference/retriever-benchmarks/delete-benchmark

delete /v1/retrievers/benchmarks/{benchmark_id}
Delete a benchmark and its results



# Get benchmark
Source: https://docs.mixpeek.com/docs/api-reference/retriever-benchmarks/get-benchmark

get /v1/retrievers/benchmarks/{benchmark_id}
Get benchmark status and results by ID



# List benchmarks
Source: https://docs.mixpeek.com/docs/api-reference/retriever-benchmarks/list-benchmarks

get /v1/retrievers/benchmarks
List benchmarks with optional filtering



# Create evaluation dataset
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/create-evaluation-dataset

post /v1/retrievers/evaluations/datasets
Create a ground truth dataset for evaluating retrievers. Include queries with their relevant documents and optional graded relevance scores.



# Discover evaluation endpoints
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/discover-evaluation-endpoints

get /v1/evaluations
Lightweight discovery endpoint. Evaluation **datasets** are namespace-scoped and evaluation **runs** are retriever-scoped, so both live under `/v1/retrievers/...` rather than a top-level `/v1/evaluations/{id}`. This endpoint exists so a bare `GET /v1/evaluations` returns the real endpoint map instead of a hintless 404.



# Generate evaluation dataset from interactions
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/generate-from-interactions

post /v1/retrievers/{retriever_id}/evaluations/generate-from-interactions
Auto-generate a ground truth evaluation dataset by mining user interactions for this retriever. Queries with sufficient positive interactions (click, purchase, bookmark, etc.) are used to infer relevance labels, creating a dataset suitable for offline evaluation.



# Get evaluation dataset
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/get-evaluation-dataset

get /v1/retrievers/evaluations/datasets/{dataset_identifier}
Retrieve a specific dataset by ID or name



# Get evaluation results
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/get-evaluation-results

get /v1/retrievers/{retriever_id}/evaluations/{evaluation_id}
Retrieve evaluation results with all calculated metrics



# List evaluations
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/list-evaluations

get /v1/retrievers/{retriever_id}/evaluations
List all evaluations for a retriever with optional filters



# Run evaluation
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/run-evaluation

post /v1/retrievers/{retriever_id}/evaluations
Evaluate a retriever's quality using a ground truth dataset. Returns immediately with a task ID - evaluation runs asynchronously.



# Score predictions vs ground truth (stateless)
Source: https://docs.mixpeek.com/docs/api-reference/retriever-evaluations/score-predictions

post /v1/evaluations/score
Compute quality metrics (Precision@K, Recall@K, F1@K, F2@K) for precomputed predictions vs ground truth, with NO retriever, namespace, or persistence. Each item's `predicted` and `ground_truth` are treated as SETS. Send a single pair (`predicted` + `ground_truth`) or an `items` batch. Returns per-item scores and the macro-average across items. Auth only (no X-Namespace). This is the stateless counterpart to the retriever-scoped evaluation runs — use it to dogfood F1/F2 on extraction/generation benchmarks.



# Add a taxonomy node derived from a document selection
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/add-taxonomy-node-derived-from-document

post /v1/taxonomies/{taxonomy_identifier}/nodes/from-documents
Add a node to an existing **flat** taxonomy from a document selection.

The node is an ordinary document in the taxonomy's node (source) collection:
its anchor vector is the **centroid of the referenced documents' stored
vectors** and its `label` is `node_label`. Future documents enriched through
this taxonomy are labeled with the nearest node's `label`.

Only stored vectors are read — no re-embedding or GPU work. Documents that
can't contribute (missing, no matching vector) are skipped and reported in
`documents_skipped`; the call fails only when NO document contributes.

Hierarchical taxonomies are not supported here (their nodes are whole
collections) — the error points you to the flat-taxonomy path.



# Clone Taxonomy
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/clone-taxonomy

post /v1/taxonomies/{taxonomy_identifier}/clone
Clone a taxonomy with optional modifications.

    **Purpose:**
    Creates a NEW taxonomy (with new ID) based on an existing one. This is the
    recommended way to iterate on taxonomy designs when you need to modify core
    logic that PATCH doesn't allow (config, retriever_id, input_mappings).

    **Clone vs PATCH vs Template:**
    - **PATCH**: Update metadata only (name, description, metadata)
    - **Clone**: Copy and modify core logic (config, retriever, collections)
    - **Template**: Start from a pre-configured pattern (for new projects)

    **Common Use Cases:**
    - Fix configuration errors without losing join history
    - Change retriever or input mappings
    - Modify enrichment fields or collection configuration
    - Test modifications before replacing production taxonomy
    - Create variants for different datasets

    **How it works:**
    1. Source taxonomy is copied
    2. You provide a new name (REQUIRED)
    3. Optionally override any other fields (description, config)
    4. A new taxonomy is created with a new ID
    5. Original taxonomy remains unchanged



# Create Taxonomy
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/create-taxonomy

post /v1/taxonomies
Create a taxonomy and return the created resource.



# Create a taxonomy from a document selection or a cluster run (one gesture)
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/create-taxonomy-from-document-selection

post /v1/taxonomies/from-documents
Promote a document selection (e.g. a lasso/cluster selection in the cluster
visualization) into a NEW flat taxonomy in one call.

Creates, in order:
1. a **node collection** mirroring the selection's vector index (same vector
   name, dimensions, and feature_uri — no re-embedding, no GPU work),
2. a **vector-search retriever** over it,
3. the **flat taxonomy** wired to both (input_mappings map an incoming
   document's stored embedding onto the retriever),
4. the **nodes**, from one of two sources:
   - `documents` + `node_label`: one node whose anchor vector is the centroid
     of the referenced documents' stored vectors;
   - `cluster_run` (`{cluster_id, run_id}`): one node per cluster in that run,
     labeled with the cluster's label and anchored on its stored centroid, so
     a whole cluster run becomes a classifier in one call. The response's
     `nodes` lists every node created.

Documents enriched through the taxonomy afterwards are labeled with the
nearest node's `label`. To add more nodes later, use
`POST /v1/taxonomies/{taxonomy_id}/nodes/from-documents`.

Resources created before a failing step are cleaned up best-effort.



# Create Taxonomy Version
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/create-taxonomy-version

post /v1/taxonomies/{taxonomy_id}/versions
Create a new version for a taxonomy with a new config snapshot.



# Delete Taxonomy
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/delete-taxonomy

delete /v1/taxonomies/{taxonomy_identifier}
This endpoint deletes a taxonomy and all its resources including:
    - Taxonomy versions (version snapshots)
    - Taxonomy metadata from MongoDB

    The deletion is performed synchronously and returns when complete.



# Get Taxonomy
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/get-taxonomy

get /v1/taxonomies/{taxonomy_identifier}
Get a taxonomy by ID or name.



# List Taxonomies
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/list-taxonomies

post /v1/taxonomies/list
List taxonomies with optional filters and pagination.



# List taxonomies (GET alias for POST /list).
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/list-taxonomies-get

get /v1/taxonomies
List taxonomies with optional filters and pagination.



# List Taxonomy Versions
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/list-taxonomy-versions

get /v1/taxonomies/{taxonomy_id}/versions
List all versions for a taxonomy (head included as latest).



# Partially Update Taxonomy
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/partially-update-taxonomy

patch /v1/taxonomies/{taxonomy_identifier}
Update a taxonomy's metadata.

    **Metadata Only Updates:**
    This endpoint allows updating ONLY metadata fields. Core taxonomy logic is immutable
    to ensure consistency for join history and dependent resources.

    **Fields You CAN Update:**
    - taxonomy_name: Rename the taxonomy
    - description: Update documentation
    - metadata: Update custom metadata

    **Fields You CANNOT Update:**
    - config: Taxonomy configuration (retriever_id, input_mappings, collections)
    - taxonomy_type: Type (flat vs hierarchical)

    **Need to Modify Core Logic?**
    Use POST /{taxonomy_identifier}/clone instead to modify configuration,
    retriever_id, input_mappings, or collections.



# Test taxonomy configuration (validation only)
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/test-taxonomy-configuration

post /v1/taxonomies/{taxonomy_identifier}/execute
⚠️ VALIDATION ENDPOINT ONLY - Not for production enrichment!



# Test taxonomy configuration (validation only) — DEPRECATED path
Source: https://docs.mixpeek.com/docs/api-reference/taxonomies/test-taxonomy-configuration-validation-only

post /v1/taxonomies/execute/{taxonomy_identifier}
⚠️ VALIDATION ENDPOINT ONLY - Not for production enrichment!

DEPRECATED path shape — prefer POST /taxonomies/{id}/execute (id-first,
consistent with retrievers). This verb-first path still works.

This endpoint validates taxonomy configuration with 1-5 sample documents.
Results are returned immediately and NOT persisted to any collection.

❌ DO NOT USE FOR:
- Enriching entire collections (use taxonomy_applications instead)
- Batch processing documents (automatic during ingestion)
- Persisting enriched documents (use retriever pipelines instead)

✅ USE THIS FOR:
- Testing taxonomy configuration is correct
- Validating retriever finds matching taxonomy nodes
- Checking enrichment fields are properly applied
- Development/debugging taxonomy setup

📚 FOR PRODUCTION ENRICHMENT:

Automatic (during ingestion):
  1. Create taxonomy: POST /v1/taxonomies
  2. Attach to collection: PATCH /v1/collections/{id} with taxonomy_applications field
  3. Ingest documents: Documents are automatically enriched by engine

On-the-fly (during retrieval):
  1. Add taxonomy_join stage to retriever pipeline
  2. Execute retriever: POST /v1/retrievers/{id}/execute
  3. Results include enriched documents (not persisted)

See API documentation for Collections and Retrievers for details.



# Analyze multi-step transition paths
Source: https://docs.mixpeek.com/docs/api-reference/taxonomy-analytics/analyze-multi-step-transition-paths

post /v1/taxonomies/{taxonomy_id}/analytics/paths
Discover the most common multi-step paths documents take between two taxonomy steps.

Unlike the `/transitions` endpoint which only analyzes direct A→B transitions,
this endpoint reveals the intermediate steps documents actually take.

## Use Cases

**Email Thread Analysis:**
- Question: What paths do emails take from "inquiry" to "closed_won"?
- Discover: Some go inquiry → followup → proposal → closed_won
- Discover: Others skip steps: inquiry → proposal → closed_won
- Discover: Fast track: inquiry → closed_won

**Content Editorial Paths:**
- Question: Common paths from "draft" to "published"?
- Discover: draft → review → edit → review → published
- Discover: draft → review → published (expedited)
- Discover: Paths that loop back (draft → review → draft → review)

**Compliance Resolution Paths:**
- Question: How do violations get resolved?
- Discover: violation → investigated → remediated → resolved
- Discover: violation → false_positive → closed
- Discover: Escalation paths: violation → escalated → legal_review → resolved

## Requirements

- Taxonomy must have `step_analytics` configured
- Collection must contain documents with timestamp and sequence_id fields

## Returns

**Completion Metrics:**
- `total_sequences`: Sequences starting at from_step
- `completed_sequences`: Number reaching to_step
- `completion_rate`: Percentage that completed

**Paths (sorted by frequency):**
- `path`: Ordered sequence of steps
- `count`: Number of sequences following this path
- `percentage`: Percentage of completing sequences
- `avg_duration_sec`: Average time for this path

## Example Request

```json
{
    "collection_id": "col_emails",
    "taxonomy_id": "tax_sales_stages",
    "from_step": "inquiry",
    "to_step": "closed_won",
    "max_path_length": 10,
    "min_support": 5
}
```

## Example Response

```json
{
    "from_step": "inquiry",
    "to_step": "closed_won",
    "total_sequences": 1000,
    "completed_sequences": 350,
    "completion_rate": 0.35,
    "paths": [
        {
            "path": ["inquiry", "followup", "proposal", "closed_won"],
            "count": 120,
            "percentage": 34.3,
            "avg_duration_sec": 604800.0
        },
        {
            "path": ["inquiry", "proposal", "closed_won"],
            "count": 90,
            "percentage": 25.7,
            "avg_duration_sec": 432000.0
        },
        {
            "path": ["inquiry", "closed_won"],
            "count": 70,
            "percentage": 20.0,
            "avg_duration_sec": 172800.0
        }
    ]
}
```

## Path Interpretation

**Length Analysis:**
- Shorter paths indicate efficient progression
- Longer paths may indicate complexity or bottlenecks
- Loops (repeated steps) indicate rework or revisions

**Duration Analysis:**
- Compare avg_duration_sec across paths
- Shorter paths may not always be faster
- Identify optimization opportunities

**Frequency Analysis:**
- High-percentage paths are "happy paths"
- Low-percentage paths may be edge cases or exceptions
- Missing expected paths indicate drop-off points



# Compute step transition analytics
Source: https://docs.mixpeek.com/docs/api-reference/taxonomy-analytics/compute-step-transition-analytics

post /v1/taxonomies/{taxonomy_id}/analytics/transitions
Analyze how documents progress from one taxonomy step to another.

This endpoint computes conversion rates, duration statistics, and predictor lifts
for documents transitioning between taxonomy labels.

## Use Cases

**Email Thread Analysis:**
- Question: How long from "inquiry" to "closed_won"?
- Question: What % of inquiries result in sales?
- Question: Which sender domains have highest conversion?

**Content Workflow Tracking:**
- Question: Conversion rate from "draft" to "published"?
- Question: How long does content stay in review?
- Question: Which authors publish fastest?

**Safety Compliance Monitoring:**
- Question: Time from violation detection to resolution?
- Question: Success rate for remediation efforts?

## Requirements

- Taxonomy must have `step_analytics` configured (or provide `override_step_analytics`)
- Collection must contain documents enriched with this taxonomy
- Documents must have timestamp and sequence grouping fields configured

## Returns

**Conversion Metrics:**
- `count`: Total sequences starting at from_step
- `converted`: Number reaching to_step
- `conversion_rate`: Percentage that converted

**Duration Statistics (if converted > 0):**
- `mean`, `median`: Average and middle duration
- `p90`, `p95`: 90th and 95th percentile durations
- `std_dev`, `min`, `max`: Distribution statistics

**Top Predictors:**
- Covariates with highest impact on conversion
- Lift values (>1.0 = increases conversion, <1.0 = decreases)
- Statistical significance via minimum support threshold

## Example Request

```json
{
    "collection_id": "col_emails",
    "taxonomy_id": "tax_sales_stages",
    "from_step": "inquiry",
    "to_step": "closed_won",
    "max_window_days": 90,
    "min_support": 10
}
```

## Example Response

```json
{
    "from_step": "inquiry",
    "to_step": "closed_won",
    "count": 1000,
    "converted": 350,
    "conversion_rate": 0.35,
    "durations_sec": {
        "mean": 432000.0,
        "median": 345600.0,
        "p50": 345600.0,
        "p90": 691200.0,
        "p95": 864000.0
    },
    "top_predictors": [
        {
            "field": "Sender Domain",
            "value": "enterprise.com",
            "count": 150,
            "conversion_rate": 0.75,
            "lift": 2.14
        }
    ]
}
```



# Get Available Steps
Source: https://docs.mixpeek.com/docs/api-reference/taxonomy-analytics/get-available-steps

get /v1/taxonomies/{taxonomy_id}/analytics/available-steps
Get all available steps for a taxonomy and collection.

This endpoint discovers what steps exist in your analytics data by querying
the taxonomy_events analytics collection. Use this before querying transitions
or paths to understand what step values you can use.

**Use Cases:**
- Discover available steps before querying analytics
- Validate step names (avoid typos in from_step/to_step)
- See which steps have the most events
- Check data freshness (first_seen/last_seen timestamps)

**Example Usage:**
```python
# 1. Get available steps
GET /v1/taxonomies/tax_sales/analytics/available-steps?collection_id=col_emails

# Response:
{
    "taxonomy_id": "tax_sales",
    "collection_id": "col_emails",
    "total_events": 5432,
    "total_sequences": 1000,
    "steps": [
        {"step_key": "inquiry", "event_count": 1000, ...},
        {"step_key": "followup", "event_count": 450, ...},
        {"step_key": "closed_won", "event_count": 350, ...}
    ]
}

# 2. Use discovered steps in transition query
POST /v1/taxonomies/tax_sales/analytics/transitions
{
    "collection_id": "col_emails",
    "from_step": "inquiry",      # From available steps
    "to_step": "closed_won"      # From available steps
}
```

Args:
    request: FastAPI request object (contains tenant context)
    taxonomy_id: Taxonomy ID to query
    collection_id: Collection ID for filtering events

Returns:
    AvailableStepsResponse with all steps sorted by event count (descending)

Raises:
    NotFoundError: If taxonomy not found
    ValidationError: If unable to query the analytics store



# Get a taxonomy analytics run
Source: https://docs.mixpeek.com/docs/api-reference/taxonomy-analytics/get-taxonomy-analytics-run

get /v1/taxonomies/{taxonomy_id}/analytics/runs/{run_id}
Retrieve a single persisted analytics run (its parameters and stored result) so it can be reviewed or re-run.



# List taxonomy analytics run history
Source: https://docs.mixpeek.com/docs/api-reference/taxonomy-analytics/list-taxonomy-analytics-run-history

get /v1/taxonomies/{taxonomy_id}/analytics/runs
List previously executed step-analytics runs (transitions and paths) for a taxonomy, newest-executed first. Runs are de-duplicated by their parameters, so re-running the same analysis updates the existing entry and bumps its run_count instead of creating a duplicate.



# Re-run a stored taxonomy analytics run
Source: https://docs.mixpeek.com/docs/api-reference/taxonomy-analytics/re-run-stored-taxonomy-analytics-run

post /v1/taxonomies/{taxonomy_id}/analytics/runs/{run_id}/rerun
Re-execute a previously persisted analytics run using its stored parameters and return the FRESH result. The stored run is updated in place with the new result (run_count is incremented).



# List Triggers (GET alias for POST /list).
Source: https://docs.mixpeek.com/docs/api-reference/triggers/list-triggers

get /v1/triggers
List triggers with filters and pagination.



# Update Trigger
Source: https://docs.mixpeek.com/docs/api-reference/update-trigger

patch /v1/triggers/{trigger_id}
Update a trigger.

    **Allowed updates:**
    - `schedule_config`: Modify trigger schedule
    - `description`: Update description
    - `status`: Change status (prefer using pause/resume endpoints)

    **Not allowed:**
    - `action_type`: Must delete and recreate
    - `trigger_type`: Must delete and recreate
    - `action_config`: Must delete and recreate



# Create Webhook
Source: https://docs.mixpeek.com/docs/api-reference/webhooks/create-webhook

post /v1/organizations/webhooks
Create a new webhook for the user's organization.



# Get Webhook
Source: https://docs.mixpeek.com/docs/api-reference/webhooks/get-webhook

get /v1/organizations/webhooks/{identifier}
Get a single webhook by its ID.



# List Webhooks
Source: https://docs.mixpeek.com/docs/api-reference/webhooks/list-webhooks

post /v1/organizations/webhooks/list
List all webhooks for the user's organization.



# Update Webhook
Source: https://docs.mixpeek.com/docs/api-reference/webhooks/update-webhook

put /v1/organizations/webhooks/{identifier}
Update an existing webhook.



# Whoami
Source: https://docs.mixpeek.com/docs/api-reference/account/whoami

get /v1/me
Return the caller's identity: organization, tier, key metadata, limits.

Works for every valid key. `/account` is an undocumented alias so both
natural spellings resolve.



# Create Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/create-alert

post /v1/alerts
Create a new alert that monitors document ingestion and sends notifications.

    Alerts attach retrievers to collections. When new documents are ingested,
    the alert runs the retriever and sends notifications if matches are found.

    **Key Components:**
    - `retriever_id`: References a retriever that defines query logic (filters, scoring, limits)
    - `notification_config`: Defines where to send notifications (webhook, Slack, email)

    **Note:** The retriever owns all query semantics. The alert's job is simply
    to run the retriever and notify if results exist.



# Create Org-Scoped Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/create-org-scoped-alert

post /v1/organizations/alerts
Create an organization-level alert that applies across namespaces via its `namespace_selector` (all or a selected set). Use this for system alerts (collection/batch/retriever health) that should cover the whole org rather than a single namespace.



# Delete Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/delete-alert

delete /v1/alerts/{alert_identifier}
Delete an alert and its execution history.

    This operation:
    - Removes the alert from MongoDB
    - Deletes all execution history for this alert
    - Does NOT affect the referenced retriever



# Delete Org-Scoped Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/delete-org-scoped-alert

delete /v1/organizations/alerts/{alert_identifier}
Delete an org-scoped alert.



# Get Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/get-alert

get /v1/alerts/{alert_identifier}
Get an alert by ID (alt_...) or name.



# Get Alert Execution
Source: https://docs.mixpeek.com/docs/api-reference/alerts/get-alert-execution

get /v1/alerts/executions/{execution_id}
Get a single alert execution result by execution ID. Use this to poll for alert results without needing a webhook.



# Get Latest Alert Result
Source: https://docs.mixpeek.com/docs/api-reference/alerts/get-latest-alert-result

get /v1/alerts/{alert_identifier}/results
Get the most recent execution result for an alert. Use this endpoint to poll for alert results without a webhook. Returns the latest execution with match details, or 404 if the alert has never executed.



# Get Org-Scoped Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/get-org-scoped-alert

get /v1/organizations/alerts/{alert_identifier}
Get an org-scoped alert by ID (alt_...) or name.



# List Alert Executions
Source: https://docs.mixpeek.com/docs/api-reference/alerts/list-alert-executions

get /v1/alerts/{alert_identifier}/executions
List execution history for a specific alert.



# List Alerts
Source: https://docs.mixpeek.com/docs/api-reference/alerts/list-alerts

post /v1/alerts/list
List all alerts in the namespace with optional filtering and pagination.

    **Filtering:**
    - Use `search` for wildcard text search across alert_id, name, description
    - Use `filters` for structured queries

    **Sorting:**
    - Default: created_at descending (newest first)



# List Alerts (GET alias for POST /list).
Source: https://docs.mixpeek.com/docs/api-reference/alerts/list-alerts-get

get /v1/alerts
List alerts with pagination and filtering.



# List All Executions
Source: https://docs.mixpeek.com/docs/api-reference/alerts/list-all-executions

get /v1/alerts/executions
List execution history for all alerts in the namespace.



# List Org-Scoped Alerts (GET alias for POST /list).
Source: https://docs.mixpeek.com/docs/api-reference/alerts/list-org-scoped-alerts

get /v1/organizations/alerts
List org-scoped alerts with pagination and filtering.



# List Org-Scoped Alerts
Source: https://docs.mixpeek.com/docs/api-reference/alerts/list-org-scoped-alerts-list

post /v1/organizations/alerts/list
List all organization-level alerts (scoped by internal_id).



# Update Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/update-alert

patch /v1/alerts/{alert_identifier}
Partially update an alert's configuration.

    **All fields are optional** - provide only what you want to update.

    Unlike taxonomies, alerts can be fully updated including:
    - `name`: Rename the alert
    - `description`: Update documentation
    - `retriever_id`: Change the retriever
    - `notification_config`: Update notification channels
    - `enabled`: Enable/disable the alert
    - `metadata`: Update custom metadata



# Update Org-Scoped Alert
Source: https://docs.mixpeek.com/docs/api-reference/alerts/update-org-scoped-alert

patch /v1/organizations/alerts/{alert_identifier}
Partially update an org-scoped alert (e.g. enable/disable, channels).



# Batch Queue Status
Source: https://docs.mixpeek.com/docs/api-reference/batches/batch-queue-status

get /v1/batches/queue
View the Ray job queue: active jobs, max concurrency, and waiting batches with priorities.



# Cancel Batch
Source: https://docs.mixpeek.com/docs/api-reference/batches/cancel-batch

post /v1/batches/{batch_id}/cancel
Cancel a submitted/processing batch by ID without requiring the bucket_id. Cancels all associated Ray jobs and queued work, releases queue slots, and marks the batch as CANCELLED.



# Get Batch by ID
Source: https://docs.mixpeek.com/docs/api-reference/batches/get-batch-by-id

get /v1/batches/{batch_id}
Retrieve a single batch by its ID without requiring the bucket_id. Returns full batch details including real-time progress (objects_processed, total_objects, items_per_second, eta_seconds) for in-flight batches.



# List All Batches
Source: https://docs.mixpeek.com/docs/api-reference/batches/list-all-batches

post /v1/batches/list
List batches across the organization. The X-Namespace header narrows the list to that namespace, exactly like every other namespaced endpoint; omit it for the org-wide view. `namespace_id` in the request body also works (and wins ties when it agrees with the header; a conflicting header + body is a 400). Other body filters: `status`, `bucket_id`, `collection_id`.



# List Dsar Requests
Source: https://docs.mixpeek.com/docs/api-reference/data-subject-access-requests/list-dsar-requests

get /v1/organizations/dsar/status
List all DSAR requests for this organization.

Returns the status of all pending and completed data subject
access requests (exports and deletions).

Requires ADMIN permission.



# Request Data Deletion
Source: https://docs.mixpeek.com/docs/api-reference/data-subject-access-requests/request-data-deletion

post /v1/organizations/dsar/delete
Request deletion of all organization data.

This is a DSAR deletion request per GDPR Article 17 (Right to Erasure)
and CCPA Section 1798.105. The request is logged and processed within
30 days. Once executed, this action is IRREVERSIBLE.

The actual deletion is performed by the delete_organization_resources_flow
after verification. This endpoint creates the request record.

Requires ADMIN permission.



# Request Data Export
Source: https://docs.mixpeek.com/docs/api-reference/data-subject-access-requests/request-data-export

post /v1/organizations/dsar/export
Request a full data export for this organization.

Returns a summary of all data categories that will be exported.
The export is processed asynchronously and delivered within 30 days
as required by GDPR Article 15 and CCPA Section 1798.100.

Requires ADMIN permission.



# Advance Funnel Stage
Source: https://docs.mixpeek.com/docs/api-reference/notifications/advance-funnel-stage

post /v1/notifications/funnel/advance
Advance the funnel stage from the client side.

Studio calls this when key milestones happen in-browser that the server
cannot observe directly (e.g. first_search executed in the test tab).
Accepts both resource-creation stages and activation funnel stages.

``source`` is optional provenance for the stage being advanced, recorded on
the analytics event so a stage reached two different ways can be counted
separately. It distinguishes a click on sample data from a real upload, and
a search of a sample retriever from a search of the caller's own data, which
the stage name alone cannot. Ignored for resource-creation stages.



# Delete Notification
Source: https://docs.mixpeek.com/docs/api-reference/notifications/delete-notification

delete /v1/notifications/{notification_id}
Delete a notification (idempotent - succeeds even if already deleted).



# Get Funnel State
Source: https://docs.mixpeek.com/docs/api-reference/notifications/get-funnel-state

get /v1/notifications/funnel/state
Get the current funnel state for the organization.

Returns the user's position in the onboarding funnel, including:
- Current stage
- When each stage was reached
- Nudges that have been sent
- Last activity timestamp



# Get Notification
Source: https://docs.mixpeek.com/docs/api-reference/notifications/get-notification

get /v1/notifications/{notification_id}
Get a single notification by its ID.



# Get Preferences
Source: https://docs.mixpeek.com/docs/api-reference/notifications/get-preferences

get /v1/notifications/preferences
Get notification preferences for the organization.



# Get recent email sends
Source: https://docs.mixpeek.com/docs/api-reference/notifications/get-recent-email-sends

get /v1/notifications/emails/recent



# Get Reminder Preferences
Source: https://docs.mixpeek.com/docs/api-reference/notifications/get-reminder-preferences

get /v1/notifications/preferences/reminders
Get reminder/nudge email preferences for the organization.

These preferences control funnel-based onboarding nudges,
re-engagement emails, and feature tips.



# Get Unread Count
Source: https://docs.mixpeek.com/docs/api-reference/notifications/get-unread-count

get /v1/notifications/unread/count
Get count of unread notifications.



# List Notifications
Source: https://docs.mixpeek.com/docs/api-reference/notifications/list-notifications

post /v1/notifications/list
List all notifications for the user's organization.



# Mark All As Read
Source: https://docs.mixpeek.com/docs/api-reference/notifications/mark-all-as-read

post /v1/notifications/read/all
Mark all notifications as read for a user.



# Mark As Read
Source: https://docs.mixpeek.com/docs/api-reference/notifications/mark-as-read

post /v1/notifications/{notification_id}/read
Mark a notification as read.



# Update Preferences
Source: https://docs.mixpeek.com/docs/api-reference/notifications/update-preferences

put /v1/notifications/preferences
Update notification preferences for the organization.



# Update Reminder Preferences
Source: https://docs.mixpeek.com/docs/api-reference/notifications/update-reminder-preferences

put /v1/notifications/preferences/reminders
Update reminder/nudge email preferences for the organization.

You can update individual preferences without affecting others:
- enabled: Master toggle for all reminder emails
- onboarding_nudges: Funnel progression nudges
- engagement_reminders: Re-engagement emails for inactivity
- feature_tips: Helpful tips and inspiration
- quiet_hours_start/end: Hours (0-23 UTC) when no emails are sent



# Create Api Key
Source: https://docs.mixpeek.com/docs/api-reference/organization-api-keys/create-api-key

post /v1/organizations/users/{user_email}/api-keys
Create a new API key for a user.



# Delete Api Key
Source: https://docs.mixpeek.com/docs/api-reference/organization-api-keys/delete-api-key

delete /v1/organizations/users/{user_email}/api-keys/{key_name}
Revoke an API key.

🔒 The "admin-key" is protected and cannot be deleted.

Studio session keys can be revoked here by name or key_id; list them with
include_session_keys=true.

Refuses when a live published app still embeds this key, naming the app.
A 2026-06-03 revocation killed 2 published apps for seven weeks
because nothing in this path looked for references and the call reported
success.



# List Api Keys
Source: https://docs.mixpeek.com/docs/api-reference/organization-api-keys/list-api-keys

get /v1/organizations/users/{user_email}/api-keys
List API keys for a user.



# Rotate Api Key
Source: https://docs.mixpeek.com/docs/api-reference/organization-api-keys/rotate-api-key

post /v1/organizations/users/{user_email}/api-keys/{key_name}/rotate
Rotate an API key and return the new secret.

🔒 The "admin-key" is protected and cannot be rotated.



# Update Api Key
Source: https://docs.mixpeek.com/docs/api-reference/organization-api-keys/update-api-key

patch /v1/organizations/users/{user_email}/api-keys/{key_name}
Update an API key's metadata or permissions.

🔒 The "admin-key" is protected and cannot be modified.



# Export Audit Logs
Source: https://docs.mixpeek.com/docs/api-reference/organization-audit/export-audit-logs

get /v1/organizations/audit/logs/export
Export a time-bounded, filterable audit evidence bundle.

Streams newline-delimited JSON (NDJSON): one audit event per line with
resolved actor identity (name/email/key name), newest first. If the cap
truncates the export, the FINAL line is a marker object
`{"__truncated__": true, "exported": N, "total": T}` — a cut bundle
always says so rather than presenting as complete. Requires ADMIN.



# Get Audit Log
Source: https://docs.mixpeek.com/docs/api-reference/organization-audit/get-audit-log

get /v1/organizations/audit/logs/{audit_id}
Get a specific audit log entry by ID.

Requires ADMIN permission.



# Get Audit Settings
Source: https://docs.mixpeek.com/docs/api-reference/organization-audit/get-audit-settings

get /v1/organizations/audit/settings
Get current audit configuration for the organization.

Returns the audit settings including whether read auditing is enabled.
Requires ADMIN permission.



# List Audit Logs
Source: https://docs.mixpeek.com/docs/api-reference/organization-audit/list-audit-logs

get /v1/organizations/audit/logs
List organization audit logs with filtering and pagination.

Returns audit events for the organization, sorted by timestamp descending.
Requires ADMIN permission.



# Update Audit Settings
Source: https://docs.mixpeek.com/docs/api-reference/organization-audit/update-audit-settings

patch /v1/organizations/audit/settings
Update audit configuration for the organization.

Use this to enable or disable read auditing.
Requires ADMIN permission.



# Apply Promo
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/apply-promo

post /v1/organizations/billing/apply-promo
Apply a promo code to the organization's active subscription.

Every failure carries a reason (invalid / expired / exhausted / product
mismatch / no subscription) — never a silent no-op.



# Cancel Subscription
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/cancel-subscription

post /v1/organizations/billing/subscription/cancel
Cancel the current subscription.

Default: cancel at end of billing period (the subscription remains
active until then, after which the organization downgrades). With
``immediate=true``: cancel NOW — the org-deletion cascade
and teardown callers need a cancel that actually ends billing today
instead of leaving a live sub on the card.



# Confirm Payment Method
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/confirm-payment-method

post /v1/organizations/billing/confirm-payment-method
Confirm payment method after frontend collects it.

After Stripe Elements confirms the SetupIntent, call this endpoint
to attach the payment method to the customer and enable auto-billing.

**Requirements:**
- Admin permission
- Must have called setup-payment-method first

**Example:**
```python
# After Stripe Elements confirms setup
response = await client.post(
    "/v1/organizations/billing/confirm-payment-method",
    json={"payment_method_id": "pm_1ABC2DEF3GHI"}
)
```



# Create Checkout
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/create-checkout

post /v1/organizations/billing/checkout
Create a Stripe Checkout Session for a one-time credit purchase.

Allows organization admins to purchase credit packages via Stripe Checkout.
After successful payment, credits are automatically added to the organization.

**Available Packages:**
- `starter`: 10,000 credits — $10
- `growth`: 50,000 credits — $40
- `scale`: 200,000 credits — $120

**Requirements:**
- Admin permission

**Example:**
```python
response = await client.post(
    "/v1/organizations/billing/checkout",
    json={"package": "growth"}
)
# Redirect user to response["checkout_url"]
```



# Disable Auto Billing
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/disable-auto-billing

post /v1/organizations/billing/disable-auto-billing
Disable automatic monthly billing.

Disables automatic billing but keeps payment method saved.
Organization can re-enable later or pay invoices manually.

**Requirements:**
- Admin permission

**Example:**
```python
response = await client.post("/v1/organizations/billing/disable-auto-billing")
```



# Enable Auto Billing
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/enable-auto-billing

post /v1/organizations/billing/enable-auto-billing
Enable automatic monthly billing.

Re-enables automatic billing if it was previously disabled.
Payment method must already be saved.

**Requirements:**
- Admin permission
- Must have payment method saved

**Example:**
```python
response = await client.post("/v1/organizations/billing/enable-auto-billing")
```



# Estimate Cost
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/estimate-cost

post /v1/organizations/billing/estimate
Quote the v2 cost of planned ingestion before submitting it.

Forgiving by design: unknown MIME types quote at the cheapest band;
wrong unit fields and inapplicable features return teaching errors.



# Generate Invoice Now
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/generate-invoice-now

post /v1/organizations/billing/generate-invoice
Manually trigger invoice generation for this organization.

Useful for testing the billing flow or generating retroactive invoices.
Requires ADMIN permission.

Line items follow the v2 (modality+features, D8) composition: flat monthly
fee + per-modality feature usage in natural units and dollars + the plan's
included-usage pool as a deduction; only usage beyond the pool is due. The
total always reconciles to the month's ledger charges (see `pricing_v2`).

**Example:**
```python
# Dry run
response = await client.post(
    "/v1/organizations/billing/generate-invoice",
    json={"dry_run": true}
)

# Actually generate
response = await client.post(
    "/v1/organizations/billing/generate-invoice",
    json={"billing_month": "2026-01"}
)
```



# Get Credit Balance
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-credit-balance

get /v1/organizations/billing/balance
Get current credit balance and tier information.

Returns the organization's credit balance, current tier, and usage statistics.
Useful for displaying billing status in dashboards.

**Requirements:**
- Read permission

**Example:**
```python
response = await client.get("/v1/organizations/billing/balance")
print(f"Balance: {response['credit_balance']} credits")
print(f"Tier: {response['account_tier']}")
```



# Get Current Usage
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-current-usage

get /v1/organizations/billing/usage/current
Get current month usage.

Returns credit consumption for the current billing period,
estimated cost, and next invoice date.

**Requirements:**
- Read permission

**Example:**
```python
response = await client.get("/v1/organizations/billing/usage/current")
print(f"Usage: {response['current_month_usage']} credits")
print(f"Estimated cost: ${response['estimated_cost_usd']}")
```



# Get Payment Method
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-payment-method

get /v1/organizations/billing/payment-method
Get current payment method.

Returns the saved payment method details (last 4 digits, brand)
and auto-billing status.

**Requirements:**
- Read permission

**Example:**
```python
response = await client.get("/v1/organizations/billing/payment-method")
if response["has_payment_method"]:
    print(f"Card: {response['payment_method']['card_brand']} ****{response['payment_method']['card_last4']}")
```



# Get Spending Caps
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-spending-caps

get /v1/organizations/billing/spending-caps
Get current spending cap configuration.

Returns spending cap settings including budget limits, alert thresholds,
and current spending status.

**Requirements:**
- Read permission

**Example:**
```python
response = await client.get("/v1/organizations/billing/spending-caps")
print(f"Monthly budget: ${response['monthly_spending_budget_usd']}")
print(f"Hard cap enabled: {response['hard_cap_enabled']}")
print(f"Current spending: ${response['current_spending_usd']}")
```



# Get Subscription Status
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-subscription-status

get /v1/organizations/billing/subscription
Get current subscription status.



# Get Usage Breakdown
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-usage-breakdown

get /v1/organizations/billing/usage/breakdown
Get detailed usage breakdown.

Returns usage breakdown by operation type and extractor
for the specified billing period.

**Query Parameters:**
- `billing_month`: Month to query (YYYY-MM format, defaults to current)

**Requirements:**
- Read permission

**Example:**
```python
# Current month
response = await client.get("/v1/organizations/billing/usage/breakdown")

# Specific month
response = await client.get(
    "/v1/organizations/billing/usage/breakdown",
    params={"billing_month": "2025-11"}
)
```



# Get Vector Backend Usage
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/get-vector-backend-usage

get /v1/organizations/billing/usage/vector-backend
Get vector backend usage breakdown.

Returns usage dimensions (vectors stored, storage, queries, writes) and
cost breakdown for the tenant's vector backend (MVS).

**Requirements:**
- Read permission



# List Invoices
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/list-invoices

get /v1/organizations/billing/invoices
List monthly invoices.

Returns paginated list of monthly invoices with links to
Stripe-hosted invoice pages.

**Query Parameters:**
- `limit`: Number of invoices (1-100, default 10)

**Requirements:**
- Read permission

**Example:**
```python
response = await client.get("/v1/organizations/billing/invoices?limit=10")
for invoice in response["invoices"]:
    print(f"{invoice['billing_month']}: ${invoice['amount_paid']/100}")
```



# List Transactions
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/list-transactions

get /v1/organizations/billing/transactions
List individual credit-usage transactions (most recent first).

The per-operation **credit ledger**: every credit debit, what consumed it
(operation/extractor/resource), and when — the transaction-level itemization
beyond the aggregated `/usage/breakdown` totals.

**Query Parameters:**
- `limit`: page size (1-200, default 50)
- `offset`: records to skip
- `billing_month`: optional YYYY-MM filter

**Requirements:** Read permission.

**Example:**
```python
response = await client.get(
    "/v1/organizations/billing/transactions",
    params={"limit": 50, "billing_month": "2026-06"},
)
```



# Open Billing Portal
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/open-billing-portal

post /v1/organizations/billing/portal
Open the Stripe customer billing portal.

This is the correct destination for a "Manage subscription" action: the
portal lets an existing customer update their card, view/download invoices,
and change or cancel their plan. It is NOT the same as POST /subscribe —
re-running /subscribe for an existing subscriber starts a brand-new checkout
instead of managing the current one.

Requires an existing Stripe customer. Orgs that have never started billing
have nothing to manage, so they get an actionable 400 pointing at /subscribe.



# Redeem Promo
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/redeem-promo

post /v1/organizations/billing/redeem-promo
Redeem a promo code on a FRESH org — no Checkout, no card.

A promo code lets you continue without entering a card. This creates the
subscription server-side with the coupon attached (the self-serve twin of the manual comp flow), stamps
``metadata.organization_id`` so the subscription webhook grants tier +
POC cap, and sets the org's plan fields inline so the user continues
IMMEDIATELY (no webhook-latency paywall).

Only codes whose window bills $0 (percent_off=100 — every POC code by
validated shape) can start service card-less: a partial discount with no
payment method would generate a FAILING first invoice (service cut +
dunning — strictly worse than the card form; the uncollectable shape
POC-PROMO-CAP-2026-07-17 exists to prevent). Partial codes get a clear
redirect to checkout, where the discount still applies. Card-ask happens
at coupon end (finances' T-7d lifecycle).

Every failure carries a reason — never a silent no-op.



# Setup Payment Method
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/setup-payment-method

post /v1/organizations/billing/setup-payment-method
Initialize payment method setup flow.

Creates a Stripe SetupIntent for collecting payment method without charging.
The client_secret should be used with Stripe Elements on the frontend.

**Flow:**
1. Frontend calls this endpoint
2. Backend creates Stripe Customer (if needed) and SetupIntent
3. Frontend uses client_secret with Stripe Elements
4. User enters card details
5. Frontend calls confirm-payment-method endpoint

**Requirements:**
- Admin permission (only org admins can set up payment methods)

**Example:**
```python
response = await client.post("/v1/organizations/billing/setup-payment-method")
client_secret = response["client_secret"]
# Use client_secret with Stripe Elements
```



# Subscribe To Plan
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/subscribe-plan

post /v1/organizations/billing/subscribe
Subscribe to a paid plan via Stripe Checkout.

Creates a Stripe Checkout Session in subscription mode. After payment,
the organization's account_type is updated via webhook.

If the organization already has an active subscription, the plan is
changed inline (upgrade/downgrade with proration) instead of creating
a new checkout session.

**Available Plans** (tier minimum is a billing floor; usage above it
bills automatically):
- `managed_build`: Managed Mixpeek Build — $25/mo min, 100K objects/mo
- `managed_scale`: Managed Mixpeek Scale — $250/mo min, 1M objects/mo
- `mvs_build`: MVS Standalone Build — $25/mo min, 1M vectors
- `mvs_scale`: MVS Standalone Scale — $250/mo min, 25M vectors

**Discounts:** each non-empty `onboarding_answers` field takes $5 off the
first invoice (max $10). An explicit `promo_code` is validated and
pre-applied; when both are sent the promo takes checkout's single
discount slot and the onboarding credit lands on the customer balance.
Without either, Stripe's own promo-code field is enabled at checkout.



# Update Spending Caps
Source: https://docs.mixpeek.com/docs/api-reference/organization-billing/update-spending-caps

post /v1/organizations/billing/spending-caps
Update spending cap configuration.

Configure spending limits and alert thresholds to control costs.

**Features:**
- **Soft Limit (Budget)**: Triggers alerts but doesn't block API access
- **Hard Cap**: Blocks API access when reached. Setting a positive
  `hard_spending_cap` arms enforcement automatically; pass
  `hard_cap_enabled: false` in the same request to set the cap without
  enforcing it. Clearing the cap (null/0) disarms enforcement.
- **Alert Thresholds**: Customize when to receive spending notifications

**Requirements:**
- Admin permission
- Only applies to organizations with auto-billing enabled

**Example:**
```python
# Set $100 budget with alerts at 75% and 100%
response = await client.post(
    "/v1/organizations/billing/spending-caps",
    json={
        "monthly_spending_budget": 10000,  # $100 in cents
        "spending_alert_thresholds": [75, 100],
        "spending_alerts_enabled": True,
    }
)

# Enable hard cap at $500
response = await client.post(
    "/v1/organizations/billing/spending-caps",
    json={
        "hard_spending_cap": 50000,  # $500 in cents
        "hard_cap_enabled": True,
    }
)

# Disable all spending limits
response = await client.post(
    "/v1/organizations/billing/spending-caps",
    json={
        "monthly_spending_budget": None,
        "hard_spending_cap": None,
        "hard_cap_enabled": False,
    }
)
```



# Create Storage Connection
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/create-storage-connection

post /v1/organizations/connections
Create a new storage provider connection.

Establishes a connection to an external storage provider (Google Drive, S3, etc.)
for use in sync operations. Credentials are validated before saving unless
test_before_save is False.

**Use Cases:**
- Connect to team Google Drive for automated file ingestion
- Link customer S3 buckets for batch processing
- Set up storage connections for sync operations

**Security:**
- Requires ADMIN permission
- Credentials never appear in read responses; only the credential type is returned
- Connection is tested before saving (unless test_before_save=False)
- Audit log entry created for compliance

**Example:**
```bash
curl -X POST "http://localhost:8000/v1/organizations/connections" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Marketing Drive",
    "provider_type": "google_drive",
    "provider_config": {
      "credentials": {...},
      "shared_drive_id": "0AH-Xabc123"
    }
  }'
```



# Delete Storage Connection
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/delete-storage-connection

delete /v1/organizations/connections/{connection_identifier}
Soft-delete a connection (mark archived).

Permanently retires a connection by marking it as ARCHIVED. The connection
cannot be reactivated after deletion. Credentials are preserved for audit
purposes but the connection is no longer usable.

**Example:**
```bash
curl -X DELETE "http://localhost:8000/v1/organizations/connections/conn_abc123" \
  -H "Authorization: Bearer YOUR_API_KEY"
```



# Get Storage Connection
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/get-storage-connection

get /v1/organizations/connections/{connection_identifier}
Retrieve a storage connection by ID or name.

Returns connection metadata including name, provider type, status, and
health information. Credential fields are not returned; provider_config.credentials
carries only the credential type.

**Identifier Resolution:**
- If identifier starts with 'conn_', treated as connection ID
- Otherwise, treated as connection name

**Example:**
```bash
# By ID
curl -X GET "http://localhost:8000/v1/organizations/connections/conn_abc123" \
  -H "Authorization: Bearer YOUR_API_KEY"

# By name
curl -X GET "http://localhost:8000/v1/organizations/connections/Marketing%20Drive" \
  -H "Authorization: Bearer YOUR_API_KEY"
```



# List Google Drive Files
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/list-google-drive-files

get /v1/organizations/connections/{connection_identifier}/files
List files in Google Drive folder for preview.

Shows a preview of files in the selected folder when configuring sync operations.
Only available for Google Drive connections.

**Use Cases:**
- Preview files in a folder before selecting it for sync
- Verify folder contains expected files
- Check file types and counts

**Example:**
```bash
curl -X GET "http://localhost:8000/v1/organizations/connections/conn_abc123/files?path=/Marketing&max_results=20" \
  -H "Authorization: Bearer YOUR_API_KEY"
```



# List Google Drive Folders
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/list-google-drive-folders

get /v1/organizations/connections/{connection_identifier}/folders
List folders in Google Drive for folder selection in sync configuration.

Enables users to browse and select folders when configuring sync operations.
Only available for Google Drive connections.

**Use Cases:**
- Browse available folders for sync configuration
- Select source folder for bucket sync
- Navigate nested folder structures

**Example:**
```bash
curl -X GET "http://localhost:8000/v1/organizations/connections/conn_abc123/folders?path=/Marketing" \
  -H "Authorization: Bearer YOUR_API_KEY"
```



# List Storage Connections
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/list-storage-connections

post /v1/organizations/connections/list
List storage connections for the authenticated organization.

Returns paginated results with optional filters for provider type, status,
and active flag. Results are sorted by creation date (newest first).

**Use Cases:**
- List all active Google Drive connections
- Find failed connections that need attention
- Filter by provider type for sync configuration

**Example:**
```bash
curl -X POST "http://localhost:8000/v1/organizations/connections/list" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider_type": "google_drive",
    "is_active": true
  }'
```



# Test Storage Connection
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/test-storage-connection

post /v1/organizations/connections/{connection_identifier}/test
Perform a credential test against the external provider.

Validates that connection credentials are still valid and the provider
is accessible. Result is logged in audit trail.

**Use Cases:**
- Validate credentials before using in sync operations
- Diagnose connection issues
- Refresh credentials after expiration

**Example:**
```bash
curl -X POST "http://localhost:8000/v1/organizations/connections/conn_abc123/test" \
  -H "Authorization: Bearer YOUR_API_KEY"
```



# Update Storage Connection
Source: https://docs.mixpeek.com/docs/api-reference/organization-connections/update-storage-connection

patch /v1/organizations/connections/{connection_identifier}
Update connection metadata or credentials.

Allows partial updates to connection metadata without changing credentials.
Credentials can be updated via provider_config. All changes are logged
in audit trail.

**What You Can Update:**
- Connection name and description
- Metadata tags
- Status (active/suspended)
- Provider credentials (via provider_config)

**Example:**
```bash
curl -X PATCH "http://localhost:8000/v1/organizations/connections/conn_abc123" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Updated Drive Name",
    "status": "suspended"
  }'
```



# Get Infrastructure
Source: https://docs.mixpeek.com/docs/api-reference/organization-infrastructure/get-infrastructure

get /v1/organizations/infrastructure/summary
Get infrastructure summary for an enterprise single-tenant deployment.

Returns compute profiles for each extractor, cloud cost breakdown
with transparent margin, and the billing period summary for the
requested (default: current) month. Enterprise-only endpoint.



# Create Secret
Source: https://docs.mixpeek.com/docs/api-reference/organization-secrets/create-secret

post /v1/organizations/secrets
Create a new secret in organization vault.

**Security**:
- Secret value is encrypted at rest using Fernet encryption
- Encrypted using ENCRYPTION_KEY from environment
- Decrypted value is NEVER returned in API responses
- Only secret names are exposed in list operations

**Use Cases**:
- Store API keys for external services (Stripe, GitHub, etc.)
- Store authentication tokens for api_call retriever stage
- Store credentials for third-party integrations

**Important**:
- Secret names must be unique within organization
- Use update endpoint to modify existing secrets
- Delete and recreate if you forget the value



# Delete Secret
Source: https://docs.mixpeek.com/docs/api-reference/organization-secrets/delete-secret

delete /v1/organizations/secrets/{secret_name}
Delete a secret from organization vault.

**Warning**:
- Deletion is permanent and immediate
- Any api_call stages using this secret will fail
- No confirmation prompt - use with caution

**Use Cases**:
- Remove unused credentials
- Clean up after service decommissioning
- Security incident response



# List Secrets
Source: https://docs.mixpeek.com/docs/api-reference/organization-secrets/list-secrets

get /v1/organizations/secrets
List all secret names in organization vault.

**Security**:
- Returns ONLY secret names, never values
- Use for discovering which secrets are configured
- Secret values can only be retrieved by internal services

**Response**:
- List of secret names (e.g., ['stripe_api_key', 'github_token'])
- Total count of secrets



# Update Secret
Source: https://docs.mixpeek.com/docs/api-reference/organization-secrets/update-secret

put /v1/organizations/secrets/{secret_name}
Update an existing secret in organization vault.

**Security**:
- Replaces existing encrypted value with new encrypted value
- Old value is permanently overwritten
- No history or audit trail of previous values

**Use Cases**:
- Rotate API keys periodically
- Update expired tokens
- Change credentials after security incident



# Get Api Key Endpoint Breakdown
Source: https://docs.mixpeek.com/docs/api-reference/organization-usage/get-api-key-endpoint-breakdown

get /v1/organizations/api-keys/{key_id}/usage/endpoints
Return endpoint-level usage metrics for a specific API key.



# Get Api Key Usage
Source: https://docs.mixpeek.com/docs/api-reference/organization-usage/get-api-key-usage

get /v1/organizations/api-keys/{key_id}/usage
Return usage metrics for a specific API key.



# Get Org Usage
Source: https://docs.mixpeek.com/docs/api-reference/organization-usage/get-org-usage

get /v1/organizations/usage
Return aggregated usage for the organization.



# Create User
Source: https://docs.mixpeek.com/docs/api-reference/organization-users/create-user

post /v1/organizations/users
Create a new organization user.



# Delete User
Source: https://docs.mixpeek.com/docs/api-reference/organization-users/delete-user

delete /v1/organizations/users/{user_email}
Delete a user and revoke their API keys.



# Get User
Source: https://docs.mixpeek.com/docs/api-reference/organization-users/get-user

get /v1/organizations/users/{user_email}
Return a user by email address.



# List Users
Source: https://docs.mixpeek.com/docs/api-reference/organization-users/list-users

get /v1/organizations/users
List organization users with pagination and optional filters.



# Update User
Source: https://docs.mixpeek.com/docs/api-reference/organization-users/update-user

patch /v1/organizations/users/{user_email}
Apply partial updates to an existing user.



# Add credits
Source: https://docs.mixpeek.com/docs/api-reference/organizations/add-credits

post /v1/organizations/credits



# Get Credits
Source: https://docs.mixpeek.com/docs/api-reference/organizations/get-credits

get /v1/organizations/credits
Read the org's credit/usage state.

GET on this path returned 405 (only POST existed) — a
natural read that every evaluator tries. Returns the ENFORCED ceiling
(`effective_monthly_credit_cap`, the number the block point uses), the
current month usage, and the decorative legacy `credit_count` clearly
labeled as non-gating.



# Get Organization
Source: https://docs.mixpeek.com/docs/api-reference/organizations/get-organization

get /v1/organizations
Get current organization details.

Security: Infrastructure configuration is NOT exposed via this endpoint.
Infrastructure (vector store URLs, Ray clusters) is only accessible via private admin endpoints.



# Get Organization Targets
Source: https://docs.mixpeek.com/docs/api-reference/organizations/get-organization-targets

get /v1/organizations/targets
Get this organization's resource + SLO targets.

Tenancy (dedicated vs shared) is resolved SERVER-SIDE from the org —
never from a client-supplied header. ``targets_writable``
is the capability flag a client should read rather than infer from
``plane``: shared-plane orgs are read-only by platform policy, and
dedicated orgs hold write access to their own spec.



# Update Organization
Source: https://docs.mixpeek.com/docs/api-reference/organizations/update-organization

patch /v1/organizations
Update organization settings (requires ADMIN permission).

Security: Infrastructure configuration cannot be modified via this endpoint.
Infrastructure updates require Mixpeek admin access via private endpoints.



# Get Pricing Configuration
Source: https://docs.mixpeek.com/docs/api-reference/pricing/get-pricing-configuration

get /v1/billing/pricing
Public pricing rate card. Price = how much content of each file type × (base rate + the search-by features enabled). Returns per-modality rates, tier usage pools, and plan definitions. No authentication required.



# Get Task Information
Source: https://docs.mixpeek.com/docs/api-reference/tasks/get-task-information

get /v1/tasks/{task_id}
Retrieve a task by its ID.

    A task may have an expiration time, after which it will still be returned but marked as expired.
    This allows tracking of historical tasks while indicating their current validity state.



# Kill Task
Source: https://docs.mixpeek.com/docs/api-reference/tasks/kill-task

delete /v1/tasks/{task_id}
Kill a task (idempotent - succeeds even if task doesn't exist or is already killed).



# List Tasks
Source: https://docs.mixpeek.com/docs/api-reference/tasks/list-tasks

post /v1/tasks/list
List tasks with optional filtering, sorting, and pagination.

    **Filter Options**:
    - `status`: Filter by specific status (PENDING, IN_PROGRESS, COMPLETED, FAILED, etc.)
    - `task_type`: Filter by task type
    - `filters`: AND/OR/NOT conditions on any task field, for example
      `{"AND": [{"field": "task_id", "operator": "eq", "value": "<task_id>"}]}`
    - `search`: case-insensitive text matched against task_id, task_type and status
    - `sort`: `{"field": "created_at", "direction": "desc"}`

    **Examples**:
    - All tasks: `{}`
    - Failed tasks only: `{"status": "FAILED"}`
    - Pending batches: `{"status": "PENDING", "task_type": "API_BUCKETS_UPLOADS_BATCH_CONFIRM"}`
    - In-progress tasks: `{"status": "IN_PROGRESS"}`



# Delete Webhook
Source: https://docs.mixpeek.com/docs/api-reference/webhooks/delete-webhook

delete /v1/organizations/webhooks/{identifier}
Delete a webhook (idempotent - succeeds even if already deleted).



# Create Session
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/create-session

post /v1/agents/sessions
Create a new agent session.

A session represents a stateful conversation with an AI agent that can
call tools to search data, filter results, and perform multi-step reasoning.

Args:
    request: FastAPI request with tenant context
    payload: Session creation request

Returns:
    CreateSessionResponse with session metadata

Example:
    ```bash
    curl -X POST http://localhost:8000/v1/agents/sessions \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{
        "agent_config": {
          "model": "claude-3-5-sonnet-20241022",
          "temperature": 0.7,
          "available_tools": ["search_retrievers", "execute_retriever"]
        },
        "quotas": {
          "max_messages": 100,
          "max_tokens_total": 100000
        }
      }'
    ```



# Detect Intent
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/detect-intent

post /v1/agents/sessions/intent/detect
Detect user intent from natural language request.

This endpoint analyzes a user's request to determine whether they want to:
- Execute queries on existing data (execution mode)
- Create new resources/infrastructure (setup mode)
- Or if the request is ambiguous and needs clarification

It performs keyword analysis and checks existing collections to provide
intelligent classification and recommendations.

Args:
    request: FastAPI request with tenant context
    payload: Intent detection request with user's input

Returns:
    IntentClassification with detected intent and recommendations

Example:
    ```bash
    curl -X POST http://localhost:8000/v1/agents/intent/detect \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{
        "user_request": "I want to search videos by faces",
        "include_collection_analysis": true
      }'
    ```



# Get History
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/get-history

get /v1/agents/sessions/{session_id}/history
Get conversation history for a session.

Returns messages in chronological order (oldest first).

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier
    limit: Maximum messages to return (default: 50, max: 200)
    offset: Pagination offset (default: 0)

Returns:
    GetHistoryResponse with message history

Raises:
    NotFoundError: If session not found

Example:
    ```bash
    curl -X GET "http://localhost:8000/v1/agents/sessions/ses_abc123/history?limit=20&offset=0" \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}"
    ```



# Get Session
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/get-session

get /v1/agents/sessions/{session_id}
Get session metadata by ID.

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier

Returns:
    GetSessionResponse with session metadata

Raises:
    NotFoundError: If session not found

Example:
    ```bash
    curl -X GET http://localhost:8000/v1/agents/sessions/ses_abc123 \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}"
    ```



# List Sessions
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/list-sessions

post /v1/agents/sessions/list
List agent sessions in the namespace.

Args:
    request: FastAPI request with tenant context
    list_request: Optional filters and sorting
    pagination: Pagination parameters

Returns:
    ListSessionsResponse with session list

Example:
    ```bash
    curl -X POST http://localhost:8000/v1/agents/sessions/list \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{"status": "active"}'
    ```



# List Tools
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/list-tools

get /v1/agents/sessions/tools
List all available agent tools.

Use this endpoint to discover available tools before creating a session.

Tool Categories:
- search: Tools for searching data (execute_retriever)
- read: Tools for reading resources (list_*, get_*)
- create: Tools for creating resources (create_*) - requires confirmation
- update: Tools for updating resources (update_*) - requires confirmation
- delete: Tools for deleting resources (delete_*) - requires confirmation
- upload: Tools for file uploads (upload_object)

Note: Write operations (create, update, delete) require user confirmation
via the /confirmations endpoint before execution.

Args:
    request: FastAPI request with tenant context
    category: Optional filter by tool category

Returns:
    ListToolsResponse with available tools

Example:
    ```bash
    # List all tools
    curl -X GET http://localhost:8000/v1/agents/tools \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}"

    # List only search tools
    curl -X GET "http://localhost:8000/v1/agents/tools?category=search" \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}"
    ```



# Patch Session
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/patch-session

patch /v1/agents/sessions/{session_id}
Update session metadata.

Only user_memory can be updated. To change agent configuration,
create a new session.

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier
    payload: Update request

Returns:
    PatchSessionResponse with update timestamp

Raises:
    NotFoundError: If session not found

Example:
    ```bash
    curl -X PATCH http://localhost:8000/v1/agents/sessions/ses_abc123 \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{
        "user_memory": {
          "preferences": {"language": "en", "domain": "tech"}
        }
      }'
    ```



# Respond To Confirmation
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/respond-to-confirmation

post /v1/agents/sessions/{session_id}/confirmations/{confirmation_id}
Respond to a pending confirmation for a write operation.

When the agent requests a write operation (create, update, delete),
the stream pauses and emits a `confirmation_required` event. The user
must call this endpoint to approve or deny the action.

After responding, this endpoint returns a new SSE stream that continues
the agent's execution from where it paused.

## Confirmation Workflow

1. User sends message via POST /sessions/{id}/messages
2. Agent proposes a write operation
3. Stream emits `confirmation_required` event with `confirmation_id`
4. User calls this endpoint with `approved: true/false`
5. This endpoint returns SSE stream continuing the agent's work
6. If approved, agent executes the tool and continues
7. If denied, agent acknowledges and continues without executing

## Confirmation Expiration

Confirmations expire after 5 minutes. Attempting to respond to an
expired confirmation returns a 400 error.

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier
    confirmation_id: Confirmation identifier from `confirmation_required` event
    payload: Approval/denial decision

Returns:
    StreamingResponse with SSE events (continuation of agent execution)

Raises:
    NotFoundError: If session or confirmation not found
    ValidationError: If confirmation already processed or expired

Example:
    ```bash
    # Approve a pending action
    curl -N -X POST http://localhost:8000/v1/agents/sessions/ses_abc/confirmations/conf_xyz \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{"approved": true}'

    # SSE Output (continuation):
    event: tool_result
    data: {"tool_name": "delete_collection", "success": true, "result": {...}}

    event: token
    data: {"content": "I've deleted the collection as requested."}

    event: done
    data: {}
    ```



# Send Message
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/send-message

post /v1/agents/sessions/{session_id}/messages
Send a message to the agent and stream the response.

This endpoint streams Server-Sent Events (SSE) back to the client as
the agent processes the message through its workflow.

## SSE Event Types

**Core Events:**
- `intent`: Intent classification result (emitted first)
    - `{intent, confidence, category, reasoning, context_scope}`
- `thinking`: Agent is analyzing/planning
    - `{step, message}`
- `tool_call`: Agent is calling a tool
    - `{tool_name, tool_call_id, inputs}`
- `tool_result`: Tool execution completed
    - `{tool_name, tool_call_id, success, output, latency_ms}`
- `token`: Response token (streaming)
    - `{content}`
- `message`: Final response content
    - `{content, message_id, is_final}`
- `session_name`: Auto-generated session name (first message only)
    - `{session_name}`
- `done`: Processing complete
    - `{latency_ms, tool_calls_made, message_id, retriever_summary, data_accessed_via_retriever}`
- `error`: Error occurred
    - `{message, recoverable}`

**Retriever Events (IMPORTANT - Primary Data Pathway):**
- `retriever_execution`: Retriever was used for data access
    - `{tool_name, execution_id, retriever_id, is_adhoc, documents_returned, latency_ms, message}`
    - Emitted whenever data is accessed via retriever (saved or ad-hoc)
- `pipeline_config`: Ad-hoc retriever configuration
    - `{tool_name, config, message}`
    - Contains the exact pipeline config users can save as a named retriever

**Retriever Summary in `done` Event:**
```json
{
  "retriever_summary": {
    "used_retrievers": true,
    "retriever_count": 2,
    "saved_retrievers": 1,
    "adhoc_retrievers": 1,
    "total_documents": 25,
    "executions": [...]
  },
  "data_accessed_via_retriever": true
}
```

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier
    payload: Message request

Returns:
    StreamingResponse with SSE events

Raises:
    NotFoundError: If session not found

Example:
    ```bash
    curl -N -X POST http://localhost:8000/v1/agents/sessions/ses_abc123/messages \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{
        "content": "Find videos about machine learning",
        "stream": true
      }'

    # SSE Output:
    event: intent
    data: {"intent": "retriever_search", "confidence": 0.92, "category": "retriever"}

    event: thinking
    data: {"step": "processing", "message": "Analyzing your request..."}

    event: tool_call
    data: {"tool_name": "execute_retriever", "tool_call_id": "run_abc", "inputs": {...}}

    event: tool_result
    data: {"tool_name": "execute_retriever", "success": true, "output": {...}}

    event: retriever_execution
    data: {"tool_name": "execute_retriever", "is_adhoc": false, "documents_returned": 5}

    event: message
    data: {"content": "I found 5 videos about machine learning...", "is_final": true}

    event: done
    data: {"latency_ms": 1250.5, "data_accessed_via_retriever": true, "retriever_summary": {...}}
    ```



# Submit Feedback
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/submit-feedback

post /v1/agents/sessions/{session_id}/feedback
Submit feedback on an assistant message.

When positive feedback is received, the conversation exchange is stored
to memory for future context. When negative feedback is received, the
exchange is NOT stored. This enables learning from quality interactions.

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier
    payload: Feedback request

Returns:
    SubmitFeedbackResponse with feedback status

Raises:
    NotFoundError: If session or message not found

Example:
    ```bash
    curl -X POST http://localhost:8000/v1/agents/sessions/ses_abc123/feedback \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}" \
      -H "Content-Type: application/json" \
      -d '{
        "message_id": "msg_xyz789",
        "rating": "positive",
        "feedback_text": "Very helpful response!"
      }'
    ```



# Terminate Session
Source: https://docs.mixpeek.com/docs/api-reference/agent-sessions/terminate-session

delete /v1/agents/sessions/{session_id}
Terminate a session and kill its actor.

This permanently ends the session and releases all associated resources.

Args:
    request: FastAPI request with tenant context
    session_id: Session identifier

Returns:
    TerminateSessionResponse with termination timestamp

Raises:
    NotFoundError: If session not found

Example:
    ```bash
    curl -X DELETE http://localhost:8000/v1/agents/sessions/ses_abc123 \
      -H "Authorization: Bearer {api_key}" \
      -H "X-Namespace: {namespace_id}"
    ```



# Add a custom domain
Source: https://docs.mixpeek.com/docs/api-reference/apps/add-domain

post /v1/apps/{app_id}/domains
Attach a custom domain to an App.

Returns a `verification_token` and `cname_target`. To prove ownership,
add a DNS TXT record:

    _mixpeek-verify.{domain}  TXT  "mixpeek-site-verification={verification_token}"

Then call `POST /{app_id}/domains/{domain}/verify` to start polling.



# Connect a GitHub repository
Source: https://docs.mixpeek.com/docs/api-reference/apps/connect-a-github-repository

post /v1/apps/{app_id}/connect-repo
Connect a GitHub repository for automatic deploys on push.

After connecting, pushes to the configured branch will trigger
a build + deploy via the GitHub App webhook.



# Create an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/create-app

post /v1/apps
Create a new App.

Apps are full-stack web applications deployed as zip bundles to
``{slug}.mxp.co``.  After creation, deploy your built frontend via the
deploy pipeline:

1. ``POST /v1/apps/{app_id}/deploy/upload-url`` — get a presigned S3 PUT URL
2. Upload your built ``dist/`` zip to that URL
3. ``POST /v1/apps/{app_id}/deploy`` — trigger the build

Only ``slug`` and ``meta`` are required.  Set ``auth_config.mode`` to
control end-user authentication (``public``, ``clerk``, ``password``,
``api_key``, ``jwt``, ``sso_oidc``, ``sso_saml``).

Slugs are globally unique across all organizations.

**Deprecated fields:** ``template``, ``sections``, ``custom_html``,
``hero``, ``theme``, ``seo``, ``stats``, ``featured_gallery``, ``tabs``,
``password_secret_name`` are accepted for backward compatibility but
should not be used for new apps.

**Tier limits:** Free = 1 app, Pro = 10, Enterprise = unlimited
(``TierLimits._max_apps``). Exceeding your tier's limit returns a 400 with
the limit + current count in ``error.details``.



# Delete an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/delete-app

delete /v1/apps/{app_id}
Delete an App permanently.

The slug is freed immediately. Any active custom domains will stop
resolving within seconds.



# Remove a domain
Source: https://docs.mixpeek.com/docs/api-reference/apps/delete-domain

delete /v1/apps/{app_id}/domains/{domain}
Remove a custom domain from an App.



# Deploy an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/deploy-app

post /v1/apps/{app_id}/deploy
Queue a build+deploy for an App, creating a new version.

**Full deploy workflow:**
1. `POST /{app_id}/deploy/upload-url` → get a presigned S3 PUT URL
2. Upload your built dist/ zip to that URL
3. Call this endpoint with the returned `bundle_s3_key`
4. Optionally include `source_files` (path→content map) for version diffing
5. Poll `GET /{app_id}/deploys/{deploy_id}` until status is `complete`

**Edit-and-redeploy workflow:**
1. `GET /{app_id}/versions/{version}/download` → download the bundle zip
2. Edit files locally
3. Re-zip and upload via upload-url → deploy (creates a new version)
4. `GET /{app_id}/versions/{old}/diff/{new}` to review changes

Each deploy creates an immutable version record. Use `POST /{app_id}/versions/{n}/restore`
for instant rollback to any previous version.



# Diff Two Versions
Source: https://docs.mixpeek.com/docs/api-reference/apps/diff-versions

GET /v1/apps/{app_id}/versions/{from_version}/diff/{to_version}
Compare two versions — returns file-level changes and optional source diffs.

Like `git diff v1..v2`: shows added, removed, modified, and unchanged files
based on content hashes. If both versions have source_files, also returns
unified diffs of the source code.



# Disconnect GitHub repository
Source: https://docs.mixpeek.com/docs/api-reference/apps/disconnect-github-repository

delete /v1/apps/{app_id}/connect-repo
Disconnect the GitHub repository from this app.



# Download Version Bundle
Source: https://docs.mixpeek.com/docs/api-reference/apps/download-version

GET /v1/apps/{app_id}/versions/{version}/download
Get a presigned download URL for a version's original bundle zip.

Use this to download → edit → re-upload → deploy as a new version.
The returned URL expires in 1 hour.



# Get an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-app

get /v1/apps/{app_id}
Get an App by its `app_id`.



# Get app analytics overview
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-app-analytics-overview

get /v1/apps/{app_id}/analytics
Per-app analytics dashboard data: error trend, Web Vitals, recent errors, event counts.

Reads the canvas_errors, canvas_vitals and canvas_app_events collections.



# Get Deploy Status
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-deploy-status

GET /v1/apps/{app_id}/deploys/{deploy_id}
Check the status of a specific deployment (queued, building, complete, failed).



# Get presigned upload URL
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-deploy-upload-url

post /v1/apps/{app_id}/deploy/upload-url
Generate a presigned S3 PUT URL so the CLI can upload a bundle zip directly.
Pass the returned `bundle_s3_key` to the deploy endpoint.



# Get the GitHub App install URL for this app
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-github-app-install-url-app

get /v1/apps/{app_id}/github/install-url
Return the GitHub App installation URL carrying a signed ``state``.

The Studio "Install GitHub App" button opens this URL; after install,
GitHub redirects to ``/webhooks/apps/github/setup`` which verifies the
signature and links the installation to this app.



# Get recent error rate
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-recent-error-rate

get /v1/apps/{app_id}/error-rate
Count of canvas_errors for this app in the last N minutes.



# Get runtime or request logs
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-runtime-or-request-logs

get /v1/apps/{app_id}/logs
Read the canvas_app_logs or canvas_request_logs collection.



# Get Version Details
Source: https://docs.mixpeek.com/docs/api-reference/apps/get-version

GET /v1/apps/{app_id}/versions/{version}
Return full details for a specific version including asset manifest and source files.



# Invite a user
Source: https://docs.mixpeek.com/docs/api-reference/apps/invite-a-user

post /v1/apps/{app_id}/users/invite
Invite a user to this app by email via Clerk.



# List app users
Source: https://docs.mixpeek.com/docs/api-reference/apps/list-app-users

get /v1/apps/{app_id}/users
List all users in this app's Clerk organization.



# List Apps
Source: https://docs.mixpeek.com/docs/api-reference/apps/list-apps

get /v1/apps
List all Apps for this organization.

The response includes ``max_apps`` (the tier's Canvas App limit, ``null`` =
unlimited) so the UI can render usage ("X of N apps used") and gate the
create action without a second round-trip.



# List custom domains
Source: https://docs.mixpeek.com/docs/api-reference/apps/list-domains

get /v1/apps/{app_id}/domains
List all custom domains attached to an App.



# List pending invitations
Source: https://docs.mixpeek.com/docs/api-reference/apps/list-pending-invitations

get /v1/apps/{app_id}/users/invitations
List pending invitations for this app's Clerk organization.



# List version history
Source: https://docs.mixpeek.com/docs/api-reference/apps/list-versions

get /v1/apps/{app_id}/versions
Return version history for an App, most recent first.



# Promote staging to production
Source: https://docs.mixpeek.com/docs/api-reference/apps/promote-staging-to-production

post /v1/apps/{app_id}/promote
Promote the current staging deployment to production.

Copies `environments.staging` (deploy_id, asset_prefix) to
`environments.production` and updates `build_config.asset_prefix`
for backward compatibility. Also invalidates the canvas asset cache
so the new production version is served immediately.



# Publish an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/publish-app

post /v1/apps/{app_id}/publish
Publish the current draft config of an App.

For deploy-based apps, code deploys go live automatically — you only
need this endpoint to publish **config changes** (auth, meta, etc.).

Creates an immutable version snapshot with a content hash.



# Remove a user
Source: https://docs.mixpeek.com/docs/api-reference/apps/remove-a-user

delete /v1/apps/{app_id}/users/{user_id}
Remove a user from this app's Clerk organization.



# Restore a Version
Source: https://docs.mixpeek.com/docs/api-reference/apps/restore-version

POST /v1/apps/{app_id}/versions/{version}/restore
Restore a previous version by pointing the environment to its assets.

This is instant — no rebuild required. Old assets are always retained in S3.



# Revoke an invitation
Source: https://docs.mixpeek.com/docs/api-reference/apps/revoke-an-invitation

delete /v1/apps/{app_id}/users/invitations/{invitation_id}
Revoke a pending invitation.



# Rollback an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/rollback-app

post /v1/apps/{app_id}/rollback
Rollback to the previous published config version.

For deploy-based apps, use ``POST /v1/apps/{app_id}/versions/{version}/restore``
to roll back to any previous code deploy instead.



# Stream deploy logs (SSE)
Source: https://docs.mixpeek.com/docs/api-reference/apps/stream-deploy-logs-sse

get /v1/apps/{app_id}/deploys/{deploy_id}/logs/stream
Server-Sent Events stream of real-time deploy logs.

Sends cached logs first (catch-up), then subscribes to live log
events via Redis pub/sub until the deploy reaches a terminal state.



# Update an App
Source: https://docs.mixpeek.com/docs/api-reference/apps/update-app

patch /v1/apps/{app_id}
Partially update an App configuration (saved as draft).

All fields are optional — only the fields you provide are updated.
Updates are draft by default. Set `publish: true` to apply and publish
in a single call.

Set `dry_run=true` to check slug availability only, with no write.



# Update user role
Source: https://docs.mixpeek.com/docs/api-reference/apps/update-user-role

patch /v1/apps/{app_id}/users/{user_id}/role
Update a user's role in this app's Clerk organization.



# Verify a presigned-URL upload landed in S3
Source: https://docs.mixpeek.com/docs/api-reference/apps/verify-a-presigned-url-upload-landed-in-s3

post /v1/apps/{app_id}/deploy/verify-upload
Confirm a bundle uploaded via a presigned PUT URL actually landed in S3.

S3 returns an empty body on a successful PUT — the ETag is only in
response headers, which browser clients often can't read due to CORS.
This endpoint does a server-side HEAD and returns {etag, size_bytes,
last_modified, exists} so clients can verify the upload before calling
`/deploy`.



# Trigger DNS verification
Source: https://docs.mixpeek.com/docs/api-reference/apps/verify-domain

post /v1/apps/{app_id}/domains/{domain}/verify
Trigger asynchronous DNS TXT record verification for a domain.

The domain status is set to `verifying` immediately. A background worker
polls DNS every 30 minutes for up to 72 hours. On success the status
advances to `provisioning_tls`.



# Apply Manifest
Source: https://docs.mixpeek.com/docs/api-reference/manifest/apply-manifest

post /v1/manifest/apply
Apply a YAML manifest to create resources.

Creates all resources defined in the manifest file in dependency order.

**Modes:**
- `create_only` (default) — fails if any resource already exists.
- `create_missing` — creates what is missing, leaves what exists alone, and resolves
  manifest references against the resources ALREADY in the namespace. This is what makes
  "here is a namespace with buckets, add these collections over them" work. Under
  `create_only` a reference to an undeclared-but-existing bucket is a validation error,
  because the manifest is treated as a closed world.

**Plan first:** before creating anything, apply runs the same checks as
`POST /v1/manifest/validate`. Any error refuses the whole manifest with
`success: false`, every resource `skipped`, and each error naming the
resource, the field and the fix. Nothing is created. A namespace budget above
the organization's bound is set to the bound and reported as a warning.

**Features:**
- Topological sorting ensures resources are created in correct dependency order
- Secret references (`${{ secrets.NAME }}`) are resolved from organization secrets
- Atomic operation: rolls back all created resources if any creation fails, and
  an apply whose process is stopped part way is rolled back when the API starts again
- Dry run returns the plan without making changes

**Example:**
```bash
curl -X POST /v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -H "X-Namespace: ns_xxx" \
  -F "manifest_file=@mixpeek.yaml"
```

**Example manifest:**
```yaml
version: "1.0"
metadata:
  name: "my-environment"

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1

buckets:
  - name: raw_videos
    namespace: video_search
    schema:
      properties:
        video: { type: video }
```



# Export Manifest Get
Source: https://docs.mixpeek.com/docs/api-reference/manifest/export-manifest

get /v1/manifest/export
Export current resources to a YAML manifest.

Exports all resources (or a specific namespace) to a YAML file
that can be version-controlled and re-applied to another environment.

**Features:**
- Resource IDs are converted to human-readable names
- Secret values are replaced with placeholder references (`${{ secrets.NAME }}`)
- Output is formatted for readability and git-friendliness

**Note:** You must configure actual secrets before applying the exported
manifest to a new environment.

**Example:**
```bash
# Export all resources
curl /v1/manifest/export \
  -H "Authorization: Bearer $API_KEY" \
  -o mixpeek.yaml

# Export specific namespace
curl "/v1/manifest/export?namespace_id=ns_abc123" \
  -H "Authorization: Bearer $API_KEY" \
  -o namespace.yaml
```



# Validate Manifest
Source: https://docs.mixpeek.com/docs/api-reference/manifest/validate-manifest

post /v1/manifest/validate
Validate a YAML manifest without applying.

Checks:
- YAML syntax validity
- Schema validation against manifest models
- Cross-resource reference validation
- Dependency resolution (no circular dependencies)
- Secret reference existence
- Every extractor a namespace or collection names exists and is turned on
- Retriever stages are registered and their parameters valid
- Every feature URI a retriever or cluster names is written by a collection
- Alert applications map every input their alert's retriever requires
- Namespace budgets above the organization's bound (a warning: apply sets
  the bound)

These are the checks apply runs before creating anything, so a manifest
that validates with no errors is one apply will not refuse on references.

Returns detailed validation results including:
- Resource counts by type
- Missing secrets that need to be configured
- Validation errors and warnings

**Example:**
```bash
curl -X POST /v1/manifest/validate \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```



# Get public App config
Source: https://docs.mixpeek.com/docs/api-reference/public-apps-api/get-app-config

get /v1/public/apps/{slug}/config
Fetch the full configuration for a public App — no authentication required.

Used by the `canvas/` host runtime and the `@mixpeek/canvas-sdk`
`useApp()` hook to render the app. Returns tabs, hero, stats, sections,
theme, and SEO settings.

Serves the `published_config` snapshot when available, so draft edits
made via `PATCH /v1/apps/{app_id}` are not visible until the app is
published.



# Get App featured gallery
Source: https://docs.mixpeek.com/docs/api-reference/public-apps-api/get-app-gallery

get /v1/public/apps/{slug}/gallery
Fetch featured gallery results for a public App — no authentication required.

Executes the app's `featured_gallery` retriever using `default_inputs`
defined in the gallery configuration. Returns an empty result set if the
gallery is disabled or not configured.



# Get taxonomy node tree for an App
Source: https://docs.mixpeek.com/docs/api-reference/public-apps-api/get-app-taxonomies

get /v1/public/apps/{slug}/taxonomies/{taxonomy_id}/tree
Fetch the node tree for a taxonomy associated with a public App — no authentication required.

Looks up the taxonomy by ``taxonomy_id`` within the namespace that owns
the app identified by ``slug``. Returns the full ``nodes`` array as stored,
which can be used to render faceted navigation or classification trees in
the app UI.



# Execute an App tab search
Source: https://docs.mixpeek.com/docs/api-reference/public-apps-api/search-app

post /v1/public/apps/{slug}/search
Execute a search on a specific tab of a public App — no authentication required.

Routes `tab_id` → retriever → results. Each tab can be backed by either:
- An **internal retriever** (`retriever_id`) — org-scoped
- A **marketplace catalog entry** (`public_name`) — proxied via public API

The `inputs` dict is passed directly to the retriever's `input_schema`.
Extra top-level fields are automatically merged into `inputs`.

**Example:**
```json
{
  "tab_id": "search",
  "inputs": { "query": "red sneakers" },
  "settings": { "limit": 20 }
}
```



# Track an App interaction event
Source: https://docs.mixpeek.com/docs/api-reference/public-apps-api/track-app-interaction

post /v1/public/apps/{slug}/interactions
Record a user interaction event for a public App — no authentication required.

Interaction events are written to the ``app_interactions`` collection for
analytics and relevance tuning. Write errors are swallowed server-side so
that a transient storage failure never surfaces as an error to the caller.

**Example:**
```json
{
  "event_type": "click",
  "document_id": "doc_abc123",
  "position": 2,
  "query": "red sneakers",
  "session_id": "sess_xyz"
}
```



# Get Public Namespace Template
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/get-public-namespace-template

get /v1/public/templates/namespaces/{template_id}
Get public namespace template details (no authentication required).

Returns template only if marked as public.



# Get Public Retriever Template
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/get-public-retriever-template

get /v1/public/templates/retrievers/{template_id}
Get public retriever template details (no authentication required).

Returns template only if marked as public.



# Get Scaffold
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/get-scaffold

get /v1/public/templates/scaffolds/{template_id}
Get scaffold template details (public, no auth required).

Returns the complete configuration including:
- Namespace: feature extractors, payload indexes
- Bucket: name, description, schema
- Collection: feature extractor config
- Retriever: stages, input schema

**To instantiate (requires auth):**
```
POST /v1/templates/scaffolds/{template_id}/instantiate
{"namespace_name": "my_app"}
```



# Get Scaffold Manifest
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/get-scaffold-manifest

get /v1/public/templates/scaffolds/{template_id}/manifest
Render a scaffold template as a Mixpeek manifest (public, no auth).

Powers the Studio namespace-templates UI, which shows each template as a
declarative manifest tree + copyable YAML (the same IaC format used by
`POST /v1/manifest/apply`). Returns both the YAML string (`manifest_yaml`)
and the parsed manifest dict (`manifest`).



# Get Scaffold Sample Data
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/get-scaffold-sample-data

get /v1/public/templates/scaffolds/{template_id}/sample-data
Live per-collection document counts of the scaffold's golden sample
namespace (public, no auth required).

Powers the Studio pre-clone preview: what a user actually gets when they
clone this scaffold with sample data. Counts are live-derived server-side
(cached ~10 min) because golden namespaces are not readable with normal
org keys.



# List Public Namespace Templates
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/list-public-namespace-templates

get /v1/public/templates/namespaces
List public namespace templates (no authentication required).

Returns only templates marked as public (is_public=True).



# List Public Retriever Templates
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/list-public-retriever-templates

get /v1/public/templates/retrievers
List public retriever templates (no authentication required).

Returns only templates marked as public (is_public=True).



# List Scaffolds
Source: https://docs.mixpeek.com/docs/api-reference/public-templates-api/list-scaffolds

get /v1/public/templates/scaffolds
List available scaffold templates (public, no auth required).

Scaffolds are pre-configured templates that create complete infrastructure:
- Namespace with feature extractors
- Bucket with data schema
- Collection with processing config
- Retriever with search pipeline

**Categories:**
- `media` - Video, image, podcast search
- `documents` - Document Q&A, RAG
- `ecommerce` - Product catalog search

**To instantiate (requires auth):**
```
POST /v1/templates/scaffolds/{template_id}/instantiate
{"namespace_name": "my_app"}
```



# Get Or Create Default Namespace
Source: https://docs.mixpeek.com/docs/api-reference/scaffold-templates/get-or-create-default-namespace

post /v1/templates/scaffolds/default
Get or create the default universal namespace.

Returns the existing default namespace if one exists, or creates a new one
using the universal template. This is the primary onboarding entry point —
Studio calls this on first upload to ensure a namespace exists without
requiring the user to choose a template.

The default namespace supports all file types (video, image, audio, PDF, text)
with pre-configured extractors and a universal retriever.

If the request carries an `X-Namespace` header that resolves to an existing
writable org namespace (e.g. one just created via the wizard), the scaffold
is provisioned into THAT namespace instead of creating a "default" one.



# Get Scaffold Template Manifest
Source: https://docs.mixpeek.com/docs/api-reference/scaffold-templates/get-scaffold-template-manifest

get /v1/templates/scaffolds/{template_id}/manifest
Render a scaffold template as a Mixpeek manifest.

Converts the scaffold's configuration (namespace, bucket, collections,
retriever, taxonomies, clusters) into declarative Infrastructure-as-Code
YAML — the same format used by `POST /v1/manifest/apply` and
`GET /v1/manifest/export`.

Returns both the YAML string (`manifest_yaml`) and the parsed manifest
dict (`manifest`).



# Create a broadcast communication (Mixpeek admin only).
Source: https://docs.mixpeek.com/docs/api-reference/support/create-broadcast-communication

post /v1/support/comms/broadcast



# Create Support Ticket
Source: https://docs.mixpeek.com/docs/api-reference/support/create-support-ticket

post /v1/support/tickets



# Get Support Ticket (with message thread).
Source: https://docs.mixpeek.com/docs/api-reference/support/get-support-ticket

get /v1/support/tickets/{ticket_identifier}



# Get a support video (with presigned playback URL + chapters).
Source: https://docs.mixpeek.com/docs/api-reference/support/get-support-video

get /v1/support/videos/{video_identifier}



# List correspondence (ticket messages + broadcasts, org + global).
Source: https://docs.mixpeek.com/docs/api-reference/support/list-correspondence

post /v1/support/comms/list



# List Correspondence
Source: https://docs.mixpeek.com/docs/api-reference/support/list-correspondence-get

get /v1/support/comms



# List curated support videos (org + global).
Source: https://docs.mixpeek.com/docs/api-reference/support/list-curated-support-videos

post /v1/support/videos/list



# List Support Tickets
Source: https://docs.mixpeek.com/docs/api-reference/support/list-support-tickets

get /v1/support/tickets



# List Support Tickets (org + global).
Source: https://docs.mixpeek.com/docs/api-reference/support/list-support-tickets-list

post /v1/support/tickets/list



# List Support Videos
Source: https://docs.mixpeek.com/docs/api-reference/support/list-support-videos

get /v1/support/videos



# Presigned PUT to upload a support video to cold storage (admin only).
Source: https://docs.mixpeek.com/docs/api-reference/support/presigned-put-upload-support-video-cold

post /v1/support/videos/upload-url



# Register a support video (admin only); triggers Gemini review for uploads.
Source: https://docs.mixpeek.com/docs/api-reference/support/register-support-video-triggers-gemini-review

post /v1/support/videos



# Reply to a support ticket.
Source: https://docs.mixpeek.com/docs/api-reference/support/reply-support-ticket

post /v1/support/tickets/{ticket_identifier}/messages



# Update a support ticket (status/priority).
Source: https://docs.mixpeek.com/docs/api-reference/support/update-support-ticket

patch /v1/support/tickets/{ticket_identifier}



# Create Bucket Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/create-bucket-template

post /v1/templates/buckets/from-bucket/{bucket_id}
Create template from existing bucket.

Supports three template scopes:
- **organization**: Available to all users in your organization (default)
- **user**: Available only to you
- **system**: Available to all organizations (requires Mixpeek admin email)



# Create Cluster Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/create-cluster-template

post /v1/templates/clusters/from-cluster/{cluster_id}
Create template from existing cluster.



# Create Collection Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/create-collection-template

post /v1/templates/collections/from-collection/{collection_id}
Create template from existing collection.

Supports three template scopes:
- **organization**: Available to all users in your organization (default)
- **user**: Available only to you
- **system**: Available to all organizations (requires Mixpeek admin email)



# Create Namespace Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/create-namespace-template

post /v1/templates/namespaces/from-namespace/{namespace_id}
Create template from existing namespace.

Supports three template scopes:
- **organization**: Available to all users in your organization (default)
- **user**: Available only to you
- **system**: Available to all organizations (requires Mixpeek admin email)



# Create Retriever Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/create-retriever-template

post /v1/templates/retrievers/from-retriever/{retriever_id}
Create template from existing retriever.

Supports three template scopes:
- **organization**: Available to all users in your organization (default)
- **user**: Available only to you
- **system**: Available to all organizations (requires Mixpeek admin email)



# Create Taxonomy Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/create-taxonomy-template

post /v1/templates/taxonomies/from-taxonomy/{taxonomy_id}
Create template from existing taxonomy.

Supports three template scopes:
- **organization**: Available to all users in your organization (default)
- **user**: Available only to you
- **system**: Available to all organizations (requires Mixpeek admin email)



# Get Bucket Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/get-bucket-template

get /v1/templates/buckets/{template_id}
Get bucket template details.



# Get Cluster Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/get-cluster-template

get /v1/templates/clusters/{template_id}
Get cluster template details.



# Get Collection Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/get-collection-template

get /v1/templates/collections/{template_id}
Get collection template details.



# Get Namespace Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/get-namespace-template

get /v1/templates/namespaces/{template_id}
Get namespace template details.



# Get Retriever Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/get-retriever-template

get /v1/templates/retrievers/{template_id}
Get retriever template details.



# Get Taxonomy Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/get-taxonomy-template

get /v1/templates/taxonomies/{template_id}
Get taxonomy template details.



# Instantiate Bucket Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/instantiate-bucket-template

post /v1/templates/buckets/{template_id}/instantiate
Instantiate bucket template.



# Instantiate Cluster Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/instantiate-cluster-template

post /v1/templates/clusters/{template_id}/instantiate
Instantiate cluster template.



# Instantiate Collection Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/instantiate-collection-template

post /v1/templates/collections/{template_id}/instantiate
Instantiate collection template.



# Instantiate Retriever Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/instantiate-retriever-template

post /v1/templates/retrievers/{template_id}/instantiate
Instantiate retriever template.



# Instantiate Taxonomy Template
Source: https://docs.mixpeek.com/docs/api-reference/templates/instantiate-taxonomy-template

post /v1/templates/taxonomies/{template_id}/instantiate
Instantiate taxonomy template.



# List Bucket Templates
Source: https://docs.mixpeek.com/docs/api-reference/templates/list-bucket-templates

post /v1/templates/buckets
List bucket templates (system + organization + user).

Supports filtering, sorting, and search like other list operations.



# List Cluster Templates
Source: https://docs.mixpeek.com/docs/api-reference/templates/list-cluster-templates

post /v1/templates/clusters
List cluster templates (system + organization + user).

Supports filtering, sorting, and search like other list operations.



# List Collection Templates
Source: https://docs.mixpeek.com/docs/api-reference/templates/list-collection-templates

post /v1/templates/collections
List collection templates (system + organization + user).

Supports filtering, sorting, and search like other list operations.



# List Namespace Templates
Source: https://docs.mixpeek.com/docs/api-reference/templates/list-namespace-templates

get /v1/templates/namespaces
List namespace templates (system + organization + user).



# List Retriever Templates
Source: https://docs.mixpeek.com/docs/api-reference/templates/list-retriever-templates

post /v1/templates/retrievers
List retriever templates (system + organization + user).

Supports filtering, sorting, and search like other list operations.

**Request Body (optional):**
- `filters`: Attribute-based filters `{"AND": [{"field": "category", "operator": "eq", "value": "semantic_search"}]}`
- `sort`: Sort options `{"field": "name", "direction": "asc"}`
- `search`: Wildcard search across template_id, name, description, tags
- `scope`: Filter by scope (system, organization, user)
- `category`: Filter by category
- `is_active`: Show only active templates (default: true)
- `tags`: Filter by tags (templates must have ALL specified tags)



# List Taxonomy Templates
Source: https://docs.mixpeek.com/docs/api-reference/templates/list-taxonomy-templates

post /v1/templates/taxonomies
List taxonomy templates (system + organization + user).

Supports filtering, sorting, and search like other list operations.



# Analyze Bottlenecks
Source: https://docs.mixpeek.com/docs/api-reference/analytics/analyze-bottlenecks

get /v1/analytics/performance/engine/bottlenecks
Identify performance bottlenecks.

Analyzes profiling data to identify the biggest bottlenecks,
ranked by total time spent.

**Use Cases:**
- Find what's slowing down your pipelines
- Prioritize optimization efforts
- Monitor bottleneck trends

**Ranking:**
- Sorted by total time spent (sum across all executions)
- Shows percentage of total execution time
- Includes execution count and average time

**Example:**
```bash
GET /v1/analytics/performance/engine/bottlenecks?hours=24&limit=10
```



# Analyze For Tuning
Source: https://docs.mixpeek.com/docs/api-reference/analytics/analyze-for-tuning

post /v1/analytics/retrievers/{retriever_id}/analyze-tuning
Analyze retriever and generate tuning recommendations.

Performs comprehensive analysis and generates actionable recommendations:
- Parameter tuning suggestions
- Cache optimization opportunities
- Performance improvement estimates

**Recommendations Include:**
- Increase/decrease k value
- Adjust reranking thresholds
- Enable/optimize caching
- Stage reordering suggestions

**Use Cases:**
- Initial retriever configuration
- Periodic performance optimization
- A/B testing parameter changes
- Cost optimization

**Example:**
```bash
curl -X POST "https://api.mixpeek.com/v1/analytics/retrievers/ret_abc123/analyze-tuning?days=7" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Assignment Metrics
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-assignment-metrics

get /v1/analytics/taxonomies/{taxonomy_id}/assignments
Get taxonomy assignment metrics over time.

Tracks assignment counts including:
- Assignment volume over time
- Average confidence scores
- Unique labels assigned

**Use Cases:**
- Monitor taxonomy usage
- Track assignment trends
- Identify popular labels



# Get Batch Diagnostics
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-batch-diagnostics

get /v1/analytics/performance/batches/{batch_id}/diagnostics
Get comprehensive diagnostics for a batch.

Combines batch status, task progress, collection info, performance metrics,
and actionable insights into a single response for easy frontend rendering.

**Use Cases:**
- Monitor batch processing in real-time
- Debug failed batches
- View performance breakdown after completion
- Get actionable next steps

**Response includes:**
- Overall batch status and progress
- Per-tier task details with Ray job links
- Collection document counts
- Performance insights and bottlenecks (if completed)
- Error details (if failed)
- Recommended next actions

**Example:**
```bash
GET /v1/analytics/performance/batches/{batch_id}/diagnostics
```

**Perfect for:**
- Real-time progress tracking UI
- Batch monitoring dashboards
- Debugging failed extractions
- Performance optimization



# Get Bucket Health
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-bucket-health

get /v1/analytics/buckets/{bucket_id}/health
Get bucket health monitoring metrics.

Analyzes bucket health including:
- Error breakdown by type
- Sync health per configuration
- Stuck/failing syncs
- Overall health status

**Use Cases:**
- Monitor bucket health
- Identify failing syncs
- Debug errors by type
- Alert on unhealthy buckets

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/buckets/bkt_abc123/health?hours=24" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Bucket Storage
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-bucket-storage

get /v1/analytics/buckets/{bucket_id}/storage
Get storage growth trends over time.

Analyzes bucket storage metrics including:
- Total storage size over time
- Object count trends
- Growth rates

**Use Cases:**
- Monitor storage capacity planning
- Identify storage growth patterns
- Track object accumulation
- Alert on unexpected growth

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/buckets/bkt_abc123/storage?group_by=day" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Bucket Usage
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-bucket-usage

get /v1/analytics/buckets/{bucket_id}/usage
A bucket's storage, upload and sync usage over time.

Per period: storage_gb_hours, the bytes the bucket held integrated over
time from its hourly storage snapshots (GB is 10^9 bytes; time before the
first snapshot in the window is not counted); upload_operations, the
objects created through the object create endpoints; and sync_operations,
the sync runs that finished, successful or failed. Objects a sync creates
count under its run, not as uploads.

Credits and estimated cost read 0: storage, uploads and syncs have no
price yet, and this route reports quantities until the invoice's price
table carries them. The default window is the last 30 days.



# Get Cache Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-cache-performance

get /v1/analytics/retrievers/{retriever_id}/cache-performance
Get cache performance metrics.

Analyzes cache effectiveness including:
- Hit/miss rates
- Latency comparison (cache vs full search)
- Hourly cache performance trends

**Use Cases:**
- Evaluate cache effectiveness
- Optimize cache TTL settings
- Monitor cache performance
- Identify cache warming opportunities

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/retrievers/ret_abc123/cache-performance?hours=168" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Collection Overview
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-collection-overview

get /v1/analytics/collections/{collection_id}/overview
Get high-level collection health and status metrics.

Provides collection health including:
- Total document count and recent growth
- Processing performance and success rates
- Active enrichments (taxonomies, clusters)

**Use Cases:**
- Monitor collection health
- Quick status check
- Identify collections needing attention



# Get Compound Index Patterns
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-compound-index-patterns

get /v1/analytics/namespaces/indexes/compound-patterns
Identify compound index opportunities.

Finds metadata fields commonly used together in filters, suggesting
opportunities for compound (multi-field) indexes.

**Use Cases:**
- Optimize multi-field queries
- Create compound indexes
- Understand query complexity
- Improve complex filter performance

**Response Includes:**
- Field combinations used together
- Frequency of combination usage
- Average and P95 latency
- Sorted by combination frequency

**Compound Index Example:**
If `brand + status` appears frequently, create:
```javascript
db.documents.createIndex({"metadata.brand": 1, "metadata.status": 1})
```

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/indexes/compound-patterns" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Confidence Distribution
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-confidence-distribution

get /v1/analytics/taxonomies/{taxonomy_id}/confidence
Get confidence score distribution.

Analyzes assignment confidence including:
- Distribution across confidence ranges
- Low-confidence assignment alerts
- Average confidence trends

**Use Cases:**
- Monitor taxonomy quality
- Identify low-confidence assignments
- Track confidence improvements



# Get Document Growth
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-document-growth

get /v1/analytics/collections/{collection_id}/growth
Get document growth trends over time.

Tracks document additions over time, useful for understanding
batch processing patterns and indexing velocity.



# Get Engine Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-engine-performance

get /v1/analytics/performance/engine
Get engine performance metrics over time.

Query profiling data logged by the engine's profiling infrastructure.
All queries are automatically filtered to your namespace.

**Time Range:**
- Specify `hours` for recent history (e.g., `hours=24` for last 24 hours)
- OR specify `start_date` and `end_date` for custom range
- Defaults to last 24 hours if neither provided

**Grouping:**
- `minute`: High-resolution (for short time ranges)
- `hour`: Standard resolution (default)
- `day`: For longer time ranges
- `week`, `month`: For historical trends

**Response:**
- Time-series metrics (avg, p50, p95, p99 latencies)
- Summary statistics across the entire time range
- All latencies in milliseconds

**Example:**
```bash
GET /v1/analytics/performance/engine?hours=24&group_by=hour
```



# Get Engine Stage Breakdown
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-engine-stage-breakdown

get /v1/analytics/performance/engine/stages
Get stage-level performance breakdown.

Breaks down engine performance by individual profiled stages
(e.g., pipeline_run, generate_input_dataset, gcs_batch_upload).

**Use Cases:**
- Identify which stages are slowest
- See percentage of total time per stage
- Optimize specific bottlenecks

**Response:**
- Per-stage metrics (count, avg, p95, max latencies)
- Total time spent in each stage
- Percentage of total execution time
- Sorted by total time (worst bottlenecks first)

**Example:**
```bash
GET /v1/analytics/performance/engine/stages?hours=24
```



# Get Enrichment History
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-enrichment-history

get /v1/analytics/taxonomies/{taxonomy_id}/enrichments
Get enrichment execution history.

Tracks enrichment operations including:
- Enrichment success rates
- Average latency
- Volume trends

**Use Cases:**
- Monitor enrichment performance
- Track success rates
- Identify performance issues



# Get Execution History
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-execution-history

get /v1/analytics/clusters/{cluster_id}/execution-history
Get cluster execution history with metrics.

Provides execution timeline including:
- Execution duration and document counts
- Algorithm and cluster counts
- Success/failure status

**Use Cases:**
- Monitor clustering performance
- Track execution patterns
- Identify execution issues



# Get Extractor Breakdown
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-extractor-breakdown

get /v1/analytics/performance/engine/extractors
Get extractor performance breakdown.

Shows performance metrics for each extractor and its stages.

**Use Cases:**
- Compare performance across extractors
- Identify slow extractor stages
- Monitor specific extractor performance

**Response:**
- Per-extractor, per-stage metrics
- Execution counts
- Latency statistics (avg, p95, max)

**Example:**
```bash
GET /v1/analytics/performance/engine/extractors?hours=24
```



# Get Extractor Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-extractor-performance

get /v1/analytics/extractors/performance
Get feature extraction performance metrics.

Returns per-extractor execution metrics from the ``extraction_events``
analytics collection over the last ``hours`` (namespace-scoped), one row per
(extractor_name, version):
- Total executions per extractor
- Average duration + P95/P99 latencies
- Success/failure counts and success rate

Returns an empty array when no extraction has run in the window or analytics
is unavailable.

**Example:**
```bash
GET /v1/analytics/extractors/performance?hours=24
GET /v1/analytics/extractors/performance?extractor_name=text_extractor&hours=168
```



# Get Extractor Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-extractor-performance-1

get /v1/analytics/collections/{collection_id}/extractors
Get feature extractor performance breakdown.

Analyzes extractor execution metrics including:
- Execution counts and success/failure rates
- Latency percentiles (P95, P99)
- Total processing time

**Use Cases:**
- Identify slow extractors for optimization
- Monitor extraction success rates
- Understanding processing bottlenecks



# Get Failure Analysis
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-failure-analysis

get /v1/analytics/collections/{collection_id}/failures
Get failure analysis metrics.

Debug processing failures including:
- Error distribution by type
- Recent error messages
- Failure patterns

**Use Cases:**
- Debug processing failures
- Identify common error patterns
- Track error trends



# Get Failure Analysis
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-failure-analysis-1

get /v1/analytics/clusters/{cluster_id}/failures
Get cluster failure analysis.

Analyzes clustering failures including:
- Error messages and types
- Failure timestamps
- Failure patterns

**Use Cases:**
- Debug clustering failures
- Identify common error patterns
- Monitor cluster health



# Get Field Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-field-performance

get /v1/analytics/namespaces/fields/performance
Analyze field performance correlation.

Shows which metadata fields correlate with slow queries, helping identify
fields that would benefit most from indexing.

**Use Cases:**
- Identify fields causing performance issues
- Quantify indexing impact potential
- Prioritize index creation
- Monitor field usage patterns

**Response Includes:**
- Field usage count
- Latency statistics (avg, P50, P95, P99, max)
- Index priority score (usage × latency)
- Sorted by priority score

**Index Priority Score:**
Higher scores indicate fields where indexing would have greatest impact.
Score = (query count) × (average latency)

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/fields/performance?days=30" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Field Usage
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-field-usage

get /v1/analytics/indexes/usage
Get usage statistics for all filtered fields.



# Get Index Recommendations
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-index-recommendations

get /v1/analytics/namespaces/indexes/recommendations
Get comprehensive MongoDB index recommendations.

Analyzes query patterns and generates prioritized index recommendations
with ready-to-use MongoDB commands.

**Use Cases:**
- Get actionable index suggestions
- Prioritize database optimization
- Copy/paste index creation commands
- Track optimization opportunities

**Recommendation Levels:**
- **HIGH PRIORITY**: >100 queries, >300ms avg OR >10 very slow queries
- **MEDIUM PRIORITY**: >50 queries, >200ms avg OR >20 slow queries
- **LOW PRIORITY**: >10 queries but acceptable performance
- **NO ACTION**: Low usage, no optimization needed

**Response Includes:**
- Prioritized recommendations
- Usage and latency statistics
- MongoDB index creation commands
- Summary by priority level

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/indexes/recommendations" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Index Suggestions
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-index-suggestions

get /v1/analytics/indexes/suggestions
Get index suggestions based on filter usage patterns.



# Get inference performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-inference-performance

get /v1/analytics/inference/performance



# Get Label Distribution
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-label-distribution

get /v1/analytics/taxonomies/{taxonomy_id}/labels
Get label distribution.

Analyzes label usage including:
- Top labels by assignment count
- Label popularity percentages
- Confidence by label

**Use Cases:**
- Understand label usage
- Identify most common categories
- Balance taxonomy coverage



# Get Latency Metrics
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-latency-metrics

get /v1/analytics/collections/{collection_id}/latency
Get processing latency distribution.

Analyzes document processing latency including:
- Latency percentiles over time
- Slowest document operations
- Performance trends



# Get Most Queried Fields
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-most-queried-fields

get /v1/analytics/namespaces/fields/most-queried
Get most frequently queried metadata fields.

Identifies which metadata fields are accessed most often in retriever queries,
helping prioritize index creation and understand query patterns.

**Use Cases:**
- Identify fields that need indexing
- Understand common query patterns
- Prioritize optimization efforts
- Plan database schema improvements

**Response Includes:**
- Field name and usage frequency
- Average and P95 latency metrics
- Total unique fields analyzed

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/fields/most-queried?days=30&limit=20" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Namespace Buckets Health
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-namespace-buckets-health

get /v1/analytics/buckets/health
Get health for ALL buckets in the namespace in one request.

Batch counterpart of GET /{bucket_id}/health: two grouped analytics
queries cover the whole namespace instead of a query fan-out per bucket.
Buckets with no events in the window are omitted from the response and
should be treated as healthy.

**Use Cases:**
- Monitoring dashboards listing many buckets (replaces the per-bucket
  health call fan-out that scaled linearly with bucket count)
- Alerting sweeps over all buckets

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/buckets/health?hours=24" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Namespace Errors
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-namespace-errors

get /v1/analytics/namespaces/errors
Batch error/failure counts for every collection & cluster in the namespace.

Returns one entry per resource that has at least one error/failure in the
window, in a single response — replacing the per-resource
``/analytics/{collections,clusters}/{id}/failures`` fan-out (the Studio
Monitoring "Errors & Health" N+1, which fired ~13+ calls per visit, most
returning zero). Collection errors come from the
``extraction_events`` analytics collection; cluster failures from Mongo
``clustering_results``. An empty ``collections``/``clusters`` list means that
resource type is healthy for the window.

**Recommended workflow:** call this once for the aggregate health view, then
call the per-resource ``/failures`` endpoint lazily ONLY for the (usually
few) resources reporting errors to get the full error-type breakdown.

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/errors?hours=24" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Namespace Summary
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-namespace-summary

get /v1/analytics/namespaces/summary
Get comprehensive namespace optimization summary.

Provides a complete overview of namespace performance including:
- Top index recommendations
- Most queried fields
- Slowest fields
- Compound index opportunities
- Summary statistics

**Use Cases:**
- Get full optimization picture
- Regular performance reviews
- Database health checks
- Planning optimization work

**Response Includes:**
- Summary statistics (field counts, priority levels)
- Top 10 index recommendations
- Top 10 most queried fields
- Top 10 slowest fields
- Top 10 compound index opportunities

**Recommended Workflow:**
1. Call this endpoint for overview
2. Use specific endpoints for details
3. Implement high-priority recommendations
4. Monitor improvement over time

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/summary?days=30" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Retriever Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-retriever-performance

get /v1/analytics/retrievers/{retriever_id}/performance
Get retriever performance metrics for tuning.

Retrieves time-series performance data including:
- Query latency (P50, P95, P99)
- Query counts
- Result counts
- Latency trends

**Use Cases:**
- Monitor retriever performance over time
- Identify performance degradations
- Compare performance across time periods
- Establish performance baselines

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/retrievers/ret_abc123/performance?hours=24&group_by=hour" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Retriever Signals
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-retriever-signals

get /v1/analytics/retrievers/{retriever_id}/signals
Get retriever signals for interaction tuning.

Retrieves fine-grained signals about retriever behavior:
- Cache hits/misses
- Reranking scores
- Filter effectiveness
- Query expansion results

**Signal Types:**
- `cache_hit`: Successful cache lookups
- `cache_miss`: Cache misses requiring full search
- `rerank_scores`: Reranking effectiveness metrics
- `filter_reduction`: Pre-filter document reduction
- `expansion_results`: Query expansion impact

**Use Cases:**
- Fine-tune retrieval parameters
- Analyze query patterns
- Optimize cache strategies
- Validate optimization improvements

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/retrievers/ret_abc123/signals?signal_type=rerank_scores&limit=50" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Slow Queries
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-slow-queries

get /v1/analytics/namespaces/queries/slow
Get slow queries and their filter patterns.

Identifies queries exceeding a latency threshold and shows which metadata
fields they're filtering on, helping pinpoint optimization opportunities.

**Use Cases:**
- Troubleshoot slow queries
- Identify unindexed fields causing slowdowns
- Debug performance issues
- Optimize query patterns

**Response Includes:**
- Query details (retriever, inputs, latency)
- Results count
- Metadata fields being filtered
- Full query context

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/namespaces/queries/slow?latency_threshold_ms=1000" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Slowest Operations
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-slowest-operations

get /v1/analytics/performance/engine/slow-operations
Get slowest individual operations.

Returns the slowest profiled operations, useful for debugging
specific slow executions.

**Use Cases:**
- Troubleshoot specific slow operations
- Identify outliers
- Deep dive into problematic executions

**Filtering:**
- `threshold_ms`: Only show operations slower than this
- Default: 1000ms (1 second)

**Response:**
- Timestamp of slow operation
- Stage name and component
- Latency in milliseconds
- Full metadata context

**Example:**
```bash
GET /v1/analytics/performance/engine/slow-operations?hours=24&threshold_ms=5000
```



# Get Slowest Queries
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-slowest-queries

get /v1/analytics/retrievers/{retriever_id}/slow-queries
Get slowest queries for troubleshooting.

Identifies slowest-performing queries for optimization:
- Query text
- Execution time
- Result counts
- Stage breakdown

**Use Cases:**
- Identify problematic queries
- Debug performance issues
- Optimize query patterns
- User experience improvements

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/retrievers/ret_abc123/slow-queries?limit=10&hours=24" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Stage Breakdown
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-stage-breakdown

get /v1/analytics/retrievers/{retriever_id}/stages
Get stage-level performance breakdown.

Analyzes individual stage performance to identify bottlenecks:
- Stage execution times
- Document flow (in/out)
- Stage-level latency distribution

**Use Cases:**
- Identify slow stages in retrieval pipeline
- Optimize stage ordering
- Debug pipeline bottlenecks
- Understand document reduction rates

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/retrievers/ret_abc123/stages?hours=24" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Sync Comparison
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-sync-comparison

get /v1/analytics/buckets/{bucket_id}/sync-comparison
Compare performance across sync configurations.

Compares sync configurations by:
- Average duration and throughput
- Success rates
- Total files and bytes synced
- Provider performance

**Use Cases:**
- Compare sync providers (S3 vs GCS)
- Optimize sync configurations
- Identify best-performing syncs
- Benchmark sync strategies

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/buckets/bkt_abc123/sync-comparison?hours=168" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Sync Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-sync-performance

get /v1/analytics/buckets/{bucket_id}/sync-performance
Get sync performance metrics.

Analyzes sync job execution including:
- Files synced/failed
- Sync duration and throughput
- Success rates by provider

**Use Cases:**
- Monitor sync reliability
- Compare sync configurations
- Identify slow syncs
- Debug sync failures

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/buckets/bkt_abc123/sync-performance?hours=168" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Upload Performance
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-upload-performance

get /v1/analytics/buckets/{bucket_id}/upload-performance
Get upload performance metrics.

Analyzes upload operations including:
- Upload latency (P50, P95, P99)
- Throughput (MB/s)
- Error rates

**Use Cases:**
- Monitor upload performance
- Identify performance degradations
- Optimize upload strategies
- Debug upload issues

**Example:**
```bash
curl -X GET "https://api.mixpeek.com/v1/analytics/buckets/bkt_abc123/upload-performance?hours=24" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Namespace: your-namespace"
```



# Get Usage Summary
Source: https://docs.mixpeek.com/docs/api-reference/analytics/get-usage-summary

get /v1/analytics/usage/summary
Get usage summary for billing.

Aggregates the organization's usage_records — the SAME billing source of
truth that feeds daily_burn_rate and monthly invoicing, so this
endpoint and the burn meter always agree.

Returns per-operation credit totals for the window with USD derived at the
canonical $0.001/credit rate (CREDIT_RATE_USD):
- ``usage``: one row per operation_type — ``{operation_type, credits,
  cost_usd}`` — biggest spend first. Refund records are negative and net
  automatically.
- ``total_credits`` / ``total_cost``: window totals across operations.
- ``multi_vector_tokens``: tokens of multi-vectors stored by writes, as
  carried by the ``mvs_write`` usage records in the window (``recorded``),
  plus tokens written since the last record (``pending``).

Scoped to the ORGANIZATION (matching the billing meter). Records are only
partially namespace-attributed (async engine work carries no namespace),
so filtering by namespace would silently under-report; ``namespace_id`` is
echoed for request context only.

**Time Range:**
- If both `start_date` and `end_date` are provided, uses that range
- If neither provided, defaults to last 30 days
- If only one provided, defaults to now as the other bound

**Example:**
```bash
GET /v1/analytics/usage/summary
GET /v1/analytics/usage/summary?start_date=2025-01-01T00:00:00Z&end_date=2025-01-31T23:59:59Z
```



# Get All Discovery Information
Source: https://docs.mixpeek.com/docs/api-reference/discovery/get-all-discovery-information

get /v1/discovery
Returns combined discovery information including extractors, stages, and manifest schema in a single request. Use this for comprehensive capability discovery.



# Get Manifest Schema for Agent Configuration
Source: https://docs.mixpeek.com/docs/api-reference/discovery/get-manifest-schema-for-agent-configuration

get /v1/discovery/schema
Returns the complete manifest schema including all resource types, their JSON schemas, dependency graph, and YAML examples. Use this for programmatic manifest generation and validation.



# List Available Feature Extractors
Source: https://docs.mixpeek.com/docs/api-reference/discovery/list-available-feature-extractors

get /v1/discovery/extractors
Discover all available feature extractors with their capabilities, supported modalities, output features, and example usage. Use this to understand what extractors are available when configuring namespaces and collections in manifests.



# List Available Retriever Stages with Examples
Source: https://docs.mixpeek.com/docs/api-reference/discovery/list-available-retriever-stages-with-examples

get /v1/discovery/stages
Discover all available retriever stages with extended information including example configurations, common use cases, and cost tiers. This endpoint provides more context than /v1/retrievers/stages for agent-driven configuration.



# List change feed events
Source: https://docs.mixpeek.com/docs/api-reference/events/list-events

get /v1/events
List this organization's change feed, ordered and resumable by cursor.

**Scope**: org-global by default — one ordered sequence across every
namespace and resource type. An `X-Namespace` header (or `namespace_id`
query param) narrows it to one namespace's events. Events recorded
before namespace stamping shipped (2026-08) carry no namespace and are
excluded by ANY namespace filter — a namespace-filtered read is not a
replay of history older than that.

A consumer can be killed mid-stream and resume from its last stored
`next_cursor` without re-reading or missing anything, **as long as it
resumes within the 90-day retention window** — events older than that
are permanently expired, not archived. A consumer that has been down
longer than 90 days must resync current state instead of resuming.

A cursor is only meaningful for a FIXED filter set: it encodes a
position in this organization's overall sequence, not a position
within any particular `namespace_id`/`event_type` filter. Changing
either filter mid-stream while reusing an old cursor silently skips
whatever the previous filter combination would have matched in
between — start a fresh cursor (or none) whenever the filters change.

High-churn event types (`object.*` under active storage syncs) can bury
sparse ones thousands of pages deep in the unfiltered walk — a consumer
looking for one type should pass `event_type` rather than walking
everything.



# Execute Raw Inference
Source: https://docs.mixpeek.com/docs/api-reference/inference/execute-raw-inference

post /v1/inference
Execute raw inference with provider+model or custom plugin.

This endpoint provides direct access to inference services without
the retriever framework overhead. Supports two modes:

1. **Provider + Model**: Use standard providers (openai, google, anthropic).
   ``gemini`` is accepted as an alias for ``google``.
2. **Custom Plugin**: Use your custom inference plugins by inference_name

## Supported Providers

- **openai**: GPT models, embeddings, Whisper transcription
- **google** (alias **gemini**): Gemini models, Vertex multimodal embeddings (1408D)
- **anthropic**: Claude models

## Examples

### Custom Plugin (by inference_name)
```json
{
    "inference_name": "my_text_embedder_1_0_0",
    "inputs": {"text": "hello world"},
    "parameters": {}
}
```

### Custom Plugin (by feature_uri)
```json
{
    "feature_uri": "mixpeek://my_custom_embedder@1.0.0/embedding",
    "inputs": {"text": "hello world"},
    "parameters": {}
}
```

### Builtin Embedder (by feature_uri)
```json
{
    "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
    "inputs": {"text": "hello world"},
    "parameters": {}
}
```

### Chat Completion
```json
{
    "provider": "openai",
    "model": "gpt-4o-mini",
    "inputs": {"prompts": ["What is AI?"]},
    "parameters": {"temperature": 0.7, "max_tokens": 500}
}
```

### Text Embedding (OpenAI)
```json
{
    "provider": "openai",
    "model": "text-embedding-3-large",
    "inputs": {"text": "machine learning"},
    "parameters": {}
}
```

### Text Embedding (Google Vertex Multimodal - 1408D)
```json
{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"text": "machine learning"},
    "parameters": {}
}
```

### Image Embedding (Google Vertex Multimodal - 1408D)
```json
{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"image_url": "https://example.com/image.jpg"},
    "parameters": {}
}
```

### Image Embedding from Base64
```json
{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"image_base64": "<base64-encoded-image>"},
    "parameters": {}
}
```

### Video Embedding (Google Vertex Multimodal - 1408D)
```json
{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"video_url": "https://example.com/video.mp4"},
    "parameters": {}
}
```

### Video Embedding from Base64
```json
{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"video_base64": "<base64-encoded-video>"},
    "parameters": {}
}
```

### Audio Transcription
```json
{
    "provider": "openai",
    "model": "whisper-1",
    "inputs": {"audio_url": "https://example.com/audio.mp3"},
    "parameters": {}
}
```

### Vision (Multimodal LLM)
```json
{
    "provider": "openai",
    "model": "gpt-4o",
    "inputs": {
        "prompts": ["Describe this image"],
        "image_url": "https://example.com/image.jpg"
    },
    "parameters": {"temperature": 0.5}
}
```

Args:
    request: FastAPI request object (populated by middleware)
    payload: Raw inference request

Returns:
    Inference response with results and metadata

Raises:
    400 Bad Request: Invalid provider, model, or inputs
    422 Unprocessable Entity: inputs carries no key the provider call
        reads (for example ``prompt`` instead of ``prompts``)
    401 Unauthorized: Missing or invalid API key
    429 Too Many Requests: Rate limit exceeded
    500 Internal Server Error: Inference execution failed



# Diff Manifest
Source: https://docs.mixpeek.com/docs/api-reference/manifest/diff-manifest

post /v1/manifest/diff
Compare a manifest file with current state.

Shows resources that would be:
- **Created**: In manifest but not in system
- **In system only**: In system but not in manifest
- **Different**: In both but with configuration differences

This is useful for understanding what changes would occur
before applying a manifest.

**Example:**
```bash
curl -X POST /v1/manifest/diff \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```



# Generate Manifest
Source: https://docs.mixpeek.com/docs/api-reference/manifest/generate-manifest

post /v1/manifest/generate
Generate a manifest from natural language description.

    Uses AI to create a valid YAML manifest from a natural language description
    of desired resources. The generated manifest can be reviewed and applied.

    **Example:**
    ```bash
    curl -X POST /v1/manifest/generate \
      -H "Authorization: Bearer $API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "description": "I need a bucket of images that feeds into a multimodal extractor and a retriever",
        "manifest_name": "image-search-setup"
      }'
    ```

    The model is given the live extractor and retriever-stage catalogs. The
    answer is checked with the same rules as POST /v1/manifest/validate, and
    when that finds errors the model is shown them and asked to repair its
    answer, up to 2 times.

    **Response:**
    ```json
    {
      "manifest": "version: '1.0'
metadata:
  name: image-search-setup
...",
      "format": "yaml",
      "manifest_name": "image-search-setup",
      "description": "I need a bucket of images...",
      "validation": {"valid": true, "errors": [], "warnings": []},
      "repair_rounds": 1
    }
    ```

    `validation.valid` is false when the manifest still has errors after the
    repair rounds; the errors are listed so you can fix them by hand.

    **Next Steps:**
    1. Review the generated manifest
    2. Apply it using POST /v1/manifest/apply



# Lint Manifest
Source: https://docs.mixpeek.com/docs/api-reference/manifest/lint-manifest

post /v1/manifest/lint
Lint a YAML manifest for best practices and potential issues.

Goes beyond basic validation to provide actionable suggestions for
improving your manifest configuration. This endpoint is designed for
AI agents and developers who want to optimize their Mixpeek setup.

**Lint Rules:**
- `UNUSED_EXTRACTOR`: Feature extractor defined but not used by any collection
- `UNUSED_COLLECTION`: Collection not referenced by any retriever
- `MISSING_INPUT_SCHEMA`: Retriever uses templates but has no input_schema
- `SUBOPTIMAL_STAGE_ORDER`: Filter stages after expensive operations
- `DUPLICATE_FEATURE_URI`: Same feature searched multiple times
- `MISSING_DESCRIPTION`: Resources without descriptions
- `NO_SEARCH_STAGE`: Retriever with no search stages
- `EXTRACTOR_NOT_IN_NAMESPACE`: Collection uses extractor not in namespace
- `MISSING_SECRET`: Secret reference not configured
- `UNUSED_BUCKET`: Bucket that no collection reads
- `TAXONOMY_INPUT_NOT_WRITTEN`: Taxonomy reads a feature its collection does not write
- `RETRIEVER_FEATURE_NOT_IN_COLLECTIONS`: Retriever searches a feature none of its collections write

**Severity Levels:**
- `error`: Must be fixed before applying
- `warning`: Best practice violation, should be fixed
- `info`: Suggestion for improvement

**Example:**
```bash
curl -X POST /v1/manifest/lint \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

**Response includes actionable suggestions:**
```json
{
  "valid": true,
  "results": [
    {
      "code": "MISSING_INPUT_SCHEMA",
      "severity": "warning",
      "message": "Retriever 'product_search' uses template variables but has no input_schema defined",
      "location": "retrievers[0]",
      "suggestion": "Define input_schema to document and validate retriever inputs",
      "fix_example": "input_schema:\n  type: object\n  properties:\n    query:\n      type: string"
    }
  ],
  "summary": {"error": 0, "warning": 1, "info": 0}
}
```



# Get the org's signup-time suggested namespace
Source: https://docs.mixpeek.com/docs/api-reference/onboarding/get-suggested-config

get /v1/onboarding/suggested-config
Returns the namespace configuration that was suggested from Mixpeek's company research at signup, if one was provisioned. Studio uses this to badge the suggested namespace.



# Infer onboarding configuration
Source: https://docs.mixpeek.com/docs/api-reference/onboarding/infer-onboarding-configuration

post /v1/onboarding/infer-config
Given a free-text use-case description, uses an LLM to recommend the best scaffold template, extractors, retriever stages, and sample queries for the user's project.



# Search Resources
Source: https://docs.mixpeek.com/docs/api-reference/resource-search/search-resources

post /v1/resources/search
Search across all resource names and IDs within your namespace.

    This endpoint performs a case-insensitive search across:
    - Buckets (bucket_name, bucket_id)
    - Collections (collection_name, collection_id)
    - Retrievers (retriever_name, retriever_id)
    - Taxonomies (taxonomy_name, taxonomy_id)
    - Clusters (cluster_name, cluster_id)
    - Namespaces (namespace_name, namespace_id)

    Results are sorted by relevance (exact matches first) and creation time (newest first).
    Use the resource_types parameter to filter searches to specific resource types.
    Pagination is supported via limit and offset parameters.