Manifests

Manifests let you define Mixpeek resources in YAML files and apply them in a single operation. This enables version-controlled, reproducible infrastructure across environments.

Quick Start

# mixpeek.yaml — a complete pipeline: ingest → index → search
version: "1.0"
metadata:
  name: "video-search-env"

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1

buckets:
  - name: raw_videos
    namespace: video_search
    schema:
      properties:
        video: { type: video }

collections:
  - name: video_index
    namespace: video_search
    source:
      type: bucket
      bucket: raw_videos
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v1

retrievers:
  - name: video_search_tool
    namespace: video_search
    collections: [video_index]
    input_schema:
      query:
        type: text
        required: true
    stages:
      - stage_name: search
        stage_type: filter
        config:
          stage_id: feature_search
          parameters:
            searches:
              - collection_identifiers: [video_index]
                query:
                  input_mode: text
                  text: "{{INPUT.query}}"
                feature_uri: "mixpeek://multimodal_extractor@v1/embedding"
                top_k: 10
            final_top_k: 10

Every collection needs a source and a feature_extractor; every retriever needs namespace, collections, and its stages pipeline. validate names any missing field, warns on unknown top-level keys (a typo like retreivers: won’t silently drop your search pipeline), and checks that references (namespace, bucket, collection names) resolve within the manifest.

curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"

Applying over resources that already exist

apply takes a mode query parameter. It defaults to create_only, which fails if any resource in the manifest already exists, because the manifest is treated as a closed world: a reference to a bucket that exists but is not declared is a validation error. Use create_missing to add to a namespace that already has resources. It creates what is missing, leaves what exists alone, and resolves references against the resources already in the namespace.

# Add these collections over buckets that are already there
curl -X POST "https://api.mixpeek.com/v1/manifest/apply?mode=create_missing" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@collections.yaml"

apply plans before it creates anything. It runs the same checks as validate, and any error refuses the whole manifest: nothing is created, and each error names the resource, the field and the fix. The plan refuses:

an extractor that no longer exists or is turned off on this deployment
a retriever stage that is not registered, such as a retired stage type
a feature URI that no collection in the manifest writes, such as one naming a renamed model
an alert application that does not map an input its alert’s retriever requires

A namespace budget above your organization’s budget is set to the organization’s budget, and apply reports the change in warnings. dry_run=true returns that plan and creates nothing. If a resource fails partway through, apply rolls back the ones it already created, and if the API is stopped in the middle of an apply, it rolls that apply back when it starts again.

Core Operations

Operation	Description
Apply	Create all resources defined in the manifest. `mode=create_only` (default) fails if anything exists; `mode=create_missing` fills the gaps
Validate	Check syntax, schema, and references without changes
Export	Generate a manifest from existing resources
Diff	Compare manifest against current state

Resource Types

Manifests support these resource types, applied in dependency order:

Namespaces - Isolation boundaries with feature extractors and a budget
Extractors - The extractors collections run, declared under extractors:
Storage connections and buckets - Object storage with schemas
Bucket syncs - Scheduled pulls from a connection
Collections - Document stores with indexing
Retrievers - Search pipelines
Alerts - Retrievers or system conditions that notify you
Clusters and taxonomies - Grouping and classification
Triggers and webhooks - Automation

Extractors

An extractors: section declares the extractors the manifest’s collections run. Each entry has a name, a namespace, a version (default v1), a source and an optional compute_profile. Collections reference an entry by name in feature_extractor, and namespaces in feature_extractors. Once a manifest has an extractors: section, every extractor a collection or namespace names must be declared in it. Extractors apply before collections, so a collection never names an extractor that does not exist yet. source is one of:

marketplace: a feature extractor name from the registry, for example text_extractor. This is the default when source is omitted ({marketplace: <name>}).
model: a model your organization uploaded through POST /v1/models/uploads, as org:<model_id>. Apply validates the model if it has not been validated, then serves it as an extractor at zero replicas until the first call.

compute_profile carries the fields of a compute profile (resource_type, batch_size, max_concurrency, memory). Apply stores them as the namespace’s batch override for that extractor, merged over the extractor’s built-in profile when a batch is built. Two nested blocks are checked rather than stored:

batch.enabled: only true is accepted. A collection that names the extractor runs it in batch.
serve: the real-time serving block (enabled, min_replicas, max_replicas, min_reason, idle_timeout_s). Validate and apply check it; realtime still runs under the extractor’s built-in serve settings, and warnings says so. A min_replicas above 0 is refused with the monthly cost of the warm replicas named, because a template adds no always-on floor. Leave it at 0 and the first call after idle waits for a replica to start.

extractors:
  - name: text_extractor
    namespace: video_search
    version: v1
    compute_profile:
      resource_type: cpu
      batch_size: 32
  - name: brand_classifier
    namespace: video_search
    source:
      model: org:mdl_7f3a9c

collections:
  - name: transcripts
    namespace: video_search
    source: {type: bucket, bucket: raw_videos}
    feature_extractor:
      feature_extractor_name: text_extractor
      version: v1

A collection naming an extractor the section does not declare is refused at plan with the name. Applying the same manifest again is a no-op for the extractors it declares.

Parameters

A parameters: section declares the values a manifest needs from whoever applies it, so a template ships without a hand edit or a secret inside it. Reference a parameter anywhere in the document as ${{ params.<name> }}. A reference that is the whole value takes the parameter’s type; one inside a longer string is formatted into it as text.

Field	Type	Default	What it does
`name`	string	required	The name you reference as `${{ params.<name> }}`
`type`	`string`, `integer`, `number`, `boolean`	`string`	The type a supplied value must match
`required`	boolean	`true`	Whether a value must be supplied. Ignored when `default` is set
`default`	matches `type`	none	Used when no value is supplied
`description`	string	none	Shown next to the input in Studio

parameters:
  - name: camera_url
    type: string
    description: Your camera's rtsp:// url
  - name: polling_seconds
    type: integer
    default: 300

buckets:
  - name: camera_feed
    namespace: site_cameras
    sync:
      source_url: ${{ params.camera_url }}
      interval_seconds: ${{ params.polling_seconds }}

Pass the values on validate and apply as the parameters form field, a JSON object of name to value:

curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@camera.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'

A missing required parameter, a value of the wrong type, a reference to an undeclared parameter, or a name declared twice is refused before any resource is created, naming parameters.<name>.

Budget

A namespace entry takes a budget with daily_usd and monthly_usd, the same fields the namespaces API takes as spend_budget. Omit it to use your organization’s namespace default. A value above your organization’s budget is set to the organization’s budget, and apply reports the clamp in warnings.

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1
    budget:
      daily_usd: 5
      monthly_usd: 100

See Budgets for what happens when work reaches the limit.

Alerts

An alerts: section creates alerts beside the retrievers they watch. An alert mirrors the create request of the alerts API, plus namespace and retriever by name.

Field	Type	Default	What it does
`name`	string	required	Unique alert name
`namespace`	string	required	Namespace by name
`source`	`retriever` or `system`	`retriever`	Run a retriever, or watch a built-in condition
`retriever`	string	none	A retriever declared in this manifest, by name (`source: retriever`)
`retriever_id`	string	none	An existing retriever’s id instead of `retriever`; set one of the two
`trigger_on`	`results` or `no_results`	`results`	When a retriever alert fires
`system_condition`	object	none	The built-in condition to watch (`source: system`)
`notification_config`	object	required	How and where to notify
`enabled`	boolean	`true`	Whether the alert runs

alerts:
  - name: no_results_for_brand_queries
    namespace: video_search
    retriever: video_search_tool
    trigger_on: no_results
    notification_config:
      channels:
        - channel_type: email
          config:
            to_addresses: [ops@example.com]

Plan refuses an alert that does not map an input its retriever requires. The alert fields are described on Alerts.

Track an apply

apply returns a task_id (absent on a dry run). GET /v1/tasks/{task_id} returns every resource of the plan with its states in order (planned, creating, created, exists, updated, or failed with the finding; a collection also passes warming and ready), each with the time it was reached. The same events arrive on the change feed and on a webhook.

curl https://api.mixpeek.com/v1/tasks/$TASK_ID \
  -H "Authorization: Bearer $API_KEY"

The apply response also carries readiness: for each collection whose extractor needs a real-time serving app, whether that app is running. An app scaled to zero is running; its first call answers 503 with Retry-After while it warms.

Secret References

Use ${{ secrets.NAME }} to reference organization secrets:

buckets:
  - name: external_data
    namespace: my_namespace
    sync:
      connection_id: ${{ secrets.S3_CONNECTION_ID }}

Secrets must exist before applying. Use the validate endpoint to check for missing secrets.

Seed Data

A seed: section gives a namespace sample objects to answer its first query. After the resources are created, apply ingests them into the named bucket with one batch, bounded by max_objects (default 50, at most 500). Applying the manifest again finds that batch and ingests nothing more. Each object names a sample file for each blob property under blobs. Keys come from the Mixpeek sample set, samples/starter/<file>. When the bucket declares other properties, set them on the object beside blobs, the way the object create API takes them:

buckets:
  - name: ad-creatives
    namespace: my_namespace
    schema:
      properties:
        creative_asset_url: {type: video}
        ad_id: {type: string, required: true}
        brand_slug: {type: string, required: true}

seed:
  - bucket: ad-creatives
    max_objects: 2
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}
        ad_id: sample-norelco
        brand_slug: norelco

Validate and apply check every object against the bucket’s schema before anything is created. A missing required property, a property the bucket does not declare, a value of the wrong type, or a blob property given outside blobs is refused, and the error names the bucket, the object and the property.

Acceptance

A template can carry an acceptance: block: the steps that show its use case works and what each must return. Apply creates nothing from it. Mixpeek’s template acceptance harness runs the steps after an apply, in order, and reports the step that missed.

Field	Type	What it does
`steps`	list, at least one	The steps, run in order. Step names must be unique
`steps[].name`	string	Shown when the step fails
`steps[].kind`	`query`, `documents`, `alert` or `realtime`	What the step does
`steps[].retriever`	string	`query`: a retriever declared in this manifest
`steps[].collection`	string	`documents`: a collection declared in this manifest
`steps[].alert`	string	`alert`: an alert declared in this manifest
`steps[].feature_uri`	string	`realtime`: the `mixpeek://` feature URI to call
`steps[].inputs`	object	`query` and `realtime`: the inputs sent
`steps[].expect`	object	The outcome the step must produce, below
`max_cost_usd`	number	The most one acceptance run may cost, read from the meter

Each kind reads its own expect fields: query reads min_results, top_field, top_equals and top_contains; documents reads min_documents; alert reads min_fires; realtime reads dims, the length of the returned vector.

acceptance:
  max_cost_usd: 0.5
  steps:
    - name: seeded documents written
      kind: documents
      collection: transcripts
      expect: {min_documents: 2}
    - name: golden query
      kind: query
      retriever: video_search_tool
      inputs: {query: "electric shaver"}
      expect: {min_results: 1, top_field: metadata.brand_slug, top_equals: norelco}

Validate refuses a step whose retriever, collection or alert this manifest does not declare, an expect field the step’s kind does not read, and a step with no outcome, each naming acceptance.steps.<name>.

Dependency Resolution

Resources are created in topological order. The manifest engine:

Resolves cross-resource references automatically
Detects circular dependencies
Rolls back all changes if any creation fails

Workflows

Environment Replication

# Export from production
curl https://api.mixpeek.com/v1/manifest/export \
  -H "Authorization: Bearer $PROD_KEY" \
  -o prod-manifest.yaml

# Apply to staging
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $STAGING_KEY" \
  -F "manifest_file=@prod-manifest.yaml"

Pre-deployment Validation

# Validate without applying
curl -X POST https://api.mixpeek.com/v1/manifest/validate \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"

# Check what would change
curl -X POST https://api.mixpeek.com/v1/manifest/diff \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"

CI/CD Integration

# GitHub Actions example
- name: Deploy Mixpeek Resources
  run: |
    curl -X POST https://api.mixpeek.com/v1/manifest/apply \
      -H "Authorization: Bearer ${{ secrets.MIXPEEK_API_KEY }}" \
      -F "manifest_file=@mixpeek.yaml"

Get started

Connect your data

Extract features

Build retrievers

Enrich & organize

Integrate & operate

Resources

Quick Start

Applying over resources that already exist

Core Operations

Resource Types

Extractors

Parameters

Budget

Alerts

Track an apply

Secret References

Seed Data

Acceptance

Dependency Resolution

Workflows

Environment Replication

Pre-deployment Validation

CI/CD Integration

References

​Quick Start

​Applying over resources that already exist

​Core Operations

​Resource Types

​Extractors

​Parameters

​Budget

​Alerts

​Track an apply

​Secret References

​Seed Data

​Acceptance

​Dependency Resolution

​Workflows

​Environment Replication

​Pre-deployment Validation

​CI/CD Integration

​References

Quick Start

Applying over resources that already exist

Core Operations

Resource Types

Extractors

Parameters

Budget

Alerts

Track an apply

Secret References

Seed Data

Acceptance

Dependency Resolution

Workflows

Environment Replication

Pre-deployment Validation

CI/CD Integration

References