NEWVectors or files. Pick a path.Start →
    Model data
    Template v0.1.0 · updated 2026-10-29

    AI data curation

    Find the data your model is missing. Turn model failures into training and evaluation datasets with multimodal search and sampling.

    Upload your samples with your model's predictions and the ground truth you have. Mixpeek finds more examples like the failures, ranks the confident misses, drops near-duplicates and balances the pick across labels, so the selection is ready for your labeling and training tools. Mixpeek finds and selects; labeling, training and scoring stay in your own tools.

    Deploy starter pipeline

    AI labs, robotics and autonomous-vehicle teams, and data and evaluation vendors who know where their model fails and need more examples like the failures, picked without near-duplicates.

    What deploys today9 ready1 need your input

    No marketplace starter set ships with this template: it applies empty and reads Your own samples with your model's predictions, so connect your own data before querying it.

    A London street in daylight seen through a rain-covered window, with a red bus and cars with their brake lights on
    The model called this rainy London street night, at 0.84 confidence. It is daytime, and hard-examples ranks it first because it is the most confident miss in the set.Wikimedia Commons (CC0) / Anjana Menon
    40 in 248 s
    samples indexed after apply
    Proof run on production, 2026-10-29: apply created the 7 resources and the seed batch in 27.5 s, and the samples collection reached 40 documents at 247.5 s.
    5 of 5
    night scenes in failure-discovery's top 5
    "a dark city street at night" returned only night samples, all of them low_light misses. 23.9 s on the first, cold call.
    16
    misses ranked by confidence, 0.84 down to 0.52
    hard-examples returned every wrong prediction in the seed set, most confident first, in 0.7 s. The rainy street the model called night ranks first.
    6 / 6
    day and night in the 12-sample curated pick
    curate with "a city street" returned 12 results balanced across both labels, in 1.4 s. The whole proof run cost 880 credits (0.88 USD).

    What deploys today

    samplesCollectionDeploys ready
    failure-discoveryRetrieverDeploys ready
    similar-to-failureRetrieverDeploys ready
    hard-examplesRetrieverDeploys ready
    curateRetrieverDeploys ready
    attribute_filterRetriever stageDeploys ready
    sort_attributeRetriever stageDeploys ready
    mmrRetriever stageDeploys ready
    sampleRetriever stageDeploys ready
    samplesData sourceNeeds your input
    What each part needs

    The states are read from the manifest, so a part it ships commented out never shows as ready.

    samples
    Deploys and processes documents as they arrive.
    failure-discovery
    Deploys and answers queries once its collections hold documents.
    similar-to-failure
    Deploys and answers queries once its collections hold documents.
    hard-examples
    Deploys and answers queries once its collections hold documents.
    curate
    Deploys and answers queries once its collections hold documents.
    attribute_filter
    The one-click manifest includes this stage.
    sort_attribute
    The one-click manifest includes this stage.
    mmr
    The one-click manifest includes this stage.
    sample
    The one-click manifest includes this stage.
    samples
    Create the s3 credentials as manifest secrets, then uncomment the sync. It ships commented out because apply tests a connection as it creates it.

    What it looks like

    Pick a frame or a search and see what fires.

    Simulated walkthrough · illustrative frames, scripted decisions
    A London street in daylight seen through a rain-covered window, with a red bus and cars with their brake lights onreview
    samples -> hard-examplesStill: Wikimedia Commons (CC0) / Anjana Menonframe 1 of 1
    Detections on this frame
    Decision path
    1. samples -> hard-examples
    2. overcast sky, read as night
    3. brake lights in daylight
    4. route: review
    label day, predicted night at 0.84: the most confident miss
    Your call on this frame
    Running tally
    0
    cleared
    1
    review
    0
    excluded
    What a person's call does here
    The selection itself, recorded on each document as curation_versions with a reason, is what a release exports.
    Try a search
    Pick a search to see which retriever answers it and whether it works on a one-click deploy.
    Where the decisions fire
    Illustrative frames; the boxes are authored to show the decision path
    A London street in daylight seen through a rain-covered window, with a red bus and cars with their brake lights onovercast sky, read as nightbrake lights in daylightreview
    samples -> hard-examples · The model called this rainy London street night, at 0.84 confidence. It is daytime, and hard-examples ranks it first because it is the most confident miss in the set.Still: Wikimedia Commons (CC0) / Anjana Menon
    What goes in
    Connect your data to get started.
    samples
    One image per sample, with your own label, prediction, confidence, correct, error_category and model_version fields. Only the image is required; every field you send passes through at the path you gave it.
    What comes out
    Outputs from this template.
    samples
    Needs your input
    One document per sample with a multimodal embedding of the image and your prediction fields carried through, so the same index answers a sentence, a photo of a failure, and a filter on correct = false.
    a dataset release
    Documented, not deployed: record the selection on each document (curation_versions), then export the collection filtered on that version as Parquet or WebDataset, with lineage and a release manifest giving every file's size and SHA-256, into your bucket.

    What you would ask it

    Example searches this namespace answers once it is applied. Each one names the retriever that serves it.

    • “a dark city street at night”

      failure-discovery

      on the seed set, the top 5 are all night scenes the model called day

    • “every sample the model got wrong”

      hard-examples

      filtered on correct = false, sorted by confidence, highest first

    • “a city street”

      curate

      12 kept, near-duplicates dropped, at least 4 per label

    How the namespace is wired

    The diagram shows 1 buckets, 1 collections, 0 clean views and 4 retrievers. The manifest below applies 1 buckets, 1 collections (clean views included) and 4 retrievers today; the other parts are commented out in it, each with the reason. The diagram generates the manifest, so they cannot drift apart.

    SourceBucketCollectionClean viewRetrieverClusterConnectionBucket syncAlertTriggerClick a node to inspect it
    syncsearchsearchsearchsearch

    Reward signals

    How reviewer decisions move the thresholds

    Thresholds at ingest drift as the corpus changes. The reviewers working the queue are the ones who see where a threshold is wrong first, so this template routes their decisions back into the model that set it.

    DocsExport a collection
    The loop runs through your tools: you send samples with predictions, Mixpeek selects, your labeling tool labels the selection, your training job uses it, and the next model's predictions come back as new samples.
    Explicit signals

    The selection itself, recorded on each document as curation_versions with a reason, is what a release exports.

    Implicit signals

    None from the template. Mixpeek records retriever executions server-side, which is telemetry about queries rather than feedback about results.

    Where they land
    samplesmxp_retriever_executions

    System collections in your namespace, on the same vector store as the rest of the template. They are yours to query.

    How the loop closes

    Each model version's predictions arrive with model_version on them, so the next round of failure-discovery runs against what the newest model still gets wrong.

    DocsManifest schema

    One file spins up the namespace. Generated from the diagram above. Also served at /templates/ai-data-curation.namespace.yaml.

    # ai-data-curation: one manifest spins up the namespace.
# Platform manifest schema (GET /v1/discovery/schema). Validate with POST /v1/manifest/validate,
# apply with POST /v1/manifest/apply or the Deploy button. Wiring comes from the flow diagram:
# edges are bucket -> collection sources, collection -> retriever scope, retriever -> view.
# Applying a SECOND time, over a namespace this template already created: use
# POST /v1/manifest/apply?mode=create_missing, which creates what is missing and leaves
# what exists alone. The default, create_only, fails the WHOLE apply and rolls it back if
# any resource already exists, so an upgrade looks like a dead end without this. Use
# mode=upsert to also patch resources that exist but have drifted from this file.
version: '1.0'
metadata:
  name: ai-data-curation
  description: "Namespace template ai-data-curation. Generated from the flow diagram on mixpeek.com/templates/ai-data-curation."
namespaces:
  - name: ai-data-curation
    description: "Everything below lives in this namespace."
    feature_extractors:
      - name: multimodal_extractor
        version: v1
    # Daily spend budget, in dollars of what you are charged (1,000 credits a dollar).
    # New batches pause once the namespace has spent this much today; queries are not capped.
    # Raise it later on the namespace: PATCH /v1/namespaces/<id> {"spend_budget": {"daily_usd": N}}.
    budget:
      daily_usd: 2

# Data sources. A storage connection carries credentials, so it is created in Studio
# (or POST /v1/organizations/storage-connections) and synced into the bucket named here.
#   samples: s3, continuous, your samples with your model's predictions, in your account -> bucket samples
buckets:
  - name: samples
    namespace: ai-data-curation
    description: "Meant to be fed by samples (s3, continuous). No source is connected yet: applying this manifest creates the bucket only. Connect the source to this bucket in Studio (Syncs) to start the feed."
    schema:
      properties:
        image:
          type: image
          required: true
        label:
          type: string
        prediction:
          type: string
        confidence:
          type: number
        correct:
          type: boolean
        error_category:
          type: string
        model_version:
          type: string

# Connect your own storage. Create the two secrets, uncomment, and apply again;
# apply live-tests the connection, so it must have real credentials to succeed.
# Until then the buckets above accept direct uploads.
# storage_connections:
#   - name: samples-connection
#     provider: s3
#     description: "Read-only access to the s3 location holding your samples with your model's predictions, in your account."
#     config:
#       region: us-east-1
#       credentials:
#         type: access_key
#         access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
#         secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

# bucket_syncs:
#   - name: samples-sync
#     bucket: samples
#     connection: samples-connection
#     source_path: "samples/"
#     sync_mode: continuous
#     polling_interval_seconds: 300
#     skip_duplicates: true
#     file_filters:
#       include_patterns: ["*.jpg", "*.jpeg", "*.png", "*.webp"]
#     schema_mapping:
#       mappings:
#         image:
#           target_type: blob
#           source: {type: file}
#           blob_type: image

collections:
  - name: samples
    namespace: ai-data-curation
    description: "multimodal_extractor@v1 over bucket samples. Feeds failure-discovery, similar-to-failure, hard-examples, curate."
    source:
      type: bucket
      bucket: samples
    feature_extractor:
      name: multimodal_extractor
      version: v1
      input_mappings:
        image: image
      field_passthrough:
        - source_path: label
        - source_path: prediction
        - source_path: confidence
        - source_path: correct
        - source_path: error_category
        - source_path: model_version
    enabled: true
retrievers:
  - name: failure-discovery
    namespace: ai-data-curation
    description: "Describe a failure in words and get back every sample that looks like it, labeled or not, so one known miss turns into the set of samples like it."
    collections:
      - samples
    input_schema:
      query:
        type: text
        required: true
        description: "What to look for; searched across every index below"
    stages:
      - stage_name: search
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
              query:
                input_mode: text
                value: "{{INPUT.query}}"
              top_k: 50
          fusion: rrf
          final_top_k: 50
    tags:
      - template:ai-data-curation
  - name: similar-to-failure
    namespace: ai-data-curation
    description: "Pass the URL of an image the model got wrong and get back the samples closest to it."
    collections:
      - samples
    input_schema:
      query:
        type: text
        required: true
        description: "URL of the image or frame to match; encoded by the index below and compared against it"
    stages:
      - stage_name: search
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
              query:
                input_mode: content
                value: "{{INPUT.query}}"
              top_k: 50
          fusion: rrf
          final_top_k: 50
    tags:
      - template:ai-data-curation
  - name: hard-examples
    namespace: ai-data-curation
    description: "Every sample the model got wrong, most confident first."
    collections:
      - samples
    input_schema:
      limit:
        type: text
        required: false
        description: "Optional cap on documents grouped per run"
    stages:
      - stage_name: verdict_filter
        config:
          stage_id: attribute_filter
          parameters:
            conditions:
              AND:
                - field: correct
                  operator: eq
                  value: false
      - stage_name: most_confident_first
        stage_id: sort_attribute
        parameters:
          field: "confidence"
          direction: "desc"
    tags:
      - template:ai-data-curation
  - name: curate
    namespace: ai-data-curation
    description: "Search for what a dataset should cover, then keep a diverse, balanced pick: near-duplicates dropped and every label represented."
    collections:
      - samples
    input_schema:
      query:
        type: text
        required: true
        description: "What to look for; searched across every index below"
    stages:
      - stage_name: search
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
              query:
                input_mode: text
                value: "{{INPUT.query}}"
              top_k: 200
          fusion: rrf
          final_top_k: 200
      - stage_name: drop_near_duplicates
        stage_id: mmr
        parameters:
          lambda: 0.5
          top_k: 100
          diversity_feature_uri: "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
      - stage_name: balance_labels
        stage_id: sample
        parameters:
          strategy: "stratified"
          stratify_by: "label"
          count: 12
          min_per_stratum: 4
          seed: 7
    tags:
      - template:ai-data-curation

# The curation sample set: 20 daytime and 20 night street photos (CC0, from
# Wikimedia Commons), each with the label it truly has and an illustrative
# prediction from a day-or-night scene model. The model gets 16 wrong, 14 of
# them at night, so the failures share a condition the retrievers can find.
seed:
  - bucket: samples
    max_objects: 40
    objects:
      - {blobs: {image: samples/curation/images/day/steam-in-new-york-street.jpg}, label: day, prediction: day, confidence: 0.97, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/doorman-on-leadenhall-street.jpg}, label: day, prediction: day, confidence: 0.95, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/muscle-cars-at-rundle-street.jpg}, label: day, prediction: day, confidence: 0.93, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/driving-outside-flinders-street-railway-station.jpg}, label: day, prediction: day, confidence: 0.91, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/first-street.jpg}, label: day, prediction: day, confidence: 0.89, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/neighborhood-street.jpg}, label: day, prediction: day, confidence: 0.97, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/street-view-urban-street.jpg}, label: day, prediction: day, confidence: 0.95, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/street.jpg}, label: day, prediction: day, confidence: 0.93, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/street-curve.jpg}, label: day, prediction: day, confidence: 0.91, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/bleak-city-street.jpg}, label: day, prediction: day, confidence: 0.89, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/shadowed-street-london.jpg}, label: day, prediction: night, confidence: 0.71, correct: false, error_category: deep_shadow, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/street-corner-japan.jpg}, label: day, prediction: day, confidence: 0.95, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/crossing-the-street.jpg}, label: day, prediction: day, confidence: 0.93, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/people-crossing-street.jpg}, label: day, prediction: day, confidence: 0.91, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/steep-city-street.jpg}, label: day, prediction: day, confidence: 0.89, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/london-street-in-the-rain.jpg}, label: day, prediction: night, confidence: 0.84, correct: false, error_category: overcast, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/cobblestone-street.jpg}, label: day, prediction: day, confidence: 0.95, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/aerial-view-of-smolenskaya-street-moscow.jpg}, label: day, prediction: day, confidence: 0.93, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/new-york-street-fashion.jpg}, label: day, prediction: day, confidence: 0.91, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/day/budapest-crossroads.jpg}, label: day, prediction: day, confidence: 0.89, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/toronto-harbour-street-at-night-in-2017-sean-pol.jpg}, label: night, prediction: day, confidence: 0.52, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/hong-kong-night.jpg}, label: night, prediction: night, confidence: 0.87, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/cityscape-and-traffic-light-trails.jpg}, label: night, prediction: day, confidence: 0.56, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/flatiron-at-night.jpg}, label: night, prediction: night, confidence: 0.81, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/speed-by-night.jpg}, label: night, prediction: day, confidence: 0.6, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/man-on-a-bicycle-at-night.jpg}, label: night, prediction: day, confidence: 0.64, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/inside-a-car-at-night-time-in-williamsburg.jpg}, label: night, prediction: day, confidence: 0.68, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/downtown-los-angeles-at-night.jpg}, label: night, prediction: night, confidence: 0.81, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/driving-through-a-bridge-at-night.jpg}, label: night, prediction: day, confidence: 0.72, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/car-light-trails-at-night.jpg}, label: night, prediction: day, confidence: 0.76, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/dim-urban-street.jpg}, label: night, prediction: day, confidence: 0.52, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/boston-firetruck-parked-on-a-street-at-night.jpg}, label: night, prediction: day, confidence: 0.56, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/cobblestone-street-at-night.jpg}, label: night, prediction: day, confidence: 0.6, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/colorful-urban-traffic-at-night.jpg}, label: night, prediction: night, confidence: 0.87, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/new-york-city-center-street-at-night-time.jpg}, label: night, prediction: night, confidence: 0.84, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/on-an-empty-road-at-night.jpg}, label: night, prediction: day, confidence: 0.64, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/highway-intersection-at-night-time.jpg}, label: night, prediction: day, confidence: 0.68, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/jette-highway-traffic-at-night.jpg}, label: night, prediction: day, confidence: 0.72, correct: false, error_category: low_light, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/neon-streets.jpg}, label: night, prediction: night, confidence: 0.84, correct: true, model_version: scene-lighting-v1}
      - {blobs: {image: samples/curation/images/night/chicago-nighttime.jpg}, label: night, prediction: day, confidence: 0.76, correct: false, error_category: low_light, model_version: scene-lighting-v1}
# Measured on production 2026-10-29: the seed and these four queries cost
# 880 credits ($0.88), nearly all of it the 40 image embeddings.
acceptance:
  max_cost_usd: 1.25
  steps:
    - name: samples-indexed
      kind: documents
      collection: samples
      expect: {min_documents: 40}
    - name: failure-discovery-finds-night
      kind: query
      retriever: failure-discovery
      inputs: {query: a dark city street at night}
      expect: {top_field: label, top_equals: night}
    - name: hard-examples-ranks-the-most-confident-miss
      kind: query
      retriever: hard-examples
      expect: {top_field: error_category, top_equals: overcast}
    - name: curate-returns-a-stratified-pick
      kind: query
      retriever: curate
      inputs: {query: a city street}
      expect: {min_results: 8}
    Start building with Mixpeek

    Deploy this template, bring your data, and go from exploration to production.

    Get startedRequest a demo