Skip to main content

Overview

Branching an AI data environment used to mean one of two things: re-processing your entire corpus (slow, expensive) or experimenting directly in production (dangerous). Mixpeek solves this with a clone-based branching model that operates at every layer of the pipeline (namespace, collection, retriever, and taxonomy), so you can create isolated environments without re-ingesting your data and promote changes deliberately. This matters most when:
  • You want to test a new retrieval pipeline on production data without affecting live traffic
  • You need a staging namespace that mirrors production for QA without re-ingesting everything
  • You’re evaluating a new embedding model and want side-by-side comparison on the same corpus
  • A taxonomy schema is about to change and you need a safe place to validate it first

Branching Primitives

Mixpeek resources are immutable by design. You can’t change a collection’s feature extractor or a retriever’s pipeline stages via PATCH. This preserves execution history, dependent results, and audit trails. The branching mechanism is a first-class clone operation available on every resource type.

Namespace Clone (full environment branch)

The broadest primitive. A namespace clone deep-copies the entire environment: collections (including MVS vectors), retrievers, buckets, and optionally taxonomies, and remaps all internal IDs so nothing points back to production.
include_resources defaults to collections and retrievers on, taxonomies off. Set ttl_seconds (60 or more) to have the clone deleted automatically after that many seconds, which suits a short-lived test environment. The clone runs in the background, and its duration grows with the number of vectors and objects in the source. Poll GET /v1/namespaces/{namespace_id} on the new namespace: The clone response’s cloned_resources holds counts (collections, retrievers, taxonomies, buckets, objects, points) and is filled in once the clone is ready.
A namespace clone copies MVS vectors directly, so your data is normally not re-processed. After copying, the clone reads its own data back. If the copied vectors cannot be read or searched, the clone re-extracts the copied objects in the new namespace, which runs your extractors again and is billed as new processing. The clone is isolated: changes to collections, retrievers, or documents in staging have no effect on production.
What gets copied:

Clone modes

mode sets how the clone stores its data.
branch is in private beta. A request with "mode": "branch" returns 403 with error.details.error_code: "branch_mode_off" and error.details.available: ["copy"] unless branches are enabled for your deployment. A branch forks within the source’s organization only. To join the beta, contact support@mixpeek.com.

Follow the source

A branch can keep applying its source’s changes after the fork. Add follow to a clone with "mode": "branch". Each source change goes through a transform you declare, then lands in the branch. Use it for a staging namespace that tracks production, or a copy that keeps only some of the source’s documents.
follow is in private beta with branch. It needs branch mode enabled for your deployment and works within one organization. Contact support@mixpeek.com to join.
The transform runs on each change in this order:
  1. Remap (always): the source’s namespace, collection and bucket ids become the branch’s.
  2. filter (optional): keeps source points that match, in the shape retriever filters use. A point that stops matching is deleted from the branch.
  3. field_map (optional): {"from", "to"} renames and {"drop"} removals of your payload paths, applied in order. You cannot map _internal, document_id or collection_id.
"follow": {"transform": null} runs the remap only. Vectors are copied as written, and source deletes are applied to the branch. You cannot write to a following branch: a write returns 409 with write_to_following_branch. Check progress. GET /v1/namespaces/{namespace_id} returns a follow object on a following branch. The read comes from stored state and does not wake the namespace. Add ?include=follow_live to read it from the running namespace. A field the read cannot determine is null. Treat null as unknown, never as 0. A change the transform cannot apply fails the follow at that change and keeps applied_seq at the last good one. The follow never skips a change. error.remedy says what clears it: Stop following. Send PATCH /v1/namespaces/{namespace_id} with {"follow": {"state": "stopped"}}. The branch keeps its data at applied_seq and accepts writes again. A stopped follow cannot resume: asking returns 409 with follow_stopped_cannot_resume. Deleting the source also stops the follow, with stopped_reason: "source_deleted". Errors carry their code in error.details.error_code:

Collection Clone (swap extractor or source)

Clone a single collection and optionally override its feature extractor or source. This is the entry point for embedding model experimentation: run two extractors on the same corpus and compare retrieval quality.
When you clone a collection without changing the feature extractor, vectors are reused. When you change the extractor, you must trigger reprocessing, because vectors are model-specific and cannot be ported across embedding spaces.

Retriever Clone (pipeline experiment)

Immutable retriever stages mean the safe way to test a new ranking strategy, add a rerank stage, or adjust fusion weights is to clone the retriever with overrides.

Taxonomy Clone (schema version branch)

Taxonomies are immutable in their core config (retriever_id, input_mappings, enrichment_fields). Clone to branch a schema: swap the backing retriever, adjust the hierarchy, or test a new classification model.

Common Patterns

Pattern 1: Staging environment for a production namespace

The most common use case: a full mirror of production where QA teams can validate changes before go-live.
After cloning, engineers can modify the staging retriever pipeline, run evaluations, and only promote to prod once quality gates pass.

Pattern 2: Embedding model A/B test

Run two embedding models on the same corpus, then compare retrieval quality with Mixpeek’s Evaluations before committing.

Pattern 3: Retriever pipeline experiment

Test a new retrieval stage (reranker, MMR, query expansion) without touching the live retriever.

Pattern 4: Taxonomy version checkpoint

Before migrating an IAB or ICD taxonomy schema, snapshot the current version so you can validate in parallel.

Vertical Examples

Problem: Your IAB 2.2 taxonomy needs to be upgraded to IAB 3.0. You can’t migrate production mid-campaign without validating classification quality first.Solution:
  1. Clone the production namespace → ns_adtech_staging
  2. Clone tax_iab_v2 with the new retriever trained on IAB 3.0 → tax_iab_v3_candidate
  3. Apply tax_iab_v3_candidate to a sample of staging documents
  4. Validate label quality against ground truth
  5. Promote: update production taxonomy config once quality gates pass
No live campaigns are affected. Staging vectors are reused from production (no reprocessing).
Problem: You’re switching the ICD-10 classification retriever from GPT-4o to a fine-tuned clinical model. Patient record classification in production cannot be interrupted.Solution:
  1. Clone tax_icd10_prod with the new retriever → tax_icd10_clinical_v2
  2. Apply to a test collection of de-identified records
  3. Compare classification accuracy against the production taxonomy’s output on the same records
  4. Only swap production once the new model meets or exceeds current accuracy
Both taxonomies run in parallel. There’s no downtime and no disruption to the production classification pipeline.
Problem: Editorial wants to test a diversity-aware ranking algorithm (MMR) on the content search endpoint before rolling out to all users.Solution:
  1. Clone ret_content_search → ret_content_search_mmr
  2. Override stages to add an MMR sort step after semantic search
  3. Route 10% of internal QA traffic to the experimental retriever (traffic splitting handled in your application layer)
  4. Compare CTR, dwell time, and result diversity metrics
  5. Promote by cloning the staging retriever config back to production
The collection (and all its vectors) is shared between both retrievers, with no extra storage cost.
Problem: A new image embedding model shows better performance on architectural/interior photos. You want to validate before migrating 5M property images.Solution:
  1. Clone col_property_images with the new extractor → col_property_images_v2
  2. Trigger reprocessing (required for model changes, because vectors are model-specific)
  3. Create ret_property_search_v2 pointing to the new collection
  4. Run offline evaluation: compare top-5 retrieval precision on a labeled query set
  5. If metrics improve, migrate production: update ret_property_search to point to the new collection
The old collection stays active during migration as a fallback. Roll back is instant: repoint the retriever.

Promotion Workflow

Branching is only useful if you have a clear path back to production. The recommended flow:
Promoting a retriever experiment to production:
  1. Run Evaluations on the experimental retriever
  2. If metrics pass, delete the old production retriever (after confirming no dependent published pages)
  3. Rename the experimental retriever to the production name via PATCH (name is mutable)
  4. Or: update your application to point to the new retriever ID directly
Rolling back is always safe. Because the old retriever and collection still exist unchanged, you can revert by updating the retriever ID in your application config.

Best Practices

One namespace per environment (dev, staging, prod). Clone from prod to create staging rather than maintaining them separately. This guarantees staging always reflects current production data and config.
  • Clone, don’t modify. Resist the urge to patch production resources for “quick experiments.” A clone takes seconds and preserves the ability to roll back.
  • Retriever clones are free. They share the underlying collection (and all its vectors). You only pay for additional MVS storage when collection vectors diverge.
  • Trigger reprocessing only when the extractor changes. Cloning a collection with the same extractor reuses existing vectors and consumes no GPU time.
  • Use evaluations before promoting. The Evaluations API lets you run offline quality checks on any retriever before it touches production traffic.
  • Name branches consistently. A naming convention like {resource}_staging, {resource}_exp_{date}, or {resource}_v{n} makes it easy to identify which resources are active experiments vs. production.
  • Clean up stale branches. Delete experimental collections and retrievers after promotion or abandonment. MVS vectors from branched collections consume storage until deleted.