Feature URIs: Evolving Embeddings Without Migration

You finally built a working multimodal search system, but the moment user queries evolve, everything breaks.

The problem: Vector indexes are inherently stateful. When you need to upgrade an embedding model or change feature extractors, you're forced to re-index millions of documents. And if the new model performs worse? You have to migrate back. Again.

The solution: Feature URIs, a core abstraction for managing the lifecycle of embeddings, extractors, and indexes without painful migrations.

Why Search Systems Break

Imagine you build a search system that answers:

"Show me all the dogs that are prancing outside in the snow."

Perfect. It works. Your users love it.

Then a new user comes in and asks:

"How many dogs have all four legs on the ground, there's snow outside, and Beethoven is playing in the background?"

Your system fails. Why?

Because your embedding model wasn't trained for this level of detail. To support the new query, you need to:

1. Choose a different (or fine-tuned) embedding model
2. Re-extract and re-embed millions of videos
3. Re-index everything into a new vector database
4. Test retrieval performance
5. If it doesn't work, revert everything back

This is the embedding lifecycle problem, and Feature URIs solve it.

What is a Feature URI?

A Feature URI is a unique identifier that couples four critical components:

The 4 Components

1. Protocol: mixpeek://: identifies this as a Mixpeek resource
2. Feature Extractor: The algorithm that decomposes raw files (videos, images, audio, documents) into atomic units
3. Extractor Version: Enables versioning of the same extractor (v1, v2, etc.)
4. Embedding Field: The specific embedding model and inference endpoint

Example Feature URIs:

These URIs are critical because they:

• Version everything: extractors, models, and inference endpoints
• Enable A/B testing: compare v1 vs v2 without migration
• Support rollback: revert to previous versions instantly
• Tightly couple retrieval: query using the exact same configuration that indexed the data

Why Versioning Embeddings Matters

Let's say you have 1 million videos indexed with clip-vit-base-patch32.

You want to test a new fine-tuned model: clip-custom-v2.

Without Feature URIs:

Cost: Weeks of compute. Endless re-indexing. Production downtime.

With Feature URIs:

# Index with BOTH models simultaneously
collection.insert(
    file="video.mp4",
    feature_extractors=[
        "mixpeek://video-extractor/v1/clip-base",
        "mixpeek://video-extractor/v2/clip-custom"
    ]
)

# Query against v1
results_v1 = retriever.search(
    query="dog in snow",
    feature_uri="mixpeek://video-extractor/v1/clip-base"
)

# Query against v2
results_v2 = retriever.search(
    query="dog in snow",
    feature_uri="mixpeek://video-extractor/v2/clip-custom"
)

# Compare precision/recall
evaluate(results_v1, results_v2)

No re-indexing. No migration. No downtime.

Rolling Forward & Rolling Back Safely

Feature URIs enable zero-downtime upgrades:

# Step 1: Add new extractor version alongside old one
collection.add_extractor(
    "mixpeek://face-detector/v3/arc-face-r100"
)

# Step 2: A/B test in production
if experiment_group == "control":
    feature_uri = "mixpeek://face-detector/v2/face-embedding"
else:
    feature_uri = "mixpeek://face-detector/v3/arc-face-r100"

# Step 3: Roll forward if v3 performs better
collection.set_default_extractor(
    "mixpeek://face-detector/v3/arc-face-r100"
)

# Step 4: Or roll back instantly if v3 is worse
collection.set_default_extractor(
    "mixpeek://face-detector/v2/face-embedding"
)

The key insight: Both extractors coexist in the same collection. You're not replacing data: you're adding versions.

Feature URIs in a Real Collection

Let's look at a real example from the video:

# Collection: portrait-gallery
# Source: S3 bucket with 100,000 portrait images

collection = mixpeek.collections.create(
    name="gallery",
    feature_extractors=[
        {
            "uri": "mixpeek://image-extractor/v1/google-siglip-base",
            "apply_to": "image"
        }
    ]
)

This single URI tells us:

• Extractor: image-extractor (analyzes images at the pixel level)
• Version: v1 (first iteration of this extractor)
• Embedding model: google-siglip-base (the SigLIP vision encoder)
• Inference endpoint: Managed by Mixpeek (no infrastructure to maintain)

When you query this collection:

results = retriever.search(
    collection="gallery",
    query="dog",
    feature_uri="mixpeek://image-extractor/v1/google-siglip-base"
)

Mixpeek automatically:

1. Encodes "dog" using the same google-siglip-base model
2. Queries the image-extractor/v1 index
3. Returns K-nearest neighbors using the exact same vector space

No guesswork. No mismatched embeddings. Everything is coupled.

Feature Search with URIs

Feature search in Mixpeek uses Feature URIs to ensure retrieval consistency:

retriever = mixpeek.retrievers.create(
    name="test-retriever",
    collection="gallery",
    stages=[
        {
            "type": "feature_search",
            "config": {
                "input": "{{ query }}",  # Jinja template from input schema
                "feature_uri": "mixpeek://image-extractor/v1/google-siglip-base",
                "top_k": 10
            }
        }
    ]
)

# Execute retrieval
results = retriever.run(inputs={"query": "dog"})

What happens under the hood:

1. Input encoding: "dog" is encoded using google-siglip-base
2. Index routing: Query is routed to the image-extractor/v1 index
3. Vector search: HNSW algorithm finds 10 nearest neighbors
4. Result assembly: Mixpeek returns the original documents + scores

The Feature URI ensures that:

• The query uses the same embedding model as the indexed data
• The query targets the correct index (not a different version)
• The inference endpoint is consistent (model version, quantization, hardware)

Migrating Extractors Without Re-Indexing

Here's the power move: namespace migration.

# Source namespace: 1M videos with old extractor
source = {
    "collection": "video-archive",
    "feature_uri": "mixpeek://video-extractor/v1/clip-base"
}

# Target namespace: new extractor
target = {
    "collection": "video-archive-v2",
    "feature_uri": "mixpeek://video-extractor/v2/clip-custom"
}

# Migrate everything
migration = mixpeek.migrations.create(
    source=source,
    target=target,
    strategy="async"  # Background job
)

# Monitor progress
print(migration.status())  # "Processing: 342,891 / 1,000,000"

What happens:

1. Mixpeek reads from the v1 namespace (no re-downloading files)
2. Re-extracts features using the v2 extractor
3. Writes to a new namespace with the new Feature URI
4. Both namespaces coexist: you can query either one

Result: You can A/B test v1 vs v2 in production without touching your original index.

A/B Testing Retrieval Performance

Now you have two namespaces. Let's measure which one performs better:

# Test queries
test_queries = [
    "dog with all four legs on the ground",
    "snow falling during sunset",
    "beethoven playing in the background"
]

# Ground truth (labeled relevant documents)
ground_truth = load_ground_truth()

# Evaluate v1
results_v1 = evaluate_retriever(
    retriever="v1-retriever",
    queries=test_queries,
    ground_truth=ground_truth
)

# Evaluate v2
results_v2 = evaluate_retriever(
    retriever="v2-retriever",
    queries=test_queries,
    ground_truth=ground_truth
)

# Compare metrics
print(f"V1 Precision@10: {results_v1['precision']:.3f}")
print(f"V2 Precision@10: {results_v2['precision']:.3f}")

print(f"V1 Recall@10: {results_v1['recall']:.3f}")
print(f"V2 Recall@10: {results_v2['recall']:.3f}")

Example output:

If v2 wins, you promote it:

# Promote v2 to production
collection.set_default_feature_uri(
    "mixpeek://video-extractor/v2/clip-custom"
)

# Archive v1 (or keep for rollback)
collection.archive_feature_uri(
    "mixpeek://video-extractor/v1/clip-base"
)

If v2 loses, you delete it and keep v1. No migration pain.

Hybrid Search with Multiple Feature URIs

You can combine different embedding models in a single query:

retriever = mixpeek.retrievers.create(
    name="hybrid-retriever",
    collection="documents",
    stages=[
        # Stage 1: Dense semantic search
        {
            "type": "feature_search",
            "config": {
                "feature_uri": "mixpeek://text-extractor/v1/colbert-v2",
                "input": "{{ query }}",
                "top_k": 20,
                "weight": 0.7
            }
        },
        # Stage 2: Sparse keyword search
        {
            "type": "feature_search",
            "config": {
                "feature_uri": "mixpeek://text-extractor/v1/splade",
                "input": "{{ query }}",
                "top_k": 20,
                "weight": 0.3
            }
        },
        # Stage 3: Fusion (combine results)
        {
            "type": "reciprocal_rank_fusion",
            "config": {
                "k": 60
            }
        }
    ]
)

Result: You get the best of both worlds, semantic understanding from ColBERT and keyword precision from SPLADE.

Each Feature URI ensures:

• The correct embedding model is used
• The correct index is queried
• The correct inference endpoint is called

Real-World Use Cases

1. Fine-Tuning Safety

You fine-tune an embedding model for your domain:

# Production: safe fallback
prod_uri = "mixpeek://text-extractor/v1/sentence-transformers"

# Experiment: fine-tuned model
experiment_uri = "mixpeek://text-extractor/v2/custom-finetuned"

# A/B test with 10% traffic
if random.random() < 0.10:
    results = retriever.search(feature_uri=experiment_uri)
else:
    results = retriever.search(feature_uri=prod_uri)

If the experiment fails, you haven't touched production.

2. Cost Optimization

Test a cheaper embedding model:

# Expensive: OpenAI ada-002
expensive_uri = "mixpeek://text-extractor/v1/openai-ada-002"

# Cheap: Open-source model
cheap_uri = "mixpeek://text-extractor/v2/sentence-transformers"

# Measure quality vs cost
if cheap_recall > 0.95 * expensive_recall:
    print("Switch to cheap model → save $10k/month")

3. Regulatory Compliance

Keep old versions for audit trails:

# Production: latest model
current_uri = "mixpeek://face-detector/v5/arc-face"

# Compliance: frozen model from Jan 2024
audit_uri = "mixpeek://face-detector/v3/arc-face-frozen"

# Regulators can verify decisions using the exact model version
historical_results = retriever.search(
    feature_uri=audit_uri,
    query=compliance_query,
    as_of="2024-01-15"  # Time-travel query
)

The Architecture Behind Feature URIs

Under the hood, Mixpeek:

1. Stores Feature URIs as metadata on every indexed vector
2. Routes queries to the correct index based on the URI
3. Manages inference endpoints for each embedding model
4. Tracks provenance: you always know which model produced which embedding
5. Enables time-travel: query historical versions of your index

Example database record:

{
  "document_id": "video-12345",
  "timestamp": "2025-12-29T10:00:00Z",
  "feature_uri": "mixpeek://video-extractor/v2/clip-custom",
  "embedding": [0.23, -0.45, 0.78, ...],
  "metadata": {
    "file": "s3://bucket/video-12345.mp4",
    "extractor_version": "v2",
    "model_name": "clip-custom",
    "inference_endpoint": "gpu-cluster-3"
  }
}

When you query:

results = retriever.search(
    query="dog in snow",
    feature_uri="mixpeek://video-extractor/v2/clip-custom"
)

Mixpeek:

1. Filters for documents with matching feature_uri
2. Encodes the query using clip-custom on gpu-cluster-3
3. Queries the v2 index (not v1)
4. Returns results with full provenance

Key Takeaways

1. Vector indexes are stateful, changing embedding models requires re-indexing
2. Feature URIs version everything: extractors, models, versions, endpoints
3. A/B testing without migration: run multiple models side-by-side
4. Roll forward and roll back safely: promote or revert with a single API call
5. Hybrid search made simple: combine multiple embedding models with different URIs
6. Provenance tracking: always know which model produced which results

What's Next?

Now that you understand Feature URIs, you're ready to:

• Build evaluation pipelines: measure precision, recall, and NDCG across model versions
• Implement re-ranking: combine multiple retrieval stages with learned re-rankers
• Explore clustering: group similar content using different embedding models

Resources

• Mixpeek Feature URIs Documentation
• Feature Extractors Guide
• A/B Testing Retrieval Systems
• Embedding Lifecycle Management

Try It Yourself

1. Create two collections with different Feature URIs
2. Index the same 1,000 documents into both collections
3. Run the same 50 queries against both and compare results
4. Measure precision@10 for each, which model wins?

Discussion Questions

• When would you choose to version an extractor vs an embedding model?
• How would you handle a scenario where v2 performs better on new queries but worse on old ones?
• What metrics would you track to decide when to promote a new embedding model to production?