Rollout & Safety

Learned fusion changes how search results are ranked, per user and per query. That power requires operational controls. This guide covers how to roll out Auto-Tune safely: start with shadow mode, ramp traffic gradually, monitor, and know how to turn it off if something goes wrong.

Traffic Splitting

The rollout_pct field controls what percentage of requests use learned fusion weights. The rest fall back to static fusion (RRF by default).

{
  "learning_config": {
    "rollout_pct": 10.0
  }
}

Bucketing is deterministic by user ID: the same user always gets the same treatment (learned or static) on consecutive requests. This prevents a user from seeing different ranking behavior on every search.

rollout_pct: 0    → all requests use static fusion
rollout_pct: 10   → ~10% of users get learned fusion
rollout_pct: 50   → ~50% of users get learned fusion
rollout_pct: 100  → all requests use learned fusion (default)

Bucketing uses a hash of the user_id. If no user_id is provided in the query, the request always uses static fusion regardless of rollout_pct.

Changing the rollout

rollout_pct lives in the feature search stage’s learning_config, so you change it by updating the retriever. Send the retriever’s stages with that stage’s learning_config edited; the next request uses the new value, with no redeploy.

curl -X PATCH "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "stages": [
      {
        "stage_name": "search",
        "config": {
          "stage_id": "feature_search",
          "parameters": {
            "fusion": "learned",
            "learning_config": { "rollout_pct": 25 }
          }
        }
      }
    ]
  }'

Include the stage’s other parameters (its searches, limits and the rest of learning_config) as they are today: the stage is replaced by what you send. In the Studio, the Learning tab rollout slider makes this update for you.

Shadow Mode

Shadow mode computes learned fusion weights and logs what the results would have been, but serves static fusion results to the user. This lets you evaluate the quality of learned fusion before it affects real users.

{
  "learning_config": {
    "shadow_mode": true,
    "rollout_pct": 100.0
  }
}

In shadow mode:

Both fusion strategies execute in parallel
Static fusion results are served to the user
Learned fusion weights and re-ranked results are logged as shadow_execution metadata
The evaluation system can compare shadow vs. served results offline

The execution response includes a learned_fusion_context field (when shadow mode is active) so you can inspect what weights would have been used:

{
  "results": [ ... ],
  "learned_fusion_context": {
    "shadow_mode": true,
    "served_fusion": "rrf",
    "sampled_weights": {
      "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1": 0.68,
      "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding": 0.32
    },
    "context_level": "personal",
    "effective_exploration": 0.42,
    "feature_uris": [
      "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
      "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding"
    ]
  }
}

Run shadow mode for at least one week before enabling live traffic. Compare NDCG and click-through rates between shadow and served results using evaluations.

Turning Learned Fusion Off

Set rollout_pct to 0 with the same update. Every request then uses static fusion, and the rest of learning_config and the accumulated interaction history stay as they are, so setting the percentage back resumes where it left off.

The /v1/retrievers/{retriever_id}/learned-fusion/* endpoints (enable, disable, rollout, per-user opt-out, opt-in and reset, weights, stats, activity) have been removed and answer 410 Gone with the replacement in the error body. Rollout and shadow mode are set in learning_config; what served each search is in the execution history (below).

Seeing Which Weights Served a Search

Each execution records the learned-fusion arm that served it as learned_fusion_context: the gate it took (serve, shadow or skip), the context level (personal, demographic, global), the sampled weights and the user. It is in the execute response and stored on the execution’s profile:

curl "https://api.mixpeek.com/v1/retrievers/$RETRIEVER_ID/executions/$EXECUTION_ID/profile" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY" \
  -H "X-Namespace: $NAMESPACE_ID"

List recent executions with POST /v1/retrievers/{retriever_id}/executions/list and read each one’s profile to see weights per feature over time, or every execution served to one user_id.

Weight Bounds

Even with strong personalization data, individual feature weights are clamped to prevent degenerate rankings:

{
  "learning_config": {
    "min_weight": 0.05,
    "max_weight": 0.95
  }
}

Setting	Default	Effect
`min_weight`	`0.05`	No feature drops below 5% weight — it always contributes something to results
`max_weight`	`0.95`	No feature exceeds 95% weight — a single feature cannot completely dominate

After Thompson Sampling produces raw weights, they are clamped to [min_weight, max_weight] and re-normalized to sum to 1.0. This means even a user with extreme interaction patterns will always see results influenced by all configured features.

Circuit Breaker

If learned fusion weight resolution takes too long (slow signal reads, Redis issues, or high load), the system automatically falls back to static fusion for that individual request:

learned fusion resolution > circuit_breaker_timeout_ms (default 1000ms) → fall back to static fusion

The timeout is configurable via the circuit_breaker_timeout_ms field in learning_config (range 1–30000ms). The circuit breaker is per-request, not per-retriever. A single slow request does not disable learned fusion for everyone. The response’s learned_fusion_context metadata indicates when a fallback occurred:

{
  "learned_fusion_context": {
    "context_level": "none",
    "sampled_weights": { ... },
    "effective_exploration": 1.0,
    "circuit_breaker_triggered": true,
    "weight_resolution_ms": 1023.4
  }
}

A context_level of "none" with circuit_breaker_triggered: true indicates the circuit breaker fired and uniform priors were used. The weight_resolution_ms field shows the actual resolution time. Monitor the mxp_learned_fusion_circuit_breaker_total Prometheus metric to detect systemic latency issues.

Recommended Rollout Plan

Shadow mode (1-2 weeks)

Enable shadow_mode: true with rollout_pct: 100. All users get static results, but learned fusion runs in parallel and logs what it would have returned.Check: Run evaluations comparing shadow vs. served results. Learned fusion NDCG should be equal to or better than static.

1% canary (1 week)

Set shadow_mode: false, rollout_pct: 1.0. A small fraction of users get learned fusion.Check: Monitor click-through rates and circuit breaker triggers. No degradation vs. control group.

10% ramp (1 week)

Set rollout_pct: 10.0. Enough traffic to produce statistically significant comparisons.Check: Run a benchmark comparing learned vs. static. Expect learned fusion to show improvement for users with sufficient interaction history.

50% ramp (1 week)

Set rollout_pct: 50.0. Half of users get personalized results.Check: Monitor weight distributions across users. Look for convergence patterns (weights stabilizing) rather than oscillation.

100% GA

Set rollout_pct: 100.0. All users get personalized fusion weights, with hierarchical fallback for new users.Check: Keep evaluations running on a schedule to catch regressions. Set up alerts for high global fallback rates and circuit breaker storms.

At any step, if you see regression:

Set rollout_pct: 0 to serve static fusion to everyone
Investigate a specific user through the learned_fusion_context on their executions’ profiles
Check whether the issue is global (bad reward map) or user-specific (outlier behavior)
Adjust the learning_config and restart from the previous step

Auto-Tune — overview of the full feedback loop
Reward Signals — tuning how interactions influence weights
Evaluations — measuring retrieval quality

Get started

Connect your data

Extract features

Build retrievers

Enrich & organize

Integrate & operate

Resources

Rollout & Safety

Traffic Splitting

Changing the rollout

Shadow Mode

Turning Learned Fusion Off

Seeing Which Weights Served a Search

Weight Bounds

Circuit Breaker

Recommended Rollout Plan

​Traffic Splitting

​Changing the rollout

​Shadow Mode

​Turning Learned Fusion Off

​Seeing Which Weights Served a Search

​Weight Bounds

​Circuit Breaker

​Recommended Rollout Plan

​Related

Traffic Splitting

Changing the rollout

Shadow Mode

Turning Learned Fusion Off

Seeing Which Weights Served a Search

Weight Bounds

Circuit Breaker

Recommended Rollout Plan

Related