Why Fine-Tuning Quietly Breaks Your Embedding Index (and AlphaWiSE's Fix)
Summary
Fine-tune a CLIP-style model on new data and the cross-modal alignment that made old queries work shifts underneath your index; recall erodes with no dashboard warning. AlphaWiSE interpolates two frozen checkpoints with one learned coefficient per tensor, fitted on a small exemplar memory, and bakes out a single model with zero added inference cost. Also covers the operational fix: pin embedding-model versions per collection. Paper: arxiv.org/abs/2607.15094
About this video
Fine-tune a CLIP-style model on new data and the cross-modal alignment that made old queries work shifts underneath your index; recall erodes with no dashboard warning. AlphaWiSE interpolates two frozen checkpoints with one learned coefficient per tensor, fitted on a small exemplar memory, and bakes out a single model with zero added inference cost. Also covers the operational fix: pin embedding-model versions per collection. Paper: arxiv.org/abs/2607.15094