NEWVectors or files. Pick a path.Start →
    Data Infrastructure
    16 min read
    Updated 2026-07-28

    How Do You Run Your Own Model Inside a Managed Search Pipeline?

    The four places a managed retrieval platform can let you run your own code (ingest-time extraction, query-time inference, reranking, and enrichment), what each one demands of the platform, and why the ingest and query sides must load the identical model or your vectors and your queries end up in different spaces. Covers the packaging contract, the cold-start and GPU-allocation problems that decide whether query-time custom inference is usable, versioning against an already-indexed corpus, and how to tell a real extension point from a webhook with good marketing.

    Custom Models
    Extractors
    Architecture
    Retrieval
    MLOps

    The Short Answer



    A managed search platform can let you run your own model in four distinct places, and they are not interchangeable:

    At ingest time (custom extraction). Your code runs on every object as it is indexed, producing embeddings or structured attributes. This is the most common extension point and the easiest to support, because it is batch work with no latency budget.

    At query time (custom inference). Your code runs inside the request path, usually to embed the incoming query with the same model you indexed with. This is the hardest to support well, because it puts your container on the critical path with a latency budget measured in milliseconds.

    At rerank time. Your model reorders an already-retrieved candidate set. Latency-sensitive but bounded, because you see tens of candidates rather than the whole corpus.

    At enrichment time. Your model runs after retrieval to annotate or transform results, off the critical path.

    The distinction that actually matters: a platform that supports only ingest-time extraction cannot support a custom embedding model. If you embed documents with your fine-tuned encoder but the platform embeds queries with its own, your query vector and your document vectors live in different spaces and the similarity scores are meaningless. Custom embedding requires ingest and query time together, running the identical model artifact. That single requirement disqualifies most "bring your own model" claims.

    Why This Is the Question People Actually Ask



    Teams evaluating a managed platform almost always arrive with a model they already trust: a fine-tuned CLIP for their product photography, a domain encoder trained on their own legal corpus, a classifier that encodes a policy no general model knows. The question is never "can this platform embed things." It is "can it embed things *my way*, and what do I give up to get that."

    The honest answer depends on architecture, not on a feature checkbox, so this guide is about what to look for.

    The Four Extension Points, and What Each Costs the Platform



    Extension pointRuns whenLatency budgetWhat the platform must provideTypical availability
    Ingest-time extractionDuring indexingMinutes; batchScheduling, GPU allocation, retries, checkpointingCommon
    Query-time inferenceInside the search requestTens of millisecondsWarm containers, autoscaling, cold-start avoidanceRare
    RerankingAfter candidate retrieval~100ms on tens of itemsSame as query-time, smaller batchUncommon
    Post-retrieval enrichmentAfter results are returnedSeconds; asyncOrdinary async executionCommon
    Read the table as a difficulty gradient. Ingest-time extraction is a scheduling problem the platform already solved for its own extractors. Query-time inference is a serving problem: your container has to be resident and warm, which means the platform is paying for idle GPU on your behalf, which is why it is usually gated behind dedicated infrastructure rather than offered on a shared multi-tenant API.

    The Trap: Ingest and Query Must Load the Same Model



    This is the failure that costs a full re-index, so it is worth being precise.

    Retrieval works because the query vector and the document vectors are produced by the same function. Cosine similarity between a vector from model A and a vector from model B is a real number, and it means nothing. There is no error, no exception, no warning. Recall simply collapses to something slightly better than random, and because the system returns *plausible-looking* results ranked in a confident order, it can take a long time to notice.

    So when you evaluate a platform's custom-model support, the question to ask is not "can I bring my own embedding model." It is: "when I issue a text query against an index built with my model, what embeds the query?" If the answer is anything other than "the same artifact you uploaded," custom embedding is not supported no matter what the marketing page says.

    A platform that gets this right exposes the same extractor in both places, and the query-time path is an explicit stage in the retriever rather than an implicit default. See Multi-Stage Retrieval for how those stages compose, and Late Interaction Retrieval for what changes when your model emits many vectors per document instead of one.

    The Packaging Contract



    Every platform that runs your code defines a contract. The shape is fairly consistent across vendors:

  1. An entry point for batch work that receives an object and returns named outputs.
  2. An entry point for real-time work that receives a payload and returns a result over HTTP.
  3. A declared compute profile (CPU, GPU class, memory) so the scheduler can place you and the biller can price you.
  4. A dependency manifest pinned tightly enough to be reproducible.
  5. An output schema naming each vector or attribute you emit.


  6. The output schema is the part people underestimate. Your extractor usually produces more than one thing (an embedding, plus a few attributes you want to filter on), and each output needs a stable name because retrievers, filters, taxonomies and clusters all reference it by that name. Rename an output and you have silently broken every retriever pointing at it.

    Versioning Against a Corpus That Already Exists



    Once your extractor has written vectors, its version is part of your data, not just your code.

    Ship v2 with a different architecture or dimension and you now have two incompatible vector populations in one index. Three ways out, in increasing order of cost:

    1. Additive outputs. v2 emits a new named output alongside the old one. Nothing breaks, storage grows, and you cut over retrievers when you are ready. 2. Shadow index. Build the v2 space separately, evaluate it against the v1 space on real queries, then swap. This is the safe default and it is what evaluating multimodal retrieval is for. 3. Full re-extraction. Correct, and it means paying GPU cost for your entire corpus again. Budget it explicitly; see what it costs to make a media library searchable.

    If the change is only the embedding model and not the upstream feature extraction, you may not need to re-run the expensive part at all. Migrating embedding models without re-embedding covers when that is possible.

    Cold Starts Decide Whether Query-Time Inference Is Usable



    A custom model in the query path is only as good as its warm-start behavior. A 2 GB checkpoint that loads in eight seconds is fine for batch and unusable for search, so the platform has to keep at least one replica resident. That has three consequences worth checking before you commit:

  7. You are paying for idle. Somebody is, anyway, and it will show up in your bill as a floor rather than a per-query cost.
  8. Autoscaling is bounded below by one. Scale-to-zero and interactive search are incompatible.
  9. Deploys are rollouts, not restarts. Replacing a resident model without dropping queries requires the platform to run both versions briefly.


  10. Ask what the p99 looks like immediately after a deploy, not in steady state. That is where the honest answer lives.

    How This Works on Mixpeek



    Mixpeek runs custom extractors on the same Ray cluster as the built-in ones. A single extractor archive supplies both sides: a batch entry point used during decomposition, and a realtime.py that exposes an HTTP endpoint retriever stages call at query time. Because it is one artifact, the feature_search stage embeds the query with exactly the model that embedded the corpus, which is the trap described above closed by construction.

    Outputs land in the vector store and MongoDB under the same feature URI scheme as built-in pipelines (mixpeek://[email protected]/my_embedding), so retrievers, taxonomies and clusters reference custom outputs the same way they reference native ones. Pricing derives from the compute profile your extractor declares, and you can quote it before running.

    The limitation, stated plainly: the upload, deploy and real-time inference lifecycle runs on dedicated infrastructure. It is *not* exposed on the shared public API, where those endpoints return 404 or 405 by design. On the shared API you can discover extractors, and you can author, lint and test locally with no API key. To get an extractor into production you either provision a dedicated deployment or submit it for review to be merged into the built-in catalog, which is the right path when the extractor is not proprietary.

  11. Custom extractors covers the authoring model and what plugs in where.
  12. Custom Extractor API is the dedicated-infrastructure HTTP contract.
  13. Custom extractor quickstart is the shortest path to a working local extractor.


  14. What To Ask Before You Commit



    1. When I query an index built with my model, what embeds the query? (The disqualifying question.) 2. Is query-time custom inference on the shared plan, or does it require dedicated infrastructure? 3. What happens to already-indexed vectors when I ship a new version of my extractor? 4. Can one extractor emit several named outputs, and can filters and taxonomies reference them? 5. What is p99 latency right after a deploy? 6. If I stop paying, can I get my vectors out, or only my source files?

    Question 6 is the one people skip. Custom models make your index harder to reproduce elsewhere, because reproducing it requires both your model artifact and the platform's chunking and preprocessing behavior. Running the store on object storage you already own is the usual hedge.

    Related Reading



  15. Multi-stage retrieval for where a custom stage sits in a pipeline
  16. Late interaction retrieval for multi-vector custom models
  17. Ingesting millions of files for what custom extraction costs at scale
  18. Tenant isolation if your custom model runs per customer
  19. Best multimodal embedding models if you are still choosing a base model
  20. Extractor catalog and pricing
  21. Managed Mixpeek

    Put multimodal search to work

    Connect a bucket and Mixpeek runs the whole multimodal search pipeline for you: extraction, indexing, and search over your own objects. No models to wire up, nothing to host.

    Start with Managed
    MVS · bring your own

    Already have vectors?

    Keep your embeddings on your own cloud and run dense, sparse, and BM25 search directly on object storage. From $25/mo.

    Start with MVS

    Run this on your own data

    Point Mixpeek at the storage you already have and search your video, images, audio, and documents the way this guide describes. Build starts at $25/mo for up to 1M vectors.

    Search your own archiveRead Docs