Custom Plugins

Custom plugins let you run your own code on Mixpeek infrastructure — inside the same Ray cluster that powers the built-in extractors and retriever stages. You keep full control of the logic, model, and I/O; Mixpeek handles packaging, scheduling, GPU allocation, caching, and observability.

​

Delivery Formats

Ship your plugin in either of two formats: Both formats go through the same upload → confirm → deploy lifecycle and expose the same runtime APIs (batch __call__, real-time run_inference, platform LLM/secret accessors).

​

What You Can Build

Custom plugins plug into two places in the warehouse:

​

1. Feature Extractors (Decomposition)

Attach custom logic to a collection’s feature_extractor so every ingested object flows through your pipeline during decomposition. Use this to:

• Embed domain-specific content with your own model (fine-tuned CLIP, proprietary audio encoder, etc.)
• Extract structured attributes via a VLM you manage (brand compliance, regulated content classification)
• Transcribe, OCR, or segment media with a custom pre/post-processing chain
• Produce multiple named vector indexes from a single pass

Outputs land in MVS and MongoDB with the same feature URI scheme as built-in extractors (mixpeek://my_plugin@1.0.0/my_embedding), so retrievers, taxonomies, and clusters can reference them.

​

2. Retriever Operations (Query Time)

A plugin’s realtime.py exposes a Ray Serve HTTP endpoint that retriever stages can call during execution. Use this to power:

• feature_search — embed queries at search time with the same model you used during ingestion, so the query vector lives in the same space as the indexed vectors
• Inference operations — on-the-fly classification, scoring, or re-ranking against your model
• LLM calls — wrap a hosted or private LLM behind a stable contract, with platform-managed secrets and cost tracking
• Classifiers — apply your own classifier to candidate results mid-pipeline

This is what lets a single custom plugin own both halves of a retrieval flow: it encodes documents on the way in, and encodes the query on the way out.

​

How It Fits Together

​

Version Management

Plugins support a git-like workflow for iterating on deployed code: Use the CLI at server/scripts/api/plugins.py or call the version management endpoints directly:

• GET /v1/namespaces/{ns}/plugins/by-name/{slug}/versions — version history
• POST /v1/namespaces/{ns}/plugins/by-name/{slug}/rollback — rollback
• GET /v1/namespaces/{ns}/plugins/by-name/{slug}/diff?v1=1.0.0&v2=1.0.1 — diff
• GET /v1/namespaces/{ns}/plugins/{id}/source — download source files

​

Pre-installed Tools

The engine runtime includes media processing tools available via run_tool from the Plugin SDK: See Cinema Camera Raw File Support for usage examples.

​

Next Steps