Skip to main content
Start with Features. features: ["image_search"] is the way to create collections — the platform resolves the pipeline and defaults the wiring. This page covers the advanced configuration underneath: explicit input mappings, field passthrough, pipeline parameters, and the feature URIs that retrievers query. The feature_extractor config object shown here still works everywhere but is a deprecated alias — see the migration guide.
Processing pipeline turning objects into reusable features
Every collection runs one processing pipeline: Ray-powered workflows that read objects, run ML models, and write features into collection documents. Each pipeline output exposes a stable feature URI so retrievers, taxonomies, and clusters can reference it with confidence. When you create a collection with features: [...], Mixpeek picks the pipeline, pins its version, and defaults its inputs; the knobs below are for when you need explicit control.

​
Anatomy of an explicit pipeline config

The explicit config is attached via the feature_extractor field — mutually exclusive with features (provide one or the other): 
{
  "feature_extractor": {
    "feature_extractor_name": "text_extractor",
    "version": "v1",
    "input_mappings": {
      "text": "product_text"
    },
    "field_passthrough": [
      { "source_path": "metadata.category" },
      { "source_path": "metadata.brand" }
    ],
    "parameters": {
      "model": "multilingual-e5-large-instruct",
      "normalize": true
    }
  }
}
Four knobs control what the pipeline sees and produces.

​
feature_extractor_name + version

Identifies the pipeline and pins its version per collection — a new v2 ships without disturbing collections on v1. You roll forward deliberately via a namespace migration, not in place. Built-in names are accepted as deprecated aliases of their feature; custom plugin names are yours and stay first-class.

​
input_mappings — bind pipeline inputs to object fields

Every pipeline declares an input_schema (e.g., the text pipeline expects a text input). input_mappings bind those named inputs to fields in your bucket schema: product_text, video_url, etc. Use flat field names directly, not prefixed paths like payload.product_text. On the features: [...] path these default to the standard uploads bucket properties (image, video, audio, pdf, content, url) — you only need explicit mappings for custom bucket schemas.

​
field_passthrough — carry source fields into documents

Pipeline outputs alone are rarely enough at query time. You want to filter by metadata.category or status without re-joining against the source bucket. field_passthrough copies selected source fields onto every output document so retrievers can use them directly in metadata filters.
If you plan to filter by a field at query time, pass it through at ingestion time. Reaching back into the source bucket from a retriever stage is 100–1000x slower than a local metadata predicate.

​
parameters — tune the pipeline

Parameters are pipeline-specific: chunk strategy, embedding model variant, OCR enable/disable, transcription language, thumbnail quality. Invalid parameters return a teaching 422 that includes the pipeline’s parameter schema (also available via GET /v1/collections/features/extractors).

​
The output schema

When you configure a collection, it immediately calculates an output_schema that merges passthrough fields with pipeline outputs. You can inspect this before any processing runs: 
curl "https://api.mixpeek.com/v1/collections/{collection_id}/features" \
  -H "Authorization: Bearer $MIXPEEK_API_KEY"
The response lists every feature URI the collection will produce, its type, its dimensions (for embeddings), and its description. Use this to validate downstream retrievers, taxonomies, and clusters before you ingest a single object.

​
Feature URIs: the stable contract

Every pipeline output publishes a URI in the form mixpeek://{name}@{version}/{output}:
  • mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1
  • mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding
  • mixpeek://universal_extractor@v1/gemini-embedding-2
The URI is the contract between ingestion and everything downstream:
  • Retrievers reference features by URI in feature_search stages.
  • Taxonomies classify against a feature URI.
  • Clusters group documents by a feature URI.
  • Migrations swap one URI for another across retrievers atomically.
Because the URI pins the pipeline version, a retriever built against v1 keeps working even after v2 is released — it simply doesn’t see v2 features unless you explicitly migrate. URIs embed internal pipeline names; that’s expected — they’re version-pinned implementation references, not config vocabulary. Discover the URIs your collection actually produces with GET /v1/collections/{id}/features rather than constructing them by hand.
Treat feature URIs like API versions: pin them in production retrievers, and roll them forward deliberately rather than in-place.

​
Vector index registration

When a pipeline publishes an embedding, Mixpeek registers a vector index in MVS at the matching URI. Retrievers then query that exact index via feature_search. For built-in pipelines this is automatic. For custom extractors, it’s the single biggest pitfall: the features list in manifest.py must use the exact keys feature_type, feature_name, embedding_dim, distance_metric. Intuitive-but-wrong names (type, name, dimensions, distance) silently create a collection with zero vector indexes, and the ingestion task reports COMPLETED with 0 documents written.

​
What happens at runtime

For a single-pipeline collection, the end-to-end path is:
  1. Flatten. API flattens the manifest into per-pipeline row artifacts (Parquet) and stores them in S3.
  2. Schedule. Ray poller discovers pending batches and submits a job.
  3. Run. Workers load the dataset, run the pipeline flow (GPU if available), and emit features plus passthrough fields.
  4. Write. MVSBatchProcessor writes vectors and payloads to MVS, emits webhook events, and updates index signatures.
Every document records lineage metadata so you can trace any feature back to the object that produced it: 
{
  "root_object_id": "obj_123",
  "source_collection_id": "col_source",
  "processing_tier": 1,
  "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1"
}
When pipelines are chained across collections, the processing_tier field becomes load-bearing — see Multi-Tier Feature Extraction for how the DAG executes.

​
Performance and scaling

  • GPU workers deliver 5–10x faster throughput for embeddings, reranking, and video processing. CPU-only pipelines skip GPU allocation (saves ~3 minutes of cluster startup and ~6x on cost).
  • Ray Data handles batching, shuffling, and parallelization automatically. Default batch size is 64 rows; tune via the pipeline’s compute profile.
  • Autoscaling maintains target utilization (0.7 CPU, 0.8 GPU by default).
  • Inference cache short-circuits repeated model calls when inputs hash to the same key — handy when reprocessing or when ingesting near-duplicates.
  • Normalization happens once at ingest (720p video mezzanine, capped image edges, 16kHz mono audio) — it bounds per-unit cost and is included in the modality rates. Opt out per collection with full_res: true (2x multiplier).

​
Operational checklist

  1. Prefer features: [...]. Reach for the explicit config only when you need custom input mappings, passthrough, or parameters.
  2. Pin versions. Upgrade deliberately via new collection + migration, never in-place.
  3. Batch uploads. Keep ingestion batches in the 1k–10k object range to maximize parallelism without overwhelming the scheduler.
  4. Use field_passthrough for every metadata filter. If you plan to filter by category at query time, pass it through at ingestion time.
  5. Inspect features before querying. GET /v1/collections/{id}/features returns the feature URIs, dimensions, and descriptions actually registered — use this to confirm the pipeline ran correctly before building retrievers.
  6. Watch lineage, not just counts. A task reporting COMPLETED with 0 features written almost always means an input_mappings mistake or (for custom extractors) a bad features manifest. The Document Lineage API and the vector_indexes field on the collection are your diagnostic tools.

​
Reference

  • Features — the feature menu per modality, discovery endpoint, and cost estimates.
  • Migration guide — mapping deprecated built-in extractor names to feature keys.
  • Custom Extractors — bring your own pipeline; runs on the same Ray infrastructure with full GPU support, versioning, and observability, and prices per unit from its declared compute profile.