> ## Documentation Index > Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Pipeline Configuration (Advanced) > Advanced knobs behind a collection's processing pipeline — input mappings, field passthrough, parameters, and feature URIs **Start with [Features](/processing/features).** `features: ["image_search"]` is the way to create collections — the platform resolves the pipeline and defaults the wiring. This page covers the advanced configuration underneath: explicit input mappings, field passthrough, pipeline parameters, and the feature URIs that retrievers query. The `feature_extractor` config object shown here still works everywhere but is a **deprecated alias** — see the [migration guide](/processing/extractor-migration). Processing pipeline turning objects into reusable features Every collection runs one processing pipeline: Ray-powered workflows that read objects, run ML models, and write features into collection documents. Each pipeline output exposes a stable **feature URI** so retrievers, taxonomies, and clusters can reference it with confidence. When you create a collection with `features: [...]`, Mixpeek picks the pipeline, pins its version, and defaults its inputs; the knobs below are for when you need explicit control. ## Anatomy of an explicit pipeline config The explicit config is attached via the `feature_extractor` field — mutually exclusive with `features` (provide one or the other): ```json theme={null} { "feature_extractor": { "feature_extractor_name": "text_extractor", "version": "v1", "input_mappings": { "text": "product_text" }, "field_passthrough": [ { "source_path": "metadata.category" }, { "source_path": "metadata.brand" } ], "parameters": { "model": "multilingual-e5-large-instruct", "normalize": true } } } ``` Four knobs control what the pipeline sees and produces. ### `feature_extractor_name` + `version` Identifies the pipeline and pins its version per collection — a new `v2` ships without disturbing collections on `v1`. You roll forward deliberately via a [namespace migration](/api-reference/namespace-migrations/list-migrations), not in place. Built-in names are accepted as deprecated aliases of their [feature](/processing/extractor-migration); custom plugin names are yours and stay first-class. ### `input_mappings` — bind pipeline inputs to object fields Every pipeline declares an `input_schema` (e.g., the text pipeline expects a `text` input). `input_mappings` bind those named inputs to fields in your bucket schema: `product_text`, `video_url`, etc. Use **flat field names** directly, not prefixed paths like `payload.product_text`. On the `features: [...]` path these default to the standard `uploads` bucket properties (`image`, `video`, `audio`, `pdf`, `content`, `url`) — you only need explicit mappings for custom bucket schemas. ### `field_passthrough` — carry source fields into documents Pipeline outputs alone are rarely enough at query time. You want to filter by `metadata.category` or `status` without re-joining against the source bucket. `field_passthrough` copies selected source fields onto every output document so retrievers can use them directly in metadata filters. If you plan to filter by a field at query time, pass it through at ingestion time. Reaching back into the source bucket from a retriever stage is 100–1000x slower than a local metadata predicate. ### `parameters` — tune the pipeline Parameters are pipeline-specific: chunk strategy, embedding model variant, OCR enable/disable, transcription language, thumbnail quality. Invalid parameters return a teaching `422` that includes the pipeline's parameter schema (also available via `GET /v1/collections/features/extractors`). ## The output schema When you configure a collection, it immediately calculates an `output_schema` that merges passthrough fields with pipeline outputs. You can inspect this before any processing runs: ```bash theme={null} curl "https://api.mixpeek.com/v1/collections/{collection_id}/features" \ -H "Authorization: Bearer $MIXPEEK_API_KEY" ``` The response lists every feature URI the collection will produce, its type, its dimensions (for embeddings), and its description. Use this to validate downstream retrievers, taxonomies, and clusters before you ingest a single object. ## Feature URIs: the stable contract Every pipeline output publishes a URI in the form `mixpeek://{name}@{version}/{output}`: * `mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1` * `mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding` * `mixpeek://universal_extractor@v1/gemini-embedding-2` The URI is the contract between ingestion and everything downstream: * **Retrievers** reference features by URI in `feature_search` stages. * **Taxonomies** classify against a feature URI. * **Clusters** group documents by a feature URI. * **Migrations** swap one URI for another across retrievers atomically. Because the URI pins the pipeline version, a retriever built against `v1` keeps working even after `v2` is released — it simply doesn't see `v2` features unless you explicitly migrate. URIs embed internal pipeline names; that's expected — they're version-pinned implementation references, not config vocabulary. Discover the URIs your collection actually produces with `GET /v1/collections/{id}/features` rather than constructing them by hand. Treat feature URIs like API versions: pin them in production retrievers, and roll them forward deliberately rather than in-place. ## Vector index registration When a pipeline publishes an embedding, Mixpeek registers a vector index in [MVS](/vector-store/overview) at the matching URI. Retrievers then query that exact index via `feature_search`. For built-in pipelines this is automatic. For [custom extractors](/processing/custom-extractors), it's the single biggest pitfall: the `features` list in `manifest.py` must use the exact keys `feature_type`, `feature_name`, `embedding_dim`, `distance_metric`. Intuitive-but-wrong names (`type`, `name`, `dimensions`, `distance`) silently create a collection with **zero vector indexes**, and the ingestion task reports `COMPLETED` with 0 documents written. ## What happens at runtime For a single-pipeline collection, the end-to-end path is: 1. **Flatten.** API flattens the manifest into per-pipeline row artifacts (Parquet) and stores them in S3. 2. **Schedule.** Ray poller discovers pending batches and submits a job. 3. **Run.** Workers load the dataset, run the pipeline flow (GPU if available), and emit features plus passthrough fields. 4. **Write.** `MVSBatchProcessor` writes vectors and payloads to MVS, emits webhook events, and updates index signatures. Every document records lineage metadata so you can trace any feature back to the object that produced it: ```json theme={null} { "root_object_id": "obj_123", "source_collection_id": "col_source", "processing_tier": 1, "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1" } ``` When pipelines are chained across collections, the `processing_tier` field becomes load-bearing — see [Multi-Tier Feature Extraction](/processing/multi-tier-extractors) for how the DAG executes. ## Performance and scaling * **GPU workers** deliver 5–10x faster throughput for embeddings, reranking, and video processing. CPU-only pipelines skip GPU allocation (saves \~3 minutes of cluster startup and \~6x on cost). * **Ray Data** handles batching, shuffling, and parallelization automatically. Default batch size is 64 rows; tune via the pipeline's compute profile. * **Autoscaling** maintains target utilization (`0.7` CPU, `0.8` GPU by default). * **Inference cache** short-circuits repeated model calls when inputs hash to the same key — handy when reprocessing or when ingesting near-duplicates. * **Normalization** happens once at ingest (720p video mezzanine, capped image edges, 16kHz mono audio) — it bounds per-unit cost and is included in the [modality rates](/platform/billing). Opt out per collection with `full_res: true` (2x multiplier). ## Operational checklist 1. **Prefer `features: [...]`.** Reach for the explicit config only when you need custom input mappings, passthrough, or parameters. 2. **Pin versions.** Upgrade deliberately via new collection + migration, never in-place. 3. **Batch uploads.** Keep ingestion batches in the 1k–10k object range to maximize parallelism without overwhelming the scheduler. 4. **Use `field_passthrough` for every metadata filter.** If you plan to filter by `category` at query time, pass it through at ingestion time. 5. **Inspect features before querying.** `GET /v1/collections/{id}/features` returns the feature URIs, dimensions, and descriptions actually registered — use this to confirm the pipeline ran correctly before building retrievers. 6. **Watch lineage, not just counts.** A task reporting `COMPLETED` with 0 features written almost always means an `input_mappings` mistake or (for custom extractors) a bad `features` manifest. The [Document Lineage API](/api-reference/document-lineage/get-document-lineage) and the `vector_indexes` field on the collection are your diagnostic tools. ## Reference * **[Features](/processing/features)** — the feature menu per modality, discovery endpoint, and cost estimates. * **[Migration guide](/processing/extractor-migration)** — mapping deprecated built-in extractor names to feature keys. * **[Custom Extractors](/processing/custom-extractors)** — bring your own pipeline; runs on the same Ray infrastructure with full GPU support, versioning, and observability, and prices per unit from its declared compute profile.