What is Feature Extractor

How It Works

Each extractor declares what inputs it needs, what it does to them, and what it emits. The platform maps fields from the source object onto those inputs, runs the extractor over batches, and writes the outputs onto the resulting documents alongside their vectors. Because extraction is batched and asynchronous, objects land immediately while documents appear when the collection next processes, which is the right trade for throughput and the wrong assumption to build a read-after-write test on.

Technical Details

Extractors are where cost concentrates, since this is the stage that runs models over GPU. That makes two properties matter: idempotency, so a re-run does not silently re-pay for work already done, and honest failure classification, so an undecodable file is recorded as a decode failure on that object rather than disappearing and resurfacing as an empty embedding two stages later. Custom extractors are ordinary code in most systems worth using, which is what lets a team bring a domain model rather than settle for the built-in set.

Common Pitfalls

• Assuming a document is queryable the moment the object is written, when extraction is asynchronous
• Re-running a pipeline to fix one field and re-paying full GPU cost for every unchanged object
• Letting a failed extraction pass through silently, so the symptom appears later as a missing embedding rather than as the decode error it was
• Configuring an extractor whose input mapping names a field the source does not have, which produces empty output rather than an error
• Treating extractor output as final when the model will be replaced, instead of keeping the source so a re-run is possible

Relevance to Multimodal Systems

This is the primitive that makes multimodal search different from text search. A text chunk is close to its own representation; two hours of video is not, and everything that makes it findable, the segmentation, the transcript, the on-screen text, the per-scene embeddings, is produced here. See multimodal extractor, the extractor documentation, and bringing your own model.