What is Bucket
Bucket - A schema-backed container for the raw objects you upload, before any processing
A bucket holds source material as you supplied it. Objects go in with their blobs (video, image, audio, text, JSON) and whatever metadata you attach, validated against a schema the bucket declares. Nothing in a bucket is searchable by meaning yet: the bucket is the durable record of what arrived, and a collection is what turns it into something you can query.
How It Works
You declare a bucket with a schema describing the blob properties and types it expects, then write objects into it. The schema is enforced at write time, so a malformed object is rejected at the door rather than surfacing three stages later as an empty embedding. Blob bytes go to object storage and the metadata is indexed, and each object keeps an identifier that everything downstream carries as lineage.
Technical Details
Buckets are the natural boundary for retention and takedown, because they hold the original bytes rather than derived features, and deleting derived documents does not delete the source. Validation happens on content as well as on the declared label: a payload whose declared mime type contradicts its actual bytes should be rejected at ingest, since the label is the part a caller controls. One object commonly becomes many documents downstream, so bucket object counts and collection document counts are different numbers and are supposed to be.
Common Pitfalls
- Expecting search to work against a bucket, when search runs over the collection derived from it
- Treating a bucket schema as advisory, then discovering the pipeline breaks on the one field nobody validated
- Trusting a caller-supplied mime type over the bytes, which lets a corrupt payload in through the transport that checks least
- Deleting documents to honour a takedown while the source object stays in the bucket
- Comparing object count to document count and reading the difference as data loss
Relevance to Multimodal Systems
Buckets matter more in multimodal work than in text, because the source is large, expensive to move, and frequently the legal artifact. Keeping the originals intact and deriving everything else means an extractor change is a re-run rather than a re-upload of terabytes. It is also why bring-your-own object storage is coherent: if the bucket is the substrate, it can be a bucket you already own. See buckets, the ingestion documentation, the derived side in collection, and ingesting millions of files.
Put multimodal search to work
Connect a bucket and Mixpeek runs the whole multimodal search pipeline for you: extraction, indexing, and search over your own objects. No models to wire up, nothing to host.
Start with ManagedAlready have vectors?
Keep your embeddings on your own cloud and run dense, sparse, and BM25 search directly on object storage. From $25/mo.
Start with MVS