The S3 Vector Warehouse: Storage-First Vector Search

A billion 768-dimensional float32 vectors is about 3TB raw.

In the memory-resident model that's a large cluster, sized for the whole corpus, running whether anyone queries it or not. On object storage it's a rounding error of monthly storage cost.

That gap is why storage-first vector search stopped being a hack. AWS made the pattern official when S3 Vectors went GA in December 2025, claiming up to 90 percent cost reduction and supporting up to two billion vectors per index. Several vendors had been shipping variations of the same idea before that.

The architecture is the same three layers either way. Only the residency of the middle one changes. Instead of loading the entire index into RAM and keeping it there, the index lives as immutable segments on object storage, a query routes to the partitions it actually needs, and a cache keeps the hot working set local.

Which means object storage is the source of truth, not a backup. That constraint is doing real work: object storage has no in-place update, so the index becomes append-only segments plus compaction, and deletes and freshness need their own design rather than falling out for free.

The honest decision rule is a latency budget, and it answers in both directions.

Storage-first is usually right when the corpus is large, the query rate is moderate, and 100 millisecond class latency is acceptable. Cold queries with nothing cached run in the high hundreds of milliseconds. Warm queries settle around 100ms.

You still want RAM when you need sub-10ms everywhere, when the query rate is high enough that nothing is ever cold, or when the corpus is small enough that memory is cheap anyway.

One thing worth asking any vendor: what does a first query on an untouched partition cost? A benchmark that only reports warm latency is describing the cache, not the architecture, and cold is what a new user sees.

mxp.co/d/s3-vector-warehouse

[image: s3-vector-warehouse.png]