One S3 Feature Unlocked the WAL-on-a-Bucket Wave
Summary
A Git server with no database, a vector store with no coordinator, and a wave of database-free systems all trace to one enabler: S3 conditional writes, added in 2024. If-None-Match creates an object only if the key is absent; If-Match replaces it only if the ETag has not moved. That is compare-and-swap on object storage, which means the bucket can be its own lock and the external coordinator most architectures bolt on can go away. Chroma's wal3 builds a write-ahead log on it, Shopify's CEO built walgit in a weekend, and Mixpeek's vector store (MVS) ships sealed WAL segments the same way, with a leader lease and boot identity check fencing every ship. The ceiling is 5 to 98 conditional writes per second per chain, so the winning shape commits locally, seals, and ships. How ours works: mixpeek.com/mvs
About this video
A Git server with no database, a vector store with no coordinator, and a wave of database-free systems all trace to one enabler: S3 conditional writes, added in 2024. If-None-Match creates an object only if the key is absent; If-Match replaces it only if the ETag has not moved. That is compare-and-swap on object storage, which means the bucket can be its own lock and the external coordinator most architectures bolt on can go away. Chroma's wal3 builds a write-ahead log on it, Shopify's CEO built walgit in a weekend, and Mixpeek's vector store (MVS) ships sealed WAL segments the same way, with a leader lease and boot identity check fencing every ship. The ceiling is 5 to 98 conditional writes per second per chain, so the winning shape commits locally, seals, and ships. How ours works: mixpeek.com/mvs