Storage format (.mvs)

Mixpeek stores each namespace as a .mvs dataset: a documented tree of objects in an object-storage bucket. There is no proprietary storage service in the path. The same layout runs on GCS, S3, MinIO, or your own hardware. A .mvs dataset is a prefix, not a single file. Ranged reads, parallel restore, and lazy per-partition loading all depend on that shape.

The layout

<bucket>/mvs/snapshots/<namespace>/<shard>/latest/
    meta.json               snapshot descriptor
    payloads.bin            document payload store
    keyword_indexes.bin     filter index image
    namespace_index.bin     namespace routing index
    indexes/<name>/
      manifest.bin          declares every partition by filename
      partitions/           one blob per index partition
<bucket>/mvs/wal/<namespace>/<shard>/
    <seq>.wal               write-ahead log segments
    highwater.json          highest shipped sequence
<bucket>/mvs/leader/<namespace>/<shard>/leader.json

meta.json carries a format identifier (lire_rs_v1 today). Readers refuse identifiers they do not recognize, so a future format change cannot silently misread your data.

What the format guarantees

Eight invariants define a valid dataset. The load-bearing ones:

Pairing. Every indexed point has both an id and a vector entry. A point cannot be countable but unsearchable.
Parts sum to whole. Partition row counts add up to the manifest’s total, and the manifest’s total reconciles with the payload store.
Manifest honesty. Every partition the manifest declares exists as an object, by name. Vanished blobs surface as findings, not as silently missing results.
Durability frontier. The snapshot’s replay floor never exceeds the shipped write-ahead log. Recovery can always replay forward from a snapshot to the last shipped write.
Checksums. The snapshot image carries an inline SHA-256 trailer. Restore refuses a mismatch.

Mixpeek validates namespaces against these invariants with an internal checker. Each check exists because a real production incident motivated it.

Why this matters to you

No lock-in. Your data sits in your bucket in a documented layout. A tool that implements this page can read it without Mixpeek in the path. Auditability. Regulated teams can verify what is stored, where, and whether it is internally consistent, from the objects alone. Portability. Copying a namespace is an object copy of three trees. The format validates identically at the source and the destination, on any object store. Tiering without translation. Warm and cold data use the same layout at different object-storage classes. Nothing is rewritten when data moves between tiers.

Status

The format specification is at v1. The identifier lire_rs_v1 ships in every production meta.json today, so v1 describes what is already stored, not a migration target. Enterprise customers can request the full byte-level specification.

The write path stays exclusive to Mixpeek: all ingestion flows through the API and processing pipeline. The format documents what we store so you can always read, verify, and move your own data.

​The layout

​What the format guarantees

​Why this matters to you

​Status

The layout

What the format guarantees

Why this matters to you

Status