How to build your own private mini AI data center

Four principles, and everything else follows

These are the reason the build order looks the way it does. Each one explains why the next purchase is the right one, and together they are the argument for starting with disks rather than with a GPU.

1. 01

   Own your storage, and own it as S3

   Owning the bytes is only half of it. Owning them behind a standard object API is what makes them useful.

   Disks you control with no standard interface in front of them is a NAS: your data, reachable only by whatever happens to mount it. An S3-compatible API in front of the same disks makes the corpus addressable by every tool, SDK and pipeline that already speaks S3, including ones that have nothing to do with us. The API is the contract, and it is the part worth insisting on, because it is what stops ownership turning into lock-in of a different shape.

2. 02

   Future-proof the storage before anything else

   The namespace outlives every machine it has ever been served from.

   Accelerators are superseded, interconnects change generation, servers get replaced. None of that matters if the data does not live inside any of them. Pick the interface that survives the move (SATA disks migrate from a desktop enclosure into a rack chassis), keep an independent copy so the migration is never a single-copy event, and address everything as s3:// objects from the first day rather than as paths on a machine you will not own in three years.

3. 03

   Extend the rack so compute comes to the data

   The endgame is NVMe and inference sitting beside the object store, so search over your own storage runs end to end without leaving the rack.

   Storage alone still sends every read somewhere else to be understood. The reason to extend into a rack is co-location: a fast local NVMe tier and an accelerator in the same cabinet as the object pool, so extraction, embedding, indexing, retrieval and inference all happen next to the bytes. That is what turns owned storage into an end-to-end system, where a query against your own object storage returns a timestamp inside a video rather than a filename, and no part of answering it crossed a network you pay for.

4. 04

   Start small, extend, never re-architect

   Every phase should reuse the last one, which is a constraint on what you buy first.

   The first purchase is four disks and an enclosure, and it is useful the day it lands. The second is a rack that adopts those same disks. After that, throughput is a compute pod, capacity is more disks or a shelf, shared multi-node access is a parallel filesystem tier, and bandwidth is a fabric generation. Because the object namespace never changes, none of those is a migration, and a rack that fills up is answered by another rack rather than by starting over.

Phase 1: V1 rack

A self-contained multimodal AI appliance

• Compute-01: RTX PRO 6000 Blackwell 96 GB, Threadripper PRO 9975WX, 512 GB ECC, mirrored boot, model and scratch NVMe.
• Storage-01: 24-bay hot-swap server starting around 12 × 20-24 TB enterprise SATA.
• Durable data exposed through an S3-compatible object API; local NVMe is cache and scratch, never the source of truth.
• The full local Mixpeek stack, Ray, vector services and video extraction, plus vLLM, SGLang and TensorRT-LLM for serving.
• 100 GbE-ready internal fabric, an isolated management network, BMC/IPMI and a local break-glass console.

Outcome: Private large-scale ingestion, multimodal inference and model training in one 27-32U cabinet.

Phase 3: Frontier and off-grid

Nothing built in V1 is invalidated

• Future NVIDIA-qualified RTX PRO, MGX, HGX or DGX nodes join the existing cluster.
• Dense storage shelves scale the object pool without touching compute.
• An optional off-grid power module: solar, a 48 V LiFePO4 bank, hybrid inverter and generator, with energy-aware scheduling.
• Satellite WAN plus the secure overlay keeps the site reachable without depending on local infrastructure.

Outcome: A private AI cluster where newer accelerator nodes join without changing the object namespace.

How the disks move between phases

PHASE 0 - NOW

Mac Studio
    │ Thunderbolt
    ▼
ThunderBay 8
├── 24TB
├── 24TB
├── 24TB
└── 24TB
= 96TB raw


PHASE 1 - V1

ThunderBay corpus
       │ copy + verify
       │ 10/100GbE
       ▼
Storage-01
S3 / ZFS
12×20-24TB
       │
       ├──────── Compute-01
       │          96GB GPU
       │
       └──────── Mixpeek


PHASE 2 - SCALE

                  Kubernetes / Ray
                        │
             100→400GbE fabric
          ┌─────────────┼─────────────┐
          ▼             ▼             ▼
     Compute-01    Compute-02      Storage-01
          │             │             │
          └─────── Lustre V2 ─────────┘
                 shared hot data


PHASE 3 - PB / FRONTIER

 Compute-01  Compute-02  MGX/HGX/DGX future
      \         |          /
       └── high-speed fabric ──┐
                               ▼
                         Storage cluster
                         S3 + Lustre
                               │
                         dense JBOD shelves
                         hundreds TB → PBs

The software you will run on it

All open or freely available, all running locally. This is the layer where the work actually is, which is why the hardware plan above can be published without hesitation.

• Mixpeek local stack
• Ray / KubeRay
• Kubernetes
• NVIDIA GPU Operator
• vLLM
• SGLang
• TensorRT-LLM
• NeMo
• Megatron
• PyTorch
• Hugging Face
• Qdrant
• S3-compatible object service
• ZFS
• Lustre (V2)
• Prometheus / Grafana-class observability