Can you run vector search on NFS, Lustre or a local NVMe drive?

Can a vector database run on NFS, Lustre or a local NVMe drive?

On a local NVMe drive, yes, and that is the storage most vector databases are designed for. On a shared network filesystem such as NFS, or a parallel filesystem such as Lustre, the answer depends on the engine and on what the filesystem promises about locking and visibility. Some engines refuse it outright, some allow it with conditions, and the ones that work well on slower shared storage are designed to make few, large reads instead of many small ones.

The question comes up most in research labs and HPC centers, where the data already sits on a cluster filesystem, and on media teams whose archive lives on a NAS. Copying everything to local disks feels wasteful, so it is natural to ask whether the index can live where the data does.

What do the major vector databases say about NFS?

Their own documentation is the best guide, and it varies more than you might expect:

Engine

What its docs say about network filesystems

Source

Qdrant	"Qdrant won't work with Network file systems such as NFS, or Object storage systems such as S3." It requires block-level storage with a POSIX-compatible filesystem.	Qdrant installation docs
Milvus	Index files and logs go to object storage (MinIO, S3, Azure Blob). NFS appears only for its Woodpecker write-ahead log: if all pods share a filesystem such as NFS, cluster mode can use the local WAL backend.	Milvus architecture, Woodpecker
LanceDB / Lance	A dataset path can be a local directory or an object store (s3://, gs://, az://). LanceDB lists local NVMe, EBS and EFS (an NFS service) among the storage it runs on. Commits depend on the storage letting exactly one writer create a given manifest.	Lance object stores, LanceDB storage
PostgreSQL + pgvector	"It is possible to use an NFS file system for storing the PostgreSQL data directory." The firm requirement is a hard mount, and a sync export is strongly recommended.	PostgreSQL docs, NFS
Chroma	The local PersistentClient writes to a directory and needs SQLite; Chroma's docs call it intended for development and testing.	Chroma clients
SQLite (under Chroma and many embedded tools)	Exclusive locks "have been known to operate incorrectly for some network filesystems. This has led to database corruption."	sqlite.org

So before you point anything at a shared mount, read what that engine says. A database that rules it out has a reason, and the reason usually shows up as corruption or lost writes rather than an error at startup.

Why does a shared filesystem cause trouble for a vector index?

Two different problems hide under "it is slower over the network".

Correctness. A database that writes needs the filesystem to honor locks, flushes and renames the way a local disk does. NFS gives close-to-open consistency: when an application closes a file, the client writes back its changes "so that the next opener can view the changes", and the protocol "is not designed to support true cluster file system cache coherence without some type of application serialization" (nfs(5)). A reader that keeps an index file open or memory-mapped while another machine rewrites it is therefore not guaranteed to see the new bytes. That is exactly how a long-running search process holds its index. It is why SQLite and Qdrant warn you off, and why PostgreSQL insists on a hard mount and a sync export: without sync, "an fsync or equivalent on the NFS client is not actually guaranteed to reach permanent storage on the server."

Latency per read. A graph index such as HNSW answers a query by hopping from node to node, and each hop reads a small, unpredictable piece of the index. On local NVMe that read finishes well under a millisecond. Over a network filesystem each one becomes a round trip, and a query that makes hundreds of them in sequence adds those round trips up. Throughput, the number your storage team quotes, does not help here: a parallel filesystem can stream gigabytes a second and still be slow at many tiny dependent reads.

Does Lustre behave differently from NFS?

Lustre is built for HPC workloads that read and write large files in parallel. Its own wiki says "Lustre file system performance is currently optimized for large files," and that small files cost "additional RPC round-trips to the OSTs, which hurt small file performance" (Lustre wiki: Data on MDT). The Lustre manual says it supports mmap() (Lustre manual).

For vector search that means a large, read-only index file that every node reads in big sequential chunks suits Lustre well, for example loading an index into each node's memory at the start of a job. An engine that keeps its index on Lustre and makes small random reads per query fights the filesystem's design.

Which index designs work on slower storage?

The question to ask of an index is how many separate reads one query needs and how big they are.

In-memory indexes. HNSW kept fully in RAM does not care where its file lives once it is loaded. Shared storage only affects load time, and a large sequential read is what Lustre and NFS do well. The cost is RAM on every node that serves queries.

Inverted-file (IVF) indexes on disk. FAISS can store an IVF index's lists in a file and read them on demand: it keeps the lookup table in RAM "so that accessing an inverted list requires a single disk seek" (FAISS wiki). A query reads a handful of fairly large lists, which suits network storage better than hundreds of small hops. FAISS can also memory-map a standard index with IO_FLAG_MMAP.

SSD-resident graphs. DiskANN keeps compressed vectors in RAM and the full graph on SSD, and was designed to make few reads per query. Its paper reports indexing and searching a billion points on one workstation with 64 GB of RAM and an inexpensive SSD, at more than 5,000 queries a second with under 3 ms mean latency and 95%+ 1-recall@1 on 16 cores (DiskANN, NeurIPS 2019). That figure is for a local SSD. Each extra millisecond of storage latency is paid on every one of those reads.

Columnar files and object-storage designs. Lance-style formats and object-storage-backed engines assume each read is expensive, so they batch reads, cache aggressively and keep the hot part of the index local. That assumption carries over reasonably to a network filesystem.

How do I test my own mount before choosing?

Measure the two things that matter: small random-read latency, and whether the mount behaves correctly with more than one machine.

Random-read latency with fio, run on the mount you plan to use and then on a local NVMe drive for comparison:

fio --name=randread --filename=/mnt/shared/fio.test --size=4G \
    --rw=randread --bs=4k --iodepth=1 --numjobs=1 --direct=1 \
    --runtime=60 --time_based --group_reporting

Read the completion latency percentiles (clat). With iodepth=1 every read waits for the previous one, which is how a graph search behaves. Multiply the p50 by a few hundred reads to estimate the storage cost of one disk-resident graph query.

A quick visibility check between two machines, in Python. Run the writer on one host and the reader on another, both pointed at the same shared directory:

# writer.py: rewrite a small file in place every second
import os, time
path = "/mnt/shared/visibility.bin"
with open(path, "r+b" if os.path.exists(path) else "w+b") as f:
    for i in range(30):
        f.seek(0); f.write(i.to_bytes(8, "little")); f.flush(); os.fsync(f.fileno())
        time.sleep(1)

# reader.py: keep the file open (as a search process would) and watch the value
import mmap, time
with open("/mnt/shared/visibility.bin", "rb") as f:
    m = mmap.mmap(f.fileno(), 8, access=mmap.ACCESS_READ)
    for _ in range(30):
        print(int.from_bytes(m[:8], "little")); time.sleep(1)

If the reader's value lags the writer's or stops changing, a process holding an open index on this mount will serve stale data after another node updates it. If it tracks the writer closely, the mount's caching settings are stricter, and you still need the engine's own documentation to say it is safe.

What setup do teams usually end up with?

Index on local NVMe, source data on the shared filesystem. The vector database runs on nodes with local SSDs, and the files being indexed stay on NFS or Lustre. This is the default for most engines and the only supported one for Qdrant.

Read-only index on the shared filesystem, loaded into memory. One job builds the index, writes it as a single file, and every query node loads it at startup. This suits Lustre and works with FAISS or any engine that can load a saved index.

An engine designed for slow storage. IVF-on-disk, DiskANN-style or object-storage-backed designs that make few large reads, with a local cache in front.

How does Mixpeek handle local and shared filesystems?

Mixpeek Vector Store normally keeps vectors in an S3-compatible bucket and serves queries from a local cache, which is the object-storage design above. Storing MVS data on a POSIX filesystem, including local NVMe, NFS and Lustre, is in private beta: it is opt-in, and you contact us to enable it for a deployment (docs). For how the object-storage design works and what it costs, read Can you run a vector database on S3?, and for choosing an engine, see the best vector databases.

Frequently Asked Questions

Can I run Qdrant on NFS?

No. Qdrant's installation docs say it "won't work with Network file systems such as NFS, or Object storage systems such as S3," and that it needs block-level storage with a POSIX-compatible filesystem. Use local disks or block volumes such as iSCSI for its data.

Can I put a PostgreSQL database with pgvector on NFS?

PostgreSQL's docs allow an NFS data directory if it is mounted with the hard option, and strongly recommend the sync export option on the server so that fsync really reaches storage. pgvector indexes live in that data directory, so they follow the same rules.

Why is my vector search slow on a network filesystem?

Graph indexes such as HNSW make many small reads per query, one after another, and each read becomes a network round trip. Measure 4 KB random-read latency at queue depth 1 with fio; that number, times the reads per query, is the storage share of each query.

Is Lustre good for vector search?

Lustre is optimized for large files and parallel throughput, and its wiki says small reads cost extra round trips. It suits building an index and loading it into memory at the start of a job, and suits small random reads per query poorly.

Which vector index works best when storage is slow?

Designs that make few, large reads: an IVF index with its lists on disk (FAISS OnDiskInvertedLists), an SSD-resident graph such as DiskANN, or an object-storage-backed engine with a local cache. An HNSW index held entirely in RAM also works, because storage only affects load time.

Is SQLite-based vector storage safe on NFS?

SQLite's own documentation says its locks have operated incorrectly on some network filesystems and that this has led to database corruption. Keep SQLite files, including a local Chroma directory, on a local disk.