Ceph, Lustre, NVMe-oF, tiering
A GPU waiting for data is a GPU you pay to do nothing
Storage is the most underestimated bottleneck in AI and HPC infrastructure. We design storage farms whose throughput is sized on your jobs, not on a datasheet.
Use cases
- Feeding GPU clusters: datasets, shuffling, massive checkpoints
- Parallel scratch for I/O-intensive HPC campaigns
- On-prem data lake: ingestion, archiving and compliance
- Backup and disaster recovery with multi-site replication
Reference architecture
- Hot tier
- Local NVMe or NVMe-oF for scratch and checkpoints: minimal latency, closest to compute.
- Parallel
- Lustre or BeeGFS for aggregated multi-node throughput: hundreds of sustained GB/s.
- Capacity
- Ceph (block, S3 object, file) for resilient capacity and automatic tiering.
- Protection
- Erasure coding, snapshots, asynchronous multi-site replication and encryption at rest.
What you receive
- 01
I/O study on your real jobs: read / write profiles, target throughput
- 02
Farm deployed and benchmarked (fio, IOR, mdtest)
- 03
Tiering, snapshot and retention policy configured
- 04
Operations runbook and training
FAQ
- Ceph or Lustre for AI?
- Lustre wins on raw parallel throughput for large-scale training; Ceph wins on versatility (S3, block, file) and operational resilience. Many farms combine both in tiers. The I/O study decides with your numbers.
- What minimum volume makes this relevant?
- From a few hundred active terabytes, or as soon as cloud storage exceeds a few thousand euros per month, a dedicated farm is worth pricing. Cloud egress fees often tip the scale earlier than expected.