Network fabric
InfiniBand or tuned RoCEv2: the choice worth 600,000 dollars
On a mid-size GPU cluster, the fabric decision can swing the budget by six figures. Here is how to decide with benchmarks instead of brand loyalty.
Written by
YAPIO
Published on
Jul 18, 2026
What is actually at stake
Distributed training is a communication problem wearing a compute costume. Gradient all-reduce synchronizes every GPU at every step; if the fabric stalls, thousands of dollars of accelerators wait on the network. That is why fabric is the second-biggest line on a GPU farm quote, and why the InfiniBand-versus-Ethernet question deserves an engineering answer.
The price gap is real. On a 64 to 128 GPU cluster, an InfiniBand NDR fabric (switches, NICs, cables, licenses) commonly costs 40 to 60 percent more than an equivalent 400G Ethernet fabric for RoCEv2. Across a mid-size deployment, that difference reaches the order of 600,000 dollars, before support contracts.
What "tuned" RoCEv2 really means
RoCEv2 out of the box is not a contender; RoCEv2 engineered as a lossless fabric is. That means PFC configured per priority class to prevent drops, ECN with DCQCN so congestion is signaled before buffers overflow, correct MTU and buffer profiles on every switch, and NIC firmware aligned across the fleet. Miss one of these and NCCL throughput collapses under congestion exactly when your largest job needs it.
Properly tuned, published results and our field experience agree: for clusters up to roughly 128 GPUs, RoCEv2 delivers all-reduce bandwidth within a few percent of InfiniBand for typical training workloads. InfiniBand keeps advantages that matter at larger scale: in-network reductions with SHARP, adaptive routing maturity, and a single-vendor stack when the job spans hundreds of nodes.
A decision method that survives procurement
Decide on three inputs. Scale: below 128 GPUs, tuned RoCEv2 is the default; above 256, InfiniBand earns its premium; in between, benchmark. Workload: inference farms and single-node fine-tuning barely stress the fabric, while frequent large-scale pretraining pushes toward InfiniBand. Team: RoCEv2 assumes network engineers comfortable with PFC and ECN; if that skill is absent and not hired for, the InfiniBand premium buys operational simplicity.
Whatever you pick, insist on acceptance criteria in the contract: NCCL all-reduce bandwidth and bus utilization targets at several message sizes, measured on the delivered cluster. YAPIO prices both fabrics in every audit and validates the winner with those benchmarks before handover. Six figures deserve at least that much rigor.