Rail-Optimized Topology

The Rule, and Why It Looks Wrong

Conventionally, all eight accelerators in a node connect to the same top-of-rack switch. A rail-optimised fabric deliberately breaks that: accelerator 0 in every node in the pod connects to leaf switch 0, accelerator 1 to leaf switch 1, and so on. Eight accelerators means eight rails, eight independent switch planes, and eight cables leaving each node in eight directions.

The justification is the traffic. The dominant collective in distributed training exchanges data between accelerators holding the same shard — which is to say, between the same index across different nodes. A rail puts exactly those peers on one switch.

What It Buys

The heaviest, most latency-sensitive collective completes in a single switch hop and never touches the spine. Traffic that would otherwise oversubscribe the upper fabric simply is not there, so the spine tier can be smaller for the same job performance.

Cross-rail traffic still exists and still costs three switches, but it is a much smaller share of the total. Some designs route it through the scale-up fabric inside the node instead — hop across NVLink to the accelerator that owns the right rail, then out — which trades a little intra-node bandwidth for a spine traversal avoided.

What It Costs in Cable

This is where the design lands on the cable plant, and it is not a small effect. A rail switch serves one accelerator position across every rack in the pod, so its links fan across the floor rather than staying in the rack. Runs that would have been one to three metres become five to fifty.

Passive copper does not reach that far. At 100 Gb/s per lane, three metres is the practical ceiling; at 200 Gb/s per lane it falls to roughly one to two metres. So the media mix inverts: a little DAC for same-rack rails, breakout AOC for the same-row majority, and transceivers over structured single-mode for anything crossing the hall. AI cluster cabling guide works a full bill of materials for a 128-accelerator pod.

How to Plan It

Three things decide the cost, and all three should be settled before the first purchase order. The breakout ratio, because one 800G switch port serving two 400G rails halves the port count on the switch side — check what the platform actually supports with the breakout configurator. The trunk sizing, because rail links are structured runs and the trunk is the item you cannot cheaply revisit; the cabling planner sizes it. And measured route lengths rather than nominal ones, which are always short.

Then there is bulk. Eight rails per node across several nodes per rack puts forty to seventy cables on one rack face, directly in the intake path of whatever sits behind it. Choosing AOC where the distance allows is a thermal decision as much as an optical one.

Related Terms

leaf-spine, scale-up and scale-out, GPUDirect RDMA, InfiniBand, DAC vs AOC vs ACC.