A purpose-built cluster interconnect that is lossless by design, with credit-based flow control instead of drop-and-retry.
What Makes It Different
InfiniBand does not avoid packet loss by tuning; it makes loss structurally impossible. A sender holds credits describing exactly how much receive buffer is free on the next hop and never transmits beyond them. There is no pause frame, no threshold, and no configuration to get wrong — a packet is never launched without somewhere to put it.
The fabric is also centrally managed. A subnet manager discovers the topology, assigns local identifiers and programmes forwarding tables, which is a very different operational model from distributed routing protocols and a large part of why it feels foreign to an Ethernet team.
Reading the Generation Names
The trap is that EDR, HDR, NDR and XDR name a lane rate, not a port rate. EDR is 25 Gb/s per lane, HDR 50, NDR 100 and XDR 200. Because the standard port width is four lanes, those become the familiar 100G, 200G, 400G and 800G figures — and XDR additionally defines an eight-lane width at 1,600 Gb/s.
Release 2.0, published in July 2025, finalised XDR at 200 Gb/s per lane along with XDR FEC and 64K-port management. Release 2.1 was announced in 2026 and brings multipath reliable connection work back toward the Ethernet and RoCE world. Lane rates track SerDes generations, so the same 100G and 200G electrical lanes appear on both sides of the InfiniBand-versus-Ethernet argument.
What You Actually Cable
Modern switch faces are twin-port: a single OSFP cage carries two independent links and presents two separate MPO connectors. A 400G NDR link is therefore often half of an 800G module, and a breakout cable that looks like a splitter is really two ports leaving one cage.
Two mechanical details cost people time. Switch cages usually want the finned riding-heatsink module while host adapters want the flat-top variant, and the two are not interchangeable. And parallel single-mode links land on MPO-12 APC, so gender and polarity have to be right through the trunk, the panel and the jumper. See MPO-12 vs MPO-16 for the connector side.
How to Choose
The technical case for InfiniBand is strongest where the workload is one large tightly synchronised job and where in-network collective aggregation genuinely helps. The case against it is rarely technical: one dominant supplier, a separate management model, and a skill set your existing team does not have.
Ethernet has closed much of the gap through lossless configuration and is closing more through Ultra Ethernet. The honest comparison is in InfiniBand vs RoCE vs Ethernet.
InfiniBand reference
| Flow control | credit-based, per virtual lane |
|---|---|
| EDR / HDR / NDR / XDR per lane | 25 / 50 / 100 / 200 Gb/s |
| 4x port rate | 100 / 200 / 400 / 800 Gb/s |
| XDR 8x width | 1,600 Gb/s |
| Governing body | InfiniBand Trade Association (IBTA) |
| Latest published release | IBTA Release 2.0, July 2025; 2.1 announced 2026, publication pending |
| Switch cage convention | twin-port OSFP — two MPO connectors per cage |
| In-network collectives | aggregation performed in the switch ASIC |
| Addressing | local IDs assigned by a subnet manager, not IP |
Related Terms
RoCE, UEC, NVLink, rail-optimised topology, standard lookup.