Ultra Ethernet vs RoCE: What UEC Actually Changes for AI Fabrics
Ethernet won the data centre by dropping packets. That sounds like an insult and it is not: dropping under congestion, and letting TCP work it out, is what let Ethernet scale cheaply for thirty years. It is also precisely the property that makes it awkward for AI.
RoCE bolted RDMA onto Ethernet and then spent a decade engineering around that mismatch. The Ultra Ethernet Consortium is trying something different: leave the switching silicon and the optics alone, and replace the transport that never fitted.
The Problem UEC Is Actually Solving
RDMA was designed for a fabric that does not lose packets. Its loss recovery is brutally inefficient — historically a go-back-N retransmission that throws away a great deal of correctly delivered data. So a RoCE fabric has to be configured never to drop, using PFC to pause traffic and ECN with DCQCN to slow senders before pausing is needed.
That works. Very large clusters run on it. But it is an end-to-end invariant, not a per-switch setting: every switch in every path needs consistent priority mapping, ECN thresholds and PFC configuration. Replace one switch during a maintenance window with a default configuration and the property is silently lost for everything flowing through it. The symptom is not an alarm — it is a training job that got slower.
That operational fragility, not raw performance, is what UEC is aimed at. If the transport recovered from occasional loss cheaply, the fabric would not have to be perfect.
Why One Flow Can Idle Half Your Fabric
There is a second problem, and it is arguably worse. Standard ECMP load balancing hashes a flow's five-tuple to pick a path, and every packet of that flow follows it. With thousands of small flows this distributes beautifully.
An AI collective does not produce thousands of small flows. It produces a handful of enormous, synchronised ones. Hash four elephant flows across four spine links and — by simple probability — two of them collide while a link sits idle. The fabric is congested and underused at the same instant.
Packet Spraying and Why It Needs a New Transport
The obvious fix is to spread a single flow across every available path — packet spraying. Every link carries a share, and the collision problem disappears.
The reason nobody simply enabled this years ago is that spraying guarantees out-of-order arrival. Different paths have different queue depths, so packets arrive shuffled. Classic RoCE treats out-of-order as a symptom of loss and reacts accordingly, which turns the cure into a worse disease.
UEC's central change is a transport designed to accept out-of-order delivery as normal and to recover from actual loss selectively rather than by retransmitting everything since the gap. Once that is true, spraying becomes not just possible but the obvious default — and the fabric's usable capacity rises to something close to its installed capacity.
What Happens to PFC and ECN
They do not disappear, but their role changes. If the transport survives occasional loss without collapsing, PFC stops being load-bearing and becomes what it should always have been: a rarely used safety net. The head-of-line blocking, the congestion spreading and the deadlock hazards that come with pause-based flow control all become far less consequential when you are no longer depending on them.
For an operations team the practical difference is the size of the invariant you have to maintain. Today, "the fabric is lossless" is a property of every switch in every path and it degrades silently. Under UEC it becomes a performance optimisation rather than a correctness requirement.
What This Means If You Are Buying Switches Now
Three things worth doing, none of which require you to bet on a timeline.
Ask the direct question. Not "do you support Ultra Ethernet" — everyone says yes to that. Ask whether the specific switch silicon and NIC you are quoting can support UEC transport through a software update, or whether it needs new hardware. The answer materially changes the useful life of the purchase.
Instrument RoCE properly in the meantime. PFC pause counters and ECN marking rates, graphed per switch, per priority, over time. If you cannot see those today you cannot tell a tuned fabric from an untuned one, and you will not be able to tell whether UEC helped either.
Do not let it change the cabling. This is the reassuring part. UEC is a transport-layer change. The optics, the fibre, the connectors and the reach classes are identical either way — the same 800G modules, the same in-rack DAC, the same MPO trunk between rows. Size the plant for the port count and the topology, not for the transport. The cabling planner does not care which fabric wins.
FAQ
Is UEC a replacement for InfiniBand? It is aimed at the same workloads. Whether it displaces InfiniBand depends on delivered performance at scale, which is not yet demonstrable. The honest position today is that both remain reasonable choices and the cabling plant should not depend on which way it goes.
Will my existing RoCE fabric stop working? No. RoCE is not deprecated by anything and large clusters will run it for years. UEC is a better answer to the same problem, not a flag day.
Does packet spraying need special optics? No. It is entirely a switching and transport behaviour. Nothing about the physical layer changes.
When does this actually matter? When you are specifying hardware with a five-year life today. That is the decision UEC should influence — not next month's cabling order.
