the stream

ipspace: Leaf-and-spine fabrics versus fabric extenders

Alexander Bochmann Monday 27 of February, 2017

Ivan Pepelnjak wrote:
It’s obvious that a bunch of fabric extenders (leafs) connected to a pair of Nexus switches (spines) form a leaf-and-spine fabric.
However, there are several important differences between a fabric extender-based fabric and a leaf-and-spine fabric built with standard data center switches:
* In a well-designed leaf-and-spine fabric the spine nodes are completely independent – they share no configuration, state or risk. Nexus switches configured as a vPC pair share a lot of configuration and state (and risk).
* Leaf nodes in a traditional leaf-and-spine fabric are independent devices, whereas fabric extenders act as linecards of the spine switches. The blast radius (how many things can go wrong based on a single failure) on a fabric extender-based architecture is much larger than in a fabric built with independent switches.
* Independent leaf nodes can do local packet switching whereas in a fabric extender environment all traffic has to traverse the spine layer.

We don't use Nexus fabrics in our datacenter, but the "blast radius" of a Juniper EX or QFX Virtual Chassis (Fabric) control plane failure isn't much different (though they can do local packet switching on linecards, unlike the FEXen). Our next DC design will probably go towards a routed fabric - though we'll have to build up all the automation infrastructure and skills that comes with that. The Virtual Chassis black box, for all it's downsides, removes a lot of the configuration complexity.