BGP EVPN VXLAN — Trivia & Interesting Facts¶
Surprising, historical, and little-known facts about BGP EVPN with VXLAN overlay networking.
VXLAN was born from a real crisis at VMware¶
VXLAN (RFC 7348, 2014) was created because VMware ran into the 4,094 VLAN limit when trying to build multi-tenant cloud infrastructure. The 24-bit VXLAN Network Identifier (VNI) supports over 16 million segments — roughly 4,000 times more than VLANs. The original draft was co-authored by engineers from VMware, Cisco, Arista, and Broadcom.
EVPN was originally designed for MPLS, not VXLAN¶
EVPN (RFC 7432, 2015) was first specified as an MPLS-based solution for Ethernet VPN services at service providers. The adaptation of EVPN as a control plane for VXLAN came later and was not the original intent. This MPLS heritage is why EVPN route types and terminology can feel alien to data center engineers who have never touched an SP network.
BGP was never meant to run inside a data center¶
BGP was designed for inter-domain routing between autonomous systems on the Internet. Using iBGP or eBGP as the underlay and overlay control plane inside a single data center — as modern leaf-spine fabrics do — would have seemed absurd to the BGP designers of the 1980s. RFC 7938 (2016) formally documented this "BGP in the data center" pattern, largely driven by Facebook and Microsoft's hyperscale designs.
The EVPN type-2 route carries both MAC and IP¶
EVPN Route Type 2 (MAC/IP Advertisement) can carry a MAC address, an IP address, or both simultaneously. This dual capability is what enables EVPN to do distributed ARP suppression — the fabric learns MAC-to-IP bindings via BGP and can answer ARP requests locally at each leaf, dramatically reducing broadcast traffic in large fabrics.
VXLAN adds exactly 50 bytes of overhead¶
Every VXLAN-encapsulated frame adds a 50-byte header: 8 bytes VXLAN + 8 bytes UDP + 20 bytes outer IP + 14 bytes outer Ethernet. This is why VXLAN fabrics require jumbo frames (typically MTU 9214) on the underlay — without them, the inner payload gets squeezed to 1450 bytes, causing fragmentation and mysterious performance problems.
Facebook's data center network helped kill Spanning Tree¶
Facebook published their "fabric" data center network design around 2014, using a Clos topology with BGP as the sole routing protocol and no Spanning Tree anywhere. This design, combined with VXLAN overlays, proved that you could build a data center network with pure Layer 3 routing and overlay-based Layer 2 extension. The approach was so successful that it became the template for modern leaf-spine architectures industry-wide.
EVPN has five route types, and most engineers only use two¶
EVPN defines Route Types 1 through 5: Ethernet Auto-Discovery, MAC/IP Advertisement, Inclusive Multicast, Ethernet Segment, and IP Prefix. In practice, most data center deployments primarily use Type 2 (MAC/IP) and Type 5 (IP Prefix) routes. Types 1, 3, and 4 handle multi-homing and BUM traffic replication — critical functions that work silently in the background.
The "flood and learn" vs. EVPN debate split the industry¶
Before EVPN became dominant, VXLAN deployments used "flood and learn" for MAC address discovery — essentially traditional Ethernet behavior tunneled through VXLAN. This approach was simpler but scaled terribly. The migration from flood-and-learn to BGP EVPN control plane was one of the most significant architectural shifts in data center networking between 2015 and 2020.
Asymmetric vs. symmetric IRB is a surprisingly heated topic¶
EVPN supports two inter-VXLAN routing models. Asymmetric IRB (Integrated Routing and Bridging) routes on ingress and bridges on egress, requiring all VNIs on all leaves. Symmetric IRB routes on both sides using a transit L3 VNI, allowing VNIs to exist only where needed. The choice between them has caused more design arguments in network engineering teams than almost any other VXLAN topic.
A single misconfigured VNI can create a silent black hole¶
If one leaf in an EVPN fabric has a VNI-to-VLAN mapping that differs from the rest, traffic entering that leaf will be encapsulated with the wrong VNI. The remote leaf will either drop the traffic silently (if the VNI doesn't exist) or deliver it to the wrong VLAN. These mismatches produce no errors, no log messages, and no alarms — just missing traffic. This is one of the most common and hardest-to-diagnose EVPN issues.
Multi-site EVPN connects data centers without stretching failure domains¶
EVPN Multi-Site (or DCI with EVPN) uses border gateways that re-originate EVPN routes between sites, preventing a MAC learning storm in one data center from propagating to another. This was a direct response to the disasters caused by earlier technologies like OTV and VPLS that transparently stretched Layer 2 across sites and allowed broadcast storms to take down multiple data centers simultaneously.