EVPN-VXLAN with Internet DCI on VyOS

Modern data centers increasingly rely on IP fabrics that use BGP EVPN as the control plane and VXLAN as the data plane. This standards-based architecture extends Layer 2 and Layer 3 services across an IP underlay while keeping a clear separation between the physical network and tenant or service networks.

This post summarizes the routing models, interconnect options, and resiliency considerations behind an Internet-based Data Center Interconnect (DCI) built on VyOS. For the complete configuration and validation steps, see the EVPN-VXLAN DCI Deployment Guide.

Why Inter-Subnet Routing Shapes the DCI Architecture

In the DCI deployment, VXLAN provides overlay encapsulation for Ethernet traffic between Virtual Tunnel Endpoints (VTEPs), while BGP EVPN distributes reachability information among participating devices. Instead of relying solely on data-plane learning, EVPN exchanges MAC and IP reachability via BGP, improving scalability and providing the fabric with a more deterministic control plane.

When workloads in different IP subnets need to communicate, the fabric must also provide Layer 3 routing. Physical servers, virtual machines, and containers typically use an IP gateway on their local subnet, and where that gateway lives determines the fabric's routing model.

How Routing Models Work on VyOS

Centralized Routing

Centralized routing uses selected VTEPs as Layer 3 gateways, while other VTEPs mainly provide Layer 2 VXLAN bridging. Inter-subnet traffic passes through these gateways for routing before reaching the destination network. This approach simplifies gateway configuration and routing policy, especially in smaller environments, but it can add hops and requires careful planning for gateway capacity, resiliency, and failure domains.

Distributed routing

Distributed routing moves Layer 3 gateway functionality closer to workloads. Each participating leaf operates as both an L2 and an L3 VTEP, providing VXLAN bridging and IP routing through IRB interfaces tied to the corresponding VLAN/VNI networks. When a workload sends traffic to another subnet, its local leaf performs the Layer 3 lookup immediately instead of forwarding the packet to a centralized device. The distributed anycast gateway model on VyOS takes this approach one step further.

DCI in Practice

Layer 2 versus Layer 3 DCI

The DCI can carry Layer 2 or Layer 3 services, depending on application requirements and the EVPN architecture. Extending Layer 2 is useful when workloads must stay in the same IP subnet across data centers. However, stretching a broadcast domain across geographically separated sites enlarges the failure domain and introduces additional considerations regarding latency, BUM traffic, MAC mobility, and convergence.

Where applications do not specifically require Layer 2 adjacency, a Layer 3 DCI model is generally simpler to operate. Each data center keeps its local Layer 2 domains, while EVPN/BGP exchanges the required Layer 3 reachability between sites. This limits broadcast domains and creates clearer failure boundaries, a pattern also used in multi-tenant L3VPN EVPN networks on VyOS.

Resiliency and design considerations

• Redundancy: avoid dependency on a single VyOS DCI router or Internet path. Redundant DCI nodes and multiple upstream paths improve availability.
• Convergence: BGP provides dynamic path selection, and BFD can be added where appropriate for faster detection of transport failures.
• Gateway placement: distributed gateways keep routing close to workloads and support a consistent gateway model across participating leaf devices.
• Failure domains: prefer routed inter-site boundaries unless applications explicitly require stretched Layer 2 connectivity.

Key Takeaways

• VyOS supports Spine, Leaf, and Border Leaf roles in EVPN-VXLAN architectures.
• Layer 2 and Layer 3 services can be interconnected across multiple data centers over the Internet.
• BGP EVPN reduces reliance on data-plane learning with a scalable, deterministic control plane.
• Distributed routing keeps Layer 3 processing close to workloads and reduces hairpinning.
• Routed inter-site boundaries limit failure domains between data centers.
• No additional feature-based licensing is required.

Conclusion

VyOS provides a flexible, scalable platform for Data Center Interconnect, supporting Spine, Leaf, and Border Leaf roles in EVPN-VXLAN architectures. It enables secure Layer 2 and Layer 3 connectivity across multiple data centers using Internet-based DCI, standards-based networking, and routing segmentation, all without additional feature-based licensing.

To compare DCI transport options and communication methods in more depth, explore VyOS Data Center Interconnect solutions.

Interconnect your data centers with VyOS.

Planning an EVPN-VXLAN fabric or an Internet-based DCI and want a second opinion on the design?

Talk to our engineers.

Additional Links

• VyOS Solutions

• BGP L2VPN/EVPN support via VXLAN transport

• L3VPN EVPN with VyOS

• EVPN-VXLAN Enhancements: Single VXLAN Device Support