Publication

Running BGP in Data Centers at Scale

USENIX Symposium on Networked Systems Design and Implementation (NSDI)


Abstract

Border Gateway Protocol (BGP) forms the foundation for routing in the Internet. More recently, BGP has made serious inroads into data centers on account of its scalability, extensive policy control, and proven track record of running the Internet for a few decades. Data center operators are known to use BGP for routing, often in different ways. Yet, because data center requirements are very different from the Internet, it is not straightforward to use BGP to achieve effective data center routing.

In this paper, we present Facebook’s BGP-based data center routing design and how it marries data center’s stringent requirements with BGP’s functionality. We present the design’s significant artifacts, including the BGP Autonomous System Number (ASN) allocation, route summarization, and our sophisticated BGP policy set. We demonstrate how this design provides us with flexible control over routing and keeps the network reliable. We also describe our in-house BGP software implementation, and its testing and deployment pipelines. These allow us to treat BGP like any other software component, enabling fast incremental updates. Finally, we share our operational experience in running BGP and specifically shed light on critical incidents over two years across our data center fleet. We describe how those influenced our current and ongoing routing design and operation.

Read blog

Related Publications

All Publications

OFC - July 9, 2021

BOW: First Real-World Demonstration of a Bayesian Optimization System for Wavelength Reconfiguration

Zhizhen Zhong, Manya Ghobadi, Maximilian Balandat, Sanjeevkumar Katti, Abbas Kazerouni, Jonathan Leach, Mark McKillop, Ying Zhang

ICASSP - June 11, 2021

Optimizing Coverage and Capacity in Cellular Networks using Machine Learning

Ryan M. Dreifuerst, Samuel Daulton, Yuchen Qian, Paul Parayil Varkey, Maximilian Balandat, Sanjay Kasturia, Anoop Tomar, Ali Yazdan Panah, Vish Ponnampalam, Robert W. Heath Jr

IEEE International Workshop on Spectrum Sharing Technology for Next Generation Communications - June 14, 2021

A Configurable 60GHz Phased Array Platform for Multi-Link mmWave Channel Characterization

Anton Shkel, Alireza Mehrabani, Julius Kusuma

ISCA - June 14, 2021

Enabling Compute-Communication Overlap in Distributed Deep Learning Training Platforms

Saeed Rashidi, Matthew Denton, Srinivas Sridharan, Sudarshan Srinivasan, Amoghavarsha Suresh, Jade Nie, Tushar Krishna

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy