Passive Realtime Datacenter Fault Detection and Localization

USENIX Symposium on Networked Systems Design and Implementation (NSDI)


Datacenters are characterized by their large scale, comprising a large number of network links and switches. However, these hardware components can develop intermittent faults, resulting in randomly occurring packet drops or delays that harm application performance—several such faults occur daily in large production datacenters. Since the effects are intermittent, traditional detection techniques involving end-host and router statistics or active probe traffic can fall short in their ability to identify and locate these errors. In this article, we present our passive hybrid approach that combines network path information with end-host-based statistics to rapidly detect and pinpoint the location of datacenter network faults inside a production Facebook datacenter.

Related Publications

All Publications

OFC - July 9, 2021

BOW: First Real-World Demonstration of a Bayesian Optimization System for Wavelength Reconfiguration

Zhizhen Zhong, Manya Ghobadi, Maximilian Balandat, Sanjeevkumar Katti, Abbas Kazerouni, Jonathan Leach, Mark McKillop, Ying Zhang

ICASSP - June 11, 2021

Optimizing Coverage and Capacity in Cellular Networks using Machine Learning

Ryan M. Dreifuerst, Samuel Daulton, Yuchen Qian, Paul Parayil Varkey, Maximilian Balandat, Sanjay Kasturia, Anoop Tomar, Ali Yazdan Panah, Vish Ponnampalam, Robert W. Heath Jr

IEEE International Workshop on Spectrum Sharing Technology for Next Generation Communications - June 14, 2021

A Configurable 60GHz Phased Array Platform for Multi-Link mmWave Channel Characterization

Anton Shkel, Alireza Mehrabani, Julius Kusuma

ISCA - June 14, 2021

Enabling Compute-Communication Overlap in Distributed Deep Learning Training Platforms

Saeed Rashidi, Matthew Denton, Srinivas Sridharan, Sudarshan Srinivasan, Amoghavarsha Suresh, Jade Nie, Tushar Krishna

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy