Publication

Presto: SQL on Everything

IEEE International Conference on Data Engineering (ICDE)


Abstract

Presto is an open source distributed query engine that supports much of the SQL analytics workload at Facebook. Presto is designed to be adaptive, flexible, and extensible. It supports a wide variety of use cases with diverse characteristics. These range from user-facing reporting applications with sub-second latency requirements to multi-hour ETL jobs that aggregate or join terabytes of data. Presto’s Connector API allows plugins to provide a high performance I/O interface to dozens of data sources, including Hadoop data warehouses, RDBMSs, NoSQL systems, and stream processing systems. In this paper, we outline a selection of use cases that Presto supports at Facebook. We then describe its architecture and implementation, and call out features and performance optimizations that enable it to support these use cases. Finally, we present performance results that demonstrate the impact of our main design decisions.

Related Publications

All Publications

ACM SIGIR - July 11, 2021

From Producer Success to Retention: a New Role of Search and Recommendation Systems on Marketplaces

Viet Ha-Thuc, Matthew Wood, Yunli Liu, Jagadeesan Sundaresan

TSE - May 6, 2021

Comparative Analysis of Constraint Handling Techniques for Constrained Combinatorial Testing

Huayao Wu, Changhai Nie, Justyna Petke, Yue Jia, Mark Harman

Information and Inference: a Journal of the IMA - January 18, 2021

Secure multiparty computations in floating-point arithmetic

Chuan Guo, Awni Hannun, Brian Knott, Laurens van der Maaten, Mark Tygert, Ruiyu Zhu

EASE - May 10, 2021

Facebook’s Cyber–Cyber and Cyber–Physical Digital Twins

John Ahlgren, Kinga Bojarczuk, Sophia Drossopoulou, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Maria Lomeli, Simon Mark Lucas, Erik Meijer, Steve Omohundro, Rubmary Rojas, Silvia Sapora, Jie M. Zhang, Norm Zhou

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy