Research from Meta

All Publications

June 19, 2023
Siying Dong, Satadru Pan, Abhinav Sharma, Albert Kim, Anand Ananthabhotla, Dhanabal Ekambaram, Jay Zhuang, Nishant Vinaybhai Parikh, Sam Dunster, Shiva Shankar P, Shobhit Dayal, Sushil Patil, Yanqin Jin, Akanksha Mahajan, Anirudh Chelluri, Chaitanya Datye, Lucas Vasconcelos Santana, Nitin Garg, Omkar Gawde

RocksDB is an open-source storage engine widely used inside and outside of Meta. Historically, it was mainly used for storing data on local SSDs.

January 9, 2023
Biswapesh Chattopadhyay, Pedro Pedreira, Sameer Agarwal, Yutian James Sun, Suketu Vakharia, Peng Li, Weiran Liu, Sundaram Narayanan

This paper describes how we transformed the legacy data lakehouse stack at Meta to adapt to the new realities through a large cross-organizational effort called Shared Foundations.

Areas
August 31, 2022
Pedro Pedreira, Orri Erling, Masha Basmanova, Kevin Wilfong, Laith Sakka, Krishna Pai, Wei He, Biswapesh Chattopadhyay

Velox provides reusable, extensible, high-performance, and dialect-agnostic data processing components for building execution engines, and enhancing data management systems.

Areas
June 18, 2022
Mark Zhao, Niket Agarwal, Aarti Basant, Buğra Gedik, Satadru Pan, Mustafa Ozdal, Rakesh Komuravelli, Jerry Pan, Tianshu Bao, Haowei Lu, Sundaram Narayanan, Jack Langman, Kevin Wilfong, Harsha Rastogi, Carole-Jean Wu, Christos Kozyrakis, Parik Pol

This paper presents Meta’s end-to-end DSI pipeline, composed of a central data warehouse built on distributed storage and a Data PreProcessing Service that scales to eliminate data stalls.