I am an Engineering Manager supporting the XLDB team. The XLDB team is the structured data storage in Data Infrastructure. We read and write 20EB of data enabling Analytics, ML and Ads across FB.
We have a lot of challenges ahead of us to support the ginormous growth of data coming over in the next few year. In order to support this we have published our vision https://fb.workplace.com/notes/682432615742336 based on adaptability. I joke about the fact that we are breaking the CAP theorem, and we do it by using ML to derive if we provide an AP or CP set of characteristics.
Data in general