Research Area
Year Published

976 Results

July 1, 2011

Many-core key-value store

International Green Computing Conference (IGCC)

Scaling data centers to handle task-parallel workloads requires balancing the cost of hardware, operations, and power. Low-power, low-core-count servers reduce costs in one of these dimensions, but may require additional nodes to provide the required quality of service or increase costs by underutilizing memory and other resources.

By: Mateusz Berezecki, Eitan Frachtenberg, Michael Paleczny, Ken Steele

June 20, 2011

YSmart: Yet Another SQL-to-MapReduce Translator

International Conference on Distributed Computing Systems (ICDCS)

MapReduce has become an effective approach to big data analytics in large cluster systems, where SQL-like queries play important roles to interface between users and systems. However, based on our Face book daily operation results, certain types of queries are executed at an unacceptable low speed by Hive (a production SQL-to-MapReduce translator). In this paper, we demonstrate that existing SQL-to-MapReduce translators that operate in a one-operation-to-one-job mode and do not consider query correlations cannot generate high-performance MapReduce programs for certain queries, due to the mismatch between complex SQL structures and simple MapReduce framework. We propose and develop a system called Y Smart, a correlation aware SQL-to-MapReduce translator. Y Smart applies a set of rules to use the minimal number of MapReduce jobs to execute multiple correlated operations in a complex query. Y Smart can significantly reduce redundant computations, I/O operations and network transfers compared to existing translators. We have implemented Y Smart with intensive evaluation for complex queries on two Amazon EC2 clusters and one Face book production cluster. The results show that Y Smart can outperform Hive and Pig, two widely used SQL-to-MapReduce translators, by more than four times for query execution.

By: Rubao Lee, Tian Luo, Yin Huai, Fusheng Wang, Yongqiang He, Xiaodong Zhang

June 12, 2011

Apache Hadoop goes realtime at Facebook

ACM Special Interest Group on Management of Data (SIGMOD)

Facebook recently deployed Facebook Messages, its first ever user-facing application built on the Apache Hadoop platform. Apache HBase is a database-like layer built on Hadoop designed to support billions of messages per day.

By: Dhruba Borthakur, Joydeep Sen Sarma, Jonathan Gray, Kannan Muthukkaruppan, Nicolas Spiegelberg, Hairong Kuang, Karthik Ranganathan, Dmytro Molkov, Aravind Menon, Samuel Rash, Rodrigo Schmidt, Amitanand Aiyer

April 10, 2011

Facebook Immune System

Workshop on Social Network Systems (SNS)

Popular Internet sites are under attack all the time from phishers, fraudsters, and spammers. They aim to steal user information and expose users to unwanted spam. The attackers have vast resources at their disposal. They are well-funded, with full-time skilled labor, control over compromised and infected accounts, and access to global botnets.

By: Tao Stein, Roger Chen, Karan Mangla

March 30, 2011

FATE and DESTINI: A Framework for Cloud Recovery Testing

USENIX Symposium on Networked Systems Design and Implementation (NSDI)

As the cloud era begins and failures become commonplace, the fate and destiny of availability, reliability and performance are in the hands of failure recovery. Unfortunately, recovery problems still take place, causing downtimes, data loss, and many other problems.

By: Haryadi S. Gunawi, Thanh Do, Pallavi Joshi, Peter Alvaro, Joseph M. Hellerstein, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, Koushik Sen, Dhruba Borthakur

February 1, 2011

Social Capital on Facebook: Differentiating Uses and Users

ACM Conference on Human Factors in Computing Systems (CHI)

Though social network site use is often treated as a monolithic activity, in which all time is equally “social” and its impact the same for all users, we examine how Facebook affects social capital depending upon: (1) types of site activities, contrasting one-on-one communication, broadcasts to wider audiences, and passive consumption of social news, and (2) individual differences among users, including social communication skill and self-esteem.

By: Moira Burke, Robert Kraut, Cameron Marlow

January 1, 2011

RCFile: A Fast and Space-efficient Data Placement Structure in MapReduce-based Warehouse Systems

IEEE International Conference on Data Engineering (ICDE)

MapReduce-based data warehouse systems are playing important roles of supporting big data analytics to understand quickly the dynamics of user behavior trends and their needs in typical Web service providers and social network sites (e.g., Facebook). In such a system, the data placement structure is a critical factor that can affect the warehouse performance in a fundamental way.

By: Yongqiang He, Rubao Lee, Yin Huai, Zheng Shao, Namit Jain, Xiaodong Zhang, Zhiwei Xu

January 1, 2011

Supervised Random Walks: Predicting and Recommending Links in Social Networks

ACM International Conference on Web Search and Data Mining (WSDM)

Predicting the occurrence of links is a fundamental problem in networks. In the link prediction problem we are given a snapshot of a network and would like to infer which interactions among existing members are likely to occur in the near future or which existing interactions are we missing. Although this problem has been extensively studied, the challenge of how to effectively combine the information from the network structure with rich node and edge attribute data remains largely open.

By: Lars Backstrom, Jure Leskovec

January 1, 2011

Network Bucket Testing

International World Wide Web Conference (WWW)

Bucket testing, also known as A/B testing, is a practice that is widely used by on-line sites with large audiences: in a simple version of the methodology, one evaluates a new feature on the site by e…

By: Lars Backstrom, Jure Leskovec

October 4, 2010

Finding a needle in Haystack: Facebook’s photo storage

USENIX Symposium on Operating Systems Design and Implementation (OSDI)

This paper describes Haystack, an object storage system optimized for Facebook’s Photos application. Facebook currently stores over 260 billion images, which translates to over 20 petabytes of data. U…

By: Doug Beaver, Sanjeev Kumar, Harry Li, Jason Sobel, Peter Vajgel