Efficient Measurement of Quality at Scale in Facebook Video Ecosystem

SPIE Optics + Photonics


This paper describes FB-MOS, a metric that is used to measure video quality at scale in FB Video Ecosystem. Facebook processes a very large number of videos daily that collectively receive billions of views each day and hence both the accuracy and computational complexity of the metric are equally important. As the quality of uploaded user-generated content (UGC) source itself varies widely, FB-MOS consists of both a no-reference metric component to assess input (upload) quality and a full-reference component to assess quality preserved in the transcoding and delivery pipeline. FB videos can be watched on a variety of devices (Mobile/Laptop/TV) in varying network conditions, and often switched between in-line view and full-screen view during the same viewing session. We show how FB-MOS metric can accurately account for all this variation in viewing condition while minimizing the computation overhead to offer such measurement. We also discuss how this metric allows for end-to-end quality monitoring at scale, as well as guide encoding and delivery optimizations. The paper also discusses some of the optimizations to enable its use to achieve real-time quality measurement for Live videos. Another aspect of the Facebook video product is the wide variation in popularity of videos where less popular UGC content may receive relative very few views while highly watched professional or viral UGC content can receive millions of views. We discuss how the computational overhead of the metric can scale with the popularity of video where more compute is expended on more popular videos to get more accurate metrics while spending less compute on less popular videos.

Related Publications

All Publications

The CacheLib Caching Engine: Design and Experiences at Scale

Benjamin Berg, Daniel S. Berger, Sara McAllister, Isaac Grosof, Sathya Gunasekar, Jimmy Lu, Michael Uhlar, Jim Carrig, Nathan Beckmann, Mor Harchol-Balter, Gregory G. Ganger

OSDI - November 4, 2020

Virtual Consensus in Delos

Mahesh Balakrishnan, Jason Flinn, Chen Shen, Mihir Dharamshi, Ahmed Jafri, Xiao Shi, Santosh Ghosh, Hazem Hassan, Aaryaman Sagar, Rhed Shi, Jingming Liu, Filip Gruszczynski, Xianan Zhang, Huy Hoang, Ahmed Yossef, Francois Richard, Yee Jiun Song

OSDI - November 4, 2020

Twine: A Unified Cluster Management System for Shared Infrastructure

Chunqiang (CQ) Tang, Kenny Yu, Kaushik Veeraraghavan, Jonathan Kaldor, Scott Michelson, Thawan Kooburat, Aravind Anbudurai, Matthew Clark, Kabir Gogia, Long Cheng, Ben Christensen, Alex Gartrell, Maxim Khutornenko, Sachin Kulkarni, Marcin Pawlowski, Tuomas Pelkonen, Andre Rodrigues, Rounak Tibrewal, Vaishnavi Venkatesan, Peter Zhang

OSDI - November 4, 2020

FlightTracker: Consistency across Read-Optimized Online Stores at Facebook

Xiao Shi, Scott Pruett, Kevin Doherty, Jinyu Han, Dmitri Petrov, Jim Carrig, John Hugg, Nathan Bronson

OSDI - November 4, 2020

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy