Publication

A Face Recognition Application for People with Visual Impairments: Understanding Use Beyond the Lab

Conference on Human Factors in Computing Systems (CHI)


Abstract

Recognizing others is a major challenge for people with visual impairments (VIPs) and can hinder their social engagement. We present Accessibility Bot, a research prototype bot on Facebook Messenger, which leverages state-of-the-art computer vision algorithms and the existing set of tagged photos of a user’s friends on Facebook to help people with visually impairments recognize their friends. Accessibility Bot provides users information about the identity of friends in camera and their facial expressions and attributes. To guide our design, we interviewed eight VIPs to understand their challenges and needs in social activities. We then conducted a diary study with six VIPs to study the use of Accessibility Bot in everyday life. While most participants found the Bot helpful, their experience was undermined by perceived low recognition accuracy, difficulty aiming a camera, and lack of knowledge about the phone’s status. We discuss these real-world challenges, identify suitable use cases for Accessibility Bot, and distill design implications for future face recognition applications.

Related Publications

All Publications

DSN - June 21, 2021

Near-Realtime Server Reboot Monitoring and Root Cause Analysis in a Large-Scale System

Fred Lin, Bhargav Bolla, Eric Pinkham, Neil Kodner, Daniel Moore, Amol Desai, Sriram Sankar

arXiv - September 23, 2020

Neural Relational Autoregression for High-Resolution COVID-19 Forecasting

Matthew Le, Mark Ibrahim, Levent Sagun, Timothée Lacroix, Maximilian Nickel

ACM SIGIR - July 11, 2021

From Producer Success to Retention: a New Role of Search and Recommendation Systems on Marketplaces

Viet Ha-Thuc, Matthew Wood, Yunli Liu, Jagadeesan Sundaresan

Information and Inference: a Journal of the IMA - January 18, 2021

Secure multiparty computations in floating-point arithmetic

Chuan Guo, Awni Hannun, Brian Knott, Laurens van der Maaten, Mark Tygert, Ruiyu Zhu

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy