Publication

A Face Recognition Application for People with Visual Impairments: Understanding Use Beyond the Lab

Conference on Human Factors in Computing Systems (CHI)


Abstract

Recognizing others is a major challenge for people with visual impairments (VIPs) and can hinder their social engagement. We present Accessibility Bot, a research prototype bot on Facebook Messenger, which leverages state-of-the-art computer vision algorithms and the existing set of tagged photos of a user’s friends on Facebook to help people with visually impairments recognize their friends. Accessibility Bot provides users information about the identity of friends in camera and their facial expressions and attributes. To guide our design, we interviewed eight VIPs to understand their challenges and needs in social activities. We then conducted a diary study with six VIPs to study the use of Accessibility Bot in everyday life. While most participants found the Bot helpful, their experience was undermined by perceived low recognition accuracy, difficulty aiming a camera, and lack of knowledge about the phone’s status. We discuss these real-world challenges, identify suitable use cases for Accessibility Bot, and distill design implications for future face recognition applications.

Related Publications

All Publications

Journal of Big Data - July 19, 2021

Cumulative deviation of a subpopulation from the full population

Mark Tygert

CSCW - November 3, 2020

The dynamics of U.S. college network formation on Facebook

Jan Overgoor, Bogdan State, Lada Adamic

PLOS ONE Journal - September 8, 2021

Large-scale decrease in the social salience of climate change during the COVID-19 pandemic

Brian R. Spisak, Bogdan State, Ingrid van de Leemput, Marten Scheffer, Yuwei Liu

AISTATS - August 31, 2021

Causal Autoregressive Flows

Ilyes Khemakhem, Ricardo P. Monti, Robert Leech, Aapo Hyvärinen

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy