Postdoctoral Researcher
Carnegie Mellon University
PhD '19 University of California, Berkeley
Areas of Interest
- Human-Computer Interaction
Poster
Describing Videos for Non-Visual Accessibility
Abstract
Online social and informational content is increasingly communicated via video rather than images or text. But, the visual content in videos is rarely described, presenting serious access barriers to blind and visually impaired users. My recent work seeks to make videos non-visually accessible through computational support for: (1) authoring audio descriptions that describe visual content in videos, (2) facilitating search for inherently non-visually accessible videos, and (3) providing real-time feedback to help authors make their videos accessible at capture time. This poster presents Rescribe, an interactive tool for authoring and automatically editing audio descriptions. Rescribe takes a user’s first-draft of an audio description and then uses an optimization algorithm that considers both text shortening, and audio track lengthening approaches to fit the audio description into gaps in the video speech. Authors can iteratively visualize and refine the audio descriptions produced by Rescribe, working in concert with the tool. Our evaluations demonstrate that Rescribe helps novice describers produce error-free descriptions more efficiently than traditional tools, and produces audio descriptions that are preferred by blind users to unedited descriptions. Professional audio describers wanted to incorporate Rescribe it into their workflow and identified new opportunities for automation.
Bio
Amy Pavel is currently a Postdoctoral Fellow at Carnegie Mellon University and a Research Scientist at Apple in the Machine Intelligence group. Her research builds and studies interactive tools, augmented with natural language processing, that support people in exploring and understanding videos using text. Her work has been published in top HCI conferences such as UIST, CHI, and the top venue for accessible technology, ASSETS. Previously, she completed her PhD in the EECS department at UC Berkeley advised by Maneesh Agrawala and Bjoern Hartmann. Her dissertation research was supported by an NDSEG Fellowship.
