Pawluk awarded $300K to develop desktop robot to assist the blind and visually impaired

In the U.S. alone, there are approximately 8 million individuals who are blind or visually impaired. In order for these individuals to reach their potential and be competitive with their peers, it is important that they have equivalent access to information to ensure timely completion of education and work tasks. Computers have become essential in these environments, creating information and collaborating with others, whether in the same room or remotely over the web. A lot of this interaction is highly spatial in nature, whether in the use and creation of graphics, interacting with others through a mouse or cursor, or searching for information in the spatial layout of text on a web page. For individuals who are blind or visually impaired, this spatial information is largely inaccessible. This project is intended to address this issue by developing a robotic assistant to provide haptic and Braille input and output to a computer, as well as aid an individual in exploration of the computer window.

The main goal of this grant is to provide a coherent haptic system that provides effective input and output of graphics and Braille on a full haptic page/window through a single device that can be used in a single hand posture. The objectives are to first develop a desktop co-robot to provide shared control with the user over a haptic interface to aid in exploring graphical and full page Braille information. A mouse-shaped tactile input/output component will be mounted on the co-robot to provide spatially distributed tactile feedback to the second and third fingers of the hand. Combined with a stationary version of the mouse, this will also allow simultaneous input of Braille text through button inputs and, in the future, the creation of graphical information. In developing this system, we will use a Participatory Action Design approach involving all stakeholders. The created system will then be validated in accessing graphical and full page text information, as well as editing full page text compared to current commercial and/or research devices. In addition, this system will be furthered by the development of techniques to effectively communicate the focus of attention between the user and collaborators.

Dianne Pawluk, Ph.D., is the Director of the Rehabilitation Technology and Haptics Laboratory housed in the Department of Biomedical Engineering in the College of Engineering.