Pawluk awarded $1.1M to develop a programming environment for the visually impaired

As computing has become integral to the practice of science, technology, engineering and mathematics (STEM), the STEM+Computing program seeks to advance applied research integrating computational thinking and computing activities within disciplinary science, technology, engineering, and mathematics teaching and learning in early childhood education through high school (preK-12). There is currently no accessibility solution that allows blind and visually impaired learners use of the new highly visual programming languages that are ubiquitous and which are specifically designed to improve sighted students' engagement and ease of learning computer programming.

The aim of this proposed project, by Virginia Commonwealth University and Smith-Kettlewell Institute, is to provide a new interface "tSCRATCH" for the programming environment to enable visually impaired students to be engaged in computation and computational thinking. This work has the potential to improve learning of computer programming for all students, but will particularly focus on providing equal and shared access to individuals who are visually impaired. The learning approach is to develop a non-visual interface for a learning application, Scratch, that takes into account the strengths and weaknesses of the haptic (physical) and auditory systems. The project will be evaluated by Praxis Associates, who will provide ongoing feedback on the project to the project team.

This work has the potential to improve learning of computer programming for all students, but it will focus on providing access to individuals who are visually impaired. The approach is to develop a tangible, non-visual interface for Scratch that takes into account the strengths and weaknesses of the haptic and auditory systems. It will adapt a low-cost tactile tile concept to represent program commands, along with a grid surface and other organizational structures, to ensure that programs of moderate complexity can be created. A key aspect of the approach is to allow users to manipulate commands as a group to enable chunking (i.e., grouping into a whole) of information for ease of manipulation and cognitive understanding. Multi-media instructional material will be developed by experts in teaching students computer programming who are visually impaired. Further, this intervention will explore the possibility of tScratch to be used alongside of Scratch in mainstream classrooms by the use of the tScratch prototype in a computer camp type setting. The assessment of the intervention will consider the change in both blind and sighted students' ability to apply programming concepts, share in teamwork, views of each other, and views of computer science.

Dr. Dianne Pawluk is the Director of the Rehabilitation Technology and Haptics Laboratory housed in the Department of Biomedical Engineering in the College of Engineering.