Seeing with sound - Blast from the past

In a recent restructuring at the University of Northampton both the areas of Computing and Engineering joined the larger Faculty of Arts, Science and Technology (FAST). This is not the first time though that the computing technology has used by members the University's computing or engineering staff to provide social benefit, but with links to areas of visual and audio creativity .

In June 2000 Michael Capp, under the supervision of Professors Phil Picton and Jackie Campbell, completed his PhD  "Alternatives Approaches to Optophonic Mappings" 

Abstract
This thesis presents a number of modifications to a blind aid, known as the video optophone, which enables a blind user to more readily interpret their local environment for enhanced mobility and navigation. Versions o f this form of blind aid are generally both difficult to use and interpret, and are therefore inadequate for safe mobility. The reason for this severe problem lies in the complexity and excessive bandwidth of the optophonic output after the conversion from scene-to-sound. 

The work herein describes a number of modifications that can be applied to the current optophonic process to make more efficient use of the limited bandwidth provided by the auditory system when converting scene images to sound. Various image processing and stereo techniques have been employed to artificially emulate the human visual system through the use o f depth maps that successfully fade out the quantity o f relatively unimportant image features, whilst emphasising the more significant regions such as nearby obstacles. 

A series of experiments were designed to test these various modifications to the optophonic mapping by studying important factors of mobility and subject response whilst going about everyday life. The devised system, labelled DeLIA for the Detection, Location, Identification, and Avoidance (or Action) of obstacles, provided a means for gathering statistical data on users’ interpretation of the optophonic output. An analysis o f this data demonstrated a significant improvement when using the stereo cartooning technique, developed as part of this work, over the more conventional plain image as an input to an optophonic mapping from scene-to-sound. Lastly, conclusions were drawn from the results, which indicated that the use of a stereo depth map as an input to a video optophone would improve its usefulness as an aid to general mobility. For the purposes of detecting and determining text or similar detail, either a plain unmodified image or some form of edge (depth) image were found to produce the best results.

Published Outputs
[CapPicOOa] - Capp, M. and Picton, P., (Feb.-Mar. 2000). ‘Fast, Low Resolution Edge Depth Maps and their Application to a Blind Mobility Aid’, International Conference on Computer Vision, Pattern Recognition and Image Processing, Atlantic City, USA, C V P R IP -12, pp. 248-251. 
[CapPicOOb] - Capp, M., & Picton, P., (Jun. 2000). “The Optophone: an Electronic Blind Aid.” Engineering Science and Education Journal, vol. 9, no. 3, pp. 137-143. 
[CapPicOOc] - Capp, M. & Picton, P., (Aug. 2000). ‘An investigation into stereo vision as a modification to optophonic mappings from scene-to-sound.’ l0th International Mobility Conference, Warwick, 4th-7th Aug. 2000. 
[CapPicOOd] - Capp, M. & Picton, P., ‘Relaying Scene Information to the Blind via Sound using Cartoon Depth Maps.’ - Vision, Image and Signal Processing, 2000.




To find out more contact Phil Picton phil.picton@northampton.ac.uk


All views and opinions are the author's and do not necessarily reflected those of any organisation they are associated with. Twitter: @scottturneruon