Multimodal perception

This group comprises a range of projects that are concerned with the coordinated use of multiple sensory systems.

Research interests

Multimodal object recognition

An individual’s mental processes during physical and social perception of a particular object or material in a reaction, act or description within a specific situation can be characterised by means of multiple discrimination scaling. Personal cognition encompasses verbal and other symbolic stimuli and the cultural meaning of physical stimuli - both manufactured (such as consumer products) and natural, such as facial expressions, gestures or body shapes and sensations from muscles, joints or the digestive tract. Applications of sensory with semantic perception include thirst, hunger and their satisfaction, chronic pain and fatigue (with or without physical pathology) and user-perceived functional quality of materials such as foods, drinks, medications and domestic goods. Basic work on complex textures of foods in collaboration with food chemists is currently supported by the BBSRC, within the Food Quality wing of the Food Quality & Nutritional Psychology Research Group. Software to support industrial research and training and consumer education is under development for the enABLEr/s system of the Archive Research Group (Health Psychology).

Vision and Other Senses

Research into the integration of vision and other senses is carried out in three projects: Professor Alan Wing and Professor Glyn Humphreys have a collaborative project with Marc Ernst to study vision and touch interactions in slope perception. Dr Andrew Schofield and Dr Paul Rock have recently completed the development of a new experimental system for testing interactions between vision and touch (pictured above). The equipment integrates an existing Phantom robot arm and ReachIN workstation with a BITS++ system for presenting calibrated visual stimuli. The system will enable the study of vision and touch using psychophysical methods and is currently being used to study the role of visual texture in shape from shading. Dr Andrew Welchman has funding from the BBSRC to study interactions between vision and touch and vision and hearing. 
Dr Max Di Luca has funding from Marie Curie to study temporal aspects of audiovisual interaction and from the Royal Society to study haptics.

Current grants

Sponsor Name Principal Investigator
EPSRC Andrew Schofield
EPSRC Andrew Welchman
EU Alan WingGlyn Humphreys
BBSRC David Booth
EU Max Di Luca 
Royal Society  Max Di Luca