The Echoes project has developed a technology enhanced learning that combines interactive multitouch screens, gaze tracking, and intelligent agent-based context-sensitive interfaces to create a novel, interactive, and multi-modal environment that can be adapted to the needs of specific individuals, and that can provide new ways of investigating and supporting the development of social skills in children.
ECHOES is a game-like learning environment. It can adapt itself to the needs of specific children, and provides new ways of supporting the development of their social understanding and communication. It allows them to explore a 'magic garden' and to play with a child-like character called Andy. Andy gives the child instructions and can also respond to the child in a way that encourages the child to develop joint attention. Andy does this by explicitly supporting the child to follow his gaze through playful activities, with exciting objects in a garden-like environment. The activities approximate the ‘real-world’, but they remove some of the unpredictability of the real world and some of the threats. The child is offered opportunities to repeat actions in order to practise them, and to try out new actions, thus engendering an awareness of alternatives, and resulting consequences. Andy communicates with the child through simple language and through gestures, such as pointing, gaze and body language.
COSPATIAL is a 3 year project (2009-2012) funded by the Seventh Framework Programme of the European Commission. The project investigates the use of collaborative technologies to support the teaching of social skills for children with autism spectrum conditions.
Several educational applications have been developed by COSPATIAL project partners at the University of Nottingham and Fondazione Bruno Kessler in Italy in partnership with the University of Southampton, University of Haifa and Bar Ilan University (Israel). A team from the University of Nottingham will be at the ESRC Festival of Science to present the applications with Dr Sarah Parsons and her team from the University of Southampton.
The applications use novel technologies to support a variety of tasks promoting cooperation and collaboration. They have been developed using a participatory design approach, where teachers, therapists, experts and children have all been involved regularly in the creation and refinement of the programmes. More information...
Somantics and ReacTickles
Somantics is a project in development from the award-winning ReacTickles team. Somantics will be at a proof of concept stage by the end of October 2011. The Somantics is a suite of applications that use touch, gesture and camera input to encourage, capture and amplify the interests of young people with Autistic Spectrum Conditions and other related communication difficulties. The overarching goal of Somantics is promote greater self-awareness, confidence and independence.
Somantics will be made available for the iPad, iPad2, and Mac platforms. PC and Linux versions are currently in development. An iPad2 will be required for camera based interaction, iPad1 users may still download and use the touch based Somantics. Desktop and laptop machines require a Microsoft Kinect or compatible device to function.
Interaction with Somantics is repetitious, flowing and highly expresssive. The applications are non-competitive; users discover their own purpose. The Somantics interface enables users choose an application and to exlpore with little, if any, assistance from others. Independence will increase through further exploration of the system.
Somantics has been developed by a research and design team with many years experience of developing interfaces that help develop creativity. We have been undertaking some formative studies with students at opposite ends of the autism spectrum, spanning ages 10-22 years. High functioning students with Asbergers Syndrome have been fundamental to design and developmentally lower functioning students have been trying out the prototypes. Results of the studies have contributed new ideas to the project and refined existing ones. From our observations we have noted some interesting changes in social behaviour when young people interact with Somantics. This may be down to the fact that Somantics sets boundaries rather than rules. Most notably the spatial boundaries of the system are discoverable through movement. Within these boundaries social interaction has been both rewarding and "appropriate". There have been no invasions of personal space or other anti social behaviours, which, according to teaching staff, is quite common with the students we have worked with. For example:
- When Somantics is projected it becomes a performance space - students have been able to test the limits of the space and explore interaction within it.
Using kinect as a motion sensor enables the student to explore proximity, for example going close to the wall has no effect, a short proximal distance is needed for visual reward.
- As the action is real-time students also detect the impact of speed - slower movements are easier to observe. Erratic and uncontrolled movement reduces potential for reward.
- The large projection space means that watching others performs becomes fascinating. While mirroring is widely understood as important for social interaction, a person needs to concentrate on watching for long enough in order to accomplish mirroring. The ability to observe the real world actions and reactions of others, either through video or graphic representation, is being evidenced as a valuable attribute of the system and levels of concentration have been much higher than in other settings.
ReacTickles are experiential interfaces, by this we suggest that they have no obvious meaning or purpose until the player interacts. Interaction disturbs the ReacTickle, bringing it to life, giving it energy. They can be explored on many everyday devices: SmartBoard, keyboard, mouse, microphone and even SmartTable. Currently in development are iPhone, iPad and motion sensor versions.
ReacTickles are fresh, novel, surprising, joyous, relaxing, funny, energetic, cheeky and enduring. The simplicity of their appearance make ReacTickles irresistible and open to interpretation and purpose. The main function of a ReacTickle is to trigger curiosity and to reward interaction with positive emotional experiences, for many people this will be the key to unlocking potential. Thie design of ReacTickles is therefore based on shapes and colours that are naturally stimulating but that do require an unnecessary level of cognitive processing and compliance.