Assessing the usability of raw machine translation output via eye tracking

Lecture Room 3 - Arts Building
Arts and Law
Tuesday 5th March 2013 (16:15-17:30)
Download the date to your calendar (.ics file)


Professional translators and translator trainers currently face an interesting challenge with the recent relative success of machine translation (MT - automatic translation by computer). In a domain that is heavily focused on quality, machine translation causes considerable concern for the translation profession. Yet, we have witnessed a considerable uptake of MT both by casual online users and by large translation clients, especially those in the IT domain. This has sparked research into the impact of MT on users, but the focus to date has been on the translator as ‘user’, with research focusing on productivity, quality and measurements of cognitive load during MT post-editing. To date, there has been very little focus on the impact of machine translated text on the actual end users of (machine) translated documentation.

During this talk I will present research that seeks to measure the usability of raw machine translated text compared with the original text, written in English. Our understanding of usability is guided by the ISO/TR 16982 definition which breaks the concept into four variables: goal completion, satisfaction, effectiveness and efficiency. We use both eye tracking and a post-task questionnaire to measure usability across four target languages (Spanish, French, German and Japanese) and one source language (English).

Dr. Sharon O’Brien,
Centre for Next Generation Localisation
Centre for Translation and Textual Studies
School of Applied Language and Intercultural Studies
Dublin City University

This research is supported by the Science Foundation Ireland (Grant 07/CE/I1142) as part of the Centre for Next Generation Localisation ( at Dublin City University and was conducted in collaboration with Dr. Stephen Doherty.

Sharon O'Brien is a lecturer in translation and language technology in the School of Applied Language and Intercultural Studies, Dublin City University. She is interested in many aspects of translating and interpreting, but her research to date has focussed on the interaction between translators and technology (including Translation Memory and Machine Translation), cognitive aspects of translation, research methods, including eye tracking and keyboard logging, localisation and content authoring. She is the current Director of the Centre for Translation and Textual Studies ( and and PI in the Centre for Next Generation Localisation (

About the English Language research seminars

  • Convener: Neil Millar

The ELR seminars are a long-running weekly research seminar series within the Department of English, English Language and Applied Linguistics Division. The seminars are aimed primarily at staff, postgraduate students, and academic visitors in the department, but everyone with an interest in language research is welcome! Seminars are usually held on Tuesdays during term time, starting at 4:15 and finishing around 5:30.