An academic from the University of Birmingham has developed a novel method for high accuracy Question Answering which is believed to be the most accurate in the world.
Question answering software automatically answers questions posed by users much like Siri, Alexa and Google Now. Harish Tayyar Madabushi, a PhD student in Computer Science, has integrated information extracted from the analysis of language with deep learning to achieve this result. Deep learning is a method commonly used by Artificial Intelligence (AI) software to learn new patterns in data.
While researchers in Natural Language Processing (NLP) have been working on this problem for over 20 years, data suggests that the software developed by PhD student Harish, with his supervisors Dr Mark Lee, Senior Lecturer in the School of Computer Science and Professor John Barnden, Professor of Artificial Intelligence is capable of the most accurate and advanced question answering so far.
The system is tested on hundreds of question and answer pairs. For example:
Question
How many years was Jack Welch with GE?
Sentence 1
"six sigma has galvanized our company with an intensity the likes of which I have never seen in my 40 years at GE, '' said Welch, chairman of General Electric.
Sentence 2
so fervent a proselytizer is Welch that GE has spent three years and more than $1 billion to convert all of its divisions to the six sigma faith
The task requires the system to distinguish the first sentence as containing the answer and not the second and is the industry standard for measuring question answering effectiveness. Finding sentences that might have the answer can be done using search engines, but finding those that definitively do is much harder. Using the industry standard, QA Answer Sentence Selection Dataset, to measure accuracy resulted in the highest score for any currently established question answering software.
A core component of the system is an innovative learning generalisation method the team calls ArCH-Learning, which is used to classify questions based on the kind of answers expected into over 50 different classes. For example, the question “Who is the actress in the movie Titanic?” requires the answer to be a person and so belongs to the class ‘Human: Individual’ - a task called question classification.
Harish Tayyar Madabushi said:
“What is exciting about this system is that it incorporates high-level linguistic knowledge with powerful data-driven deep learning techniques. This work shows that the decades of work that exists in linguistics can be directly integrated into deep learning systems to improve performance while also making such systems easier to understand and debug.”
Dr Mark Lee, Senior Lecturer in the School of Computer Science said:
"The growth of the Internet of Things (IoT) has made question answering systems ubiquitous at home and the workplace and will change how people interact with technology in the future.
Professor John Barnden, Professor of Artificial Intelligence in the School of Computer Science said:
“This research is a good example of how there is much more to AI than currently popular techniques for machine learning, powerful though the latter are in certain respects.”
For more information, please contact Dominic Benson, Deputy Director of Communications on or +44 (0)121 414 5134.
The University of Birmingham is ranked amongst the world’s top 100 institutions. Its work brings people from across the world to Birmingham, including researchers, teachers and more than 6,500 international students from over 150 countries.