Professor Judith Smith Director, Health Services Management Centre, University of Birmingham
In UK universities, the assessment of research quality and effectiveness – the Research Excellence Framework (REF) – is increasingly focusing on research impact alongside the more traditional appraisal via publications in the top-ranking academic journals. As a recent returnee to university life (after seven years working in a charitable research foundation), I am struck by how many academics seem to find it easier to work out how to do the publications part of REF, and appear more reticent or even puzzled about how to achieve and demonstrate 'research impact'.
As evidence of the increased attention being given to research impact, Health Services Research UK (whose mission is focused on linking health services research with policy and practice) organised their recent autumn symposium at Swansea University with a programme that sought to answer the following questions: What is research impact? How do you achieve it? And how do you measure it?
There was clear agreement from the outset of the HSRUK symposium that research impact is an implicit duty for researchers, not 'something that is nice to do if we have some time left after completing the study'. Participants pointed out that millions of pounds of taxpayers' funds are entrusted to us, in the expectation that we will deliver impact (in our case on the quality and effectiveness of health services delivery and policy) for the public good.
Steve Hanney of Brunel and Vicky Ward of Leeds set the scene at the start of the symposium. Steve explained the history of policy and government attention to research impact, and in particular the concern with ‘research payback’ in the NHS, something he has researched for over 20 years. He pointed out that there are all sorts of research impact – e.g. guidelines, policy, new service delivery approach, change in the law - at different levels (international, national, local) and some may be about health gain, others improved equity or productivity, and others about improving care quality and patient experience.
Steve also set out the six steps to impactful research, from topic identification through research processes to dissemination, adoption and final outcomes. Important issues included collaborative agenda setting with finders and commissioners/likely end users, early presentation of findings to such customers, and multitude of ways to have impact. However, impact was concluded by Steve to be more than anything about reach, and significance – as indeed is expected within REF impact case studies.
Vicky Ward explored the tricky landscape of knowledge mobilisation (a term agreed to be clumsy and meaning little to those outside the research world), pointing to the fact that this is about making sure knowledge is moved to where it can be most useful. She emphasised the need to triangulate different sorts of knowledge, including science, practical wisdom (that which is gained be professionals through years of experience and practice – and sometimes under-valued in an era of guidelines and regulation), and technical skills. Vicky highlighted some of the many ways in which such knowledge from research can be translated or handed on to others, for example discussions, workshops, databases, learning networks, briefing papers, blogs and articles in the professional as well as academic press.
In discussion, participants at the symposium reflected that research and its impact are too often viewed as linear - one stage after the other across the research project plan, but in reality this has to be messy and iterative if to be effective. Thus the importance of researchers having multiple collaborations with funders, policy makers, the relevant community of practice (e.g. health services management, a clinical service, royal medical college) so that they are iterating research questions and ultimate impact throughout the course of their projects and wider programmes of study.
We then examined a case study of research impact – 'reducing unnecessary attendance at hospital emergency departments by improving care out of hospital' - with Professor Helen Snooks of Swansea University being interviewed by the chair and audience about her experience of crafting, evidencing, and submitting this to REF 2014. The questions came thick and fast, and Helen drew out the following messages:
- an impact case study can be about a longer term body of work beyond individual projects
- it is all about being able to tell clearly and simply the story of the work, its purpose, nature and impact
- you need to collect evidence constantly over the years with a focus on specific information about how practice or policy are known to have changed as a result of the research, with, if possible, evidence of improved outcomes for patients, practitioners, the wider population or the economy
- Information can include mentions in the media, citations in policy guidance, changed clinical protocols, letters from policy makers or politicians
- seeking testimonials from policy makers or leaders in the world of practice is a risky and uncomfortable process, but can be very helpful
- we should all build in measurement of impact to every research project, so that we are thinking about and collecting our own evidence of impact as we go
Helen Snooks made it clear that for her, research impact is in the warp and the weft of all she and her team does - it is vital and important of itself, and REF is a welcome focus to ensure it can receive the institutional priority it deserves.
The rest of our discussions at the symposium focused on who feels the impact of research, with input from colleagues whose work focuses on trying to bridge research and practice: Jo Cooke (Yorkshire and Humber CLAHRC), Kathryn Oliver (University of Oxford), Sue Bale (SE Wales Academic Health Science Partnership [SEWAHSP], and Robyn Davies, SEWAHSP). At the conclusion of the meeting, participants were unanimous in their view that achieving research impact is an imperative and not an option. Likewise, it was clear that making impact a core focus of all our research will challenge profoundly the way that (at least some) research is scoped, undertaken, and disseminated.
As the evolution of REF shows, the academic paper is important to dissemination of research findings but not sufficient. With the Comprehensive Spending Review looming, justifying the allocation of increasingly scarce research funds is likely to become an even more pressing priority for the research community. Achieving, claiming and measuring impact will be central to this mission.