Impact measurement in the third sector is still inconsistent

2 August 2013 - Press release 

The Third Sector Research Centre has conducted a review of impact measurement in the UK third sector. It notes that, despite the recent growth in impact measurement, practice and understanding are inconsistent.  

Many studies have raised concerns over variations in the quality of both impact measurement practice and tools. Organisations need more evidence about which tools are available and their strengths and weaknesses for different contexts. 

Practice appears to be concentrated in larger organisations. The review notes a lack of low-cost, off the shelf tools available for smaller organisations without the capacity to pay. Many organisations need help to navigate the tools and support available. While there has been increasing support for organisations to measure impact, this support is uncoordinated and not accessible to all. 

Other key challenges highlighted by the review include:

• Some organisations lack the research and evaluation skills necessary to conduct robust impact measurement
• Evaluation may be shaped too highly by requirements of funders, at the expense of other concerns and stakeholders 
• The cost – in terms of time and money - especially for smaller organisations
• Organisations have uneven access to support, guidance and tools, as well as capacity
• Variation in the meaning of impact measurement and discretion in the impact measurement process create problems for comparing data across organisations

The review notes that impact measurement in the sector has been largely focused on individual organisations. Developing shared measurements and tools is a key challenge. Shared measures could enable comparisons between organisations, and help draw conclusions about the impact of the sector more broadly. However, this must be balanced with the need for organisations to develop measures that are truly relevant to their own work.

Jenny Harlock, of the Third Sector Research Centre who conducted the review, said “There is emerging consensus from some practitioners in the sector – such as the Inspiring Impact Network - around best practice for impact measurement. But despite this, there are continued challenges for defining what impact measurement is, or ought to be. Measuring impact means different things to different people, and different approaches reflect different values, and prioritise different outcomes. Understanding the impact and value of third sector organisations remains a key question for both practitioners and policy makers. In this context, it is important to know more about who is undertaking evaluation, the practices and tools they are using, and the effect these have on both how organisations operate, and how they are assessed.” 

The research review ‘Impact measurement practice in the UK third sector’ was written by Jenny Harlock, Third Sector Research Centre, University of Birmingham