A brand new study by the University of Birmingham suggests that the system used by the Government to inform key decisions about the performance of NHS hospitals is inadequate.
Research published online today in the journal BMJ Quality and Safety shows that Standardised Mortality Ratios (SMRs) are not a reliable indicator of the quality of hospital care and therefore should not be used to trigger inquiries such as the high-profile probe being conducted into the performance of Mid Staffordshire Hospital.
A recent academic paper concluded that an observed lack of agreement between different methods for calculating hospital-wide mortality rates may result from ‘fundamental flaws in the hypothesised association between hospital-wide mortality and quality of care.’ Such a flaw would arise if preventable deaths were low in relation to inevitable deaths.
The Birmingham team’s paper models the correlation between overall risk-adjusted deaths and deaths due to poor care. The researchers developed a mathematical model - which was vetted and verified by international referees - to estimate the proportionate variation in SMRs that can be accounted for by fluctuations in preventable deaths.
‘The relationship between overall mortality and deaths preventable by better care is not linear,’ explains Professor Richard Lilford, Professor of Clinical Epidemiology at Birmingham and lead author of the study. ‘The signal (preventable deaths) gets lost in the noise (inevitable deaths) unless more than a quarter of all deaths are preventable. ‘The appalling idea that one in four of all NHS hospital deaths are preventable is not backed up by current evidence and does not make sense given that we all have to die and nearly half of us do so in hospital.’
‘It may be an inconvenient truth but we are saying that even when you have obtained risk-adjusted figures, they are still not a good measure of quality unless a large proportion (over 25 per cent) of the overall deaths are preventable. Therefore, we should not study overall mortality rates, even as a survey tool, in situations where people are going to die anyway.’ says Professor Lilford.
‘The fallacy is to assume that by doing the risk-adjustment process you have solved the problem,’ he adds. ‘This model offers a reality check for case-mix adjustment schemes designed to isolate the preventable component of any outcome rate. We would advise caution about using overall rates of mortality even after risk adjustment. It is preferable to look directly at quality of care.’
* Case mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ quality and safety.
For more information please contact Jenni Ameghino or Ben Hill, Press Office, University of Birmingham. 0121 415 8134. Mobile 07768 924 156.
- Hospital mortality rates are widely used as a measure of quality in developed countries.
- In England, adjusted mortality statistics for NHS hospitals have been published since 2001 by the Dr Foster organisation and the Department of Health has recently sponsored a similar approach: the Summary Hospital Mortality Index.
- Professor Lilford is an acknowledged authority on patient safety and quality. He is available for interview. Please contact the press office to arrange.