Patient Reported Outcomes

Group Leader: Prof Melanie Calvert


Patient reported outcomes (PROs) are measures reported directly by the patient, which provide “the patient voice” in clinical trials investigating evidence of treatment effectiveness. The aim of the PRO Research Group is to develop best practice for PRO use in trials, to improve the quality of PRO information collected, so that clinical care for patients may be better informed.

Our research group

PRO data may be used to inform clinical care and decision-making, predict long-term outcomes and inform health policy. Despite this, research suggests that the design, data collection, reporting and dissemination of PROs in trials are frequently suboptimal. These practices devalue important patient-centred data and limit the impact of PRO results in the clinical setting.

The PROs Research Group lead a programme of work, with international collaborators, aiming to enhance the design, implementation, analysis and reporting of PROs in trials and to improve the way that PRO results are used to inform clinical care. Recent success includes the development of the new CONSORT PRO extension which provides guidance for reporting PRO trial results


Specifically, the PRO research group focuses on the development of best practice in the following areas:

  1. Trial Design and Protocol Development
  2. Implementation of PROs in Clinical Trials 
  3. Reporting of trials where PROs are an outcome
  4. Use of PRO trial data to inform clinical care

The cross cutting themes underpinning this work are ‘ethics, education, training and knowledge transfer’ for clinicians, researchers, policymakers, patients and carers.

Related activities

Members of the PROs Research Group are involved in the ‘International Society for Quality of Life Research (ISOQOL). Best practices for PROs in Randomized Clinical Trials Task Force’ The PRO Research Group is also involved in broader outcomes research.

Prof Melanie Calvert is an expert advisor on PROs for the NIHR West Midland Research Design Service  and leads the outcomes research work stream within the MRC Midland Hub for Trials Methodology research and has recently been elected to serve on the ISOQOL Board of Directors. Researchers at the University of Birmingham are actively engaging with the COMET initiative. The COMET (Core Outcome Measures in Effectiveness Trials) Initiative which brings together people interested in the development and application of agreed standardised sets of outcomes, known as ‘core outcome sets’”.  These sets will make it easier for researchers and clinicians to compare results of trials on specific conditions in a standardised way. This encompasses both applied and methodological research.

Current projects

Trial Design and Protocol Development

Evaluation of patient reported outcomes in clinical trials: systematic review of trial protocols Investigators: Calvert M (PI), Kyte D, Draper H, Ives J, Gheorge A, Brundage M, King M, Mercieca-Bebber R, Blazeby J.Funder: NIHR NSPCR

Centre selection for clinical trials and the generalisability of results: a mixed methods studyInvestigators: Gheorghe A, Roberts T, Ives, J, Fletcher, B, Calvert MFunder: DoH RSF

Study Protocol: Evaluation of Patient Reported Outcomes (PROs) in Clinical Trials: Systematic Review of Trial Protocols
Investigators: Calvert M (PI) 

Implementation of PROs in Clinical Trials

Methodological and ethical issues in health-related quality of life research (ME-QOL)
Doctoral Researcher: Derek Kyte; Supervisors: Calvert M, Draper H, Ives JFunder: NIHR (NSPCR)
Link to ME-QOL Survey Dataset

Reporting of trials where PROs are an outcome

CONSORT Patient Reported Outcomes; (PRO) Extension Investigators: Calvert M (PI), Blazeby J, Brundage M, Moher D, Revicki D
Funder: MRC
In order to improve the reporting of PROs in clinical trials the International Society for Quality of Life Research, MRC Midland and ConDuCT Hubs for Trials Methodology Research and members of the CONSORT Executive have led the development of the CONSORT PRO Extension. The international project has involved a systematic review, survey of key stakeholders (members of the International Society for Quality of Life Research, International Society for Pharmacoeconomics and Outcomes Research, UK and European clinical trials units, the NIHR Research Design Service, MRC Hubs for Methodology Research, the Cochrane PRO Methodology Group, NETSCC and NICE Appraisal Chairs) and a Consensus Meeting. The guidance will complement the existing CONSORT recommendations and will be featured on the website of the EQUATOR network

Knowledge to action: Knowledge translation activities relating to the CONSORT PRO Extension
Investigators Brundage M (PI), Blazeby J, Calvert M, de Vet HCW, Efficace F, Moher D, Revicki D, Scott J, Snyder C, Yount SFunder: CIHR

Use of PRO trial data to inform clinical care

Oncologists attitudes to the use of PRO trial data to inform clinical practiceMasters Research: Student: Julie Rouette J; Supervisor Brundage MExpert International Advisory Group: Calvert M, Blazeby J, King M, Ringash J, Meyer R.

Recent selected publications

Calvert M, Brundage M, Jacobsen PB, Schünemann HJ, Efficace F. The CONSORT Patient-Reported Outcome (PRO) extension: implications for clinical trials and practice. Health Qual Life Outcomes. 2013 Oct 29;11(1):184. [Epub ahead of print]

Kyte D, Ives J, Draper H, Keeley T, Calvert M. Inconsistencies in quality of life data collection in clinical trials: a potential source of bias? Interviews with research nurses and trialists. PLoS One. 2013 Oct 4;8(10):e76625. doi: 10.1371/journal.pone.0076625.

Kirkby HM, Calvert M, McManus RJ, Draper H. Informing Potential Participants about Research: Observational Study with an Embedded Randomized Controlled Trial. PLoS One. 2013 Oct 3;8(10):e76435. doi: 10.1371/journal.pone.0076435.

Kyte D, Draper H, Calvert M. Patient-reported outcome alerts: ethical and logistical considerations in clinical trials. JAMA. 2013 Sep 25;310(12):1229-30. doi: 10.1001/jama.2013.277222.

Moran GM, Fletcher B, Calvert M, Feltham MG, Sackley C, Marshall T. A systematic review investigating fatigue, psychological and cognitive impairment following TIA and minor stroke: protocol paper. Syst Rev. 2013 Sep 8;2(1):72. doi: 10.1186/2046-4053-2-72.

Pinkney TD, Calvert M, Bartlett DC, Gheorghe A, Redman V, Dowswell G, Hawkins W, Mak T, Youssef H, Richardson C, Hornby S, Magill L, Haslop R, Wilson S, Morton D; West Midlands Research Collaborative; ROSSINI Trial Investigators. Impact of wound edge protection devices on surgical site infection after laparotomy: multicentre randomised controlled trial (ROSSINI Trial). BMJ. 2013 Jul 31;347:f4305. doi: 10.1136/bmj.f4305.

Guillemin F, Martinez L, Calvert M, Cooper C, Ganiats T, Gitlin M, Horne R, Marciniak A, Pfeilschifter J, Shepherd S, Tosteson A, Wade S, Macarios D, Freemantle N. Fear of falling, fracture history, and comorbidities are associated with health-related quality of life among European and US women with osteoporosis in a large international study. Osteoporos Int. 2013 Dec;24(12):3001-10. doi: 10.1007/s00198-013-2408-4. Epub 2013 Jun 11.

Kyte D, Draper H, Ives J, Liles C, Gheorghe A, Calvert M. Patient reported outcomes (PROs) in clinical trials: is 'in-trial' guidance lacking? a systematic review. PLoS One. 2013;8(4):e60684. doi: 10.1371/journal.pone.0060684. Epub 2013 Apr 1.

Module evaluation: a comparison of standard evaluation with nominal group technique. Educ Prim Care. 2013 Feb;24(2):111-8. Ives J, Skelton J, Calvert M.

Gheorghe A, Roberts TE, Ives JC, Fletcher BR, Calvert M. Centre selection for clinical trials and the generalisability of results: a mixed methods study. PLoS One. 2013;8(2):e56560. doi: 10.1371/journal.pone.0056560. Epub 2013 Feb 22.

Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD; CONSORT PRO Group. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA. 2013 Feb 27;309(8):814-22. doi: 10.1001/jama.2013.879.

Calvert M, Pall H, Hoppitt T, Eaton B, Savill E, Sackley C. Health-related quality of life and supportive care in patients with rare long-term neurological conditions. Qual Life Res. 2013 Aug;22(6):1231-8. doi: 10.1007/s11136-012-0269-5. Epub 2012 Sep 23.

Brundage M, Blazeby J, Revicki D, Bass B, de Vet H, Duffy H, Efficace F, King M, Lam CL, Moher D, Scott J, Sloan J, Snyder C, Yount S, Calvert M. Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards. Qual Life Res. 2013 Aug;22(6):1161-75. doi: 10.1007/s11136-012-0252-1. Epub 2012 Sep 18.

Research group members

Group lead: Professor Melanie Calvert [], Reader in Epidemiology
Deputy lead: Derek Kyte
Researchers: Prof Heather Draper, Dr Jon Ives
Doctoral Researchers: Derek Kyte, Grace Moran, Janet Jones
Research Officer: Helen Duffy
Research Administrator: Karen Biddle


University of Birmingham 
Prof Joanna Coast, Prof Paul Cockwell, Prof Jon Deeks, Dr Paramjit Gill, Dr Thomas Keeley, Prof Paulus Kirchhof, Prof Dion Morton, Prof Tracy Roberts, Dr Sabrina Grant

Birmingham Health Partners and the Institute for Translational Medicine
Dr Alastair Denniston

West Midlands Research Collaborative: including David Bartlett, William Hawkins, Thomas Pinkney, Haney Youseff

Prof. Doug Altman, Centre for Statistics in Medicine, Oxford #
Prof. Jane Blazeby, MRC ConDuCT HTMR, Bristol*#  
Prof. Julia Brown, University of Leeds
Dr Peter Davidson, NIHR Evaluation, Trials and Studies Coordinating Centre #
Nancy Devlin OHE
Prof. Michael Drummond, Co-Editor-in-Chief , Value in Health #
Prof. Martin Eccles, Newcastle University #
Mr. Ben Fletcher, Oxford University
Prof. Nick Freemantle, UCL
Dr Adrian Gheorghe, LSHTM
Dr. Trish Groves, Deputy Editor / Editor-in-Chief BMJ Open, BMJ #
Mr. Richard Haslop, Research Associate, UCL
Dr. Astrid James, Deputy Editor, The Lancet #
Dr. Xavier Luria, Head of Safety and Efficacy of Medicines, European Medicines Agency (EMA) #
Rhiannon Macefield, University of Bristol
Prof. Richard McManus, Oxford University
Mrs Lesley Roberts, Patient Representative
Dr Jane Scott, PRO Strategic Relations*#
Mr. Richard Stephens, Patient Representative #
Professor Andrew Stevens, Chair, NICE Appraisals Committee #
Rob Thwaites ABPT Health Informatics Group
Elizabeth Tremain, NIHR Evaluation, Trials and Studies Coordinating Centre
Dr. Emma Veitch, Acting Deputy Editor, PLoS Medicine #
Ms. Maria Von Hildebrand, Patient & Carer Representative #
Dr Chris Weir, University of Edinburgh
Prof. Paula Williamson, MRC Northwest Hub for Trials Methodology research, Liverpool
Philippa Yeeles, Head of Patient and Public Involvement NIHR Central Commissioning Facility

International Collaborators 
Dr.Ethan Basch, Associate Attending Physician and Health Services Researcher, Memorial Sloan-Kettering Cancer Center, USA #
Prof. Michael Brundage, Queen’s University, Canada* #
Dr. Deborah Watkins Bruner,
Prof. An-Wen Chan, University of Toronto
Prof. Henrica de Vet, VU University Medical Centre, The Netherlands*#
Dr. Fabio Efficace, GIMEMA, Italy*#
Dr. Cynthia J. Girman, Senior Director, Department of Epidemiology, Merck Research Laboratories, USA,
Dr. Robert M. Golub, Deputy Editor, JAMA, USA #
Prof. Paul B. Jacobsen, Associate Editor, JCO, USA #
Prof. Madeleine King, University of Sydney, Australia*#
Prof. Cindy Lam, The University of Hong Kong, Hong Kong*#
Rebecca Mercieca-Bebber, University of Sydney, Australia
Prof. David Moher, Ottawa Hospital Research Institute, Canada*#
Michael Palmer, Queen’s University, Canada
Prof. Donald Patrick, Professor and Board of Director Member, ISPOR, USA #
Dr. Bryce Reeve, UNC Lineberger, USA
Prof. Dennis Revicki, United Biosource Corporation, USA*#
Julie Rouette, Queen’s University, Canada
Prof. Claire Snyder, Johns Hopkins Medicine, USA*#
Dr. Sharon Straus, Director, Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Canada #
Associate Professor Rosalie Viney, Deputy Director, Centre for Health Economics Research and Evaluation, University of Technology Sydney, Australia #
Robert W. Woodruff Professor of Nursing, Emory University, USA #
Prof. Susan Yount, North Western University Feinberg School of Medicine, USA*#

* ISOQOL Taskforce members
# Collaborators on the CONSORT PRO Extension

CONSORT PRO Executive Prof. Doug Altman, Centre for Statistics in Medicine, Oxford #
Prof. Jane Blazeby, MRC ConDuCT HTMR, Bristol*#
Prof. Michael Brundage, Queen’s University, Canada* #
Prof. Melanie Calvert, University of Birmingham *#
Prof. David Moher, Ottawa Hospital Research Institute, Canada*#
Prof. Dennis Revicki, United Biosource Corporation, USA*#

Health Research Authority (Clive Collett, Hugh Davies, Amanda Hunn, Janet Wisely)

UKCRC Registered CTU Network

MRC HTMR and Outcomes Working Group


How should we manage PRO Alerts in Clinical Trials?
Please see our JAMA Viewpoint to find out more.

The Institute of Advanced Studies Workshop

The University of Birmingham PRO Research Group led an Institute of Advanced Studies Workshop on 2nd-3rd July 2013. The workshop aimed to disseminate the latest research from the group and international collaborators, to promote and consider best practices for PRO trial design, identify research priorities and develop collaborations to obtain funding to support these activities. The workshop identified a number of research priorities described in the workshop report

CONSORT PRO Extension Published in JAMA
A successful collaboration between the MRC Midland and ConDuCT Hubs for Trials Methodology Research, UK, ISOQOL Reporting Guidelines Taskforce and the CONSORT Executive has led to the new CONSORT PRO Extension being published in JAMA. The new guidance informed by key international stakeholders aims to facilitate the transparent reporting of PRO data from trials to inform patient care and health policy.

Calvert M, Blazeby J , Altman D , Revicki D , Moher D, Brundage Reporting of Patient Reported Outcomes in Randomised Trials: the CONSORT PRO Extension. JAMA 2013; 309 (8) 814-822

Centre selection for clinical Trials
The results of the group's research on centre selection in RCTs have now been published in PLOS One. Using a mixed methods study comprising a systematic review of NIHR HTA-funded RCTs, focus groups with trialists and an electronic survey of CRCUK Clinical Trials Units, we mapped the considerations that trialists account for when including centres in RCTs. We found that generalisability often plays only a small role in this process at the detriment of pragmatic issues, thus threatening the external validity of trial findings. The potential impact of this practice on the generalisability of trial-based cost-effectiveness results and possible ways forward will be the subject of a panel at the ISPOR (International Society for Pharmacoeconomics and Outcomes Research) Annual Meeting in New Orleans, USA (May 2013), where Adrian Gheorghe will join a panel of prominent health economists in discussing the issue.

Gheorghe A, Roberts TE, Ives JC, Fletcher BR, Calvert M (2013) Centre Selection for Clinical Trials and the Generalisability of Results: A Mixed Methods Study. PLoS ONE 8(2): e56560. doi:10.1371/journal.pone.0056560.