Reviewing PRO research

I’m a reviewer of a funding application involving PROs.  What should I look out for?

Although many trials include PROs, such as quality of life and symptoms, as outcomes, both the design and reporting of such studies is often poor.  Even though patients spend substantial time completing the questionnaires and the data have the potential to inform shared-decision-making, clinical guidelines and health policy many trials poorly report PRO data and some do not report at all. 

Key considerations when reviewing a trial protocol include:

• Why are they assessing PROs in the study? What is the rationale and hypothesis?
• How does this fit with existing literature in the population of interest?
• Which PRO have they selected? Do they provide a rationale for this and evidence of the psychometric properties of the measure and patient acceptability (this could be informed through patient and public involvement or feasibility work).
• When do they plan to assess the PRO – is there a clinical justification for this?  Is the assessment standardised?
• What processes do the team have in place to optimise PRO data collection (e.g. responsible staff, training, methods to minimise missing data, quality assurance)?
• How do they plan to analyse the data – have they pre-specified the principal analyses and addressed issues of multiplicity, missing data and clinical significance?
• Will patients be protected?  How will PRO-alerts be managed?  Will they be provided with appropriate information? Who will access their data? (see also ethics review). 

We recommend that authors prepare trial protocols in accordance with SPIRIT guidance; however this does not cover PRO specific information. We are currently leading work with international partners to provide evidence based guidance on this. In the meantime the following resources may be helpful:

• Draft PROtocol checklist –PDF
• Systematic review of PRO protocol guidance – PLOS One
• Evaluation of NIHR HTA PRO protocol content – PLOS One
• Putting patients at the heart of healthcare research – the Lancet

A SPIRIT-PRO Extension is currently under development for further details see the ISOQOL website

I’m a reviewer of an ethics application involving PROs: what should I look out for? 

The role of Research Ethics Committees is to safeguard the rights, safety, dignity and well-being of people participating in research, as well as facilitating and promoting ethical research that is of potential benefit to participants, science and society.   We are currently working in partnership with the HRA and international ethics groups to develop formal guidance for ethics committees reviewing research including PROs.  

Patient reported outcome data can provide valuable information to inform patient-centred care but currently the quality of data collection/study reporting is suboptimal which devalues the contribution of patients and limits the use of these data to inform clinical practice and health policy (research waste).  As a member of an ethics committee you have an opportunity to help ensure high quality PRO data collection that can make a real difference to patient care. 

Here are some key considerations: 

• Why are they assessing PROs in the study?   What is the rationale and hypothesis?
• How many questionnaires are being used and how often?   There should be a clear justification for the assessment of PROs ideally informed through patient and public involvement of feasibility work with patients.
• Are the questionnaires validated?   If not patient time may be wasted completing unreliable measures.
• What processes do the team have in place to optimise PRO data collection (e.g. responsible staff, training, methods to minimise missing data, quality assurance)?  If the data is poor quality this will have wasted patients time and valuable resources.
• What will happen if a patient completes a questionnaire and indicates concerning levels of psychological distress or physical symptoms that may require an immediate response? Have they considered the management of these PRO Alerts?
• Does the patient information sheet adequately describe how the PRO data will be collected, how often and who will access the PRO data?
• Has the patient been provided with information regarding support services?
• Does the consent form detail PRO specific information?  

Please see PROs in Trials Step 5 - Managing PRO Alerts for further details and examples of PIS/consent form content.

I’m a reviewer of a trial protocol/study report for a journal.  What should I look out for?

In reviewing a study protocol or trial publication you have a real opportunity to help ensure that the PRO data is of high quality and has the potential to usefully inform patient centred care.

Tips for reviewing a trial protocol

• Ensure that the protocol follows the SPIRIT checklist.  Whilst the journal publication will be a shortened version the full protocol can still be made available as a web-appendix and a SPIRIT checklist provided.
• Whilst we are developing the SPIRIT-PRO Extension use our PRO protocol checklist  to ensure PRO-specific information has been included to promote high quality data collection.

Tips for reviewing a trial publication

• Ensure that the trial publication follows CONSORT (and any relevant extensions including CONSORT-PRO).
• Please check – if the PRO was specified either as a primary or key secondary outcome, then at a minimum: the principal PRO analyses should be presented alongside clinical data in the main trial publication.  There are many examples of these data being omitted which hinders the use of PRO data in clinical decision making.

I’m a reviewer of PRO data involved in evidence synthesis. What should I look out for?

An excellent summary for those involved in systematic review and evidence synthesis of PRO data has been produced by the Cochrane PRO Methods Group.  This includes

• Background information regarding PROs
• How to locate and select studies with PROs
• A checklist for describing and assessing PROs from trials in the context of evidence synthesis
• How to interpret the result.

Interested readers can view this information here