CRMDE Research

Since the establishment of CRMDE, its members have undertaken a wide range of research projects with the aim of improving the evidence base for the education of health professionals. The Centre has built local, regional, national and international networks of collaborators and applies a range of methodological approaches to projects. Areas of enquiry include: 

  • Selection; 
  • Curriculum and assessment; 
  • Careers support; 
  • Continuing Professional Development; 
  • Multi-Professional education and training; 
  • Systematic Reviews.

Findings from projects are disseminated through a range of journals and at national and international conferences. Project reports can also be obtained from the Centre.

Current research

Evaluation of General Practice Recruitment Processes

There is national concern that too few General Practice doctors (GPs) are being trained, but any lowering of selection standards may result in unacceptable increases in remediation costs (and threats to patient safety) if poorer trainees are appointed.  Consequently, Health Education England (HEE) has commissioned this evaluation to see if the GP recruitment process is fit for purpose.

This evaluation investigates the reliability, validity, educational impact, acceptability of the method to the stakeholders, and cost of the GP recruitment process. Reliability will be investigated within a multilevel framework to increase the factors that can be included in generalizability analysis. Likewise, multi-facet Rasch modelling will allow modelling of examiner, simulator and case differences. Differential validity and differential prediction analyses will allow exploration of issues related to fairness in greater depth and provide more accurate estimates for the economic analyses than could be otherwise undertaken.  Cost-effectiveness analyses will provide estimates of the impact of potential changes to selection on GP summative assessments. Educational impact and acceptability will be investigated using surveys of applicants.

The principle investigator is Ian Davison, who is working with Celia Taylor, University of Warwick and Professor Chris McManus, University College, London.

Generalisability analysis of PACES

PACES is the Part 2 Clinical Examination for Membership of the Royal College of Physicians (MRCP(UK)). It is designed to test the clinical knowledge and skills of trainee doctors who wish to enter higher specialist training (ST3).   In collaboration with MRCP(UK) and Professor Chris McManus, University College, London,  statistical analyses are being undertaken of anonymised test data. This is employing MCMC multilevel modelling to estimate sources of unreliability of PACES using a Generalisability framework. 

Previous Research and Dissemination

Work related training in the health sector

This project examined the impact of individual and organisational characteristics on participation in four types of training in NHS trusts in England in 2006 and 2009. The analysis included measures of the perceived effectiveness of this training and data on high and low-skilled staff also provides evidence on the effect of the NHS’ commitment to the development of low skilled staff. The principal investigator was Professor Hywel Thomas

Publication

Thomas, H. and Qiu, T. (2012) Work-related continuing education and training: participation and effectiveness, Journal of Workplace Learning, 24(3)

Workforce reform and work related education and training

This research is a comparative analysis of the work related education and training of teaching assistants and nursing assistants. Comparative studies such as this are rare and provide an opportunity to appraise aspects of training that could not be undertaken by focusing on a single occupational group. The principal investigator is Professor Hywel Thomas.

Health Innovation and Education Cluster (HIEC)

Creating innovative ways to combine advances in education, service and industry to speed up the translation of new developments into health service practice, the HIEC is a partnership between NHS organisations, the higher education sector, industry and other public and private sector organisations. Its purpose is to enable high quality patient care and services by more quickly bringing to health care practice the benefits of research and innovation, co-ordinated with appropriate education and training. The Principal investigator is Professor Richard Lilford together with Professor Hywel Thomas as Co-investigator. The researchers are Ian Davison, Vickie Firmstone and Sandra Cooke. More information...

Contact:Dr Ian Davison 

Publication

Beavan, J., Burke, S., Davison, I., Firmstone, V., Gutteridge, R., Kitchen, C. and Thorpe,G. (2011) Increasing the Uptake of Home Therapies. Report to the West Midlands Specialist Commissioners Project Implementation Group. West Midlands Central Innovation and Education Centre. 

Evaluation of modernising medical careers pilots

The National Evaluation of Specialty Selection (NESS) covered the selection processes for 71% of 4,060 training posts in England and 2,751 General Practice (GP) posts in the UK available at ST1/CT1 level in 2009, making it the largest evaluation of specialty selection in the world. Designed as a cross-sectional study using quantitative and qualitative methods, the report describes and analyses the effectiveness and suitability of selection processes across 13 specialties. Taking account of the GMC’s requirement that ‘processes for recruitment, selection and appointment must be open, fair and effective’, our principal analytical model examined the acceptability, reliability, validity and cost-effectiveness of the selection processes.

Our analysis of personal characteristics as determinants of selection scores raised concerns about fairness, particularly with respect to candidates trained overseas. Shortlisting scores were generally good predictors of final selection scores. However, internal consistency and pass mark reliability in the majority of selection systems were undesirably low given the high-stakes nature of specialty selection. Consequently, we recommended: having more stations but with a single assessor; more routine data analysis and sharing results between specialties; greater attention to assessor training and the piloting and evaluating of alternative training models; and using rigorous standard setting processes. Recent moves towards integrating selection within each specialty at a national, rather than a Deanery level, and eliminating shortlisting have improved the cost-effectiveness of selection. 

The principal investigator was Professor Hywel Thomas.

Download the National Evaluation of Specialty Selection Final Report (PDF)

A summary of the findings has also been published in the British Journal of Hospital Medicine The fairness, effectiveness and acceptability of selection for specialty training in the UK

A systematic review of the effectiveness of continuing dental professional development on learning, behaviour or patient outcomes

 The team undertook a systematic review of studies using a randomised controlled trial or quasi-experimental design in order to synthesise the existing evidence to evaluate the effectiveness of continuing professional development (CPD) interventions in dentistry on learning gains, behaviour change or patient outcome. We searched a range of electronic databases from 1986 to the present, and screened all potentially relevant studies for inclusion, using pre-established inclusion/exclusion criteria. Following data extraction and quality appraisal of all included studies a narrative synthesis of the studies was undertaken.

Publication

Systematic Review of the Effectiveness of Coninuing Dental Professional Development on Learning, Behaviour, or Patient Outcomes  (September 2012) 

Evaluation of the RCGP GP training curriculum

Commissioned by the Royal College of General Practitioners (RCGP), this £189k three-year project evaluated the introduction of a new training curriculum for general practice trainees. It examined trainee and trainer experience of the curriculum and its assessment and whether it is ‘fit for purpose’ in preparing trainees for independent practice. The principal investigator was Professor Hywel Thomas.

Report

Download the Evaluation of the RCGP GP Training Curriculum final Report (June 2011). (PDF, opens new window)

An Evaluation of Public Health posts for General Practice specialty training for West Midlands SHA.

The new RCGP GP Training Curriculum, which came into effect in August 2007, enabled the introduction of new, innovative structures of GP training. In the West Midlands, four month public health posts were introduced as part of the GP Specialty Training Scheme which provided trainees with the opportunity to gain experience in public health.

This research was an evaluation to assess the impact of the posts on trainee learning, identify any problematical issues regarding the organisation of the posts, and further develop them to ensure they meet trainees’ educational needs.

The funder was the West Midlands Deanery/SHA and the project ran from  July 2010 to February 2011. The principal investigator was Ian Davison.

Publication

Davison, I., Cooke, S. and Thomas, H. (2011) An evaluation of Public Health placements for General Practice specialty training.

A feasibility and pilot trial to test the recruitment and retention of medical students in a randomised controlled trial of medical education (on-line simulation) and to investigate the promise of the intervention

A pilot feasibility randomised controlled trial of a web-based educational intervention in undergraduate medical education. Ian Davison was the trial co-ordinator of this study in the Medical School which was designed to see if such trials were feasible. Although participation in the trial was a little low, embedding the outcome statements within the end of year exam was a success and useful lessons have been learnt for future trials.

The principal investiator was Sharon Buckley and the named researchers were Jamie Coleman, Ian Davison, David Morely and Carole Torgerson.

Dentists' appraisal: a pilot implementation study

Funded by NHS West Midlands Workforce Deanery for £47k over one year, the purpose of this study was to develop and test an appraisal system for general dental practitioners in the West Midlands. The study was focussed in the Telford area. A sample of dentists were trained in appraisal, the appraisal implementation process was followed, and the pilot was then evaluated using diaries, document analysis, surveys and interviews. The study also drew on the views of appraisers and those appraised.

Download the research report (PDF 693KB, opens new window)  

Transition from Undergraduate to Practitioner: are dental graduates fit for purpose?

Funded for £48k by Birmingham & The Black Country Strategic Health Authority this project ran from 2008 - 2011. The study is investigated how well prepared dental graduates were for work in NHS dentistry. This was a longitudinal design focused on West Midlands VT trainees in 2008. 

A New Foundation Programme for Dental Therapists: Educational Evaluation

Funding was provided from the West Midlands Strategic Health Authority (SHA) to deliver a one-year pilot foundation programme for dental therapists. The new pilot foundation programme was led by the dental division of the West Midlands Workforce Deanery and launched in 2009.

The purpose of the pilot was to test out and evaluate the core components of a foundation year, the purpose of which is to develop and expand dental therapists’ clinical and personal skills within an NHS dental practice during their first year after graduation. This research tested out the educational components of the proposed foundation programme.

An Evaluation of the NHS Education South Central Developing Dental Educator Course

The Developing Dental Education Course was launched in August 2008. It aims to build the capacity of dental educators by providing experienced and new trainers in dental education with a foundation of knowledge and skills in some of the core areas detailed in the Guidelines for Dental Educators (COPDEND, 2008). It is a four day course, running over a 6 - 8 week period, rather than a one-off, or two-session course. By 2010, nine Developing Dental Educator Courses had been delivered (two of these were specifically for dental care professionals, which is a tailor made two-day course), and more than 150 participants had attended the courses and a further course was scheduled.

To determine the future funding and development of such a programme, however, it is important and timely to critically reflect on its content, format and impact: how is the programme received by its participants and what is its impact on their dental educator role? Do they feel motivated and/or prepared for further study such as a CertMedEd or PGDip MedEd. This evaluation study provided a formative evaluation of this programme, drawing together evidence for example, of attendance rates, immediate feedback from participants, as well as later, more ‘medium-term’, reflections from those who have taken part in the course.

A comprehensive evaluation is central to ensuring this course is ‘fit for purpose’ and that it meets the learning needs of those in a range of dental educator roles.