Bi-cultural bi-national benchmarking and assessment of clinical reasoning in Obstetrics and Gynaecology

Abstract

Background: The Script Concordance Test (SCT) is being increasingly used in professional development in clinical reasoning (CR) in postgraduate medicine. On-line delivery favours multi-institutional collaboration. Objectives: To establish if: 1) SCT questions developed in the French-speaking University of Montreal were readily adaptable for use in the English-speaking University of Adelaide 2) expert reference panels (ERP) from both institutions could be used interchangeably 3) student cohorts would perform similarly in the same test. Study Design: 82 SCT questions based on 27 clinical cases in Obstetrics and Gynaecology were developed in Montreal and run in a volunteer cohort of year 3 and year 4 medical students (n=154). Local faculty translated all questions, selecting 31 based on 17 clinical cases for use in summative examinations a year 5 student cohort in Adelaide (n=123). Results: Mean (SD) percentage scores using each ERP key were: 74.2 (6.4) versus 73.3 (6.9), p<0.001 for Adelaide students and 72.5 (7.8) versus 70.6 (8.8), p<0.001 for Montreal students. The correlation coefficients were ≥ 0.928 (p<0.001). Conclusions: Student cohorts performed similarly regardless of which ERP key was used. With appropriate editorial control, SCT’s can be effectively shared between French and English-speaking institutions located in different hemispheres. Potential advantages include the creation of an international database of assessment items, benchmarking and cost sharing.