Using Kane's validity framework to compare an integrated and single-skill objective structured clinical examination
Date
2024
Authors
Lim, A.
Abeyaratne, C.
Reeve, E.
Desforges, K.
Malone, D.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
American Journal of Pharmaceutical Education, 2024; 88(9, article no. 100756):1-6
Statement of Responsibility
Conference Name
Abstract
Objective: The aim of this study was to compare the validity of an integrated objective structured clinical examination (OSCE) station assessing both oral and written components with that of an OSCE station assessing 1 single skill (oral only), both targeted at assessing taking a best possible medication history.
Methods: A convergent mixed-methods design that used the 4 inferences of Kane’s validity framework (scoring, generalization, extrapolation, and implications) as a scaffold to integrate qualitative data (post-OSCE reflections) and quantitative data (assessment grades and categories of medication errors) was applied.
ResultsIn 2022, 216 students completed the OSCE station with the oral component alone, while in 2023, 254 students completed the integrated (oral and written) OSCE station. Students in 2023 performed significantly better, with a median score of 88% vs 80% in 2022. There was a greater proportion of commission errors in the integrated assessment (20.4% vs 15.3%), but fewer omission errors (29.9% vs 31.8%) and patient profile errors (5.1% vs 69.4%). Student reflections revealed that conversations were rushed in the integrated assessment, with a greater focus on written formatting, but an appreciation for the authenticity and structured format of the integrated OSCE compared with the single-skill OSCE alone.
Conclusion: Students completing the integrated OSCE (with oral and written components) had fewer patient profile and medication omission errors than students who completed the oral-only OSCE. Considering Kane’s validity framework, there was a stronger argument for the more authentic integrated OSCE in terms of the inferences of extrapolation and implications.
School/Discipline
Dissertation Note
Provenance
Description
Data source: Supplementary material, https://ars.els-cdn.com/content/image/1-s2.0-S0002945924104755-mmc1.docx
Access Status
Rights
Copyright 2024 The Author(s). Published by Elsevier Inc. on behalf of American Association of Colleges of Pharmacy. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)