Closing the implementation gap in pre-deployment medical AI study design
Date
2021
Authors
Oakden-Rayner, Luke
Editors
Advisors
Palmer, Lyle
Carneiro, Gustavo
Bessen, Taryn
Carneiro, Gustavo
Bessen, Taryn
Journal Title
Journal ISSN
Volume Title
Type:
Thesis
Citation
Statement of Responsibility
Conference Name
Abstract
The rapid development of clinical artificial intelligence, AI, technologies has outpaced the development of robust regulatory and clinical safety mechanisms. AI systems are cleared for use and deployed in practice relying on pre-clinical performance studies, without evidence of the impact this will have on patient and provider outcomes. This has led to concerns of an, implementation gap, where systems that appear to perform well on pre-clinical testing fail to produce the expected outcomes in practice. While there is an urgent need for direct clinical testing of AI systems and evaluation of the impact of these systems on patient and provider outcomes, it is implausible to expect the clinical evaluation will be performed at the scale necessary to mitigate potential AI harms of the many AI systems already in use and currently under development. In this body of work I look at factors which may contribute to the implementation gap, in particular the effects of low-quality training and testing data, flawed and incomplete study design methodologies, and an over-reliance on explainability methods to address safety. I suggest a series of improvements to how we design, evaluate, and utilise AI systems in clinical practice, with the goal of better estimating the potential harms of AI during the pre-clinical testing phase, and by doing so closing the implementation gap.
School/Discipline
School of Public Health
Dissertation Note
Thesis (Ph.D.) -- University of Adelaide, School of Public Health, 2022
Provenance
This electronic version is made publicly available by the University of Adelaide in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. This thesis may incorporate third party material which has been used by the author pursuant to Fair Dealing exceptions. If you are the owner of any included third party copyright material you wish to be removed from this electronic version, please complete the take down form located at: http://www.adelaide.edu.au/legals