Zero-shot LLM-based Visual Acuity Extraction: A Pilot Study
Files
(Published version)
Date
2025
Authors
Satheakeerthy, S.
Jesudason, D.
Bahrami, B.
Bacchi, S.
Lee, Y.M.
Casson, R.
Sun, M.
Chan, W.O.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
BMC Ophthalmology, 2025; 25(1):359-1-359-8
Statement of Responsibility
Shrirajh Satheakeerthy, Daniel Jesudason, Bobak Bahrami, Stephen Bacchi, Yong Min Lee, Robert Casson, Michelle Sun, and WengOnn Chan
Conference Name
Abstract
Introduction: Visual acuity (VA) and intraocular pressure (IOP) measurements are crucial indicators of ocular health. However, the documentation of these vital parameters in clinical notes often lacks standardisation. The aim of this study was to evaluate the feasibility of using of large language models (LLMs) for automated extraction of VA and IOP data from unstructured ophthalmology clinic notes. Method: Outpatient clinic notes from The Queen Elizabeth Hospital Ophthalmology department were analysed using a 70 billion parameter LLaMA-3 model in a zero-shot learning approach. Nine data points per eye were extracted, including various VA measurements, IOP, and measurement methods. Results were compared to expertverified ground truth data. Results: Sixty-nine outpatient clinic notes were collected. Locally deployed LLM analysis of clinic notes was feasible. High accuracy was observed in extracting best corrected VA and IOP (above 90% for both eyes). Performance varied for other measurements, with lower accuracy for uncorrected VA (67–75%) and challenges in interpreting ambiguous documentation. The model struggled to identify assumed uncorrected VA cases, highlighting issues which may correlate with documentation clarity. Conclusion: This study has demonstrated that it is feasible to deploy a LLM locally to extract VA data from free-text notes. However, the accuracy of extracted data was lacking in several domains. These performance issues highlight that further research is required to optimise the accuracy of the retrieved data.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
© The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.