Dynamic Updating of Psychosis Prediction Models in Individuals at Ultra-High Risk of Psychosis

Files

hdl_146904.pdf (618.27 KB)
  (Published version)

Date

2025

Authors

Hartmann, S.
Dwyer, D.
Scott, I.
Wannan, C.M.J.
Nguyen, J.
Lin, A.
Middeldorp, C.M.
Wood, S.J.
Yung, A.R.
McGorry, P.D.

Editors

Advisors

Journal Title

Journal ISSN

Volume Title

Type:

Journal article

Citation

Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 2025; 10(7):699-708

Statement of Responsibility

Simon Hartmann, Dominic Dwyer, Isabelle Scott, Cassandra M.J. Wannan, Josh Nguyen, Ashleigh Lin, Christel M. Middeldorp, Stephen J. Wood, Alison R. Yung, Patrick D. McGorry, Barnaby Nelson, and Scott R. Clark

Conference Name

DOI

10.1016/j.bpsc.2025.03.006

Abstract

Background: The performance of psychiatric risk calculators can deteriorate over time due to changes in patient population, referral pathways, and medical advances. Such temporal biases in existing models may lead to suboptimal decisions when translated into clinical practice. Methods are available to correct this bias, but no research has been conducted to investigate their utility in psychiatry. Methods: We aimed to analyze the performance of model updating methods for predicting psychosis onset by 1 year in 780 individuals at ultra-high risk (UHR) of psychosis from the UHR 1000+ cohort, a longitudinal cohort of UHR individuals recruited to research studies at Orygen, Melbourne, Australia, between 1995 and 2020. Model updating was performed using a yearly adjusted model (recalibration), a continuously updated model (refitting), and a continuous Bayesian updating model (dynamic updating) and compared with a static logistic regression prediction model (original) regarding calibration, discrimination, and clinical net benefit. Results: The original model was poorly calibrated over the entire validation period. All 3 updating methods improved the predictive performance compared with the original model (recalibration: p = .009; refitting: p = .020; dynamic updating: p = .001). The dynamic updating method demonstrated the best predictive performance (Harrell’s C-index = 0.71; 95% CI, 0.60 to 0.82), calibration slope (slope = 1.12; 95% CI, 0.46 to 1.87), and clinical net benefit over the entire validation period. Conclusions: Dynamic updating of psychosis prediction models may help to mitigate decreases in performance over time. Therefore, existing psychosis prediction models need to be monitored for temporal biases to mitigate potentially harmful decisions.

School/Discipline

Dissertation Note

Provenance

Description

Access Status

Rights

© 2025 Society of Biological Psychiatry. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)

License

Grant ID

http://purl.org/au-research/grants/nhmrc/1198304
 http://purl.org/au-research/grants/nhmrc/2010063
 http://purl.org/au-research/grants/nhmrc/566593
 http://purl.org/au-research/grants/nhmrc/2026339
 http://purl.org/au-research/grants/nhmrc/1137687
 http://purl.org/au-research/grants/nhmrc/1060996
 http://purl.org/au-research/grants/nhmrc/2034232

Published Version

https://doi.org/10.1016/j.bpsc.2025.03.006

Call number

Persistent link to this record

https://hdl.handle.net/2440/146904