On how data are partitioned in model development and evaluation: Confronting the elephant in the room to enhance model generalization

Simple item page
dc.contributor.authorMaier, H.R.
dc.contributor.authorZheng, F.
dc.contributor.authorGupta, H.
dc.contributor.authorChen, J.
dc.contributor.authorMai, J.
dc.contributor.authorSavic, D.
dc.contributor.authorLoritz, R.
dc.contributor.authorWu, W.
dc.contributor.authorGuo, D.
dc.contributor.authorBennett, A.
dc.contributor.authorJakeman, A.
dc.contributor.authorRazavi, S.
dc.contributor.authorZhao, J.
dc.date.issued2023
dc.description.abstractModels play a pivotal role in advancing our understanding of Earth’s physical nature and environmental systems, aiding in their efficient planning and management. The accuracy and reliability of these models heavily rely on data, which are generally partitioned into subsets for model development and evaluation. Surprisingly, how this partitioning is done is often not justified, even though it determines what model we end up with, how we assess its performance and what decisions we make based on the resulting model outputs. In this study, we shed light on the paramount importance of meticulously considering data partitioning in the model development and evaluation process, and its significant impact on model generalization. We identify flaws in existing data-splitting approaches and propose a forward-looking strategy to effectively confront the “elephant in the room”, leading to improved model generalization capabilities.
dc.description.statementofresponsibilityHolger R. Maier, Feifei Zheng, Hoshin Gupta, Junyi Chen, Juliane Mai, Dragan Savic, Ralf Loritz, Wenyan Wu, Danlu Guo, Andrew Bennett, Anthony Jakeman, Saman Razavi, Jianshi Zhao
dc.identifier.citationEnvironmental Modelling and Software, 2023; 167:105779-1-105779-8
dc.identifier.doi10.1016/j.envsoft.2023.105779
dc.identifier.issn1364-8152
dc.identifier.issn1873-6726
dc.identifier.orcidMaier, H.R. [0000-0002-0277-6887]
dc.identifier.orcidWu, W. [0000-0003-3907-1570]
dc.identifier.urihttps://hdl.handle.net/2440/139464
dc.language.isoen
dc.publisherElsevier BV
dc.relation.granthttp://purl.org/au-research/grants/arc/DE210100117
dc.rights© 2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
dc.source.urihttps://doi.org/10.1016/j.envsoft.2023.105779
dc.subjectCalibration
dc.subjectData partitioning
dc.subjectData splitting
dc.subjectEarth systems
dc.subjectModel development
dc.subjectModel evaluation
dc.subjectUncertainty
dc.subjectValidation
dc.titleOn how data are partitioned in model development and evaluation: Confronting the elephant in the room to enhance model generalization
dc.typeJournal article
pubs.publication-statusPublished

Files

Collections

Civil and Environmental Engineering publications