A Bayesian method to improve the extrapolation ability of ANNs

Abstract

Although artificial neural networks have been shown to be superior prediction models in many hydrology-related areas, their known lack of extrapolation capability has limited the wider use and acceptance of ANNs as forecasting models. This problem lies mainly with the fact that a single 'most likely' weight vector, which is determined by calibration with a finite set of data, is used to define the function modelled by the ANN. There are, in fact, many different weight vectors that result in approximately equal model performance; however, standard ANN development approaches do not allow for any weight vectors, other than that which provides the best fit to the calibration data, to impact on the predictions made. In this paper, a Bayesian method is presented that enables the entire range of plausible weight vectors to be accounted for in the model predictions. In doing so, the relationship modelled by the ANN is more general and less dominated by the information contained in the calibration data. The method is applied to a real-world case study known to require extrapolation and the resulting ANN is shown to perform significantly better than an ANN developed using standard approaches.