A parameter-bounding approach to sensitivity assessment of large simulation models

Abstract

Traditional sensitivity assessment (SA) methods have limitations which motivate a new approach, the subject of a new project at ANU and the Universities of Adelaide and Melbourne, with the Murray-Darling Basin Commission and the South Australia Dept. of Water, Land and Biodiversity Conservation as partners. The limitations include high computing load, restricted scope and validity of the results, excessive volume of results and failure to distinguish SA from uncertainty assessment. The new approach has three main aims: (i) to investigate sensitivity of a wide range of model outcomes, not only the values of individual output variables; (ii) to examine sensitivity to changes which are not small; (iii) to find efficiently features such as critical or nearredundant parameter combinations. Requirements such as output ranges, credible behaviour or given rank order of scenario outcomes define an acceptable outcome set. SA then explores the feasible set of parameter values producing acceptable outcomes. This inverts the mapping by the model from parameters to outcomes. Existing techniques for inverting an output set through a non-linear model work only on small numbers of parameters and outputs, and assume that the output set is bounded by either a box (pairs of bounds on individual variables) or an ellipsoid. Consequently it is proposed to simplify SA by set inversion by two tactics. First, the model is split into simpler sections, e.g. with linear dynamics, to allow use of efficient, approximate inversion methods such as ellipsoidal, orthotopic or parallelotopic bounding. Second, attention is confined to features of the feasible set which can answer specific questions, such as largest or smallest diameter, indicating the least and most critical linear parameter combinations. Numerical search from approximate bounds, computed with the help of standard bounding algorithms, is contemplated to find such features. Even with these tactics, SA by set inversion faces several difficulties: (i) Approximation error increases as the set is propagated through stages of the model. Existing algorithms process many successive bounded-error output observations one by one, updating the feasible parameter set with the bounds inferred from each by a one-step model inversion. By contrast, SA by set inversion through a non-linear model is likely to handle only a modest number of output bounds, but may have to propagate each through a cascade of model sections. This raises new variations on the problems tackled by established set-inversion algorithms. They produce bounds on model parameters or state from bounds on outputs, whereas SA by set inversion through a number of model sections requires bounds on inputs to all but the last section. (ii) Almost all existing algorithms produce outerbound approximations to the feasible set, whereas for SA a conservative estimate of the parameter range is required, i.e. inner bounds. (iii) The standard algorithms assume instantaneous bounds on each output variable or an ellipsoidal instantaneous bound on a vector of outputs. If the flexibility of set-inversion SA is to be exploited, bounds in other metrics have to be permitted. (iv) Some non-linearities effectively contain switches which can disconnect parts of the model. It is not obvious whether inversion of a bound through such a switch is possible. (v) A model with stable dynamics has an unstable inverse. The significance of these difficulties and the factors affecting their resolution are outlined in the paper, with particular reference to how established parameter-bounding algorithms fit into the new scheme.