×

From screening to quantitative sensitivity analysis. A unified approach. (English) Zbl 1219.93120

Summary: The present work is a sequel to a recent one published on this journal where the superiority of ‘radial design’ to compute the ‘total sensitivity index’ was ascertained. Both concepts belong to sensitivity analysis of model output. A radial design is the one whereby starting from a random point in the hyperspace of the input factors one step in turn is taken for each factor. The procedure is iterated a number of times with a different starting random point as to collect a sample of elementary shifts for each factor. The total sensitivity index is a powerful sensitivity measure which can be estimated based on such a sample. Given the similarity between the total sensitivity index and a screening test known as method of the elementary effects (or method of Morris), we test the radial design on this method. Both methods are best practices: the total sensitivity index in the class of the quantitative measures and the elementary effects in that of the screening methods. We find that the radial design is indeed superior even for the computation of the elementary effects method. This opens the door to a sensitivity analysis strategy whereby the analyst can start with a small number of points (screening-wise) and then - depending on the results - possibly increase the numeral of points up to compute a fully quantitative measure. Also of interest to practitioners is that a radial design is nothing else than an iterated ‘One factor At a Time’ (OAT) approach. OAT is a radial design of size one. While OAT is not a good practice, modelers in all domains keep using it for sensitivity analysis for reasons discussed elsewhere [A. Saltelli and P. Annoni, How to avoid a perfunctory sensitivity analysis, Environmental Modelling and Software 25, 1508–1517 (2010)]. With the present approach modelers are offered a straightforward and economic upgrade of their OAT which maintain OAT’s appeal of having just one factor moved at each step.

MSC:

93E10 Estimation and detection in stochastic control theory
93C41 Control/observation systems with incomplete information
Full Text: DOI

References:

[1] Archer, G.; Saltelli, A.; Sobol, I., Sensitivity measures, anova-like techniques and the use of bootstrap, Journal of Statistical Computation and Simulation, 58, 99-120 (1997) · Zbl 1102.62335
[2] Bratley, B.; Fox, B. L.; Niederreiter, H., Implementation and test of low discrepancy sequences, ACM Transactions on Modeling and Computer Simulation, 2, 3, 195-213 (1992) · Zbl 0846.11044
[3] Campolongo, F.; Cariboni, J.; Saltelli, A., An effective screening design for sensitivity analysis of large models, Environmental Modelling and Software, 22, 1509-1518 (2007)
[4] Cariboni, J.; Gatelli, D.; Liska, R.; Saltelli, A., The role of sensitivity analysis in ecological modelling, Ecological Modelling, 203, 167-182 (2007)
[5] Chan, K.; Saltelli, A.; Tarantola, S., Winding stairs: A sampling tool to compute sensitivity indices, Statistics and Computing, 10, 187-196 (2000)
[6] (2009), EC. Impact assessment guidelines, 15 January 2009. Technical Report 92, SEC, 2009, p. 24, accessed April 10
[7] Efron, B.; Tibshirani, R. J., An Introduction to Bootstrap (1998), Chapman and Hall/CRC
[8] (2009), EPA. Guidance on the development, evaluation, and application of environmental models. Technical report, Office of the Science Advisor, Council for Regulatory Environmental Modeling, March 2009. EPA /100/K-09/003, p. 26, accessed April 10
[9] Hall, J.; Boyce, S.; Dawson, R.; Tarantola, S.; Wang, Y.; Saltelli, A., Sensitivity analysis for hydraulic models, Journal of Hydraulic Engineering, 135, 11, 959-969 (2009)
[10] Helton, J. C.; Davis, F. J., Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems, Reliability Engineering and System Safety, 81, 1, 23-69 (2003)
[11] Helton, J. C.; Johnson, J. D.; Salaberry, C. J.; Storlie, C. B., Survey of sampling-based methods for uncertainty and sensitivity analysis, Reliability Engineering and System Safety, 91, 1175-1209 (1996)
[12] Homma, T.; Saltelli, A., Importance measures in global sensitivity analysis of model output, Reliability Engineering and System Safety, 52, 1, 1-17 (1996)
[13] Jansen, M. J.W., Analysis of variance designs for model output, Computer Physics Communication, 117, 35-43 (1999) · Zbl 1015.68218
[14] Jansen, M. J.W.; Rossing, W. A.H.; Daamen, R. A., Monte Carlo estimation of uncertainty contributions from several independent multivariate sources, (Gasmanand, J.; van Straten, G., Predictability and Nonlinear Modelling in Natural Sciences and Economics (1994), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht), 334-343 · Zbl 0875.65019
[15] Kleijnen, J. P.C., Design and Analysis of Simulation Experiments (2008), Springer · Zbl 1269.62067
[16] Kucherenko, S.; Rodriguez-Fernandez, M.; Pantelides, C.; Shah, N., Monte Carlo evaluation of derivative-based global sensitivity measures, Reliability Engineering and System Safety, 94, 1135-1148 (2009)
[17] McKay, M. D.; Beckman, R. J.; Conover, W. J., A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics, 21, 2, 239-245 (1979) · Zbl 0415.62011
[18] Morris, M. D., Factorial sampling plans for preliminary computational experiments, Technometrics, 33, 2, 161-174 (1991)
[19] Oakley, J. E.; O’Hagan, A., Probabilistic sensitivity analysis of complex models: A Bayesian approach, Journal of the Royal Statistical Society Series B, 66, 751-769 (2004) · Zbl 1046.62027
[20] (2009), OMB. Proposed risk assessment bulletin. Technical report, The Office of Management and Budget’s - Office of Information and Regulatory Affairs (OIRA), January 2006, 010906.pdf, p. 16-17, accessed April 10, 2009
[21] (De Rocquigny, E.; Devictor, N.; Tarantola, S., Uncertainty in Industrial Practice (2008), Wiley) · Zbl 1161.90001
[22] Saltelli, A., Making best use of model valuations to compute sensitivity indices, Computer Physics Communications, 145, 280-297 (2002) · Zbl 0998.65065
[23] Saltelli, A.; Annoni, P., How to avoid a perfunctory sensitivity analysis, Environmental Modelling and Software, 25, 1508-1517 (2010)
[24] Saltelli, A.; Tarantola, S.; Chan, K., Quantitative model-independent method for global sensitivity analysis of model output, Technometrics, 41, 1, 39-56 (1999)
[25] (Saltelli, A.; Chan, K.; Scott, M., Sensitivity Analysis Probability and Statistics Series. Sensitivity Analysis Probability and Statistics Series, John Wiley and Sons (2000)) · Zbl 0961.62091
[26] Saltelli, A.; Tarantola, S.; Campolongo, F.; Ratto, M., Sensitivity Analysis in Practice. A Guide to Assessing Scientific Models (2004), Wiley · Zbl 1049.62112
[27] Saltelli, A.; Ratto, M.; Tarantola, S.; Campolongo, F., Sensitivity analysis for chemical models, Chemical Reviews, 105, 7, 2811-2828 (2005)
[28] Saltelli, A.; Ratto, M.; Andres, T.; Campolongo, F.; Cariboni, J.; Gatelli, D.; Saisana, M.; Tarantola, S., Global Sensitivity Analysis. The Primer (2008), John Wiley and Sons · Zbl 1161.00304
[29] Saltelli, A.; Campolongo, F.; Cariboni, J., Screening important inputs in models with strong interaction properties, Reliability Engineering and System Safety, 94, 7, 1149-1155 (2009)
[30] Saltelli, A.; Annoni, P.; Azzini, I.; Campolongo, F.; Ratto, M.; Tarantola, S., Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index, Computer Physics Communication, 181, 2, 259-270 (2010) · Zbl 1219.93116
[31] Santner, T. J.; Williams, B. J.; Notz, W. I., Design and Analysis of Computer Experiments (2003), Springer-Verlag · Zbl 1041.62068
[32] Sobol’, I. M., On the distribution of points in a cube and the approximate evaluation of integrals, USSR Computational Mathematics and Mathematical Physics, 7, 86-112 (1967) · Zbl 0185.41103
[33] Sobol’, I. M., Uniformly distributed sequences with an addition uniform property, USSR Computational Mathematics and Mathematical Physics, 16, 236-242 (1976) · Zbl 0391.10033
[34] Sobol’, I. M., Sensitivity estimates for nonlinear mathematical models, Matematicheskoe Modelirovanie, 2, 112-118 (1990), Translated from Russian: · Zbl 0974.00506
[35] Sobol’, I. M., Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates, Mathematics and Computers in Simulation, 55, 271-280 (2001) · Zbl 1005.65004
[36] Sobol’, I. M., Global sensitivity analysis indices for the investigation of nonlinear mathematical models, Matematicheskoe Modelirovanie, 19, 11, 23-24 (2007), in Russian · Zbl 1140.60323
[37] Sobol’, I. M.; Kucherenko, S., On global sensitivity analysis of quasi-Monte Carlo algorithms, Monte Carlo Methods and Applications, 11, 1, 1-9 (2005) · Zbl 1072.65004
[38] Sobol’, I. M.; Kucherenko, S., Derivative based global sensitivity measures and their link with global sensitivity indices, Mathematics and Computers Simulation, 79, 10, 3009-3017 (2009) · Zbl 1167.62005
[39] Sobol, I. M.; Kucherenko, S., A new derivative based importance criterion for groups of variables and its link with the global sensitivity indices, Computer Physics Communications, 181, 7, 1212-1217 (2010) · Zbl 1219.65005
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.