Bayesian threshold selection for extremal models using measures of surprise. (English) Zbl 1507.62098
Summary: Statistical extreme value theory is concerned with the use of asymptotically motivated models to describe the extreme values of a process. A number of commonly used models are valid for observed data that exceed some high threshold. However, in practice a suitable threshold is unknown and must be determined for each analysis. While there are many threshold selection methods for univariate extremes, there are relatively few that can be applied in the multivariate setting. In addition, there are only a few Bayesian-based methods, which are naturally attractive in the modelling of extremes due to data scarcity. The use of Bayesian measures of surprise to determine suitable thresholds for extreme value models is proposed. Such measures quantify the level of support for the proposed extremal model and threshold, without the need to specify any model alternatives. This approach is easily implemented for both univariate and multivariate extremes.
MSC:
| 62-08 | Computational methods for problems pertaining to statistics |
| 62F15 | Bayesian inference |
| 62G32 | Statistics of extreme values; tail inference |
Keywords:
Bayesian inference; extremes; generalised Pareto distribution; posterior predictive \(p\)-value; spectral density function; surprise; threshold selectionSoftware:
ismevReferences:
| [1] | Anderson, T. W.; Darling, D. A., A test of goodness of fit, J. Amer. Statist. Assoc., 49, 765-769, (1954) · Zbl 0059.13302 |
| [2] | Balkema, A. A.; de Haan, L., Residual life time at great age, Ann. Probab., 2, 792-804, (1974) · Zbl 0295.60014 |
| [3] | Bayarri, M. J.; Berger, J. O., Bayesian statistics 6, 53-82, (1998), Oxford University Press, Chapter Quantifying surprise in the data and model verification · Zbl 0974.62021 |
| [4] | Bayarri, M. J.; Castellanos, M. E., Bayesian checking of the second levels of hierarchical models, Statist. Sci., 22, 322-343, (2007) · Zbl 1246.62029 |
| [5] | Bayarri, M. J.; Morales, J., Bayesian measures of surprise for outlier detection, J. Statist. Plann. Inference, 111, 3-22, (2003) · Zbl 1027.62014 |
| [6] | Behrens, C. N.; Hedibert, F. L.; Gamerman, D., Bayesian analysis of extreme events with threshold estimation, Stat. Model., 4, 227-244, (2004) · Zbl 1111.62023 |
| [7] | Beirlant, J.; Vynckier, P.; Teugels, J. L., Tail index estimation, Pareto quantile plots and regression diagnostics, J. Amer. Statist. Assoc., 91, 1659-1667, (1996) · Zbl 0881.62077 |
| [8] | Berger, J. O., Statistical decision theory and Bayesian analysis, (1985), Springer-Verlag · Zbl 0572.62008 |
| [9] | Boldi, M. O.; Davison, A. C., A mixture model for multivariate extremes, J. R. Stat. Soc. Ser. B, 69, 217-229, (2007) · Zbl 1120.62030 |
| [10] | Box, G. E.P., Sampling and Bayes inference in scientific modeling and robustness, J. Roy. Statist. Soc. Ser. A, 143, 383-430, (1980) · Zbl 0471.62036 |
| [11] | Cabras, S., Castellanos, M., 2009a. An objective Bayesian approach for threshold estimation in the Peaks Over the Threshold model. In: The 2009 International Workshop on Objective Bayes Methodology. |
| [12] | Cabras, S.; Castellanos, M. E., New frontiers in insurance and bank risk management, 207-216, (2009), McGraw-Hill, Chapter Comparison of two approaches to threshold selection in the extreme value POT model |
| [13] | Cabras, S.; Castellanos, M. E., A default Bayesian approach for estimating extreme quantiles, Astin Bull., 41, 87-106, (2011) · Zbl 1220.62056 |
| [14] | Cabras, S.; Morales, J., Extreme value analysis within a parametric outlier detection framework, Appl. Stoch. Models Bus. Ind., 23, 157-164, (2007) · Zbl 1164.62013 |
| [15] | Castellanos, M.; Cabras, S., A default Bayesian procedure for the generalised Pareto distribution, J. Statist. Plann. Inference, 137, 473-483, (2007) · Zbl 1102.62023 |
| [16] | Castillo, E.; Hadi, A.; Balakrishnan, H.; Sarabia, J., Extreme value and related models with applications in engineering and science, (2004), Wiley Interscience |
| [17] | Choulakian, V.; Stephens, M. A., Goodness-of-fit tests for the generalized Pareto distribution, Technometrics, 43, 478-484, (2001) |
| [18] | Coles, S. G., An introduction to statistical modeling of extreme values, (2001), Springer · Zbl 0980.62043 |
| [19] | Coles, S. G.; Tawn, J. A., Modelling extreme multivariate events, J. R. Stat. Soc. Ser. B, 53, 377-392, (1991) · Zbl 0800.60020 |
| [20] | Coles, S. G.; Tawn, J. A., Statistical methods for multivariate extremes: an application to structural design, J. Appl. Stat., 43, 1-48, (1994) · Zbl 0825.62717 |
| [21] | Danielsson, J.; de Haan, L.; Peng, L.; de Vries, C. G., Using a bootstrap method to choose the sample fraction in tail index estimation, J. Multivariate Anal., 76, 226-248, (2001) · Zbl 0976.62044 |
| [22] | David, H. A.; Nagaraja, H. N., Order statistics, (2003), Wiley New Jersey · Zbl 1053.62060 |
| [23] | Davison, A. C.; Smith, R. L., Models for exceedances over high thresholds (with discussion), J. R. Stat. Soc. Ser. B, 52, 393-442, (1990) · Zbl 0706.62039 |
| [24] | de Haan, L., Extremes in higher dimensions: the model and some statistics, (Proc. Int. Statist. Inst, (1985), International Statistical Institute The Hauge), paper 26.3 · Zbl 0646.62016 |
| [25] | Dekkers, A. L.M., Optimal choice of sample fraction in extreme-value estimation, J. Multivariate Anal., 47, 173-195, (1993) · Zbl 0797.62016 |
| [26] | Doucet, A.; de Freitas, N.; Gordon, N., Sequential Monte Carlo in practice, (2001), Springer · Zbl 0967.00022 |
| [27] | Drees, H.; de Haan, L.; Resnick, S. I., How to make a Hill plot, Ann. Statist., 28, 254-274, (2000) · Zbl 1106.62333 |
| [28] | Drees, H.; Kaufmann, E., Selecting the optimal sample fraction in univariate extreme value estimation, Stochastic Process. Appl., 75, 149-172, (1998) · Zbl 0926.62013 |
| [29] | Dupuis, D. J., Exceedances over high thresholds: a guide to threshold selection, Extremes, 1, 251-261, (1998) · Zbl 0921.62030 |
| [30] | Embrechts, P.; Klüppelberg, C.; Mikosch, T., Modelling extremal events for insurance and finance, (2003), Springer New York |
| [31] | Evans, M., Bayesian inference procedures derived via the concept of relative surprise, Commun. Stat., 26, 1125-1143, (1997) · Zbl 0934.62026 |
| [32] | Ferreira, A.; de Haan, L.; Peng, L., On optimising the estimation of high quantiles of a probability distribution, Statistics, 37, 401-434, (2003) · Zbl 1210.62052 |
| [33] | Feuerverger, A.; Hall, P. G., Estimating a tail exponent by modelling departure from a Pareto distribution, Ann. Statist., 27, 760-781, (1999) · Zbl 0942.62059 |
| [34] | Frigessi, A.; Haug, O.; Rue, H., A dynamic mixture model for unsupervised tail estimation without threshold selection, Extremes, 5, 219-235, (2003) · Zbl 1039.62042 |
| [35] | Gelman, A., Two simple examples for understanding posterior \(p\)-values whose distributions are far from uniform, Electron. J. Stat., 2595-2602, (2013) · Zbl 1294.62049 |
| [36] | Goegebeur, Y.; Berliant, J.; de Wet, T., Linking Pareto-tail kernel goodness-of-fit statistics with tail index at optimal threshold and second order estimation, REVSTAT, 6, 51-69, (2008) · Zbl 1153.62035 |
| [37] | Good, I. J., Surprise index, (Kotz, S.; Johnson, N. L.; Reid, C. B., Encyclopedia of Statistical Sciences, Vol. 7, (1988)), 104-109 · Zbl 0706.62002 |
| [38] | Guttman, I., The use of the concept of a future observation in goodness-of-fit problems, J. R. Stat. Soc. Ser. B, 29, 83-100, (1967) · Zbl 0158.37305 |
| [39] | Han, C.; Carlin, B. P., Markov chain Monte Carlo methods for computing Bayes factors: a comparative review, J. Amer. Statist. Assoc., 96, 1122-1132, (2001) |
| [40] | Heffernan, J.; Tawn, J., A conditional approach for multivariate extreme values, J. R. Stat. Soc. Ser. B, 66, 497-546, (2004) · Zbl 1046.62051 |
| [41] | Joe, H.; Smith, R. L.; Weissman, I., Bivariate threshold methods for extremes, J. R. Stat. Soc. Ser. B, 54, 171-183, (1992) · Zbl 0775.62083 |
| [42] | Kass, R. E.; Raftery, A. E., Bayes factors, J. Amer. Statist. Assoc., 90, 773-795, (1995) · Zbl 0846.62028 |
| [43] | Kotz, S.; Nadarajah, S., Extreme value distributions, (2002), Imperial College Press |
| [44] | MacDonald, A.; Scarrott, C. J.; Lee, D., Boundary correction, consistency and robustness of kernel densities using extreme value theory. technical report, (2011), University of Canterbury New Zealand |
| [45] | MacDonald, A.; Scarrott, C. J.; Lee, D.; Darlow, B.; Reale, M.; Russell, G., A flexible extreme value mixture model, Comput. Statist. Data Anal., 55, 2137-2157, (2011) · Zbl 1328.62296 |
| [46] | McNeil, A. J., Estimating the tails of loss severity distributions using extreme value theory, Astin Bull., 117-137, (1997) |
| [47] | Meng, X.-L., Posterior predictive \(p\)-values, Ann. Statist., 22, 1142-1160, (1994) · Zbl 0820.62027 |
| [48] | Moral, P. D.; Doucet, A.; Jasra, A., Sequential Monte Carlo samplers, J. R. Stat. Soc. Ser. B, 68, 411-436, (2006) · Zbl 1105.62034 |
| [49] | Pickands, J., Statistical inference using extreme order statistics, Ann. Statist., 3, 119-131, (1975) · Zbl 0312.62038 |
| [50] | Pugh, D. T.; Vassie, J. M., Extreme sea levels from tide and surge probability, (Proc. 16th Coastal Engineering Conference 1978, Hamburg, Vol. 1, (1979), American Society of Civil Engineers New York), 911-930 |
| [51] | Resnick, S. L., Extreme values, regular variation and point processes, (1987), Springer New York · Zbl 0633.60001 |
| [52] | Robin, J. M.; Van der Vaart, A.; Ventura, V., The asymptotic distribution of \(p\)-values in composite null models, J. Amer. Statist. Assoc., 95, 1143-1156, (2000) · Zbl 1072.62522 |
| [53] | Rousseau, J.; Mengersen, K., Asymptotic behavior of the posterior distribution in overfitted mixture models, J. R. Stat. Soc. Ser. B, 73, 689-710, (2011) · Zbl 1228.62034 |
| [54] | Rubin, D. B., Bayesianly justifiable and relevant frequency calculations for the applied statisticians, Ann. Statist., 12, 1151-1172, (1984) · Zbl 0555.62010 |
| [55] | Sabourin, A.; Naveau, P., Dirichlet mixture model for multivariate extremes: a re-parameterisation, Comput. Statist. Data Anal., 71, 542-567, (2014) · Zbl 1471.62176 |
| [56] | Scarrott, C. J.; MacDonald, A., A review of extreme value threshold estimation and uncertainty quantification, REVSTAT, 10, 33-60, (2012) · Zbl 1297.62120 |
| [57] | Smith, R. L., Threshold methods for sample extremes, (Tiago de Oliveira, J., Statistical Extremes and Applications, (1984), D. Reidel Publishing Company), 621-638 · Zbl 0574.62090 |
| [58] | Stephens, M. A., Goodness of fit for the extreme value distribution, Biometrika, 66, 591-595, (1977) · Zbl 0417.62035 |
| [59] | Tancredi, A.; Anderson, C.; O’Hagan, A., Accounting for threshold uncertainty in extreme value estimation, Extremes, 9, 87-106, (2006) · Zbl 1164.62326 |
| [60] | Weaver, W., Probability, rarity, interest and surprise, Scientif. Monthly, 67, 390-392, (1948) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
