Goodness-of-fit tests for a heavy tailed distribution. (English) Zbl 1146.62033
Summary: We study the Kolmogorov-Smirnov test, R. H. Berk and D. H. Jones test [Z. Wahrscheinlichkeitstheor. Verw. Geb. 47, 47–59 (1979; Zbl 0379.62026)], the score test and their integrated versions in the context of testing the goodness-of-fit of a heavy tailed distribution function. A comparison of these tests is conducted via Bahadur efficiency and simulations. In the simulations, the score test and the integrated score test show the best performance. Although the Berk-Jones test is more powerful than the Kolmogorov-Smirnov test, this does not hold true for their integrated versions; this differs from results of J. H. J. Einmahl and I. W. McKeague [Empirical likelihood based hypothesis testing. Bernoulli 9, No. 2, 267–290 (2003; Zbl 1015.62048)], which shows the difference of Berk-Jones test in testing distributions and tails.
MSC:
| 62G10 | Nonparametric hypothesis testing |
| 62G32 | Statistics of extreme values; tail inference |
| 62G20 | Asymptotic properties of nonparametric inference |
| 65C60 | Computational problems in statistics (MSC2010) |
| 62G30 | Order statistics; empirical distribution functions |
References:
| [1] | Abrahamson, I. G., Exact Bahadur efficiencies for the Kolmogorov-Smirnov and Kuiper one- and two-sample statistics, Ann. Math. Statist., 38, 1475-1490 (1967) · Zbl 0157.48003 |
| [2] | Aki, S., Some test statistics based on the martingale term of the empirical distribution function, Ann. Inst. Statist. Math., 38, 1, 1-21 (1986) · Zbl 0659.62055 |
| [3] | Anderson, C., The Long Tail: The Revolution Changing Small Markets into Big Business (2006), Hyperion: Hyperion New York |
| [4] | Bahadur, R. R., Some Limit Theorems in Statistics (1971), Society for Industrial and Applied Mathematics: Society for Industrial and Applied Mathematics Philadelphia, PA · Zbl 0257.62015 |
| [5] | Beirlant, J.; de Wet, T.; Goegebeur, Y., A goodness-of-fit statistic for Pareto-type behaviour, J. Comput. Appl. Math., 186, 1, 99-116 (2006) · Zbl 1073.62045 |
| [6] | Berk, R. H.; Jones, D. H., Relatively optimal combinations of test statistics, Scand. J. Statist., 5, 3, 158-162 (1978) · Zbl 0403.62021 |
| [7] | Berk, R. H.; Jones, D. H., Goodness-of-fit test statistics that dominate the Kolmogorov statistics, Z. Wahrscheinlichkeitstheorie Verwandte Gebiete, 47, 1, 47-59 (1979) · Zbl 0379.62026 |
| [8] | Bernstein, S., 1924. Sur une modification de l’inégalité de Tchebichef. Annals Science Institute Sav. Ukraine Sect. Math. IRussian, French Summary.; Bernstein, S., 1924. Sur une modification de l’inégalité de Tchebichef. Annals Science Institute Sav. Ukraine Sect. Math. IRussian, French Summary. |
| [9] | Cheng, S. H., Large deviation theorem for Hill’s estimator, Acta Math. Sinica New Ser., 8, 3, 243-254 (1992) · Zbl 0813.62041 |
| [10] | Chernoff, H., A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations, Ann. Math. Statist., 23, 493-507 (1952) · Zbl 0048.11804 |
| [11] | Choulakian, V.; Stephens, M. A., Goodness-of-fit tests for the generalized Pareto distribution, Technometrics, 43, 4, 478-484 (2001) |
| [12] | Danielsson, J.; de Vries, C. G., Tail index and quantile estimation with very high frequency data, J. Empirical Finance, 4, 2-3, 241-257 (1997) |
| [13] | Danielsson, J.; de Haan, L.; Peng, L.; de Vries, C. G., Using a bootstrap method to choose the sample fraction in tail index estimation, J. Multivariate Anal., 76, 2, 226-248 (2001) · Zbl 0976.62044 |
| [14] | Davison, A. C.; Smith, R. L., Models for exceedances over high thresholds, J. Roy. Statist. Soc. Ser. B: Methodological, 52, 3, 393-442 (1990) · Zbl 0706.62039 |
| [15] | de Haan, L.; Ferreira, A., Extreme Value Theory: An Introduction. Springer Series in Operations Research and Financial Engineering (2006), Springer: Springer New York · Zbl 1101.62002 |
| [16] | de Haan, L.; Resnick, S., On asymptotic normality of the Hill estimator, Comm. Statist. Stochastic Models, 14, 4, 849-866 (1998) · Zbl 1002.60519 |
| [17] | de Haan, L.; Resnick, S. I., Estimating the limit distribution of multivariate extremes, Comm. Statist. Stochastic Models, 9, 2, 275-309 (1993) · Zbl 0777.62036 |
| [18] | de Haan, L.; Stadtmüller, U., Generalized regular variation of second order, Australian Math. Soc. J. Ser A: Pure Math. Statist., 61, 3, 381-395 (1996) · Zbl 0878.26002 |
| [19] | Drees, H.; Kaufmann, E., Selecting the optimal sample fraction in univariate extreme value estimation, Stochastic Process. Appl., 75, 2, 149-172 (1998) · Zbl 0926.62013 |
| [20] | Drees, H.; de Haan, L.; Li, D., Approximations to the tail empirical distribution function with application to testing extreme value conditions, J. Statist. Plann. Inference, 136, 3498-3538 (2006) · Zbl 1093.62052 |
| [21] | Durbin, J., 1973a. Distribution Theory for Tests Based on the Sample Distribution Function. Society for Industrial and Applied Mathematics, Philadelphia, PA, p. vi+64.; Durbin, J., 1973a. Distribution Theory for Tests Based on the Sample Distribution Function. Society for Industrial and Applied Mathematics, Philadelphia, PA, p. vi+64. · Zbl 0267.62002 |
| [22] | Durbin, J., Weak convergence of the sample distribution function when parameters are estimated, Ann. Statist., 1, 2, 279-290 (1973) · Zbl 0256.62021 |
| [23] | Einmahl, J. H.J.; McKeague, I. W., Empirical likelihood based hypothesis testing, Bernoulli, 9, 2, 267-290 (2003) · Zbl 1015.62048 |
| [24] | Groeneboom, P.; Shorack, G. R., Large deviations of goodness of fit statistics and linear combinations of order statistics, Ann. Probab., 9, 6, 971-987 (1981) · Zbl 0473.60035 |
| [25] | Guillou, A.; Hall, P., A diagnostic for selecting the threshold in extreme value analysis, J. Roy. Statist. Soc. Ser. B: Statist. Methodology, 63, 2, 293-305 (2001) · Zbl 0979.62039 |
| [26] | Hill, B. M., A simple general approach to inference about the tail of a distribution, Ann. Statist., 3, 5, 1163-1174 (1975) · Zbl 0323.62033 |
| [27] | Hjort, N. L.; Koning, A., Tests for constancy of model parameters over time, J. Nonparametric Statist., 14, 1-2, 113-132 (2002) · Zbl 1017.62015 |
| [28] | Hoeffding, W., Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc., 58, 13-30 (1963) · Zbl 0127.10602 |
| [29] | Katz, R. W.; Parlange, M. B.; Naveau, P., Statistics of extremes in hydrology, Adv. Water Resources, 25, 1287-1304 (2002) |
| [30] | Khmaladze, È. V., A martingale approach in the theory of goodness-of-fit tests, Akad. Nauk SSSR. Teor. Veroyatnosteı˘i Primenen., 26, 2, 246-265 (1981) · Zbl 0454.60049 |
| [31] | Kolmogorov, A., Sulla determinazione empirica di una legge di distribuzione, Giorn. Ist. Ital. Attuari., 4, 83-91 (1933) · Zbl 0006.17402 |
| [32] | Koning, A.J., Peng, L., 2005. Goodness-of-fit tests for a heavy tailed distribution. Technical Report EI 2005-44, Econometric Institute, Erasmus University, Rotterdam \(\langle;\) URL: http://hdl.handle.net/\(1765/7031 \rangle;\).; Koning, A.J., Peng, L., 2005. Goodness-of-fit tests for a heavy tailed distribution. Technical Report EI 2005-44, Econometric Institute, Erasmus University, Rotterdam \(\langle;\) URL: http://hdl.handle.net/\(1765/7031 \rangle;\). · Zbl 1146.62033 |
| [33] | Leland, W. E.; Taqqu, M. S.; Willinger, W.; Wilson, D. V., On the self-similar nature of Ethernet traffic extended version, IEEE/ACM Trans. Networking, 2, 1-15 (1994) |
| [34] | Li, G., Nonparametric likelihood ratio goodness-of-fit tests for survival data, J. Multivariate Anal., 86, 1, 166-182 (2003) · Zbl 1025.62020 |
| [35] | Marohn, F., A characterization of generalized Pareto distributions by progressive censoring schemes and goodness-of-fit tests, Comm. Statist. Theory Methods, 31, 7, 1055-1065 (2002) · Zbl 1075.62514 |
| [36] | McNeil, A. J., Estimating the tails of loss severity distributions using extreme value theory, Astin Bull., 27, 1, 117-137 (1997) |
| [37] | Nikitin, Y., Asymptotic Efficiency of Nonparametric Tests (1995), Cambridge University Press: Cambridge University Press Cambridge · Zbl 0879.62045 |
| [38] | Podkorytova, O.A., 1990. Large deviations and Bahadur efficiency of the Khmaladze-Aki statistic. Zapiski Nauchnykh Seminarov Leningradskogo Otdeleniya Matematicheskogo Instituta imeni V. A. Steklova Akademii Nauk SSSR (LOMI) 184 (Issled. po Mat. Statist. 9), 227-233, 324-325.; Podkorytova, O.A., 1990. Large deviations and Bahadur efficiency of the Khmaladze-Aki statistic. Zapiski Nauchnykh Seminarov Leningradskogo Otdeleniya Matematicheskogo Instituta imeni V. A. Steklova Akademii Nauk SSSR (LOMI) 184 (Issled. po Mat. Statist. 9), 227-233, 324-325. · Zbl 0757.62025 |
| [39] | Potter, H. S.A., The mean values of certain Dirichlet series, II, Proc. London Math. Soc. (2), 47, 1-19 (1940) · Zbl 0025.26302 |
| [40] | Reittu, H.; Norros, I., On the power-law random graph model of massive data networks, Performance Evaluation, 55, 1-2, 3-23 (2004) |
| [41] | Resnick, S. I., Discussion of the Danish data on large fire insurance losses, Astin Bull., 27, 1, 139-151 (1997) |
| [42] | Resnick, S. I., Heavy tail modeling and teletraffic data, Ann. Statist., 25, 5, 1805-1869 (1997) · Zbl 0942.62097 |
| [43] | Shorack, G. R.; Wellner, J. A., Empirical processes with applications to statistics. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics (1986), Wiley: Wiley New York · Zbl 1170.62365 |
| [44] | Smirnov, N. V., Table for estimating the goodness of fit of empirical distributions, Ann. Math. Statist., 19, 2, 279-281 (1948) · Zbl 0031.37001 |
| [45] | Wellner, J.A., Koltchinskii, V., 2003. A note on the asymptotic distribution of Berk-Jones type statistics under the null hypothesis. In: High Dimensional Probability, III, Sandjberg, 2002, Progress of Probability, vol. 55. Birkhäuser, Basel, pp. 321-332.; Wellner, J.A., Koltchinskii, V., 2003. A note on the asymptotic distribution of Berk-Jones type statistics under the null hypothesis. In: High Dimensional Probability, III, Sandjberg, 2002, Progress of Probability, vol. 55. Birkhäuser, Basel, pp. 321-332. · Zbl 1042.62009 |
| [46] | Zipf, G., Human Behavior and Principle of Least Effort: An Introduction to Human Ecology (1949), Addison Wesley: Addison Wesley Cambridge, MA |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
