Alternatives to maximum likelihood estimation based on spacings and the Kullback-Leibler divergence. (English) Zbl 1255.62049
Summary: An alternative to the maximum likelihood (ML) method, the maximum spacing (MSP) method, was introduced by R.C.H. Cheng and N.A.K. Amin [J. R. Stat. Soc., Ser. B 45, 394–403 (1983; Zbl 0528.62017)], and independently by B. Ranneby [Scand. J. Stat., Theory Appl. 11, 93–112 (1984; Zbl 0545.62006)]. The method, as described by B. Ranneby, is derived from an approximation of the Kullback-Leibler divergence. Since the introduction of the MSP method, several closely related methods have been suggested. This article is a survey of such methods based on spacings and the Kullback-Leibler divergence. These estimation methods possess good properties and they work in situations where the ML method does not. Important issues such as the handling of ties and incomplete data are discussed, and it is argued that by using P.A.P. Moran’s [J. R. Stat. Soc., Ser. B 13, 147–150 (1951; Zbl 0045.08705)] statistic, on which the MSP method is based, we can effectively combine: (a) a test on whether an assigned model of distribution functions is correct or not, (b) an asymptotically efficient estimation of an unknown parameter \(\theta _{0}\), and (c) a computation of a confidence region for \(\theta _{0}\).
Keywords:
non-regular estimation problems; efficient estimation; sample spacings; maximum spacing estimatorReferences:
| [1] | Anatolyev, S.; Kosenok, G., An alternative to maximum likelihood based on spacings, Econometric Theory, 21, 472-476 (2005) · Zbl 1062.62040 |
| [2] | Bahadur, R. R., Some Limit Theorems in Statistics (1971), SIAM: SIAM Philadelphia · Zbl 0257.62015 |
| [3] | Baryshnikov, Y., Yukich, J.E., 2004. Gaussian limits for generalized spacings. Available at \(\langle;\) http://www.lehigh.edu/\( \sim;\) jey0/by3Dec2004.pdf \(\rangle;\).; Baryshnikov, Y., Yukich, J.E., 2004. Gaussian limits for generalized spacings. Available at \(\langle;\) http://www.lehigh.edu/\( \sim;\) jey0/by3Dec2004.pdf \(\rangle;\). · Zbl 1159.60315 |
| [4] | Bradley, R. A.; Gart, J. J., The asymptotic properties of ML estimators when sampling from associated populations, Biometrika, 49, 205-214 (1962) · Zbl 0108.15301 |
| [5] | Brown, S.; Heathcote, A., QMLE: fast, robust and efficient estimation of distribution functions based on quantiles, Behav. Res. Meth. Ins. C, 35, 485-492 (2003) |
| [6] | Cheng, R. C.H.; Amin, N. A.K., Estimating parameters in continuous univariate distributions with a shifted origin, J. Roy. Statist. Soc. Ser. B, 45, 394-403 (1983) · Zbl 0528.62017 |
| [7] | Cheng, R. C.H.; Iles, T. C., Corrected maximum likelihood in non-regular problems, J. Roy. Statist. Soc. Ser. B, 49, 95-101 (1987) · Zbl 0615.62028 |
| [8] | Cheng, R. C.H.; Stephens, M. A., A goodness-of-fit test using Moran’s statistic with estimated parameters, Biometrika, 76, 385-392 (1989) · Zbl 0671.62050 |
| [9] | Cheng, R. C.H.; Traylor, L., Non-regular maximum likelihood problems, J. Roy. Statist. Soc. Ser. B, 57, 3-44 (1995) · Zbl 0812.62024 |
| [10] | Darling, D. A., On a class of problems related to the random division of an interval, Ann. Math. Statist., 24, 239-253 (1953) · Zbl 0053.09902 |
| [11] | Davison, A. C.; Hinkley, D. V., Bootstrap Methods and their Application (1997), Cambridge University Press: Cambridge University Press Cambridge, UK · Zbl 0886.62001 |
| [12] | Ekström, M., Strong consistency of the maximum spacing estimate, Theory Probab. Math. Statist., 55, 55-72 (1996) · Zbl 0921.62025 |
| [13] | Ekström, M., On the consistency of the maximum spacing method, J. Statist. Plann. Inference, 70, 209-224 (1998) · Zbl 0941.62027 |
| [14] | Ekström, M., Strong limit theorems for sums of logarithms of high order spacings, Statistics, 33, 153-169 (1999) · Zbl 0949.60047 |
| [15] | Ekström, M., Consistency of generalized maximum spacing estimates, Scand. J. Statist., 28, 343-354 (2001) · Zbl 0973.62040 |
| [16] | Fisher, R. A., On an absolute criterion for fitting frequency curves, Messenger Math., 41, 155-160 (1912) · JFM 43.0302.01 |
| [17] | Fitzgerald, D. L., Maximum product of spacings estimators for the generalised Pareto and log logistic distributions, Stoch. Hydrol. Hydraul., 10, 1-15 (1996) · Zbl 0870.62009 |
| [18] | Ghosh, K.; Jammalamadaka, S. R., A general estimation method using spacings, J. Statist. Plann. Inference, 93, 71-82 (2001) · Zbl 0965.62019 |
| [19] | Hall, M. J.; van den Boogaard, H. F.P.; Fernando, R. C.; Mynett, A. E., The construction of confidence intervals for frequency analysis using resampling techniques, Hydrol. Earth Syst. Sci., 8, 235-246 (2004) |
| [20] | Heathcote, A.; Brown, S., Reply to Speckman and Rouder: a theoretical basis for QML, Psychon. Bull. Rev., 11, 577-578 (2004) |
| [21] | Heathcote, A.; Brown, S.; Mewhort, D. J.K., Quantile maximum likelihood estimation of response time distributions, Psychon. Bull. Rev., 9, 394-401 (2002) |
| [22] | Heathcote, A.; Brown, S.; Cousineau, D., QMPE: estimating lognormal, Wald and Weibull RT distributions with a parameter-dependent lower bound, Behav. Res. Meth. Ins. C, 36, 277-290 (2004) |
| [23] | Jaynes, E. T., Information theory and statistical mechanics, Phys. Rev., 106, 620-630 (1957) · Zbl 0084.43701 |
| [24] | Jaynes, E. T., Information theory and statistical mechanics, Phys. Rev., 108, 171-190 (1957) · Zbl 0084.43701 |
| [25] | Jiménez, R.; Yukich, J., Asymptotics for statistical distances based on Voronoi tessellations, J. Theoret. Probab., 15, 503-541 (2002) · Zbl 1011.62021 |
| [26] | Kapur, J. N.; Kesavan, H. K., Entropy Optimization Principles with Applications (1992), Academic Press: Academic Press New York · Zbl 0718.62007 |
| [27] | Le Cam, L., Maximum likelihood: an introduction, Internat. Statist. Rev., 59, 153-171 (1990) · Zbl 0715.62045 |
| [28] | Leonenko, N., Pronzato, L., Savani, V., 2007. A class of Rényi information estimators for multidimensional densities. Available at \(\langle;\) http://eprints.pascal-network.org/archive/\(00001031/ \rangle;\).; Leonenko, N., Pronzato, L., Savani, V., 2007. A class of Rényi information estimators for multidimensional densities. Available at \(\langle;\) http://eprints.pascal-network.org/archive/\(00001031/ \rangle;\). · Zbl 1205.94053 |
| [29] | Lindsay, B. G., Efficiency versus robustness: the case for minimum Hellinger distance and related methods, Ann. Statist., 22, 1081-1114 (1994) · Zbl 0807.62030 |
| [30] | Lindsay, B. G., Mixture models: theory, geometry, and applications, (In: NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5 (1995), IMS: IMS Hayward, CA) · Zbl 0434.62028 |
| [31] | Mayoral, A. M.; Morales, D.; Morales, J.; Vajda, I., On efficiency of estimation and testing with data quantized to fixed number of cells, Metrika, 57, 1-27 (2003) · Zbl 1433.62021 |
| [32] | Menéndez, M. L.; Morales, D.; Pardo, L., Maximum entropy principle and statistical inference on condensed ordered data, Statist. Probab. Lett., 34, 85-93 (1997) · Zbl 0899.62031 |
| [33] | Menéndez, M. L.; Morales, D.; Pardo, L.; Vajda, I., Minimum divergence estimators based on grouped data, Ann. Inst. Statist. Math., 53, 277-288 (2001) · Zbl 1027.62011 |
| [34] | Menéndez, M. L.; Pardo, L.; Vajda, I., Minimum disparity estimators for discrete and continuous models, Appl. Math., 46, 439-466 (2001) · Zbl 1059.62001 |
| [35] | Moran, P. A.P., The random division of an interval—Part II, J. Roy. Statist. Soc. Ser. B, 13, 147-150 (1951) · Zbl 0045.08705 |
| [36] | Pitman, E. J.G., Some Basic Theory for Statistical Inference (1979), Chapman & Hall: Chapman & Hall London · Zbl 0442.62002 |
| [37] | Rahman, M.; Pearson, L. M., Estimation in two-parameter exponential distributions, J. Statist. Comput. Simul., 70, 371-386 (2001) · Zbl 1003.62019 |
| [38] | Ranneby, B., The maximum spacing method. An estimation method related to the maximum likelihood method, Scand. J. Statist., 11, 93-112 (1984) · Zbl 0545.62006 |
| [39] | Ranneby, B., 1996. Spatial and temporal models in contextual classification. In: Mowrer, H.T., Czaplewski, R.L., Hamre, R.H. (Eds.), Spatial Accuracy Assessment in Natural Resources and Environmental Sciences: Second International Symposium. Fort Collins, Colorado, pp. 451-458.; Ranneby, B., 1996. Spatial and temporal models in contextual classification. In: Mowrer, H.T., Czaplewski, R.L., Hamre, R.H. (Eds.), Spatial Accuracy Assessment in Natural Resources and Environmental Sciences: Second International Symposium. Fort Collins, Colorado, pp. 451-458. |
| [40] | Ranneby, B., Ekström, M., 1997. Maximum spacing estimates based on different metrics. Research Report No. 5, Department of Mathematical Statistics, Umeå University.; Ranneby, B., Ekström, M., 1997. Maximum spacing estimates based on different metrics. Research Report No. 5, Department of Mathematical Statistics, Umeå University. |
| [41] | Ranneby, B.; Jammalamadaka, S. R.; Teterukovskiy, A., The maximum spacing estimation for multivariate observations, J. Statist. Plann. Inference, 129, 427-446 (2005) · Zbl 1087.62004 |
| [42] | Roeder, K., Density estimation with confidence sets exemplified by superclusters and voids in the alaxies, J. Amer. Statist. Assoc., 85, 617-624 (1990) · Zbl 0704.62103 |
| [43] | Roeder, K., Semiparametric estimation of normal mixture densities, Ann. Statist., 20, 929-943 (1992) · Zbl 0746.62044 |
| [44] | Shah, A.; Gokhale, D. V., On maximum product of spacings (MSP) estimation for Burr XII distributions, Commn. Statist. Simulation Comput., 22, 615-641 (1993) · Zbl 0800.62116 |
| [45] | Shao, Y., Hahn, M.G., 1994. Maximum spacing estimates: a generalization and improvement of maximum likelihood estimates I. Progress in Probability, vol. 35. Birkhaüser, Boston, MA, pp. 417-431.; Shao, Y., Hahn, M.G., 1994. Maximum spacing estimates: a generalization and improvement of maximum likelihood estimates I. Progress in Probability, vol. 35. Birkhaüser, Boston, MA, pp. 417-431. · Zbl 0804.62028 |
| [46] | Shao, Y.; Hahn, M. G., Strong consistency of maximum product of spacings estimates with applications in nonparametrics and in estimation of unimodal densities, Ann. Inst. Statist. Math., 51, 31-49 (1999) · Zbl 1049.62504 |
| [47] | Shao, Y.; Hahn, M. G., Maximum product of spacings method: a unified formulation with illustration of strong consistency, Ill. J. Math., 43, 489-499 (1999) · Zbl 0932.62027 |
| [48] | Shao, Y.; Jiménez, R., Entropy for random partitions and its applications, J. Theoret. Probab., 11, 417-433 (1998) · Zbl 0931.62004 |
| [49] | Smith, R. L., Maximum likelihood estimation in a class of non-regular cases, Biometrika, 72, 67-90 (1985) · Zbl 0583.62026 |
| [50] | Titterington, D. M., Comment on estimating parameters in continuous univariate distribution, J. Roy. Statist. Soc. Ser. B, 47, 115-116 (1985) |
| [51] | Wells, M. T.; Jammalamadaka, S. R.; Tiwari, R. C., Tests of fit using spacings statistics with estimated parameters, Statist. Probab. Lett., 13, 365-372 (1992) · Zbl 0743.62037 |
| [52] | Yu, J.; Ekström, M., Asymptotic properties of high order spacings under dependence assumptions, Math. Methods Statist., 9, 437-448 (2000) · Zbl 1008.62050 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
