×

Moderate deviations for Bayes posteriors. (English) Zbl 1017.62006

Let the \(X_{k}\) be i.i.d. random variables with unknown distribution which is to be inferred by a Bayesian procedure. Let the unknown probability distribution of \(X_1\) be assigned a prior \(\pi\in M_1(\Omega)\), with support \(\text{ supp} \pi\), where \(M_1(\Omega)\) is the space of probability measures on \(\Omega\). The posterior distribution, given the first \(n\) observations, is a function of the empirical measure \(L_{n}(\omega)=n^{-1}\sum_{k=1}^{n}\delta_{X_{k}}(\omega)\), where \(\delta_{X_{k}}\) denotes unit mass at \(X_{k}\) and is denoted by \(\pi^{n}(L_{n})\). Given a sequence \((X_{n})_{n\in N}\), let \((\nu_{n})_{n\in N}\) be a sequence of \(M_1(\Omega)\)-valued random variables such that \(\nu_{n}\) has distribution \(\pi^{n}(L_{n})\) for each \(n\geq 1\). Let \((b_{n})_{n\in N}\) be a sequence satisfying the conditions \(b_{n}/n\to 0\) and \(b_{n}^2/n\to\infty\) as \(n\to\infty\). For finite sets \(\Omega\) the authors prove that, on the set \((n/b_{n})(L_{n}-\mu)\to 0\), for any “regular” point \(\mu\) in the support of the prior, the sequence \((P((n/b_{n})(\nu_{n}-\mu)\in\cdot))_{n\in N}\) satisfies the large deviation principle on \(M_{b}(\Omega)\) - the set of finite signed measures on \(\Omega\) with speed \(n/b_{n}^2\) and the rate function \[ I(\nu)=\begin{cases} 2^{-1}\sum_{x\in \Omega}\nu^2(x)/\mu(x), & \text{if\quad \(\sum_{x\in \Omega}\nu(x)=0\)}\\ +\infty, & \text{otherwise}. \end{cases} \] Then the authors extend the result to Polish spaces for priors which are exchangeable with respect to a nested sequence of partitions. Pólya trees are considered as an illustrative example.

MSC:

62C10 Bayesian problems; characterization of Bayes procedures
62F10 Point estimation
62E20 Asymptotic distribution theory in statistics
62F15 Bayesian inference
Full Text: DOI

References:

[1] Barron A., Ann. Statist 27 pp 536– (1999)
[2] Borovkov A., Siberian Math. J. 19 pp 697– (1978)
[3] Cox D., Ann. Statist 21 pp 903– (1993)
[4] de Acosta A., Trans. Amer. Math. Soc 329 pp 357– (1992)
[5] de Acosta A., J. Appl. Probab 31 pp 41– (1994)
[6] 6A. de Acosta (1994 ). Projective systems in large deviation theory II: some applications . InProbability in Banach spaces,9(eds J. Hoffman-Jorgensen, J. Kuelbs, and M. Marcus),Progress in Probability35, 241-250. Birkhauser, Boston.
[7] 7A. Dembo, and O. Zeitouni (1998 ).Large deviations techniques and applications. Springer-Verlag, New York. · Zbl 0896.60013
[8] Diaconis P., Ann. Statist 14 pp 1– (1986)
[9] Doksum K., Ann. Probab 2 pp 183– (1974)
[10] 10P. Dupuis, and R. S. Ellis (1997 ).A weak convergence approach to the theory of large deviations. Wiley, New York. · Zbl 0904.60001
[11] DOI: 10.1016/S0304-4149(98)00047-7 · Zbl 0934.60023 · doi:10.1016/S0304-4149(98)00047-7
[12] Freedman D. A., Ann. Math Statist 34 pp 1386– (1963)
[13] Freedman D. A., Ann. Statist 27 pp 1119– (1999)
[14] Fu J. C., Ann. Inst. Statist. Math 40 pp 683– (1988)
[15] DOI: 10.1016/S0167-7152(98)00215-6 · Zbl 0927.62014 · doi:10.1016/S0167-7152(98)00215-6
[16] Ganesh A. J., Bernoulli 6 pp 1021– (2000)
[17] 17H.O. Georgii (1988 ).Gibbs measures and phase transitions. Walter de Gruyter, Berlin. · Zbl 0657.60122
[18] 18S. Ghosal, J. K. Ghosh, and R. V. Ramamoorthi (1998 ). Consistency issues in Bayesian nonparametrics . InAsymptotics, nonparametrics and time series: a tribute to Madan Lal Puri(ed. S. Ghosh), Marcel Dekker, New York.
[19] Ghosal S., Ann. Statist 28 pp 501– (2000) · Zbl 1105.62315 · doi:10.1214/aos/1016218228
[20] 20I. A. Ibragimov, and R. Z. Has’minskii (1981 ).Statistical estimation: asymptotic theory. Springer-Verlag, New York.
[21] Johnson R., Ann. Math. Statist 38 pp 1899– (1967)
[22] Johnson R., Ann. Math. Statist 41 pp 851– (1970)
[23] Lavine M., Ann. Statist 20 pp 1222– (1992)
[24] Le Cam L. M., Ann. Statist. 22 pp 38– (1973)
[25] 25D. G. Luenberger (1965 ).Introduction to linear and nonlinear programming. Addison-Wesley, New York.
[26] Lynch M., Ann. Probab 15 pp 610– (1987)
[27] Mauldin R. D., Ann. Statist 20 pp 1203– (1992)
[28] Mogulskii A., Theory Probab. Appl. 21 pp 300– (1976)
[29] Sanov I., Selected Transl. Math. Statist. Probab pp 214– (1961)
[30] Schwartz L., Z. Wahrscheinlichkeitstheorie verw. Geb. 4 pp 10– (1965)
[31] 31X. Shen, and L. Wasserman (1998 ). Rates of convergence of posterior distributions . Technical report, Department of Statistics, Carnegie-Mellon University.
[32] 32L. Wasserman (1999 ). Asymptotic properties of nonparametric Bayesian procedures . InPractical nonparametric and semiparametric Bayesian statistics, Lecture Notes in Statistics,133, 293-304. Springer-Verlag, New York.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.