Limited information likelihood and Bayesian analysis. (English) Zbl 1030.62016
Summary: We study how to embed the optimal generalized method of moments (GMM) estimate in a likelihood-based inference framework and the Bayesian framework. First, we derive a limited information likelihood (LIL) under some moment-based limited information available in GMM based on entropy theory of \(I\)-projection theory. Second, we study a limited information Bayesian framework in which the posterior is derived from the LIL and a prior. As the LIL enables us to incorporate GMM or related inference methods in the likelihood-based inference framework, it allows us a rich set of practical applications in the Bayesian framework in which the posterior is obtained from a likelihood and a prior.
Our results are primarily large sample results as inference in the underlying GMM framework is usually justified in asymptotics. Investigation of large sample properties of the posterior derived from the LIL reveals an interesting relation between the Bayesian and the classical distribution theories.
Our results are primarily large sample results as inference in the underlying GMM framework is usually justified in asymptotics. Investigation of large sample properties of the posterior derived from the LIL reveals an interesting relation between the Bayesian and the classical distribution theories.
MSC:
| 62F12 | Asymptotic properties of parametric estimators |
| 62F15 | Bayesian inference |
| 62B10 | Statistical aspects of information-theoretic topics |
Keywords:
limited information likelihood; entropy; \(I\)-projection; limited information posterior; correspondence between classical and Bayesian distribution theoriesSoftware:
nlmdlReferences:
| [1] | Andrews, D. W. K.: Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica 59, 817-858 (1991) · Zbl 0732.62052 |
| [2] | Andrews, D. W. K.; Monahan, J. C.: An improved heteroskedasticity and autocorrelation consistent covariance matrix estimator. Econometrica 60, 953-966 (1992) · Zbl 0778.62103 |
| [3] | Berk, R.: Consistency a posteriori. Annals of mathematical statistics 41, 894-906 (1970) · Zbl 0214.45703 |
| [4] | Chen, S. X.: On the accuracy of empirical likelihood confidence regions for linear regression models. Annals of the institute of statistical mathematics 45, 621-637 (1993) · Zbl 0799.62070 |
| [5] | Chen, S. X.: Empirical likelihood confidence intervals for linear regression coefficients. Journal of multivariate analysis 49, 24-40 (1994) · Zbl 0796.62040 |
| [6] | Cover, T. M.; Thomas, J. A.: Elements of information theory. (1991) · Zbl 0762.94001 |
| [7] | Csiszar, I.: I-divergence geometry of probability distributions and minimization problems. The annals of probability 3, No. 1, 146-158 (1975) · Zbl 0318.60013 |
| [8] | Diciccio, T.; Romano, J.: On adjustments to the signed root of the empirical likelihood statistics. Biometrika 76, 447-456 (1989) · Zbl 0676.62043 |
| [9] | Diciccio, T.; Romano, J.: Nonparametric confindence limits by resampling methods and least favorable families. International statistics review 58, 59-76 (1990) · Zbl 0715.62090 |
| [10] | Gallant, A. R.: Nonlinear statistical models. (1987) · Zbl 0611.62071 |
| [11] | Golan, A.; Judge, G. G.; Miller, D.: Maximum entropy econometrics, robust estimation with limited data. (1996) · Zbl 0884.62126 |
| [12] | Golan, A.; Judge, G. G.; Perloff, J.: A generalized maximum entropy approach to recovering information from multinomial response data. Journal of the American statistical association 91, 841-853 (1996) · Zbl 0869.62009 |
| [13] | Gordin, M. I.: The central limit theorem for stationary processes. Soviet mathematics doklady 10, 1174-1176 (1969) · Zbl 0212.50005 |
| [14] | Haberman, S. J.: Adjustment by minimum discriminant information. The annals of statistics 12, 971-988 (1984) · Zbl 0583.62020 |
| [15] | Hall, P.: Pseudo-likelihood theory for empirical likelihood. The annals of statistics 18, 121-140 (1990) · Zbl 0699.62040 |
| [16] | Hansen, L. P.: Large sample properties of generalized method of moments estimators. Econometrica 50, 1029-1054 (1982) · Zbl 0502.62098 |
| [17] | Imbens, G. W.: One step estimators for over-identified generalized method moments models. Review of economic studies 64, 359-383 (1997) · Zbl 0889.90039 |
| [18] | Imbens, G. W.; Spady, R. H.; Johnson, P.: Information theoretic approaches to inference in moment condition models. Econometrica 66, 333-357 (1998) · Zbl 1055.62512 |
| [19] | Inoue, A., 2000. A Bayesian GMM in large samples. Mimeo. |
| [20] | Jaynes, E. T.: On the rationale of maximum-entropy methods. Proceedings of the IEEE 70, 939-952 (1982) |
| [21] | Jones, L. K.: Approximation-theoretic derivation of logarithmic entropy principles for inverse problems and unique extension of the maximum entropy method to incorporate prior knowledge. SIAM journal of applied mathematics 49, 650-661 (1989) · Zbl 0672.62007 |
| [22] | Kim, J. Y.: Large sample properties of posterior densities in a time series with nonstationary components, Bayesian information criterion, and the likelihood principle. Econometrica 66, No. 2, 359-380 (1998) · Zbl 1056.62510 |
| [23] | Kim, J.Y., 2000. The generalized method of moments in the bayesian framework. Manuscript. |
| [24] | Kitamura, Y.: Empirical likelihood methods with weakly dependence processes. The annals of statistics 25, No. 5, 2084-2102 (1997) · Zbl 0881.62095 |
| [25] | Kitamura, Y.; Stutzer, M.: An information-theoretic alternative to procedures generalized method of moments estimation. Econometrica 65, 861-874 (1997) · Zbl 0894.62011 |
| [26] | Kolaczyk, E. D.: Empirical likelihood for generalized linear models. Statistica sinica 4, 199-218 (1994) · Zbl 0824.62062 |
| [27] | Kullback, S.: Information theory and statistics. (1959) · Zbl 0088.10406 |
| [28] | Kwan, Y. K.: Asymptotic Bayesian analysis based on a limited information estimator. Journal of econometrics 88, No. 1, 99-121 (1999) · Zbl 0933.62019 |
| [29] | Mittelhammer, R. C.; Judge, G.; Miller, D.: Econometric foundations. (2000) · Zbl 0961.62110 |
| [30] | Newey, W. K.; West, K. D.: A simple positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55, 703-708 (1987) · Zbl 0658.62139 |
| [31] | Owen, A.: Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75, 237-249 (1988) · Zbl 0641.62032 |
| [32] | Owen, A.: Empirical likelihood for linear models. The annals of statistics 19, 1725-1747 (1991) · Zbl 0799.62048 |
| [33] | Quin, J.: Empirical likelihood in biased sample problems. The annals of statistics 21, 1182-1196 (1993) · Zbl 0791.62052 |
| [34] | Quin, J.; Lawless, J.: Empirical likelihood and general estimating equations. The annals of statistics 23, 300-325 (1994) · Zbl 0799.62049 |
| [35] | Serfling, R.J., 1980. Approximation Theorems on Mathematical Statistics. John Wiley. · Zbl 0538.62002 |
| [36] | Shore, J. E.; Johnson, R. W.: Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE transactions 26, No. 1, 26-37 (1980) · Zbl 0429.94011 |
| [37] | Sims, C.A., 1988. Bayesian skepticism on unit root econometrics. Journal of Economic Dynamics and Control (12) 463–474. · Zbl 0647.62098 |
| [38] | Smith, R. J.: Alternative semi-parametric likelihood approaches to generalized method of moments estimation. The economics journal 107, 503-519 (1997) |
| [39] | Soofi, E. S.: Capturing the intangible concept of information. Journal of the American statistical association 89, No. 428, 1243-1254 (1994) · Zbl 0810.62012 |
| [40] | Sweeting, T.J., 1992. On asymptotic posterior normality in the multivariate case. In: Bernardo, J.M., Berger, J.O., David, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics 4, 825–835. Oxford University Press. |
| [41] | Sweeting, T. J.; Andekola, A. O.: Asymptotic posterior normality for stochastic processes revisited. Journal of the royal statistical society, series B 49, 215-222 (1987) · Zbl 0645.62092 |
| [42] | White, H.: Maximum likelihood estimation of misspecified models. Econometrica 50, 1-25 (1982) · Zbl 0478.62088 |
| [43] | Zellner, A.: Model, prior information and Bayesian analysis. Journal of econometrics 75, 51-68 (1994) · Zbl 0864.62014 |
| [44] | Zellner, A.: Bayesian method of moments/instrumental variable (BMOM/IV) analysis of mean and regression model. Modelling and prediction honoring seymour geisser, 61-74 (1996) · Zbl 0895.62037 |
| [45] | Zellner, A., 1997. The Bayesian method of moments (BMOM): theory and application. In: Fomby, T., Hill, R.C (Eds.). Advances in Econometrics. Cambridge University Press. |
| [46] | Zellner, A.: The finite sample properties of simultaneous equations’ estimates and estimators: Bayesian and non-Bayesian approaches. Journal of econometrics 83, 185-212 (1998) · Zbl 0888.62119 |
| [47] | Zellner, A.; Highfield, R. A.: Calculation of maximum entropy distributions and approximation of marginal posterior distributions. Journal of econometrics 37, 195-210 (1988) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
