Efficient Monte Carlo computation of Fisher information matrix using prior information. (English) Zbl 1464.62052
Summary: The Fisher information matrix (FIM) is a critical quantity in several aspects of mathematical modeling, including input selection and confidence region calculation. Analytical determination of the FIM in a general setting, especially in nonlinear models, may be difficult or almost impossible due to intractable modeling requirements or/and intractable high-dimensional integration.To circumvent these difficulties, a Monte Carlo simulation based technique, known as resampling algorithm, is usually recommended, in which values of the log-likelihood function or its exact stochastic gradient computed based on a set of pseudo-data vectors are used. The current work proposes an extension of this resampling algorithm in order to enhance the statistical qualities of the estimator of the FIM. This modified resampling algorithm is useful in those cases when some elements of the FIM are analytically known from prior information and the rest of the elements are unknown. The estimator of the FIM resulting from the proposed algorithm simultaneously preserves the analytically known elements and reduces variances of the estimators of the unknown elements. This is achieved by capitalizing on the information contained in the known elements.
MSC:
| 62-08 | Computational methods for problems pertaining to statistics |
References:
| [1] | Badeau, R.; David, B.; Richard, G., Cramér-Rao bounds for multiple poles and coefficients of quasi-polynomials in colored noise, IEEE Transactions on Signal Processing, 56, 8, 3458-3467 (2008) · Zbl 1390.94084 |
| [2] | Bickel, P. J.; Doksum, K. A., Mathematical Statistics: Basic Ideas and Selected Topics, vol. I (2001), Prentice Hall: Prentice Hall New Jersey · Zbl 0403.62001 |
| [3] | Chryssolouris, G.; Lee, M.; Ramsey, A., Confidence interval prediction for neural networks models, IEEE Transactions on Neural Networks, 7, 1, 229-232 (1996) |
| [4] | Cover, T. C.; Thomas, J. A., Elements of Information Theory (2006), Wiley-Interscience: Wiley-Interscience New Jersey · Zbl 1140.94001 |
| [6] | Das, S.; Ghanem, R.; Spall, J. C., Asymptotic sampling distribution for polynomial chaos representation from data: A maximum entropy and Fisher information approach, SIAM Journal on Scientific Computing, 30, 5, 2207-2234 (2008) · Zbl 1181.62004 |
| [7] | Efron, B.; Hinkley, D. V., Assessing the accuracy of the maximum likelihood estimator: Observed versus expected fisher information, Biometrika, 65, 3, 457-487 (1978), (with discussion) · Zbl 0401.62002 |
| [8] | Gómez, H. W.; Venegas, O.; Bolfarine, H., Skew-symmetric distributions generated by the distribution function of the normal distribution, Environmetrics, 18, 4, 395-407 (2007) |
| [9] | Hernandez, M. L.; Ristic, B.; Farina, A.; Sathyan, T.; Kirubarajan, T., Performance measure for Markovian switching systems using best-fitting Gaussian distributions, IEEE Transactions on Aerospace and Electric Systems, 44, 2, 724-747 (2008) |
| [10] | Hwang, J.; Ding, A., Prediction intervals for artificial neural networks, Journal of the American Statistical Association, 92, 438, 748-757 (1997) · Zbl 1090.62559 |
| [11] | Jeffreys, H., An invariant form for the prior probability in estimation problems, Proceedings of the Royal Society of London. Series A Mathematical and Physical Sciences, 186, 1007, 453-461 (1946) · Zbl 0063.03050 |
| [13] | Klein, A.; Mélard, G.; Spreij, P., On the resultant property of the Fisher information matrix of a vector ARMA process, Linear Algebra and its Applications, 403, 291-313 (2005) · Zbl 1084.15021 |
| [14] | Kurita, T., Iterative weighted least squares algorithms for neural networks classifiers, (Proceedings of the Third Workshop on Algorithmic Learning Theory (1992), Springer-Verlag: Springer-Verlag London, UK), 77-86 |
| [15] | Levy, L. J., Generic maximum likelihood identification algorithms for linear state space models, (Proceedings of the Conference on Information Science and Systems (1995), Baltimore: Baltimore MD), 659-667 |
| [16] | Mathai, A., Jacobians of Matrix Transformations and Functions of Matrix Argument (1997), World Scientific: World Scientific Mountain View, California · Zbl 0889.33001 |
| [17] | Nadarajah, S., An alternative inverse Gaussian distribution, Mathematics and Computers in Simulation, 79, 5, 1721-1729 (2009) · Zbl 1163.62007 |
| [18] | Neerchal, N. K.; Morel, J. G., An improved method for the computation of maximum likeliood estimates for multinomial overdispersion models, Computational Statistics and Data Analysis, 49, 33-43 (2005) · Zbl 1430.62015 |
| [19] | Rao, C. R., Linear Statistical Inference and its Applications (1973), John Wiley & Sons: John Wiley & Sons New York · Zbl 0256.62002 |
| [20] | Retout, S.; Duffull, S.; Mentre, F., Development and implementation of the population Fisher information matrix for the evaluation of population pharmacokinetic designs, Computer Methods and Programs in Biomedicine, 65, 141-151 (2001) |
| [21] | Retout, S.; Mentre, F.; Bruno, R., Fisher information matrix for non-linear mixed-effects models: Evaluation and application for optimal design of enoxaparin population pharmacokinetics, Statistics in Medicine, 21, 2623-2639 (2002) |
| [22] | Shumway, R. H.; Olsen, D. E.; Levy, L. J., Estimation and tests of hypotheses for the initial mean and covariance in the Kalman filter model, Communications in Statistics — Theory and Methods, 10, 16, 1625-1641 (1981) · Zbl 0472.62095 |
| [23] | Spall, J. C., Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Transactions on Automatic Control, 37, 3, 332-341 (1992) · Zbl 0745.60110 |
| [24] | Spall, J. C., Adaptive stochastic approximation by the simultaneous perturbation method, IEEE Transactions on Automatic Control, 45, 10, 1839-1853 (2000) · Zbl 0990.93125 |
| [25] | Spall, J. C., Introduction to Stochastic Search and Optimization: Estimation, Simulation and Control (2003), Wiley-Interscience: Wiley-Interscience New Jersey · Zbl 1088.90002 |
| [26] | Spall, J. C., Monte Carlo computation of the Fisher information matrix in nonstandard settings, Journal of Computational and Graphical Statistics, 14, 4, 889-909 (2005) |
| [27] | Spall, J. C., Feedback and weighting mechanisms for improving Jacobian estimates in the adaptive simultaneous perturbation algorithm, IEEE Transactions on Automatic Control, 54, 6, 1216-1229 (2009) · Zbl 1367.93734 |
| [28] | Spall, J. C.; Garner, J. P., Parameter identification for state-space models with nuisance parameters, IEEE Transactions on Aerospace and Electronic Systems, 26, 6, 992-998 (1990) |
| [29] | Tegmark, M.; Taylor, A. N.; Heavens, A., Karhunen-Loève eigenvalue problems in cosmology: How should we tackle large data sets?, The Astrophysical Journal, 480, 1, 22-35 (1997) |
| [30] | Šimandl, M.; Královec, J.; Tichavský, P., Filtering, predictive, and smoothing Cramér-Rao bounds for discrete-time nonlinear dynamic systems, Automatica, 37, 11, 1703-1716 (2001) · Zbl 1031.93146 |
| [31] | Yang, Z.; Laaksonen, J., Principal whitened gradient for information geometry, Neural Networks, 21, 2-3, 232-240 (2008) · Zbl 1254.68326 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
