Efficient sampling methods for truncated multivariate normal and Student-\(t\) distributions subject to linear inequality constraints. (English) Zbl 1423.62047
Summary: Sampling from a truncated multivariate distribution subject to multiple linear inequality constraints is a recurring problem in many areas in statistics and econometrics, such as the order-restricted regressions, censored data models, and shape-restricted non-parametric regressions. However, the sampling problem remains nontrivial due to the analytically intractable normalizing constant of the truncated multivariate distribution. We first develop an efficient rejection sampling method for the truncated univariate normal distribution, and analytically establish its superiority in terms of acceptance rates compared to some of the popular existing methods. We then extend our methodology to obtain samples from a truncated multivariate normal distribution subject to convex polytope restriction regions. Finally, we generalize the sampling method to truncated scale mixtures of multivariate normal distributions. Empirical results are presented to illustrate the superior performance of our proposed Gibbs sampler in terms of various criteria (e.g., mixing and integrated auto-correlation time).
MSC:
| 62H10 | Multivariate distribution of statistics |
| 62D05 | Sampling theory, sample surveys |
| 62E15 | Exact distribution theory in statistics |
Keywords:
Gibbs sampler; rejection sampling; truncated multivariate normal distribution; truncated multivariate Student-\(t\) distribution; truncated scale mixture of multivariate normal distributionReferences:
| [1] | Breslaw, J. 1994. Random sampling from a truncated multivariate normal distribution. Appl. Math. Lett., 7(1), 1-6. · Zbl 0795.62060 · doi:10.1016/0893-9659(94)90042-6 |
| [2] | Damien, P., and S. G. Walker. 2001. Sampling truncated normal, beta, and gamma densities. J. Comput. Graph. Stat., 10(2), 206-215. · doi:10.1198/10618600152627906 |
| [3] | Gelfand, A. E., and A. F. M. Smith. 1990. Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc., 85(410), 398-409. · Zbl 0702.62020 · doi:10.1080/01621459.1990.10476213 |
| [4] | Gelfand, A. E., A. F. M. Smith, and T.-M. Lee. 1992. Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. J. Am. Stat. Assoc., 87(418), 523-532. · doi:10.1080/01621459.1992.10475235 |
| [5] | Geman, S., and D. Geman. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intell., PAMI-6, 721-741. · Zbl 0573.62030 · doi:10.1109/TPAMI.1984.4767596 |
| [6] | Geweke, J.; Keramidas, E. M (ed.), Efficient simulation from the multivariate normal and Student-t distributions subject to linear constraints and the evaluation of constraint probabilities, 571-578 (1991), Fairfax Station, VA |
| [7] | Geweke, J.; Lee, J. C (ed.); Johnson, W. O (ed.); Zellner, A. (ed.), Bayesian inference for linear models subject to linear inequality constraints, 248-263 (1996), New York, NY · Zbl 0895.62028 · doi:10.1007/978-1-4612-2414-3_15 |
| [8] | Gómez-Sánchez-Manzano, E., M. Gómez-Villegas, and J. Marín. 2008. Multivariate exponential power distributions as mixtures of normal distributions with bayesian applications. Commun. Stat. Theory Methods, 37(6), 972-985. · Zbl 1135.62041 · doi:10.1080/03610920701762754 |
| [9] | Hajivassiliou, V. A., and D. L. Mcfadden, 1990. The method of simulated scores for the estimation of ldv models: With an application to the problem of external debt crises. Cowles Foundation Discussion Paper no. 967. https://ideas.repec.org/p/cwl/cwldpp/967.html |
| [10] | Horrace, W. C. 2005. Some results on the multivariate truncated normal distribution. J. Multivariate Anal., 94(1), 209-221. · Zbl 1065.62098 · doi:10.1016/j.jmva.2004.10.007 |
| [11] | Liechty, M. W., and J. Lu. 2010. Multivariate normal slice sampling. J. Comput. Graph. Stat., 19(2), 281-294. With supplementary material available online. · doi:10.1198/jcgs.2009.07138 |
| [12] | Neal, R. M. 2003. Slice sampling. Ann. Stat., 31(3), 705-767. With discussions and a rejoinder by the author. · Zbl 1051.65007 · doi:10.1214/aos/1056562461 |
| [13] | Robert, C. P. 1995. Simulation of truncated normal variables. Stat. Comput., 5, 121-125. · doi:10.1007/BF00143942 |
| [14] | Robert, C. P., and G. Casella. 2004. Monte Carlo statistical methods. 2nd ed. Springer Texts in Statistics. New York, NY: Springer-Verlag. · Zbl 1096.62003 · doi:10.1007/978-1-4757-4145-2 |
| [15] | Rodrigues-Yam, G., R. A. Davis, and L. L. Scharf. 2004. Efficient Gibbs sampling of truncated multivariate normal with application to constrained linear regression. Technical report, Colorado State University, Fort Collins, CO. |
| [16] | Tierney, L. 1994. Markov chains for exploring posterior distributions. Ann. Stat., 22(4), 1701-1762. With discussion and a rejoinder by the author. · Zbl 0829.62080 · doi:10.1214/aos/1176325750 |
| [17] | Yu, J., and G. Tian. 2011. Efficient algorithms for generating truncated multivariate normal distributions. Acta Math. Appl. Sin. (English Ser.), 27, 601-612. · Zbl 1272.62046 · doi:10.1007/s10255-011-0110-x |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
