×

High-dimensional vector autoregressive time series modeling via tensor decomposition. (English) Zbl 1506.62373

Summary: The classical vector autoregressive model is a fundamental tool for multivariate time series analysis. However, it involves too many parameters when the number of time series and lag order are even moderately large. This article proposes to rearrange the transition matrices of the model into a tensor form such that the parameter space can be restricted along three directions simultaneously via tensor decomposition. In contrast, the reduced-rank regression method can restrict the parameter space in only one direction. Besides achieving substantial dimension reduction, the proposed model is interpretable from the factor modeling perspective. Moreover, to handle high-dimensional time series, this article considers imposing sparsity on factor matrices to improve the model interpretability and estimation efficiency, which leads to a sparsity-inducing estimator. For the low-dimensional case, we derive asymptotic properties of the proposed least squares estimator and introduce an alternating least squares algorithm. For the high-dimensional case, we establish nonasymptotic properties of the sparsity-inducing estimator and propose an ADMM algorithm for regularized estimation. Simulation experiments and a real data example demonstrate the advantages of the proposed approach over various existing methods.

MSC:

62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
68T09 Computational aspects of data analysis and big data
62H99 Multivariate analysis

Software:

Cross; SOFAR

References:

[1] Amengual, D.; Watson, M. W., “Consistent Estimation of the Number of Dynamic Factors in Large N and T Panel, Journal of Business & Economic Statistics, 25, 91-96 (2007)
[2] Athanasopoulos, G.; Vahid, F., “VARMA Versus VAR for Macroeconomic Forecasting,”, Journal of Business & Economic Statistics, 26, 237-252 (2008)
[3] Bai, J.; Ng, S., “Large Dimensional Factor Analysis, Foundations and Trends® in Econometrics, 3, 89-163 (2008) · doi:10.1561/0800000002
[4] Bai, J.; Wang, P., “Econometric Analysis of Large Factor Models, Annual Review of Economics, 8, 53-80 (2016) · doi:10.1146/annurev-economics-080315-015356
[5] Bai, Z. D., “Circular Law, The Annals of Probability, 25, 494-529 (1997) · Zbl 0871.62018 · doi:10.1214/aop/1024404298
[6] Basu, S.; Li, X.; Michailidis, G., “Low Rank and Structured Modeling of High-Dimensional Vector Autoregressions, IEEE Transactions on Signal Processing, 67, 1207-1222 (2019) · Zbl 1415.94051 · doi:10.1109/TSP.2018.2887401
[7] Basu, S.; Michailidis, G., “Regularized Estimation in Sparse High-Dimensional Time Series Models, The Annals of Statistics, 43, 1535-1567 (2015) · Zbl 1317.62067 · doi:10.1214/15-AOS1315
[8] Bernanke, B. S.; Boivin, J.; Eliasz, P., “Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) Approach,”, Quarterly Journal of Economics, 120, 387-422 (2005)
[9] Bickel, P. J.; Ritov, Y.; Tsybakov, A. B., “Simultaneous Analysis of Lasso and Dantzig Selector, The Annals of Statistics, 37, 1705-1732 (2009) · Zbl 1173.62022 · doi:10.1214/08-AOS620
[10] Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J., “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Foundations and Trends in Machine learning, 3, 1-122 (2011) · Zbl 1229.90122 · doi:10.1561/2200000016
[11] Bunea, F.; She, Y.; Wegkamp, M. H., “Joint Variable and Rank Selection for Parsimonious Estimation of High-Dimensional Matrices, The Annals of Statistics, 40, 2359-2388 (2012) · Zbl 1373.62246 · doi:10.1214/12-AOS1039
[12] Carriero, A.; Kapetanios, G.; Marcellino, M., “Forecasting Large Datasets With Bayesian Reduced Rank Multivariate Models, Journal of Applied Econometrics, 26, 735-761 (2011) · doi:10.1002/jae.1150
[13] Chan, J. C. C.; Eisenstat, E.; Koop, G., “Large Bayesian VARMAs, Journal of Econometrics, 192, 374-390 (2016) · Zbl 1420.62373 · doi:10.1016/j.jeconom.2016.02.005
[14] Chen, K.; Chan, K.-S.; Stenseth, N. C., “Reduced Rank Stochastic Regression With a Sparse Singular Value Decomposition, Journal of the Royal Statistical Society, Series B, 74, 203-221 (2012) · Zbl 1411.62182 · doi:10.1111/j.1467-9868.2011.01002.x
[15] Chen, K.; Dong, H.; Chan, K.-S., “Reduced Rank Regression via Adaptive Nuclear Norm Penalization, Biometrika, 100, 901-920 (2013) · Zbl 1279.62115 · doi:10.1093/biomet/ast036
[16] Chen, L.; Huang, J. Z., “Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection, Journal of the American Statistical Association, 107, 1533-1545 (2012) · Zbl 1258.62075 · doi:10.1080/01621459.2012.734178
[17] Davis, R. A.; Zang, P.; Zheng, T., “Sparse Vector Autoregressive Modeling, Journal of Computational and Graphical Statistics, 25, 1077-1096 (2016) · doi:10.1080/10618600.2015.1092978
[18] De Lathauwer, L.; De Moor, B.; Vandewalle, J., “A Multilinear Singular Value Decomposition, SIAM Journal on Matrix Analysis and Applications, 21, 1253-1278 (2000) · Zbl 0962.15005 · doi:10.1137/S0895479896305696
[19] De Mol, C.; Giannone, D.; Reichlin, L., “Forecasting Using a Large Number of Predictors: Is Bayesian Shrinkage a Valid Alternative to Principal Components?, Journal of Econometrics, 146, 318-328 (2008) · Zbl 1429.62659 · doi:10.1016/j.jeconom.2008.08.011
[20] Dias, G. F.; Kapetanios, G., “Estimation and Forecasting in Vector Autoregressive Moving Average Models for Rich Datasets, Journal of Econometrics, 202, 75-91 (2018) · Zbl 1378.62065 · doi:10.1016/j.jeconom.2017.06.022
[21] Ding, S.; Cook, R. D., “Matrix Variate Regressions and Envelope Models, Journal of the Royal Statistical Society, Series B, 80, 387-408 (2018) · Zbl 06849260 · doi:10.1111/rssb.12247
[22] Guo, S.; Wang, Y.; Yao, Q., “High-Dimensional and Banded Vector Autoregressions, Biometrika, 103, 889-903 (2016) · doi:10.1093/biomet/asw046
[23] Hallin, M.; Lippi, M., “Factor Models in High-Dimensional Time Series: A Time-Domain Approach,”, Stochastic Processes and Their Applications, 123, 2678-2695 (2013) · Zbl 1285.62106
[24] Han, F.; Lu, H.; Liu, H., “A Direct Estimation of High Dimensional Stationary Vector Autoregressions,”, The Journal of Machine Learning Research, 16, 3115-3150 (2015) · Zbl 1351.62165
[25] Kock, A. B.; Callot, L., “Oracle Inequalities for High Dimensional Vector Autoregressions, Journal of Econometrics, 186, 325-344 (2015) · Zbl 1331.62348 · doi:10.1016/j.jeconom.2015.02.013
[26] Kolda, T. G.; Bader, B. W., “Tensor Decompositions and Applications, SIAM Review, 51, 455-500 (2009) · Zbl 1173.65029 · doi:10.1137/07070111X
[27] Koop, G. M., “Forecasting With Medium and Large Bayesian VARs, Journal of Applied Econometrics, 28, 177-203 (2013) · doi:10.1002/jae.1270
[28] Lai, R.; Osher, S., “A Splitting Method for Orthogonality Constrained Problems, Journal of Scientific Computing, 58, 431-449 (2014) · Zbl 1296.65087 · doi:10.1007/s10915-013-9740-x
[29] Li, G.; Leng, C.; Tsai, C.-L., “A Hybrid Bootstrap Approach to Unit Root Tests, Journal of Time Series Analysis, 35, 299-321 (2014) · Zbl 1311.62133 · doi:10.1111/jtsa.12019
[30] Li, G.; Yang, D.; Nobel, A. B.; Shen, H., “Supervised Singular Value Decomposition and Its Asymptotic Properties, Journal of Multivariate Analysis, 146, 7-17 (2016) · Zbl 1336.62129 · doi:10.1016/j.jmva.2015.02.016
[31] Li, L.; Zhang, X., “Parsimonious Tensor Response Regression, Journal of the American Statistical Association, 112, 1131-1146 (2017) · doi:10.1080/01621459.2016.1193022
[32] Li, X.; Xu, D.; Zhou, H.; Li, L., “Tucker Tensor Regression and Neuroimaging Analysis, Statistics in Biosciences, 10, 520-545 (2018) · doi:10.1007/s12561-018-9215-6
[33] Lian, H.; Feng, S.; Zhao, K., “Parametric and Semiparametric Reduced-Rank Regression With Flexible Sparsity, Journal of Multivariate Analysis, 136, 163-174 (2015) · Zbl 1308.62144 · doi:10.1016/j.jmva.2015.01.013
[34] Lütkepohl, H., New Introduction to Multiple Time Series Analysis (2005), Berlin: Springer, Berlin · Zbl 1072.62075
[35] Negahban, S.; Wainwright, M. J., “Estimation of (Near) Low-Rank Matrices With Noise and High-Dimensional Scaling, The Annals of Statistics, 39, 1069-1097 (2011) · Zbl 1216.62090 · doi:10.1214/10-AOS850
[36] Raskutti, G.; Yuan, M.; Chen, H., “Convex Regularization for High-Dimensional Multi-Response Tensor Regression, The Annals of Statistics, 47, 1554-1584 (2019) · Zbl 1428.62324 · doi:10.1214/18-AOS1725
[37] Ravenna, F., “Vector Autoregressions and Reduced Form Representations of DSGE Models, Journal of Monetary Economics, 54, 2048-2064 (2007) · doi:10.1016/j.jmoneco.2006.09.002
[38] Reinsel, G., “Some Results on Multivariate Autoregressive Index Models, Biometrika, 70, 145-156 (1983) · Zbl 0509.62083 · doi:10.1093/biomet/70.1.145
[39] Said, E. S.; Dickey, D. A., “Testing for Unit Roots in Autoregressive-Moving Average Models of Unknown Order, Biometrika, 71, 599-607 (1984) · Zbl 0564.62075 · doi:10.1093/biomet/71.3.599
[40] Shapiro, A., “Asymptotic Theory of Overparameterized Structural Models, Journal of the American Statistical Association, 81, 142-149 (1986) · Zbl 0596.62069 · doi:10.1080/01621459.1986.10478251
[41] Staudenmayer, J.; Buonaccorsi, J. P., “Measurement Error in Linear Autoregressive Models, Journal of the American Statistical Association, 100, 841-852 (2005) · Zbl 1117.62430 · doi:10.1198/016214504000001871
[42] Stock, J. H.; Watson, M. W., Implications of Dynamic Factor Models for VAR Analysis, National Bureau of Economic Research Working Paper, 11467 (2005)
[43] Stock, J. H.; Watson, M. W.; Castle, J. L.; Shephard, N., The Methodology and Practice of Econometrics: A Festschrift in Honour of David F. Hendry, Forecasting in Dynamic Factor Models Subject to Structural Instability,”, 173-205 (2009), Oxford: Oxford University Press, Oxford · Zbl 1456.62219
[44] Stock, J. H.; Watson, M. W.; Clements, M. P.; Hendry, D. F., The Oxford Handbook of Economic Forecasting, Dynamic Factor Models,”, 35-60 (2011), Oxford: Oxford University Press, Oxford
[45] Sun, W. W.; Li, L., “Store: Sparse Tensor Response Regression and Neuroimaging Analysis, The Journal of Machine Learning Research, 18, 4908-4944 (2017) · Zbl 1442.62773
[46] Tibshirani, R., “Regression Shrinkage and Selection via the Lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288 (1996) · Zbl 0850.62538 · doi:10.1111/j.2517-6161.1996.tb02080.x
[47] Tsay, R. S., “Testing and Modeling Multivariate Threshold Models, Journal of the American Statistical Association, 93, 1188-1202 (1998) · Zbl 1063.62578 · doi:10.1080/01621459.1998.10473779
[48] Tsay, R. S., Analysis of Financial Time Series (2010), Hoboken NJ: Wiley, Hoboken NJ · Zbl 1209.91004
[49] Tucker, L. R., “Some Mathematical Notes on Three-Mode Factor Analysis, Psychometrika, 31, 279-311 (1966) · doi:10.1007/BF02289464
[50] Uematsu, Y.; Fan, Y.; Chen, K.; Lv, J.; Lin, W., “SOFAR: Large-Scale Association Network Learning, IEEE Transactions on Information Theory, 65, 4924-4939 (2019) · Zbl 1432.68402 · doi:10.1109/TIT.2019.2909889
[51] Velu, R. P.; Reinsel, G. C., Multivariate Reduced-Rank Regression: Theory and Applications, 136 (2013), New York: Springer-Verlag, New York · Zbl 0909.62066
[52] Velu, R. P.; Reinsel, G. C.; Wichern, D. W., “Reduced Rank Models for Multiple Time Series, Biometrika, 73, 105-118 (1986) · Zbl 0612.62121 · doi:10.1093/biomet/73.1.105
[53] Vu, V. Q.; Lei, J., “Minimax Sparse Principal Subspace Estimation in High Dimensions, The Annals of Statistics, 41, 2905-2947 (2013) · Zbl 1288.62103 · doi:10.1214/13-AOS1151
[54] Wang, D.; Liu, X.; Chen, R., “Factor Models for Matrix-Valued High-Dimensional Time Series, Journal of Econometrics, 208, 231-248 (2019) · Zbl 1452.62684 · doi:10.1016/j.jeconom.2018.09.013
[55] Wang, Y.; Yin, W.; Zeng, J., “Global Convergence of ADMM in Nonconvex Nonsmooth Optimization, Journal of Scientific Computing, 78, 29-63 (2019) · Zbl 1462.65072 · doi:10.1007/s10915-018-0757-z
[56] Wang, Z.; Gu, Q.; Ning, Y.; Liu, H., High Dimensional EM Algorithm: Statistical Optimization and Asymptotic Normality, Advances in Neural Information Processing Systems, 2521-2529 (2015)
[57] Wilms, I., Basu, S., Bien, J., and Matteson, D. S. (2017), “Sparse Identification and Estimation of Large-Scale Vector Autoregressive Moving Averages,” arXiv no. 1707.09208.
[58] Wu, W.-B.; Wu, Y. N., “Performance Bounds for Parameter Estimates of High-Dimensional Linear Models With Correlated Errors, Electronic Journal of Statistics, 10, 352-379 (2016) · Zbl 1333.62172 · doi:10.1214/16-EJS1108
[59] Xia, Q.; Xu, W.; Zhu, L., “Consistently Determining the Number of Factors in Multivariate Volatility Modelling, Statistica Sinica, 25, 1025-1044 (2015) · Zbl 1415.62067 · doi:10.5705/ss.2013.252
[60] Yuan, M.; Ekici, A.; Lu, Z.; Monteiro, R., “Dimension Reduction and Coefficient Estimation in Multivariate Linear Regression, Journal of the Royal Statistical Society, Series B, 69, 329-346 (2007) · Zbl 07555355 · doi:10.1111/j.1467-9868.2007.00591.x
[61] Zhang, A., “Cross: Efficient Low-Rank Tensor Completion, The Annals of Statistics, 47, 936-964 (2019) · Zbl 1416.62298 · doi:10.1214/18-AOS1694
[62] Zhao, J.; Leng, C., “Structure Lasso for Regression With Matrix Covariates,”, Statistica Sinica, 24, 799-814 (2014) · Zbl 1285.62064
[63] Zheng, L.; Raskutti, G., “Testing for High-Dimensional Network Parameters in Auto-Regressive Models, Electronic Journal of Statistics, 13, 4977-5043 (2019) · Zbl 1434.62201 · doi:10.1214/19-EJS1646
[64] Zheng, Y.; Cheng, G., “Finite Time Analysis of Vector Autoregressive Models Under Linear Restrictions, Biometrika (2020) · Zbl 07458265 · doi:10.1093/biomet/asaa065
[65] Zhou, H.; Li, L.; Zhu, H., “Tensor Regression With Applications in Neuroimaging Data Analysis, Journal of the American Statistical Association, 108, 540-552 (2013) · Zbl 06195959 · doi:10.1080/01621459.2013.776499
[66] Zhu, X.; Pan, R.; Li, G.; Liu, Y.; Wang, H., “Network Vector Autoregression, The Annals of Statistics, 45, 1096-1123 (2017) · Zbl 1381.62256 · doi:10.1214/16-AOS1476
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.