On Koopman mode decomposition and tensor component analysis. (English) Zbl 1462.37089
Summary: Koopman mode decomposition and tensor component analysis [also known as CANDECOMP (canonical decomposition)/PARAFAC (parallel factorization)] are two popular approaches of decomposing high dimensional datasets into modes that capture the most relevant features and/or dynamics. Despite their similar goal, the two methods are largely used by different scientific communities and are formulated in distinct mathematical languages. We examine the two together and show that, under certain conditions on the data, the theoretical decomposition given by the tensor component analysis is the same as that given by Koopman mode decomposition. This provides a “bridge” with which the two communities should be able to more effectively communicate. Our work provides new possibilities for algorithmic approaches to Koopman mode decomposition and tensor component analysis and offers a principled way in which to compare the two methods. Additionally, it builds upon a growing body of work showing that dynamical systems theory and Koopman operator theory, in particular, can be useful for problems that have historically made use of optimization theory.
©2021 American Institute of Physics
©2021 American Institute of Physics
MSC:
| 37M10 | Time series analysis of dynamical systems |
References:
| [1] | Koopman, B. O., Hamiltonian systems and transformation in Hilbert space, Proc. Natl. Acad. Sci., 17, 315 (1931) · Zbl 0002.05701 · doi:10.1073/pnas.17.5.315 |
| [2] | Koopman, B. O.; Neumann, J. V., Dynamical systems of continuous spectra, Proc. Natl. Acad. Sci., 18, 255 (1932) · Zbl 0006.22702 · doi:10.1073/pnas.18.3.255 |
| [3] | Mezic, I., Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dyn., 41, 309 (2005) · Zbl 1098.37023 · doi:10.1007/s11071-005-2824-x |
| [4] | Budisíc, M.; Mohr, R.; Mezic, I., Applied Koopmanism, Chaos, 22, 047510 (2012) · Zbl 1319.37013 · doi:10.1063/1.4772195 |
| [5] | Mezic, I., Spectrum of the Koopman operator, spectral expansions in functional spaces, and state-space geometry, J. Nonlinear Sci., 30, 2091 (2019) · Zbl 1467.37085 · doi:10.1007/s00332-019-09598-5 |
| [6] | Brunton, S. L.; Budišić, M.; Kaiser, E.; Kutz, J. N. |
| [7] | Mezic, I. |
| [8] | Rowley, C. W.; Mezic, I.; Bagheri, S.; Schlatter, P.; Henningson, S. S., Spectral analysis of nonlinear flows, J. Fluid Mech., 641, 115 (2009) · Zbl 1183.76833 · doi:10.1017/S0022112009992059 |
| [9] | Mezic, I., Analysis of fluid flows via spectral properties of the Koopman operator, Annu. Rev. Fluid Mech., 45, 357 (2013) · Zbl 1359.76271 · doi:10.1146/annurev-fluid-011212-140652 |
| [10] | Arbabi, H.; Mezic, I., Study of dynamics in post transient flows using Koopman mode decomposition, Phys. Rev. Fluids, 2, 124402 (2017) · doi:10.1103/PhysRevFluids.2.124402 |
| [11] | Susuki, Y.; Mezíc, I.; Raak, F.; Hikihara, T., Applied Koopman operator theory for power systems technology, Nonlinear Theor. Appl. IEICE, 7, 430 (2016) · doi:10.1587/nolta.7.430 |
| [12] | Korda, M.; Susuki, Y.; Mezic, I., Power grid transient stabilization using Koopman model predictive control, IFAC-PapersOnLine, 51, 297 (2018) · doi:10.1016/j.ifacol.2018.11.718 |
| [13] | Dogra, A. S. and Redman, W. T., “Optimizing neural networks via Koopman operator theory,” in Advances in Neural Information Processing Systems 33, NeurIPS 2020 (Neural Information Processing Systems Foundation, 2020). |
| [14] | Manojlović, I.; Fonoberova, M.; Mohr, R.; Andrejčuk, A.; Drmač, Z.; Kevrekidis, Y.; Mezić, I. |
| [15] | Tano, M. E.; Portwood, G. D.; Ragusa, J. C. |
| [16] | Naiman, I.; Azencot, O. |
| [17] | Brunton, B. W.; Johnson, L. A.; Ojemann, J. G.; Kutz, J. N., Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition, J. Neurosci. Methods, 258, 1 (2016) · doi:10.1016/j.jneumeth.2015.10.010 |
| [18] | Klus, S.; Nüske, F.; Koltai, P.; Wu, H.; Kevrekidis, I.; Schutte, C.; Noe, F., Data-driven model reduction and transfer operator approximation, J. Nonlinear Sci., 28, 985 (2018) · Zbl 1396.37083 · doi:10.1007/s00332-017-9437-7 |
| [19] | Lu, H.; Tartakovsky, D. M., Prediction accuracy of dynamic mode decomposition, SIAM J. Sci. Comput., 42, A1639-A1662 (2020) · Zbl 1448.65134 · doi:10.1137/19M1259948 |
| [20] | Bradde, S.; Bialek, W., PCA meets RG, J. Stat. Phys., 167, 462-475 (2017) · Zbl 1378.82027 · doi:10.1007/s10955-017-1770-6 |
| [21] | Redman, W. T., Renormalization group as a Koopman operator, Phys. Rev. E, 101, 060104 (2020) · doi:10.1103/PhysRevE.101.060104 |
| [22] | Marrouch, N.; Slawinska, J.; Giannakis, D.; Read, H., Data-driven Koopman operator approach for computational neuroscience, Ann. Math. Artif. Intell., 88, 1155-1173 (2020) · Zbl 1466.37073 · doi:10.1007/s10472-019-09666-2 |
| [23] | Balakhrishnan, S., Hasnain, A., Boddupalli, N., Joshy, D. M., Egbet, R. G., and Yeung, E., “Prediction of fitness in bacteria with causal jump dynamic mode decomposition,” in 2020 American Control Conference (IEEE, 2020), pp. 3749-3756. |
| [24] | Carroll, J. D.; Chang, J.-J., Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition, Psychometrika, 35, 283-319 (1970) · Zbl 0202.19101 · doi:10.1007/BF02310791 |
| [25] | Harshman, R., “Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multi-model factor analysis,” UCLA Working Papers in Phonetics, 1970. |
| [26] | Vervliet, N., Debals, O., Sorber, L., Van Barel, M., and De Lathauwer, L., Tensorlab 3.0. |
| [27] | Kolda, T. G.; Bader, B. W., Tensor decompositions and applications, SIAM Rev., 51, 455-500 (2009) · Zbl 1173.65029 · doi:10.1137/07070111X |
| [28] | Hong, D.; Kolda, T. G.; Duersch, J. D., Generalized canonical polyadic tensor decomposition, SIAM Rev., 62, 133-163 (2020) · Zbl 1432.68385 · doi:10.1137/18M1203626 |
| [29] | Rambhatla, S., Li, X., and Haupt, J., “Provable online CP/PARAFAC decomposition of a structured tensor via dictionary learning,” in Advances in Neural Information Processing Systems 33, NeurIPS \(2020 (b l a c k s q u a r e, 2020)\). |
| [30] | Kruskal, J. B., Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics, Linear Algebra Appl., 18, 95-138 (1977) · Zbl 0364.15021 · doi:10.1016/0024-3795(77)90069-6 |
| [31] | Martınez-Montes, E.; Valdés-Sosa, P. A.; Miwakeichi, F.; Goldman, R. I.; Cohen, M. S., Concurrent EEG/fMRI analysis by multiway partial least squares, NeuroImage, 22, 1023-1034 (2004) · doi:10.1016/j.neuroimage.2004.03.038 |
| [32] | Miwakeichi, F.; Martınez-Montes, E.; Valdés-Sosa, P. A.; Nishiyama, N.; Mizuhara, H.; Yamaguchi, Y., Decomposing EEG data into space-time-frequency components using parallel factor analysis, NeuroImage, 22, 1035-1045 (2004) · doi:10.1016/j.neuroimage.2004.03.039 |
| [33] | Beckmann, C.; Smith, S., Tensorial extensions of independent component analysis for multisubject FMRI analysis, NeuroImage, 25, 294-311 (2005) · doi:10.1016/j.neuroimage.2004.10.043 |
| [34] | Acar. C. Aykut-Bingol, E.; Bingol, H.; Bro, R.; Yener, B., Multiway analysis of epilepsy tensors, Bioinformatics, 23, i10-i18 (2007) · doi:10.1093/bioinformatics/btm210 |
| [35] | Cichocki, A.; De Vos, M.; De Lathauwer, L.; Vanrumste, B.; Van Huffel, S.; Van Paesschen, W., Canonical decomposition of ictal scalp EEG and accurate source localisation: Principles and simulation study, Comput. Intell. Neurosci., 2007, 058253 (2007) |
| [36] | De Vos, M.; Vergult, A.; De Lathauwer, L.; De Clercq, W.; Van Huffel, S.; Dupont, P.; Palmini, A.; Van Paesschen, W., Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone, NeuroImage, 37, 844-854 (2007) · doi:10.1016/j.neuroimage.2007.04.041 |
| [37] | Mørup, M.; Hansen, L. K.; Herrmann, C. S.; Parnas, J.; Arnfred, S. M., Parallel factor analysis as an exploratory tool for wavelet transformed event-related EEG, NeuroImage, 29, 938-947 (2006) · doi:10.1016/j.neuroimage.2005.08.005 |
| [38] | Mørup, M.; Hansen, L. K.; Arnfred, S. M., ERPWAVELAB: A toolbox for multi-channel analysis of time-frequency transformed event related potentials, J. Neurosci. Methods, 161, 361-368 (2007) · doi:10.1016/j.jneumeth.2006.11.008 |
| [39] | Williams, A. H.; Kim, T. H.; Wang, F.; Vyas, S.; Ryu, S. I.; Shenoy, K. V.; Schnitzer, M.; Kolda, T. G.; Ganguli, S., Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis, Neuron, 98, 1099-1115 (2018) · doi:10.1016/j.neuron.2018.05.015 |
| [40] | Constantinople, C. M.; Piet, A. T.; Bibawi, P.; Akrami, A.; Kopec, C.; Brody, C. D., Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases, eLife, 8, e49744 (2019) · doi:10.7554/eLife.49744 |
| [41] | Xia, J.; Marks, T. D.; Goard, M. J.; Wessel, R., Diverse co-active neurons encode stimulus-driven and stimulus-independent variables, J. Neurophysiol., 124, 1505-1517 (2020) · doi:10.1152/jn.00431.2020 |
| [42] | Zhu, Y.; Liu, J.; Mathiak, K.; Ristaniemi, T.; Cong, F., Deriving electrophysiological brain network connectivity via tensor component analysis during freely listening to music, IEEE Trans. Neural Syst. Rehabil. Eng., 28, 409-418 (2020) · doi:10.1109/TNSRE.2019.2953971 |
| [43] | Zhu, Y.; Liu, J.; Ye, C.; Mathiak, K.; Astikainen, P.; Ristaniemi, T.; Cong, F., Discovering dynamic task-modulated functional networks with specific spectral modes using MEG, NeuroImage, 218, 116924 (2020) · doi:10.1016/j.neuroimage.2020.116924 |
| [44] | Xia, J., Marks, T. D., Goard, M. J., and Wessel, R., “Stable representation of a naturalistic movie emerges from episodic activity with gain variability,” Version 1, Research Square; see doi:10.21203/rs.3.rs-126977/v1 (2021). |
| [45] | Klus, S.; Gelß, P.; Peitz, S.; Schütte, C., Tensor-based dynamic mode decomposition, Nonlinearity, 31, 7, 3359 (2018) · Zbl 1404.65313 · doi:10.1088/1361-6544/aabc8f |
| [46] | Gelß, P.; Klus, S.; Eiset, J.; Schütte, C., Multidimensional approximation of nonlinear dynamical systems, J. Comput. Nonlinear Dyn., 14, 6, 061006 (2019) · doi:10.1115/1.4043148 |
| [47] | Lusch, B., Chi, E. C., and Kutz, J. N., “Shape constrained tensor decompositions,” in 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA) (IEEE, 2019), pp. 287-297. |
| [48] | Schmid, P. J., Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656, 5-28 (2010) · Zbl 1197.76091 · doi:10.1017/S0022112010001217 |
| [49] | Tu, J. H.; Rowley, C. W.; Luchtenburg, D. M.; Brunton, S. L.; Kutz, J. N., On dynamic mode decomposition: Theory and applications, J. Comput. Dyn., 1, 2, 391-421 (2014) · Zbl 1346.37064 · doi:10.3934/jcd.2014.1.391 |
| [50] | Chen, K. K.; Tu, J. H.; Rowley, C. W., Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22, 6, 887-915 (2012) · Zbl 1259.35009 · doi:10.1007/s00332-012-9130-9 |
| [51] | Williams, M. O.; Kevrekidis, I. G.; Rowley, C. W., A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25, 1307 (2015) · Zbl 1329.65310 · doi:10.1007/s00332-015-9258-5 |
| [52] | Arbabi, H.; Mezić, I., Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the Koopman operator, SIAM J. Appl. Dyn. Syst., 16, 4, 2096-2126 (2017) · Zbl 1381.37096 · doi:10.1137/17M1125236 |
| [53] | Takens, F., “Detecting strange attractors in turbulence,” in Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics Vol. 898, edited by D. Rand and L. S. Young (Springer, Berlin, 1981). · Zbl 0513.58032 |
| [54] | Kamb, M.; Kaiser, E.; Brunton, S. L.; Kutz, J. N., Time-delay observables for Koopman: Theory and applications, SIAM J. Appl. Dyn. Syst., 19, 2, 886-917 (2020) · Zbl 1441.37090 · doi:10.1137/18M1216572 |
| [55] | Hitchcock, F. L., The expression of a tensor or a polyadic as a sum of products, J. Math. Phys., 6, 164-189 (1927) · JFM 53.0095.01 · doi:10.1002/sapm192761164 |
| [56] | Hitchcock, F. L., Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys., 7, 39-79 (1928) · JFM 53.0097.01 · doi:10.1002/sapm19287139 |
| [57] | Muti, D.; Bourennane, S., Multidimensional filtering based on a tensor approach, Signal Process., 85, 2338-2353 (2005) · Zbl 1160.94349 · doi:10.1016/j.sigpro.2004.11.029 |
| [58] | De Lathauwer, L.; Castaing, J., Tensor-based techniques for the blind separation of DS-CDMA signal, Signal Process., 87, 322-336 (2007) · Zbl 1186.94413 · doi:10.1016/j.sigpro.2005.12.015 |
| [59] | De Lathauwer, L.; De Baynast, A., Blind deconvolution of DS-CDMA signals by means of decomposition in rank-(1, L, L terms, IEEE Trans. Signal Process., 56, 1562-1571 (2008) · Zbl 1390.94147 · doi:10.1109/TSP.2007.910469 |
| [60] | Tucker, L. R., Some mathematical notes on three-mode factor analysis, Psychometrika, 31, 279-311 (1966) · doi:10.1007/BF02289464 |
| [61] | Appellof, C. J.; Davidson, E. R., Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents, Anal. Chem., 53, 2053-2056 (1981) · doi:10.1021/ac00236a025 |
| [62] | Leurgans, S.; Ross, R. T., Multilinear models: Applications in spectroscopy, Stat. Sci., 7, 289-310 (1992) · Zbl 0955.62592 · doi:10.1214/ss/1177011225 |
| [63] | Henrion, R., Body diagonalization of core matrices in three-way principal components analysis: Theoretical bounds and simulation, J. Chemometrics, 7, 477-494 (1993) · doi:10.1002/cem.1180070604 |
| [64] | Smilde, A. K.; Wang, Y.; Kowalski, B. R., Theory of medium-rank second-order calibration with restricted-Tucker models, J. Chemometrics, 8, 21-36 (1994) · doi:10.1002/cem.1180080104 |
| [65] | Kiers, H. A. L., A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity, J. Chemometrics, 12, 155-171 (1998) · doi:10.1002/(SICI)1099-128X(199805/06)12:3<155::AID-CEM502>3.0.CO;2-5 |
| [66] | Wise, B. M.; Gallagher, N. B.; Martin, E. B., Application of PARAFAC2 to fault detection and diagnosis in semiconductor etch, J. Chemometrics, 15, 285-298 (2001) · doi:10.1002/cem.689 |
| [67] | Bro, R.; Andersson, C. A.; Kiers, H. A. L., PARAFAC2—Part II. Modeling chromotographic data with retention time shifts, J. Chemometrics, 13, 295-309 (1999) · doi:10.1002/(SICI)1099-128X(199905/08)13:3/4<295::AID-CEM547>3.0.CO;2-Y |
| [68] | Andersson, C. M.; Bro, R., Practical aspects of PARAFAC modeling fluorescence excitation-emission data, J. Chemometrics, 17, 200-215 (2003) · doi:10.1002/cem.790 |
| [69] | Smilde, A.; Bro, R.; Geladi, P., Multi-Way Analysis: Applications in the Chemical Sciences (2004), Wiley: Wiley, Chicester, England |
| [70] | Kruskal, J. B., “Rank, decomposition, and uniqueness for 3-way and N-way arrays,” in Multiway Data Analysis, edited by R. Coppi and S. Bolasco (North-Holland, Amsterdamn, 1989), pp. 7-18. |
| [71] | Ten Berge, J. M. F.; Kiers, H. A. L., Simplicity of core arrays in three-way principal component analysis and the typical rank of p \(\times\) q \(\times 2\) arrays, Linear Algebra Appl., 294, 169-179 (1999) · Zbl 0931.62050 · doi:10.1016/S0024-3795(99)00057-9 |
| [72] | Ten Berge, J. M. F., The typical rank of tall three-way arrays, Psychometrika, 65, 525-532 (2000) · Zbl 1291.62255 · doi:10.1007/BF02296342 |
| [73] | https://github.com/william-redman/Koopman-mode-decomposition-and-tensor-component-analysis. |
| [74] | Bagheri, S., Koopman-mode decomposition of the cylinder wake, J. Fluid Mech., 726, 596-623 (2013) · Zbl 1287.76116 · doi:10.1017/jfm.2013.249 |
| [75] | Takeishi, N., Kawahara, Y., and Yairi, T., “Learning Koopman invariant subspace for dynamic mode decomposition,” in Advances in Neural Information Processing Systems 30, NIPS 17 (Neural Information Processing Systems Foundation, 2017), pp. 1130-1140. |
| [76] | Li, Q.; Dietrich, F.; Bollt, E. M.; Kevrekidis, I. G., Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator, Chaos, 27, 103111 (2017) · Zbl 06876982 · doi:10.1063/1.4993854 |
| [77] | Lusch, B.; Kutz, J. N.; Brunton, S. L., Deep learning for universal linear embeddings of nonlinear dynamics, Nat. Commun., 9, 4950 (2018) · doi:10.1038/s41467-018-07210-0 |
| [78] | Otto, S. E.; Rowley, C. W., Linearly recurrent autoencoder networks for learning dynamics, SIAM J. Appl. Dyn. Syst., 18, 1, 558-593 (2019) · Zbl 1489.65164 · doi:10.1137/18M1177846 |
| [79] | Yeung, E., Kundu, S., and Hodas, N., “Learning deep neural network representations for Koopman operators of nonlinear dynamical systems,” in American Control Conference 2019 (IEEE, 2019), pp. 4832-4839. |
| [80] | Battaglino, C.; Ballard, G.; Kolda, T. G., A practical randomized CP tensor decomposition, SIAM J. Matrix Anal. Appl., 39, 2, 876-901 (2018) · Zbl 1444.65016 · doi:10.1137/17M1112303 |
| [81] | Erichson, N. B.; Manohar, K.; Brunton, S. L.; Kutz, J. N., Randomized CP tensor decomposition, Mach. Learn.: Sci. Technol., 1, 025012 (2020) · doi:10.1088/2632-2153/ab8240 |
| [82] | Paatero, P.; Tapper, U., Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 5, 2, 111-126 (1994) · doi:10.1002/env.3170050203 |
| [83] | Lee, D. D.; Seung, H. S., Learning the parts of objects by non-negative matrix factorization, Nature, 401, 788-791 (1999) · Zbl 1369.68285 · doi:10.1038/44565 |
| [84] | Qi, Y.; Comon, P.; Lim, L. H., Uniqueness of nonnegative tensor approximations, IEEE Trans. Inf. Theor., 62, 2170-2183 (2016) · Zbl 1359.94155 · doi:10.1109/TIT.2016.2532906 |
| [85] | Maćešić, S.; Črnjarić-Žic, N.; Mezić, I., Koopman operator family spectrum for nonautonomous systems, SIAM J. Appl. Dyn. Syst., 17, 4, 2478-2515 (2018) · Zbl 1403.37092 · doi:10.1137/17M1133610 |
| [86] | Dietrich, F.; Thiem, T. N.; Kevrekidis, I. G., On the Koopman operator of algorithms, SIAM J. Appl. Dyn. Syst., 19, 2, 860-885 (2020) · Zbl 1437.47048 · doi:10.1137/19M1277059 |
| [87] | Sahai, T., “Dynamical systems theory and algorithms for NP-hard problems,” in Advances in Dynamics, Optimization and Computation. SON 2020. Studies in Systems, Decision and Control, edited by O. Junge, O. Schütze, G. Froyland, S. Ober-Blöbaum, and K. Padberg-Gehle (Springer, Cham, 2020), Vol. 304. · Zbl 1528.37072 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
