Approximate joint diagonalization with Riemannian optimization on the general linear group. (English) Zbl 1432.49062
Summary: We consider the classical problem of approximate joint diagonalization of matrices, which can be cast as an optimization problem on the general linear group. We propose a versatile Riemannian optimization framework for solving this problem-unifiying existing methods and creating new ones. We use two standard Riemannian metrics (left- and right-invariant metrics) having opposite features regarding the structure of solutions and the model. We introduce the Riemannian optimization tools (gradient, retraction, vector transport) in this context, for the two standard nondegeneracy constraints (oblique and nonholonomic constraints). We also develop tools beyond the classical Riemannian optimization framework to handle the non-Riemannian quotient manifold induced by the nonholonomic constraint with the right-invariant metric. We illustrate our theoretical developments with numerical experiments on both simulated data and a real electroencephalographic recording.
MSC:
| 49Q20 | Variational problems in a geometric measure-theoretic setting |
| 15A23 | Factorization of matrices |
| 49M15 | Newton-type methods |
| 58B20 | Riemannian, Finsler and other geometric structures on infinite-dimensional manifolds |
| 65K05 | Numerical mathematical programming methods |
| 90C25 | Convex programming |
| 90C26 | Nonconvex programming, global optimization |
| 90C30 | Nonlinear programming |
Keywords:
approximate joint diagonalization; Riemannian optimization; oblique constraint; nonholonomic constraintSoftware:
ManoptReferences:
| [1] | P.-A. Absil and K. A. Gallivan, Joint diagonalization on the oblique manifold for independent component analysis, in Proceedings of the 2006 IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 5, 2006, pp. 945-948. |
| [2] | P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds, Princeton University Press, Princeton, NJ, 2008. · Zbl 1147.65043 |
| [3] | P.-A. Absil and J. Malick, Projection-like retractions on matrix manifolds, SIAM J. Optim., 22 (2012), pp. 135-158. · Zbl 1248.49055 |
| [4] | B. Afsari, Simple LU and QR based non-orthogonal matrix joint diagonalization, in International Conference on Independent Component Analysis and Signal Separation, Lecture Notes in Comput. Sci. 3889, Springer, New York, 2006, pp. 1-7. · Zbl 1147.65307 |
| [5] | B. Afsari, Sensitivity analysis for the problem of matrix joint diagonalization, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1148-1171. · Zbl 1184.15010 |
| [6] | B. Afsari and P. S. Krishnaprasad, Some Gradient Based Joint Diagonalization Methods for ICA, in Independent Component Analysis and Blind Signal Separation, Lecture Notes in Comput. Sci. 3195, Springer, New York, 2004, pp. 437-444. |
| [7] | K. Alyani, M. Congedo, and M. Moakher, Diagonality measures of Hermitian positive-definite matrices with application to the approximate joint diagonalization problem, Linear Algebra Appl., 528 (2017), pp. 290-320. · Zbl 1398.15043 |
| [8] | S.-I. Amari, Natural gradient works efficiently in learning, Neural Comput., 10 (1998), pp. 251-276. |
| [9] | S.-I. Amari, T. Chen, and A. Cichocki, Nonholonomic orthogonal learning algorithms for blind source separation, Neural Comput., 12 (2000), pp. 1463-1484. |
| [10] | E. Andruchow, G. Larotonda, L. Recht, and A. Varela, The left invariant metric in the general linear group, J. Geom. Phys., 86 (2014), pp. 241-257. · Zbl 1306.53039 |
| [11] | F. Bouchard, J. Malick, and M. Congedo, Riemannian optimization and approximate joint diagonalization for blind source separation, IEEE Trans. Signal Process., 66 (2018), pp. 2041-2054. · Zbl 1414.94085 |
| [12] | N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre, MANOPT, a MATLAB toolbox for optimization on manifolds, J. Mach. Learn. Res., 15 (2014), pp. 1455-1459. · Zbl 1319.90003 |
| [13] | J.-F. Cardoso and A. Souloumiac, Blind beamforming for non-Gaussian signals, IEEE Proc. F, 140 (1993), pp. 362-370. |
| [14] | J.-F. Cardoso and A. Souloumiac, Jacobi angles for simultaneous diagonalization, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 161-164. · Zbl 0844.65028 |
| [15] | G. Chabriel, M. Kleinsteuber, E. Moreau, H. Shen, P. Tichavsky, and A. Yeredor, Joint matrices decompositions and blind source separation: A survey of methods, identification, and applications, IEEE Signal Process. Mag., 31 (2014), pp. 34-43. |
| [16] | P. Comon and C. Jutten, Handbook of Blind Source Separation: Independent Component Analysis and Applications, Academic Press, New York, 2010. |
| [17] | M. Congedo, M. Goyat, N. Tarrin, G. Ionescu, L. Varnet, B. Rivet, R. Phlypo, N. Jrad, M. Acquadro, and C. Jutten, “Brain Invaders”: A prototype of an open-source p300-based video game working with the openvibe platform, in Proceedings of the 5th International Brain-Computer Interface Conference, 2011, pp. 280-283. |
| [18] | M. Congedo, L. Korczowski, A. Delorme, and F. Lopes da Silva, Spatio-temporal common pattern: A companion method for ERP analysis in the time domain, J. Neurosci. Methods, 267 (2016), pp. 74-88. |
| [19] | M. Congedo, R. Phlypo, and D. T. Pham, Approximate joint singular value decomposition of an asymmetric rectangular matrix set, IEEE Trans. Signal Process., 59 (2011), pp. 415-424. · Zbl 1391.65080 |
| [20] | M. Congedo, S. Rousseau, and C. Jutten, An introduction to EEG source analysis with an illustration of a study on error-related potentials, in Guide to Brain-Computer Music Interfacing, Springer, New York, 2014, pp. 163-189. |
| [21] | B. N. Flury and W. Gautschi, An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 169-184. · Zbl 0614.65043 |
| [22] | S. Gallot, D. Hulin, and J. Lafontaine, Riemannian Geometry, 3rd ed., Springer, New York, 2004. · Zbl 1068.53001 |
| [23] | W. Huang, P.-A. Absil, and K. Gallivan, A Riemannian BFGS Method for Nonconvex Optimization Problems, Springer, New York, 2016, pp. 627-634. · Zbl 1352.65153 |
| [24] | T. Kim, T. Eltoft, and T. Lee, Independent Vector Analysis: An Extension of ICA to Multivariate Components, in International Conference on Independent Component Analysis and Signal Separation, Lecture Notes in Comput. Sci. 3889, Springer, New York, 2006, pp. 165-172. · Zbl 1178.94080 |
| [25] | D. Lahat and C. Jutten, Joint independent subspace analysis using second-order statistics, IEEE Trans. Signal Process., 64 (2016), pp. 4891-4904. · Zbl 1414.94318 |
| [26] | J. Lee, Introduction to Smooth Manifolds, Grad. Texts in Math. 218, Springer, New York, 2003. |
| [27] | S. Lee, M. Choi, H. Kim, and F. C. Park, Geometric direct search algorithms for image registration, IEEE Trans. Image Process., 16 (2007), pp. 2215-2224. |
| [28] | M. I. Miller, A. Trouvé, and L. Younes, The metric spaces, Euler equations, and normal geodesic image motions of computational anatomy, in Proceedings of the 2003 International Conference on Image Processing, vol. 2, IEEE, 2003, pp. 635-638. |
| [29] | E. Moreau and O. Macchi, A one stage self-adaptive algorithm for source separation, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, 1994, pp. 49-52. |
| [30] | D.-T. Pham, Joint approximate diagonalization of positive definite Hermitian matrices, SIAM J. Matrix Anal. Appl., 22 (2000), pp. 1136-1152. · Zbl 1008.65020 |
| [31] | D.-T. Pham and J.-F. Cardoso, Blind separation of instantaneous mixtures of nonstationary sources, IEEE Trans. Signal Process., 49 (2001), pp. 1837-1848. · Zbl 1369.94262 |
| [32] | D.-T. Pham and M. Congedo, Least square joint diagonalization of matrices under an intrinsic scale constraint, in Independent Component Analysis and Signal Separation, Lecture Notes in Comput. Sci. 5441, Springer, New York, 2009, pp. 298-305. |
| [33] | A. Souloumiac, Nonorthogonal joint diagonalization by combining Givens and hyperbolic rotations, IEEE Trans. Signal Process., 57 (2009), pp. 2222-2231. · Zbl 1391.15050 |
| [34] | P. Tichavskỳ and A. Yeredor, Fast approximate joint diagonalization incorporating weight matrices, IEEE Trans. Signal Process., 57 (2009), pp. 878-891. · Zbl 1391.94416 |
| [35] | B. Vandereycken, P.-A. Absil, and S. Vandewalle, A Riemannian geometry with complete geodesics for the set of positive semidefinite matrices of fixed rank, IMA J. Numer. Anal., 33 (2012), pp. 481-514. · Zbl 1271.53039 |
| [36] | A. Yeredor, A. Ziehe, and K.-R. Müller, Approximate joint diagonalization using a natural gradient approach, in Independent Component Analysis and Blind Signal Separation, Lecture Notes in Comput. Sci. 3195, Springer, New York, 2004, pp. 89-96. |
| [37] | E. Zacur, M. Bossa, and S. Olmos, Left-invariant Riemannian geodesics on spatial transformation groups, SIAM J. Imaging Sci., 7 (2014), pp. 1503-1557. · Zbl 1308.58006 |
| [38] | A. Ziehe, P. Laskov, G. Nolte, and K.-R. Müller, A fast algorithm for joint diagonalization with non-orthogonal transformations and its application to blind source separation, J. Mach. Learn. Res., 5 (2004), pp. 777-800. · Zbl 1222.65043 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
