×

Benign landscapes of low-dimensional relaxations for orthogonal synchronization on general graphs. (English) Zbl 07836051

Summary: Orthogonal group synchronization is the problem of estimating \(n\) elements \(Z_1,\dots,Z_n\) from the \(r\times r\) orthogonal group given some relative measurements \(R_{ij}\approx Z_iZ_j^{-1}\). The least-squares formulation is nonconvex. To avoid its local minima, a Shor-type convex relaxation squares the dimension of the optimization problem from \(O(n)\) to \(O(n^2)\). Alternatively, Burer-Monteiro-type nonconvex relaxations have generic landscape guarantees at dimension \(O(n^{3/2})\). For smaller relaxations, the problem structure matters. It has been observed in the robotics literature that, for simultaneous localization and mapping problems, it seems sufficient to increase the dimension by a small constant multiple over the original. We partially explain this. This also has implications for Kuramoto oscillators. Specifically, we minimize the least-squares cost function in terms of estimators \(Y_1,\dots,Y_n\). For \(p\geq r\), each \(Y_i\) is relaxed to the Stiefel manifold \(\mathrm{St}(r,p)\) of \(r\times p\) matrices with orthonormal rows. The available measurements implicitly define a (connected) graph \(G\) on \(n\) vertices. In the noiseless case, we show that, for all connected graphs \(G\), second-order critical points are globally optimal as soon as \(p\geq r+2\). (This implies that Kuramoto oscillators on \(\mathrm{St}(r,p)\) synchronize for all \(p\geq r+2\)). This result is the best possible for general graphs; the previous best known result requires \(2p\geq 3(r+1)\). For \(p>r+2\), our result is robust to modest amounts of noise (depending on \(p\) and \(G)\). Our proof uses a novel randomized choice of tangent direction to prove (near-)optimality of second-order critical points. Finally, we partially extend our noiseless landscape results to the complex case (unitary group); we show that there are no spurious local minima when \(2p\geq 3r\).

MSC:

90C26 Nonconvex programming, global optimization
90C30 Nonlinear programming
90C35 Programming involving graphs or networks
90C46 Optimality conditions and duality in mathematical programming

References:

[1] Abdalla, P., Bandeira, A. S., and Invernizzi, C., Guarantees for spontaneous synchronization on random geometric graphs, SIAM J. Appl. Dyn. Syst., 23 (2024), pp. 779-790. · Zbl 1539.34037
[2] Abdalla, P., Bandeira, A. S., Kassabov, M., Souza, V., Strogatz, S. H., and Townsend, A., Expander Graphs are Globally Synchronising, https://arxiv.org/abs/2210.12788, 2022.
[3] Bandeira, A. S., Random Laplacian matrices and convex relaxations, Found. Comput. Math., 18 (2018), pp. 345-379, doi:10.1007/s10208-016-9341-9. · Zbl 1386.15065
[4] Bandeira, A. S., Boumal, N., and Voroninski, V., On the low-rank approach for semidefinite programs arising in synchronization and community detection, Proc. Mach. Learn. Res. (PMLR), 49 (2016), pp. 361-382, http://proceedings.mlr.press/v49/bandeira16.html.
[5] Bandeira, A. S., Singer, A., and Spielman, D. A., A Cheeger inequality for the graph connection Laplacian, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 1611-1630, doi:10.1137/120875338. · Zbl 1287.05081
[6] Bandeira, A. S. and van Handel, R., Sharp nonasymptotic bounds on the norm of random matrices with independent entries, Ann. Probab., 44 (2016), pp. 2479-2506, doi:10.1214/15-AOP1025. · Zbl 1372.60004
[7] Boumal, N., A Riemannian Low-Rank Method for Optimization Over Semidefinite Matrices with Block-Diagonal Constraints, preprint, arXiv:1506.00575, 2015.
[8] Boumal, N., An Introduction to Optimization on Smooth Manifolds, Cambridge University Press, Cambridge, 2023. · Zbl 1532.90001
[9] Boumal, N., Mishra, B., Absil, P.-A., and Sepulchre, R., Manopt, a Matlab toolbox for optimization on manifolds, J. Mach. Learn. Res., 15 (2014), pp. 1455-1459. · Zbl 1319.90003
[10] Boumal, N., Voroninski, V., and Bandeira, A. S., Deterministic guarantees for Burer-Monteiro factorizations of smooth semidefinite programs, Comm. Pure Appl. Math., 73 (2019), pp. 581-608, doi:10.1002/cpa.21830. · Zbl 1445.90074
[11] Burer, S. and Monteiro, R. D. C., Local minima and convergence in low-rank semidefinite programming, Math. Program., 103 (2005), pp. 427-444, doi:10.1007/s10107-004-0564-1. · Zbl 1099.90040
[12] Burer, S., Monteiro, R. D. C., and Zhang, Y., Rank-two relaxation heuristics for MAX-CUT and other binary quadratic programs, SIAM J. Optim., 12 (2002), pp. 503-521, doi:10.1137/s1052623400382467. · Zbl 1152.90532
[13] Carlone, L., Tron, R., Daniilidis, K., and Dellaert, F., Initialization techniques for 3d SLAM: A survey on rotation estimation and its use in pose graph optimization, in Proceedings of the IEEE Conference on Robotics and Automation (ICRA), , Seattle, Washington, IEEE, Piscataway, NJ, 2015, pp. 4597-4604, doi:10.1109/icra.2015.7139836.
[14] Dellaert, F., Rosen, D. M., Wu, J., Mahony, R., and Carlone, L., Shonan rotation averaging: Global optimality by surfing \(SO(p)^n\), in Proceedings of the European Conference on Computer Vision (ECCV), Virtual Conference, , Springer, Cham, Switzerland, 2020, pp. 292-308, doi:10.1007/978-3-030-58539-6_18.
[15] DeVille, L., Synchronization and stability for quantum Kuramoto, J. Stat. Phys., 174 (2019), pp. 160-187, doi:10.1007/s10955-018-2168-9. · Zbl 1412.82021
[16] Doherty, K. J., Rosen, D. M., and Leonard, J. J., Performance guarantees for spectral initialization in rotation averaging and pose-graph SLAM, in Proceedings of the IEEE Conference on Robotics and Automation (ICRA), , Philadelphia, IEEE, Piscataway, NJ, 2022, doi:10.1109/icra46639.2022.9811788.
[17] Filbir, F., Krahmer, F., and Melnyk, O., On recovery guarantees for angular synchronization, J. Fourier Anal. Appl., 27 (2021), 31, doi:10.1007/s00041-021-09834-1. · Zbl 1470.90065
[18] Gao, C. and Zhang, A. Y., Exact minimax estimation for phase synchronization, IEEE Trans. Inform. Theory, 67 (2021), pp. 8236-8247, doi:10.1109/tit.2021.3112712. · Zbl 1489.94034
[19] Gao, C. and Zhang, A. Y., SDP achieves exact minimax optimality in phase synchronization, IEEE Trans. Inform. Theory, 68 (2022), pp. 5374-5390, doi:10.1109/tit.2022.3167603. · Zbl 1505.65225
[20] Geshkovski, B., Letrouit, C., Polyanskiy, Y., and Rigollet, P., A Mathematical Perspective on Transformers, preprint, https://arxiv.org/abs/2312.10794, 2023.
[21] Ha, S.-Y., Ko, D., Park, J., and Zhang, X., Collective synchronization of classical and quantum oscillators, EMS Surv. Math. Sci., 3 (2016), pp. 209-267, doi:10.4171/emss/17. · Zbl 1403.34032
[22] Hartley, R., Trumpf, J., Dai, Y., and Li, H., Rotation averaging, Int. J. Comput. Vis., 103 (2013), pp. 267-305, doi:10.1007/s11263-012-0601-0. · Zbl 1270.68346
[23] Hatcher, A., Algebraic Topology, Cambridge University Press, Cambridge, 2001.
[24] Horn, R. A. and Johnson, C. R., Topics in Matrix Analysis, Cambridge University Press, Cambridge, 1991. · Zbl 0729.15001
[25] Iwen, M. A., Preskitt, B., Saab, R., and Viswanathan, A., Phase retrieval from local measurements: Improved robustness via eigenvector-based angular synchronization, Appl. Comput. Harmon. Anal., 48 (2020), pp. 415-444, doi:10.1016/j.acha.2018.06.004. · Zbl 07140142
[26] Journée, M., Bach, F., Absil, P.-A., and Sepulchre, R., Low-rank optimization on the cone of positive semidefinite matrices, SIAM J. Optim., 20 (2010), pp. 2327-2351, doi:10.1137/080731359. · Zbl 1215.65108
[27] Kassabov, M., Strogatz, S. H., and Townsend, A., Sufficiently dense Kuramoto networks are globally synchronizing, Chaos, 31 (2021), 073135, doi:10.1063/5.0057659. · Zbl 1468.34046
[28] Kassabov, M., Strogatz, S. H., and Townsend, A., A global synchronization theorem for oscillators on a random graph, Chaos, 32(2022), 093119. · Zbl 07878949
[29] Lageman, C., Convergence of Gradient-like Dynamical Systems and Optimization Algorithms, Ph.D. thesis, Universität Würzburg, Würzburg, Germany, 2007.
[30] Ling, S., Solving orthogonal group synchronization via convex and low-rank optimization: Tightness and landscape analysis, Math. Program., 200 (2023), pp. 589-628, doi:10.1007/s10107-022-01896-3. · Zbl 1519.90156
[31] Ling, S., Xu, R., and Bandeira, A. S., On the landscape of synchronization networks: A perspective from nonconvex optimization, SIAM J. Optim., 29 (2019), pp. 1879-1907, doi:10.1137/18m1217644. · Zbl 1427.90234
[32] Lohe, M. A., Non-Abelian Kuramoto models and synchronization, J. Phys. A, 42 (2009), 395101, doi:10.1088/1751-8113/42/39/395101. · Zbl 1187.37048
[33] Lu, J. and Steinerberger, S., Synchronization of Kuramoto oscillators in dense networks, Nonlinearity, 33 (2020), pp. 5905-5918, doi:10.1088/1361-6544/ab9baa. · Zbl 1480.34069
[34] Markdahl, J., A geometric obstruction to almost global synchronization on riemannian manifolds, https://arxiv.org/abs/1808.00862, 2018.
[35] Markdahl, J., Synchronization on Riemannian manifolds: Multiply connected implies multistable, IEEE Trans. Automat. Control, 66 (2021), pp. 4311-4318, doi:10.1109/tac.2020.3030849. · Zbl 1471.93020
[36] Markdahl, J., Proverbio, D., Mi, L., and Goncalves, J., Almost global convergence to practical synchronization in the generalized Kuramoto model on networks over the \(n\)-sphere, Commun. Phys., 4 (2021), 187, doi:10.1038/s42005-021-00689-y.
[37] Markdahl, J., Thunberg, J., and Goncalves, J., Almost global consensus on the \(n\)-sphere, IEEE Trans. Automat. Control, 63 (2018), pp. 1664-1675, doi:10.1109/tac.2017.2752799. · Zbl 1395.93446
[38] Markdahl, J., Thunberg, J., and Goncalves, J., Towards almost global synchronization on the Stiefel manifold, in Proceedings of the IEEE Conference on Decision and Control (CDC), , Miami, Florida, IEEE, Piscataway, NJ, 2018, doi:10.1109/cdc.2018.8619593.
[39] Markdahl, J., Thunberg, J., and Goncalves, J., High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globally, Automatica J. IFAC, 113 (2020), 108736, doi:10.1016/j.automatica.2019.108736. · Zbl 1440.93237
[40] Martinec, D. and Pajdla, T., Robust rotation and translation estimation in multiview reconstruction, in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), , Minneapolis, Minnesota, IEEE, Piscataway, NJ, 2007, doi:10.1109/cvpr.2007.383115.
[41] Mei, S., Misiakiewicz, T., Montanari, A., and Oliveira, R. I., Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality, Proc. Mach. Learn. Res. (PMLR), 65 (2017), pp. 1476-1515, http://proceedings.mlr.press/v65/mei17a.html.
[42] O’Carroll, L., Srinivas, V., and Vijayaraghavan, A., The Burer-Monteiro SDP method can fail even above the Barvinok-Pataki bound, in Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), , Vol. 35, New Orleans, Louisiana, 2022, pp. 31254-31264, https://proceedings.neurips.cc/paper_files/paper/2022/hash/ca92ff06d973ece92cecc561757d500e-Abstract-Conference.html.
[43] Pikovsky, A. and Rosenblum, M., Introduction to focus issue: Dynamics of oscillator populations, Chaos, 33 (2023), 010401, doi:10.1063/5.0139277. · Zbl 1541.00058
[44] Preskitt, B. P., Phase Retrieval from Locally Supported Measurements, Ph.D. thesis, University of California San Diego, San Diego, CA, 2018, https://escholarship.org/uc/item/97v5k8j9.
[45] Pumir, T., Jelassi, S., and Boumal, N., Smoothed analysis of the low-rank approach for smooth semidefinite programs, in Proceedings of the Conference of Neural Information Processing Systems (NeurIPS), , Vol. 35, Montréal, Canada, 2018, pp. 2281-2290, https://proceedings.neurips.cc/paper_files/paper/2018/hash/a1d50185e7426cbb0acad1e6ca74b9aa-Abstract.html.
[46] Rastegin, A. E., Relations for certain symmetric norms and anti-norms before and after partial trace, J. Stat. Phys., 148 (2012), pp. 1040-1053, doi:10.1007/s10955-012-0569-8. · Zbl 1260.81011
[47] Rosen, D. M., Carlone, L., Bandeira, A. S., and Leonard, J. J., SE-sync: A certifiably correct algorithm for synchronization over the special Euclidean group, Int. J. Robot. Res., 38 (2019), pp. 95-125, doi:10.1177/0278364918784361.
[48] Rosen, D. M., Doherty, K. J., Espinoza, A. T., and Leonard, J. J., Advances in inference and representation for simultaneous localization and mapping, Annu. Rev. Control Robot. Auton. Syst., 4 (2021), pp. 215-242, doi:10.1146/annurev-control-072720-082553.
[49] Singer, A., Angular synchronization by eigenvectors and semidefinite programming, Appl. Comput. Harmon. Anal., 30 (2011), pp. 20-36, doi:10.1016/j.acha.2010.02.001. · Zbl 1206.90116
[50] Singer, A. and tieng Wu, H., Vector diffusion maps and the connection Laplacian, Comm. Pure Appl. Math., 65 (2012), pp. 1067-1144, doi:10.1002/cpa.21395. · Zbl 1320.68146
[51] Taylor, R., There is no non-zero stable fixed point for dense networks in the homogeneous Kuramoto model, J. Phys. A, 45 (2012), 5, doi:10.1088/1751-8113/45/5/055102. · Zbl 1247.34092
[52] Townsend, A., Stillman, M., and Strogatz, S. H., Dense networks that do not synchronize and sparse ones that do, Chaos, 30 (2020), 083142, doi:10.1063/5.0018322. · Zbl 1445.34061
[53] Waldspurger, I. and Waters, A., Rank optimality for the Burer-Monteiro factorization, SIAM J. Optim., 30 (2020), pp. 2577-2602, doi:10.1137/19m1255318. · Zbl 1451.90114
[54] Wang, L. and Singer, A., Exact and stable recovery of rotations for robust synchronization, Inf. Inference, 2 (2013), pp. 145-193, doi:10.1093/imaiai/iat005. · Zbl 1309.65070
[55] Wang, L., Singer, A., and Wen, Z., Orientation determination of cryo-EM images using least unsquared deviations, SIAM J. Imaging Sci., 6 (2013), pp. 2450-2483, doi:10.1137/130916436. · Zbl 1402.92449
[56] Yoneda, R., Tatsukawa, T., and nosuke Teramae, J., The lower bound of the network connectivity guaranteeing in-phase synchronization, Chaos, 31 (2021), 063124, doi:10.1063/5.0054271. · Zbl 1465.34052
[57] Zhang, A. Y., Exact Minimax Optimality of Spectral Methods in Phase Synchronization and Orthogonal Group Synchronization, preprint, https://arxiv.org/abs/2209.04962, 2022.
[58] Zhang, R. Y., Improved Global Guarantees for the Nonconvex Burer-Monteiro Factorization via Rank Overparameterization, preprint, https://arxiv.org/abs/2207.01789, 2022.
[59] Łojasiewicz, S., Sur les trajectoires du gradient d’une fonction analytique, in Seminari di Geometria, Università di Bologna, Dipartimento di matematica, Bologna, 1983, pp. 115-117. · Zbl 0606.58045
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.