Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifolds. (English) Zbl 1466.37068
Summary: We present a novel kernel-based machine learning algorithm for identifying the low-dimensional geometry of the effective dynamics of high-dimensional multiscale stochastic systems. Recently, the authors developed a mathematical framework for the computation of optimal reaction coordinates of such systems that is based on learning a parameterization of a low-dimensional transition manifold in a certain function space. In this article, we enhance this approach by embedding and learning this transition manifold in a reproducing kernel Hilbert space, exploiting the favorable properties of kernel embeddings. Under mild assumptions on the kernel, the manifold structure is shown to be preserved under the embedding, and distortion bounds can be derived. This leads to a more robust and more efficient algorithm compared to the previous parameterization approaches.
MSC:
| 37M99 | Approximation methods and numerical treatment of dynamical systems |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T10 | Pattern recognition, speech recognition |
References:
| [1] | Abraham, I.; Bartal, Y.; Neiman, O., Advances in metric embedding theory, Adv. Math., 228, 6, 3026-3126 (2011) · Zbl 1250.46016 · doi:10.1016/j.aim.2011.08.003 |
| [2] | Baxter, JR; Rosenthal, JS, Rates of convergence for everywhere-positive Markov chains, Stat. Probab. Lett., 22, 4, 333-338 (1995) · Zbl 0819.60056 · doi:10.1016/0167-7152(94)00085-M |
| [3] | Berendsen, H.; van der Spoel, D.; van Drunen, R., Gromacs: a message-passing parallel molecular dynamics implementation, Comput. Phys. Commun., 91, 1, 43-56 (1995) · doi:10.1016/0010-4655(95)00042-E |
| [4] | Berry, T.; Harlim, J., Variable bandwidth diffusion kernels, Appl. Comput. Harmonic Anal., 40, 1, 68-96 (2016) · Zbl 1343.94020 · doi:10.1016/j.acha.2015.01.001 |
| [5] | Best, RB; Hummer, G., Reaction coordinates and rates from transition paths, Proc. Natl. Acad. Sci., 102, 19, 6732-6737 (2005) · doi:10.1073/pnas.0408098102 |
| [6] | Bittracher, A.; Koltai, P.; Klus, S.; Banisch, R.; Dellnitz, M.; Schütte, C., Transition manifolds of complex metastable systems: theory and data-driven computation of effective dynamics, J. Nonlinear Sci., 28, 2, 471-512 (2017) · Zbl 06860496 · doi:10.1007/s00332-017-9415-0 |
| [7] | Bittracher, A.; Banisch, R.; Schütte, C., Data-driven computation of molecular reaction coordinates, J. Chem. Phys., 149, 15, 154103 (2018) · doi:10.1063/1.5035183 |
| [8] | Bouvrie, J., Hamzi, B.: Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space. In: Proceedings of 48th Annual Allerton Conference on Communication, Control, and Computing, pp. 294-301 (2010) |
| [9] | Bouvrie, J.; Hamzi, B., Kernel methods for the approximation of some key quantities of nonlinear systems, J. Comput. Dyn., 4, 1, 1-19 (2017) · Zbl 1394.37115 · doi:10.3934/jcd.2017001 |
| [10] | Bouvrie, J.; Hamzi, B., Kernel methods for the approximation of nonlinear systems, SIAM J. Control Optim., 55, 4, 2460-2492 (2017) · Zbl 1368.93248 · doi:10.1137/14096815X |
| [11] | Bowman, G.; Volez, V.; Pande, VS, Taming the complexity of protein folding, Curr. Opin. Struct. Biol., 21, 1, 4-11 (2011) · doi:10.1016/j.sbi.2010.10.006 |
| [12] | Bowman, GR; Pande, VS; Noé, F., An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation (2014), Berlin: Springer, Berlin · Zbl 1290.92004 |
| [13] | Camacho, CJ; Thirumalai, D., Kinetics and thermodynamics of folding in model proteins, Proc. Natl. Acad. Sci., 90, 13, 6369-6372 (1993) · doi:10.1073/pnas.90.13.6369 |
| [14] | Chekmarev, DS; Ishida, T.; Levy, RM, Long-time conformational transitions of alanine dipeptide in aqueous solution: continuous and discrete-state kinetic models, J. Phys. Chem. B, 108, 50, 19487-19495 (2004) · doi:10.1021/jp048540w |
| [15] | Coifman, RR; Kevrekidis, IG; Lafon, S.; Maggioni, M.; Nadler, B., Diffusion maps, reduction coordinates, and low dimensional representation of stochastic systems, Multiscale Model. Simul., 7, 2, 842-864 (2008) · Zbl 1175.60058 · doi:10.1137/070696325 |
| [16] | Duvenaud, D., Lloyd, J., Grosse, R., Tenenbaum, J., Zoubin, G.: Structure discovery in nonparametric regression through compositional kernel search. In: Dasgupta, S., McAllester, D.: (eds), Proceedings of the 30th International Conference on Machine Learning, Volume 28 of Proceedings of Machine Learning Research, pp. 1166-1174, Atlanta, Georgia, USA, 17-19 Jun (2013). PMLR |
| [17] | E, W., Vanden-Eijnden, E.: Towards a theory of transition paths. J. Stat. Phys. 123(3), 503-523 (2006) · Zbl 1101.82016 |
| [18] | E, W., Ren, W., Vanden-Eijnden, E.: String method for the study of rare events. Phys. Rev. B 66, 052301 (2002) · Zbl 1050.60068 |
| [19] | E, W., Ren, W., Vanden-Eijnden, E.: Simplified and improved string method for computing the minimum energy paths in barrier-crossing events. J. Chem. Phys. 126(16), 164103 (2007) |
| [20] | Elber, R.; Bello-Rivas, JM; Ma, P.; Cardenas, AE; Fathizadeh, A., Calculating iso-committor surfaces as optimal reaction coordinates with milestoning, Entropy, 19, 5, 219 (2017) · doi:10.3390/e19050219 |
| [21] | Freddolino, PL; Harrison, CB; Liu, Y.; Schulten, K., Challenges in protein folding simulations: timescale, representation, and analysis, Nat. Phys., 6, 10, 751 (2010) · doi:10.1038/nphys1713 |
| [22] | Frewen, TA; Hummer, G.; Kevrekidis, IG, Exploration of effective potential landscapes using coarse reverse integration, J. Chem. Phys., 131, 13, 10B603 (2009) · doi:10.1063/1.3207882 |
| [23] | Froyland, G.; Gottwald, GA; Hammerlindl, A., A trajectory-free framework for analysing multiscale systems, Phys. D Nonlinear Phenom., 328, 34-43 (2016) · Zbl 1366.65065 · doi:10.1016/j.physd.2016.04.010 |
| [24] | Fukumizu, K., Gretton, A., Sun, X., Schölkopf, B.: Kernel measures of conditional dependence. In: Proceedings of the 20th International Conference on Neural Information Processing Systems, NIPS’07, pp. 489-496 (2007) |
| [25] | Gaspar, P.; Carbonell, J.; Oliveira, JL, On the parameter optimization of support vector machines for binary classification, J. Integr. Bioinform., 9, 3, 33-43 (2012) · doi:10.1515/jib-2012-201 |
| [26] | Gesùa, GD; Lelièvre, T.; Peutreca, DL; Nectouxa, B., Jump Markov models and transition state theory: the quasi-stationary distribution approach, Faraday Discuss., 195, 469-495 (2016) · doi:10.1039/C6FD00120C |
| [27] | Gönen, M.; Alpaydin, E., Multiple kernel learning algorithms, J. Mach. Learn. Res., 12, 64, 2211-2268 (2011) · Zbl 1280.68167 |
| [28] | Gretton, A.; Borgwardt, KM; Rasch, MJ; Schölkopf, B.; Smola, A., A kernel two-sample test, J. Mach. Learn. Res., 13, Mar, 723-773 (2012) · Zbl 1283.62095 |
| [29] | Hunt, B.; Kaloshin, V., Regularity of embeddings of infinite-dimensional fractal sets into finite-dimensional spaces, Nonlinearity, 12, 5, 1263-1275 (1999) · Zbl 0932.28006 · doi:10.1088/0951-7715/12/5/303 |
| [30] | Klein, R., Scale-dependent models for atmospheric flows, Annu. Rev. Fluid Mech., 42, 1, 249-274 (2010) · Zbl 1213.86002 · doi:10.1146/annurev-fluid-121108-145537 |
| [31] | Klus, S.; Bittracher, A.; Schuster, I.; Schütte, C., A kernel-based approach to molecular conformation analysis, J. Chem. Phys., 149, 24, 244109 (2018) · doi:10.1063/1.5063533 |
| [32] | Klus, S.; Nüske, F.; Koltai, P.; Wu, H.; Kevrekidis, I.; Schütte, C.; Noé, F., Data-driven model reduction and transfer operator approximation, J. Nonlinear Sci., 28, 985-1010 (2018) · Zbl 1396.37083 · doi:10.1007/s00332-017-9437-7 |
| [33] | Klus, S.; Schuster, I.; Muandet, K., Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces, J. Nonlinear Sci., 30, 1, 283-315 (2020) · Zbl 1437.37104 · doi:10.1007/s00332-019-09574-z |
| [34] | Kruskal, JB, Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 29, 1, 1-27 (1964) · Zbl 0123.36803 · doi:10.1007/BF02289565 |
| [35] | Lee, JA; Verleysen, M., Quality assessment of dimensionality reduction: rank-based criteria, Neurocomputing, 72, 7, 1431-1443 (2009) · doi:10.1016/j.neucom.2008.12.017 |
| [36] | Majda, AJ; Klein, R., Systematic multiscale models for the tropics, J. Atmos. Sci., 60, 2, 393-408 (2003) · doi:10.1175/1520-0469(2003)060<0393:SMMFTT>2.0.CO;2 |
| [37] | Mardt, A.; Pasquali, L.; Wu, H.; Noé, F., Vampnets for deep learning of molecular kinetics, Nat. Commun., 9, 1, 5 (2018) · doi:10.1038/s41467-017-02388-1 |
| [38] | Mattingly, JC; Stuart, AM, Geometric ergodicity of some hypo-elliptic diffusions for particle motions, Markov Process. Relat. Fields, 8, 2, 199-214 (2002) · Zbl 1014.60059 |
| [39] | McGibbon, RT; Husic, BE; Pande, VS, Identification of simple reaction coordinates from complex dynamics, J. Chem. Phys., 146, 4, 44109 (2017) · doi:10.1063/1.4974306 |
| [40] | Melzer, T., Reiter, M., Bischof, H.: Nonlinear feature extraction using generalized canonical correlation analysis. In: Dorffner, G., Bischof, H., Hornik, K. (eds), Artificial Neural Networks—ICANN 2001, pp. 353-360 (2001) · Zbl 1005.68935 |
| [41] | Mercer, J., Functions of positive and negative type, and their connection the theory of integral equations, Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 209, 441-458, 415-446 (1909) · JFM 40.0408.02 |
| [42] | Muandet, K.; Fukumizu, K.; Sriperumbudur, B.; Schölkopf, B., Kernel mean embedding of distributions: a review and beyond, Found. Trends Mach. Learn., 10, 1-2, 1-141 (2017) · Zbl 1380.68336 · doi:10.1561/2200000060 |
| [43] | Müller, K., Reaction paths on multidimensional energy hypersurfaces, Angewandte Chemie Int. Ed. Engl., 19, 1, 1-13 (1980) · doi:10.1002/anie.198000013 |
| [44] | Munkres, JR, Topology (2000), Upper Saddle River: Prentice Hall, Upper Saddle River · Zbl 0951.54001 |
| [45] | Nadler, B.; Lafon, S.; Coifman, RR; Kevrekidis, IG, Diffusion maps, spectral clustering and reaction coordinates of dynamical systems, Appl. Comput. Harmon. Anal., 21, 1, 113-127 (2006) · Zbl 1103.60069 · doi:10.1016/j.acha.2005.07.004 |
| [46] | Noé, F.; Schütte, C.; Vanden-Eijnden, E.; Reich, L.; Weikl, TR, Constructing the full ensemble of folding pathways from short off-equilibrium simulations, Proc. Natl. Acad. Sci., 106, 45, 19011-19016 (2009) · Zbl 1254.92001 · doi:10.1073/pnas.0905466106 |
| [47] | Owhadi, H.; Yoo, GR, Kernel flows: from learning kernels from data into the abyss, J. Comput. Phys., 389, 22-47 (2019) · Zbl 1452.65028 · doi:10.1016/j.jcp.2019.03.040 |
| [48] | Prinz, J-H; Wu, H.; Sarich, M.; Keller, B.; Senne, M.; Held, M.; Chodera, JD; Schütte, C.; Noé, F., Markov models of molecular kinetics: generation and validation, J. Chem. Phys., 134, 17, 174105 (2011) · doi:10.1063/1.3565032 |
| [49] | Roweis, ST; Saul, LK, Nonlinear dimensionality reduction by locally linear embedding, Science, 290, 5500, 2323-2326 (2000) · doi:10.1126/science.290.5500.2323 |
| [50] | Schervish, MJ; Carlin, BP, On the convergence of successive substitution sampling, J. Comput. Graph. Stat., 1, 2, 111-127 (1992) |
| [51] | Schölkopf, B.; Smola, AJ, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2001), Cambridge: MIT Press, Cambridge |
| [52] | Schölkopf, B.; Smola, A.; Müller, K-R, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., 10, 5, 1299-1319 (1998) · doi:10.1162/089976698300017467 |
| [53] | Schölkopf, B.; Muandet, K.; Fukumizu, K.; Harmeling, S.; Peters, J., Computing functions of random variables via reproducing kernel Hilbert space representations, Stat. Comput., 25, 4, 755-766 (2015) · Zbl 1331.62206 · doi:10.1007/s11222-015-9558-5 |
| [54] | Schütte, C.; Sarich, M., Metastability and Markov State Models in Molecular Dynamics: Modeling, Analysis, Algorithmic Approaches. Number 24 in Courant Lecture Notes (2013), Providence: American Mathematical Society, Providence · Zbl 1305.60004 · doi:10.1090/cln/024 |
| [55] | Schwantes, CR; Pande, VS, Modeling molecular kinetics with tICA and the kernel trick, J. Chem. Theory Comput., 11, 2, 600-608 (2015) · doi:10.1021/ct5007357 |
| [56] | Smith, PE, The alanine dipeptide free energy surface in solution, J. Chem. Phys., 111, 12, 5568-5579 (1999) · doi:10.1063/1.479860 |
| [57] | Smola, A., Gretton, A., Song, L., Schölkopf, B.: A Hilbert space embedding for distributions. In: Proceedings of the 18th International Conference on Algorithmic Learning Theory, pp. 13-31. Springer-Verlag (2007) · Zbl 1142.68407 |
| [58] | Socci, ND; Onuchic, JN; Wolynes, PG, Diffusive dynamics of the reaction coordinate for protein folding funnels, J. Chem. Phys., 104, 15, 5860-5868 (1996) · doi:10.1063/1.471317 |
| [59] | Sriperumbudur, BK; Gretton, A.; Fukumizu, K.; Schölkopf, B.; Lanckriet, GR, Hilbert space embeddings and metrics on probability measures, J. Mach. Learn. Res., 11, 1517-1561 (2010) · Zbl 1242.60005 |
| [60] | Steinwart, I.; Christmann, A., Support Vector Machines (2008), New York: Springer, New York · Zbl 1203.68171 |
| [61] | Vanden-Eijnden, E.; Venturoli, M., Revisiting the finite temperature string method for the calculation of reaction tubes and free energies, J. Chem. Phys., 130, 19, 194103 (2009) · doi:10.1063/1.3130083 |
| [62] | Young, FW, Multidimensional Scaling: History, Theory, and Applications (2013), New York: Psychology Press, New York |
| [63] | Zhang, W.; Hartmann, C.; Schütte, C., Effective dynamics along given reaction coordinates, and reaction rate theory, Faraday Discuss., 195, 365-394 (2016) · doi:10.1039/C6FD00147E |
| [64] | Zwanzig, R., Memory effects in irreversible thermodynamics, Phys. Rev., 124, 983-992 (1961) · Zbl 0131.45006 · doi:10.1103/PhysRev.124.983 |
| [65] | Zwanzig, R., Nonequilibrium Statistical Mechanics (2001), Oxford: Oxford University Press, Oxford · Zbl 1267.82001 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
