×

Empirical evolution equations. (English) Zbl 1388.62102

Summary: Evolution equations comprise a broad framework for describing the dynamics of a system in a general state space: when the state space is finite-dimensional, they give rise to systems of ordinary differential equations; for infinite-dimensional state spaces, they give rise to partial differential equations. Several modern statistical and machine learning methods concern the estimation of objects that can be formalized as solutions to evolution equations, in some appropriate state space, even if not stated as such. The corresponding equations, however, are seldom known exactly, and are empirically derived from data, often by means of non-parametric estimation. This induces uncertainties on the equations and their solutions that are challenging to quantify, and moreover the diversity and the specifics of each particular setting may obscure the path for a general approach. In this paper, we address the problem of constructing general yet tractable methods for quantifying such uncertainties, by means of asymptotic theory combined with bootstrap methodology. We demonstrates these procedures in important examples including gradient line estimation, diffusion tensor imaging tractography, and local principal component analysis. The bootstrap perspective is particularly appealing as it circumvents the need to simulate from stochastic (partial) differential equations that depend on (infinite-dimensional) unknowns. We assess the performance of the bootstrap procedure via simulations and find that it demonstrates good finite-sample coverage.

MSC:

62G07 Density estimation
62G09 Nonparametric statistical resampling methods
37N20 Dynamical systems in other branches of physics (quantum mechanics, general relativity, laser physics)

References:

[1] Arias-Castro, E., Mason, D. and Pelletier, B. (2016). On the Estimation of the Gradient Lines of a Density and the Consistency of the Mean-Shift Algorithm., Journal of Machine Learning Research 17 1-28. · Zbl 1360.62150
[2] Atkinson, K. and Han, W. (2009). Linear Operators on Normed Spaces. In, Theoreticl Numerical Analysis: A Functional Analysis Framework 51-113. · Zbl 1181.47078
[3] Basser, P. J., Mattiello, J. and LeBihan, D. (1994). MR diffusion tensor spectroscopy and imaging., Biophysical journal 66 259-67.
[4] Bellman, R. (2008)., Stability Theory of Differential Equations. Dover Publications, New York.
[5] Bierens, H. J. (1987). Kernel estimators of regression functions. In, Advances in econometrics: Fifth world congress 99-144. · Zbl 0850.62348
[6] Blanes, S., Casas, F., Oteo, J. a. and Ros, J. (2009). The Magnus expansion and some of its applications., Physics Reports 470 151-238.
[7] Bottou, L. (2010)., Large-Scale Machine Learning with Stochastic Gradient Descent In Proceedings of COMPSTAT’2010: 19th International Conference on Computational Statistics Paris France, August 22-27, 2010 Keynote, Invited and Contributed Papers 177-186. Physica-Verlag HD, Heidelberg. · Zbl 1436.68293
[8] Brunel, N. J.-B., Clairon, Q. and D’Alché-Buc, F. (2014). Parametric Estimation of Ordinary Differential Equations With Orthogonality Conditions., Journal of the American Statistical Association 109 173-185. · Zbl 1367.62081 · doi:10.1080/01621459.2013.841583
[9] Chandra, J. (1970). On a Generalization of the Gronwall-Bellman in Partially Ordered Banach Spaces., Journal of Mathematical Analysis and Applications 681 668-681. · Zbl 0179.20302 · doi:10.1016/0022-247X(70)90016-8
[10] Chaudhuri, P. and Marron, J. S. (2000). Scale Space View of Curve Estimation., The Annals of Statistics 28 408-428. · Zbl 1106.62318 · doi:10.1214/aos/1016218224
[11] Chen, Y.-C. C., Genovese, C. R. and Wasserman, L. (2015). Asymptotic theory for density ridges., Annals of Statistics 43 1896-1928. · Zbl 1327.62303 · doi:10.1214/15-AOS1329
[12] Chen, Y.-C., Genovese, C. R. and Wasserman, L. (2016). A Comprehensive Approach to Mode Clustering., Electronic Journal of Statistics 10 210-241. · Zbl 1332.62200 · doi:10.1214/15-EJS1102
[13] Chen, Y. C., Genovese, C. R. and Wasserman, L. (2017a). Density Level Sets: Asymptotics, Inference, and Visualization., Journal of the American Statistical Association 1-13.
[14] Chen, Y. C., Genovese, C. R. and Wasserman, L. (2017b). Statistical inference using the morse-smale complex., Electronic Journal of Statistics 11 1390-1433. · Zbl 1362.62078 · doi:10.1214/17-EJS1271
[15] Chen, Y.-C., Genovese, C. R., Ho, S. and Wasserman, L. (2015). Optimal Ridge Detection using Coverage Risk. In, Advances in Neural Information Processing Systems 28 316-324.
[16] Ciollaro, M., Genovese, C. R. and Wang, D. (2016). Nonparametric Clustering of Functional Data Using, Pseudo-Densities. · Zbl 1357.62162 · doi:10.1214/16-EJS1198
[17] Delaigle, A. and Hall, P. (2010). Defining probability density for a distribution of random functions., Annals of Statistics 38 1171-1193. · Zbl 1183.62061 · doi:10.1214/09-AOS741
[18] Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife., The Annals of Statistics 7 1-26. · Zbl 0406.62024 · doi:10.1214/aos/1176344552
[19] Efron, B. (1982)., The jackknife, the bootstrap, and other resampling plans. Society for Industrial and Applied Mathematics. · Zbl 0496.62036
[20] Einbeck, J. and Dwyer, J. (2011). Using principal curves to analyse traffic patterns on freeways., Transportmetrica 7 229-246.
[21] Einbeck, J., Evers, L. and Bailer-Jones, C. (2008). Representing complex data using localized principal components with application to astronomical data. In, Lecture Notes in Computational Science and Engineering 58 178-201.
[22] Einbeck, J., Tutz, G. and Evers, L. (2005). Local principal curves., Statistics and Computing 15 301-313. · doi:10.1007/s11222-005-4073-8
[23] Genovese, C. R., Perone-Pacifico, M., Verdinelli, I. and Wasserman, L. (2009). On the path density of a gradient field., Annals of Statistics 37 3236-3271. · Zbl 1191.62062 · doi:10.1214/08-AOS671
[24] Hageman, N. S., Toga, A. W., Narr, K. L. and Shattuck, D. W. (2009). A diffusion tensor imaging tractography algorithm based on Navier-Stokes fluid mechanics., IEEE Transactions on Medical Imaging 28 348-60.
[25] Hall, P. and Heckman, N. E. (2002). Estimating and depicting the structure of a distribution of random functions., Biometrika 89 145-158. · Zbl 0996.62030 · doi:10.1093/biomet/89.1.145
[26] Hastie, T. and Stuetzle, W. (1989). Principal Curves., Journal of the American Statistical Association 84 502-516. · Zbl 0679.62048 · doi:10.1080/01621459.1989.10478797
[27] Hill, B. J., Kendall, W. S. and Thönnes, E. (2012). Fibre-generated point processes and fields of orientations., Annals of Applied Statistics 6 994-1020. · Zbl 1254.62101 · doi:10.1214/12-AOAS553
[28] Huckemann, S., Hotz, T. and Munk, A. (2008). Global models for the orientation field of fingerprints: An approach based on quadratic differentials., IEEE Transactions on Pattern Analysis and Machine Intelligence 30 1507-1519.
[29] Koltchinskii, V., Sakhanenko, L. and Cai, S. (2007). Integral curves of noisy vector fields and statistical problems in diffusion tensor imaging: nonparametric kernel estimation and hypotheses testing., Annals of Statistics 35 1576-1607. · Zbl 1195.62040 · doi:10.1214/009053607000000073
[30] Kosorok, M. R. (2008). Introduction to Empirical Processes and Semiparametric Inference., Springer August 1-491. · Zbl 1180.62137
[31] Krein, S. G. (1971)., Linear differential equations in Banach space. American Mathematical Society, Providence, R.I.
[32] Lee, K. K. (1992)., Lectures on Dynamical Systems, Structural Stability and Their Applications. World Scientific, Singapore. · Zbl 0798.58025
[33] Lee, J. M. (2012)., Introduction to Smooth Manifolds, 2 ed. Springer New York, New York.
[34] Lisini, S. (2009). Nonlinear diffusion equations with variable coefficients as gradient flows in Wasserstein spaces., ESAIM: Control, Optimisation and Calculus of Variations 15 712-740. · Zbl 1178.35201 · doi:10.1051/cocv:2008044
[35] Mikusiński, J. (1978)., The Bochner integral. Birkhauser Verlag. · Zbl 0369.28010
[36] Moler, C. andVan Loan, C. (2003). Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later., SIAM Review 45 3-49. · Zbl 1030.65029 · doi:10.1137/S00361445024180
[37] Mukherjee, P., Berman, J. I., Chung, S. W., Hess, C. P. and Henry, R. G. (2008). Diffusion tensor MR imaging and fiber tractography: theoretic underpinnings., American Journal of Neuroradiology 29 632-41.
[38] Nelson, E. (1969)., Topics in Dynamics I: Flows. Princeton University Press, Princeton. · Zbl 0197.10702
[39] Ozertem, U. and Erdogmus, D. (2011). Locally Defined Principal Curves and Surfaces., Journal of Machine Learning Research 12 1249-1286. · Zbl 1280.62071
[40] Panaretos, V. M., Pham, T. and Yao, Z. (2014). Principal flows., Journal of the American Statistical Association 109 424-436. · Zbl 1367.62187 · doi:10.1080/01621459.2013.849199
[41] Praestgaard, J. and Wellner, J. A. (1993). Exchangeably Weighted Bootstraps of the General Empirical Process., The Annals of Probability 21 2053-2086. · Zbl 0792.62038 · doi:10.1214/aop/1176989011
[42] Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007). Parameter Estimation for Differential Equations: A Generalized Smoothing Approach., Differential Equations 69 741-796. · Zbl 07555374
[43] Rinaldo, A. and Nugent, R. (2012). Stability of Density-Based Clustering., Journal of Machine Learning Reasearch 13 905-948. · Zbl 1283.62130
[44] Rinaldo, A. and Wasserman, L. (2010). Generalized density clustering., Annals of Statistics 38 2678-2722. · Zbl 1200.62066 · doi:10.1214/10-AOS797
[45] Theisel, H., Weinkauf, T., Hege, H. C. and Seidel, H. P. (2004). Stream line and path line oriented topology for 2D time-dependent vector fields., IEEE Visualization 2004 - Proceedings, VIS 2004 321-328.
[46] van der Vaart, A. W. and Wellner, J. A. (1996)., Weak Convergence and Empirical Processes. Springer series in statistics. Springer. · Zbl 0862.60002
[47] Vanhems, A. (2006). Nonparametric study of solutions of differential equations., Econometric Theory 22 127-157. · Zbl 1084.62031 · doi:10.1017/S0266466606060051
[48] Walker, J. A. (1980)., Dynamical Systems and Evolution Equations. Springer, Boston. · Zbl 0421.34050
[49] Zachmanoglou, E. · Zbl 0327.35001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.