Kernel ordinary differential equations. (English) Zbl 1515.62075
Summary: Ordinary differential equation (ODE) is widely used in modeling biological and physical processes in science. In this article, we propose a new reproducing kernel-based approach for estimation and inference of ODE given noisy observations. We do not assume the functional forms in ODE to be known, or restrict them to be linear or additive, and we allow pairwise interactions. We perform sparse estimation to select individual functionals, and construct confidence intervals for the estimated signal trajectories. We establish the estimation optimality and selection consistency of kernel ODE under both the low-dimensional and high-dimensional settings, where the number of unknown functionals can be smaller or larger than the sample size. Our proposal builds upon the smoothing spline analysis of variance (SS-ANOVA) framework, but tackles several important problems that are not yet fully addressed, and thus extends the scope of existing SS-ANOVA as well. We demonstrate the efficacy of our method through numerous ODE examples.
MSC:
| 62J10 | Analysis of variance and covariance (ANOVA) |
| 62R10 | Functional data analysis |
| 62G05 | Nonparametric estimation |
Keywords:
component selection and smoothing operator; high dimensionality; ordinary differential equations; smoothing spline analysis of variance; reproducing kernel Hilbert spaceReferences:
| [1] | Bachoc, F.; Leeb, H.; Pötscher, B. M., “Valid Confidence Intervals for Post-Model-Selection Predictors, The Annals of Statistics, 47, 1475-1504 (2019) · Zbl 1419.62164 · doi:10.1214/18-AOS1721 |
| [2] | Berk, R.; Brown, L.; Buja, A.; Zhang, K.; Zhao, L., “Valid Post-Selection Inference, The Annals of Statistics, 41, 802-837 (2013) · Zbl 1267.62080 · doi:10.1214/12-AOS1077 |
| [3] | Brunton, S. L.; Proctor, J. L.; Kutz, J. N., “Discovering Governing Equations From Data by Sparse Identification of Nonlinear Dynamical Systems, Proceedings of the National Academy of Sciences, 113, 3932-3937 (2016) · Zbl 1355.94013 · doi:10.1073/pnas.1517384113 |
| [4] | Buxton, R. B.; Uludağ, K.; Dubowitz, D. J.; Liu, T. T., “Modeling the Hemodynamic Response to Brain Activation, Neuroimage, 23, S220-S233 (2004) · doi:10.1016/j.neuroimage.2004.07.013 |
| [5] | Cao, J.; Zhao, H., “Estimating Dynamic Models for Gene Regulation Networks, Bioinformatics, 24, 1619-1624 (2008) · doi:10.1093/bioinformatics/btn246 |
| [6] | Cao, X.; Sandstede, B.; Luo, X., “A Functional Data Method for Causal Dynamic Network Modeling of Task-Related fMRI, Frontiers in Neuroscience, 13, 127 (2019) · doi:10.3389/fnins.2019.00127 |
| [7] | Chen, S.; Shojaie, A.; Witten, D. M., “Network Reconstruction From High-Dimensional Ordinary Differential Equations, Journal of the American Statistical Association, 112, 1697-1707 (2017) · doi:10.1080/01621459.2016.1229197 |
| [8] | Chernozhukov, V.; Hansen, C.; Spindler, M., “Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach, Annual Review of Economics, 7, 649-688 (2015) · doi:10.1146/annurev-economics-012315-015826 |
| [9] | Chou, I.-C.; Voit, E. O., “Recent Developments in Parameter Estimation and Structure Identification of Biochemical and Genomic Systems, Mathematical Biosciences, 219, 57-83 (2009) · Zbl 1168.92019 · doi:10.1016/j.mbs.2009.03.002 |
| [10] | Cox, D. D., “Asymptotics for M-Type Smoothing Splines, The Annals of Statistics, 11, 530-551 (1983) · Zbl 0519.62034 · doi:10.1214/aos/1176346159 |
| [11] | Dattner, I.; Klaassen, C. A. J., “Optimal Rate of Direct Estimators in Systems of Ordinary Differential Equations Linear in Functions of the Parameters, Electronic Journal of Statistics, 9, 1939-1973 (2015) · Zbl 1327.62120 · doi:10.1214/15-EJS1053 |
| [12] | Friston, K. J.; Harrison, L.; Penny, W., “Dynamic Causal Modelling, Neuroimage, 19, 1273-1302 (2003) · doi:10.1016/S1053-8119(03)00202-7 |
| [13] | Friston, K. J.; Preller, K. H.; Mathys, C.; Cagnan, H.; Heinzle, J.; Razi, A.; Zeidman, P., “Dynamic Causal Modelling Revisited, Neuroimage, 199, 730-744 (2019) · doi:10.1016/j.neuroimage.2017.02.045 |
| [14] | González, J.; Vujačić, I.; Wit, E., “Reproducing Kernel Hilbert Space Based Estimation of Systems of Ordinary Differential Equations, Pattern Recognition Letters, 45, 26-32 (2014) · doi:10.1016/j.patrec.2014.02.019 |
| [15] | Gu, C., Smoothing Spline ANOVA Models (2013), New York: Springer-Verlag, New York · Zbl 1269.62040 |
| [16] | Henderson, J.; Michailidis, G., “Network Reconstruction Using Nonparametric Additive ODE Models, PLOS ONE, 9, 1-15 (2014) · doi:10.1371/journal.pone.0094003 |
| [17] | Huang, J. Z., “Projection Estimation in Multiple Regression With Application to Functional ANOVA Models, The Annals of Statistics, 26, 242-272 (1998) · Zbl 0930.62042 · doi:10.1214/aos/1030563984 |
| [18] | Izhikevich, E., Dynamical Systems in Neuroscience (2007), Cambridge, MA: MIT Press, Cambridge, MA |
| [19] | Javanmard, A.; Montanari, A., “Confidence Intervals and Hypothesis Testing for High-Dimensional Regression,”, Journal of Machine Learning Research, 15, 2869-2909 (2014) · Zbl 1319.62145 |
| [20] | Koltchinskii, V.; Yuan, M., “Sparsity in Multiple Kernel Learning, The Annals of Statistics, 38, 3660-3695 (2010) · Zbl 1204.62086 · doi:10.1214/10-AOS825 |
| [21] | Liang, H.; Wu, H., “Parameter Estimation for Differential Equation Models Using a Framework of Measurement Error in Regression Models, Journal of the American Statistical Association, 103, 1570-1583 (2008) · Zbl 1286.62039 · doi:10.1198/016214508000000797 |
| [22] | Lin, Y., “Tensor Product Space ANOVA Models, The Annals of Statistics, 28, 734-755 (2000) · Zbl 1105.62329 · doi:10.1214/aos/1015951996 |
| [23] | Lin, Y.; Zhang, H. H., “Component Selection and Smoothing in Multivariate Nonparametric Regression, The Annals of Statistics, 34, 2272-2297 (2006) · Zbl 1106.62041 · doi:10.1214/009053606000000722 |
| [24] | Loh, P.-L.; Wainwright, M. J., “High-Dimensional Regression With Noisy and Missing Data: Provable Guarantees With Nonconvexity, The Annals of Statistics, 40, 1637-1664 (2012) · Zbl 1257.62063 · doi:10.1214/12-AOS1018 |
| [25] | Lu, J.; Kolar, M.; Liu, H., “Kernel Meets Sieve: Post-Regularization Confidence Bands for Sparse Additive Model, Journal of the American Statistical Association, 115, 2084-2099 (2020) · Zbl 1453.62476 · doi:10.1080/01621459.2019.1689984 |
| [26] | Lu, T.; Liang, H.; Li, H.; Wu, H., “High-Dimensional ODEs Coupled With Mixed-Effects Modeling Techniques for Dynamic Gene Regulatory Network Identification, Journal of the American Statistical Association, 106, 1242-1258 (2011) · Zbl 1234.62146 · doi:10.1198/jasa.2011.ap10194 |
| [27] | Ma, W.; Trusina, A.; El-Samad, H.; Lim, W. A.; Tang, C., “Defining Network Topologies That Can Achieve Biochemical Adaptation, Cell, 138, 760-773 (2009) · doi:10.1016/j.cell.2009.06.013 |
| [28] | Marbach, D.; Prill, R. J.; Schaffter, T.; Mattiussi, C.; Floreano, D.; Stolovitzky, G., “Revealing Strengths and Weaknesses of Methods for Gene Network Inference, Proceedings of the National Academy of Sciences of the United States of America, 107, 6286-6291 (2010) · doi:10.1073/pnas.0913357107 |
| [29] | Marbach, D.; Schaffter, T.; Mattiussi, C.; Floreano, D., “Generating Realistic in Silico Gene Networks for Performance Assessment of Reverse Engineering Methods, Journal of Computational Biology, 16, 229-239 (2009) · doi:10.1089/cmb.2008.09TT |
| [30] | Mikkelsen, F. V., and Hansen, N. R. (2017), “Learning Large Scale Ordinary Differential Equation Systems,” arXiv no. 1710.09308. |
| [31] | Opsomer, J. D.; Ruppert, D., “Fitting a Bivariate Additive Model by Local Polynomial Regression, The Annals of Statistics, 25, 186-211 (1997) · Zbl 0869.62026 · doi:10.1214/aos/1034276626 |
| [32] | Pfister, N.; Bauer, S.; Peters, J., “Learning Stable and Predictive Structures in Kinetic Systems, Proceedings of the National Academy of Sciences of the United States of America, 116, 25405-25411 (2019) · Zbl 1456.70002 · doi:10.1073/pnas.1905688116 |
| [33] | Raskutti, G., Wainwright, M. J., and Yu, B. (2011), “Minimax Rates of Estimation for High-Dimensional Linear Regression Over \(####\)-Balls,” IEEE Transactions on Information Theory, 57, 6976-6994. · Zbl 1365.62276 |
| [34] | Ravikumar, P.; Wainwright, M. J.; Lafferty, J., “High-Dimensional Ising Model Selection Using l_1-Regularized Logistic Regression, The Annals of Statistics, 38, 1287-1319 (2010) · Zbl 1189.62115 · doi:10.1214/09-AOS691 |
| [35] | Rubenstein, P. K.; Bongers, S.; Schölkopf, B.; Mooij, J. M. (2018) |
| [36] | Schaffter, T.; Marbach, D.; Floreano, D., “GeneNetWeaver: In Silico Benchmark Generation and Performance Profiling of Network Inference Methods, Bioinformatics, 27, 2263-2270 (2011) · doi:10.1093/bioinformatics/btr373 |
| [37] | Talagrand, M., New Concentration Inequalities in Product Spaces,, Inventiones Mathematicae, 126, 505-563 (1996) · Zbl 0893.60001 · doi:10.1007/s002220050108 |
| [38] | Tzafriri, A. R., “Michaelis-Menten Kinetics at High Enzyme Concentrations, Bulletin of Mathematical Biology, 65, 1111-1129 (2003) · Zbl 1334.92185 · doi:10.1016/S0092-8240(03)00059-4 |
| [39] | van der Vaart, A. W.; Wellner, J. A., Weak Convergence and Empirical Processes (1996), New York: Springer-Verlag, New York · Zbl 0862.60002 |
| [40] | Varah, J. M., “A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations, SIAM Journal on Scientific and Statistical Computing, 3, 28-46 (1982) · Zbl 0481.65050 · doi:10.1137/0903003 |
| [41] | Volterra, V., “Variations and Fluctuations of the Number of Individuals in Animal Species Living Together, ICES Journal of Marine Science, 3, 3-51 (1928) · doi:10.1093/icesjms/3.1.3 |
| [42] | Wahba, G., “Bayesian ‘Confidence Intervals’ for the Cross-Validated Smoothing Spline, Journal of the Royal Statistical Society, Series B, 45, 133-150 (1983) · Zbl 0538.65006 · doi:10.1111/j.2517-6161.1983.tb01239.x |
| [43] | Wahba, G., Spline Models for Observational Data (1990), Philadelphia, PA: SIAM, Philadelphia, PA · Zbl 0813.62001 |
| [44] | Wahba, G.; Wang, Y.; Gu, C.; Klein, R.; Klein, B., “Smoothing Spline ANOVA for Exponential Families, With Application to the Wisconsin Epidemiological Study of Diabetic Retinopathy, The Annals of Statistics, 23, 1865-1895 (1995) · Zbl 0854.62042 · doi:10.1214/aos/1034713638 |
| [45] | Wang, S.; Nan, B.; Zhu, N.; Zhu, J., “Hierarchically Penalized Cox Regression With Grouped Variables, Biometrika, 96, 307-322 (2009) · Zbl 1163.62089 · doi:10.1093/biomet/asp016 |
| [46] | Wu, H.; Lu, T.; Xue, H.; Liang, H., “Sparse Additive Ordinary Differential Equations for Dynamic Gene Regulatory Network Modeling, Journal of the American Statistical Association, 109, 700-716 (2014) · Zbl 1367.62221 · doi:10.1080/01621459.2013.859617 |
| [47] | Yuan, M.; Zhou, D.-X., “Minimax Optimal Rates of Estimation in High Dimensional Additive Models, The Annals of Statistics, 44, 2564-2593 (2016) · Zbl 1360.62200 · doi:10.1214/15-AOS1422 |
| [48] | Zhang, C.-H.; Zhang, S. S., “Confidence Intervals for Low Dimensional Parameters in High Dimensional Linear Models, Journal of the Royal Statistical Society, Series B, 76, 217-242 (2014) · Zbl 1411.62196 · doi:10.1111/rssb.12026 |
| [49] | Zhang, T.; Wu, J.; Li, F.; Caffo, B.; Boatman-Reich, D., “A Dynamic Directional Model for Effective Brain Connectivity Using Electrocorticographic (ECoG) Time Series, Journal of the American Statistical Association, 110, 93-106 (2015) · Zbl 1374.92101 · doi:10.1080/01621459.2014.988213 |
| [50] | Zhang, T.; Yin, Q.; Caffo, B.; Sun, Y.; Boatman-Reich, D., “Bayesian Inference of High-Dimensional, Cluster-Structured Ordinary Differential Equation Models With Applications to Brain Connectivity Studies, The Annals of Applied Statistics, 11, 868-897 (2017) · Zbl 1391.62262 · doi:10.1214/17-AOAS1021 |
| [51] | Zhang, X.; Cao, J.; Carroll, R. J., “On the Selection of Ordinary Differential Equation Models With Application to Predator-Prey Dynamical Models, Biometrics, 71, 131-138 (2015) · Zbl 1419.62489 · doi:10.1111/biom.12243 |
| [52] | Zhao, P.; Yu, B., “On Model Selection Consistency of Lasso, Journal of Machine Learning Research, 7, 2541-2563 (2006) · Zbl 1222.62008 |
| [53] | Zhu, H.; Yao, F.; Zhang, H. H., “Structured Functional Additive Regression in Reproducing Kernel Hilbert Spaces, Journal of the Royal Statistical Society, Series B, 76, 581-603 (2014) · Zbl 1411.62111 · doi:10.1111/rssb.12036 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
