Total variation regularized Fréchet regression for metric-space valued data. (English) Zbl 1486.62326
Summary: Non-Euclidean data that are indexed with a scalar predictor such as time are increasingly encountered in data applications, while statistical methodology and theory for such random objects are not well developed yet. To address the need for new methodology in this area, we develop a total variation regularization technique for nonparametric Fréchet regression, which refers to a regression setting where a response residing in a metric space is paired with a scalar predictor and the target is a conditional Fréchet mean. Specifically, we seek to approximate an unknown metric-space valued function by an estimator that minimizes the Fréchet version of least squares and at the same time has small total variation, appropriately defined for metric-space valued objects. We show that the resulting estimator is representable by a piece-wise constant function and establish the minimax convergence rate of the proposed estimator for metric data objects that reside in Hadamard spaces. We illustrate the numerical performance of the proposed method for both simulated and real data, including metric spaces of symmetric positive-definite matrices with the affine-invariant distance, of probability distributions on the real line with the Wasserstein distance, and of phylogenetic trees with the Billera-Holmes-Vogtmann metric.
MSC:
| 62R20 | Statistics on metric spaces |
| 62R30 | Statistics on manifolds |
| 62G08 | Nonparametric regression and quantile regression |
| 62H12 | Estimation in multivariate analysis |
| 62G20 | Asymptotic properties of nonparametric inference |
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
Keywords:
Alexandrov space; brain imaging; Fréchet mean; Hadamard space; non-Euclidean data; phylogenetic tree; random objects; symmetric positive-definite matrix; Wasserstein metricReferences:
| [1] | Afsari, B. (2011). Riemannian \[{L^p}\] center of mass: Existence, uniqueness, and convexity. Proc. Amer. Math. Soc. 139 655-673. · Zbl 1220.53040 · doi:10.1090/S0002-9939-2010-10541-5 |
| [2] | Alquier, P., Cottet, V. and Lecué, G. (2019). Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions. Ann. Statist. 47 2117-2144. · Zbl 1466.62289 · doi:10.1214/18-AOS1742 |
| [3] | Arsigny, V., Fillard, P., Pennec, X. and Ayache, N. (2006). Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magn. Reson. Med. 56 411-421. |
| [4] | Arsigny, V., Fillard, P., Pennec, X. and Ayache, N. (2007). Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 29 328-347. · Zbl 1144.47015 · doi:10.1137/050637996 |
| [5] | Bačák, M. (2015). Convergence of nonlinear semigroups under nonpositive curvature. Trans. Amer. Math. Soc. 367 3929-3953. · Zbl 1310.58004 · doi:10.1090/S0002-9947-2015-06087-5 |
| [6] | Bergmann, R. and Weinmann, A. (2016). A second-order TV-type approach for inpainting and denoising higher dimensional combined cyclic and vector space data. J. Math. Imaging Vision 55 401-427. · Zbl 1338.65155 · doi:10.1007/s10851-015-0627-3 |
| [7] | Bergmann, R., Laus, F., Steidl, G. and Weinmann, A. (2014). Second order differences of cyclic data and applications in variational denoising. SIAM J. Imaging Sci. 7 2916-2953. · Zbl 1309.65022 · doi:10.1137/140969993 |
| [8] | Bhattacharya, R. and Patrangenaru, V. (2003). Large sample theory of intrinsic and extrinsic sample means on manifolds. I. Ann. Statist. 31 1-29. · Zbl 1020.62026 · doi:10.1214/aos/1046294456 |
| [9] | Billera, L. J., Holmes, S. P. and Vogtmann, K. (2001). Geometry of the space of phylogenetic trees. Adv. in Appl. Math. 27 733-767. · Zbl 0995.92035 · doi:10.1006/aama.2001.0759 |
| [10] | Bridson, M. R. (1991). Geodesics and curvature in metric simplicial complexes. Ph.D. thesis, Cornell University. · Zbl 0844.53034 |
| [11] | Bridson, M. R. and Haefliger, A. (1999). Metric Spaces of Non-positive Curvature. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 319. Springer, Berlin. · Zbl 0988.53001 · doi:10.1007/978-3-662-12494-9 |
| [12] | Burago, D., Burago, Y. and Ivanov, S. (2001). A Course in Metric Geometry. Graduate Studies in Mathematics 33. Amer. Math. Soc., Providence, RI. · Zbl 0981.51016 · doi:10.1090/gsm/033 |
| [13] | Chambolle, A., Caselles, V., Cremers, D., Novaga, M. and Pock, T. (2010). An introduction to total variation for image analysis. In Theoretical Foundations and Numerical Methods for Sparse Recovery. Radon Ser. Comput. Appl. Math. 9 263-340. de Gruyter, Berlin. · Zbl 1209.94004 · doi:10.1515/9783110226157.263 |
| [14] | Chang, T. (1989). Spherical regression with errors in variables. Ann. Statist. 17 293-306. · Zbl 0669.62042 · doi:10.1214/aos/1176347017 |
| [15] | Chinot, G., Lecué, G. and Lerasle, M. (2020). Robust statistical learning with Lipschitz and convex loss functions. Probab. Theory Related Fields 176 897-940. · Zbl 1436.62178 · doi:10.1007/s00440-019-00931-3 |
| [16] | Cornea, E., Zhu, H., Kim, P. and Ibrahim, J. G. (2017). Regression models on Riemannian symmetric spaces. J. R. Stat. Soc. Ser. B. Stat. Methodol. 79 463-482. · Zbl 1414.62177 · doi:10.1111/rssb.12169 |
| [17] | Dai, X. and Müller, H.-G. (2018). Principal component analysis for functional data on Riemannian manifolds and spheres. Ann. Statist. 46 3334-3361. · Zbl 1454.62553 · doi:10.1214/17-AOS1660 |
| [18] | Davis, B. C., Fletcher, P. T., Bullitt, E. and Joshi, S. (2010). Population shape regression from random design data. Int. J. Comput. Vis. 90 255-266. |
| [19] | Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, R. P. et al. (2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage 31 968-980. |
| [20] | Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921. · Zbl 0935.62041 · doi:10.1214/aos/1024691081 |
| [21] | Dryden, I. L., Koloydenko, A. and Zhou, D. (2009). Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. Ann. Appl. Stat. 3 1102-1123. · Zbl 1196.62063 · doi:10.1214/09-AOAS249 |
| [22] | Dubey, P. and Müller, H.-G. (2020a). Fréchet change-point detection. Ann. Statist. 48 3312-3335. · Zbl 1461.62243 · doi:10.1214/19-AOS1930 |
| [23] | Dubey, P. and Müller, H.-G. (2020b). Functional models for time-varying random objects. J. R. Stat. Soc. Ser. B. Stat. Methodol. 82 275-327. · Zbl 07554756 |
| [24] | Eltzner, B. and Huckemann, S. F. (2019). A smeary central limit theorem for manifolds with application to high-dimensional spheres. Ann. Statist. 47 3360-3381. · Zbl 1436.60032 · doi:10.1214/18-AOS1781 |
| [25] | Essen, D. C. V., Smith, S. M., Barch, D. M., Behrens, T. E. J., Yacoub, E., Ugurbil, K. and Wu-Minn HCP Consortium (2013). The Wu-Minn human connectome project: An overview. NeuroImage 80 62-79. |
| [26] | Fang, B., Guntuboyina, A. and Sen, B. (2021). Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation. Ann. Statist. 49 769-792. · Zbl 1471.62372 · doi:10.1214/20-aos1977 |
| [27] | Faraway, J. J. (2014). Regression for non-Euclidean data using distance matrices. J. Appl. Stat. 41 2342-2357. · Zbl 1514.62555 · doi:10.1080/02664763.2014.909794 |
| [28] | Fillard, P., Arsigny, V., Ayache, N. and Pennec, X. (2005). A Riemannian framework for the processing of tensor-valued images. In International Workshop on Deep Structure, Singularities, and Computer Vision 112-123. |
| [29] | Fisher, N. I. (1995). Statistical Analysis of Circular Data. Cambridge Univ. Press, Cambridge. · Zbl 0788.62047 · doi:10.1017/CBO9780511564345 |
| [30] | Fletcher, P. T. (2013). Geodesic regression and the theory of least squares on Riemannian manifolds. Int. J. Comput. Vis. 105 171-185. · Zbl 1304.62092 · doi:10.1007/s11263-012-0591-y |
| [31] | Fletcher, T. and Joshi, S. (2007). Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Process. 87 250-262. · Zbl 1186.94126 |
| [32] | Friston, K. J. (2011). Functional and effective connectivity: A review. Brain Connect. 1 13-36. |
| [33] | Green, M. F., Horan, W. P. and Lee, J. (2015). Social cognition in schizophrenia. Nat. Rev. Neurosci. 16 620-631. |
| [34] | Hein, M. (2009). Robust nonparametric regression with metric-space valued output. In Advances in Neural Information Processing Systems 718-726. |
| [35] | Hinkle, J., Fletcher, P. T. and Joshi, S. (2014). Intrinsic polynomials for regression on Riemannian manifolds. J. Math. Imaging Vision 50 32-52. · Zbl 1310.53038 · doi:10.1007/s10851-013-0489-5 |
| [36] | Hotz, T., Huckemann, S., Le, H., Marron, J. S., Mattingly, J. C., Miller, E., Nolen, J., Owen, M., Patrangenaru, V. et al. (2013). Sticky central limit theorems on open books. Ann. Appl. Probab. 23 2238-2258. · Zbl 1293.60006 · doi:10.1214/12-AAP899 |
| [37] | Hütter, J.-C. and Rigollet, P. (2016). Optimal rates for total variation denoising. In 29th Annual Conference on Learning Theory (V. Feldman, A. Rakhlin and O. Shamir, eds.). Proceedings of Machine Learning Research 49 1115-1146. PMLR, Columbia University, New York, New York, USA. |
| [38] | Kim, S.-J., Koh, K., Boyd, S. and Gorinevsky, D. (2009). \[{l_1}\] trend filtering. SIAM Rev. 51 339-360. · Zbl 1171.37033 · doi:10.1137/070690274 |
| [39] | Kloeckner, B. (2010). A geometric study of Wasserstein spaces: Euclidean spaces. Ann. Sc. Norm. Super. Pisa Cl. Sci. (5) 9 297-323. · Zbl 1218.53079 |
| [40] | Kolar, M. and Xing, E. P. (2012). Estimating networks with jumps. Electron. J. Stat. 6 2069-2106. · Zbl 1295.62032 · doi:10.1214/12-EJS739 |
| [41] | Ledoux, M. and Talagrand, M. (2011). Probability in Banach Spaces: Isoperimetry and Processes. Classics in Mathematics. Springer, Berlin. · Zbl 1226.60003 |
| [42] | Lang, S. (1995). Differential and Riemannian Manifolds. Springer, New York. · Zbl 0824.58003 |
| [43] | Lellmann, J., Strekalovskiy, E., Koetter, S. and Cremers, D. (2013). Total variation regularization for functions with values in a manifold. In 2013 IEEE International Conference on Computer Vision 2944-2951. IEEE, New York. |
| [44] | Lin, Z. (2019). Riemannian geometry of symmetric positive definite matrices via Cholesky decomposition. SIAM J. Matrix Anal. Appl. 40 1353-1370. · Zbl 07141455 · doi:10.1137/18M1221084 |
| [45] | Lin, Z. and Müller, H.-G. (2021). Supplement to “Total variation regularized Fréchet regression for metric-space valued data.” https://doi.org/10.1214/21-AOS2095SUPP |
| [46] | Mammen, E. and van de Geer, S. (1997). Locally adaptive regression splines. Ann. Statist. 25 387-413. · Zbl 0871.62040 · doi:10.1214/aos/1034276635 |
| [47] | Moakher, M. (2005). A differential geometric approach to the geometric mean of symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 26 735-747. · Zbl 1079.47021 · doi:10.1137/S0895479803436937 |
| [48] | Ortelli, F. and van de Geer, S. (2018). On the total variation regularized estimator over a class of tree graphs. Electron. J. Stat. 12 4517-4570. · Zbl 1411.62208 · doi:10.1214/18-ejs1519 |
| [49] | Owen, M. and Provan, J. S. (2011). A fast algorithm for computing geodesic distances in tree space. IEEE/ACM Trans. Comput. Biol. Bioinform. 8 2-13. |
| [50] | Patrangenaru, V. and Ellingson, L. (2015). Nonparametric Statistics on Manifolds and Their Applications to Object Data Analysis. CRC Press, Boca Raton, FL. · Zbl 1331.62007 |
| [51] | Pelletier, B. (2006). Non-parametric regression estimation on closed Riemannian manifolds. J. Nonparametr. Stat. 18 57-67. · Zbl 1088.62053 · doi:10.1080/10485250500504828 |
| [52] | Pennec, X. (2018). Barycentric subspace analysis on manifolds. Ann. Statist. 46 2711-2746. · Zbl 1410.60018 · doi:10.1214/17-AOS1636 |
| [53] | Pennec, X., Fillard, P. and Ayache, N. (2006). A Riemannian framework for tensor computing. Int. J. Comput. Vis. 66 41-66. · Zbl 1287.53031 |
| [54] | Petersen, A. and Müller, H.-G. (2016). Fréchet integration and adaptive metric selection for interpretable covariances of multivariate functional data. Biometrika 103 103-120. · Zbl 1452.62400 · doi:10.1093/biomet/asv054 |
| [55] | Petersen, A. and Müller, H.-G. (2019). Fréchet regression for random objects with Euclidean predictors. Ann. Statist. 47 691-719. · Zbl 1417.62091 · doi:10.1214/17-AOS1624 |
| [56] | Petrunin, A. and Tuschmann, W. (1999). Diffeomorphism finiteness, positive pinching, and second homotopy. Geom. Funct. Anal. 9 736-774. · Zbl 0941.53026 · doi:10.1007/s000390050101 |
| [57] | Rathi, Y., Tannenbaum, A. and Michailovich, O. (2007). Segmenting images on the tensor manifold. In 2007 IEEE Conference on Computer Vision and Pattern Recognition 1-8. IEEE, New York. |
| [58] | Rudin, L. I., Osher, S. and Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Phys. D, Nonlinear Phenom. 60 259-268. · Zbl 0780.49028 · doi:10.1016/0167-2789(92)90242-F |
| [59] | Sacks, J. and Ylvisaker, D. (1970). Designs for regression problems with correlated errors. III. Ann. Math. Stat. 41 2057-2074. · Zbl 0234.62025 · doi:10.1214/aoms/1177696705 |
| [60] | Sadhanala, V., Wang, Y.-X. and Tibshirani, R. J. (2016). Total variation classes beyond 1d: Minimax rates, and the limitations of linear smoothers. In Neural Information Processing Systems 3521-3529. |
| [61] | Shi, X., Styner, M., Lieberman, J., Ibrahim, J. G., Lin, W. and Zhu, H. (2009). Intrinsic regression models for manifold-valued data. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 12 192-199. |
| [62] | Steinke, F., Hein, M. and Schölkopf, B. (2010). Nonparametric regression between general Riemannian manifolds. SIAM J. Imaging Sci. 3 527-563. · Zbl 1195.41011 · doi:10.1137/080744189 |
| [63] | Strong, D. and Chan, T. (2003). Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl. 19 S165-S187. · Zbl 1043.94512 · doi:10.1088/0266-5611/19/6/059 |
| [64] | Sturm, K.-T. (2003). Probability measures on metric spaces of nonpositive curvature. In Heat Kernels and Analysis on Manifolds, Graphs, and Metric Spaces (Paris, 2002). Contemp. Math. 338 357-390. Amer. Math. Soc., Providence, RI. · Zbl 1040.60002 · doi:10.1090/conm/338/06080 |
| [65] | Tibshirani, R. J. (2014). Adaptive piecewise polynomial estimation via trend filtering. Ann. Statist. 42 285-323. · Zbl 1307.62118 · doi:10.1214/13-AOS1189 |
| [66] | Tibshirani, R., Saunders, M., Rosset, S., Zhu, J. and Knight, K. (2005). Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 91-108. · Zbl 1060.62049 · doi:10.1111/j.1467-9868.2005.00490.x |
| [67] | van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924. · Zbl 0709.62040 · doi:10.1214/aos/1176347632 |
| [68] | van de Geer, S. (2001). Least squares estimation with complexity penalties. Math. Methods Statist. 10 355-374. · Zbl 1005.62043 |
| [69] | Wang, X., Zhu, H. and ADNI (2017). Generalized scalar-on-image regression models via total variation. J. Amer. Statist. Assoc. 112 1156-1168. · doi:10.1080/01621459.2016.1194846 |
| [70] | Wang, Y.-X., Sharpnack, J., Smola, A. J. and Tibshirani, R. J. (2016). Trend filtering on graphs. J. Mach. Learn. Res. 17 Paper No. 105, 41 pp. · Zbl 1369.62082 |
| [71] | Weinmann, A., Demaret, L. and Storath, M. (2014). Total variation regularization for manifold-valued data. SIAM J. Imaging Sci. 7 2226-2257. · Zbl 1309.65071 · doi:10.1137/130951075 |
| [72] | Yuan, Y., Zhu, H., Lin, W. and Marron, J. S. (2012). Local polynomial regression for symmetric positive definite matrices. J. R. Stat. Soc. Ser. B. Stat. Methodol. 74 697-719. · Zbl 1411.62110 · doi:10.1111/j.1467-9868.2011.01022.x |
| [73] | Zhou, D., Dryden, I. L., Koloydenko, A. A., Audenaert, K. M. R. and Bai, L. (2016). Regularisation, interpolation and visualisation of diffusion tensor images using non-Euclidean statistics. J. Appl. Stat. 43 943-978. · Zbl 1514.62982 · doi:10.1080/02664763.2015.1080671 |
| [74] | Zhu, H., Fan, J. and Kong, L. (2014). Spatially varying coefficient model for neuroimaging data with jump discontinuities. J. Amer. Statist. Assoc. 109 1084-1098 · Zbl 1368.62297 · doi:10.1080/01621459.2014.881742 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
