×

Transporting deformations of face emotions in the shape spaces: a comparison of different approaches. (English) Zbl 1522.65029

Summary: Studying the changes of shape is a common concern in many scientific fields. We address here two problems: (1) quantifying the deformation between two given shapes and (2) transporting this deformation to morph a third shape. These operations can be done with or without point correspondence, depending on the availability of a surface matching algorithm, and on the type of mathematical procedure adopted. In computer vision, the re-targeting of emotions mapped on faces is a common application. We contrast here four different methods used for transporting the deformation toward a target once it was estimated upon the matching of two shapes. These methods come from very different fields such as computational anatomy, computer vision and biology. We used the large diffeomorphic deformation metric mapping and thin plate spline, in order to estimate deformations in a deformational trajectory of a human face experiencing different emotions. Then we use naive transport (NT), linear shift (LS), direct transport (DT) and fanning scheme (FS) to transport the estimated deformations toward four alien faces constituted by 240 homologous points and identifying a triangulation structure of 416 triangles. We used both local and global criteria for evaluating the performance of the 4 methods, e.g., the maintenance of the original deformation. We found DT, LS and FS very effective in recovering the original deformation while NT fails under several aspects in transporting the shape change. As the best method may differ depending on the application, we recommend carefully testing different methods in order to choose the best one for any specific application.

MSC:

65D18 Numerical aspects of computer graphics, image analysis, and computational geometry

References:

[1] Aaron, O.: bezier: Bezier Curve and Spline Toolkit (2014). R package version 1.1
[2] Bookstein, FL, Principal warps: thin-plate splines and the decomposition of deformations, IEEE Trans. Pattern Anal. Mach. Intell., 11, 6, 567-585 (1989) · Zbl 0691.65002 · doi:10.1109/34.24792
[3] Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Calculus of non-rigid surfaces for geometry and texture manipulation. In: IEEE Transactions on Visualization and Computer Graphics (2007)
[4] Buckley, PF; Dean, D.; Bookstein, FL; Friedman, L.; Kwon, D.; Lewin, JS; Kamath, J.; Lys, C., Three-dimensional magnetic resonance-based morphometrics and ventricular dysmorphology in schizophrenia, Biol. Psychiat., 45, 1, 62-67 (1999) · doi:10.1016/S0006-3223(98)00067-5
[5] Campbell, K., Fletcher, P.: Efficient parallel transport in the group of diffeomorphisms via reduction to the lie algebra. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10551 LNCS, pp. 186-198 (2017)
[6] Carlson, K.B., De Ruiter, D.J., Dewitt, T.J., Mcnulty, K.P., Carlson, K.J., Tafforeau, P., Berger, L.R.: Developmental simulation of the adult cranial morphology of australopithecus sediba. S. Afr. J. Sci. (2016). doi:10.17159/sajs.2016/20160012
[7] do Carmo Valero, M.P.: Riemannian Geometry. Mathematics: Theory and Applications. Birkhäuser, London (1992) · Zbl 0752.53001
[8] Dryden, I.; Koloydenko, A.; Zhou, D., Non-euclidean statistics for covariance matrices, with applications to diffusion tensor imaging, Ann. Appl. Stat., 3, 1102-1123 (2009) · Zbl 1196.62063 · doi:10.1214/09-AOAS249
[9] Dryden, I.L.: Shapes: Statistical Shape Analysis (2019). R package version 1.2.5
[10] Dryden, IL; Mardia, KV, Statistical Shape Analysis, with Applications in R (2016), Chichester: Wiley, Chichester · Zbl 1381.62003 · doi:10.1002/9781119072492
[11] Durrleman, S.; Pennec, X.; Trouvé, A.; Ayache, N.; Braga, J., Comparison of the endocranial ontogenies between chimpanzees and bonobos via temporal regression and spatiotemporal registration, J Hum Evol, 62, 74-88 (2012) · doi:10.1016/j.jhevol.2011.10.004
[12] Durrleman, S.; Prastawa, M.; Charon, N.; Korenberg, JR; Joshi, S.; Gerig, G.; Trouvé, A., Morphometry of anatomical shape complexes with dense deformations and sparse parameters, NeuroImage, 101, 35-49 (2014) · doi:10.1016/j.neuroimage.2014.06.043
[13] Evangelista, A.; Gabriele, S.; Nardinocchi, P.; Piras, P.; Puddu, P.; Teresi, L.; Torromeo, C.; Varano, V., Non-invasive assessment of functional strain lines in the real human left ventricle via speckle tracking echocardiography, J. Biomech., 48, 3, 465-471 (2015) · doi:10.1016/j.jbiomech.2014.12.028
[14] Gabriele, S.; Nardinocchi, P.; Varano, V., Evaluation of the strain-line patterns in a human left ventricle: a simulation study, Comput. Methods Biomech. Biomed. Eng., 18, 7, 790-798 (2015) · doi:10.1080/10255842.2013.847094
[15] Guigui, N., Jia, S., Sermesant, M., Pennec, X.: Symmetric algorithmic components for shape analysis with diffeomorphisms. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11712 LNCS, pp. 759-768 (2019) · Zbl 1458.94033
[16] Gunz, P., Mitteroecker, P.: Semilandmarks: A method for quantifying curves and surfaces. Hystrix 24(1), (2013)
[17] Le, H., Unrolling shape curves, J. Lond. Math. Soc., 68, 511 (2003) · Zbl 1040.60009 · doi:10.1112/S0024610703004393
[18] Huckemann, S.; Hotz, T.; Munk, A., Intrinsic Manova for Riemannian manifolds with an application to Kendall’s space of planar shapes, IEEE Trans. Pattern Anal. Mach. Intell., 32, 4, 593-603 (2010) · doi:10.1109/TPAMI.2009.117
[19] Joshi, S.; Miller, M., Landmark matching via large deformation diffeomorphisms, IEEE Trans. Image Process., 9, 8, 1357-1370 (2000) · Zbl 0965.37065 · doi:10.1109/83.855431
[20] Kume, A.; Dryden, I.; Le, H., Shape space smoothing splines for planar landmark data, Biometrika, 94, 513-528 (2007) · Zbl 1134.62044 · doi:10.1093/biomet/asm047
[21] Laidlaw, DH; Weickert, J., Visualization and Processing of Tensor Fields (2009), Berlin: Springer, Berlin · Zbl 1159.68003 · doi:10.1007/978-3-540-88378-4
[22] Lorenzi, M.; Pennec, X., Geodesics, parallel transport and one-parameter subgroups for diffeomorphic image registration, Int. J. Comput. Vis., 105, 111-127 (2013) · Zbl 1304.92076 · doi:10.1007/s11263-012-0598-4
[23] Louis, M., Bône, A., Charlier, B., Durrleman, S.: Parallel transport in shape analysis: a scalable numerical scheme. In International Conference on Geometric Science of Information, pp. 29-37. Springer, (2017) · Zbl 1428.53046
[24] Louis, M.; Charlier, B.; Jusselin, P.; Pal, S.; Durrleman, S., A fanning scheme for the parallel transport along geodesics on Riemannian manifolds, SIAM J. Numer. Anal., 56, 4, 2563-2584 (2018) · Zbl 1396.53065 · doi:10.1137/17M1130617
[25] Madeo, A., Piras, P., Re, F., Gabriele, S., Nardinocchi, P., Teresi, L., Torromeo, C., Chialastri, C., Schiariti, M., Giura, G., Evangelista, A., Dominici, T., Varano, V., Zachara, E., Puddu, P.: A new 4d trajectory-based approach unveils abnormal lv revolution dynamics in hypertrophic cardiomyopathy. PLoS ONE 10(4), e0122376 (2015)
[26] Miller, M.; Trouvé, A.; Younes, L., Hamiltonian systems and optimal control in computational anatomy: 100 years since D’arcy Thompson, Annu. Rev. Biomed. Eng., 17, 447-509 (2015) · doi:10.1146/annurev-bioeng-071114-040601
[27] Miller, MI; Qiu, A., The emerging discipline of computational functional anatomy, Neuroimage, 45, 1, S16-S39 (2009) · doi:10.1016/j.neuroimage.2008.10.044
[28] Miller, MI; Trouvé, A.; Younes, L., Geodesic shooting for computational anatomy, J. Math. Imaging Vis., 24, 2, 209-228 (2006) · Zbl 1478.92084 · doi:10.1007/s10851-005-3624-0
[29] Miller, MI; Younes, L.; Trouvé, A., Diffeomorphometry and geodesic positioning systems for human anatomy, Technology, 2, 1, 36-43 (2014) · doi:10.1142/S2339547814500010
[30] Nardinocchi, P.; Teresi, L.; Varano, V., The elastic metric: a review of elasticity with large distortions, Int. J. Non-Linear Mech., 56, 34-42 (2013) · doi:10.1016/j.ijnonlinmec.2013.05.002
[31] Niethammer, M., Vialard, F.: Riemannian metrics for statistics on shapes: parallel transport and scale invariance. In:Proceedings of the 4th MICCAI workshop on Mathematical Foundations of Computational Anatomy (MFCA) (2013)
[32] Piras, P., Evangelista, A., Gabriele, S., Nardinocchi, P., Teresi, L., Torromeo, C., Schiariti, M., Varano, V., Puddu, P.: 4d-analysis of left ventricular heart cycle using procrustes motion analysis. PLoS ONE 9(1), e86896 (2014)
[33] Piras, P.; Marcolini, F.; Raia, P.; Curcio, M.; Kotsakis, T., Testing evolutionary stasis and trends in first lower molar shape of extinct Italian populations of Terricola savii (arvicolidae, rodentia) by means of geometric morphometrics, J. Evolut. Biol., 22, 1, 179-191 (2009) · doi:10.1111/j.1420-9101.2008.01632.x
[34] Piras, P.; Profico, A.; Pandolfi, L.; Raia, P.; Di Vincenzo, F.; Mondanaro, A.; Castiglione, S.; Varano, V., Current options for visualization of local deformation in modern shape analysis applied to paleobiological case studies, Front. Earth Sci., 8, 66 (2020) · doi:10.3389/feart.2020.00066
[35] Piras, P., Sansalone, G., Teresi, L., Moscato, M., Profico, A., Eng, R., Cox, T., Loy, A., Colangelo, P., Kotsakis, T.: Digging adaptation in insectivorous subterranean eutherians. the enigma of mesoscalops montanensis unveiled by geometric morphometrics and finite element analysis. J. Morphol. 276(10), 1157-1171 (2015)
[36] Piras, P., Torromeo, C., Evangelista, A., Gabriele, S., Esposito, G., Nardinocchi, P., Teresi, L., Madeo, A., Schiariti, M., Varano, V., Puddu, P.: Homeostatic left heart integration and disintegration links atrio-ventricular covariation’s dyshomeostasis in hypertrophic cardiomyopathy. Sci. Rep. 7(1), 6257 (2017)
[37] Piras, P., Torromeo, C., Re, F., Evangelista, A., Gabriele, S., Esposito, G., Nardinocchi, P., Teresi, L., Madeo, A., Chialastri, C., Schiariti, M., Varano, V., Uguccioni, M., Puddu, P.: Left atrial trajectory impairment in hypertrophic cardiomyopathy disclosed by geometric morphometrics and parallel transport. Sci. Rep. 6, 34906 (2016)
[38] Pokrass, J.; Bronstein, A.; Bronstein, M., Partial shape matching without point-wise correspondence, Numer. Math., 6, 1, 223-244 (2013) · Zbl 1289.65029
[39] Profico A.V.: Arothron: R Functions for Geometric Morphometrics Analyses (2015). R package
[40] Qiu, A.; Albert, M.; Younes, L.; Miller, M., Time sequence diffeomorphic metric mapping and parallel transport track time-dependent shape changes, Neuroimage, 45, 1, S51-60 (2009) · doi:10.1016/j.neuroimage.2008.10.039
[41] Raia, P., Piras, P., Kotsakis, T.: Detection of plio-quaternary large mammal communities of Italy. An integration of fossil faunas biochronology and similarity. Quatern. Sci. Rev. 25(7-8), 846-854 (2006)
[42] Schlager, S.: Morpho and RVCG—shape analysis in R. In: Zheng, G., Li, S., Szekely, G. (eds). Statistical Shape and Deformation Analysis, pp. 217-256. Academic Press, London (2017)
[43] Schlager, S.: Morpho and RVCG—Shape Analysis in R: R-Packages for Geometric Morphometrics, Shape Analysis and Surface Manipulations (2017)
[44] Schlager, S.; Profico, A.; Di Vincenzo, F.; Manzi, G., Retrodeformation of fossil specimens based on 3d bilateral semi-landmarks: implementation in the r package “morpho”, PLoS ONE, 13, 3, e0194073 (2018) · doi:10.1371/journal.pone.0194073
[45] Srivastava, A.; Klassen, E.; Joshi, S.; Jermyn, I., Shape analysis of elastic curves in Euclidean spaces, IEEE Trans. Pattern Anal. Mach. Intell., 33, 7, 1415-1428 (2011) · doi:10.1109/TPAMI.2010.184
[46] Sundaramoorthi, G.; Mennucci, A.; Soatto, S.; Yezzi, A., A new geometric metric in the space of curves, and applications to tracking deforming objects by prediction and filtering, SIAM J. Imag. Sci., 4, 1, 109-145 (2011) · Zbl 1214.93033 · doi:10.1137/090781139
[47] Trouvé, A.: Diffeomorphisms groups and pattern matching in image analysis. Int. J. Comput. Vis. 28, 213-221 (1998)
[48] Varano, V., Gabriele, S., Teresi, L., Dryden, I., Puddu, P., Torromeo, C., Piras, P.: Comparing shape trajectories of biological soft tissues in the size-and-shape space. In: Biomat 2014 International Symposium on Mathematical and Computational Biology, pp. 351-365 (2015) · Zbl 1344.92109
[49] Varano, V.; Gabriele, S.; Teresi, L.; Dryden, I.; Puddu, P.; Torromeo, C.; Piras, P., The TPS direct transport: a new method for transporting deformations in the size-and-shape space, Int. J. Comput. Vis., 124, 3, 384-408 (2017) · Zbl 1458.68255 · doi:10.1007/s11263-017-1031-9
[50] Xie, Q., Kurtek, S., Le, H., Srivastava, A.: Parallel transport of deformations in shape space of elastic surfaces. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 865-872 (2013)
[51] Younes, L., Shapes and Diffeomorphisms (2010), Heidelberg: Springer, Heidelberg · Zbl 1205.68355 · doi:10.1007/978-3-642-12055-8
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.