Higher-order total directional variation: imaging applications. (English) Zbl 07292249
Summary: We introduce a class of higher-order anisotropic total variation regularizers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the total generalized variation regularizer and its variants. We propose a primal-dual hybrid gradient approach to approximating numerically the associated gradient flow. This choice of regularizers allows us to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements.
MSC:
| 47A52 | Linear operators and ill-posed problems, regularization |
| 49M30 | Other numerical methods in calculus of variations (MSC2010) |
| 49N45 | Inverse problems in optimal control |
| 65J22 | Numerical solution to inverse problems in abstract spaces |
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
Keywords:
total directional variation; anisotropy; denoising; wavelet-based zooming; digital elevation mapReferences:
| [1] | T. Aach, C. Mota, I. Stuke, M. Muhlich, and E. Barth, Analysis of superimposed oriented patterns, IEEE Trans. Image Process., 15 (2006), pp. 3690-3700, https://doi.org/10.1109/TIP.2006.884921. |
| [2] | A. Almansa, F. Cao, Y. Gousseau, and B. Rouge, Interpolation of digital elevation models using AMLE and related methods, IEEE Trans. Geosci. Remote Sensing, 40 (2002), pp. 314-325, https://doi.org/10.1109/36.992791. |
| [3] | L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel, Axioms and fundamental equations of image processing, Arch. Rational Mech. Anal., 123 (1993), pp. 199-257. · Zbl 0788.68153 |
| [4] | C. Ballester, M. Bertalmio, V. Caselles, G. Sapiro, and J. Verdera, Filling-in by joint interpolation of vector fields and gray levels, IEEE Trans. Image Process., 10 (2001), pp. 1200-1211, https://doi.org/10.1109/83.935036. · Zbl 1037.68771 |
| [5] | I. Bayram and M. E. Kamasak, Directional total variation, IEEE Signal Process. Lett., 19 (2012), pp. 781-784, https://doi.org/10.1109/LSP.2012.2220349. |
| [6] | F. Beauducel, READHGT: Import/download NASA SRTM data files (.HGT), 2012, https://www.mathworks.com/matlabcentral/fileexchange/36379. |
| [7] | B. Berkels, M. Burger, M. Droske, O. Nemitz, and M. Rumpf, Cartoon extraction based on anisotropic image classification, in Vision, Modeling, and Visualization Proceedings, 2006, pp. 293-300, http://numod.ins.uni-bonn.de/research/papers/public/BeBuDr06.pdf. |
| [8] | T. R. Bin Wu and X.-C. Tai, Sparse-Data Based 3D Surface Reconstruction for Cartoon and Map, Internal report, UCLA, 2017, ftp://ftp.math.ucla.edu/pub/camreport/cam17-38.pdf. |
| [9] | P. Blomgren and T. F. Chan, Color TV: Total variation methods for restoration of vector-valued images, IEEE Trans. Image Process., 7 (1998), pp. 304-309, https://doi.org/10.1109/83.661180. |
| [10] | K. Bredies and M. Holler, A TGV regularized wavelet based zooming model, in Scale Space and Variational Methods in Computer Vision (SSVM 2013), Springer, 2013, pp. 149-160, https://doi.org/10.1007/978-3-642-38267-3_13. |
| [11] | K. Bredies, K. Kunisch, and T. Pock, Total generalized variation, SIAM J. Imaging Sci., 3 (2010), pp. 492-526, https://doi.org/10.1137/090769521. · Zbl 1195.49025 |
| [12] | V. Caselles, A. Chambolle, D. Cremers, M. Novaga, and T. Pock, An introduction to total variation for image analysis, Theoret. Found. Numer. Methods Sparse Recovery, 9 (2010), pp. 263-340, https://doi.org/10.1515/9783110226157.263. · Zbl 1209.94004 |
| [13] | V. Caselles, A. Chambolle, and M. Novaga, The discontinuity set of solutions of the TV denoising problem and some extensions, Multiscale Model. Simul., 6 (2007), pp. 879-894, https://doi.org/10.1137/070683003. · Zbl 1145.49024 |
| [14] | V. Caselles, B. Tang, and G. Sapiro, Direction diffusion, in Proceedings of the Seventh IEEE International Conference on Computer Vision (ICCV), Vol. 2, 1999, pp. 1245-1252 , https://doi.org/10.1109/ICCV.1999.790423. |
| [15] | A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis., 40 (2011), pp. 120-145, https://doi.org/10.1007/s10851-010-0251-1. · Zbl 1255.68217 |
| [16] | A. Chambolle and T. Pock, An introduction to continuous optimization for imaging, Acta Numer., 25 (2016), pp. 161-319, https://doi.org/10.1017/S096249291600009X. · Zbl 1343.65064 |
| [17] | T. Chan, S. Esedoglu, and F. Park, A fourth order dual method for staircase reduction in texture extraction and image restoration problems, in 2010 IEEE International Conference on Image Processing, 2010, pp. 4137-4140, https://doi.org/10.1109/ICIP.2010.5653199. |
| [18] | T. Chan, A. Marquina, and P. Mulet, High-order total variation-based image restoration, SIAM J. Sci. Comput., 22 (2000), pp. 503-516, https://doi.org/10.1137/S1064827598344169. · Zbl 0968.68175 |
| [19] | K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3-D transform-domain collaborative filtering, IEEE Trans. Image Process., 16 (2007), pp. 2080-2095, https://doi.org/10.1109/TIP.2007.901238. |
| [20] | R. Dalgas Kongskov, Y. Dong, and K. Knudsen, Directional Total Generalized Variation Regularization, preprint, https://arxiv.org/abs/1701.02675, 2017. · Zbl 1429.49035 |
| [21] | J. C. De los Reyes, C.-B. Schönlieb, and T. Valkonen, Bilevel parameter learning for higher-order total variation regularisation models, J. Math. Imaging Vis., 57 (2017), pp. 1-25, https://doi.org/10.1007/s10851-016-0662-8. · Zbl 1425.94010 |
| [22] | Y. Dong and M. Hintermüller, Multi-scale Total Variation with Automated Regularization Parameter Selection for Color Image Restoration, Springer, 2009, pp. 271-281, https://doi.org/10.1007/978-3-642-02256-2_23. |
| [23] | M. J. Ehrhardt and M. M. Betcke, Multicontrast MRI reconstruction with structure-guided total variation, SIAM J. Imaging Sci., 9 (2016), pp. 1084-1106, https://doi.org/10.1137/15M1047325. · Zbl 1346.47006 |
| [24] | V. Estellers, S. Soatto, and X. Bresson, Adaptive regularization with the structure tensor, IEEE Trans. Image Process., 24 (2015), pp. 1777-1790, https://doi.org/10.1109/TIP.2015.2409562. · Zbl 1408.94166 |
| [25] | J. Fehrenbach and J.-M. Mirebeau, Sparse non-negative stencils for anisotropic diffusion, J. Math. Imaging Vis., 49 (2014), pp. 123-147, https://doi.org/10.1007/s10851-013-0446-3. · Zbl 1365.68451 |
| [26] | W. Förstner, A feature based correspondence algorithm for image matching, Int. Arch. Photogrammetry Remote Sensing, 26 (1986), pp. 150-166. |
| [27] | M. Grasmair and F. Lenzen, Anisotropic total variation filtering, Appl. Math. Optim., 62 (2010), pp. 323-339, https://doi.org/10.1007/s00245-010-9105-x. · Zbl 1205.35152 |
| [28] | P. K. Hansma, G. Schitter, G. E. Fantner, and C. Prater, High-speed atomic force microscopy, Science, 314 (2006), pp. 601-602, https://doi.org/10.1126/science.1133497. |
| [29] | C. Harris and M. Stephens, A combined corner and edge detector, in Proceedings of the Alvey Vision Conference, Alvety Vision Club, 1988, pp. 23.1-23.6, https://doi.org/10.5244/C.2.23. |
| [30] | M. Kass and A. Witkin, Analyzing oriented patterns, Computer Vision Graphics Image Process., 37 (1987), pp. 362-385, https://doi.org/10.1016/0734-189X(87)90043-0. |
| [31] | R. Kimmel, R. Malladi, and N. Sochen, Images as embedded maps and minimal surfaces: Movies, color, texture, and volumetric medical images, Internat. J. Comput. Vis., 39 (2000), pp. 111-129, https://doi.org/10.1023/A:1008171026419. · Zbl 1060.68656 |
| [32] | F. Knoll, K. Bredies, T. Pock, and R. Stollberger, Second order total generalized variation (TGV) for MRI, Magnetic Resonance in Medicine, 65 (2011), pp. 480-491, https://doi.org/10.1002/mrm.22595. |
| [33] | S. Lefkimmiatis, A. Roussos, P. Maragos, and M. Unser, Structure tensor total variation, SIAM J. Imaging Sci., 8 (2015), pp. 1090-1122, https://doi.org/10.1137/14098154X. · Zbl 1315.65019 |
| [34] | J. Lellmann, J.-M. Morel, and C.-B. Schönlieb, Anisotropic third-order regularization for sparse digital elevation models, in Scale Space and Variational Methods in Computer Vision, Springer, 2013, pp. 161-173, https://doi.org/10.1007/978-3-642-38267-3_14. |
| [35] | F. Lenzen, F. Becker, J. Lellmann, S. Petra, and C. Schnörr, A class of quasi-variational inequalities for adaptive image denoising and decomposition, Comput. Optim. Appl., 54 (2013), pp. 371-398, https://doi.org/10.1007/s10589-012-9456-0. · Zbl 1269.90115 |
| [36] | J. Meinguet, Surface Spline Interpolation: Basic Theory and Computational Aspects, Springer, 1984, pp. 127-142, https://doi.org/10.1007/978-94-009-6466-2_6. · Zbl 0549.41006 |
| [37] | D. Múller and Y. Dufrêne, Atomic Force Microscopy: A nanoscopic window on the cell surface, Trends Cell Biol., 21 (2011), pp. 461 – 469, https://doi.org/10.1016/j.tcb.2011.04.008. |
| [38] | M. Nikolova, Local strong homogeneity of a regularized estimator, SIAM J. Appl. Math., 61 (2000), pp. 633-658, https://doi.org/10.1137/S0036139997327794. · Zbl 0991.94015 |
| [39] | C. S. Oxvig, T. Arildsen, and T. Larsen, Structure assisted compressed sensing reconstruction of undersampled AFM images, Ultramicroscopy, 172 (2017), pp. 1-9, https://doi.org/10.1016/j.ultramic.2016.09.011. |
| [40] | K. Papafitsoros and C. Schönlieb, A combined first and second order variational approach for image reconstruction, J. Math. Imaging Vis., 48 (2014), pp. 308-338, https://doi.org/10.1007/s10851-013-0445-4. · Zbl 1362.94009 |
| [41] | S. Parisotto, Anisotropic Variational Models and PDEs for Inverse Imaging Problems, Ph.D. thesis, University of Cambridge Repository, 2019, https://doi.org/10.17863/cam.38240. |
| [42] | S. Parisotto, S. Masnou, and C.-B. Schönlieb, Higher-order total directional variation: Analysis, SIAM J. Imaging Sci., 13 (2020), pp. 474-496, https://doi.org/10.1137/19M1239210. · Zbl 1444.47020 |
| [43] | S. Parisotto and C.-B. Schönlieb, Total directional variation for video denoising, in Scale Space and Variational Methods in Computer Vision, J. Lellmann, M. Burger, and J. Modersitzki, eds., Springer, Cham, 2019, pp. 522-534, https://doi.org/10.1007/978-3-030-22368-7_41. · Zbl 1536.94007 |
| [44] | C. Rankl, Atomic Force Microscopy Images of Cell Specimens, 2015, https://doi.org/10.5281/zenodo.17573. |
| [45] | L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259-268, https://doi.org/10.1016/0167-2789(92)90242-F. · Zbl 0780.49028 |
| [46] | G. Sapiro and D. L. Ringach, Anisotropic diffusion of multivalued images with applications to color filtering, IEEE Trans. Image Process., 5 (1996), pp. 1582-1586, https://doi.org/10.1109/83.541429. · Zbl 0852.68113 |
| [47] | O. Savin, \(C^1\) regularity for infinity harmonic functions in two dimensions, Arch. Ration. Mech. Anal., 176 (2005), pp. 351-361, https://doi.org/10.1007/s00205-005-0355-8. · Zbl 1112.35070 |
| [48] | G. Schitter and M. Rost, Scanning probe microscopy at video-rate, Materials Today, 11 (2008), pp. 40-48, https://doi.org/10.1016/S1369-7021(09)70006-9. |
| [49] | S. Setzer and G. Steidl, Variational methods with higher-order derivatives in image processing, in Approximation Theory XII, Nashboro Press, Brentwood, TN, 2008, pp. 360-386. · Zbl 1175.68520 |
| [50] | S. Setzer, G. Steidl, and T. Teuber, Infimal convolution regularizations with discrete \(\ell_1\)-type functionals, Commun. Math. Sci., 9 (2011), pp. 797-827, https://doi.org/10.4310/CMS.2011.v9.n3.a7. · Zbl 1269.49063 |
| [51] | G. Steidl and T. Teuber, Anisotropic smoothing using double orientations, in Scale Space and Variational Methods in Computer Vision, X.-C. Tai, K. Mørken, M. Lysaker, and K.-A. Lie, eds., Springer, Berlin, 2009, pp. 477-489, https://doi.org/10.1007/978-3-642-02256-2_40. · Zbl 1371.94352 |
| [52] | U. Trottenberg, C. Oosterlee, and A. Schüller, Multigrid, Elsevier Science, Academic Press, San Diego, CA, 2001. |
| [53] | D. Tschumperlé and R. Deriche, Vector-valued image regularization with PDEs: A common framework for different applications, IEEE Trans. Pattern Anal. Mach. Intell., 27 (2005), pp. 506-517, https://doi.org/10.1109/TPAMI.2005.87. |
| [54] | USGS, Shuttle Radar Topography Mission, 2006, https://www2.jpl.nasa.gov/srtm/. |
| [55] | T. Valkonen and T. Pock, Acceleration of the PDHGM on partially strongly convex functions, J. Math. Imaging Vis., 59 (2017), pp. 394-414, https://doi.org/10.1007/s10851-016-0692-2. · Zbl 1382.90080 |
| [56] | Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004), pp. 600-612, https://doi.org/10.1109/TIP.2003.819861. |
| [57] | J. Weickert, Anisotropic Diffusion in Image Processing, B. G. Teubner, Stuttgart, 1998. · Zbl 0886.68131 |
| [58] | J. Weickert, Coherence-enhancing diffusion filtering, Internat. J. Comput. Vis., 31 (1999), pp. 111-127, https://doi.org/10.1023/A:1008009714131. · Zbl 1505.94010 |
| [59] | J. Weickert and H. Scharr, A scheme for coherence-enhancing diffusion filtering with optimized rotation invariance, J. Visual Commun. Image Representation, 13 (2002), pp. 103-118, https://doi.org/10.1006/jvci.2001.0495. |
| [60] | C. Wu and X.-C. Tai, Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models, SIAM J. Imaging Sci., 3 (2010), pp. 300-339, https://doi.org/10.1137/090767558. · Zbl 1206.90245 |
| [61] | S. D. Zenzo, A note on the gradient of a multi-image, Computer Vision Graphics Image Process., 33 (1986), pp. 116-125, https://doi.org/10.1016/0734-189X(86)90223-9. · Zbl 0625.68065 |
| [62] | H. Zhang and Y. Wang, Edge adaptive directional total variation, J. Engrg., (2013), https://doi.org/10.1049/joe.2013.0116. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
