Partial differential equations for zooming, deinterlacing and dejittering. (English) Zbl 1235.68312
Summary: In this paper, for imaging applications, we introduce partial differential equations (PDEs), which allow for correcting displacement errors, for dejittering, and for deinterlacing, respectively, in multi-channel data. These equations are derived via semi-groups for non-convex energy functionals. As a particular example, for gray valued data, we find the mean curvature equation and the corresponding non-convex energy functional. As a further application for correction of displacement errors we study image interpolation, in particular zooming, of digital color images. For actual image zooming, the solutions of the proposed PDEs are projected onto a space of functions satisfying interpolation constraints. A comparison of the test results with standard and state-of-the-art interpolation algorithms shows the competitiveness of this approach.
References:
| [1] | Aubert, G., Deriche, R., & Kornprobst, P. (1999). Computing optical flow via variational techniques. The SIAM Journal on Applied Mathematics, 60, 156–182. · Zbl 0942.35057 · doi:10.1137/S0036139998340170 |
| [2] | Ballester, C., Bertalmio, M., Caselles, V., Garrido, L., Ranchin, A., & Marques, V. (2007). An inpainting-based deinterlacing method. IEEE Transactions on Image Processing, 16(10), 2476–2491. · doi:10.1109/TIP.2007.903844 |
| [3] | Belahmidi, A., & Guichard, F. (2004). A partial differential equation approach to image zoom. In Proc. of the 2004 int. conf. on image processing (pp. 649–652). |
| [4] | Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. (2000). Image inpainting. In S. Hoffmeyer (Ed.), Proceedings of the computer graphics conference 2000 (SIGGRAPH-00) (pp. 417–424). New York: ACM Press. |
| [5] | Brézis, H. (1973). Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert. Amsterdam: North-Holland. North-Holland Mathematics Studies, No. 5. Notas de Matemática (50). · Zbl 0252.47055 |
| [6] | Burger, W., & Burge, M. (2005). Digitale Bildverarbeitung. Berlin: Springer. |
| [7] | Chambolle, A. (1994). Partial differential equations and image processing. In Proceedings 1994 international conference on image processing (Vol. 1, pp. 16–20). · Zbl 0837.35146 |
| [8] | Chan, R., Setzer, S., & Steidl, G. (2008). Inpainting by flexible Haar wavelet shrinkage. The SIAM Journal on Imaging Sciences, 1(3), 273–293. · Zbl 1187.68649 · doi:10.1137/070711499 |
| [9] | Chan, T., Kang, S., & Shen, J. (2002). Euler’s elastica and curvature based inpaintings. The SIAM Journal on Applied Mathematics, 63(2), 564–592. · Zbl 1028.68185 |
| [10] | Dacorogna, B. (1982). Lecture notes in computer science : Vol. 922. Weak continuity and weak lower semicontinuity of non-linear functionals. Berlin/Heidelberg/New York: Springer. · Zbl 0484.46041 |
| [11] | Dacorogna, B. (1989). Applied mathematical sciences : Vol. 78. Direct methods in the calculus of variations. Berlin: Springer. · Zbl 0703.49001 |
| [12] | De Haan, G., & Bellers, E. B. (1997). De-interlacing of video data. IEEE Transactions on Consumer Electronics, 43, 819–825. · doi:10.1109/30.628721 |
| [13] | De Haan, G., & Bellers, E. B. (1998). Deinterlacing–an overview. Proceedings of the IEEE, 86, 1839–1857. · doi:10.1109/5.705528 |
| [14] | Deriche, R., Kornprobst, P., & Aubert, G. (1995). Optical flow estimation while preserving its discontinuities: a variational approach. In Proceedings of the 2nd Asian conference on computer vision (Vol. 2, pp. 71–80). |
| [15] | Elbau, P., Grasmair, M., Lenzen, F., & Scherzer, O. (2008). Evolution by non-convex energy functionals (Reports of FSP S092–”Industrial Geometry” 75). University of Innsbruck, Austria. Submitted. |
| [16] | Ghodstinat, M., Bruhn, A., & Weickert, J. (2009). Deinterlacing with motion-compensated anisotropic diffusion. In D. Cremers, B. Rosenhahn, A. Yuille, & F. Schmidt (Eds.), Lecture notes in computer science : Vol. 5604. Statistical and geometrical approaches to visual motion analysis (pp. 91–106). Berlin: Springer. |
| [17] | Grasmair, M., Lenzen, F., Obereder, A., Scherzer, O., & Fuchs, M. (2005). A non-convex PDE scale space. In R. Kimmel, N. A. Sochen, & J. Weickert (Eds.), Lecture notes in computer science : Vol. 3459. Scale space and pde methods in computer vision (pp. 303–315). Berlin: Springer. · Zbl 1119.68475 |
| [18] | Guichard, F., & Malgouyres, F. (1998). Total variation based interpolation. In Proceedings of the European signal processing conference (Vol. 3, pp. 1741–1744). |
| [19] | Guichard, F., Morel, J. M., & Ryan, R. (2007). textitContrast invariant image analysis and PDE’s. Preprint, Paris. Work in preparation. |
| [20] | Jähne, B. (2002). Digitale Bildverarbeitung (5th ed.). Berlin: Springer. · Zbl 0682.68006 |
| [21] | Kang, S.-H., & Shen, J. (2006). Video dejittering by Bake and Shake. Image and Vision Computing, 24(2), 143–152. · doi:10.1016/j.imavis.2005.09.022 |
| [22] | Keller, S., Lauze, F., & Nielsen, M. (2005). A total variation motion adaptive deinterlacing scheme. In R. Kimmel, N. A. Sochen, & J. Weickert (Eds.), Lecture notes in computer science : Vol. 3459. Scale space and PDE methods in computer vision (pp. 408–419). Berlin: Springer. · Zbl 1119.68485 |
| [23] | Keller, S. H., Lauze, F., & Nielsen, M. (2008). Deinterlacing using variational methods IEEE. Transactions on Image Processing, 17(11), 2015–2028. · Zbl 1371.94192 · doi:10.1109/TIP.2008.2003394 |
| [24] | Lenzen, F., & Scherzer, O. (2009). A geometric PDE for interpolation of m-channel data. In X. C. Tai, K. Mørken, M. Lysaker, & K. A. Lie (Eds.), Lecture notes in computer science : Vol. 5567. Scale space and variational methods in computer vision (pp. 413–425). Second international conference, SSVM 2009, Voss, Norway, June 1–5. Berlin: Springer. |
| [25] | Malgouyres, F., & Guichard, F. (2001). Edge direction preserving image zooming: a mathematical and numerical analysis. SIAM Journal on Numerical Analysis, 39, 1–37. · Zbl 1001.68174 · doi:10.1137/S0036142999362286 |
| [26] | Nashed, M. (Ed.) (1976). Generalized inverses and applications. New York: Academic Press [Harcourt Brace Jovanovich Publishers]. · Zbl 0346.15001 |
| [27] | Nesi, P. (1993). Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11(7), 419–439. · doi:10.1016/0262-8856(93)90046-J |
| [28] | Nikolova, M., (2009). Fast dejittering for digital video frames. In X. C. Tai, K. Mørken, M. Lysaker, & K. A. Lie (Eds.), Lecture notes in computer science : Vol. 5567. Scale space and variational methods in computer vision (pp. 439–451). Second International Conference, SSVM 2009, Voss, Norway, June 1-5. Berlin: Springer. |
| [29] | Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158. · doi:10.1007/s11263-005-3960-y |
| [30] | Roussos, A., & Maragos, P. (2007). Vector-valued image interpolation by an anisotropic diffusion-projection PDE. In F. Sgallari, A. Murli, & N. Paragios (Eds.), Lecture notes in computer science : Vol. 4485. Scale space and variational methods in computer vision (pp. 104–115). First International Conference, SSVM 2007, Ischia, Italy, May 30–June 2. Berlin: Springer. |
| [31] | Roussos, A., & Maragos, P. (2009). Reversible interpolation of vectorial images by an anisotropic diffusion-projection PDE. International Journal of Computer Vision, 84(2), 130–145. · Zbl 1483.65034 · doi:10.1007/s11263-008-0132-x |
| [32] | Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42. · Zbl 1012.68731 · doi:10.1023/A:1014573219977 |
| [33] | Scherzer, O., & Weickert, J. (2000). Relations between regularization and diffusion filtering. Journal of Mathematical Imaging and Vision, 12(1), 43–63. · Zbl 0945.68183 · doi:10.1023/A:1008344608808 |
| [34] | Shen, J. (2004). Bayesian video dejittering by the BV image model. The SIAM Journal on Applied Mathematics, 64(5), 1691–1708. · Zbl 1055.94505 · doi:10.1137/S0036139902418699 |
| [35] | Szu, H. H., Buss, J. R., Garcia, J. P., Breaux, N. A. et al. (2004). Subpixel jitter video restoration on board of micro-UAV. Proceedings of SPIE, 5439(183). |
| [36] | Thevenaz, P., Blu, T., & Unser, M. (2000). Image interpolation and resampling. In I. N. Bankman (Ed.), Handbook of medical imaging, processing and analysis (pp. 393–420). San Diego: Academic Press. Chap. 25. |
| [37] | Tschumperlé, D. (2006). Fast anisotropic smoothing of multi-valued images using curvature-preserving PDEs. International Journal of Computer Vision, 68, 65–82. · Zbl 1481.94031 · doi:10.1007/s11263-006-5631-z |
| [38] | Tschumperle, D., & Besserer, B. (2004). High quality deinterlacing using inpainting and shutter-model directed temporal interpolation. In Int. conf. on computer vision and graphics (pp. 301–307). |
| [39] | Tschumperlé, D., & Deriche, R. (2005). Vector valued image regularization with PDEs: a common framework for different applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 506–517. · doi:10.1109/TPAMI.2005.87 |
| [40] | Wang, Z., Glazowski, C. E., & Zavislan, J. M. (2007). Modulation transfer function measurement of scanning reflectance microscopes. Journal of Biomedical Optics, 12(5), 051802. |
| [41] | Weickert, J., & Welk, M. (2006). Tensor field interpolation with PDEs. In H. Hagen, & J. Weickert (Eds.), Mathematics and visualization. Visualization and processing of tensor fields (pp. 315–325). Berlin: Springer. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
