×

Neural blind deconvolution with Poisson data. (English) Zbl 07672404

Summary: Blind Deconvolution problem is a challenging task in several scientific imaging domains, such as Microscopy, Medicine and Astronomy. The Point Spread Function inducing the blur effect on the acquired image can be solely approximately known, or just a mathematical model may be available. Blind deconvolution aims to reconstruct the image when only the recorded data is available. In the last years, among the standard variational approaches, Deep Learning techniques have gained interest thanks to their impressive performances. The Deep Image Prior framework has been employed for solving this task, giving rise to the so-called neural blind deconvolution (NBD), where the unknown blur and image are estimated via two different neural networks. In this paper, we consider microscopy images, where the predominant noise is of Poisson type, hence signal-dependent: this leads to consider the generalized Kullback-Leibler as loss function and to couple it with regularization terms on both the blur operator and on the image. Furthermore, we propose to modify the standard NBD formulation problem, by including for the blur kernel an upper bound which depends on the optical instrument. A numerical solution is obtained by an alternating Proximal Gradient Descent-Ascent procedure, which results in the Double Deep Image Prior for Poisson noise algorithm. We evaluate the proposed strategy on both synthetic and real-world images, achieving promising results and proving that the correct choice of the loss and regularization functions strongly depends on the application at hand.

MSC:

68Txx Artificial intelligence
65-XX Numerical analysis

References:

[1] Bertero, M.; Boccacci, P.; Ruggiero, V., Inverse Imaging With Poisson Data (2018), Bristol: IOP Publish, Bristol · doi:10.1088/2053-2563/aae109
[2] Campisi, P.; Egiazarian, K., Blind Image Deconvolution: Theory and Applications (2016), Boca Raton, FL: CRC Press, Boca Raton, FL
[3] Levin, A.; Weiss, Y.; Durand, F.; Freeman, W. T., Understanding and evaluating blind deconvolution algorithm, pp 1964-71 (2009)
[4] van Kempen, G. M P.; van Vliet, L. J.; Verveer, P. J.; van der Voort, H. T M., A quantitative comparison of image restoration methods for confocal microscopy, J. Microsc., 185, 354-65 (1997) · doi:10.1046/j.1365-2818.1997.d01-629.x
[5] van Kempen, G. M P.; van Vliet, L. J., Background estimation in nonlinear image restoration, J. Opt. Soc. Am. A, 17, 425-33 (2000) · doi:10.1364/JOSAA.17.000425
[6] Bertsekas, D. P., Nonlinear Programming (1999), Belmont: Athena Scientific, Belmont · Zbl 1015.90077
[7] Grippo, L.; Sciandrone, M., On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26, 127-36 (2000) · Zbl 0955.90128 · doi:10.1016/S0167-6377(99)00074-7
[8] Bonettini, S., Inexact block coordinate descent methods with application to the nonnegative matrix factorization, IMA J. Num. Anal., 31, 1431-52 (2011) · Zbl 1235.65061 · doi:10.1093/imanum/drq024
[9] Prato, M.; La Camera, A.; Bonettini, S.; Bertero, M., A convergent blind deconvolution method for post-adaptive-optics astronomical imaging, Inverse Problems, 29 (2013) · Zbl 1286.68477 · doi:10.1088/0266-5611/29/6/065017
[10] Zuo, W.; Ren, D.; Zhang, D.; Gu, S.; Zhang, L., Learning iteration-wise generalized shrinkage-thresholding operators for blind deconvolution, IEEE Trans. Image Process., 25, 1751-64 (2016) · Zbl 1408.94856 · doi:10.1109/TIP.2016.2531905
[11] Pan, J.; Sun, D.; Pfister, H.; Yang, M. H., Deblurring images via dark channel prior, IEEE Trans. Pattern Anal. Mach. Intell., 40, 2315-28 (2018) · doi:10.1109/TPAMI.2017.2753804
[12] Zhang, M.; Fang, Y.; Ni, G.; Zeng, T., Pixel screening based intermediate correction for blind deblurring, pp 5892-900 (2022)
[13] Liu, J.; Yan, M.; Zeng, T., Surface-aware blind image deblurring, IEEE Trans. Pattern Anal. Mach. Intell., 43, 1041-55 (2021) · doi:10.1109/TPAMI.2019.2941472
[14] Nah, S.; Hyun Kim, T.; Mu Lee, K., Deep multi-scale convolutional neural network for dynamic scene deblurring, pp 257-65 (2017)
[15] Tao, X.; Gao, H.; Shen, X.; Wang, J.; Jia, J., Scale-recurrent network for deep image deblurring, pp 8174-82 (2018)
[16] Zhang, J.; Pan, J.; Ren, J.; Song, Y.; Bao, L.; Lau, R. W H.; Yang, M., Dynamic scene deblurring using spatially variant recurrent neural networks, pp 2521-9 (2018)
[17] Gao, H.; Tao, X.; Shen, X.; Jia, J., Dynamic scene deblurring with parameter selective sharing and nested skip connections, pp 3843-51 (2019)
[18] Koh, J.; Lee, J.; Yoon, S., Single-image deblurring with neural networks: a comparative survey, Comput. Vis. Image Underst., 203 (2021) · doi:10.1016/j.cviu.2020.103134
[19] Tran, P.; Tran, A.; Phung, Q.; Hoai, M., Explore image deblurring via encoded blur kernel space, p 2021 (2021)
[20] Asim, M.; Shamshad, F.; Ahmed, A., Blind image deconvolution using deep generative priors, IEEE Trans. Comput. Imaging, 6, 1493-506 (2020) · doi:10.1109/TCI.2020.3032671
[21] Ren, D.; Zhang, K.; Wang, Q.; Hu, Q.; Zuo, W., Neural blind deconvolution using deep priors, pp 3338-47 (2020)
[22] Ulyanov, D.; Vedaldi, A.; Lempitsky, V.; Prior, D. I., pp 9446-54 (2018)
[23] Ulyanov, D.; Vedaldi, A.; Lempitsky, V., Deep Image Prior, Int. J. Comput. Vis., 128, 1867-88 (2020) · doi:10.1007/s11263-020-01303-4
[24] Gandelsman, Y.; Shocher, A.; Irani, M., Double-DIP: unsupervised image decomposition via coupled deepimage- priors (2019)
[25] Wang, Z.; Wang, Z.; Li, Q.; Bilen, H., Image deconvolution with deep image and kernel priors (2019)
[26] Kotera, J.; Šroubek, F.; Šmídl, W., Improving neural blind deconvolution, pp 1954-8 (2021)
[27] Zhuang, Z.; Li, T.; Wang, H.; Sun, J., Blind image deblurring with unknown kernel size and substantial noise (2022)
[28] Cascarano, P.; Franchini, G.; Kobler, E.; Porta, F.; Sebastiani, A., Constrained and unconstrained deep image prior optimization models with automatic regularization, Comput. Optim. Appl., 84, 1-25 (2022) · Zbl 1507.65104 · doi:10.1007/s10589-022-00392-w
[29] Boţ, R. I.; Böhm, A., Alternating proximal-gradient steps for (stochastic) nonconvex-concave minimax problems (2022)
[30] Chen, Z.; Zhou, Y.; Xu, T.; Liang, Y., Proximal gradient descent-ascent: variable convergence under kl geometry (2021)
[31] Wang, H.; Li, T.; Zhuang, Z.; Chen, T.; Liang, H.; Sun, J., Early stopping for deep image prior, CoRR (2021)
[32] Bertero, M.; Boccacci, P.; Talenti, G.; Zanella, R.; Zanni, L., A discrepancy principle for Poisson data, Inverse Problems, 26 (2010) · Zbl 1200.62067 · doi:10.1088/0266-5611/26/10/105004
[33] Bauschke, H. H.; Combettes, P. L., Convex Analysis and Monotone Operator Theory in Hilbert Spaces (2011), New York: Springer Science & Business Media, New York · Zbl 1218.47001
[34] Pardalos, P. M.; Kovoor, N., An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds, Math. Program., 46, 321-8 (1990) · Zbl 0711.90061 · doi:10.1007/BF01585748
[35] Dai, Y. H.; Fletcher, R., New algorithms for singly linearly constrained quadratic programming problems subject to lower and upper bounds, Math. Program., 106, 403-21 (2006) · Zbl 1134.90030 · doi:10.1007/s10107-005-0595-2
[36] Kiwiel, K. C., Breakpoint searching algorithms for the continuous quadratic knapsack problem, Math. Program., 112, 473-91 (2008) · Zbl 1190.90121 · doi:10.1007/s10107-006-0050-z
[37] Chen, W. T.; Zhu, A. Y.; Khorasaninejad, M.; Shi, Z.; Sanjeev, V.; Capasso, F., Immersion meta-lenses at visible wavelengths for nanoscale imaging, Nano Lett., 17, 3188-94 (2017) · doi:10.1021/acs.nanolett.7b00717
[38] Ashida, Y.; Honma, Y.; Miura, N.; Shibuya, T.; Kikuchi, H.; Tamada, Y.; Kamei, Y.; Matsuda, A.; Hattori, M., Imaging performance of microscopy adaptive-optics system using scene-based wavefront sensing, J. Biomed. Opt., 25 (2020) · doi:10.1117/1.JBO.25.12.123707
[39] Sitzmann, V.; Martel, J. N P.; Bergman, A. W.; Lindell, D. B.; Wetzstein, G., Implicit neural representations with periodic activation functions, CoRR (2020)
[40] Cascarano, P.; Franchini, G.; Porta, F.; Sebastiani, A., On the First-Order Optimization Methods in Deep Image Prior, J. Verif. Valid. Uncertain. Quantif., 7 (2023) · doi:10.1115/1.4056470
[41] Borgia, G. C.; Brown, R. J S.; Fantazzini, P., Uniform-penalty inversion of multiexponential decay data, J. Magn. Reson., 132, 65-77 (1998) · doi:10.1006/jmre.1998.1387
[42] Grasmair, M., Locally adaptive total variation regularization, Scale Space and Variational Methods in Computer Vision (Lecture Notes in Computer Science), pp 331-42 (2009), Berlin: Springer, Berlin · Zbl 1233.68026
[43] Bortolotti, V.; Brown, R.; Fantazzini, P.; Landi, G.; Zama, F., Uniform penalty inversion of two-dimensional NMR relaxation data, Inverse Problems, 33 (2016) · Zbl 1361.65098 · doi:10.1088/1361-6420/33/1/015003
[44] Zanella, R.; Boccacci, P.; Zanni, L.; Bertero, M., Efficient gradient projection methods for edge-preserving removal of poisson noise, Inverse Problems, 25 (2009) · Zbl 1163.65042 · doi:10.1088/0266-5611/25/4/045010
[45] Zanella, R.; Boccacci, P.; Zanni, L.; Bertero, M., Corrigendum: efficient gradient projection methods for edge-preserving removal of poisson noise, Inverse Problems, 29 (2013) · Zbl 1284.65071 · doi:10.1088/0266-5611/29/11/119501
[46] Willett, R. M.; Nowak, R. D., Platelets: a multiscale approach for recovering edges and surfaces in photon limited medical imaging, IEEE Trans. Med. Imaging, 22, 332-50 (2003) · doi:10.1109/TMI.2003.809622
[47] Benfenati, A., upU-Net approaches for background emission removal in fluorescence microscopy, J. Imaging, 8, 142 (2022) · doi:10.3390/jimaging8050142
[48] Kirshner, H.; Aguet, F.; Sage, D.; Unser, M., 3-D PSF fitting for fluorescence microscopy: Implementation and localization application, J. Microsc., 249, 13-25 (2013) · doi:10.1111/j.1365-2818.2012.03675.x
[49] Sage, D.; Donati, L.; Soulez, F.; Fortun, D.; Schmit, G.; Seitz, A.; Guiet, R.; Vonesch, C.; Unser, M., DeconvolutionLab2: an open-source software for deconvolution microscopy, Methods Image Process. Biol., 115, 28-41 (2017) · doi:10.1016/j.ymeth.2016.12.015
[50] Sage, D., Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software, Nat. Methods Tech. Life Sci. Chem., 16, 387-95 (2019) · doi:10.1038/s41592-019-0364-4
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.