Abstract
In this paper, we propose a new image blind deblurring model, based on a novel low-rank prior. As the low-rank prior, we employ the weighted Schatten p-norm minimization (WSNM), which can represent both the sparsity and self-similarity of the image structure more accurately. In addition, the L0-regularized gradient prior is introduced into our model, to extract significant edges quickly and effectively. Moreover, the WSNM prior can effectively eliminate harmful details and maintain dominant edges, to generate sharper intermediate images, which is beneficial for blur kernel estimation. To optimize the model, an efficient optimization algorithm is developed by combining the half-quadratic splitting strategy with the generalized soft-thresholding algorithm. Extensive experiments have demonstrated the validity of the WSNM prior. Our flexible low-rank prior enables the proposed algorithm to achieve excellent results in various special scenarios, such as the deblurring of text, face, saturated, and noise-containing images. In addition, our method can be extended naturally to non-uniform deblurring. Quantitative and qualitative experimental evaluations indicate that the proposed algorithm is robust and performs favorably against state-of-the-art algorithms.
Similar content being viewed by others
References
E.J. Candes, X. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58(3), 1–37 (2011)
A. Chakrabarti, A neural approach to blind motion deblurring, in European Conference on Computer Vision (ECCV) (2016), pp. 221–235
L. Chen, F. Fang, T. Wang, G. Zhang, Blind image deblurring with local maximum gradient prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019), pp. 1742–1750
S. Cho, S. Lee, Fast motion deblurring. ACM Trans. Graph. 28(5), 1–8 (2009)
S. Cho, J. Wang, S. Lee, Handling outliers in non-blind image deconvolution, in International Conference on Computer Vision (ICCV) (2011), pp. 495–502
K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
J. Dong, J. Pan, Z. Su, Blur kernel estimation via salient edges and low rank prior for blind image deblurring. Sig. Process. Image Commun. 58, 134–145 (2017)
W. Dong, G. Shi, X. Li, Image deblurring with low-rank approximation structured sparse representation, in Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (2012), pp. 1–5
W. Dong, G. Shi, X. Li, Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans. Image Process. 22(2), 700–711 (2013)
W. Dong, L. Zhang, G. Shi, X. Wu, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. 20(7), 1838–1857 (2011)
R. Fergus, B. Singh, A. Hertzmann, S.T. Roweis, W.T. Freeman, Removing camera shake from a single photograph. ACM Trans. Graph. 25(3), 787–794 (2006)
S. Gu, Q. Xie, D. Meng, W. Zuo, X. Feng, L. Zhang, Weighted nuclear norm minimization and its applications to low level vision. Int. J. Comput. Vision 121(2), 183–208 (2017)
S. Gu, L. Zhang, W. Zuo, X. Feng, Weighted nuclear norm minimization with application to image denoising, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 2862–2869
Y. Guo, H. Ma, Image blind deblurring using an adaptive patch prior. Tsinghua Sci. Technol. 24(2), 238–248 (2019)
M. Hirsch, C.J. Schuler, S. Harmeling, B. Schölkopf, Fast removal of non-uniform camera shake, in International Conference on Computer Vision (ICCV) (2011), pp. 463–470
H. Ji, C. Liu, Z. Shen, Y. Xu, Robust video denoising using low rank matrix completion, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010), pp. 1791–1798
K.H. Jin, J.C. Ye, Annihilating filter-based low-rank hankel matrix approach for image inpainting. IEEE Trans. Image Process. 24(11), 3498–3511 (2015)
N. Joshi, R. Szeliski, D.J. Kriegman, PSF estimation using sharp edge prediction, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008), pp. 1–8
B.R. Kapuriya, D. Pradhan, R. Sharma, Detection and restoration of multi-directional motion blurred objects. SIViP 13(5), 1001–1010 (2019)
R. Köhler, M. Hirsch, B. Mohler, B. Schölkopf, S. Harmeling, Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database, in European Conference on Computer Vision (ECCV) (2012), pp. 27–40
D. Krishnan, T. Tay, R. Fergus, Blind deconvolution using a normalized sparsity measure, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011), pp. 233–240
O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, J. Matas, DeblurGAN: blind motion deblurring using conditional adversarial networks, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 8183–8192
W.-S. Lai, J.-B. Huang, Z. Hu, N. Ahuja, M.-H. Yang, A comparative study for single image blind deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 1701–1709
A. Levin, R. Fergus, F. Durand, W.T. Freeman, Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26(3), 70–78 (2007)
A. Levin, Y. Weiss, F. Durand, W.T. Freeman, Understanding and evaluating blind deconvolution algorithms, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 1964–1971
A. Levin, Y. Weiss, F. Durand, W.T. Freeman, Efficient marginal likelihood optimization in blind deconvolution, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011), pp. 2657–2664
J. Li, W. Lu, Blind image motion deblurring with L0-regularized priors. J. Vis. Commun. Image Represent. 40, 14–23 (2016)
L. Li, J. Pan, W.-S. Lai, C. Gao, N. Sang, M.-H. Yang, Blind image deblurring via deep discriminative priors. Int. J. Comput. Vision 127(8), 1025–1043 (2019)
M. Li, J. Liu, Z. Xiong, X. Sun, Z. Guo, MARLow: a joint multiplanar autoregressive and low-rank approach for image completion, in European Conference on Computer Vision (ECCV), (2016), pp. 819–834
L. Ma, L. Xu, T. Zeng, Low rank prior and total variation regularization for image deblurring. J. Sci. Comput. 70(3), 1336–1357 (2017)
J. Mairal, F. Bach, J. Ponce, G. Sapiro, A. Zisserman, Non-local sparse models for image restoration, in International Conference on Computer Vision (ICCV) (2009), pp. 2272–2279
T. Michaeli, M. Irani, Blind deblurring using internal patch recurrence, in European Conference on Computer Vision (ECCV) (2014), pp. 783–798
S. Nah, T. Kim, K. Lee, Deep multi-scale convolutional neural network for dynamic scene deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 3883–3891
F. Nie, H. Huang, C. Ding, Low-rank matrix recovery via efficient Schatten p-norm minimization, in AAAI Conference on Artificial Intelligence (AAAI) (2012), pp. 655–661
J. Pan, Z. Hu, Z. Su, M.-H. Yang, Deblurring text images via L0-regularized intensity and gradient prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), pp. 2901–2908
J. Pan, Z. Lin, Z. Su, M.-H. Yang, Robust kernel estimation with outliers handling for image deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 2800-2808
J. Pan, R. Liu, Z. Su, X. Gu, Kernel estimation from salient structure for robust motion deblurring. Sig. Process. Image Commun. 28(9), 1156–1170 (2013)
J. Pan, D. Sun, H. Pfister, M.-H. Yang, Blind image deblurring using dark channel prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 1628–1636
D. Perrone, R. Diethelm, P. Favaro, Blind deconvolution via lower-bounded logarithmic image priors, in International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR) (2015), pp. 112–125
W. Ren, X. Cao, J. Pan, X. Guo, W. Zuo, M.-H. Yang, Image deblurring via enhanced low rank prior. IEEE Trans. Image Process. 25(7), 3426–3437 (2016)
C.J. Schuler, M. Hirsch, S. Harmeling, B. Scholkopf, Learning to Deblur. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1439–1451 (2016)
Q. Shan, J. Jia, A. Agarwala, High-quality motion deblurring from a single image. ACM Trans. Graph. 27(3), 1–10 (2008)
L. Sun, S. Cho, J. Wang, J. Hays, Edge-based blur kernel estimation using patch priors, in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–8
Y. Tang, Y. Xue, Y. Chen, L. Zhou, Blind deblurring with sparse representation via external patch priors. Digit. Signal Proc. 78, 322–331 (2018)
X. Tao, H. Gao, X. Shen, J. Wang, J. Jia, Scale-recurrent network for deep image deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 8174–8182
O. Whyte, J. Sivic, A. Zisserman, Deblurring shaken and partially saturated images. Int. J. Comput. Vision 110(2), 185–201 (2014)
O. Whyte, J. Sivic, A. Zisserman, J. Ponce, Non-uniform deblurring for shaken images. Int. J. Comput. Vis. 98(2), 168–186 (2011)
Y. Xie, S. Gu, Y. Liu, W. Zuo, W. Zhang, L. Zhang, Weighted Schatten p-norm minimization for image denoising and background subtraction. IEEE Trans. Image Process. 25(10), 4842–4857 (2016)
L. Xu, J. Jia, Two-phase kernel estimation for robust motion deblurring, in European Conference on Computer Vision (ECCV) (2010), pp. 157–170
L. Xu, C. Lu, Y. Xu, J. Jia, Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30(6), 1–12 (2011)
L. Xu, S. Zheng, J. Jia, Unnatural L0 sparse representation for natural image deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013), pp. 1107–1114
Y. Yan, W. Ren, Y. Guo, R. Wang, X. Cao, Image deblurring via extreme channels prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6978–6986
Z. Zha, X. Zhang, Y. Wu, Q. Wang, X. Liu, L. Tang, X. Yuan, Non-convex weighted lp nuclear norm based ADMM framework for image restoration. Neurocomputing 311, 209–244 (2018)
J. Zhang, D. Zhao, W. Gao, Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014)
D. Zoran, Y. Weiss, From learning models of natural image patches to whole image restoration, in International Conference on Computer Vision (ICCV) (2011), pp. 479–486
W. Zuo, D. Meng, L. Zhang, X. Feng, D. Zhang, A generalized iterated shrinkage algorithm for non-convex sparse coding, in International Conference on Computer Vision (ICCV) (2013), pp. 217–224
Acknowledgements
We thank the anonymous reviewers for their valuable comments, which have helped to improve the quality of this paper. Additionally, we thank Liwen Bianji, Edanz Editing China (www.liwenbianji.cn/ac), for editing the English text of a draft of this manuscript.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
ZX contributed to conceptualization, methodology, and writing—original draft preparation; ZX and HC performed formal analysis and investigation and provided software; ZX, HC, and ZL performed writing—review and editing; ZL helped in supervision and project administration.
Corresponding authors
Ethics declarations
Conflicts of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xu, Z., Chen, H. & Li, Z. Blind Image Deblurring via the Weighted Schatten p-norm Minimization Prior. Circuits Syst Signal Process 39, 6191–6230 (2020). https://doi.org/10.1007/s00034-020-01457-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00034-020-01457-z