A nonlocal method for image shadow removal. (English) Zbl 1524.94006
Summary: This paper proposes a new model for image shadow removal. The model reformulates a recent osmosis model with nonlocal differential operators. This allows to benefit from distant pixels similarities and thus improves restoration results. Some properties of this model are established and discussed, making it suitable for our application. Experimental results show that the nonlocal model obtained very good qualitative and quantitative results compared with state-of-the-art techniques.
MSC:
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
| 35A15 | Variational methods applied to PDEs |
| 68T45 | Machine vision and scene understanding |
| 68U10 | Computing methodologies for image processing |
| 70H20 | Hamilton-Jacobi equations in mechanics |
References:
| [1] | Barrow, H.; Tenenbaum, J.; Hanson, A.; Riseman, E., Recovering intrinsic scene characteristics, Comput. Vis. Syst., 2, 3-26, 2 (1978) |
| [2] | Finlayson, G.; Hordley, S.; Drew, M. S., Removing shadows from images using retinex, (Color and Imaging Conference, vol. 1 (2002)), 73-79 · Zbl 1039.68627 |
| [3] | Arbel, E.; Hel-Or, H., Shadow removal using intensity surfaces and texture anchor points, IEEE Trans. Pattern Anal. Mach. Intell., 33, 6, 1202-1216 (2010) |
| [4] | Finlayson, G. D.; Hordley, S. D.; Drew, M. S., Removing shadows from images, (European Conference on Computer Vision (2002), Springer), 823-836 · Zbl 1039.68627 |
| [5] | Fredembach, C.; Finlayson, G., Hamiltonian path-based shadow removal, (Proc. of the 16th British Machine Vision Conference (BMVC), vol. 2 (2005)), 502-511 |
| [6] | Mohan, A.; Tumblin, J.; Choudhury, P., Editing soft shadows in a digital photograph, IEEE Comput. Graph. Appl., 27, 2, 23-31 (2007) |
| [7] | Yang, Q.; Tan, K.-H.; Ahuja, N., Shadow removal using bilateral filtering, IEEE Trans. Image Process., 21, 10, 4361-4368 (2012) · Zbl 1373.94458 |
| [8] | Finlayson, G.; Drew, M.; Lu, C., Entropy minimization for shadow removal, Int. J. Comput. Vis., 85, 35-37 (2009) |
| [9] | Finlayson, G. D.; Hordley, S. D.; Lu, C.; Drew, M. S., On the removal of shadows from images, IEEE Trans. Pattern Anal. Mach. Intell., 28, 1, 59-68 (2006) |
| [10] | Amin, B.; Riaz, M. M.; Ghafoor, A., Automatic shadow detection and removal using image matting, Signal Process., 170 (2020) |
| [11] | Shor, Y.; Lischinski, D., The shadow meets the mask: pyramid-based shadow removal, Comput. Graph. Forum, 27 (2008) |
| [12] | Wu, T.-P.; Tang, C.-K., A bayesian approach for shadow extraction from a single image, (Tenth IEEE International Conference on Computer Vision (ICCV’05), vol. 1 (2005), IEEE), 480-487 |
| [13] | Wu, T.-P.; Tang, C.-K.; Brown, M. S.; Shum, H.-Y., Natural shadow matting, ACM Trans. Graph., 26, 2 (2007), 8-es |
| [14] | Khan, S. H.; Bennamoun, M.; Sohel, F.; Togneri, R., Automatic shadow detection and removal from a single image, IEEE Trans. Pattern Anal. Mach. Intell., 38, 3, 431-446 (2015) |
| [15] | Gong, H.; Cosker, D., Interactive shadow removal and ground truth for variable scene categories, (BMVC, Citeseer (2014)), 1-11 |
| [16] | Gryka, M.; Terry, M.; Brostow, G. J., Learning to remove soft shadows, ACM Trans. Graph., 34, 5, 1-15 (2015) |
| [17] | Guo, R.; Dai, Q.; Hoiem, D., Paired regions for shadow detection and removal, IEEE Trans. Pattern Anal. Mach. Intell., 35, 12, 2956-2967 (2012) |
| [18] | Vicente, T. F.Y.; Hoai, M.; Samaras, D., Leave-one-out kernel optimization for shadow detection and removal, IEEE Trans. Pattern Anal. Mach. Intell., 40, 3, 682-695 (2017) |
| [19] | Liu, F.; Gleicher, M., Texture-consistent shadow removal, (Proceedings of the European Conference on Computer Vision (2008)), 437-450 |
| [20] | Qu, L.; Tian, J.; He, S.; Tang, Y.; Lau, R. W., Deshadownet: a multi-context embedding deep network for shadow removal, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)), 4067-4075 |
| [21] | Wang, J.; Li, X.; Yang, J., Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)), 1788-1797 |
| [22] | Vicente, T. F.Y.; Hou, L.; Yu, C.-P.; Hoai, M.; Samaras, D., Large-scale training of shadow detectors with noisily-annotated shadow examples, (European Conference on Computer Vision (2016), Springer), 816-832 |
| [23] | Hu, X.; Zhu, L.; Fu, C.-W.; Qin, J.; Heng, P.-A., Direction-aware spatial context features for shadow detection, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)), 7454-7462 |
| [24] | Weickert, J., Anisotropic Diffusion in Image Processing, vol. 1 (1998), Teubner: Teubner Stuttgart · Zbl 0886.68131 |
| [25] | Le, H.; Samaras, D., Shadow removal via shadow image decomposition, (Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)), 8578-8587 |
| [26] | Weickert, J.; Hagenburg, K.; Breuß, M.; Vogel, O., Linear osmosis models for visual computing, (International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (2013), Springer), 26-39 |
| [27] | Parisotto, S.; Calatroni, L.; Caliari, M.; Schönlieb, C.-B.; Weickert, J., Anisotropic osmosis filtering for shadow removal in images, Inverse Probl., 35, 5, Article 054001 pp. (2019) · Zbl 1448.94030 |
| [28] | Hagenburg, K.; Breuß, M.; Weickert, J.; Vogel, O., Novel schemes for hyperbolic pdes using osmosis filters from visual computing, (International Conference on Scale Space and Variational Methods in Computer Vision (2011), Springer), 532-543 |
| [29] | Vogel, O.; Hagenburg, K.; Weickert, J.; Setzer, S., A fully discrete theory for linear osmosis filtering, (International Conference on Scale Space and Variational Methods in Computer Vision (2013), Springer), 368-379 |
| [30] | d’Autume, M.; Morel, J.-M.; Meinhardt-Llopis, E., A flexible solution to the osmosis equation for seamless cloning and shadow removal, (2018 25th IEEE International Conference on Image Processing (ICIP) (2018), IEEE), 2147-2151 |
| [31] | Parisotto, S.; Calatroni, L.; Bugeau, A.; Papadakis, N.; Schönlieb, C.-B., Variational osmosis for non-linear image fusion, IEEE Trans. Image Process., 29, 5507-5516 (2020) |
| [32] | Parisotto, S.; Calatroni, L.; Daffara, C., Digital cultural heritage imaging via osmosis filtering, (International Conference on Image and Signal Processing (2018), Springer), 407-415 |
| [33] | Calatroni, L.; d’Autume, M.; Hocking, R.; Panayotova, S.; Parisotto, S.; Ricciardi, P.; Schönlieb, C.-B., Unveiling the invisible: mathematical methods for restoring and interpreting illuminated manuscripts, Herit. Sci., 6, 1, 1-21 (2018) |
| [34] | Gilboa, G.; Osher, S., Nonlocal operators with applications to image processing, Multiscale Model. Simul., 7, 3, 1005-1028 (2009) · Zbl 1181.35006 |
| [35] | Franken, E.; van Almsick, M.; Rongen, P.; Florack, L.; Romeny, B. H., An efficient method for tensor voting using steerable filters, (European Conference on Computer Vision (2006), Springer), 228-240 |
| [36] | Guo, R.; Dai, Q.; Hoiem, D., Paired regions for shadow detection and removal, IEEE Trans. Pattern Anal. Mach. Intell., 35, 12, 2956-2967 (2012) |
| [37] | Yang, Q.; Tan, K.-H.; Ahuja, N., Shadow removal using bilateral filtering, IEEE Trans. Image Process., 21, 10, 4361-4368 (2012) · Zbl 1373.94458 |
| [38] | Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P., Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13, 4, 600-612 (2004) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
