Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery. (English) Zbl 1441.94020
Summary: The volume of data grows with the advent of multiple types of remote sensing sensors and in order to extract the most useful information there is a need to combine the data gathered from the different sources. The widely used panchromatic and multispectral imageries in many applications offer decimetric and metric spatial resolution. However, the spectral resolution of these images is poor. Hyperspectral imaging has unique characteristics of providing very fine spectral resolution in a large number of bands with decametric spatial resolution and found to be highly useful for a wide span of application areas that requires high spectral resolution. The fusion of spectral and spatial information provides an effective way of enhancing the spatial quality of hyperspectral imagery as well as a method for preserving spectral quality. This fusion process is not a trivial task as always there has been a tradeoff between the preservation of spatial and spectral quality information as in the original sources of fusion. In this paper, a review on hyperspectral pansharpening and hyperspectral multispectral fusion based approaches has been reported. The widely adopted quantitative and qualitative performance measures to verify the fusion results are highlighted. In addition, the challenges in existing fusion techniques have also been discussed.
MSC:
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
Keywords:
hyperspectral image fusion; pansharpening; hyperspectral data; spectral preservation; spatial enhancementReferences:
| [1] | Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing, 40, 2300-2312. · doi:10.1109/TGRS.2002.803623 |
| [2] | Aiazzi, B., Baronti, S., & Selva, M. (2007). Improving component substitution pansharpening through multivariate regression of MS+Pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3230-3239. · doi:10.1109/TGRS.2007.901007 |
| [3] | Alparone, L., Aiazzi, B., Baronti, S. & Garzelli, A. (2003). Sharpening of very high resolution images with spectral distortion minimization. In Proceedings of IEEE international conference geoscience and remote sensing (IGARSS), pp. 458-460. |
| [4] | Amro, I., Mateos, J., Vega, M., Molina, R., & Katsaggelos, A. K. (2011). A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing, 79, 1-22. |
| [5] | Bieniarz, J., Cerra, D., Avbelj, J., Reinartz, P., & Miller, R. (2011). Hyperspectral image resolution enhancement based on spectral unmixing and information fusion. In ISPRS Hannover Workshop 2011: High-resolution earth imaging for geospatial information. |
| [6] | Chaudhuri, S., & Kotwal, K. (2013). Hyperspectral image fusion. New York: Springer. · doi:10.1007/978-1-4614-7470-8 |
| [7] | Chembe, C., Engels, J., Hahn, M., & Gülch, E. (2012). Pansharpening of hyperspectral images in urban areas. In Proceedings on international archives of the photogrammetry, remote sensing and spatial information sciences, XXII ISPRS Congress, XXXIX-B7, pp. 387-392. |
| [8] | Cheng, G., Han, J., Guo, L., Liu, Z., Shuhui, B., & Ren, J. (2015). Effective and efficient midlevel visual elements-oriented land-use classification using VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 53(8), 4238-4249. · doi:10.1109/TGRS.2015.2393857 |
| [9] | Cheng, G., Han, J., Guo, L., Qian, X., Zhou, P., Yao, X., et al. (2013). Object detection in remote sensing imagery using a discriminatively trained mixture model. ISPRS Journal of Photogrammetry and Remote Sensing, 85, 32-43. · doi:10.1016/j.isprsjprs.2013.08.001 |
| [10] | Cheng, G., Han, J., Zhou, P., & Guo, L. (2014). Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS Journal of Photogrammetry and Remote Sensing, 98, 119-132. · doi:10.1016/j.isprsjprs.2014.10.002 |
| [11] | Choi, M. (2006). A new intensity hue saturation fusion approach to image fusion with a tradeoff parameter. IEEE Transactions on Geoscience and Remote Sensing, 44(6), 1672-1682. · doi:10.1109/TGRS.2006.869923 |
| [12] | Choi, Y., Sharifahmadian, E., & Latifi, S. (2013). Performance analysis of contourlet based hyperspectral image fusion methods. International Journal on Information Theory, 2(1-2), 1-14. · doi:10.5121/ijit.2014.2401 |
| [13] | Do, M. N., & Vetterli, M. (2005). The Contourlet transform: An efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12), 2091-2106. · doi:10.1109/TIP.2005.859376 |
| [14] | Du, Q., Younan, N. H., King, R., & Shah, V. P. (2007). On the performance evaluation of pan-sharpening techniques. IEEE Geoscience and Remote Sensing Letters, 4(4), 518-522. · doi:10.1109/LGRS.2007.896328 |
| [15] | Eismann, M. T., & Hardie, R. C. (2005). Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 455-465. · doi:10.1109/TGRS.2004.837324 |
| [16] | Fauvel, M., Benediktsson, J. A., Chanussot, J., & Sveinsson, J. R. (2008). Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 46(11), 3804-3814. · doi:10.1109/TGRS.2008.922034 |
| [17] | Fasbneder, D., Radoux, J., & Bogaert, P. (2008). Bayesian data fusion for adaptable image pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 46(6), 1847-1857. · doi:10.1109/TGRS.2008.917131 |
| [18] | Fonseca, L., Namikawal, L., Castejon, E., Carvalho, L., Pinho, C., & Pagamisse, A. (2011). Image fusion for remote sensing applications. In Y. Zheng (Ed.), Image fusion and its applications. InTech. http://www.intechopen.com/books/image-fusion-and-its-applications/image-fusion-for-remote-sensing-applications. |
| [19] | Garzell, A., Aiazzi, B., Barronti, S., Selva, M., & Alparone, L. (2010). Hyperspectral image fusion. In Proceedings of the hyperspectral 2010 workshop, pp. 17-19. |
| [20] | Ghamisi, P., Mura, M. D., & Benediktsson, J. A. (2015). A survey on spectral spatial classification techniques based on attribute profiles. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2335-2353. · doi:10.1109/TGRS.2014.2358934 |
| [21] | González-Audícana, M., Saleta, J. L., Catalán, R. G., & García, R. (2004). Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 42(6), 1291-1299. · doi:10.1109/TGRS.2004.825593 |
| [22] | Gu, Y., Zhang, Y., & Zhang, J. (2008). Integration of spatial-spectral information for resolution enhancement in hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1347-1358. · doi:10.1109/TGRS.2008.917270 |
| [23] | Grohnfeldt, C., Burns, T. M., & Zhu, X. X. (2015). Dynamic dictionary learning strategies for sparse representation based hyperspectral image enhancement. http://www.researchgate.net/publication/277279536. |
| [24] | Han, S. S., Li, H. T., & Gu, H. Y. (2008). The study on image fusion for high spatial resolution remote sensing images. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 37, 1159-1163. |
| [25] | Hardie, R. C., Eismann, M. T., & Wilson, G. L. (2004). MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Transactions on Image Processing, 13(9), 1174-1184. · doi:10.1109/TIP.2004.829779 |
| [26] | Hoyer, P. O. (2002). Non negative sparse coding. In Proceedings of IEEE workshop neural network signal processing, pp. 557-565. |
| [27] | Hsu, S. M., & Burke, H-h K. (2003). Multisensor fusion with hyperspectral imaging data: Detection and classification. Lincoln Laboratory Journal, 14(1), 145-159. |
| [28] | Idrees, M. O., Saiedi, V., Pradhan, I., & Shafri, H. Z. M. (2015). Maximising urban features extraction from multi-sensor data with Dempster-Shafer theory and HSI data fusion techniques. Asian Journal of Applied Sciences, 3(2), 218-228. |
| [29] | Kaplan, N. H., Erer, I., & Elibol, F. (2012). Fusion of multispectral and panchromatic images by combining bilateral filter and HIS transform. In 20th European signal processing conference, pp. 2501-2505. |
| [30] | Khan, M. M., Chanussot, J., Condat, L., & Montanvert, A. (2008). Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geoscience and Remote Sensing Letters, 5(1), 98-102. · doi:10.1109/LGRS.2007.909934 |
| [31] | Khandelwal, A., & Rajan, K. S. (2011). Hyperspectral image enhancement based on sensor simulation and vector decomposition. In 2011 proceedings of the 14th international conference on information fusion (FUSION), pp. 1-6. |
| [32] | Kotwal, K., & Chaudhuri, S. (2010). A fast approach for fusion of hyperspectral images through redundancy elimination. In Proceedings of the seventh Indian conference on computer vision, graphics and Image processing, pp. 506-511. doi:10.1145/1924559.1924627. |
| [33] | Laben, C., & Brower, B. (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. U.S. Patent No. 6,011,875. Washington, DC: U.S. Patent and Trademark Office. |
| [34] | Lanaras, L., Baltsavias, E., & Schindler, K. (2015a). Advances in hyperspectral and multispectral image fusion and spectral unmixing. In The international archives of photogrammetry, remote sensing and spatial information sciences, Vol. XL-3, pp. 451-458. doi:10.5194/isprsarchives-XL-3-W3-451-2015. |
| [35] | Lanaras, C., Baltsavias, E., & Schindler, K. (2015b). Hyperspectral image fusion. In 36th Asian conference on remote sensing (ACRS), Philippines. |
| [36] | Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non negative matrix factorization. Nature, 401, 788-791. · Zbl 1369.68285 · doi:10.1038/44565 |
| [37] | Liao, W., Pizurica, A., Bellens, R., Gautama, S., & Philips, W. (2015). Generalized graph-based fusion of hyperspectral and LiDAR data using morphological features. IEEE Geoscience and Remote Sensing Letters, 12(3), 552-556. · doi:10.1109/LGRS.2014.2350263 |
| [38] | Licciardi, G., Khan, M. M., Chanussot, J., Montanvert, A., Condat, L., & Jutten, C. (2012). Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction. EURASIP Journal on Advances in Signal processing, 1, 1-17. |
| [39] | Licciardi, G., Veganzones, M. A., Vivone, G., Loncan, L., & Chanussot, J. (2015). Impact of hybrid pansharpening approaches applied to hyperspectral images. https://hal.archives-ouverts.fr/hal-0121191. |
| [40] | Li, H., Manjunath, B. S., & Mitra, S. K. (1994). Multi-sensor image fusion using the wavelet transforms. IEEE Transactions in Signal Processing, 2012(207), 51-55. |
| [41] | Lin, H., Zhang, A., & Yang, S. (2015). Comparison of several hyperspectral image fusion methods for visualization (pp. 646-651). In International conference on advances in mechanical engineering and industrial informatics: Atlantis Press. |
| [42] | Liu, J. G. (2000). Smoothing filter based intensity modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, 21(18), 3461-3472. · doi:10.1080/014311600750037499 |
| [43] | Loncan, L., Almeida, L. B., Bioucas-Dias, J. M., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., et al. (2015). Hyperspectral pansharpening: A review. arXiv preprint arXiv:1504.04531. |
| [44] | Matsuoka, M. (2012). The influence of spectral wavelength on the quality of pansharpened image simulated using hyperspectral data. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 1, 297-302. · doi:10.5194/isprsannals-I-7-297-2012 |
| [45] | Mianji, F. A., Zhang, Y., & Babakhani, A. (2009a). Optimum method selection for resolution enhancement of hyperspectral imagery. Information Technology Journal, 8(3), 263-274. · doi:10.3923/itj.2009.263.274 |
| [46] | Mianji, F. A., Zhang, Y., & Babakhani, A. (2010). Key information retrieval in hyperspectral imagery through spatial-spectral data fusion. Radio Engineering, 19(4), 734-744. |
| [47] | Mianji, F. A., Zhang, Y., Gu, Y., & Babakhani, A. (2009b). Spatial-spectral data fusion for resolution enhancement of hyperspectral imagery. IEEE IGRASS, III, 1011-1014. |
| [48] | Mianji, F. A., Zhang, Y., & Sulehria, H. K. (2008). Super resolution challenges in hyperspectral Imagery. Information Technology Journal, 7(7), 1030-1036. · doi:10.3923/itj.2008.1030.1036 |
| [49] | Moeller, M., Wittman, T., & Bertozzi, A. L. (2009). A variational approach to hyperspectral Image fusion. In SPIE defense, security, and sensing, International Society for Optics and Photonics, pp. 73341E-73341E. |
| [50] | Naidu, V. P. S., & Raol, J. R. (2008). Pixel-level image fusion using wavelets and principal component analysis. Defense Science Journal, 58(3), 338-352. · doi:10.14429/dsj.58.1653 |
| [51] | Nguyen, M. Q., Atkinson, P. M., & Lewis, H. G. (2006). Superresolution mapping using a Hopfield neural network with fused images. IEEE Transactions on Geoscience and Remote Sensing, 44(3), 736-749. · doi:10.1109/TGRS.2005.861752 |
| [52] | Pande, H., & Tiwari, P. S. (2013). High-resolution and hyperspectral data fusion for classification. In Q. Miao (Ed.), New advances in image fusion. InTech. http://www.intechopen.com/books/new-advances-in-image-fusion/high-resolution-and-hyperspectral-data-fusion-for-classification. |
| [53] | Rahmani, S., Strait, M., Merkurjev, D., Moeller, M., & Wittman, T. (2010). An adaptive IHS pan-sharpening method. IEEE Geoscience and Remote Sensing Letters, 7(4), 746-750. · doi:10.1109/LGRS.2010.2046715 |
| [54] | Ranchin, T., & Wald, L. (2000). Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation. Photogrammetric Engineering and Remote Sensing, 66(1), 49-61. |
| [55] | Sadjadi, F. (2005). Comparative image fusion analysis. In IEEE Computer Society conference on computer vision and pattern recognition-workshops (CVPR), p. 8, IEEE. |
| [56] | Villa, A., Chanussot, J., Benediktsson, J. A., & Jutten, C. (2011). Spectral unmixing for the classification of hyperspectral images at a finer spatial resolution. IEEE Journal of Selected Topics in Signal Processing, 5(3), 521-533. · doi:10.1109/JSTSP.2010.2096798 |
| [57] | Vivone, G., Alpharone, L., Chanussot, J., Mura, M. D., Garzelli, A., Giorgio, A., et al. (2015). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565-2586. · doi:10.1109/TGRS.2014.2361734 |
| [58] | Vivone, G., Restaino, R., Licciardi, G., Dalla Mura, M., & Chanussot, J. (2014). Multiresolution analysis and component substitution techniques for hyperspectral pansharpening. In Geoscience and remote sensing symposium (IGARSS), 2014 IEEE international, pp. 2649-2652. IEEE. |
| [59] | Wang, Q., Shen, Y., Zhang, Y., & Zhang, J. Q. (2003). A quantitative method for evaluating the performances of hyperspectral image fusion. IEEE Transactions on Instrumentation and Measurement, 52(4), 1041-1047. · doi:10.1109/TIM.2003.814821 |
| [60] | Wei, Q., Bioucas Dias, J. M., Dobigeon, N., & Tourneret, J. Y. (2015a). Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3658-3668. · doi:10.1109/TGRS.2014.2381272 |
| [61] | Wei, Q., Dobigeon, N., & Tourneret, J.-Y. (2014). Bayesian fusion of hyperspectral and multispectral images. In Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP), pp. 3176-3180. |
| [62] | Wei, Q., Dobigeon, N., & Tourneret, J.-Y. (2015b). Bayesian fusion of multispectral and hyperspectral images using a block coordinate descent method. In Proceedings of IEEE GRSS workshop hyperspectral image signal process evolution in remote sensing (WHISPERS), Tokyo, Japan. |
| [63] | Yokoya, N., Yairi, T., & Iwasaki, A. (2011). Hyperspectral, multispectral, and panchromatic data fusion based on coupled non-negative matrix factorization. In 2011 3rd workshop on hyperspectral image and signal processing: Evolution in remote sensing (WHISPERS), pp. 1-4. IEEE. |
| [64] | Zhang, Y. (2008). Understanding image fusion. Photogrammetric Engineering and Remote Sensing, 70(6), 657-661. |
| [65] | Zhang, Z., & Shi, Z. (2013). Nonnegative matrix factorization-based hyperspectral and panchromatic image fusion. Neural Computing and Applications, 23(3-4), 895-905. · doi:10.1007/s00521-012-1014-2 |
| [66] | Zhang, H., Zhang, L., & Shen, H. (2012). A super-resolution reconstruction algorithm for hyperspectral images. Signal Processing, 92, 2082-2096. · doi:10.1016/j.sigpro.2012.01.020 |
| [67] | Zhao, Y.-Q., Zhang, L., & Kong, S. G. (2010). Band subset based clustering and fusion for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 49(2), 747-756. · doi:10.1109/TGRS.2010.2059707 |
| [68] | Zheng, Y. (2011). Image fusion and its applications. Rijeka: Intech. · doi:10.5772/691 |
| [69] | Zhukov, O., Oertel, D., Lanzl, F., & Reinhackel, G. (1999). Unmixing based multisensory multiresolution image fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1212-1226. · doi:10.1109/36.763276 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
