High spectral quality pansharpening approach based on MTF-matched filter banks. (English) Zbl 1441.94039
Summary: Pansharpening consists in merging a low-resolution multispectral image (MS) with a high spatial resolution panchromatic image (PAN) to produce a high resolution pansharpened MS image. It consists in enhancing spatially the low-resolution MS image by injecting the missing details provided by the high-resolution PAN image. In this paper, we propose a novel pansharpening approach based on decomposition/reconstruction processing using low-pass and high-pass filter banks. On the one hand, the low-pass approximation (taking into account the imaging system modulation transfer function MTF) of the pansharpened MS image is assumed to be equal to the original MS image in order to preserve the spectral quality. On the other hand, the high-pass filter allowing us to extract the high-frequency PAN details is designed as complementary filter to the low-pass one in order to provide perfect reconstruction in the ideal case. Quantitative assessment performed on reduced and full-resolution images are used to validate the proposed technique and compare it to state-of-art. Experimental results using Pléaides and GeoEye-1 data show that our proposed fusion schema outperforms the pre-existing methods visually as well as quantitatively.
MSC:
| 94A12 | Signal theory (characterization, reconstruction, filtering, etc.) |
Keywords:
pansharpening; modulation transfer function (MTF); spectral quality; filter banks; image reconstructionReferences:
| [1] | Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing, 40(10), 2300-2312. · doi:10.1109/TGRS.2002.803623 |
| [2] | Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2006). MTF-tailored multiscale fusion of high-resolution MS and pan imagery. Photogrammetric Engineering and Remote Sensing, 72(5), 591-596. |
| [3] | Aiazzi, B., Baronti, S., & Selva, M. (2007). Improving component substitution pansharpening through multivariate regression of MS+ Pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3230-3239. · doi:10.1109/TGRS.2007.901007 |
| [4] | Aiazzi, B., Alparone, S. Land Baronti, Garzelli, A., & Selva, M. (2011). Twenty-five years of pansharpening: A critical review and new developments. In Signal and image processing for remote sensing (pp. 533-548). Boca Raton, FL: Taylor and Francis Books. |
| [5] | Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2012). Advantages of Laplacian pyramids over “à trous” wavelet transforms for pansharpening of multispectral images (Vol. 8537, pp. 853704-853704-10). |
| [6] | Alparone, L., Baronti, S., Garzelli, A., & Nencini, F. (2004). A global quality measurement of pan-sharpened multispectral imagery. IEEE Geoscience and Remote Sensing Letters, 1, 313-317. · doi:10.1109/LGRS.2004.836784 |
| [7] | Alparone, L., Wald, L., Chanussot, J., Thomas, C., Gamba, P., & Bruce, L. (2007). Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3012-3021. · doi:10.1109/TGRS.2007.904923 |
| [8] | Alparone, L., Aiazzi, B., Baronti, S., Garzelli, A., Nencini, F., & Selva, M. (2008). Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering and Remote Sensing, 74(2), 193-200. · doi:10.14358/PERS.74.2.193 |
| [9] | Amro, I., Mateos, J., Vega, M., Molina, R., & Katsaggelos, A. K. (2011). A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing, 2011, 79. · doi:10.1186/1687-6180-2011-79 |
| [10] | Baronti, S., Aiazzi, B., Selva, M., Garzelli, A., & Alparone, L. (2011). A theoretical analysis of the effects of aliasing and misregistration on pansharpened imagery. IEEE Journal of Selected Topics in Signal Processing, 5, 446-453. · doi:10.1109/JSTSP.2011.2104938 |
| [11] | Bijaoui, A., Murtagh, F., & Starck, J. (1994). Restauration des Images Multi-Echelles par l’Algorithme a Trous. Traitement du Signal, 11(3), 229-243. · Zbl 0939.68917 |
| [12] | Blanc, P., Wald, L., & Ranchin, T. (1998). Importance and effect of co-registration quality in an example of “pixel to pixel” fusion process. In T. R. et L Wald (Ed.) 2nd international conference “ Fusion of earth data: Merging point measurements, raster maps and remotely sensed images ”, SEE/URISCA, Nice, France, Sophia Antipolis, France, pp. 67-74. |
| [13] | Blanchet, G., Moisan, L., & Rougé, B. (2005). A linear prefilter for image sampling with ringing artifact control. In IEEE International Conference Images Processing (pp. 577-580). · Zbl 0709.94650 |
| [14] | Blanchet, G., Lebeque, L., Fourest, S., Latry, C., Porez Nadal, F., Lacherade, S. & Thiebaut, C. (2012). Pleiades-HR innovative techniques for radiometric image quality commissioning (pp. XXXIX-B1:513-518). |
| [15] | Burt, P., & Adelson, E. (1983). The Laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4), 532-540. |
| [16] | Carla, R., Santurri, L., Aiazzi, B., & Baronti, S. (2015). Full-scale assessment of pansharpening through polynomial fitting of multiscale measurements. IEEE Transactions on Geoscience and Remote Sensing, 53(12), 6344-6355. · doi:10.1109/TGRS.2015.2436699 |
| [17] | Carper, W., Lillesand, T., & Kiefer, P. (1990). The use of intensity-hue-saturation transformations for merging spot panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing, 56(4), 459-467. |
| [18] | Chavez, P., Sides, S., & Anderson, J. (1991). Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogrammetric Engineering and Remote Sensing, 57(3), 295-303. |
| [19] | Cheng, M., Wang, C., & Li, J. (2014). Sparse representation based pansharpening using trained dictionary. IEEE Geoscience and Remote Sensing Letters, 11(1), 293-297. · doi:10.1109/LGRS.2013.2256875 |
| [20] | Chibani, Y., & Houacine, A. (2002). The joint use of IHS transform and redundant wavelet decomposition for fusing multispectral and panchromatic images. International Journal of Remote Sensing, 23, 3821-3833. · doi:10.1080/01431160110107626 |
| [21] | CNES. (2012). Pléaides user guide. Toulouse: ASTRUIM Services. |
| [22] | Croisier, A., Esteban, D., & Galand, C. (1976). Perfect channel splitting by use of interpolation/decimation/tree decomposition techniques. International Conference on Information Sciences and Systems, Patras, Greece, 2, 443-446. |
| [23] | Dai, X., & Khorram, S. (1998). The effects of image misregistration on the accuracy of remotely sensed change detection. IEEE Transactions on Geoscience and Remote Sensing, 36(5), 1566-1577. · doi:10.1109/36.718860 |
| [24] | Daubechies, I. (1988). Adjustable model-based fusion method for multispectral and panchromatic image. Communications on Pure and Applied Mathematics, 41, 909-996. · Zbl 0644.42026 · doi:10.1002/cpa.3160410705 |
| [25] | Delleji, T., Kallel, A., & Hamida, A. B. (2014). Multispectral image adaptive pansharpening based on wavelet transformation and NMDB approaches. International Journal of Remote Sensing, 35(19), 7069-7098. · doi:10.1080/01431161.2014.967883 |
| [26] | Ehlers, M., Klonus, S., Johan Åstrand, P., & Rosso, P. (2010). Multi-sensor image fusion for pansharpening in remote sensing. International Journal of Image and Data Fusion, 1(1), 25-45. · doi:10.1080/19479830903561985 |
| [27] | Esteban, D., & Galand, C. (1977). Application of quadrature mirror filters to split band voice coding schemes. In: IEEE international conference on acoustics, speech, and signal processing, ICASSP’77 (Vol. 2, pp. 191-195). |
| [28] | Ghahremani, M., & Ghassemian, H. (2015). Remote sensing image fusion using Ripplet transform and compressed sensing. IEEE Geoscience and Remote Sensing Letters, 12(3), 502-506. · doi:10.1109/LGRS.2014.2347955 |
| [29] | González-Audícana, M., Saleta, J. L., Catalán, R. G., & García, R. (2004). Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 42, 1291-1299. · doi:10.1109/TGRS.2004.825593 |
| [30] | Harikumar, V., Gajjar, P., Joshi, M., & Raval, M. (2014). Multiresolution image fusion: use of compressive sensing and graph cuts. Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of, 7(5), 1771-1780. · doi:10.1109/JSTARS.2013.2287891 |
| [31] | He, X., Condat, L., Bioucas-Dias, J., Chanussot, J., & Xia, J. (2014). A new pansharpening method based on spatial and spectral sparsity priors. IEEE Transactions on Image Processing, 23(9), 4160-4174. · Zbl 1374.94137 · doi:10.1109/TIP.2014.2333661 |
| [32] | Jiang, C., Zhang, H., Shen, H., & Zhang, L. (2012). A practical compressed sensing-based pan-sharpening method. IEEE Geoscience and Remote Sensing Letters, 9(4), 629-633. · doi:10.1109/LGRS.2011.2177063 |
| [33] | Jing, L., & Cheng, Q. (2011). An image fusion method for misaligned panchromatic and multispectral data. International Journal of Remote Sensing, 32(4), 1125-1137. · doi:10.1080/01431160903527405 |
| [34] | Kallel, A. (2015). MTF-adjusted pansharpening approach based on coupled multiresolution decompositions. IEEE Transactions on Geoscience and Remote Sensing, 53(6), 3124-3145. · doi:10.1109/TGRS.2014.2369056 |
| [35] | Khan, M., Alparone, L., & Chanussot, J. (2009). Pansharpening quality assessment using the modulation transfer functions of instruments. IEEE Transactions on Geoscience and Remote Sensing, 47(11), 3880-3891. · doi:10.1109/TGRS.2009.2029094 |
| [36] | Laben, C., & Brower, B. (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. US Patent 6011875. |
| [37] | Li, S., & Yang, B. (2011). A new pan-sharpening method using a compressed sensing technique. IEEE Transactions on Geoscience and Remote Sensing, 49(2), 738-746. · doi:10.1109/TGRS.2010.2067219 |
| [38] | Li, S., Yin, H., & Fang, L. (2013). Remote sensing image fusion via sparse representations over learned dictionaries. IEEE Transactions on Geoscience and Remote Sensing, 51(9), 4779-4789. · doi:10.1109/TGRS.2012.2230332 |
| [39] | Li, Z., & Leung, H. (2009). Fusion of multispectral and panchromatic images using a restoration-based method. IEEE Transactions on Geoscience and Remote Sensing, 47(5), 1482-1491. · doi:10.1109/TGRS.2008.2005639 |
| [40] | Liu, J. G. (2000). Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, 21(18), 3461-3472. · doi:10.1080/014311600750037499 |
| [41] | Luo, B., Khan, M., Bienvenu, T., Chanussot, J., & Zhang, L. (2013). Decision-based fusion for pansharpening of remote sensing images. IEEE Geoscience and Remote Sensing Letters, 10(1), 19-23. · doi:10.1109/LGRS.2012.2189933 |
| [42] | Mallat, S. (1989). A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7), 674-693. · Zbl 0709.94650 · doi:10.1109/34.192463 |
| [43] | Nencini, F., Garzelli, A., Baronti, S., & Alparone, L. (2007). Remote sensing image fusion using the curvelet transform. Information Fusion, 8(2), 143-156. · doi:10.1016/j.inffus.2006.02.001 |
| [44] | Nunez, J., Otazu, X., Fors, O., Prades, A., Pala, V., & Arbiol, R. (1999). Multiresolution-based image fusion with additive wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1204-1211. · doi:10.1109/36.763274 |
| [45] | Oppenheim, A. V., & Schafer, R. W. (2009). Discrete-time signal processing (3rd ed.). Upper Saddle River, NJ: Prentice Hall Press. · Zbl 0676.42001 |
| [46] | Otazu, X., González-Audícana, M., Fors, O., & Nunez, J. (2005). Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods. IEEE Transactions on Geoscience and Remote Sensing, 43(10), 2376-2385. · doi:10.1109/TGRS.2005.856106 |
| [47] | Palsson, F., Sveinsson, J., & Ulfarsson, M. (2014). A new pansharpening algorithm based on total variation. IEEE Geoscience and Remote Sensing Letters, 11(1), 318-322. · doi:10.1109/LGRS.2013.2257669 |
| [48] | Palsson, F., Sveinsson, J., Ulfarsson, M., & Benediktsson, J. (2015). Model-based fusion of multi- and hyperspectral images using PCA and wavelets. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2652-2663. · doi:10.1109/TGRS.2014.2363477 |
| [49] | Patra, S. K., Mishra, N., Chandrakanth, R., & Ramachandran, R. (2002). Image quality improvement through MTF compensation—A treatment to high resolution data. Indian Cartographer, 86-93. |
| [50] | Pohl, C., & Van Genderen, J. (1998). Review article multisensor image fusion in remote sensing: Concepts, methods and applications. International Journal of Remote Sensing, 19(5), 823-854. · doi:10.1080/014311698215748 |
| [51] | Ranchin, T., & Wald, L. (2000). Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation. Photogrammetric Engineering and Remote Sensing, 66(1), 49-61. |
| [52] | Shah, V. P., Younan, N. H., & King, R. L. (2008). An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Transactions on Geoscience and Remote Sensing, 46, 1323-1335. · doi:10.1109/TGRS.2008.916211 |
| [53] | Starck, J. L., Fadili, J., & Murtagh, F. (2007). The undecimated wavelet decomposition and its reconstruction. IEEE Transactions on Image Processing, 16(2), 297-309. · doi:10.1109/TIP.2006.887733 |
| [54] | Thomas, C., Ranchin, T., Wald, L., & Chanussot, J. (2008). Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1301-1312. · doi:10.1109/TGRS.2007.912448 |
| [55] | Vaidyanathan, P. (1990). Multirate digital filters, filter banks, polyphase networks, and applications: A tutorial. Proceedings of the IEEE, 78(1), 56-93. · doi:10.1109/5.52200 |
| [56] | Vetterli, M. (1986). Filter banks allowing perfect reconstruction. Signal Processing (Elsevier), 10(3), 219-244. · doi:10.1016/0165-1684(86)90101-5 |
| [57] | Vetterli, M., & Kovačević, J. (1995). Wavelets and subband coding. Prentice-Hall signal processing series. Prentice Hall PTR. · Zbl 0885.94002 |
| [58] | Vicinanza, M., Restaino, R., Vivone, G., Dalla Mura, M., & Chanussot, J. (2015). A pansharpening method based on the sparse representation of injected details. IEEE Geoscience and Remote Sensing Letters, 12(1), 180-184. · Zbl 1445.94009 · doi:10.1109/LGRS.2014.2331291 |
| [59] | Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G., et al. (2015a). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565-2586. · doi:10.1109/TGRS.2014.2361734 |
| [60] | Vivone, G., Simões, M., Dalla Mura, M., Restaino, R., Bioucas-Dias, J., Licciardi, G., et al. (2015b). Pansharpening based on semiblind deconvolution. IEEE Transactions on Geoscience and Remote Sensing, 53(4), 1997-2010. · Zbl 1327.80003 · doi:10.1109/TGRS.2014.2351754 |
| [61] | Wald, L. (1999). Some terms of reference in data fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1190-1193. · doi:10.1109/36.763269 |
| [62] | Wald, L., Ranchin, T., & Mangolini, M. (1997). Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing, 63(6), 691-699. |
| [63] | Wang, Z., & Bovik, A. (2002). A universal image quality index. IEEE Signal Processing Letters, 9, 81-84. · doi:10.1109/97.995823 |
| [64] | Wei, Q., Bioucas-Dias, J., Dobigeon, N., & Tourneret, J. Y. (2015). Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3658-3668. · doi:10.1109/TGRS.2014.2381272 |
| [65] | Xu, Q., Zhang, Y., & Li, B. (2014). Recent advances in pansharpening and key problems in applications. International Journal of Image and Data Fusion, 5(3), 175-195. · doi:10.1080/19479832.2014.889227 |
| [66] | Yin, H. (2015). Sparse representation based pansharpening with details injection model. Signal Processing, 113, 218-227. · doi:10.1016/j.sigpro.2014.12.017 |
| [67] | Yocky, D. (1996). Artifacts in wavelet image merging. Optical Engineering, 35, 2094-2101. · doi:10.1117/1.600765 |
| [68] | Zhang, G., Fang, F., Zhou, A., & Li, F. (2015). Pan-sharpening of multi-spectral images using a new variational model. International Journal of Remote Sensing, 36(5), 1484-1508. · doi:10.1080/01431161.2015.1014973 |
| [69] | Zhang, L., Shen, H., Gong, W., & Zhang, H. (2012). Adjustable model-based fusion method for multispectral and panchromatic images. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 42(6), 1693-1704. · doi:10.1109/TSMCB.2012.2198810 |
| [70] | Zhang, Y., De Backer, S., & Scheunders, P. (2009). Noise-resistant wavelet-based bayesian fusion of multispectral and hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 47(11), 3834-3843. · doi:10.1109/TGRS.2009.2017737 |
| [71] | Zhou, J., Civco, D. L., & Silander, J. A. (1998). A wavelet transform method to merge Landsat TM and SPOT panchromatic data. International Journal of Remote Sensing, 19(4), 743-757. · doi:10.1080/014311698215973 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
