Edge-based multi-modal registration and application for night vision devices. (English) Zbl 1343.68254
Summary: Multi-modal image sequence registration is a challenging problem that consists in aligning two image sequences of the same scene acquired with a different sensor, hence containing different characteristics. We focus in this paper on the registration of optical and infra-red image sequences acquired during the flight of a helicopter. Both cameras are located at different positions and they provide complementary informations. We propose a fast registration method based on the edge information: a new criterion is defined in order to take into account both the magnitude and the orientation of the edges of the images to register. We derive a robust technique based on a gradient ascent and combined with a reliability test in order to quickly determine the optimal transformation that matches the two image sequences. We show on real multi-modal data that our method outperforms classical registration methods, thanks to the shape information provided by the contours. Besides, results on synthetic images and real experimental conditions show that the proposed algorithm manages to find the optimal transformation in few iterations, achieving a rate of about 8 frames per second.
MSC:
| 68T45 | Machine vision and scene understanding |
| 68U10 | Computing methodologies for image processing |
References:
| [1] | Allaire, S., Kim, J.J., Breen, S.L., Jaffray, D.A., Pekar, V.: Full orientation invariance and improved feature selectivity of 3d sift with application to medical image analysis. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2008. CVPRW’08, pp. 1-8. IEEE (2008) |
| [2] | Aubert, G., Kornprobst, P.: Mathematical problems in image processing: partial differential equations and the calculus of variationsx, vol. 147. Springer, New York (2006) · Zbl 1110.35001 |
| [3] | Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: SURF: Speeded up robust features. Comput. Vis. Image Underst. (CVIU) 110(3), 346-359 (2008) · doi:10.1016/j.cviu.2007.09.014 |
| [4] | Brown, L.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325-376 (1992) · doi:10.1145/146370.146374 |
| [5] | Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490-530 (2005) · Zbl 1108.94004 · doi:10.1137/040616024 |
| [6] | Dellinger, F., Delon, J., Gousseau, Y., Michel, J., Tupin, F.: SAR-SIFT: a SIFT-like algorithm for applications on sar images. In: 2012 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 3478-3481. IEEE (2012) |
| [7] | Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381-395 (1981) · doi:10.1145/358669.358692 |
| [8] | Haber, E., Modersitzki, J.: Intensity gradient based registration and fusion of multi-modal images. Medical image computing and computer-assisted intervention: MICCAI 46, 292-299 (2006) |
| [9] | Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge university press, Cambridge (2003) · Zbl 0956.68149 |
| [10] | Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, S.M., Schnabel, J.A.: Mind: Modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Image Anal. 16(7), 1423-1435 (2012) · doi:10.1016/j.media.2012.05.008 |
| [11] | Hermosillo, G., Chefd’Hotel, C., Faugeras, O.: Variational methods for multimodal image matching. Int. J. Comput. Vis. 50(3), 329-343 (2002) · Zbl 1012.68788 · doi:10.1023/A:1020830525823 |
| [12] | Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: Sixth International Conference on Computer Vision, 1998, pp. 959-966. IEEE (1998) |
| [13] | Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91-110 (2004) · doi:10.1023/B:VISI.0000029664.99615.94 |
| [14] | Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187-198 (1997) · doi:10.1109/42.563664 |
| [15] | Mann, S., Picard, R.W.: Video orbits of the projective group a simple approach to featureless estimation of parameters. IEEE Trans. Image Process. 6(9), 1281-1295 (1997) · doi:10.1109/83.623191 |
| [16] | Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615-1630 (2005) · doi:10.1109/TPAMI.2005.188 |
| [17] | Odobez, J.M., Bouthemy, P.: Robust multiresolution estimation of parametric motion models. J. Vis. Commun. Image Represent. 6(4), 348-365 (1995) · doi:10.1006/jvci.1995.1029 |
| [18] | Paganelli, C., Peroni, M., Pennati, F., Baroni, G., Summers, P., Bellomi, M., Riboldi, M.: Scale invariant feature transform as feature tracking method in 4 D imaging: a feasibility study. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6543-6546 (2012) |
| [19] | Pluim, J., Maintz, J., Viergever, M.: Image registration by maximization of combined mutual information and gradient information. IEEE Trans. Med. Imaging 19(8), 809-814 (2000) · doi:10.1109/42.876307 |
| [20] | Pratt, W.: Digital Image Processing. Wiley, New York (1978) · Zbl 0728.68142 |
| [21] | Roshni, V., Revathy, K.: Using mutual information and cross correlation as metrics for registration of images. J. Theor. Appl. Inf. Technol. 4 (6) (2008) |
| [22] | Spontón, H., Cardelino, J.: A review of classic edge detectors. In: Image Processing On Line (2012) |
| [23] | Sun, Y., Jolly, M.P., Moura, J.F.: Integrated registration of dynamic renal perfusion MRimages. In: ICIP, pp. 1923-1926 (2004) |
| [24] | Toews, M., Zöllei, L., Wells, W.M.: Feature-based alignment of volumetric multi-modal images. In: Information Processing in Medical Imaging, pp. 25-36. Springer (2013) |
| [25] | Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137-154 (1997) · doi:10.1023/A:1007958904918 |
| [26] | Yu, L., Zhang, D., Holden, E.J.: A fast and fully automatic registration approach based on point features for multi-source remote-sensing images. Computers & Geosciences 34(7), 838-848 (2008) · doi:10.1016/j.cageo.2007.10.005 |
| [27] | Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977-1000 (2003) · doi:10.1016/S0262-8856(03)00137-9 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
