Multi-view 3D reconstruction and modeling of the unknown 3D scenes using genetic algorithms. (English) Zbl 1402.90223
Summary: This paper presents a complete pipeline of the reconstruction and the modeling of the unknown complex 3D scenes from a sequence of unconstrained images. The proposed system is based on the formulation of a nonlinear cost function by determining the relationship between 2D points of the images and the cameras parameters; the optimization of this function by a genetic algorithm makes finding the optimal cameras parameters. The determination of these parameters allows thereafter to estimate the 3D points of the observed scene. Then, the mesh of the 3D points is achieved by 3D Crust algorithm and the texture mapping is performed by multiple dependent viewpoints. Extensive experiments on synthetic and real data are performed to validate the proposed approach, and the results indicate that our system is robust and can achieve a very satisfactory reconstruction quality.
MSC:
| 90C90 | Applications of mathematical programming |
| 90C59 | Approximation methods and heuristics in mathematical programming |
| 68T45 | Machine vision and scene understanding |
Keywords:
camera self-calibration; nonlinear optimization; genetic algorithms; 3D mesh; 3D reconstruction; 3D modelingSoftware:
PowercrustReferences:
| [1] | Amenta N (1999) The crust algorithm for 3D surface reconstruction. In: Proceedings of symposium on computational geometry, pp 423-424 |
| [2] | Amenta N, Choi S, Kolluri RK (2001) The power crust. In: Proceedings of the sixth ACM symposium on solid modeling and applications, University of Texas at Austin, pp 249-266 |
| [3] | Anam S, Islam MS, Kashem MA, Islam MN, Islam MR, Islam MS (2009) Face recognition using genetic algorithm and back propagation neural network. In: International multi conference of engineers and computer scientists, vol I |
| [4] | Baumgart BG et al (1974) Geometric modeling for computer vision. Doctoral dissertation, Stanford University |
| [5] | Cazals F, Giesen J (2004) Delaunay triangulation based surface reconstruction: ideas and algorithms. Technical report, RR-5393, INRIA · Zbl 1116.65022 |
| [6] | Chang, CC; Kuo, Y-T, Genetic-based approach for synthesizing texture, Int J Artif Intell Tools, 17, 731-743, (2008) · doi:10.1142/S0218213008004126 |
| [7] | Craciun DI (2011) Modélisation des équivalents dynamiques des réseaux électriques. Thèse, Université de Grenoble, p 174 |
| [8] | Dipanda, A.; Woo, S.; Marzani, F.; Bilbault, JM, 3D shape reconstruction in an active stereo vision system using genetic algorithms, J Pattern Recognit Soc, 36, 2143-2159, (2003) · Zbl 1047.68635 · doi:10.1016/S0031-3203(03)00049-9 |
| [9] | Hazzat, S.; Saaidi, A.; Satori, K., Euclidean 3D reconstruction of unknown objects from multiple images, J Emerg Technol Web Intell, 6, 59-63, (2014) |
| [10] | Faugeras O, Luong QT, Papadopoulou T (2001) The geometry of multiple images: the laws that govern the formation of images of a scene and some of their applications. MIT Press, Cambridge · Zbl 1002.68183 |
| [11] | Franco, J., Efficient polyhedral modeling from silhouettes, IEEE Trans Pattern Anal Mach Intell, 31, 853-861, (2010) |
| [12] | Fuhrmann, S.; etal., MVE—an image-based reconstruction environment, Comput Graph, 53, 44-53, (2015) · doi:10.1016/j.cag.2015.09.003 |
| [13] | Furukawa, Y.; Ponce, J., Accurate, dense, and robust multi-view stereopsis, Trans Pattern Anal Mach Intell, 32, 1362-1376, (2010) · doi:10.1109/TPAMI.2009.161 |
| [14] | Furukawa Y, Curless B, Seitz SM, Szeliski R (2010) Towards internet-scale multi-view stereo. In: Conference on computer vision and pattern recognition |
| [15] | Goldberg DE (1989) Genetic algorithms in search, optimization & machine learning. Addison-Wesley, Boston · Zbl 0721.68056 |
| [16] | Goldberg, DE; Deb, K.; Rawlins, G. (ed.), A comparative analysis of selection scheme used in genetic algorithms, 69-93, (1991), San Mateo |
| [17] | Harris C, Stephens M (1988) A combined corner et edge detector. In: 4th Alvey vision conference, pp 147-151 |
| [18] | Hartley RI, Zisserman A (2000) Multiple view geometry in computer vision. Cambridge University Press, Cambridge, p 265. ISBN: 0521623049 · Zbl 0956.68149 |
| [19] | Holland JH (1992) Adaptation in natural and artificial systems. MIT Press, Cambridge |
| [20] | Hornung A, Kobbelt L, (2006) Robust reconstruction of watertight 3D models from non-uniformly sampled point-clouds without normal information. In: Eurographics symposium on geometry processing, pp 41-50 |
| [21] | Janko, Z.; Chetverikov, D.; Ekart, A., Using genetic algorithms in computer vision: registering images to 3D surface model, Acta Cybern, 18, 193-212, (1995) · Zbl 1135.68601 |
| [22] | Jean-Denis D, Adrien B, Pierre G (1998) Interactive 3D modeling from multiple images using scene regularities. Lecture notes in computer science, vol, 1506, pp 236-252 |
| [23] | Jean-Denis D, Adrien B, Pierre G (2010) Shape-from-texture revisited. In: Francophone congress of pattern recognition and artificial intelligence, pp 1-8 |
| [24] | Johnson CM, Bhat A, et Thibault W (2006) An evolutionary approach to camera-based projector calibration. In: Genetic and evolutionary computation conference, pp 1871-1872 |
| [25] | Kazhdan, M.; Hoppe, H., Screened Poisson surface reconstruction, ACM Trans Graph, 32, 1-29, (2013) · Zbl 1322.68228 · doi:10.1145/2487228.2487237 |
| [26] | Kolev, K.; Klodt, M.; etal., Continuous global optimization in multiview 3D reconstruction, Int J Comput Vis, 4, 80-96, (2009) · doi:10.1007/s11263-009-0233-1 |
| [27] | Kolev, K.; Brox, T.; Cremers, D., Fast joint estimation of silhouettes and dense 3D geometry from multiple images, Trans Pattern Anal Mach Intell, 34, 493-505, (2012) · doi:10.1109/TPAMI.2011.150 |
| [28] | Kutulakos, KN; Seitz, SM, A theory of shape by space carving, Int J Comput Vis, 38, 199-218, (2000) · Zbl 1012.68691 · doi:10.1023/A:1008191222954 |
| [29] | Lobay, A.; Forsyth, DA, Shape from texture without boundaries, Int J Comput Vis, 67, 71-91, (2006) · doi:10.1007/s11263-006-4068-8 |
| [30] | Loh M, Hartley R (2005) Shape from non homogeneous, non-stationary, anisotropic, perspective texture. In: BMVC’05. Royaume-Uni, Oxford, pp 69-78 |
| [31] | Ma Y, Soatto S, Kosecka J, Sastry SS (2003) An invitation to 3-D vision: from images to geometric models. Springer, Berlin · Zbl 1043.65040 |
| [32] | Matusik W, Buehler C, McMillan L (2001) Polyhedral visual hulls for real-time rendering. In: Euro graphics workshop on rendering, pp 115-125 |
| [33] | Merras, M.; Akkad, N.; Saaidi, A.; Nazih, AG; Satori, K., Camera calibration with varying parameters based on improved genetic algorithm, WSEAS Trans Comput, 13, 129-137, (2014) |
| [34] | Merras, M.; etal., Camera self calibration with varying parameters by an unknown three dimensional scene using the improved genetic algorithm, 3D Res, 6, 1-14, (2015) · doi:10.1007/s13319-015-0039-6 |
| [35] | Merras, M.; Hazzat, S.; Saaidi, A.; Nazih, AG; Satori, K., 3D face reconstruction using images from cameras with varying parameters, Int J Autom Comput, (2016) · doi:10.1007/s11633-016-0999-x |
| [36] | Nguyen, MH; etal., Modeling of 3D object using unconstrained and uncalibrated images taken with a handheld camera, Comput Vis Imaging Comput Graph Theory Appl, 274, 1-5, (2011) |
| [37] | Nguyen MH et al (2013) A hybrid image base modeling algorithm. In: Proceedings of the thirty sixth Australasian computer sciences conference, vol 135, pp 115-123 |
| [38] | Nistér, D., Preemptive RANSAC for live structure and motion estimation, Mach Vis Appl, 16, 321-329, (2005) · doi:10.1007/s00138-005-0006-y |
| [39] | Olsson C, Enqvist O (2011) Stable structure from motion for unordered image collections. In: Scandinavian conference on image analysis, SCIA 2011 |
| [40] | Pighin, F., Modeling and animating realistic faces from images, Int J Comput Vis, 50, 143-169, (2002) · Zbl 1012.68776 · doi:10.1023/A:1020393915769 |
| [41] | Pighin, F.; Hecker, J.; Dani, L.; Richard, S.; Salesin, DH, Synthesizing realistic facial expressions from photographs, Comput Graph, (1998) · doi:10.1145/280814.280825 |
| [42] | Pollefeys, M.; Koch, R.; Gool, LV, Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters, Int J Comput Vis, 32, 7-25, (1999) · doi:10.1023/A:1008109111715 |
| [43] | Quan, L.; etal., Image-based plant modeling, ACM Trans Graph, 25, 599-604, (2006) · doi:10.1145/1141911.1141929 |
| [44] | Ren, Z-W; San, Y.; Chen, J-F, Hybrid implex-improved genetic algorithm for global numerical optimization, Acta Autom Sin, 33, 91-95, (2007) · doi:10.1360/aas-007-0091 |
| [45] | Roberts R, Szeliski R (2011) Structure from motion for scenes with large duplicate structures. In: Computer vision and pattern recognition, pp 3137-3144 |
| [46] | Saaidi A, Tairi H, Satori K (2006) Fast stereo matching using rectification and correlation techniques. In: ISCCSP, second international symposium on communications, control and signal processing. Marrakech, Morrocco, pp 1-4 |
| [47] | Salman, N.; Yvinec, M., Surface reconstruction from multi-view stereo of large-scale outdoor scenes, Int J Virtual Real, 5, 1-6, (2010) |
| [48] | Seitz S, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Conference on computer vision and pattern recognition |
| [49] | Snavely, N.; Seitz, SM; Szeliski, R., Photo tourism: exploring photo collections in 3D, ACM Trans Graph, 25, 835-846, (2006) · doi:10.1145/1141911.1141964 |
| [50] | Tan, P.; etal., Image based tree graphics, ACM Trans Graph, 27, 418-433, (2006) |
| [51] | Triggs B, McLauchlan P, Hartley RI, Fitzgibbon A (1999) Bundle adjustment—a modern synthesis. In: Vision algorithms, pp 298-372 |
| [52] | Wang, G.; Wu, QMJ, Perspective 3-d Euclidean reconstruction with varying camera parameters, IEEE Trans Circuits Syst Video Technol, 19, 1793-1803, (2009) · doi:10.1109/TCSVT.2009.2031380 |
| [53] | Wilczkowiak M, Boyer E, Sturm P (2001) Camera calibration and 3D reconstruction from single images using parallelepipeds. In: ICCV. Vancouver, Canada, pp 142-148 |
| [54] | Wojciech et al (2000) Image based visual hulls. In: 27th conference on computer graphics and interactive techniques, pp 369-374 |
| [55] | Wu C (2013) Towards linear-time incremental structurefrom motion. In: International conference on 3D vision, pp 127-134 |
| [56] | Wu C, Agarwal S, Curless B, Seitz S (2011) Multicore bundle adjustment. In: Conference on computer vision and pattern recognition, pp 3057-3064 |
| [57] | Xiao, J.; etal., Image based façade modeling, ACM Trans Graph, 27, 26-34, (2008) · doi:10.1145/1409060.1409114 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
