A generative machine learning model for the 3D reconstruction of material microstructure and performance evaluation. (English) Zbl 07904381
Summary: The 3D reconstruction is generally defined as the process of capturing the shape and appearance of real objects. By reconstructing 3D digital model from a series of 2D slices, it brings considerable convenience to visualize internal structure and decipher structure-property relation of a material. Nowadays, the 3D reconstruction is becoming a cutting-edge technique in depicting the internal structure and evaluating the physical performance of targeted materials. Recent years, generative machine learning methods, such as generative adversarial networks (GAN), have achieved tremendous success in AI-generated physical content (AIGPC). However, lots of technical challenges remain, including oversimplified models, oversized dataset requirement and inefficient convergence. These difficulties are caused by the insufficient ability to capture detailed features and the inadequacy of the generated model quality. To this end, a novel generative model is developed, which combines the multiscale features of U-net and the synthesis ability of GANs. With the help of the multiscale channel aggregation module, the hierarchical feature aggregation module and the convolutional block attention module, our model is developed to capture the features of the material microstructure better. The loss function is refined by combining the image regularization loss with the Wasserstein distance loss. In addition, the anisotropy index is adopted to measure anisotropic degree of selected samples quantitively. The results demonstrate that the 3D structures generated by the proposed model retain high fidelity with ground truth samples. With remarkable performance, the proposed model not only overtakes traditional GAN, but will shed a brilliant light on AIGPC and physical 3D reconstruction.
Keywords:
3D reconstruction; generative adversarial networks; multiscale modelling; anisotropic degreeReferences:
| [1] | Gao, Y.; Jiao, Y.; Liu, Y., Ultra-efficient reconstruction of 3D microstructure and distribution of properties of random heterogeneous materials containing multiple phases, Acta Mater, 204, Article 116526 pp., 2021 |
| [2] | Fu, J.; Xiao, D.; Li, D.; Thomas, H. R.; Li, C., Stochastic reconstruction of 3D microstructures from 2D cross-sectional images using machine learning-based characterization, Comput. Methods Appl. Mech. Engrg., 390, Article 114532 pp., 2022 · Zbl 1507.74565 |
| [3] | Zhang, Y.; Yan, M.; Wan, Y.; Jiao, Z.; Chen, Y.; Chen, F.; Xia, C.; Ni, M., High-throughput 3D reconstruction of stochastic heterogeneous microstructures in energy storage materials, Npj Comput. Mater., 5, 2019 |
| [4] | Kang, I.; Goy, A.; Barbastathis, G., Dynamical machine learning volumetric reconstruction of objects’ interiors from limited angular views, Light Sci. Appl., 10, 2021 |
| [5] | Gayon-Lombardo, A.; Mosser, L.; Brandon, N. P.; Cooper, S. J., Pores for thought: generative adversarial networks for stochastic reconstruction of 3D multi-phase electrode microstructures with periodic boundaries, Npj Comput. Mater., 6, 2020 |
| [6] | Hsu, Y.-C.; Yang, Z.; Buehler, M. J., Generative design, manufacturing, and molecular modeling of 3D architected materials based on natural language input, APL Mater, 10, 2022 |
| [7] | Buehler, M. J., DeepFlames: Neural network-driven self-assembly of flame particles into hierarchical structures, MRS Commun, 12, 257-265, 2022 |
| [8] | Cecen, A.; Dai, H.; Yabansu, Y. C.; Kalidindi, S. R.; Song, L., Material structure-property linkages using three-dimensional convolutional neural networks, Acta Mater, 146, 76-84, 2018 |
| [9] | Li, X.; Zhang, Y.; Zhao, H.; Burkhart, C.; Brinson, L. C.; Chen, W., A Transfer Learning Approach for Microstructure Reconstruction and Structure-property Predictions, Sci. Rep., 8, 2018 |
| [10] | Ha, C. S.; Yao, D.; Xu, Z.; Liu, C.; Liu, H.; Elkins, D.; Kile, M.; Deshpande, V.; Kong, Z.; Bauchy, M.; Zheng, X., Rapid inverse design of metamaterials based on prescribed mechanical behavior through machine learning, Nat. Commun., 14, 2023 |
| [11] | Bastek, J.-H.; Kochmann, D. M., Inverse design of nonlinear mechanical metamaterials via video denoising diffusion models, Nat. Machine Intell., 5, 1466-1475, 2023 |
| [12] | Buehler, M. J., Generating 3D architectured nature-inspired materials and granular media using diffusion models based on language cues, Oxford Open Mater. Sci., 2, 2022 |
| [13] | Watson, J. L.; Juergens, D.; Bennett, N. R.; Trippe, B. L.; Yim, J.; Eisenach, H. E.; Ahern, W.; Borst, A. J.; Ragotte, R. J.; Milles, L. F.; Wicky, B. I.M.; Hanikel, N.; Pellock, S. J.; Courbet, A.; Sheffler, W.; Wang, J.; Venkatesh, P.; Sappington, I.; Torres, S. V.; Lauko, A.; De Bortoli, V.; Mathieu, E.; Ovchinnikov, S.; Barzilay, R.; Jaakkola, T. S.; DiMaio, F.; Baek, M.; Baker, D., De novo design of protein structure and function with RFdiffusion, Nat, 620, 1089-1100, 2023 |
| [14] | Yang, Z.; Hsu, Y.-C.; Buehler, M. J., Generative multiscale analysis of de novo proteome-inspired molecular structures and nanomechanical optimization using a VoxelPerceiver transformer model, J. Mech. and Phy. of Sol., 170, Article 105098 pp., 2023 |
| [15] | Waibel, D. J.E.; Röell, E.; Rieck, B.; Giryes, R.; Marr, C., A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images, (2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), 2023) |
| [16] | Sanchez-Lengeling, B.; Aspuru-Guzik, A., Inverse molecular design using machine learning: Generative models for matter engineering, Sci, 361, 360-365, 2018 |
| [17] | Cnudde, V.; Boone, M. N., High-resolution X-ray computed tomography in geosciences: A review of the current technology and applications, Earth-Science Rev, 123, 1-17, 2013 |
| [18] | Schlüter, S.; Sheppard, A.; Brown, K.; Wildenschild, D., Image processing of multiphase images obtained via X-ray microtomography: A review, Water Resour. Res., 50, 3615-3639, 2014 |
| [19] | Zong, C.; Bronckaers, A.; Vande Velde, G.; Willems, G.; Cadenas de Llano-Pérula, M., In Vivo Micro-Computerized Tomography Tracking of Human Periodontal Ligament Stem Cells Labeled with Gold Nanocomplexes, Adv. Healthcare Mater., 11, 2021 |
| [20] | Kim, T.; Lee, N.; Arifin, D. R.; Shats, I.; Janowski, M.; Walczak, P.; Hyeon, T.; Bulte, J. W.M., In Vivo Micro-CT Imaging of Human Mesenchymal Stem Cells Labeled with Gold-Poly-l-Lysine Nanocomplexes, Adv. Funct. Mater., 27, Article 1604213 pp., 2016 |
| [21] | Stachelek, P.; MacKenzie, L.; Parker, D.; Pal, R., Circularly polarised luminescence laser scanning confocal microscopy to study live cell chiral molecular interactions, Nat. Commun., 13, 2022 |
| [22] | Lymberopoulos, D. P.; Payatakes, A. C., Derivation of topological, geometrical, and correlational properties of porous media from pore-chart analysis of serial section data, J. Coll. Inter. Sci., 150, 61-80, 1992 |
| [23] | Izadi, H.; Baniassadi, M.; Hasanabadi, A.; Mehrgini, B.; Memarian, H.; Soltanian-Zadeh, H.; Abrinia, K., Application of full set of two point correlation functions from a pair of 2D cut sections for 3D porous media reconstruction, J. Pet. Sci. Eng., 149, 789-800, 2017 |
| [24] | Mac, M. J.; Yio, M. H.N.; Desbois, G.; Casanova, I.; Wong, H. S.; Buenfeld, N. R., 3D imaging techniques for characterising microcracks in cement-based materials, Cem. and Concr. Res., 140, Article 106309 pp., 2021 |
| [25] | Tahmasebi, P.; Sahimi, M., Enhancing multiple-point geostatistical modeling: 2. Iterative simulation and multiple distance function, Water Resour. Res., 52, 2099-2122, 2016 |
| [26] | Zhou, X.-P.; Xiao, N., A hierarchical-fractal approach for the rock reconstruction and numerical analysis, International J. Rock Mech. Min. Sci., 109, 68-83, 2018 |
| [27] | Lee, K.-H.; Yun, G. J., Multi-plane denoising diffusion-based dimensionality expansion for 2D-to-3D reconstruction of microstructures with harmonized sampling, Npj Comput. Mater., 10, 2024 |
| [28] | Valsecchi, A.; Damas, S.; Tubilleja, C.; Arechalde, J., Stochastic reconstruction of 3D porous media from 2D images using generative adversarial networks, Neurocomputing, 399, 227-236, 2020 |
| [29] | Volkhonskiy, D.; Muravleva, E.; Sudakov, O.; Orlov, D.; Burnaev, E.; Koroteev, D.; Belozerov, B.; Krutko, V., Generative adversarial networks for reconstruction of three-dimensional porous media from two-dimensional slices, Phys. Rev. E, 105, 2022 |
| [30] | Müller, S.; Sauter, C.; Shunmugasundaram, R.; Wenzler, N.; De Andrade, V.; De Carlo, F.; Konukoglu, E.; Wood, V., Deep learning-based segmentation of lithium-ion battery microstructures enhanced by artificially generated electrodes, Nat. Commun., 12, 2021 |
| [31] | Alexander, S. K.; Fieguth, P.; Ioannidis, M. A.; Vrscay, E. R., Hierarchical Annealing for Synthesis of Binary Images, Math Geosci, 41, 357-378, 2009 · Zbl 1163.86303 |
| [32] | Chen, D.; Wang, X.; Nan, J., A new framework for the reconstruction of porous media based on statistical characteristics: Multiscale erosion simulated annealing method, Phy. A: Statistical Mechanics and Its Applications, 616, Article 128616 pp., 2023 · Zbl 07679928 |
| [33] | Feinauer, J.; Brereton, T.; Spettl, A.; Weber, M.; Manke, I.; Schmidt, V., Stochastic 3D modeling of the microstructure of lithium-ion battery anodes via Gaussian random fields on the sphere, Comput. Mater. Sci., 109, 137-146, 2015 |
| [34] | Jiang, Z.; Chen, W.; Burkhart, C., Efficient 3D porous microstructure reconstruction via Gaussian random field and hybrid optimization, J. Microsc., 252, 135-148, 2013 |
| [35] | Feng, J.; Teng, Q.; He, X.; Wu, X., Accelerating multi-point statistics reconstruction method for porous media via deep learning, Acta Mater, 159, 296-308, 2018 |
| [36] | Tang, T.; Teng, Q.; He, X.; Luo, D., A pixel selection rule based on the number of different-phase neighbours for the simulated annealing reconstruction of sandstone microstructure, J. Microsc., 234, 262-268, 2009 |
| [37] | Diógenes, A. N.; Dos Santos, L. O.E.; Fernandes, C. P.; Moreira, A. C.; Apolloni, C. R., Porous media microstructure reconstruction using pixel-based and object-based simulated annealing – comparison with other reconstruction methods, Reterm, 8, 35, 2009 |
| [38] | Quiblier, J. A., A new three-dimensional modeling technique for studying porous media, J. Coll. Inter. Sci., 98, 84-102, 1984 |
| [39] | Jiao, Y.; Stillinger, F. H.; Torquato, S., Modeling heterogeneous materials via two-point correlation functions: Basic principles, Phys. Rev. E, 76, 2007 |
| [40] | Jiao, Y.; Stillinger, F. H.; Torquato, S., Modeling heterogeneous materials via two-point correlation functions. II. Algorithmic details and applications, Phys. Rev. E, 77, 2008 |
| [41] | Lee, K.-H.; Yun, G. J., Microstructure reconstruction using diffusion-based generative models, Mechanics of Advanced Materials and Structures, 31, 4443-4461, 2023 |
| [42] | Düreth, C.; Seibert, P.; Rücker, D.; Handford, S.; Kästner, M.; Gude, M., Conditional diffusion-based microstructure reconstruction, Mater. Today Commun., 35, Article 105608 pp., 2023 |
| [43] | Zhang, F.; He, X.; Teng, Q.; Wu, X.; Dong, X., 3D-PMRNN: Reconstructing three-dimensional porous media from the two-dimensional image with recurrent neural network, J. Pet. Sci. Eng., 208, Article 109652 pp., 2022 |
| [44] | Zhang, F.; He, X.; Teng, Q.; Wu, X.; Cui, J.; Dong, X., PM-ARNN: 2D-TO-3D reconstruction paradigm for microstructure of porous media via adversarial recurrent neural network, Knowledge-Based Systems, 264, Article 110333 pp., 2023 |
| [45] | Kim, Y.; Park, H. K.; Jung, J.; Asghari-Rad, P.; Lee, S.; Kim, J. Y.; Jung, H. G.; Kim, H. S., Exploration of optimal microstructure and mechanical properties in continuous microstructure space using a variational autoencoder, Mater. & Des., 202, Article 109544 pp., 2021 |
| [46] | Dan, Y.; Zhao, Y.; Li, X.; Li, S.; Hu, M.; Hu, J., Generative adversarial networks (GAN) based efficient sampling of chemical composition space for inverse design of inorganic materials, Npj Comput. Mater., 6, 2020 |
| [47] | Sciazko, A.; Komatsu, Y.; Shimura, T.; Shikazono, N., Prediction of electrode microstructure evolutions with physically constrained unsupervised image-to-image translation networks, Npj Comput. Mater., 10, 2024 |
| [48] | Öngün, C.; Temizel, A., Paired 3D Model Generation with Conditional Generative Adversarial Networks, 2019, Springer International Publishing |
| [49] | Murgas, B.; Stickel, J.; Ghosh, S., Generative adversarial network (GAN) enabled Statistically equivalent virtual microstructures (SEVM) for modeling cold spray formed bimodal polycrystals, Npj Comput. Mater., 10, 2024 |
| [50] | Zhang, T.; Shen, T.; Hu, G.; Lu, F.; Du, X., Stochastic reconstruction of digital cores using two-discriminator VAE-GAN, Geo. Sci. and Eng., 236, Article 212744 pp., 2024 |
| [51] | Shen, S. C.; Buehler, M. J., Nature-inspired architected materials using unsupervised deep learning, Commun. Eng., 1, 2022 |
| [52] | Mosser, L.; Dubrule, O.; Blunt, M. J., Reconstruction of three-dimensional porous media using generative adversarial neural networks, Phys. Rev. E, 96, 2017 |
| [53] | Mosser, L.; Dubrule, O.; Blunt, M. J., Stochastic Reconstruction of an Oolitic Limestone by Generative Adversarial Networks, Transp. Porous. Med., 125, 81-103, 2018 |
| [54] | Kench, S.; Cooper, S. J., Generating three-dimensional structures from a two-dimensional slice with generative adversarial network-based dimensionality expansion, Nat. Mach. Intell., 3, 299-305, 2021 |
| [55] | Chi, P.; Sun, J.; Luo, X.; Cui, R.; Dong, H., Reconstruction of 3D digital rocks with controllable porosity using CVAE-GAN, Geo. Sci. and Eng., 230, Article 212264 pp., 2023 |
| [56] | Li, Y.; Jian, P.; Han, G., Cascaded Progressive Generative Adversarial Networks for Reconstructing Three-Dimensional Grayscale Core Images From a Single Two-Dimensional Image, Front. Phys., 10, 2022 |
| [57] | Bangaru, S. S.; Wang, C.; Zhou, X.; Hassan, M., Scanning electron microscopy (SEM) image segmentation for microstructure analysis of concrete using U-net convolutional neural network, Automation in Construction, 144, Article 104602 pp., 2022 |
| [58] | Ronneberger, O.; Fischer, P.; Brox, T., U-Net: Convolutional Networks for Biomedical Image Segmentation, 2015, Springer International Publishing |
| [59] | Rücker, M.; Bartels, W.-B.; Singh, K.; Brussee, N.; Coorn, A.; van der Linde, H. A.; Bonnin, A.; Ott, H.; Hassanizadeh, S. M.; Blunt, M. J.; Mahani, H.; Georgiadis, A.; Berg, S., The Effect of Mixed Wettability on Pore-Scale Flow Regimes Based on a Flooding Experiment in Ketton Limestone, Geo. Res. Lett., 46, 3225-3234, 2019 |
| [60] | Yu, X.; Qu, Y.; Hong, M., Underwater-GAN: Underwater Image Restoration via Conditional Generative Adversarial Network, 2018, Springer International Publishing |
| [61] | Isola, P.; Zhu, J.-Y.; Zhou, T.; Efros, A. A., (2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017) |
| [62] | Hore, A.; Ziou, D., (Image Quality Metrics: PSNR vs. SSIM, 2010 20th International Conference on Pattern Recognition, 2010) |
| [63] | Zhang, T.; Liu, Q.; Wang, X.; Ji, X.; Du, Y., A 3D reconstruction method of porous media based on improved WGAN-GP, Comput. & Geo.s, 165, Article 105151 pp., 2022 |
| [64] | Gabbouj, M.; Partio, M.; Cramariuc, B.; Visa, A., Rock Texture Retrieval Using Gray Level Co-occurrence Matrix, (Proc. of the 5th Nordic Signal Processing Symposium, NORSIG 2002. Proc. of the 5th Nordic Signal Processing Symposium, NORSIG 2002, Norway, 2002) |
| [65] | Gebejes, A.; Huertas, R., Texture characterization based on grey-level co-occurrence matrix, Databases, 9, 375-378, 2013 |
| [66] | Lehoucq, R.; Weiss, J.; Dubrulle, B.; Amon, A.; Le Bouil, A.; Crassous, J.; Amitrano, D.; Graner, F., Analysis of image vs. position, scale and direction reveals pattern texture anisotropy, Front. Phys., 2, 2015 |
| [67] | Özarslan, E.; Vemuri, B. C.; Mareci, T. H., Generalized scalar measures for diffusion MRI using trace, variance, and entropy, Magnetic Resonance in Med, 53, 866-876, 2005 |
| [68] | Neumann, R.; Andreeta, M.; Lucas-Oliveira, E., Sandstones: raw, filtered and segmented data, Digital Rocks Portal, 2020, Online |
| [69] | Pepper, J. F.; de Witt, W.; Demarest, D. F., Geology of the Bedford Shale and Berea Sandstone in the Appalachian Basin, Sci., 119, 512-513, 1954 |
| [70] | Pedretti, D.; Bianchi, M., GEOENT: A Toolbox for Calculating Directional Geological Entropy, Geo, 12, 206, 2022 |
| [71] | Hsu, T.; Epting, W. K.; Mahbub, R.; Nuhfer, N. T.; Bhattacharya, S.; Lei, Y.; Miller, H. M.; Ohodnicki, P. R.; Gerdes, K. R.; Abernathy, H. W.; Hackett, G. A.; Rollett, A. D.; De Graef, M.; Litster, S.; Salvador, P. A., Mesoscale characterization of local property distributions in heterogeneous electrodes, J. Power Sour., 386, 1-9, 2018 |
| [72] | Albuquerque, V. H.C. D.; Tavares, J. M.R. S.; Cortez, P. C., Quantification of the microstructures of hypoeutectic white cast iron using mathematical morphology and an artificial neural network, IJMMP, 5, 52, 2010 |
| [73] | Pero-Sanz, J. A.; Plaza, D.; Verdeja, J. I.; Asensio, J., Metallographic Characterization of Hypoeutectic Martensitic White Cast Irons, Mater. Charact., 43, 33-39, 1999 |
| [74] | Cvikel, D.; Cohen, M.; Inberg, A.; Klein, S.; Iddan, N.; Ashkenazi, D., Metallurgical characterization of brass sheet from the 19th-century Akko Tower Wreck (Israel), Mater. Character., 131, 175-187, 2017 |
| [75] | Yu, Y.; Chen, W.; Xie, L.; Song, Y., Effect of heat treatment on microstructure evolution, phase transformation and mechanical properties of dual phase Cu-Zn alloy, J. Alloy. Compd., 904, Article 163960 pp., 2022 |
| [76] | Semih, Ö.; Recep, A., Investigation of microstructure, machinability, and mechanical properties of new-generation hybrid lead-free brass alloys, High Temperature Mater. and Proc., 42, 2023 |
| [77] | Oluigbo, C. V.; Agunsoye, J. O.; Agboola, J. B., Effects of graphite addition on the mechanical property and wear resistance of 60/40 brass, ITEGAM, 8, 2022 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
