Projection pursuit adaptation on polynomial chaos expansions. (English) Zbl 1539.65028
Summary: The present work addresses the issue of accurate stochastic approximations in high-dimensional parametric space using tools from uncertainty quantification (UQ). The basis adaptation method and its accelerated algorithm in polynomial chaos expansions (PCE) were recently proposed to construct low-dimensional approximations adapted to specific quantities of interest (QoI). The present paper addresses one difficulty with these adaptations, namely their reliance on quadrature point sampling, which limits the reusability of potentially expensive samples. Projection pursuit (PP) is a statistical tool to find the “interesting” projections in high-dimensional data and thus bypass the curse-of-dimensionality. In the present work, we combine the fundamental ideas of basis adaptation and projection pursuit regression (PPR) to propose a novel method to simultaneously learn the optimal low-dimensional spaces and PCE representation from given data. While this projection pursuit adaptation (PPA) can be entirely data-driven, the constructed approximation exhibits mean-square convergence to the solution of an underlying governing equation and thus captures the supports and probability distributions associated with the physics constraints. The proposed approach is demonstrated on a borehole problem and a structural dynamics problem, demonstrating the versatility of the method and its ability to discover low-dimensional manifolds with high accuracy with limited data. In addition, the method can learn surrogate models for different quantities of interest while reusing the same data set.
MSC:
| 65C30 | Numerical solutions to stochastic differential and integral equations |
| 62L20 | Stochastic approximation |
| 62J12 | Generalized linear models (logistic models) |
Keywords:
surrogate modeling; polynomial chaos expansion; projection pursuit; high-dimensional models; dimension reduction; data-drivenReferences:
| [1] | Ghanem, R.; Spanos, P. D., Stochastic Finite Elements: A Spectral Approach (1991), Springer-Verlag · Zbl 0722.73080 |
| [2] | Ghanem, R.; Red-Horse, J., Propagation of probabilistic uncertainty in complex physical systems using a stochastic finite element approach, Physica D, 133, 1-4, 137-144 (1999) · Zbl 1194.74400 |
| [3] | Sudret, B., Uncertainty Propagation and Sensitivity Analysis in Mechanical Models-Contributions to Structural Reliability and Stochastic Spectral Methods, Vol. 147, 53 (2007), Université Blaise Pascal: Université Blaise Pascal Clermont-Ferrand, France, (Habilitationa diriger des recherches) |
| [4] | Najm, H. N., Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics, Annu. Rev. Fluid Mech., 41, 35-52 (2009) · Zbl 1168.76041 |
| [5] | Soize, C., Uncertainty Quantification (2017), Springer · Zbl 1377.60002 |
| [6] | Chen, J.; Zeng, X.; Peng, Y., Probabilistic analysis of wind-induced vibration mitigation of structures by fluid viscous dampers, J. Sound Vib., 409, 287-305 (2017) |
| [7] | Zeng, X.; Peng, Y.; Chen, J., Serviceability-based damping optimization of randomly wind-excited high-rise buildings, Struct. Des. Tall Special Build., 26, 11, Article e1371 pp. (2017) |
| [8] | Marzouk, Y. M.; Najm, H. N.; Rahn, L. A., Stochastic spectral methods for efficient Bayesian solution of inverse problems, J. Comput. Phys., 224, 2, 560-586 (2007) · Zbl 1120.65306 |
| [9] | Marzouk, Y. M.; Najm, H. N., Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems, J. Comput. Phys., 228, 6, 1862-1902 (2009) · Zbl 1161.65308 |
| [10] | Marzouk, Y.; Xiu, D., A stochastic collocation approach to Bayesian inference in inverse problems (2009) · Zbl 1364.62064 |
| [11] | Sudret, B.; Marelli, S.; Wiart, J., Surrogate models for uncertainty quantification: An overview, (2017 11th European Conference on Antennas and Propagation. 2017 11th European Conference on Antennas and Propagation, EUCAP (2017), IEEE), 793-797 |
| [12] | Ghanem, R., Ingredients for a general purpose stochastic finite elements implementation, Comput. Methods Appl. Mech. Engrg., 168, 19-34 (1999) · Zbl 0943.65008 |
| [13] | Xiu, D.; Karniadakis, G. E., The Wiener-Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24, 619-644 (2002) · Zbl 1014.65004 |
| [14] | Xiu, D.; Karniadakis, G. E., Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos, Comput. Methods Appl. Mech. Engrg., 191, 4927-4948 (2002) · Zbl 1016.65001 |
| [15] | MacKay, D. J., Introduction to Gaussian processes, NATO ASI Series F Comput. Syst. Sci., 168, 133-166 (1998) |
| [16] | Seeger, M., Gaussian processes for machine learning, Int. J. Neural Syst., 14, 02, 69-106 (2004) |
| [17] | Bilionis, I.; Zabaras, N., Multi-output local Gaussian process regression: Applications to uncertainty quantification, J. Comput. Phys., 231, 17, 5718-5746 (2012) · Zbl 1277.60066 |
| [18] | Li, J.; Chen, J., Stochastic Dynamics of Structures (2009), John Wiley & Sons · Zbl 1170.74003 |
| [19] | Chen, J.; Yang, J.; Li, J., A GF-discrepancy for point selection in stochastic seismic response analysis of structures with uncertain parameters, Struct. Saf., 59, 20-31 (2016) |
| [20] | Soize, C.; Ghanem, R., Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets, Comput. Methods Appl. Mech. Engrg., 380, Article 113777 pp. (2021) · Zbl 1506.65175 |
| [21] | Zhang, H.; Guilleminot, J.; Gomez, L. J., Stochastic modeling of geometrical uncertainties on complex domains, with application to additive manufacturing and brain interface geometries, Comput. Methods Appl. Mech. Engrg., 385, Article 114014 pp. (2021) · Zbl 1502.60080 |
| [22] | Giovanis, D. G.; Shields, M. D., Data-driven surrogates for high dimensional models using Gaussian process regression on the grassmann manifold, Comput. Methods Appl. Mech. Engrg., 370, Article 113269 pp. (2020) · Zbl 1506.62549 |
| [23] | Kougioumtzoglou, I.; Spanos, P., An analytical Wiener path integral technique for non-stationary response determination of nonlinear oscillators, Probab. Eng. Mech., 28, 125-131 (2012) |
| [24] | Psaros, A. F.; Petromichelakis, I.; Kougioumtzoglou, I. A., Wiener path integrals and multi-dimensional global bases for non-stationary stochastic response determination of structural systems, Mech. Syst. Signal Process., 128, 551-571 (2019) |
| [25] | Petromichelakis, I.; Kougioumtzoglou, I. A., Addressing the curse of dimensionality in stochastic dynamics: A Wiener path integral variational formulation with free boundaries, Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 476, 2243, Article 20200385 pp. (2020) · Zbl 1472.60089 |
| [26] | Babuška, I.; Nobile, F.; Tempone, R., A stochastic collocation method for elliptic partial differential equations with random input data, SIAM J. Numer. Anal., 45, 3, 1005-1034 (2007) · Zbl 1151.65008 |
| [27] | Le Maitre, O.; Knio, O. M., Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics (2010), Springer Science & Business Media · Zbl 1193.76003 |
| [28] | Smolyak, S. A., Quadrature and interpolation formulas for tensor products of certain classes of functions, Dokl. Akad. Nauk, 4, 240-243 (1963) · Zbl 0202.39901 |
| [29] | Gerstner, T.; Griebel, M., Numerical integration using sparse grids, Numer. Algorithms, 18, 3-4, 209-232 (1998) · Zbl 0921.65022 |
| [30] | Novak, E.; Ritter, K., Simple cubature formulas with high polynomial exactness, Constr. Approx., 15, 4, 499-522 (1999) · Zbl 0942.41018 |
| [31] | Blatman, G.; Sudret, B., An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis, Probab. Eng. Mech., 25, 2, 183-197 (2010) |
| [32] | Blatman, G.; Sudret, B., Adaptive sparse polynomial chaos expansion based on least angle regression, J. Comput. Phys., 230, 6, 2345-2367 (2011) · Zbl 1210.65019 |
| [33] | Doostan, A.; Owhadi, H., A non-adapted sparse approximation of PDEs with stochastic inputs, J. Comput. Phys., 230, 8, 3015-3034 (2011) · Zbl 1218.65008 |
| [34] | Sargsyan, K.; Safta, C.; Najm, H. N.; Debusschere, B. J.; Ricciuto, D.; Thornton, P., Dimensionality reduction for complex models via Bayesian compressive sensing, Int. J. Uncertain. Quantif., 4, 1, 63-93 (2014) · Zbl 1513.65004 |
| [35] | Hampton, J.; Doostan, A., Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies, J. Comput. Phys., 280, 363-386 (2015) · Zbl 1349.94110 |
| [36] | Constantine, P. G.; Dow, E.; Wang, Q., Active subspace methods in theory and practice: applications to kriging surfaces, SIAM J. Sci. Comput., 36, 4, A1500-A1524 (2014) · Zbl 1311.65008 |
| [37] | Constantine, P. G., Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies, Vol. 2 (2015), SIAM · Zbl 1431.65001 |
| [38] | Tipireddy, R.; Ghanem, R., Basis adaptation in homogeneous chaos spaces, J. Comput. Phys., 259, 304-317 (2014) · Zbl 1349.60058 |
| [39] | Thimmisetty, C.; Tsilifis, P.; Ghanem, R., Homogeneous chaos basis adaptation for design optimization under uncertainty: Application to the oil well placement problem, (AI EDAM) Artif. Intell. Eng. Des. Anal. Manuf., 31, 3, 265-276 (2017) |
| [40] | Ghauch, Z. G.; Aitharaju, V.; Rodgers, W. R.; Pasupuleti, P.; Dereims, A.; Ghanem, R. G., Integrated stochastic analysis of fiber composites manufacturing using adapted polynomial chaos expansions, Composites A, 118, 179-193 (2019) |
| [41] | Zeng, X.; Red-Horse, J.; Ghanem, R., Accelerated basis adaptation in homogeneous chaos spaces, Comput. Methods Appl. Mech. Engrg., 386, Article 114109 pp. (2021) · Zbl 1507.65036 |
| [42] | Cai, J.; Luo, J.; Wang, S.; Yang, S., Feature selection in machine learning: A new perspective, Neurocomputing, 300, 70-79 (2018) |
| [43] | Murphy, K. P., Machine Learning: A Probabilistic Perspective (2012), MIT Press · Zbl 1295.68003 |
| [44] | Howley, T.; Madden, M. G.; O’Connell, M.-L.; Ryder, A. G., The effect of principal component analysis on machine learning accuracy with high dimensional spectral data, (International Conference on Innovative Techniques and Applications of Artificial Intelligence (2005), Springer), 209-222 |
| [45] | Reddy, G. T.; Reddy, M. P.K.; Lakshmanna, K.; Kaluri, R.; Rajput, D. S.; Srivastava, G.; Baker, T., Analysis of dimensionality reduction techniques on big data, IEEE Access, 8, 54776-54788 (2020) |
| [46] | McInnes, L.; Healy, J.; Melville, J., Umap: Uniform manifold approximation and projection for dimension reduction (2018), arXiv preprint arXiv:1802.03426 |
| [47] | Lin, T.; Zha, H., Riemannian manifold learning, IEEE Trans. Pattern Anal. Mach. Intell., 30, 5, 796-809 (2008) |
| [48] | Han, K.; Wang, Y.; Zhang, C.; Li, C.; Xu, C., Autoencoder inspired unsupervised feature selection, (2018 IEEE International Conference on Acoustics, Speech and Signal Processing. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2018), IEEE), 2941-2945 |
| [49] | P. Baldi, Autoencoders, unsupervised learning, and deep architectures, in: Proceedings of ICML Workshop on Unsupervised and Transfer Learning, in: JMLR Workshop and Conference Proceedings, 2012, pp. 37-49. |
| [50] | Friedman, J. H.; Tukey, J. W., A projection pursuit algorithm for exploratory data analysis, IEEE Trans. Comput., 100, 9, 881-890 (1974) · Zbl 0284.68079 |
| [51] | Huber, P. J., Projection pursuit, Ann. Statist., 435-475 (1985) · Zbl 0595.62059 |
| [52] | Friedman, J. H., Exploratory projection pursuit, J. Amer. Statist. Assoc., 82, 397, 249-266 (1987) · Zbl 0664.62060 |
| [53] | Lee, E.-K.; Cook, D.; Klinke, S.; Lumley, T., Projection pursuit for exploratory supervised classification, J. Comput. Graph. Statist., 14, 4, 831-846 (2005) |
| [54] | Grochowski, M.; Duch, W., Projection pursuit constructive neural networks based on quality of projected clusters, (International Conference on Artificial Neural Networks (2008), Springer), 754-762 |
| [55] | Barcaru, A., Supervised projection pursuit-A dimensionality reduction technique optimized for probabilistic classification, Chemometr. Intell. Lab. Syst., 194, Article 103867 pp. (2019) |
| [56] | Grear, T.; Avery, C.; Patterson, J.; Jacobs, D. J., Molecular function recognition by supervised projection pursuit machine learning, Sci. Rep., 11, 1, 1-15 (2021) |
| [57] | Olivier, A.; Shields, M. D.; Graham-Brady, L., Bayesian neural networks for uncertainty quantification in data-driven materials modeling, Comput. Methods Appl. Mech. Engrg., 386, Article 114079 pp. (2021) · Zbl 1507.65021 |
| [58] | Yang, L.; Meng, X.; Karniadakis, G. E., B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data, J. Comput. Phys., 425, Article 109913 pp. (2021) · Zbl 07508507 |
| [59] | Leibig, C.; Allken, V.; Ayhan, M. S.; Berens, P.; Wahl, S., Leveraging uncertainty information from deep neural networks for disease detection, Sci. Rep., 7, 1, 1-14 (2017) |
| [60] | Friedman, J. H.; Stuetzle, W., Projection pursuit regression, J. Amer. Statist. Assoc., 76, 376, 817-823 (1981) |
| [61] | Hastie, T.; Tibshirani, R.; Friedman, J. H.; Friedman, J. H., The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Vol. 2 (2009), Springer · Zbl 1273.62005 |
| [62] | Qianjian, G.; Jianguo, Y., Application of projection pursuit regression to thermal error modeling of a CNC machine tool, Int. J. Adv. Manuf. Technol., 55, 5, 623-629 (2011) |
| [63] | Ferraty, F.; Goia, A.; Salinelli, E.; Vieu, P., Functional projection pursuit regression, Test, 22, 2, 293-320 (2013) · Zbl 1367.62117 |
| [64] | Durocher, M.; Chebana, F.; Ouarda, T. B., A nonlinear approach to regional flood frequency analysis using projection pursuit regression, J. Hydrometeorol., 16, 4, 1561-1574 (2015) |
| [65] | Cui, H.-Y.; Zhao, Y.; Chen, Y.-N.; Zhang, X.; Wang, X.-Q.; Lu, Q.; Jia, L.-M.; Wei, Z.-M., Assessment of phytotoxicity grade during composting based on EEM/PARAFAC combined with projection pursuit regression, J. Hard Mater., 326, 10-17 (2017) |
| [66] | Rosenblatt, M., Remarks on a multivariate transformation, Ann. Math. Stat., 23, 3, 470-472 (1952) · Zbl 0047.13104 |
| [67] | Janson, S., Gaussian Hilbert Spaces (1997), Cambridge University Press · Zbl 0887.60009 |
| [68] | Cohen, A.; Migliorati, G., Optimal weighted least-squares methods, SMAI J. Comput. Math., 3, 181-203 (2017) · Zbl 1416.62177 |
| [69] | Schwab, C.; Gittelson, C. J., Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs, Acta Numer., 20, 291-467 (2011) · Zbl 1269.65010 |
| [70] | Tsilifis, P.; Ghanem, R. G., Reduced Wiener chaos representation of random fields via basis adaptation and projection, J. Comput. Phys., 341, 102-120 (2017) · Zbl 1378.65038 |
| [71] | Harper, W. V.; Gupta, S. K., Sensitivity/Uncertainty Analysis of a Borehole Scenario Comparing Latin Hypercube Sampling and Deterministic Sensitivity Approaches (1983), Office of Nuclear Waste Isolation, Battelle Memorial Institute Columbus: Office of Nuclear Waste Isolation, Battelle Memorial Institute Columbus Ohio |
| [72] | Morris, M. D.; Mitchell, T. J.; Ylvisaker, D., Bayesian design and analysis of computer experiments: use of derivatives in surface prediction, Technometrics, 35, 3, 243-255 (1993) · Zbl 0785.62025 |
| [73] | Gramacy, R. B.; Lian, H., Gaussian process single-index models as emulators for computer experiments, Technometrics, 54, 1, 30-41 (2012) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
