A hybrid Bayesian back-propagation neural network approach to multivariate modelling. (English) Zbl 1118.74330
Summary: There is growing interest in the use of back-propagation neural networks to model nonlinear multivariate problems in geotechnical engineering. To overcome the shortcomings of the conventional back-propagation neural network, such as overfitting, where the neural network learns the spurious details and noise in the training examples, a hybrid back-propagation algorithm has been developed. The method utilizes the genetic algorithms search technique and the Bayesian neural network methodology. The genetic algorithms enhance the stochastic search to locate the global minima for the neural network model. The Bayesian inference procedures essentially provide better generalization and a statistical approach to deal with data uncertainty in comparison with the conventional back-propagation. The uncertainty of data can be indicated using error bars. Two examples are presented to demonstrate the convergence and generalization capabilities of this hybrid algorithm.
MSC:
| 74L10 | Soil and rock mechanics |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 92B20 | Neural networks for/in biological studies, artificial life and related topics |
Keywords:
back-propagation neural network; Bayesian neural network; genetic algorithms; neural network; nonlinear modelling; piling; skin frictionReferences:
| [1] | Goh, Journal of Geotechnical Engineering 120 pp 1467– (1994) |
| [2] | Juang, Canadian Geotechnical Journal 36 pp 443– (1999) |
| [3] | Goh, Geotechnique 45 pp 709– (1995) |
| [4] | Intelligent hybrid system for the design of pile foundations. In New Technological and Design Developments in Deep Foundations, (eds). ASCE Geotechnical Special Publication No. 100: New York, 2000; 312-326. · doi:10.1061/40511(288)22 |
| [5] | Ghaboussi, Journal of Engineering Mechanics 117 pp 132– (1991) |
| [6] | Ellis, Journal of Geotechnical Engineering 121 pp 429– (1995) |
| [7] | Characterizing the 3-D stress-strain behavior of sandy soils. A neuro-mechanistic approach. In Numerical Methods in Geotechnical Engineering, (eds). ASCE Geotechnical Special Publication No. 96: New York, 2000; 43-57. · doi:10.1061/40502(284)4 |
| [8] | Learning internal representation by error propagation. In Parallel Distributed Processing, (eds). MIT Press: Cambridge, 1986; 1:318-362. |
| [9] | Hornik, Neural Networks 4 pp 251– (1991) |
| [10] | Naturally Intelligent Systems. MIT Press: Cambridge, 1991. |
| [11] | Neural Network PC tools: A Practical Guide. Academic Press: San Diego, 1990. |
| [12] | Neural Networks for Pattern Recognition. Oxford University Press: Oxford, 1995. |
| [13] | Fundamentals of Artificial Neural Networks. MIT Press: Cambridge, 1995. · Zbl 0850.68271 |
| [14] | Masri, ASCE Journal of Engineering Mechanics 125 pp 123– (1999) |
| [15] | Gori, Transactions on Pattern Analysis and Machine Learning 14 pp 76– (1992) |
| [16] | Bayesian methods for adaptive models. PhD Thesis, California Institute of Technology, 1991. |
| [17] | Bayesian training of back-propagation networks by the hybrid Monte Carlo method. Technical Report CRG-TG-92-1, Department of Computer Science, University of Toronto, 1992. |
| [18] | Bayesian Inference in Statistical Analysis Addison-Wesley: Reading, 1973. |
| [19] | Bayesian inductive inference and maximum entropy. In Maximum Entropy and Bayesian Methods in Science and Engrg., Foundations, vol. 1, (eds). Kluwer: Norwell 1988; 53-74. · doi:10.1007/978-94-009-3049-0_4 |
| [20] | Mackay, Neural Computation 4 pp 415– (1992) |
| [21] | Thodberg, Transactions on Neural Networks 7 pp 56– (1996) |
| [22] | Mackay, ASHRAE Transactions 100 pp 1053– (1995) |
| [23] | Adaptation in Natural and Artificial Systems. University of Michigan Press: Ann Arbor, 1975. |
| [24] | Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley: Reading, MA, 1989. · Zbl 0721.68056 |
| [25] | Evolutionary Bayesian Back-propagation?an artificial neural network program. Geotechnical Research Report No. NTU/GT/2001-1, Nanyang Technological University: Singapore, 2001. |
| [26] | An analysis of the behavior of a class of genetic adaptive systems. Ph.D Thesis, Department of Computer and Communications Sciences, University of Michigan, Ann Arbor, 1975. |
| [27] | Nguyen, Proceedings of the International Joint Conference on Neural Networks 3 pp 21– (1990) |
| [28] | Metroplis, Journal of Chemical Physics 21 pp 1087– (1953) |
| [29] | Automatic Generation of Neural Network Architecture using Evolutionary Computation. World Scientific: Singapore, 1997. · Zbl 0942.68105 |
| [30] | Finite Element Procedures. Prentice-Hall: NJ, 1996. |
| [31] | Choueiki, Transactions on Neural Networks 10 pp 56– (1999) |
| [32] | Adhesion of piles driven in clay soils. Proceedings of the 4th International Conference on Soil Mechanics and Foundation Engineering, vol. 2, London, 1957; 66-71. |
| [33] | Shaft capacity of driven piles in clay. Proceedings of the 17th Offshore Technology Conference, Houston, 1985; 371-378. |
| [34] | Semple, Ground Engineering 19 pp 11– (1986) |
| [35] | A new way to predict capacity of piles in clay. Proceedings of the 4th Offshore Technology Conference, vol. 2, Houston, 1972; 865-874. |
| [36] | Side friction of piles in clay. Proceedings of the 9th International Conference on Soil Mechanics and Foundation Engineering, vol. 1, Tokyo, 1977; 517-522. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
