The design of self-organizing neural networks based on PNS and FPNs with the aid of genetic optimization and extended GMDH method. (English) Zbl 1102.68575
Summary: We introduce a class of neural architectures of Self-Organizing Neural Networks (SONN) that is based on a genetically optimized multilayer perceptron with polynomial neurons or fuzzy polynomial neurons (FPNs). We discuss its comprehensive design methodology involving mechanisms of genetic optimization, especially Genetic Algorithms (GAs). The conventional SONN is based on some mechanisms of self-organization and an evolutionary algorithm rooted in the extended group method of data handling method. In contrast, the proposed genetically optimized SONN (called “gSONN”, for brief) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one encountered in the conventional SONN. This structural optimization is realized via GAs whereas in the case of the parametric optimization we proceed with a standard least square method-based learning. Through the consecutive process of such structural and parametric optimization, the gSONN becomes generated in a highly dynamic fashion. The performance of the network is quantified through experimentation in which we exploit standard data already used in fuzzy or neurofuzzy modeling. These results highlight the superiority of the proposed networks over the existing fuzzy and neural models.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
Keywords:
self-organizing neural networks; polynomial neuron; fuzzy polynomial neuron; group method of data handling; multi-layer perceptron; genetic algorithms; design procedureReferences:
| [1] | Cherkassky, V.; Gehring, D.; Mulier, F., Comparison of adaptive methods for function estimation from samples, IEEE Transactions on Neural Networks, 7, July, 969-984 (1996) |
| [2] | Dickerson, J. A.; Kosko, B., Fuzzy function approximation with ellipsoidal rules, IEEE Transactions on Systems, Man and Cybernetics, Part B, 26, August, 542-560 (1996) |
| [3] | Sommer, V.; Tobias, P.; Kohl, D.; Sundgren, H.; Lundstrom, L., Neural networks and abductive networks for chemical sensor signals: a case comparison, Sensors and Actuators B, B28, 217-222 (1995) |
| [4] | Kleinsteuber, S.; Sepehri, N., A polynomial network modeling approach to a class of large-scale hydraulic systems, Computers and Electrical Engineering, 22, 151-168 (1996) |
| [5] | Cordon, O., Ten years of genetic fuzzy systems: current framework and new trends, Fuzzy Sets and Systems, 141, 1, 5-31 (2004) · Zbl 1050.93513 |
| [6] | Ivakhnenko, A. G., Polynomial theory of complex systems, IEEE Transactions on Systems, Man and Cybernetics, SMC-1, 364-378 (1971) |
| [7] | Ivakhnenko, A. G.; Madala, H. R., Inductive Learning Algorithms for Complex Systems Modeling (1994), CRC Press: CRC Press Boca Raton, Fl · Zbl 0858.93001 |
| [8] | Ivakhnenko, A. G.; Ivakhnenko, G. A., The review of problems solvable by algorithms of the group method of data handling (GMDH), Pattern Recognition and Image Analysis, 5, 4, 527-535 (1995) |
| [9] | Ivakhnenko, A. G.; Ivakhnenko, G. A.; Muller, J. A., Self-organization of neural networks with active neurons, Pattern Recognition and Image Analysis, 4, 2, 185-196 (1994) |
| [10] | Oh, S.-K.; Pedrycz, W., The design of self-organizing polynomial neural networks, Information Science, 141, 237-258 (2002) · Zbl 1008.68098 |
| [11] | Oh, S.-K.; Pedrycz, W.; Park, B.-J., Polynomial neural networks architecture: analysis and design, Computers and Electrical Engineering, 29, 6, 703-725 (2003) |
| [12] | Park, H.-S.; Park, B.-J.; Oh, S.-K., Optimal design of self-organizing polynomial neural networks by means of genetic algorithms, Journal of the Research Institute of Engineering Technology Development, 22, 111-121 (2002), (in Korean) |
| [13] | Oh, S.-K.; Pedrycz, W., Fuzzy polynomial neuron-based self-organizing neural networks, International Journal of General Systems, 32, 3, 237-250 (2003) · Zbl 1036.93039 |
| [14] | Pedrycz, W.; Reformat, M., Evolutionary optimization of fuzzy models in fuzzy logic: a framework for the new millennium, (Dimitrov, V.; Korotkich, V., Studies in Fuzziness and Soft Computing, vol. 8 (1996), Physica-Verlag: Physica-Verlag Wurzburg), 51-67 |
| [15] | Pedrycz, W., An identification algorithm in fuzzy relational system, Fuzzy Sets and Systems, 13, 153-167 (1984) · Zbl 0554.93070 |
| [16] | Oh, S.-K.; Pedrycz, W., Identification of Fuzzy Systems by means of an Auto-Tuning Algorithm and Its Application to Nonlinear Systems, Fuzzy Sets and Systems, 115, 2, 205-230 (2000) · Zbl 0965.93045 |
| [17] | Michalewicz, Z., Genetic Algorithms + Data Structures = Evolution Programs (1996), Springer-Verlag: Springer-Verlag Berlin, Heidelberg · Zbl 0841.68047 |
| [18] | Holland, J. H., Adaptation in Natural and Artificial Systems (1975), The University of Michigan Press: The University of Michigan Press Ann Arbour · Zbl 0317.68006 |
| [19] | K.A. De Jong, Are genetic algorithms function optimizers? in: R. Manner, B. Manderick (Eds.), Parallel Problem Solving from Nature, vol. 2, North-Holland, Amsterdam.; K.A. De Jong, Are genetic algorithms function optimizers? in: R. Manner, B. Manderick (Eds.), Parallel Problem Solving from Nature, vol. 2, North-Holland, Amsterdam. |
| [20] | Box, D. E.; Jenkins, G. M., Time Series Analysis, Forecasting and Control (1976), Holden Day: Holden Day California · Zbl 0363.62069 |
| [21] | Tong, R. M., The evaluation of fuzzy models derived from experimental data, Fuzzy Sets and Systems, 13, 1-12 (1980) · Zbl 0453.93003 |
| [22] | Sugeno, M.; Yasukawa, T., Linguistic modeling based on numerical data, IFSA 91, Brussels, Computer, Management and Systems Science, 264-267 (1991) |
| [23] | Sugeno, M.; Yasukawa, T., A fuzzy-logic-based approach to qualitative modeling, IEEE Transactions on Fuzzy Systems, 1, 1, 7-31 (1993) |
| [24] | Xu, C. W.; Zailu, Y., Fuzzy model identification self-learning for dynamic system, IEEE Transactions on Systems, Man and Cybernetics, SMC-17, 4, 683-689 (1987) · Zbl 0643.93018 |
| [25] | Chen, J. Q.; Xi, Y. G.; Zhang, Z. J., A clustering algorithm for fuzzy model identification, Fuzzy Sets and Systems, 98, 319-329 (1998) |
| [26] | Gomez-Skarmeta, A. F.; Delgado, M.; Vila, M. A., About the use of fuzzy clustering techniques for fuzzy model identification, Fuzzy Sets and Systems, 106, 179-188 (1999) |
| [27] | Kim, E.-T., A new approach to fuzzy modeling, IEEE Transactions on Fuzzy Systems, 5, 3, 328-337 (1997) |
| [28] | Kim, E.-T., A simple identified Sugeno-type fuzzy model via double clustering, Information Science, 110, 25-39 (1998) |
| [29] | Leski, J.; Czogala, E., A new artificial neural networks based fuzzy inference system with moving consequents in if-then rules and selected applications, Fuzzy Sets and Systems, 108, 289-297 (1999) · Zbl 0942.93022 |
| [30] | Lin, Y.; Cunningham, G. A., A new approach to fuzzy-neural modeling, IEEE Transactions on Fuzzy Systems, 3, 2, 190-197 (1995) |
| [31] | Wang, Y.; Rong, G., A self-organizing neural-network-based fuzzy system, Fuzzy Sets and Systems, 103, 1-11 (1999) |
| [32] | Park, H.-S.; Oh, S.-K.; Yoon, Y.-W., A new modeling approach to fuzzy-neural networks architecture, Journal of Control Automation and Systems Engineering, 7, 8, 664-674 (2001), (in Korean) |
| [33] | Oh, S.-K.; Kim, D.-W.; Park, B.-J., A study on the optimal design of polynomial neural networks structure, The Transactions of The Korean Institute of Electrical Engineers, 49D, 3, 145-156 (2000), (in Korean) |
| [34] | Oh, S.-K.; Pedrycz, W.; Kim, D.-W., Hybrid fuzzy polynomial neural networks, International Journal of Uncertainty Fuzziness and Knowledge-Based Systems, 10, 3, 257-280 (2002) · Zbl 1075.93020 |
| [35] | Oh, S.-K.; Kim, D.-W.; Park, B.-J.; Hwang, H.-S., Advanced polynomial neural networks architecture with new adaptive nodes, Transactions on Control, Automation and Systems Engineering, 3, 1, 43-50 (2001), (in Korean) |
| [36] | Park, B.-J.; Pedrycz, W.; Oh, S.-K., Fuzzy polynomial neural networks: hybrid architectures of fuzzy modeling, IEEE Transactions on Fuzzy Systems, 10, 5, 607-621 (2002) |
| [37] | Park, B.-J.; Oh, S.-K.; Jang, S.-W., The design of adaptive fuzzy polynomial neural networks architectures based on fuzzy neural networks and self-organizing networks, Journal of Control Automation and Systems Engineering, 8, 2, 126-135 (2002), (in Korean) |
| [38] | Park, B.-J.; Oh, S.-K., The analysis and design of advanced neurofuzzy polynomial networks, Journal of the Institute of Electronics Engineers of Korea, 39-CI, 3/5, 18-31 (2002), (in Korean) |
| [39] | Kim, D.-W.; Oh, S.-K.; Kim, H.-K., A study on the self-organizing fuzzy polynomial neural networks, Journal of Korean Institute of Electrical Engineers, 11, 2, 79-89 (2001), (in Korean) |
| [40] | Wang, L. X.; Mendel, J. M., Generating fuzzy rules from numerical data with applications, IEEE Transactions on Systems, Man and Cybernetics, 22, 6, 1414-1427 (1992) |
| [41] | R.S. Crowder III, Predicting the Mackey-Glass time series with cascade-correlation learning, in: D. Touretzky, G. Hinton, T. Sejnowski (Eds.), Proceedings of the 1990 Connectionist Models Summer School, Carnegic Mellon University, 1990, pp. 117-123.; R.S. Crowder III, Predicting the Mackey-Glass time series with cascade-correlation learning, in: D. Touretzky, G. Hinton, T. Sejnowski (Eds.), Proceedings of the 1990 Connectionist Models Summer School, Carnegic Mellon University, 1990, pp. 117-123. |
| [42] | Jang, J. S.R., ANFIS: Adaptive-network-based fuzzy inference system, IEEE Transactions on Systems, Man and Cybernetics, 23, 3, 665-685 (1993) |
| [43] | Maguire, L. P.; Roche, B.; McGinnity, T. M.; McDaid, L. J., Predicting a chaotic time series using a fuzzy neural network, Information Sciences, 112, 125-136 (1998) · Zbl 0953.68590 |
| [44] | James Li, C.; Huang, T.-Y., Automatic structure and parameter training methods for modeling of mechanical systems by recurrent neural networks, Applied Mathematical Modeling, 23, 933-944 (1999) · Zbl 1062.70614 |
| [45] | Oh, S.-K.; Pedrycz, W.; Ahn, T.-C., Self-organizing neural networks with fuzzy polynomial neurons, Applied Soft Computing, 2, 1F, 1-10 (2002) |
| [46] | A.S. Lapedes, R. Farber, Non-linear signal processing using neural networks: prediction and system modeling, Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, 1987.; A.S. Lapedes, R. Farber, Non-linear signal processing using neural networks: prediction and system modeling, Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, 1987. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
