Cross-validation based weights and structure determination of Chebyshev-polynomial neural networks for pattern classification. (English) Zbl 1373.68362
Summary: This paper first proposes a new type of single-output Chebyshev-polynomial feed-forward neural network (SOCPNN) for pattern classification. A new type of multi-output Chebyshev-polynomial feed-forward neural network (MOCPNN) is then proposed based on such an SOCPNN. Compared with multi-layer perceptron, the proposed SOCPNN and MOCPNN have lower computational complexity and superior performance, substantiated by both theoretical analyses and numerical verifications. In addition, two weight-and-structure-determination (WASD) algorithms, one for the SOCPNN and another for the MOCPNN, are proposed for pattern classification. These WASD algorithms can determine the weights and structures of the proposed neural networks efficiently and automatically. Comparative experimental results based on different real-world classification datasets with and without added noise prove that the proposed SOCPNN and MOCPNN have high accuracy, and that the MOCPNN has strong robustness in pattern classification when equipped with WASD algorithms.
MSC:
| 68T10 | Pattern recognition, speech recognition |
| 33C45 | Orthogonal polynomials and functions of hypergeometric type (Jacobi, Laguerre, Hermite, Askey scheme, etc.) |
| 68T05 | Learning and adaptive systems in artificial intelligence |
References:
| [1] | Nakayama, H.; Kagaku, N., Pattern classification by linear goal programming and its extensions, J. Global Optim., 12, 2, 111-126 (1998) · Zbl 0907.90277 |
| [2] | Duda, R.; Hart, P.; Stork, D., Pattern Classification and Scene AnalysisPart I. Pattern Classification (1995), John Wiley & Sons, Inc.: John Wiley & Sons, Inc. New York |
| [3] | Zhang, G. P., Neural networks for classificationa survey, IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev., 30, 4, 451-462 (2000) |
| [4] | Zhang, H.; Liu, J.; Ma, D.; Wang, Z., Data-core-based fuzzy min-max neural network for pattern classification, IEEE Trans. Neural Netw., 22, 12, 2339-2352 (2011) |
| [5] | Jiang, X.; Wah, A. H.K. S., Constructing and training feed-forward neural networks for pattern classification, Pattern Recognit., 36, 4, 853-867 (2003) |
| [6] | Ou, G.; Murphey, Y. L., Multi-class pattern classification using neural networks, Pattern Recognit., 40, 1, 4-18 (2007) · Zbl 1103.68777 |
| [7] | Paola, J. D.; Schowengerdt, R. A., A detailed comparison of backpropagation neural network and maximum-likelihood classifiers for urban land use classification, IEEE Trans. Geosci. Remote Sens., 33, 4, 981-996 (1995) |
| [8] | Rumelhart, D. E.; McClelland, J. L., Parallel Distributed Processing (1986), MIT Press: MIT Press Cambridge |
| [9] | Zhou, W., Verification of the nonparametric characteristics of backpropagation neural networks for image classification, IEEE Trans. Geosci. Remote Sens., 37, 2, 771-779 (1999) |
| [11] | Yu, X.; Chen, G., On the local minima free condition of backpropagation learning, IEEE Trans. Neural Netw., 6, 5, 1300-1303 (1995) |
| [12] | Zhang, Y.; Tan, N., Weights direct determination of feedforward neural networks without iterative BP-training, (Wang, L.; Hong, T., Intelligent Soft Computation and Evolving Data Mining (2010), IGI Globa: IGI Globa Hershey), 197-225 |
| [13] | Zhang, Y.; Yu, X.; Xiao, L.; Li, W.; Fan, Z., Weights and structure determination of artificial neuronets, (Zhang, W. J., Self-Organization: Theories and Methods (2013), Nova Science Publishers: Nova Science Publishers New York), 109-153, (Chapter 5) |
| [15] | Smyth, G. K., Polynomial approximation, (Armitage, P.; Colton, T., Encyclopedia of Biostatistics Second Edition (2004), Wiley: Wiley London), 4136-4140 |
| [16] | Caporale, G.; Cerrato, M., Using Chebyshev polynomials to approximate partial differential equations, Comput. Econ., 35, 3, 235-244 (2010) · Zbl 1182.93119 |
| [17] | Lee, T.; Jeng, J., The Chebyshev-polynomials-based unified model neural networks for function approximation, IEEE Trans. Syst. Man Cybern. Part B: Cybern., 28, 6, 925-935 (1998) |
| [20] | Patra, J.; Kot, A., Nonlinear dynamic system identification using Chebyshev functional link artificial neural networks, IEEE Trans. Syst. Man Cybern. Part B: Cybern., 32, 4, 505-511 (2002) |
| [21] | D׳Ambrosio, L., Extension of Bernstein polynomials to infinite dimensional case, J. Approx. Theory, 140, 2, 191-202 (2006) · Zbl 1098.41016 |
| [23] | Leung, F.; Lam, H.; Ling, S.; Tam, P., Tuning of the structure and parameters of a neural network using an improved genetic algorithm, IEEE Trans. Neural Netw., 14, 1, 79-88 (2003) |
| [24] | Svozil, D.; Kvasnicka, V.; Pospichal, J., Introduction to multi-layer feed-forward neural networks, Chemom. Intell. Lab. Syst., 39, 43-62 (1997) |
| [25] | Huang, G.; Zhun, Q.; Siew, C., Extreme learning machinetheory and applications, Neurocomputing, 70, 489-501 (2006) |
| [26] | Fujita, O., Statistical estimation of the number of hidden units for feedforward neural networks, Neural Netw., 11, 5, 851-859 (1998) |
| [27] | Teoh, E. J.; Tan, K. C.; Xiang, C., Estimating the number of hidden neurons in a feedforward network using the singular value decomposition, IEEE Trans. Neural Netw., 17, 6, 1623-1629 (2006) |
| [28] | Zhang, R.; Lan, Y.; Huang, G.; Xu, Z., Universal approximation of extreme learning machine with adaptive growth of hidden nodes, IEEE Trans. Neural Netw. Learn. Syst., 23, 2, 365-371 (2012) |
| [29] | Zhang, Z.; Ma, X.; Yang, Y., Bounds on the number of hidden neurons in three-layer binary neural networks, Neural Netw., 16, 7, 995-1002 (2003) |
| [30] | Geisser, S., The predictive sample reuse method with applications, J. Am. Stat. Assoc., 70, 350, 320-328 (1975) · Zbl 0321.62077 |
| [31] | Arlot, S.; Celisse, A., A survey of cross-validation procedures for model selection, Stat. Surv., 4, 40-79 (2010) · Zbl 1190.62080 |
| [32] | Chen, S.; Wolfgang, A.; Harris, C. J.; Hanzo, L., Symmetric RBF classifier for nonlinear detection in multiple-antenna-aided systems, IEEE Trans. Neural Netw., 19, 5, 737-745 (2008) |
| [33] | Hong, X.; Chen, S., A new RBF neural network with boundary value constraints, IEEE Trans. Syst. Man Cybern. Part B: Cybern., 39, 1, 298-303 (2009) |
| [34] | Dunkl, C. F.; Xu, Y., Orthogonal Polynomials of Several Variables (2001), Cambridge University Press: Cambridge University Press Cambridge · Zbl 0964.33001 |
| [35] | Su, J.; Ruan, S., MATLAB Practical Guide (2008), Publishing House of Electronics Industry: Publishing House of Electronics Industry Beijing |
| [36] | Frank, A.; Asuncion, A., UCI Machine Learning Repository (2010), School of Information and Computer Science, University of California: School of Information and Computer Science, University of California Irvine, CA, 〈http://archive.ics.uci.edu/ml〉 |
| [38] | Hamm, Lonnie; Brorsen, B. W.; Hagan, Martin T., Comparison of stochastic global optimization methods to estimate neural network weights, Neural Process. Lett., 26, 3, 145-158 (2007) |
| [39] | Hagan, Martin T.; Demuth, Howard B.; Beale, Mark H., Neural Network Design (2002), China Machine Press: China Machine Press Beijing |
| [41] | Haykin, S. S., Neural NetworksA Comprehensive Foundation (1999), Prentice Hall International Editions Series: Prentice Hall International Editions Series London · Zbl 0934.68076 |
| [42] | Zhang, Y.; Li, K.; Tang, N., An RBF neural network classifier with centers, variances and weights directly determined, Comput. Technol. Autom., 28, 3, 5-9 (2009) |
| [43] | Chang, C.; Lin, C., LIBSVMa library for support vector machines, ACM Trans. Intell. Syst. Technol., 2, 3, 27:1-27:27 (2011), Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm |
| [45] | Chuang, C.; Su, S.; Jeng, J.; Hsiao, C., Robust support vector regression networks for function approximation with outliers, IEEE Trans. Neural Netw., 13, 6, 1322-1330 (2002) |
| [46] | Brazdil, P. B.; Soares, C., A comparison of ranking methods for classification algorithm selection, (Mantaras, R. L.; Plaza, E., Machine Learning: ECML 2000 (2000), Springer: Springer Berlin, Heidelberg), 63-75 |
| [47] | Zhang, Y.; Yang, Y.; Cai, B.; Guo, D., Zhang neural network and its application to Newton iteration for matrix square root estimation, Neural Comput. Appl., 21, 3, 453-460 (2012) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
