A new approach for function approximation incorporating adaptive particle swarm optimization and a priori information. (English) Zbl 1157.65323
Summary: A new approach coupling adaptive particle swarm optimization (APSO) and a priori information for function approximation problem is proposed to obtain better generalization performance and faster convergence rate. It is well known that gradient-based learning algorithms such as backpropagation (BP) algorithm have good ability of local search, whereas PSO has good ability of global search.
Therefore, in the new approach, first, APSO encoding the first-order derivative information of the approximated function is applied to train network to near global minima. Second, with the connection weights produced by APSO, the network is trained with a gradient-based algorithm. Due to combining APSO with local search algorithm and considering a priori information, the new approach has better generalization performance and convergence rate than traditional learning ones. Finally, simulation results are given to verify the efficiency and effectiveness of the proposed approach.
Therefore, in the new approach, first, APSO encoding the first-order derivative information of the approximated function is applied to train network to near global minima. Second, with the connection weights produced by APSO, the network is trained with a gradient-based algorithm. Due to combining APSO with local search algorithm and considering a priori information, the new approach has better generalization performance and convergence rate than traditional learning ones. Finally, simulation results are given to verify the efficiency and effectiveness of the proposed approach.
MSC:
| 65D15 | Algorithms for approximation of functions |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 92B20 | Neural networks for/in biological studies, artificial life and related topics |
| 90C15 | Stochastic programming |
Keywords:
feedforward neural networks; function approximation; particle swarm optimization; a priori information; gradient-based learning algorithms; numerical examples; convergence; backpropagation algorithm; local search algorithmReferences:
| [1] | Yu, S. W.; Zhu, K. J.; Diao, F. Q., A dynamic all parameters adaptive BP neural networks model and its application on oil reservoir prediction, Applied Mathematics and Computation, 195, 66-75 (2008) · Zbl 1205.86035 |
| [2] | Huang, D. S., A constructive approach for finding arbitrary roots of polynomials by neural networks, IEEE Transactions on Neural Networks, 15, 477-491 (2004) |
| [3] | Huang, D. S.; Chi, Zheru, Finding roots of arbitrary high order polynomials based on neural network recursive partitioning method, Science in China Series F Information Sciences, 47, 232-245 (2004) · Zbl 1161.68723 |
| [4] | Huang, D. S.; Horace, H. S.Ip.; Chi, Zheru., A neural root finder of polynomials based on root moments, Neural Computation, 16, 1721-1762 (2004) · Zbl 1102.68560 |
| [5] | Huang, D. S.; Horace, H. S.Ip.; Chi, Zheru; Wong, H. S., Dilation method for finding close roots of polynomials based on constrained learning neural networks, Physics Letters A, 309, 443-451 (2003) · Zbl 1011.65024 |
| [6] | Han, Fei; Huang, D. S., Improved constrained learning algorithms by incorporating additional functional constraints into neural networks, Applied Mathematics and Computation, 174, 34T-50T (2006) · Zbl 1093.68080 |
| [7] | Poggio, F.; Girosi, Regularization algorithms for learning that are equivalent to multilayer networks, Science, 247, 2, 978-982 (1990) · Zbl 1226.92005 |
| [8] | Cottrell, M.; Girard, B., Neural modeling for time series: a statistical stepwise method for weight elimination, IEEE Transactions on Neural Networks, 6, 1355-1364 (1992) |
| [9] | Jeong, S. Y.; Lee, S. Y., Adaptive learning algorithms to incorporate additional functional constraints into neural networks, Neurocomputing, 35, 73-90 (2000) · Zbl 1003.68622 |
| [10] | Jeong, D.-G.; Lee, S.-Y., Merging back-propagation and Hebbian learning rules for robust classifications, Neural Networks, 9, 1213-1222 (1996) |
| [11] | Fei Han; Huang, D. S.; Yiu-Ming; Cheung; Guang-Bin Huang, A New Modified Hybrid Learning Algorithm for Feedforward Neural Networks. A New Modified Hybrid Learning Algorithm for Feedforward Neural Networks, Lecture Notes in Computer Science, vol. 3496 (2005), Springer-Verlag, (International Symposium on Neural Network, Chongqing, May 30-June 1, China), pp. 572-577 · Zbl 1082.68627 |
| [12] | Parrott, Daniel; Li, Xiaodong, Locating and tracking multiple dynamic optima by a particle swarm model using speciation, IEEE Transactions on Evolutionary Computation, 10, 440-458 (2006) |
| [13] | Blackwell, Tim; Branke, Jürgen, Multiswarms, exclusion, and anti-convergence in dynamic environments, IEEE Transactions on Evolutionary Computation, 10, 459-472 (2006) |
| [14] | Goldberg, D. E., Genetic Algorithms in Search, Optimization, Machine Learning (1989), Addison Wesley: Addison Wesley MA · Zbl 0721.68056 |
| [15] | Li, L. L.; Wang, L.; Liu, L. H., An effective hybrid PSOSA strategy for optimization and its application to parameter estimation, Applied Mathematics and Computation, 179, 135-146 (2006) · Zbl 1100.65052 |
| [16] | Yang, X. M.; Yuan, J. S.; Yuan, J. Y.; Mao, H. N., A modified particle swarm optimizer with dynamic adaptation, Applied Mathematics and Computation, 189, 1205-1213 (2007) · Zbl 1122.65364 |
| [17] | Funahashi, K., On the approximate realization of continuous mapping by neural networks, Neural Networks, 2, 3, 183-192 (1989) |
| [18] | Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal, approximators, Neural Networks, 2, 5, 359-366 (1989) · Zbl 1383.92015 |
| [19] | B. Irie, S. Miyake, Capabilities of three-layered perceptions, in: Proceedings of the IEEE Conference on Neural Networks, vol. I, 24-27 July, San Diego, CA, 1988, pp. 641-648.; B. Irie, S. Miyake, Capabilities of three-layered perceptions, in: Proceedings of the IEEE Conference on Neural Networks, vol. I, 24-27 July, San Diego, CA, 1988, pp. 641-648. |
| [20] | R.C. Eberhart, J. Kennedy. A new optimizer using particles swarm theory, in: Proceeding of Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 1995, pp. 39-43.; R.C. Eberhart, J. Kennedy. A new optimizer using particles swarm theory, in: Proceeding of Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 1995, pp. 39-43. |
| [21] | R.C. Eberhart, J. Kennedy, Particle swarm optimization, in: Proceeding of IEEE International Conference on Neural Network, Perth, Australia, 1995, pp. 1942-1948.; R.C. Eberhart, J. Kennedy, Particle swarm optimization, in: Proceeding of IEEE International Conference on Neural Network, Perth, Australia, 1995, pp. 1942-1948. |
| [22] | Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, in: Proceeding of IEEE World Conference on Computation Intelligence, 1998, pp. 69-73.; Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, in: Proceeding of IEEE World Conference on Computation Intelligence, 1998, pp. 69-73. |
| [23] | Y.H. Shi, R.C. Eberhart, Parameter selection in particle swarm optimization, in: 1998 Annual conference on Evolutionary Programming, San Diego, March 1998.; Y.H. Shi, R.C. Eberhart, Parameter selection in particle swarm optimization, in: 1998 Annual conference on Evolutionary Programming, San Diego, March 1998. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
