Recursive least mean \(p\)-power extreme learning machine. (English) Zbl 1434.68480
Summary: As real industrial processes have measurement samples with noises of different statistical characteristics and obtain the sample one by one usually, on-line sequential learning algorithms which can achieve better learning performance for systems with noises of various statistics are necessary. This paper proposes a new online Extreme Learning Machine (ELM) algorithm, namely recursive least mean \(p\)-power ELM (RLMP-ELM). In RLMP-ELM, a novel error criterion for cost function, namely the least mean \(p\)-power (LMP) error criterion, provides a mechanism to update the output weights sequentially. The LMP error criterion aims to minimize the mean \(p\)-power of the error that is the generalization of the mean square error criterion used in the ELM. The proposed on-line learning algorithm is able to provide on-line predictions of variables with noises of different statistics and obtains better performance than ELM and online sequential ELM (OS-ELM) while the non-Gaussian noises impact the processes. Simulations are reported to demonstrate the performance and effectiveness of the proposed methods.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68W27 | Online algorithms; streaming algorithms |
Keywords:
recursive least mean \(p\)-power; extreme learning machine; online sequential learning; non-Gaussian noises; alpha-stable noisesReferences:
| [2] | Bhotto, M. Z.A.; Antoniou, A., Robust recursive least-squares adaptive-filtering algorithm for impulsive-noise environments, Signal Processing Letters IEEE, 18, 3, 185-188 (2011) |
| [3] | Bouvet, M.; Schwartz, S. C., Comparison of adaptive and robust receivers for signal detection in ambient underwater noise, IEEE Transactions on Acoustics, Speech, and Signal Processing, 37, 5, 621-626 (1989) |
| [5] | Chambers, J. A.; Tanrikulu, O.; Constantinides, A. G., Least mean mixed-norm adaptive filtering, Electronics Letters, 30, 19, 1574-1575 (1994) |
| [6] | Chan, S. C.; Zou, Y. X., A recursive least m-estimate algorithm for robust adaptive filtering in impulsive noise: Fast algorithm and convergence performance analysis, IEEE Transactions on Signal Processing, 52, 4, 975-991 (2004) · Zbl 1369.94107 |
| [7] | Chang, P. C.; Fan, C. Y., A hybrid system integrating a wavelet and tsk fuzzy rules for stock price forecasting, IEEE Transactions on Systems, Man and Cybernetics Part C, 38, 6, 802-815 (2008) |
| [8] | Chen, B.; Liu, X.; Zhao, H.; Principe, J. C., Maximum correntropy kalman filter, Automatica, 76, 70-77 (2017) · Zbl 1352.93095 |
| [9] | Chen, B.; Wang, J.; Zhao, H.; Zheng, N., Convergence of a fixed-point algorithm under maximum correntropy criterion, IEEE Signal Processing Letters, 22, 10, 1723-1727 (2015) |
| [10] | Chen, B.; Xing, L.; Liang, J.; Zheng, N.; Príncipe, J. C., Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion, IEEE Signal Processing Letters, 21, 7, 880-884 (2014) |
| [11] | Chen, B.; Xing, L.; Wu, Z.; Liang, J.; Príncipe, J. C.; Zheng, N., Smoothed least mean p-power error criterion for adaptive filtering, Digital Signal Processing, 40, 154-163 (2015) |
| [12] | Chen, B.; Xing, L.; Zhao, H.; Zheng, N.; Príncipe, J. C., Generalized correntropy for robust adaptive filtering, IEEE Transactions on Signal Processing, 64, 13, 3376-3387 (2016) · Zbl 1414.94113 |
| [13] | Chen, B.; Zhao, S.; Zhu, P.; Príncipe, J. C., Quantized kernel least mean square algorithm, IEEE Transactions on Neural Networks and Learning Systems, 23, 1, 22-32 (2012) |
| [14] | Chen, B.; Zhao, S.; Zhu, P.; Príncipe, J. C., Quantized kernel recursive least squares algorithm, IEEE Transactions on Neural Networks and Learning Systems, 24, 9, 1484-1491 (2013) |
| [15] | Chen, B.; Zhu, Y.; Hu, J.; Principe, J. C., System parameter identification: information criteria and algorithms (2013), Elsevier |
| [17] | Deng, W.-Y.; Zheng, Q.-H.; Wang, Z.-M., Cross-person activity recognition using reduced kernel extreme learning machine, Neural Networks, 53, 1-7 (2014) |
| [18] | Diniz, P. S., Adaptive filtering: algorithms and practical implementation (2008), Kluwer Academic Publishers · Zbl 1155.93002 |
| [19] | Ferrari, S.; Stengel, R. F., Smooth function approximation using neural networks, IEEE Transactions on Neural Networks, 16, 1, 24-38 (2005) |
| [20] | Golestaneh, F.; Pinson, P.; Gooi, H. B., Very short-term nonparametric probabilistic forecasting of renewable energy generation - with application to solar energy, IEEE Transactions on Power Systems, 1-14 (2016) |
| [21] | Horata, P.; Chiewchanwattana, S.; Sunat, K., Robust extreme learning machine, Neurocomputing, 102, 31-44 (2013) |
| [22] | Hou, M.; Han, X., Constructive approximation to multivariate function by decay rbf neural network, IEEE Transactions on Neural Networks, 21, 9, 1517-1523 (2010) |
| [23] | Huang, G.-B.; Chen, L.; Siew, C.-K., Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Transactions on Neural Networks, 17, 4, 879-892 (2006) |
| [25] | Huang, G.-B.; Zhu, Q.-Y.; Siew, C.-K., Extreme learning machine: Theorey and applications, Neurocomputing, 70, 1, 489-501 (2006) |
| [26] | Ilow, J.; Hatzinakos, D.; Venetsanopoulos, A. N., Performance of fh ss radio networks with interference modeled as a mixture of gaussian and alpha-stable noise, IEEE Transactions on Communications, 46, 4, 509-520 (1998) |
| [28] | LeCun, Y.; Bottou, L.; Orr, G. B.; Müller, K. R., Efficient BackProp, (Neural networks: tricks of the trade (1998), Springer Berlin, Heidelberg: Springer Berlin, Heidelberg Berlin, Heidelberg), 9-50, (Chapter) |
| [29] | Lee, J.; Tepedelenlioglu, C., Distributed detection in coexisting large-scale sensor networks, IEEE Sensors Journal, 14, 4, 1028-1034 (2014) |
| [30] | Li, Y.; Jia, Z.; Li, X., Task scheduling based on weather forecast in energy harvesting sensor systems, IEEE Sensors Journal, 14, 14, 3763-3765 (2014) |
| [31] | Liang, N.-Y.; Huang, G.-B.; Saratchandran, P.; Sundararajan, N., A fast and accurate online sequential learning algorithm for feedforward networks, IEEE Transactions on Neural Networks, 17, 6, 1411-1423 (2006) |
| [32] | Liao, S.; Chung, A. C.S., Feature based nonrigid brain mr image registration with symmetric alpha stable filters, IEEE Transactions on Medical Imaging, 29, 1, 106-119 (2010) |
| [33] | Lim, J.-s.; Lee, S.; Pang, H.-S., Low complexity adaptive forgetting factor for online sequential extreme learning machine (os-elm) for application to nonstationary system estimations, Neural Computing and Applications, 22, 3, 569-576 (2013) |
| [35] | MacKay, D. J., A practical Bayesian framework for backpropagation networks, Neural Computation, 4, 3, 448-472 (1992) |
| [36] | Man, Z.; Lee, K.; Wang, D.; Cao, Z.; Khoo, S., Robust single-hidden layer feedforward network-based pattern classifier, IEEE Transactions on Neural Networks and Learning Systems, 23, 12, 1974-1986 (2012) |
| [37] | Man, Z.; Lee, K.; Wang, D.; Cao, Z.; Miao, C., A new robust training algorithm for a class of single-hidden layer feedforward neural networks, Neurocomputing, 74, 16, 2491-2501 (2011) |
| [38] | Mao, G., A timescale decomposition approach to network traffic prediction, IEICE Transactions on Communications, E88B, 10, 3974-3981 (2005) |
| [40] | Meir, R.; Maiorov, V. E., On the optimality of neural-network approximation using incremental algorithms, IEEE Transactions on Neural Networks, 11, 2, 323-337 (2000) |
| [41] | Nikias, C. L.; Shao, M., Signal processing with alpha-stable distributions and applications (1995), Wiley-Interscience: Wiley-Interscience New York, NY, USA |
| [42] | Pei, S.-C.; Tseng, C.-C., Least mean p-power error criterion for adaptive FIR filter, IEEE Journal on Selected Areas in Communications, 12, 9, 1540-1547 (1994) |
| [43] | Pei, S.; Tseng, C., Least mean p-power error criterion for adaptive FIR filter, IEEE Journal on Selected Areas in Communications, 12, 9, 1540-1547 (1994) |
| [44] | Qin, P.; Nishii, R.; Yang, Z. J., Selection of narx models estimated using weighted least squares method via gic-based method and l 1-norm regularization methods, Nonlinear Dynamics, 70, 3, 1831-1846 (2012) |
| [45] | Rong, H. J.; Huang, G. B.; Sundararajan, N.; Saratchandran, P., Online sequential fuzzy extreme learning machine for function approximation and classification problems, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39, 4, 1067-1072 (2009) |
| [46] | Samorodnitsky, G.; Taqqu, M. S., Stable non-gaussian random processes: Stochastic models with infinite variance, Journal of the American Statistical Association, 90, 430 (1996) |
| [47] | Sapankevych, N. I.; Sankar, R., Time series prediction using support vector machines: A survey, IEEE Computational Intelligence Magazine, 4, 2, 24-38 (2009) |
| [48] | Shao, M.; Nikias, C. L., Signal processing with fractional lower order moments: stable processes and their applications, Proceedings of the IEEE, 81, 7, 986-1010 (1993) |
| [49] | Shi, Z.; Han, M., \( \gamma - C\) plane and robustness in static reservoir for nonlinear regression estimation, Neurocomputing, 72, 7-9, 1732-1743 (2009) |
| [50] | Soares, S. G.; Araújo, R., An adaptive ensemble of on-line extreme learning machines with variable forgetting factor for dynamic system prediction, Neurocomputing, 171, 693-707 (2016) |
| [51] | Sun, S.; Zhang, C.; Yu, G., A Bayesian network approach to traffic flow forecasting, IEEE Transactions on Intelligent Transportation Systems, 7, 1, 124-132 (2006) |
| [53] | Walach, E.; Widrow, B., The least mean fourth (lmf) adaptive algorithm and its family, IEEE Transactions on Information Theory, 30, 2, 275-283 (1984) |
| [54] | Wang, X.; Han, M., Online sequential extreme learning machine with kernels for nonstationary time series prediction, Neurocomputing, 145, 90-97 (2014) |
| [55] | Wen, F., Diffusion least-mean p-power algorithms for distributed estimation in alpha-stable noise environments, Electronics Letters, 49, 21, 1355-1356 (2013) |
| [56] | Xiao, Y. T.Y.; Shida, K., Adaptive algorithm based on least mean p-power error criterion for fourier analysis in additive noise, IEEE Transactions on Signal Processing, 47, 4, 1172-1181 (1999) |
| [57] | Xiao, Y.; Tadokoro, Y.; Shida, K., Adaptive algorithm based on least mean p-power error criterion for fourier analysis in additive noise, IEEE Transactions on Signal Processing, 47, 4, 1172-1181 (1999) |
| [59] | Yang, J.; Shi, Y.; Rong, H.-J., Random neural q-learning for obstacle avoidance of a mobile robot in unknown environments, Advances in Mechanical Engineering, 8, 7 (2016) |
| [60] | Ye, Y.; Squartini, S.; Piazza, F., Online sequential extreme learning machine in nonstationary environments, Neurocomputing, 116, 94-101 (2013) |
| [61] | Zha, D., Robust multiuser detection method based on least p -norm state space criterion, Wireless Personal Communications An International Journal, 40, 2, 191-204 (2006) |
| [62] | Zha, D., Robust multiuser detection method based on least p-norm state space criterion, Wireless Personal Communications, 40, 2, 191-204 (2007) |
| [63] | Zhang, X. D., Matrix analysis and applications, 59-64 (2004) |
| [64] | Zhong, X.; Premkumar, A. B.; Madhukumar, A. S., Particle filtering for acoustic source tracking in impulsive noise with alpha-stable process, IEEE Sensors Journal, 13, 2, 589-600 (2013) |
| [65] | Zimmermann, M.; Dostert, K., Analysis and modeling of impulsive noise in broad-band powerline communications, IEEE Transactions on Electromagnetic Compatibility, 44, 1, 249-258 (2002) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
