Online sequential extreme learning machine with generalized regularization and adaptive forgetting factor for time-varying system prediction. (English) Zbl 1428.62381
Summary: Many real world applications are of time-varying nature and an online learning algorithm is preferred in tracking the real-time changes of the time-varying system. Online sequential extreme learning machine (OSELM) is an excellent online learning algorithm, and some improved OSELM algorithms incorporating forgetting mechanism have been developed to model and predict the time-varying system. But the existing algorithms suffer from a potential risk of instability due to the intrinsic ill-posed problem; besides, the adaptive tracking ability of these algorithms for complex time-varying system is still very weak. In order to overcome the above two problems, this paper proposes a novel OSELM algorithm with generalized regularization and adaptive forgetting factor (AFGR-OSELM). In the AFGR-OSELM, a new generalized regularization approach is employed to replace the traditional exponential forgetting regularization to make the algorithm have a constant regularization effect; consequently the potential
ill-posed problem of the algorithm can be completely avoided and a persistent stability can be guaranteed. Moreover, the AFGR-OSELM adopts an adaptive scheme to adjust the forgetting factor dynamically and automatically in the online learning process so as to better track the dynamic changes of the time-varying system and reduce the adverse effects of the outdated data in time; thus it tends to provide desirable prediction results in time-varying environment. Detailed performance comparisons of AFGR-OSELM with other representative algorithms are carried out using artificial and real world data sets. The experimental results show that the proposed AFGR-OSELM has higher prediction accuracy with better stability than its counterparts for predicting time-varying system.
MSC:
| 62M09 | Non-Markovian processes: estimation |
| 62M20 | Inference from stochastic processes and prediction |
| 68T05 | Learning and adaptive systems in artificial intelligence |
References:
| [1] | Pérez-Sánchez, B.; Fontenla-Romero, O.; Guijarro-Berdiñas, B.; Martínez-Rego, D., An online learning algorithm for adaptable topologies of neural networks, Expert Systems with Applications, 40, 18, 7294-7304 (2013) · doi:10.1016/j.eswa.2013.06.066 |
| [2] | Qiao, J.; Zhang, Z.; Bo, Y., An online self-adaptive modular neural network for time-varying systems, Neurocomputing, 125, 7-16 (2014) · doi:10.1016/j.neucom.2012.09.038 |
| [3] | Lughofer, E., On-line active learning: A new paradigm to improve practical useability of data stream modeling methods, Information Sciences, 415-416, 356-376 (2017) · doi:10.1016/j.ins.2017.06.038 |
| [4] | Zhang, Q.; Zhang, P.; Long, G.; Ding, W.; Zhang, C.; Wu, X., Online learning from trapezoidal data streams, IEEE Transactions on Knowledge and Data Engineering, 28, 10, 2709-2723 (2016) · doi:10.1109/TKDE.2016.2563424 |
| [5] | Zhou, Z.; Zheng, W.-S.; Hu, J.-F.; Xu, Y.; You, J., One-pass online learning: A local approach, Pattern Recognition, 51, 346-357 (2016) · Zbl 1394.68333 · doi:10.1016/j.patcog.2015.09.003 |
| [6] | Liang, N.; Huang, G.; Saratchandran, P.; Sundararajan, N., A fast and accurate online sequential learning algorithm for feedforward networks, IEEE Transactions on Neural Networks and Learning Systems, 17, 6, 1411-1423 (2006) · doi:10.1109/TNN.2006.880583 |
| [7] | Huang, G. B.; Zhu, Q. Y.; Siew, C. K., Extreme learning machine: theory and applications, Neurocomputing, 70, 1-3, 489-501 (2006) · doi:10.1016/j.neucom.2005.12.126 |
| [8] | Ye, Y.; Squartini, S.; Piazza, F., Online sequential extreme learning machine in nonstationary environments, Neurocomputing, 116, 94-101 (2013) · doi:10.1016/j.neucom.2011.12.064 |
| [9] | Lu, X.; Zhou, C.; Huang, M.; Lv, W., Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system, Neurocomputing, 174, 617-626 (2016) · doi:10.1016/j.neucom.2015.09.068 |
| [10] | Yin, J. C.; Zou, Z. J.; Xu, F.; Wang, N. N., Online ship roll motion prediction based on grey sequential extreme learning machine, Neurocomputing, 129, 168-174 (2014) · doi:10.1016/j.neucom.2013.09.043 |
| [11] | Wang, H.; Qian, G.; Feng, X.-Q., Predicting consumer sentiments using online sequential extreme learning machine and intuitionistic fuzzy sets, Neural Computing and Applications, 22, 3-4, 479-489 (2013) · doi:10.1007/s00521-012-0853-1 |
| [12] | Guo, W.; Xu, T.; Tang, K., M-estimator-based online sequential extreme learning machine for predicting chaotic time series with outliers, Neural Computing and Applications, 28, 12, 4093-4110 (2017) · doi:10.1007/s00521-016-2301-0 |
| [13] | Wang, X.; Han, M., Improved extreme learning machine for multivariate time series online sequential prediction, Engineering Applications of Artificial Intelligence, 40, 28-36 (2015) · doi:10.1016/j.engappai.2014.12.013 |
| [14] | Yadav, B.; Ch, S.; Mathur, S.; Adamowski, J., Discharge forecasting using an Online Sequential Extreme Learning Machine (OS-ELM) model: A case study in Neckar River, Germany, Measurement, 92, 433-445 (2016) · doi:10.1016/j.measurement.2016.06.042 |
| [15] | Lima, A. R.; Cannon, A. J.; Hsieh, W. W., Forecasting daily streamflow using online sequential extreme learning machines, Journal of Hydrology, 537, 431-443 (2016) · doi:10.1016/j.jhydrol.2016.03.017 |
| [16] | Huynh, H. T.; Won, Y., Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks, Pattern Recognition Letters, 32, 14, 1930-1935 (2011) · doi:10.1016/j.patrec.2011.07.016 |
| [17] | Gu, S.; Tan, Y.; He, X., Recentness biased learning for time series forecasting, Information Sciences, 237, 29-38 (2013) · Zbl 1320.68136 · doi:10.1016/j.ins.2010.09.004 |
| [18] | Zhao, J.; Wang, Z.; Park, D. S., Online sequential extreme learning machine with forgetting mechanism, Neurocomputing, 87, 15, 79-89 (2012) · doi:10.1016/j.neucom.2012.02.003 |
| [19] | Wang, X.; Han, M., Online sequential extreme learning machine with kernels for nonstationary time series prediction, Neurocomputing, 145, 90-97 (2014) · doi:10.1016/j.neucom.2014.05.068 |
| [20] | Zhou, X.; Liu, Z.; Zhu, C., Online regularized and kernelized extreme learning machines with forgetting mechanism, Mathematical Problems in Engineering, 2014 (2014) · doi:10.1155/2014/938548 |
| [21] | Simon, H., Adaptive Filter Theory, Upper Saddle River: Prentice Hall (2002) |
| [22] | Lim, J. S.; Lee, S.; Pang, H. S., Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimations, Neural Computing and Applications, 22, 3-4, 569-576 (2013) · doi:10.1007/s00521-012-0873-x |
| [23] | Soares, S. G.; Araújo, R., An adaptive ensemble of on-line Extreme Learning Machines with variable forgetting factor for dynamic system prediction, Neurocomputing, 171, 693-707 (2016) · doi:10.1016/j.neucom.2015.07.035 |
| [24] | Du, Z.; Li, X.; Zheng, Z.; Zhang, G.; Mao, Q., Extreme learning machine based on regularization and forgetting factor and its application in fault prediction, Yi Qi Yi Biao Xue Bao/Chinese Journal of Scientific Instrument, 36, 7, 1546-1553 (2015) |
| [25] | Golub, G. H.; Van Loan, C. F., Matrix Computations. Matrix Computations, Johns Hopkins Studies in the Mathematical Sciences (2013), Baltimore, Md, USA: Johns Hopkins University Press, Baltimore, Md, USA · Zbl 1268.65037 |
| [26] | Leung, C.-S.; Tsoi, A.-C.; Chan, L. W., Two regularizers for recursive least squared algorithms in feedforward multilayered neural networks, IEEE Transactions on Neural Networks and Learning Systems, 12, 6, 1314-1332 (2001) · doi:10.1109/72.963768 |
| [27] | Beza, M.; Bongiorno, M., Application of recursive least squares algorithm with variable forgetting factor for frequency component estimation in a generic input signal, IEEE Transactions on Industry Applications, 50, 2, 1168-1176 (2014) · doi:10.1109/TIA.2013.2279195 |
| [28] | Funaya, H.; Ikeda, K., Statistical properties of support vector machines with forgetting factor, Neural Networks, 27, 45-50 (2012) · Zbl 1250.68222 · doi:10.1016/j.neunet.2011.03.011 |
| [29] | Cai, Y.; de Lamare, R. C.; Zhao, M.; Zhong, J., Low-complexity variable forgetting factor mechanism for blind adaptive constrained constant modulus algorithms, IEEE Transactions on Signal Processing, 60, 8, 3988-4002 (2012) · Zbl 1393.94186 · doi:10.1109/TSP.2012.2199317 |
| [30] | Paleologu, C.; Benesty, J.; Ciochiňa, S., A robust variable forgetting factor recursive least-squares algorithm for system identification, IEEE Signal Processing Letters, 15, 3, 597-600 (2008) · doi:10.1109/lsp.2008.2001559 |
| [31] | Leung, S.-H.; So, C. F., Gradient-based variable forgetting factor {RLS} algorithm in time-varying environments, IEEE Transactions on Signal Processing, 53, 8, part 2, 3141-3150 (2005) · Zbl 1373.62465 · doi:10.1109/TSP.2005.851110 |
| [32] | Lughofer, E.; Angelov, P., Handling drifts and shifts in on-line data streams with evolving fuzzy systems, Applied Soft Computing, 11, 2, 2057-2068 (2011) · doi:10.1016/j.asoc.2010.07.003 |
| [33] | Wang, Y.-S.; Sun, J.; Wang, C.-J.; Fan, H.-D., Prediction of the chaotic time series from parameter-varying systems using artificial neural networks, Wuli Xuebao/Acta Physica Sinica, 57, 10, 6120-6131 (2008) · Zbl 1199.37092 |
| [34] | Fortuna, L.; Graziani, S.; Xibilia, M., Soft sensors for product quality monitoring in debutanizer distillation columns, Control Engineering Practice, 13, 4, 499-508 (2007) · doi:10.1016/j.conengprac.2004.04.013 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
