On-line anomaly detection with advanced independent component analysis of multi-variate residual signals from causal relation networks. (English) Zbl 1479.62040
Summary: Anomaly detection in todays industrial environments is an ambitious challenge to detect possible faults/problems which may turn into severe waste during production, defects, or systems components damage, at an early stage. Data-driven anomaly detection in multi-sensor networks rely on models which are extracted from multi-sensor measurements and which characterize the anomaly-free reference situation. Therefore, significant deviations to these models indicate potential anomalies. In this paper, we propose a new approach which is based on causal relation networks (CRNs) that represent the inner causes and effects between sensor channels (or sensor nodes) in form of partial sub-relations, and evaluate its functionality and performance on two distinct production phases within a micro-fluidic chip manufacturing scenario. The partial relations are modeled by non-linear (fuzzy) regression models for characterizing the (local) degree of influences of the single causes on the effects. An advanced analysis of the multi-variate residual signals, obtained from the partial relations in the CRNs, is conducted. It employs independent component analysis (ICA) to characterize hidden structures in the fused residuals through independent components (latent variables) as obtained through the demixing matrix. A significant change in the energy content of latent variables, detected through automated control limits, indicates an anomaly. Suppression of possible noise content in residuals – to decrease the likelihood of false alarms – is achieved by performing the residual analysis solely on the dominant parts of the demixing matrix. Our approach could detect anomalies in the process which caused bad quality chips (with the occurrence of malfunctions) with negligible delay based on the process data recorded by multiple sensors in two production phases: injection molding and bonding, which are independently carried out with completely different process parameter settings and on different machines (hence, can be seen as two distinct use cases). Our approach furthermore i.) produced lower false alarm rates than several related and well-known state-of-the-art methods for (unsupervised) anomaly detection, and ii.) also caused much lower parametrization efforts (in fact, none at all). Both aspects are essential for the useability of an anomaly detection approach.
MSC:
| 62H25 | Factor analysis and principal components; correspondence analysis |
| 60G35 | Signal detection and filtering (aspects of stochastic processes) |
| 62P30 | Applications of statistics in engineering and industry; control charts |
Keywords:
online anomaly detection; causal relation networks; advanced multivariate residual analysis; dominant parts of independent component analysis; automated control limits; online production systemsReferences:
| [1] | An, J.; Cho, S., Variational autoencoder based anomaly detection using reconstruction probability, Special Lecture on IE, 2, 1-18 (2015) |
| [2] | Aumi, S.; Corbett, B.; Mhaskary, P., Model predictive quality control of batch processes, (2012 American Control Conference (2012), Fairmont Queen Elizabeth: Fairmont Queen Elizabeth Montral, Canada), 5646-5651 |
| [3] | Campello, R. J.G. B.; Moulavi, D.; Zimek, A.; Sander, J., Hierarchical density estimates for data clustering, visualization, and outlier detection, ACM Trans. Knowl. Discovery Data, 10, 1, 1-51 (2015) |
| [4] | Cernuda, C.; Lughofer, E.; Mayr, G.; Röder, T.; Hintenaus, P.; Märzinger, W.; Kasberger, J., Incremental and decremental active learning for optimized self-adaptive calibration in viscose production, Chemometr. Intell. Lab. Syst., 138, 14-29 (2014) |
| [5] | Chandola, V.; Banerjee, A.; Kumar, V., Anomaly detection: a survey, ACM Comput. Surveys, 41, 3 (2009) |
| [6] | Chen, H.; Tino, P.; Yao, X.; Rodan, A., Learning in the model space for fault diagnosis, IEEE Trans. Neural Networks Learn. Syst., 25, 1, 124-136 (2014) |
| [7] | Chiang, L. H.; Russell, E. L.; Braatz, R. D., Fault Detection and Diagnosis in Industrial Systems (2001), Springer: Springer London Berlin Heidelberg · Zbl 0982.93005 |
| [8] | Chih-Chung, C.; Chih-Jen, L., LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol., 2 (2011), 27:1-27:27 |
| [9] | Comon, P., Independent component analysis: a new concept?, Signal Process., 36, 3, 287-314 (1994) · Zbl 0791.62004 |
| [10] | Dargie, W.; Poellabauer, C., Fundamentals of Wireless Sensor Networks: Theory and Practice (2010), John Wiley and Sons: John Wiley and Sons Hoboken, New Jersey |
| [11] | Dou, D.; Zhou, S., Comparison of four direct classification methods for intelligent fault diagnosis of rotating machinery, Appl. Soft Comput., 46, 459-468 (2016) |
| [12] | Duan, L.; Xie, M.; Bai, T.; Wang, J., A new support vector data description method for machinery fault diagnosis with unbalanced datasets, Expert Syst. Appl., 64, 239-246 (2016) |
| [13] | Fernandez-Francos, D.; Martnez-Rego, D.; Fontenla-Romero, O.; Alonso-Betanzos, A., Automatic bearing fault diagnosis based on one-class v-svm, Comput. Ind. Eng., 64, 1, 357-365 (2013) |
| [14] | Fortuna, L.; Graziani, S.; Rizzo, A.; Xibilia, M. G., Soft Sensor for Monitoring and Control of Industrial Processes (2007), Springer Verlag: Springer Verlag London · Zbl 1136.93001 |
| [15] | He, Q. P.; Wang, J., Fault detection using the k-nearest neighbor rule for semiconductor manufacturing processes, IEEE Trans. Semicond. Manuf., 20, 4, 345-354 (2007) |
| [16] | Hsu, C.-W.; Chang, C.-C.; Lin, C.-J., A practical guide to support vector classification (2006) |
| [17] | Hyvärinen, A.; Oja, E., Independent component analysis: algorithms and applications, Neural Networks, 13, 4-5, 411-430 (2000) |
| [18] | Kano, M.; Nakagawa, Y., Data-based process monitoring, process control, and quality improvement: recent developments and applications in steel industry, Comput. Chem. Eng., 32, 12-24 (2008) |
| [19] | Le, T. D.; Hoang, T.; Li, J.; Liu, L.; Liu, H.; Hu, S., A fast PC algorithm for high dimensional causal discovery with multi-core PCs, J. LaTeX Class Files, 13, 9, 1-13 (2014) |
| [20] | Lemos, A.; Caminhas, W.; Gomide, F., Multivariable gaussian evolving fuzzy modeling system, IEEE Trans. Fuzzy Syst., 19, 1, 91-104 (2011) |
| [21] | Lughofer, E., On-line active learning: a new paradigm to improve practical useability of data stream modeling methods, Inf. Sci., 415-416, 356-376 (2017) |
| [22] | Lughofer, E.; Cernuda, C.; Kindermann, S.; Pratama, M., Generalized smart evolving fuzzy systems, Evolv. Syst., 6, 4, 269-292 (2015) |
| [23] | Lughofer, E.; Pratama, M., On-line active learning in data stream regression using uncertainty sampling based on evolving generalized fuzzy models, IEEE Trans. Fuzzy Syst., 26, 1, 292-309 (2018) |
| [24] | Lughofer, E.; Pratama, M.; Skrjanc, I., Incremental rule splitting in generalized evolving fuzzy systems for autonomous drift compensation, IEEE Trans. Fuzzy Syst., 26, 4, 1854-1865 (2018) |
| [25] | Lughofer, E.; Sayed-Mouchaweh, M., Predictive Maintenance in Dynamic Systems—Advanced Methods, Decision Support Tools and Real-World Application (2019), Springer: Springer New York |
| [26] | Lughofer, E.; Weigl, E.; Heidl, W.; Eitzinger, C.; Radauer, T., Integrating new classes on the fly in evolving fuzzy classifier designs and its application in visual inspection, Appl. Soft Comput., 35, 558-582 (2015) |
| [27] | Lughofer, E.; Zavoianu, A.-C.; Pollak, R.; Pratama, M.; Meyer-Heye, P.; Zörrer, H.; Eitzinger, C.; Haim, J.; Radauer, T., Self-adaptive evolving forecast models with incremental PLS space updating for on-line prediction of micro-fluidic chip quality, Eng. Appl. Artif. Intell., 68, 131-151 (2018) |
| [28] | Malathi, V.; Marimuthu, N. S.; Baskar, S., A comprehensive evaluation of multicategory classification methods for fault classification in series compensated transmission line, Neural Comput. Appl., 19, 595-600 (2010) |
| [29] | Mendona, L. F.; Sousa, J. M.C.; da Costa, J. M.G. S., An architecture for fault detection and isolation based on fuzzy methods, Expert Syst. Appl., 36, 2, 1092-1104 (2009) |
| [30] | Myklebust, O., Zero defect manufacturing: a product and plant oriented lifecycle approach, Procedia CIRP, 12, 246-251 (2013) |
| [31] | Nelles, O., Nonlinear System Identification (2001), Springer: Springer Berlin · Zbl 0963.93001 |
| [32] | Odgaard, P. F.; Lin, B.; Jorgensen, S. B., Observer and data-driven-model-based fault detection in power plant coal mills, IEEE Trans. Energy Convers., 23, 2, 659-668 (2008) |
| [33] | Orchard, M. E.; Acuna, D. E., On prognostic algorithm design and fundamental precision limits in long-term prediction, (Lughofer, E.; Sayed-Mouchaweh, M., Predictive Maintenance in Dynamic Systems, page to appear (2018), Springer: Springer New York) |
| [34] | M. Pratama, E. Dimla, T. Tjahjowidodo, E. Lughofer, W. Pedrycz, Online tool condition monitoring based on parsimonious ensemble+. IEEE Transactions on Cybernetics, on-line and in press:doi: 2018, 10.1109/TCYB.2018.2871120. |
| [35] | Samuel, R. T.; Cao, Y., Nonlinear process fault detection and identification using kernel pca and kernel density estimation, Syst. Sci. Control Eng., 4, 1, 165-174 (2016) |
| [36] | Schölkopf, B., Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., 10, 1299-1319 (1998) |
| [37] | Scholkopf, B., Estimating the support of a high-dimensional distribution, Neural Comput., 13, 7 (2001) · Zbl 1009.62029 |
| [38] | Serdio, F.; Lughofer, E.; Pichler, K.; Buchegger, T.; Efendic, H., Residual-based fault detection using soft computing techniques for condition monitoring at rolling mills, Inf. Sci., 259, 304-320 (2014) |
| [39] | Serdio, F.; Lughofer, E.; Zavoianu, A.-C.; Pichler, K.; Pichler, M.; Buchegger, T.; Efendic, H., Improved fault detection employing hybrid memetic fuzzy modeling and adaptive filters, Appl. Soft Comput., 51, 60-82 (2017) |
| [40] | Skrjanc, I., Confidence interval of fuzzy models: an example using a waste-water treatment plant, Chemometr. Intell. Lab. Syst., 96, 182-187 (2009) |
| [41] | Skrjanc, I.; Blazic, S.; Lughofer, E.; Dovzan, D., Inner matrix norms in evolving cauchy possibilistic clustering for classification and regression from data streams, Inf. Sci., 478, 540-563 (2019) |
| [42] | Spirtes, P.; Glymour, C.; Scheines, R., Causation, Prediction, and Search (2000), MIT Press: MIT Press Cambridge · Zbl 0806.62001 |
| [43] | Stojanovic, V.; Nedic, N., Robust identification of oe model with constrained output using optimal input design, J. Franklin Inst., 353, 2, 576-593 (2016) · Zbl 1395.93182 |
| [44] | Tax, D. M.J.; Duin, R. P.W., Support vector data description, Mach. Learn., 1, 45-66 (2004) · Zbl 1078.68728 |
| [45] | Tipping, M. E.; Bishop, C. M., Probabilistic principal component analysis, J. Roy. Stat. Soc. Ser. B (Stat. Methodol.), 61, 3, 611-622 (1999) · Zbl 0924.62068 |
| [46] | Wand, M. P.; Jones, M. C., Kernel Smoothing (1994), CRC Press: CRC Press Boca Raton · Zbl 0854.62043 |
| [47] | Welling, M., Kernel principal components analysis, Small Tutorial (2000), welling/classnotes/classnotes.html |
| [48] | Yu, J.; Jang, J.; Yoo, J.; Park, J. H.; Kim, S., Bagged auto-associative kernel regression-based fault detection and identification approach from steam boilers in thermal power plants, J. Electr. Eng. Technol., 12, 4, 1406-1416 (2017) |
| [49] | C. Zhang, X. Gao, Y. Li, L. Feng, Fault detection strategy based on weighted distance of k nearest neighbors for semiconductor manufacturing processes. IEEE Trans. Semincond. Manuf., on-line and in press (DOI 2018, 10.1109/TSM.2018.2857818). |
| [50] | H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. Roy. Stat. Soc. Ser. B (2005) 301-320 · Zbl 1069.62054 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
