Advances in clustering and visualization of time series using GTM through time. (English) Zbl 1254.68219
Summary: Most of the existing research on multivariate time series concerns supervised forecasting problems. In comparison, little research has been devoted to their exploration through unsupervised clustering and visualization. In this paper, the capabilities of generative topographic mapping through Time, a model with foundations in probability theory, that performs simultaneous time series clustering and visualization, are assessed in detail. Focus is placed on the visualization of the evolution of signal regimes and the exploration of sudden transitions, for which a novel identification index is defined. The interpretability of time series clustering results may become extremely difficult, even in exploratory visualization, for high dimensional datasets. Here, we define and test an unsupervised time series relevance determination method, fully integrated in the generative topographic mapping through time model, that can be used as a basis for time series selection. This method should ease the interpretation of time series clustering results.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T10 | Pattern recognition, speech recognition |
| 62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |
Keywords:
multivariate time series; generative topographic mapping; unsupervised relevance determination; clustering; visualization; change point detectionSoftware:
GTMReferences:
| [1] | Baum, L. E.; Egon, J. A., An inequality with applications to statistical estimation for probabilistic functions for a Markov process and to a model for ecology, Bulletin of the American Mathematical Society, 73, 360-363 (1967) · Zbl 0157.11101 |
| [2] | Bengio, Y.; Chapados, N., Extensions to metric-based model selection, Journal of Machine Learning Research, 3, 1209-1227 (2003) · Zbl 1102.68529 |
| [3] | Bishop, C., Hinton, G., & Strachan, I. 1997 GTM through time. In: Proceedings of the IEE fifth international conference on artificial neural networks; Bishop, C., Hinton, G., & Strachan, I. 1997 GTM through time. In: Proceedings of the IEE fifth international conference on artificial neural networks |
| [4] | Bishop, C.; Svensén, M.; Williams, C., GTM: The generative topographic mapping, Neural Computation, 10, 1, 215-234 (1998) |
| [5] | Bishop, C.; Svensén, M.; Williams, C., Developments of the generative topographic mapping, Neurocomputing, 21, 1, 203-224 (1998) · Zbl 0936.68091 |
| [6] | Bullen, R. J.; Cornford, D.; Nabney, I. T., Outlier detection in scatterometer data: Neural network approaches, Neural Networks, 16, 3-4, 419-426 (2003) |
| [7] | Carreira-Perpiñan, M. A., Reconstruction of sequential data with probabilistic models and continuity constraints, (Solla, S.; Leen, T.; Muller, K. R., Advances in neural information processing systems 12. Advances in neural information processing systems 12, NIPS’1999 (2000)), 414-420 |
| [8] | Chappell, G.; Taylor, J., The temporal Kohonen map, Neural Networks, 6, 441-445 (1993) |
| [9] | Chatfield, C., Time series forecasting (2000), Chapman & Hall/CRC Press |
| [10] | Friston, K., Learning and inference in the brain, Neural Networks, 16, 1325-1352 (2003) |
| [11] | Girolami, M., Latent variable models for the topographic organisation of discrete and strictly positive data, Neurocomputing, 48, 185-198 (2002) · Zbl 1006.68771 |
| [12] | Grabmeier, J.; Rudolph, A., Techniques of cluster algorithms in data mining, Data Mining and Knowledge Discovery, 6, 303-360 (2002) |
| [13] | Kabán, A.; Girolami, M., A dynamic probabilistic model to visualise topic evolution in text streams, Journal of Intelligence Information Systems, 18, 2-3, 107-125 (2002) |
| [14] | Keogh, E.; Lin, J., Clustering of time series subsequences is meaningless: Implications for previous and future research, Knowledge and Information Systems, 8, 2, 154-177 (2005) |
| [15] | Kohonen, T., Self-organizing maps (2001), Springer-Verlag: Springer-Verlag Berlin · Zbl 0957.68097 |
| [16] | Kostiainen, T.; Lampinen, J., On the generative probability density model in the self-organizing map, Neurocomputing, 48, 1-4, 217-228 (2002) · Zbl 1006.68783 |
| [17] | Law, M. H.C.; Figueiredo, M. A.T.; Jain, A. K., Simultaneous feature selection and clustering using mixture models, IEEE Transactions on Pattern Analysis, 26, 9, 1154-1166 (2004) |
| [18] | Lin, J.; Vlachos, M.; Keogh, E.; Gunopulos, D., Iterative incremental clustering of time series, (Bertino, E.; etal., Advances in database technology-EDBT 2004 proceedings. Advances in database technology-EDBT 2004 proceedings, LNCS, Vol. 2992 (2004)), 106-122 |
| [19] | Olier, I.; Vellido, A., Comparative assessment of the robustness of missing data imputation through Generative Topographic Mapping, (Cabestany, J.; Prieto, A.; Sandoval, F., Computational intelligence and bioinspired systems, IWANN 2005 proceedings. Computational intelligence and bioinspired systems, IWANN 2005 proceedings, LNCS, Vol. 3512 (2005)), 787-794 |
| [20] | Sharifzadeh, M.; Azmoodeh, F.; Shahabi, C., Change detection in time series using wavelet footprints, (Medeiros, C. B.; Egenhofer, M.; Bertino, E., Advances in spatial and temporal databases, SSTD 2005 proceedings. Advances in spatial and temporal databases, SSTD 2005 proceedings, LNCS, Vol. 3633 (2005)), 127-144 |
| [21] | Simon, G.; Lee, J. A.; Verleysen, M., Unfolding preprocessing for meaningful time series clustering, Neural Networks, 19, 6-7, 877-888 (2006) · Zbl 1102.68596 |
| [22] | Strickert, M.; Hammer, B., Merge SOM for temporal data, Neurocomputing, 64, 39-71 (2005) |
| [23] | Tikka, J., & Hollmén, J. 2004. Learning linear dependency trees from multivariate time-series data. In: Proceedings of the IEEE international conference on data mining (ICDM 2004) workshop on temporal data mining: Algorithms, theory and applications; Tikka, J., & Hollmén, J. 2004. Learning linear dependency trees from multivariate time-series data. In: Proceedings of the IEEE international conference on data mining (ICDM 2004) workshop on temporal data mining: Algorithms, theory and applications |
| [24] | Tino, P.; Nabney, I., Hierarchical GTM: Constructing localized nonlinear projection manifolds in a principled way, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 5, 639-656 (2002) |
| [25] | Tino, P.; Farkaš, I.; van Mourik, J., Dynamics and topographic organization of recursive self-organizing maps, Neural Computation, 18, 2529-2567 (2006) · Zbl 1107.68085 |
| [26] | Vellido, A. (2005). Preliminary theoretical results on a feature relevance determination method for Generative Topographic Mapping. Technical report LSI-05-13-R. Universitat Politècnica de Catalunya (UPC); Vellido, A. (2005). Preliminary theoretical results on a feature relevance determination method for Generative Topographic Mapping. Technical report LSI-05-13-R. Universitat Politècnica de Catalunya (UPC) |
| [27] | Vellido, A., Missing data imputation through GTM as a mixture of \(t\)-distributions, Neural Networks, 19, 10, 1624-1635 (2006) · Zbl 1178.68472 |
| [28] | Vellido, A.; El-Deredy, W.; Lisboa, P. J.G., Selective smoothing of the generative topographic mapping, IEEE Transactions on Neural Networks, 14, 847-852 (2003) |
| [29] | Vellido, A.; Lisboa, P. J.G.; Vicente, D., Robust analysis of MRS brain tumour data using \(t\)-GTM, Neurocomputing, 69, 7-9, 754-768 (2006) |
| [30] | Vesanto, J., SOM-based data visualization methods, Intelligent Data Analysis, 3, 2, 111-126 (1999) · Zbl 0970.68570 |
| [31] | Vesanto, J.; Hollmén, J., An automated report generation tool for the data understanding phase, (Abraham, A.; Koeppen, M., Hybrid information systems. Proceedings of HIS’01 (2001)), 611-625 · Zbl 1061.68586 |
| [32] | Voegtlin, T., Recursive self-organizing maps, Neural Networks, 15, 8-9, 979-991 (2002) |
| [33] | Wong, P. C., Visual data mining, IEEE Transactions on Computer Graphics Applications, 19, 5, 20-21 (1999) |
| [34] | Yamanishi, K., & Takeuchi, J.-I. 2002. A unifying framework for detecting outliers and change points from non-stationary time series data. In: Proceedings of the 8th ACM SIGKDD intenational conference on knowledge discovery and data mining; Yamanishi, K., & Takeuchi, J.-I. 2002. A unifying framework for detecting outliers and change points from non-stationary time series data. In: Proceedings of the 8th ACM SIGKDD intenational conference on knowledge discovery and data mining |
| [35] | Yin, H.; Allinson, N., Self-organizing mixture networks for probability density estimation, IEEE Transaction on Neural Networks, 12, 405-411 (2001) |
| [36] | Yoon, H.; Yang, K.; Shahabi, C., Feature subset selection and feature ranking for multivariate time series, IEEE Transactions on Knowledge and Data Engineering, 17, 9, 1186-1198 (2005) |
| [37] | Zhang, G.; Patuwo, B.; Hu, M., Forecasting with artificial neural networks: The state of the art, International Journal on Forecasting, 14, 1, 35-62 (1998) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
