×

Axiomatic generalization of the membership degree weighting function for fuzzy \(C\) means clustering: theoretical development and convergence analysis. (English) Zbl 1435.62246

Summary: For decades practitioners have been using the center-based partitional clustering algorithms like Fuzzy \(C\) Means (FCM), which rely on minimizing an objective function, comprising of an appropriately weighted sum of distances of each data point from the cluster representatives. Numerous generalizations in terms of choice of the distance function have been introduced for FCM since its inception. In a stark contrast to this fact, to the best of our knowledge, there has not been any significant effort to address the issue of convergence of the algorithm under the structured generalization of the weighting function. Here, by generalization we mean replacing the conventional weighting function \(u_{ij}^m\) (where \(u_{ij}\) indicates the membership of data \(x_i\) to cluster \(C_{j}\) and \(m\) is the real-valued fuzzifier with \(1\leq m <\infty\)) with a more general function \(g(u_{ij})\) which can assume a wide variety of forms under some mild assumptions. In this article, for the first time, we present a novel axiomatic development of the general weighting function based on the membership degrees. We formulate the fuzzy clustering problem along with the intuitive justification of the technicalities behind the axiomatic approach. We develop an Alternative Optimization (AO) procedure to solve the main clustering problem by solving the partial optimization problems in an iterative way. The novelty of the article lies in the development of an axiomatic generalization of the weighting function of FCM, formulation of an AO algorithm for the fuzzy clustering problem with the extension to this general class of weighting functions, and a detailed convergence analysis of the proposed algorithm.

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
62H86 Multivariate analysis and fuzziness
62A01 Foundations and philosophical topics in statistics
Full Text: DOI

References:

[1] Bandyopadhyay, S.; Maulik, U., Genetic clustering for automatic evolution of clusters and application to image classification, Pattern Recognit., 35, 6, 1197-1208 (2002) · Zbl 1001.68926
[2] Bandyopadhyay, S.; Pal, S. K., Classification and Learning using Genetic Algorithms: Applications in Bioinformatics and Web Intelligence (2007), Springer Science & Business Media · Zbl 1138.68486
[3] Banerjee, A.; Guo, X.; Wang, H., On the optimality of conditional expectation as a Bregman predictor, Inf. Theory IEEE Trans., 51, 7, 2664-2669 (2005) · Zbl 1284.94025
[4] Banerjee, A.; Merugu, S.; Dhillon, I. S.; Ghosh, J., Clustering with Bregman divergences, J. Mach. Learn. Res., 6, 1705-1749 (2005) · Zbl 1190.62117
[5] Ben-Tal, A.; Charnes, A.; Teboulle, M., Entropic means, J. Math. Anal. Appl., 139, 2, 537-551 (1989) · Zbl 0675.26007
[6] Bezdek, J. C., Pattern Recognition with Fuzzy Objective Function Algorithms (1981), Kluwer Academic Publishers: Kluwer Academic Publishers Norwell, MA, USA · Zbl 0503.68069
[7] Bezdek, J. C.; Hathaway, R. J., Convergence of alternating optimization, Neural Parallel Sci. Comput., 11, 4, 351-368 (2003) · Zbl 1063.90051
[8] Bezdek, J. C.; Hathaway, R. J.; Sabin, M. J.; Tucker, W. T., Convergence theory for fuzzy c-means: counterexamples and repairs, Syst. Man Cybern. IEEE Trans., 17, 5, 873-877 (1987) · Zbl 0653.68091
[9] Bobrowski, L.; Bezdek, J. C., c-means clustering with the \(l_1\) and \(l_∞\) norms, Syst. Man Cybern. IEEE Trans., 21, 3, 545-554 (1991) · Zbl 0735.62059
[10] Bregman, L. M., The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming, USSR Comput. Math. Math. Phys., 7, 3, 200-217 (1967) · Zbl 0186.23807
[11] Chaomurilige; Yu, J.; Yang, M.-S., Analysis of parameter selection for gustafson-kessel fuzzy clustering using jacobian matrix, IEEE Trans. Fuzzy Syst., 23, 6, 2329-2342 (2015)
[12] Csisz, I., On topological properties of f-divergences, Studia Sci. Math. Hungar., 2, 329-339 (1967) · Zbl 0157.25803
[13] Csiszár, I., I-divergence geometry of probability distributions and minimization problems, Ann. Probab., 146-158 (1975) · Zbl 0318.60013
[14] Dunn, J. C., A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters, Cybern. Syst., 32-57 (1973) · Zbl 0291.68033
[15] Fazendeiro, P.; de Oliveira, J. V., Observer-biased fuzzy clustering, IEEE Trans. Fuzzy Syst., 23, 1, 85-97 (2015)
[16] Fitzpatrick, P., Advanced Calculus, vol. 5 (2006), American Mathematical Soc. · Zbl 1229.26001
[17] Fränti, P.; Virmajoki, O., Iterative shrinking method for clustering problems, Pattern Recognit., 39, 5, 761-775 (2006) · Zbl 1161.68764
[18] Ganguly, S.; Bose, D.; Konar, A., Clustering using vector membership: an extension of the fuzzy c-means algorithm, 2013 Fifth International Conference on Advanced Computing (ICoAC), 27-32 (2013)
[19] Groll, L.; Jakel, J., A new convergence proof of fuzzy c-means, IEEE Trans. Fuzzy Syst., 13, 5, 717-720 (2005)
[20] Guo, Y.; Sengur, A., Ncm: neutrosophic c-means clustering algorithm, Pattern Recognit., 48, 8, 2710-2724 (2015)
[21] Hall, L. O.; Goldgof, D. B., Convergence of the single-pass and online fuzzy c-means algorithms, IEEE Trans. Fuzzy Syst., 19, 4, 792-794 (2011)
[22] Hathaway, R. J.; Bezdek, J. C.; Hu, Y., Generalized fuzzy c-means clustering strategies using \(l_p\) norm distances, Fuzzy Syst. IEEE Trans., 8, 5, 576-582 (2000)
[23] Hoppner, F.; Klawonn, F., A contribution to convergence theory of fuzzy c-means and derivatives, IEEE Trans. Fuzzy Syst., 11, 5, 682-694 (2003)
[24] Huang, M.; Xia, Z.; Wang, H.; Zeng, Q.; Wang, Q., The range of the value for the fuzzifier of the fuzzy c-means algorithm, Pattern Recognit. Lett., 33, 16, 2280-2284 (2012)
[25] Hubert, L.; Arabie, P., Comparing partitions, J. classification, 2, 1, 193-218 (1985)
[26] Jain, A. K., Data clustering: 50 years beyond k-means, Pattern Recognit. Lett., 31, 8, 651-666 (2010)
[27] Jain, A. K.; Murty, M. N.; Flynn, P. J., Data clustering: a review, ACM Comput. Surv., 31, 3, 264-323 (1999)
[28] Klawonn, F.; Höppner, F., An alternative approach to the fuzzifier in fuzzy clustering to obtain better clustering results, Proceedings of the 3rd Eusflat Conference, 730-734 (2003)
[29] Klawonn, F.; Höppner, F., What is fuzzy about fuzzy clustering? Understanding and improving the concept of the fuzzifier, Advances in Intelligent Data Analysis V, 254-264 (2003), Springer
[30] Klawonn, F.; Keller, A., Fuzzy clustering based on modified distance measures, Advances in Intelligent Data Analysis, 291-301 (1999), Springer
[31] Lei, Y.; Bezdek, J.; Chan, J.; Vinh, N.; Romano, S.; Bailey, J., Extending information-theoretic validity indices for fuzzy clustering, IEEE Trans. Fuzzy Syst., PP, 99 (2016)
[33] Lin, J., Divergence measures based on the shannon entropy, Inf. Theory IEEE Trans., 37, 1, 145-151 (1991) · Zbl 0712.94004
[34] Miyamoto, S.; Agusta, Y., An efficient algorithm for \(l_1\) fuzzy c-means and its termination, Control Cybern., 24, 421-436 (1995) · Zbl 0852.62058
[35] Miyamoto, S.; Agusta, Y., Algorithms for \(L_1\) and \(L_p\) fuzzy c-means and their convergence, Data Science, Classification, and Related Methods, 295-302 (1998), Springer · Zbl 0897.62065
[36] Munkres, J. R., Topology (2000), Prentice Hall: Prentice Hall Prentice Hall, Upper Saddle River, NJ 07458 · Zbl 0951.54001
[37] Nielsen, F.; Boltz, S., The burbea-rao and bhattacharyya centroids, Inf. Theory IEEE Trans., 57, 8, 5455-5466 (2011) · Zbl 1365.94159
[38] Nielsen, F.; Nock, R., Sided and symmetrized Bregman centroids, Inf. Theory IEEE Trans., 55, 6, 2882-2904 (2009) · Zbl 1367.94138
[39] Nock, R.; Nielsen, F.; Amari, S. I., On conformal divergences and their population minimizers, IEEE Trans. Inf. Theory, 62, 1, 527-538 (2016) · Zbl 1359.94347
[40] Qiu, C.; Xiao, J.; Han, L.; Iqbal, M. N., Enhanced interval type-2 fuzzy c-means algorithm with improved initial center, Pattern Recognit. Lett., 38, 86-92 (2014)
[41] Saha, A.; Das, S., Geometric divergence based fuzzy clustering with strong resilience to noise features, Pattern Recognit. Lett., 79, 60-67 (2016)
[42] Teboulle, M., A unified continuous optimization framework for center-based clustering methods, J. Mach. Learn. Res., 8, 65-102 (2007) · Zbl 1222.68318
[43] Veenman, C. J.; Reinders, M. J.; Backer, E., A maximum variance cluster algorithm, Pattern Anal. Mach. Intell. IEEE Trans., 24, 9, 1273-1280 (2002)
[44] Wu, C. H.; Ouyang, C. S.; Chen, L. W.; Lu, L. W., A new fuzzy clustering validity index with a median factor for centroid-based clustering, IEEE Trans. Fuzzy Syst., 23, 3, 701-718 (2015)
[45] Wu, J.; Xiong, H.; Liu, C.; Chen, J., A generalization of distance functions for fuzzy-means clustering with centroids of arithmetic means, Fuzzy Syst. IEEE Trans., 20, 3, 557-571 (2012)
[46] Xenaki, S. D.; Koutroumbas, K. D.; Rontogiannis, A. A., A novel adaptive possibilistic clustering algorithm, IEEE Trans. Fuzzy Syst., 24, 4, 791-810 (2016)
[47] Xu, R.; Wunsch, D., Survey of clustering algorithms, Neural Netw. IEEE Trans., 16, 3, 645-678 (2005)
[48] Yeung, K. Y.; Ruzzo, W. L., Details of the adjusted rand index and clustering algorithms, supplement to the paper “an empirical study on principal component analysis for clustering gene expression data”, Bioinformatics, 17, 9, 763-774 (2001)
[49] Yu, J.; Cheng, Q.; Huang, H., Analysis of the weighting exponent in the fcm, IEEE Trans. Syst. Man Cybern. Part B, 34, 1, 634-639 (2004)
[50] Zangwill, W. I., Nonlinear Programming: A Unified Approach, vol. 196 (1969), Prentice-Hall Englewood Cliffs, NJ · Zbl 0195.20804
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.