Abstract
The success of fuzzy clustering heavily relies on the proper feature space constructed by the input data. For nonspherical and overlapped clusters, kernel fuzzy clustering is more effective because it finds more proper feature space compared to conventional fuzzy clustering. Unfortunately, poor scalability of kernel fuzzy clustering is induced by the construction of a kernel matrix. To solve the problem, random feature-based method was presented to approximate the kernel function. More interestingly, these features exposed in the approximate feature space are directly manipulable. Inspired by the architecture of functional-link neural network, to exploit more information from both the original data space and the approximate kernel space, a new feature space called cascaded feature space (CF) is constructed in this paper. By performing classical fuzzy c-means (FCM) in CF space, a new fuzzy clustering framework called FCM-CF is developed. To reduce computational complexity and perform FCM in CF space well, dimension reduction methods are adopted to generate two variants of FCM-CF. The experiment results of our proposed algorithms verify their superiority in comparison of other classical fuzzy clustering algorithms.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alzate, C., Suykens, J.A.: Multiway spectral clustering with out-of-sample extensions through weighted kernel pca. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2008)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Norwell (1981)
Camastra, F., Verri, A.: A novel kernel method for clustering. In: Apolloni, B., Marinaro, M., Tagliaferri, R. (eds.) Biological and Artificial Intelligence Environments, pp. 245–250. Springer, Netherlands (2005)
Chen, C.L.P.: A rapid supervised learning neural network for function interpolation and approximation. IEEE Trans. Neural Netw. 7(5), 1220–1230 (1996)
Chen, C.L.P., Liu, Z.: Broad learning system: an effective and efficient incremental learning system without the need for deep architecture. IEEE Trans. Neural Netw. Learn. Syst. 29(1), 10–24 (2018)
Chen, C.L.P., Wan, J.Z.: A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction. IEEE Trans. Syst. Man. Cybern. B Cybern. 29(1), 62–72 (1999)
Chen, L., Zou, J., Chen, C.L.P.: Kernel non-local shadowed c-means for image segmentation. In: 2014 IEEE international conference on fuzzy systems (FUZZ-IEEE), pp. 2085–2090 (2014)
Chiang, J.H., Hao, P.Y.: A new kernel-based fuzzy clustering approach: support vector clustering with cell growing. IEEE Trans. Fuzzy Syst. 11(4), 518–527 (2003)
Chitta, R., Jin, R., Havens, T.C., Jain, A.K.: Approximate kernel k-means: solution to large scale kernel clustering. In: Proceedings of the international conference on knowledge discovery and data mining, pp. 895–903
Deng, Z., Choi, K.S., Jiang, Y., Wang, J., Wang, S.: A survey on soft subspace clustering. Inf. Sci. 348, 84–106 (2016)
Deshmukh, A.A.: Kernel approximation. Stats 608, 1–3
Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern. Anal. Mach. Intell. 35(11), 2765–2781 (2013)
Frank, A.: On kuhn’s hungarian method—a tribute from hungary. Naval Res. Logist. 52, 2–5 (2005)
Gong, M., Zhao, J., Liu, J., Miao, Q., Jiao, L.: Change detection in synthetic aperture radar images based on deep neural networks. IEEE Trans. Neural. Netw. Learn. Syst. 27(1), 125–138 (2016)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Graves, D., Pedrycz, W.: Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study. Fuzzy Sets Syst. 161(4), 522–543 (2010)
Hamid, R., Xiao, Y., Gittens, A., DeCoste, D.: Compact random feature maps. In: Proceedings of the 31st international conference on international conference on machine learning, vol. 32, ICML’14, pp. II–19–II–27. JMLR.org (2014)
Havens, T.C., Bezdek, J.C., Leckie, C., Hall, L.O., Palaniswami, M.: Fuzzy c-means algorithms for very large data. IEEE Trans. Fuzzy Syst. 20(6), 1130–1146 (2012)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Hou, W., Gao, X., Tao, D., Li, X.: Blind image quality assessment via deep learning. IEEE Trans. Neural. Netw. Learn. Syst. 26(6), 1275–1286 (2015)
Hu, H., Lin, Z., Feng, J., Zhou, J.: Smooth representation clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3834–3841 (2014)
Kim, D.W., Lee, K.Y., Lee, D., Lee, K.H.: Rapid and brief communication. Pattern Recognit. 38(4), 607–611 (2005)
Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)
Miyamoto, S., Umayahara, K.: Fuzzy clustering by quadratic regularization. In: 1998 IEEE international conference on fuzzy systems proceedings. IEEE World Congress on Computational Intelligence, vol. 2, pp. 1394–1399 vol. 2 (1998)
Muller, K., Mika, S., Ratsch, G., Tsuda, K., Scholkopf, B.: An introduction to kernel-based learning algorithms. IEEE Trans. Neural Netw. 12(2), 181–201 (2001)
Nadler, B., Lafon, S., Coifman, R.R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl. Comput. Harmon. Anal. 21, 113–27 (2005)
Pao, Y.H., Park, G.H., Sobajic, D.J.: Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6, 163–180 (1994)
Peng, X., Feng, J., Lu, J., Yau, W.Y., Yi, Z.: Cascade subspace clustering. In: Thirty-First AAAI conference on artificial intelligence (2017)
Peng, X., Feng, J., Xiao, S., Yau, W.Y., Zhou, J.T., Yang, S.: Structured autoencoders for subspace clustering. IEEE Tran. Image Process. 27(10), 5076–5086 (2018)
Peng, X., Xiao, S., Feng, J., Yau, W.Y., Yi, Z.: Deep subspace clustering with sparsity prior. In: IJCAI, pp. 1925–1931 (2016)
Peng, X., Zhou, J.T., Zhu, H.: k-meansnet: When k-means meets differentiable programming. arXiv preprint arXiv:1808.07292 (2018)
Pennington, J., Yu, F.X., Kumar, S.: Spherical random features for polynomial kernels. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28, pp. 1846–1854. Curran Associates Inc, New York (2015)
Pham, N., Pagh, R.: Fast and scalable polynomial kernels via explicit feature maps. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’13, pp. 239–247. ACM, New York, NY, USA (2013)
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems, vol. 20, pp. 1177–1184. Curran Associates Inc, New York (2008)
la Rosa, J.I.D., Fleury, G.A., Davoust, M.E.: Minimum-entropy, pdf approximation, and kernel selection for measurement estimation. IEEE Trans. Instrum. Meas. 52(4), 1009–1020 (2003)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR arXiv:1409.1556 (2014)
Sloan, I.H., Woźniakowski, H.: When are quasi-monte carlo algorithms efficient for high dimensional integrals? J. Complex. 14(1), 1–33 (1998)
Tang, J., Deng, C., Huang, G.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural. Netw. Learn. Syst. 27(4), 809–821 (2016)
Tenenbaum, J.B., Silva, V.D., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Tran, D., Wagner, M.: Fuzzy entropy clustering. In: Ninth IEEE international conference on fuzzy systems. FUZZ—IEEE 2000, vol. 1, pp. 152–157 (2000)
Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative review. J. Mach. Learn. Res. 10, 66–71 (2009)
Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)
Wang, J., Deng, Z., Choi, K.S., Jiang, Y., Luo, X., Chung, F.L., Wang, S.: Distance metric learning for soft subspace clustering in composite kernel space. Pattern Recognit. 52, 113–134 (2016)
Wang, J., Liu, H., Qian, X., Jiang, Y., Deng, Z., Wang, S.: Cascaded hidden space feature mapping, fuzzy clustering, and nonlinear switching regression on large datasets. IEEE Trans. Fuzzy Syst. 26(2), 640–655 (2017)
Xie, Z., Wang, S., Chung, F.L.: An enhanced possibilistic c-means clustering algorithm epcm. Soft Comput. 12(6), 593–611 (2008)
Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)
Yang, D.R., Lan, L.S., Pao, W.C.: A new fuzzy entropy clustering method with controllable membership characteristics. In: 2006 49th IEEE international midwest symposium on circuits and systems, vol. 1, pp. 187–191 (2006)
Yu, Z., Chen, H., Liu, J., You, J., Leung, H., Han, G.: Hybrid\(k\)-nearest neighbor classifier. IEEE Trans. Cybern. 46(6), 1263–1275 (2016)
Yu, Z., Luo, P., You, J., Wong, H., Leung, H., Wu, S., Zhang, J., Han, G.: Incremental semi-supervised clustering ensemble for high dimensional data clustering. IEEE Trans. Knowl. Data Eng. 28(3), 701–714 (2016)
Zarinbal, M., Zarandi, M.F., Turksen, I.: Relative entropy fuzzy c-means clustering. Inf. Sci. 260, 74–97 (2014)
Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems, pp. 1601–1608 (2005)
Zhang, D., Chen, S.: Fuzzy clustering using kernel method. In: The 2002 international conference on control and automation, 2002. ICCA. Final program and book of abstracts., pp. 162–163 (2002)
Zhang, L., Suganthan, P.: A comprehensive evaluation of random vector functional link networks. Inf. Sci. 367–368, 1094–1105 (2016)
Zhang, T., Chen, C.L.P., Chen, L., Xu, X., Hu, B.: Design of highly nonlinear substitution boxes based on i-ching operators. IEEE Trans. Cybern. 48(12), 3349–3358 (2018). https://doi.org/10.1109/TCYB.2018.2846186
Zhao, Y.P., Chen, L., Gan, M., Chen, C.P.: Multiple kernel fuzzy clustering with unsupervised random forests kernel and matrix-induced regularization. IEEE Access 7, 3967–3979 (2019)
Zhou, S., Gan, J.Q.: Mercer kernel, fuzzy c-means algorithm, and prototypes of clusters. In: Yang, Z.R., Yin, H., Everson, R.M. (eds.) Intelligent Data Engineering and Automated Learning—IDEAL 2004, pp. 613–618. Springer, Berlin (2004)
Acknowledgements
This research is funded by The Science and Technology Development Fund, Macao SAR (File no. 196/2017/A3). This research is supported by National Nature Science Foundation of China (61673405) and University of Macau RC MYRG2018-00132-FST.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhao, YP., Chen, L. & Chen, C.L.P. Fuzzy Clustering in Cascaded Feature Space. Int. J. Fuzzy Syst. 21, 2155–2167 (2019). https://doi.org/10.1007/s40815-019-00714-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40815-019-00714-x