Skip to main content
Springer Nature Link
Log in

Fuzzy Clustering in Cascaded Feature Space

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

The success of fuzzy clustering heavily relies on the proper feature space constructed by the input data. For nonspherical and overlapped clusters, kernel fuzzy clustering is more effective because it finds more proper feature space compared to conventional fuzzy clustering. Unfortunately, poor scalability of kernel fuzzy clustering is induced by the construction of a kernel matrix. To solve the problem, random feature-based method was presented to approximate the kernel function. More interestingly, these features exposed in the approximate feature space are directly manipulable. Inspired by the architecture of functional-link neural network, to exploit more information from both the original data space and the approximate kernel space, a new feature space called cascaded feature space (CF) is constructed in this paper. By performing classical fuzzy c-means (FCM) in CF space, a new fuzzy clustering framework called FCM-CF is developed. To reduce computational complexity and perform FCM in CF space well, dimension reduction methods are adopted to generate two variants of FCM-CF. The experiment results of our proposed algorithms verify their superiority in comparison of other classical fuzzy clustering algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Alzate, C., Suykens, J.A.: Multiway spectral clustering with out-of-sample extensions through weighted kernel pca. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2008)

    Article  Google Scholar 

  2. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  3. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Norwell (1981)

    Book  MATH  Google Scholar 

  4. Camastra, F., Verri, A.: A novel kernel method for clustering. In: Apolloni, B., Marinaro, M., Tagliaferri, R. (eds.) Biological and Artificial Intelligence Environments, pp. 245–250. Springer, Netherlands (2005)

    Chapter  Google Scholar 

  5. Chen, C.L.P.: A rapid supervised learning neural network for function interpolation and approximation. IEEE Trans. Neural Netw. 7(5), 1220–1230 (1996)

    Article  Google Scholar 

  6. Chen, C.L.P., Liu, Z.: Broad learning system: an effective and efficient incremental learning system without the need for deep architecture. IEEE Trans. Neural Netw. Learn. Syst. 29(1), 10–24 (2018)

    Article  MathSciNet  Google Scholar 

  7. Chen, C.L.P., Wan, J.Z.: A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction. IEEE Trans. Syst. Man. Cybern. B Cybern. 29(1), 62–72 (1999)

    Article  Google Scholar 

  8. Chen, L., Zou, J., Chen, C.L.P.: Kernel non-local shadowed c-means for image segmentation. In: 2014 IEEE international conference on fuzzy systems (FUZZ-IEEE), pp. 2085–2090 (2014)

  9. Chiang, J.H., Hao, P.Y.: A new kernel-based fuzzy clustering approach: support vector clustering with cell growing. IEEE Trans. Fuzzy Syst. 11(4), 518–527 (2003)

    Article  Google Scholar 

  10. Chitta, R., Jin, R., Havens, T.C., Jain, A.K.: Approximate kernel k-means: solution to large scale kernel clustering. In: Proceedings of the international conference on knowledge discovery and data mining, pp. 895–903

  11. Deng, Z., Choi, K.S., Jiang, Y., Wang, J., Wang, S.: A survey on soft subspace clustering. Inf. Sci. 348, 84–106 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Deshmukh, A.A.: Kernel approximation. Stats 608, 1–3

  13. Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern. Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  14. Frank, A.: On kuhn’s hungarian method—a tribute from hungary. Naval Res. Logist. 52, 2–5 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gong, M., Zhao, J., Liu, J., Miao, Q., Jiao, L.: Change detection in synthetic aperture radar images based on deep neural networks. IEEE Trans. Neural. Netw. Learn. Syst. 27(1), 125–138 (2016)

    Article  MathSciNet  Google Scholar 

  16. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  17. Graves, D., Pedrycz, W.: Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study. Fuzzy Sets Syst. 161(4), 522–543 (2010)

    Article  MathSciNet  Google Scholar 

  18. Hamid, R., Xiao, Y., Gittens, A., DeCoste, D.: Compact random feature maps. In: Proceedings of the 31st international conference on international conference on machine learning, vol. 32, ICML’14, pp. II–19–II–27. JMLR.org (2014)

  19. Havens, T.C., Bezdek, J.C., Leckie, C., Hall, L.O., Palaniswami, M.: Fuzzy c-means algorithms for very large data. IEEE Trans. Fuzzy Syst. 20(6), 1130–1146 (2012)

    Article  Google Scholar 

  20. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  21. Hou, W., Gao, X., Tao, D., Li, X.: Blind image quality assessment via deep learning. IEEE Trans. Neural. Netw. Learn. Syst. 26(6), 1275–1286 (2015)

    Article  MathSciNet  Google Scholar 

  22. Hu, H., Lin, Z., Feng, J., Zhou, J.: Smooth representation clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3834–3841 (2014)

  23. Kim, D.W., Lee, K.Y., Lee, D., Lee, K.H.: Rapid and brief communication. Pattern Recognit. 38(4), 607–611 (2005)

    Article  Google Scholar 

  24. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)

    Article  Google Scholar 

  25. Miyamoto, S., Umayahara, K.: Fuzzy clustering by quadratic regularization. In: 1998 IEEE international conference on fuzzy systems proceedings. IEEE World Congress on Computational Intelligence, vol. 2, pp. 1394–1399 vol. 2 (1998)

  26. Muller, K., Mika, S., Ratsch, G., Tsuda, K., Scholkopf, B.: An introduction to kernel-based learning algorithms. IEEE Trans. Neural Netw. 12(2), 181–201 (2001)

    Article  Google Scholar 

  27. Nadler, B., Lafon, S., Coifman, R.R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl. Comput. Harmon. Anal. 21, 113–27 (2005)

    Article  MATH  Google Scholar 

  28. Pao, Y.H., Park, G.H., Sobajic, D.J.: Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6, 163–180 (1994)

    Article  Google Scholar 

  29. Peng, X., Feng, J., Lu, J., Yau, W.Y., Yi, Z.: Cascade subspace clustering. In: Thirty-First AAAI conference on artificial intelligence (2017)

  30. Peng, X., Feng, J., Xiao, S., Yau, W.Y., Zhou, J.T., Yang, S.: Structured autoencoders for subspace clustering. IEEE Tran. Image Process. 27(10), 5076–5086 (2018)

    Article  MathSciNet  Google Scholar 

  31. Peng, X., Xiao, S., Feng, J., Yau, W.Y., Yi, Z.: Deep subspace clustering with sparsity prior. In: IJCAI, pp. 1925–1931 (2016)

  32. Peng, X., Zhou, J.T., Zhu, H.: k-meansnet: When k-means meets differentiable programming. arXiv preprint arXiv:1808.07292 (2018)

  33. Pennington, J., Yu, F.X., Kumar, S.: Spherical random features for polynomial kernels. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28, pp. 1846–1854. Curran Associates Inc, New York (2015)

    Google Scholar 

  34. Pham, N., Pagh, R.: Fast and scalable polynomial kernels via explicit feature maps. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’13, pp. 239–247. ACM, New York, NY, USA (2013)

  35. Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems, vol. 20, pp. 1177–1184. Curran Associates Inc, New York (2008)

    Google Scholar 

  36. la Rosa, J.I.D., Fleury, G.A., Davoust, M.E.: Minimum-entropy, pdf approximation, and kernel selection for measurement estimation. IEEE Trans. Instrum. Meas. 52(4), 1009–1020 (2003)

    Article  Google Scholar 

  37. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  38. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  39. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR arXiv:1409.1556 (2014)

  40. Sloan, I.H., Woźniakowski, H.: When are quasi-monte carlo algorithms efficient for high dimensional integrals? J. Complex. 14(1), 1–33 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  41. Tang, J., Deng, C., Huang, G.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural. Netw. Learn. Syst. 27(4), 809–821 (2016)

    Article  MathSciNet  Google Scholar 

  42. Tenenbaum, J.B., Silva, V.D., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    Article  Google Scholar 

  43. Tran, D., Wagner, M.: Fuzzy entropy clustering. In: Ninth IEEE international conference on fuzzy systems. FUZZ—IEEE 2000, vol. 1, pp. 152–157 (2000)

  44. Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative review. J. Mach. Learn. Res. 10, 66–71 (2009)

    Google Scholar 

  45. Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)

    Article  MathSciNet  Google Scholar 

  46. Wang, J., Deng, Z., Choi, K.S., Jiang, Y., Luo, X., Chung, F.L., Wang, S.: Distance metric learning for soft subspace clustering in composite kernel space. Pattern Recognit. 52, 113–134 (2016)

    Article  MATH  Google Scholar 

  47. Wang, J., Liu, H., Qian, X., Jiang, Y., Deng, Z., Wang, S.: Cascaded hidden space feature mapping, fuzzy clustering, and nonlinear switching regression on large datasets. IEEE Trans. Fuzzy Syst. 26(2), 640–655 (2017)

    Article  Google Scholar 

  48. Xie, Z., Wang, S., Chung, F.L.: An enhanced possibilistic c-means clustering algorithm epcm. Soft Comput. 12(6), 593–611 (2008)

    Article  MATH  Google Scholar 

  49. Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)

    Article  Google Scholar 

  50. Yang, D.R., Lan, L.S., Pao, W.C.: A new fuzzy entropy clustering method with controllable membership characteristics. In: 2006 49th IEEE international midwest symposium on circuits and systems, vol. 1, pp. 187–191 (2006)

  51. Yu, Z., Chen, H., Liu, J., You, J., Leung, H., Han, G.: Hybrid\(k\)-nearest neighbor classifier. IEEE Trans. Cybern. 46(6), 1263–1275 (2016)

    Article  Google Scholar 

  52. Yu, Z., Luo, P., You, J., Wong, H., Leung, H., Wu, S., Zhang, J., Han, G.: Incremental semi-supervised clustering ensemble for high dimensional data clustering. IEEE Trans. Knowl. Data Eng. 28(3), 701–714 (2016)

    Article  Google Scholar 

  53. Zarinbal, M., Zarandi, M.F., Turksen, I.: Relative entropy fuzzy c-means clustering. Inf. Sci. 260, 74–97 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  54. Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems, pp. 1601–1608 (2005)

  55. Zhang, D., Chen, S.: Fuzzy clustering using kernel method. In: The 2002 international conference on control and automation, 2002. ICCA. Final program and book of abstracts., pp. 162–163 (2002)

  56. Zhang, L., Suganthan, P.: A comprehensive evaluation of random vector functional link networks. Inf. Sci. 367–368, 1094–1105 (2016)

    Article  Google Scholar 

  57. Zhang, T., Chen, C.L.P., Chen, L., Xu, X., Hu, B.: Design of highly nonlinear substitution boxes based on i-ching operators. IEEE Trans. Cybern. 48(12), 3349–3358 (2018). https://doi.org/10.1109/TCYB.2018.2846186

    Article  Google Scholar 

  58. Zhao, Y.P., Chen, L., Gan, M., Chen, C.P.: Multiple kernel fuzzy clustering with unsupervised random forests kernel and matrix-induced regularization. IEEE Access 7, 3967–3979 (2019)

    Article  Google Scholar 

  59. Zhou, S., Gan, J.Q.: Mercer kernel, fuzzy c-means algorithm, and prototypes of clusters. In: Yang, Z.R., Yin, H., Everson, R.M. (eds.) Intelligent Data Engineering and Automated Learning—IDEAL 2004, pp. 613–618. Springer, Berlin (2004)

    Chapter  Google Scholar 

Download references

Acknowledgements

This research is funded by The Science and Technology Development Fund, Macao SAR (File no. 196/2017/A3). This research is supported by National Nature Science Foundation of China (61673405) and University of Macau RC MYRG2018-00132-FST.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Macau, China

    Yin-Ping Zhao, Long Chen & C. L. Philip Chen

Authors
  1. Yin-Ping Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Long Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. L. Philip Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Long Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, YP., Chen, L. & Chen, C.L.P. Fuzzy Clustering in Cascaded Feature Space. Int. J. Fuzzy Syst. 21, 2155–2167 (2019). https://doi.org/10.1007/s40815-019-00714-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-019-00714-x

Keywords

Search

Navigation