Abstract
DNNs (Deep Neural Networks) have been proved to be a successful technique in many areas. Understanding the reasons behind DNN is, however, quite important in assessing trust, which is fundamental if one plans to take action based on leaning results, or when choosing whether to deploy a new model. Lack of insights into the model can be an obstacle that hinders the development and application of the DNN. AdaBoost is a well-known boosting learning technique with better interpretability, combining many relatively weak and inaccurate rules to increase the performance. And as it modified sample weights in the training process, it shows excellent adaptability even in complex cases as it can alleviate overfitting. In the paper, we propose a method training the nets by the AdaBoost based on the soft label, which we call flexibly soft-labeling AdaBoost (FSL AdaBoost). Our soft labels are made in a novel and sequential way, adding further interpretability and adaptability to the learning process. Experimental results on several well-known datasets have validated the effectiveness and novelty of FSL AdaBoost.
The authors extend their appreciation to National Key Research and Development Program of China (International Technology Cooperation Project No. 2021YFE014400) and National Science Foundation of China (No. 42175194) for funding this work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2(4), 303–314 (1989). https://doi.org/10.1007/BF02551274
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016). http://www.deeplearningbook.org
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)
Krueger, D., et al.: Deep nets don’t learn via memorization (2017)
Arpit, D., et al.: A closer look at memorization in deep networks. In: Proceedings of the 34th International Conference on Machine Learning-vol. 70, pp. 233–242 (2017)
Lee, W., Jun, C.H., Lee, J.S.: Instance categorization by support vector machines to adjust weights in AdaBoost for imbalanced data classification. Inf. Sci. (Ny) 381, 92–103 (2017)
Lima, N.H.C., Neto, A.D.D., De Melo, J.D.: Creating an ensemble of diverse support vector machines using Adaboost. In: International Joint Conference on Neural Networks (2009)
Gan, Y., Chen, J., Xu, L.: Facial expression recognition boostedby soft label with a diverse ensemble. Pattern Recogn. Lett. 125, 105–112 (2019)
Li , S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. Computer Vision and Pattern Recognition (CVPR). In: 2017 IEEE Conference on IEEE, pp. 2584–2593 (2017)
Zhao, M., Zhang, Z., Chow, T.W.S., Li, B.: Soft label based linear discriminant analysis for image recognition and retrieval. Comput. Vis. Image Underst. 121, 86–99 (2014). https://doi.org/10.1016/j.cviu.2014.01.008
Liu, T., Wang, K., Chang, B., Sui, Z.: A soft-label method for noise-tolerant distantly supervised relation extraction. In: Proceedings of the 2017 Conference on Empirical Methods (2017)
Nguyen, Q., Valizadega, H., Hauskrecht, M.: Learning classification models with soft-label information. J. Am. Med. Inform. Assoc. 21, 501–508 (2014)
Algan, G., Ulusoy, I.: Meta soft label generation for noisy labels. In: 2020 25th International Conference on Pattern Recognition (ICPR) (2021)
Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5, 197–227 (1990)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Machine Learning: Proceedings of the Thirteenth International Conference, Morgan Kauman, SanFrancisco, pp. 148–156 (1996)
Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 297–304. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45243-0_39
Collins, M., Schapire, R.E., Singer, Y.: Logistic regression, adaboost and bregman distances. In: Thirteenth Conference on Computational (2000)
Domingo, C.: MadaBoost: A modification of AdaBoost. In: Proceedings of the 13th Conference on Computational Learning Theory (2013)
Wang, W., Sun, D.: The improved AdaBoost algorithms for imbalanced data classification. Inform. Sci. 563, 358–374 (2021)
Cao, J., Kwong, S., Wang, R.: A noise-detection based AdaBoost algorithm for mislabeled data. Pattern Recog. 45(12), 4451–4465 (2012)
Ji, Z., Hui, Z., Rosset, S., Hastie, T.: Multi-class AdaBoost. Stat. Its Interface 2, 349–360 (2009)
Taherkhani, A., Cosma, G., McGinnity, T.M.: AdaBoost-CNN: an adaptive boosting algorithm for convolutional neural networks to classify multi-class imbalanced datasets using transfer learning. Neurocomputing 404, 351–366 (2020)
Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16
Geng, X., Xia, Y.: Head pose estimation based on multivariate label distribution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1837–1842 (2014)
Xu, L., Chen, J., Gan, Y.: Head pose estimation with soft labels using regularized convolutional neural network. Neurocomputing 337, 339–353 (2019)
Wang, X., Wang, X., Ni, Y.: Unsupervised domain adaptation for facial expression recognition using generative adversarial networks. Comput. Intell. Neurosci. 2018, 1–10 (2018). https://doi.org/10.1155/2018/7208794
Barsoum, E., Zhang, C., Ferrer, C.C., Zhang, Z.: Training deep networks for facial expression recognition with crowd-sourced label distribution. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, ACM, pp. 279–283 (2016)
Ho, T.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)
Shirai, S., Kudo, M., Nakamura, A.: Comparison of bagging and boosting algorithms on sample and feature weighting. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds.) MCS 2009. LNCS, vol. 5519, pp. 22–31. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02326-2_3
García-Pedrajas, N., Ortiz-Boyer, D.: Boosting random subspace method. Neural Netw. 21(9), 1344–1362 (2008)
Rao, Q., Yu, B., He, K., Feng, B.: Regularization and Iterative Initialization of Softmax for Fast Training of Convolutional Neural Networks. In: 2019 International Joint Conference on Neural Networks (IJCNN), 30 Sep 2019
Masko, D., Hensman. P.: The Impact of Imbalanced Training Data for Convolutional Neural Networks. Bachelor thesis, KTH, Sch. Comput. Sci. Commun. (2015)
Hu, W., Maybank, S.: AdaBoost-based algorithm for network intrusion detection. In: IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 1 Apr 2008
Ahfock, D., McLachlan, G.J.: Harmless label noise and informative soft-labels in supervised classification. Comput. Stat. Data Anal. 161, 107253 (2021)
Ochiuddin Miah, M., Shahriar Khan, S., Shatabda, S., Farid, D.M.: Improving detection accuracy for imbal-anced network intrusion classification using cluster-based under-sampling with random forests. In: 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), pp. 1–5. IEEE (2019)
Bennin, K.E., Keung, J., Phannachitta, P., Monden, A., Mensah, S.: Mahakil: diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction. IEEE Trans. Software Eng. 44(6), 534–550 (2017)
Song, Q., Guo, Y., Shepperd, M.: A comprehensive investigation of the role of imbalanced learning for software defect prediction. IEEE Trans. Software Eng. 45(12), 1253–1269 (2018)
Cruz-Roa, A.A., Ovalle, J.E.A., Madabhushi, A., Osorio, F.A.G.: A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) Advanced Information Systems Engineering: 25th International Conference, CAiSE 2013, Valencia, Spain, June 17-21, 2013. Proceedings, pp. 403–410. Springer Berlin Heidelberg, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_50
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, J., Song, B., Zhang, X., Tian, Y., Guo, R. (2023). An Innovative AdaBoost Process Using Flexible Soft Labels on Imbalanced Big Data. In: Tian, Y., Ma, T., Jiang, Q., Liu, Q., Khan, M.K. (eds) Big Data and Security. ICBDS 2022. Communications in Computer and Information Science, vol 1796. Springer, Singapore. https://doi.org/10.1007/978-981-99-3300-6_13
Download citation
DOI: https://doi.org/10.1007/978-981-99-3300-6_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-3299-3
Online ISBN: 978-981-99-3300-6
eBook Packages: Computer ScienceComputer Science (R0)