Skip to main content
Springer Nature Link
Log in

Neuroevolutionary Models Based on Quantum-Inspired Evolutionary Algorithms

  • Chapter
  • First Online:
Women in Computational Intelligence

Part of the book series: Women in Engineering and Science ((WES))

Abstract

Artificial neural networks are powerful and flexible models that have proved useful in solving a countless number of real-world problems. More recently, deep neural networks have gained the attention of the machine learning community for surpassing human-level performance in a variety of tasks. To reach these excellent results, an expert needs to spend a significant amount of time designing the neural configuration, such as the best architecture, input variables, and training parameters, with long trial-and-error sessions. Neuroevolution is a method that tries to automate this design process by applied evolutionary computation algorithms to optimise the neural network hyperparameters. However, the high computational cost associated with the use of traditional evolutionary algorithms for neural network configuration is still an issue. In this regard, quantum-inspired evolutionary algorithms emerged as an excellent optimisation method for neuroevolution, as they provide faster convergence, with a reduced computational cost. This chapter presents an overview of quantum-inspired evolutionary algorithms and their application to the configuration of different neural network models, such as Multi-Layer Perceptrons, Echo-State Networks and Convolutional Neural Networks. Applications in system identification, image classification and concept drift environments are also detailed, with promising results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
eBook
USD 79.99
Price excludes VAT (USA)
Softcover Book
USD 99.99
Price excludes VAT (USA)
Hardcover Book
USD 129.99
Price excludes VAT (USA)

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. A. Abraham, Meta-learning evolutionary artificial neural networks. Neurocomputing 56, 1–38 (2004)

    Article  Google Scholar 

  2. A.V. Abs da Cruz, Algoritmos evolutivos com inspiração quântica para otimização de problemas com representação numérica. PhD thesis, Pontifical Catholic University of Rio de Janeiro, Brazil, (2007) (in Portuguese)

    Google Scholar 

  3. A.V. Abs da Cruz, M.M.B.R. Vellasco, M.A.C. Pacheco, Quantum-inspired evolutionary algorithm for numerical optimisation, in Book Series in Computational Intelligence, Vol. 75 – Hybrid Evolutionary Algorithms, (Springer, Berlin/Heidelberg, 2007), pp. 19–37

    Google Scholar 

  4. A.V. Abs da Cruz, M.M.B.R. Vellasco, M.A.C. Pacheco, Quantum-inspired evolutionary algorithms applied to numerical optimisation problems, in IEEE Congress on Evolutionary Computation (IEEE CEC 2010), (2010), pp. 3899–3904

    Google Scholar 

  5. P.J. Angeline, G.M. Saunders, J.B. Pollack, An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans. Neural Netw. 5(1), 54–65 (1994)

    Article  Google Scholar 

  6. P.P. Angelov, X. Zhou, Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans. Fuzzy Syst. 16(6), 1462–1475 (2008)

    Article  Google Scholar 

  7. F. Assunção, N. Lourenço, P. Machado, B. Ribeiro, DENSER: deep evolutionary network structured representation. Genet. Program Evolvable Mach. 20(1), 5–35 (2019)

    Article  Google Scholar 

  8. S. Basterrech, E. Alba, V. Snasel, An experimental analysis of the echo state network initialization using the particle swarm optimization, in 2014 Sixth World Congress on Nature and Biologically Inspired Computing (NaBIC 2014), (2014), pp. 214–219

    Chapter  Google Scholar 

  9. A. Blanco et al., A real-coded genetic algorithm for training recurrent neural networks. Neural Netw. 14(1), 93–105 (2001)

    Article  MathSciNet  Google Scholar 

  10. A. Cano, B. Krawczyk, Learning classification rules with differential evolution for high-speed data stream mining on GPU s, in 2018 IEEE Congress on Evolutionary Computation (IEEE CEC 2018), (2018)

    Google Scholar 

  11. A. Cano, B. Krawczyk, Evolving rule-based classifiers with genetic programming on GPUs for drifting data streams. Pattern Recogn. 87, 248–268 (2019)

    Article  Google Scholar 

  12. G. Capi, K. Doya, Evolution of recurrent neural controllers using an extended parallel genetic algorithm. Robot. Auton. Syst. 52(2), 148–159 (2005)

    Article  Google Scholar 

  13. V. Carvalho, W. Cohen, Single-pass online learning: performance, voting schemes and online feature selection, in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’06), (2006), pp. 548–553

    Chapter  Google Scholar 

  14. D. Chevitarese, D. Szwarcman, E. Brazil, and B. Zadrozny, “Efficient Classification of Seismic Textures”, 2018 International Joint Conference on Neural Networks (IJCNN 2018), 2018

    Google Scholar 

  15. N. Chouikhi, B. Ammar, N. Rokbani, A.M. Alimi, PSO-based analysis of Echo State Network parameters for time series forecasting. Appl. Soft Comput. 55, 211–225 (2017)

    Article  Google Scholar 

  16. B. L. R. De Moor (ed.), DaISy: Database for the Identification of Systems, Technical Report 97–70, http://homes.esat.kuleuven.be/~smc/daisy/ (Department of Electrical Engineering, ESAT/STADIUS, KU Leuven, Belgium, 2018)

    Google Scholar 

  17. P.R.M. de Paiva, Modelos neuroevolucionários com Echo State Networks aplicados à Identificação de Sistemas, MSc dissertation, Pontifical Catholic University of Rio de Janeiro, Brazil, 2018 (in Portuguese)

    Google Scholar 

  18. P.R.M. de Paiva, M. Vellasco, J. Amaral, Quantum-inspired optimisation of echo state networks applied to system identification, in 2018 IEEE Congress on Evolutionary Computation (IEEE CEC 2018), (2018), pp. 2089–2096

    Google Scholar 

  19. M. Delgado, M.C. Pegalajar, A multiobjective genetic algorithm for obtaining the optimal size of a recurrent neural network for grammatical inference. Pattern Recogn. 38(9), 1444–1456 (2005)

    Article  MATH  Google Scholar 

  20. E.D.M. Dias, M.M.B.R. Vellasco, A.V.A. Cruz, Quantum-inspired neuro coevolution model applied to coordination problems. Expert Syst. Appl. (2020). https://doi.org/10.1016/j.eswa.2020.114133

  21. R. Elwell, R. Polikar, Incremental learning of concept drift in nonstationary environments. IEEE Trans. Neural Netw. 22(10), 1517–1531 (2011)

    Article  Google Scholar 

  22. T. Escovedo, A. Koshiyama, A. Abs da Cruz, M. Vellasco, DetectA: abrupt concept drift detection in nonstationary environments. Appl. Soft Comput. 62, 119–133 (2018)

    Article  Google Scholar 

  23. T. Escovedo, A. Abs Da Cruz, M. Vellasco, A. Koshiyama, Neuroevolutionary learning in nonstationary environments. Appl. Intell. 50, 1590–1608 (2020)

    Article  Google Scholar 

  24. A.A. Ferreira, T.B. Ludermir, Genetic algorithm for reservoir computing optimization, in 2009 International Joint Conference on Neural Networks, Atlanta, GA, (2009), pp. 811–815

    Chapter  Google Scholar 

  25. A.A. Ferreira, T.B. Ludermir, R.R.B. De Aquino, An approach to reservoir computing design and training. Expert Syst. Appl. 40(10), 4172–4182 (2013)

    Article  Google Scholar 

  26. R.S. Ferreira, G. Zimbrão, L.G.M. Alvim, AMANDA: semi-supervised density-based adaptive model for nonstationary data with extreme verification latency. Inf. Sci. 488, 219–237 (2019)

    Article  Google Scholar 

  27. J. Gama, I. Žliobaite, A. Bifet, M. Pechenizkiy, A. Bouchachia, A survey on concept drift adaptation. ACM Comput. Surv. 46(4), Article 44 (2014)

    Article  MATH  Google Scholar 

  28. F. Gomez et al., Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res. 9, 937–965 (2008)

    MathSciNet  MATH  Google Scholar 

  29. I.J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, Y. Bengio, Maxout networks. Proc. Mach. Learn. Res. 28(3), 1319–1327 (2013)

    Google Scholar 

  30. K. Han, J. Kim, Genetic quantum algorithm and its application to combinatorial optimisation problem, in Proceedings of the 2000 Congress on Evolutionary Computation (IEEE CEC 2000), vol. 2, (2000), pp. 1354–1360

    Google Scholar 

  31. K. Han, J. Kim, Quantum-inspired evolutionary algorithm for a class of combinatorial optimisation. IEEE Trans. Evol. Comput. 6(6), 580–593 (2002)

    Article  Google Scholar 

  32. K. Han, J. Kim, On setting the parameters of QEA for practical applications: some guidelines based on empirical evidence, in Genetic and Evolutionary Computation Conference (GECCO 2003), (2003), pp. 427–428

    Chapter  Google Scholar 

  33. K. Han, J. Kim, Quantum-inspired evolutionary algorithms with a new termination criterion, He gate, and two-phase scheme. IEEE Trans. Evol. Comput. 8(2), 156–169 (2004)

    Article  Google Scholar 

  34. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, (2016a), pp. 770–778

    Chapter  Google Scholar 

  35. K. He, X. Zhang, S. Ren, J. Sun, Identity mappings in deep residual networks, in European Conference on Computer Vision (ECCV 2016), (2016b), pp. 630–645

    Chapter  Google Scholar 

  36. G. Hulten, L. Spencer, P. Domingos, Mining time-changing data streams, in Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’01), (2001), pp. 97–106

    Chapter  Google Scholar 

  37. F. Hutter, L. Kotthoff, J. Vanschoren (eds.), Automated Machine Learning: Methods, Systems, Challenges (The Springer Series on Challenges in Machine Learning, 2019)

    Google Scholar 

  38. K. Ishu, T. Van Der Zant, V. Becanovic, P. Ploger, Identification of motion with echo state network, in Oceans’04 MTS/IEEE Techno-Ocean ’04, Kobe, vol. 3, (2004), pp. 1205–1210

    Google Scholar 

  39. H. Jaeger, The “echo state”: approach to analysing and training recurrent neural networks, in GMD Report, vol. 148, (2001)

    Google Scholar 

  40. H. Jaeger, Simple toolbox for ESNs, http://reservoircomputing.org/software (2009)

  41. W. Jia, D. Zhao, T. Shen, C. Su, C. Hu, A new optimized GA-RBF neural network algorithm. Comput. Intell. Neurosci. 2014, Article 982045 (2014)

    Google Scholar 

  42. M.T. Karnick, M. Ahiskali, M. Muhlbaier, R. Polikar, Learning concept drift in nonstationary environments using an ensemble of classifiers based approach, in 2008 International Joint Conference on Neural Networks (IJCNN 2008), (2008), pp. 3455–3462

    Google Scholar 

  43. K.-J. Kim, S.-B. Cho, Evolutionary ensemble of diverse artificial neural networks using speciation. Neurocomputing 71(7–9), 1604–1618 (2008)

    Article  Google Scholar 

  44. B. Krawczyk, A. Cano, Online ensemble learning with abstaining classifiers for drifting and noisy data streams. Appl. Soft Comput. 68, 677–692 (2018)

    Article  Google Scholar 

  45. B. Krawczyk, L.L. Minku, J. Gama, J. Stefanowski, M. Woźniak, Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)

    Article  Google Scholar 

  46. A. Krizhevsky, Learning Multiple Layers of Features from Tiny Images, Technical Report TR-2009 (University of Toronto, 2009)

    Google Scholar 

  47. L.I. Kuncheva, Classifier ensemble for changing environments, in Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 3077, (Springer, Berlin/Heidelberg, 2004)

    Google Scholar 

  48. E. Lacerda, A.C.L.F. Carvalho, A.P. Braga, Evolutionary radial basis functions for credit assessment. Appl. Intell. 22, 167–181 (2005)

    Article  MATH  Google Scholar 

  49. H. Liu, K. Simonyan, O. Vinyals, C. Fernando, K. Kavukcuoglu, Hierarchical representations for efficient architecture search, in International Conference on Learning Representations (ICLR 2018), (2018)

    Google Scholar 

  50. G. Martins, M. Vellasco, R. Schirru, P. Vellasco, Closed-loop identification of nuclear steam generator water level using ESN network tuned by genetic algorithm, in Engineering Applications of Neural Networks (EANN 2015), Communications in Computer and Information Science, vol. 517, (Springer, 2015)

    Google Scholar 

  51. E. Paz, C. Kamath, An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans. Syst. Man Cybern. B Cybern. 35(5), 915–927 (2005)

    Article  Google Scholar 

  52. N. Pedrajas, D.O. Boyer, A cooperative constructive method for neural networks for pattern recognition. Pattern Recogn. 40(1), 80–98 (2007)

    Article  MATH  Google Scholar 

  53. N. Pedrajas, C. H-Martínez, J. Muñoz-Perez, Multiobjective cooperative coevolution of artificial neural networks (multiobjective cooperative networks). Neural Netw. 15(10), 1259–1278 (2002)

    Article  Google Scholar 

  54. N. Pedrajas, C. Hervas-Martinez, J. Munoz-Perez, COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Trans. Neural Netw. 14(3), 575–596 (2003)

    Article  Google Scholar 

  55. N. Pedrajas, C. Hervas-Martinez, D. Ortiz-Boyer, Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans. Evol. Comput. 9(3), 271–302 (2005)

    Article  Google Scholar 

  56. A. Pinho, Algoritmo evolucionário com inspiração quântica e representação mista aplicado a Neuroevolução. Master’s dissertation, Pontifical Catholic University of Rio de Janeiro, Brazil, (2010) (in Portuguese)

    Google Scholar 

  57. A. Pinho, M. Vellasco, A. Abs da Cruz, A new model for credit approval problems: a quantum-inspired neuro-evolutionary algorithm with binary-real representation, in 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC), (2009), pp. 445–450

    Chapter  Google Scholar 

  58. M. Platel, S. Schliebs, N. Kasabov, Quantum-inspired evolutionary algorithm: a multimodel EDA. IEEE Trans. Evol. Comput. 13(6), 1218–1232 (2009)

    Article  Google Scholar 

  59. E. Real, S. Moore, A. Selle, S. Saxena, Y.L. Suematsu, J. Tan, Q.V. Le, A. Kurakin, Large-scale evolution of image classifiers, in Proceedings of the 34th International Conference on Machine Learning, vol. 70, (2017), pp. 2902–2911

    Google Scholar 

  60. J. Schlimmer, R. Granger, Incremental learning from noisy data. Mach. Learn. 1, 317–354 (1986)

    Google Scholar 

  61. R. Schumacher, G.H.C. Oliveira, Uma nova abordagem vector fitting para identificação de sistemas com dados no domínio do tempo. XII Simpósio Brasileiro de Automação Inteligente, Brazil, 283–288 (2015) (in Portuguese)

    Google Scholar 

  62. R.S. Sexton, R.E. Dorsey, Reliable classification using neural networks: a genetic algorithm and backpropagation comparison. Decis. Support. Syst. 30(1), 11–22 (2000)

    Article  Google Scholar 

  63. L. Silveira, R. Tanscheit, M. Vellasco, Quantum inspired evolutionary algorithm for ordering problems. Expert Syst. Appl. 67, 71–83 (2017)

    Article  Google Scholar 

  64. M. Skowronski, J. Harris, Automatic speech recognition using a predictive echo state network classifier. Neural Netw. 20(3), 414–423 (2007)

    Article  MATH  Google Scholar 

  65. R. Stanley, O. Kenneth, Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  66. M. Suganuma, S. Shirakawa, T. Nagao, A genetic programming approach to designing convolutional neural network architectures, in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18), (2018), pp. 5369–5373

    Google Scholar 

  67. D. Szwarcman, Quantum-inspired neural architecture search. PhD thesis, Pontifical Catholic University of Rio de Janeiro, Brazil, 2020

    Google Scholar 

  68. D. Szwarcman, D. Civitarese, M. Vellasco, Quantum-inspired neural architecture search, in 2019 International Joint Conference on Neural Networks (IJCNN 2019), (2019a)

    Google Scholar 

  69. D. Szwarcman, D. Civitarese, M. Vellasco, Q-nas revisited: exploring evolution fitness to improve efficiency, in 2019 8th Brazilian Conference on Intelligent Systems (BRACIS), (2019b), pp. 509–514

    Chapter  Google Scholar 

  70. D.L. Tong, R. Mintram, Genetic algorithm-neural network (GANN): a study of neural network activation functions and depth of genetic algorithm search applied to feature selection. Int. J. Mach. Learn. Cybern. 1, 75–87 (2010)

    Article  Google Scholar 

  71. A. Tsymbal, The problem of concept drift: definitions and related work, in Technical Report, Trinity College Dublin, ICD-CS-2004-15, (2004)

    Google Scholar 

  72. M.M.B.R. Vellasco, A.V. Abs da Cruz, A.G. Pinho, Quantum-inspired evolutionary algorithms applied to neural network modeling, in IEEE World Congress on Computational Intelligence (IEEE WCCI 2010), Plenary and Invited Lectures, ed. by J. Aranda, S. Xambó, (2010), pp. 125–150

    Google Scholar 

  73. L. Xie, A. Yuille, Genetic CNN, in 2017 IEEE International Conference on Computer Vision (ICCV), (2017), pp. 1388–1397

    Chapter  Google Scholar 

  74. X. Yao, Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)

    Article  Google Scholar 

  75. R. Ye, Q. Dai, A novel greedy randomised dynamic ensemble selection algorithm. Neural. Process. Lett. 47, 565–599 (2018)

    Google Scholar 

  76. L. Zhan et al., ANN-GA approach of credit scoring for mobile customers, in IEEE Conference on Cybernetics and Intelligent Systems, (2004), pp. 1148–1153

    Google Scholar 

  77. L.M. Zhang, Genetic deep neural networks using different activation functions for financial data mining, in 2015 IEEE International Conference on Big Data (Big Data), (2015), pp. 2849–2851

    Chapter  Google Scholar 

  78. B. Zhang, L. Xue, W. Wang, S. Qin, D. Wang, Model updating mechanism of concept drift detection in data stream based on classifier pool. EURASIP J. Wirel. Commun. Netw. (2016)

    Google Scholar 

  79. B. Zoph, Q.V. Le, Neural architecture search with reinforcement learning. https://arxiv.org/abs/1611.01578. (2016)

Download references

Acknowledgements

The authors would like to thank the Brazilian Agencies CNPQ (Conselho Nacional de Pesquisa e Desenvolvimento) and FAPERJ (Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro) for their financial support in the development of the research projects presented in this chapter.

Author information

Authors and Affiliations

  1. Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Rio de Janeiro, Brazil

    Tatiana Escovedo & Marley Vellasco

  2. State University of Rio de Janeiro (UERJ), Rio de Janeiro, Brazil

    Karla Figueiredo

  3. IBM Research, Rio de Janeiro, Brazil

    Daniela Szwarcman

Authors
  1. Tatiana Escovedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karla Figueiredo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniela Szwarcman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marley Vellasco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marley Vellasco .

Editor information

Editors and Affiliations

  1. Department of Industrial and Systems Engineering and Department of Computer Science and Software Engineering, Auburn University, Auburn, AL, USA

    Alice E Smith

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Escovedo, T., Figueiredo, K., Szwarcman, D., Vellasco, M. (2022). Neuroevolutionary Models Based on Quantum-Inspired Evolutionary Algorithms. In: Smith, A.E. (eds) Women in Computational Intelligence. Women in Engineering and Science. Springer, Cham. https://doi.org/10.1007/978-3-030-79092-9_14

Download citation

Publish with us

Policies and ethics

Search

Navigation