Skip to main content
Springer Nature Link
Log in

Evolution of Hybrid Robotic Controllers for Complex Tasks

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

We propose an approach to the synthesis of hierarchical control systems comprising both evolved and manually programmed control for autonomous robots. We recursively divide the goal task into sub-tasks until a solution can be evolved or until a solution can easily be programmed by hand. Hierarchical composition of behavior allows us to overcome the fundamental challenges that typically prevent evolutionary robotics from being applied to complex tasks: bootstrapping the evolutionary process, avoiding deception, and successfully transferring control evolved in simulation to real robotic hardware. We demonstrate the proposed approach by synthesizing control systems for two tasks whose complexity is beyond state of the art in evolutionary robotics. The first task is a rescue task in which all behaviors are evolved. The second task is a cleaning task in which evolved behaviors are combined with a manually programmed behavior that enables the robot to open doors in the environment. We demonstrate incremental transfer of evolved control from simulation to real robotic hardware, and we show how our approach allows for the reuse of behaviors in different tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Becerra, J.A., Bellas, F., Reyes, J.S., Duro, R.J.: Complex behaviours through modulation in autonomous robot control. In Proceedings of the International Work-Conference on Artificial Neural Networks (IWANN), volume 3512 of Lecture Notes in Computer Science, pp 717–724. Springer, Berlin (2005)

    Google Scholar 

  2. Beer, R.D., Gallagher, J.C.: Evolving dynamical neural networks for adaptive behavior. Adapt. Behav. 1 (1), 91–122 (1992)

    Article  Google Scholar 

  3. Blynel, J., Floreano, D.: Exploring the T-Maze: Evolving learning-like robot behaviors using CTRNNs. In Applications of Evolutionary Computing, pp 593–604. Springer, Berlin (2003)

    Google Scholar 

  4. Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools 25, 11 (2000)

    Google Scholar 

  5. Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2 (1), 14–23 (1986)

    Article  Google Scholar 

  6. Celis, S., Hornby, G.S, Bongard, J.: Avoiding local optima with user demonstrations and low-level control. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 3403–3410. IEEE Press, Piscataway (2013)

    Google Scholar 

  7. Cheney, N., MacCurdy, R., Clune, J., Lipson, H.: Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 167–174. ACM Press, New York (2013)

    Google Scholar 

  8. Christensen, A.L., Dorigo, M.: Incremental evolution of robot controllers for a highly integrated task. In From Animals to Animats 9: Proceedings of the International Conference on the Simulation of Adaptive Behavior (SAB), pp 473–484. Springer, Berlin (2006a)

    Google Scholar 

  9. Christensen, A.L., Dorigo, M.: Evolving an integrated phototaxis and hole avoidance behavior for a swarm-bot. In Proceedings of the International Conference on the Simulation & Synthesis of Living Systems (ALIFE), pp 248–254. MIT Press, Cambridge (2006b)

    Google Scholar 

  10. Christensen, A.L., Oliveira, S., Duartem, M.: JBotEvolver (2014). http://code.google.com/p/jbotevolver

  11. de Nardi, R., Togelius, J., Holland, O.E., Lucas, S.M.: Evolution of neural networks for helicopter control, Why modularity matters. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 1799–1806. IEEE Press, Piscataway (2006)

    Google Scholar 

  12. Duarte, M., Oliveira, S., Christensen, A.L.: Towards artificial evolution of complex behavior observed in insect colonies. In Proceedings of the Portuguese Conference on Artificial Intelligence (EPIA), pp 153–167. Springer, Berlin (2011)

    Google Scholar 

  13. Duarte, M., Oliveira, S., Christensen, A.L.: Hierarchical evolution of robotic controllers for complex tasks. In IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), pp 1–6. IEEE Press, Piscataway (2012)

    Google Scholar 

  14. Duarte, M., Oliveira, S., Christensen, A.L.: Automatic synthesis of controllers for real robots based on preprogrammed behaviors. In From Animals to Animats 12: Proceedings of the International Conference on Adaptive Behaviour (SAB), pp 249–258. Springer, Berlin (2012)

    Google Scholar 

  15. Duarte, M., Oliveira, S., Christensen, A.L.: Videos and source code from real robot experiments (2014). http://home.iscte-iul.pt/~alcen/hybridcontrollers/

  16. Duarte, M., Silva, F., Rodrigues, T., Oliveira, S.M., Christensen, A.L.: JBotEvolver: A versatile simulation platform for evolutionary robotics. In: Proceedings of the International Conference on the Simulation & Synthesis of Living Systems (ALIFE), pp 210–211. MIT Press, Cambridge (2014)

  17. Er, M.J., Kee, B.H., Tan, C.C.: Design and development of an intelligent controller for a pole-balancing robot. Microprocess. Microsyst. 26 (9-10), 433–448 (2002)

    Article  Google Scholar 

  18. Floreano, D., Keller, L.: Evolution of adaptive behaviour in robots by means of Darwinian selection. PLoS Biol. 8 (1), 1–8 (2010)

    Article  Google Scholar 

  19. Floreano, D., Mondada, F.: Automatic creation of an autonomous agent, Genetic evolution of a neural-network driven robot. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 421–430. MIT Press, Cambridge (1994). USA

    Google Scholar 

  20. Floreano, D., Mondada, F.: Evolution of homing navigation in a real mobile robot. IEEE Trans. Syst. Man Cybern.–Part B: Cybern. 26 (3), 396–407 (1996)

    Article  Google Scholar 

  21. Floreano, D., Urzelai, J.: Evolutionary robots with on-line self-organization and behavioral fitness. Neural Netw. 13 (4–5), 431–443 (2000)

    Article  Google Scholar 

  22. Fogel, D.B., Fogel, L.J., Porto, V.W.: Evolving neural networks. Biol. Cybern. 63 (6), 487–493 (1990)

    Article  Google Scholar 

  23. Goldberg, D.E.: Genetic algorithms in search, optimization and machine learning. Addison-Wesley, MA (1989)

    MATH  Google Scholar 

  24. Gomes, J., Urbano, P., Christensen, A.L.: Evolution of swarm robotics systems with novelty search. Swarm Intell. 7 (2-3), 115–144 (2013)

    Article  Google Scholar 

  25. Gomes, J., Urbano, P., Christensen, A.L.: PMCNS: using a progressively stricter fitness criterion to guide novelty search. In International Journal of Natural Computing Research, in press (2014)

  26. Gomez, F., Miikkulainen, R.: Incremental evolution of complex general behavior. Adapt. Behav. 3-4 (5), 317–342 (1997)

    Article  Google Scholar 

  27. Groß, R., Bonani, M., Mondada, F., Dorigo, M.: Autonomous self-assembly in swarmbots. IEEE Trans. Robot. 22 (6), 1115–1130 (2006)

    Article  Google Scholar 

  28. Gutiérrez, A., Campo, A., Dorigo, M., Amor, D., Magdalena, L., Monasterio-Huelin, F.: An open localization and local communication embodied sensor. Sensors 8 (11), 7545–7563 (2008)

    Article  Google Scholar 

  29. Hamann, H., Schmickl, T., Crailsheim, K.: A hormone-based controller for evaluation-minimal evolution in decentrally controlled systems. Artif. Life 18 (2), 165–198 (2012)

    Article  MathSciNet  Google Scholar 

  30. Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12 (10), 993–1001 (1990)

    Article  Google Scholar 

  31. Harvey, I., Husbands, P., Cliff, D.: Seeing the light: artificial evolution, real vision. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 392–401. MIT Press, Cambridge (1994)

    Google Scholar 

  32. Heaton, J.: Encog machine learning framework (2008). http://www.heatonresearch.com/encog

  33. Hehn, M., D’Andrea, R.: A flying inverted pendulum. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp 763–770. IEEE Press, Piscataway (2011)

    Google Scholar 

  34. Hiller, J., Lipson, H.: Automatic design and manufacture of soft robots. IEEE Trans. Robot. 28 (2), 457–466 (2012)

    Article  Google Scholar 

  35. Hornby, G.S., Takamura, S., Yamamoto, T., Fujita, M.: Autonomous evolution of dynamic gaits with two quadruped robots. IEEE Trans. Robot. 21 (3), 402–410 (2005)

    Article  Google Scholar 

  36. Husbands, P.: Evolving robot behaviours with diffusing gas networks. In Proceedigs of the European Workshop Evolutionary Robotics (EvoRobot), pp 71–86. Springer, Berlin (1998)

    Google Scholar 

  37. Jakobi, N.: Evolutionary robotics and the radical envelope-of-noise hypothesis. Adapt. Behav. 6 (2), 325–368 (1997)

    Article  Google Scholar 

  38. Koos, S., Mouret, J.-B., Doncieux, S.: The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Trans. Evol. Comput. 17 (1), 122–145 (2013)

    Article  Google Scholar 

  39. Larsen, T., Hansen, S.T.: Evolving composite robot behaviour - a modular architecture. In Proceedings of the International Workshop on Robot Motion and Control (RoMoCo), pp 271–276. IEEE Press, Piscataway (2005)

    Google Scholar 

  40. Lee, W.-P.: Evolving complex robot behaviors. Inf. Sci. 121 (1-2), 1–25 (1999)

    Article  Google Scholar 

  41. Lehman, J., Stanley, K.O.: Abandoning objectives: Evolution through the search for novelty alone. Evol. Comput. 19 (2), 189–223 (2011)

    Article  Google Scholar 

  42. Meyer, J.-A., Husbands, P., Harvey, I.: Evolutionary robotics: a survey of applications and problems. In Proceedings of the European Workshop on Evolutionary Robotics (EvoRobot), pp 1–21. Springer, Berlin (1998)

    Book  Google Scholar 

  43. Miglino, O., Lund, H.H., Nolfi, S.: Evolving mobile robots in simulated and real environments. Artif. Life 2 (4), 417–434 (1996)

    Article  Google Scholar 

  44. Moioli, R.C., Vargas, P.A., Von Zuben, F.J., Husbands, P.: Towards the evolution of an artificial homeostatic system. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 4023–4030. IEEE Press, Piscataway (2008)

    Google Scholar 

  45. Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, J.-C., Floreano, D., Martinoli, A.: The e-puck, a robot designed for education in engineering. In Proceedings of the Conference on Autonomous Robot Systems and Competitions (ROBOTICA), pp 59–65 (2009). Instituto Politecnico de Castelo Branco, Castelo Branco, Portugal

  46. Nakamura, H., Ishiguro, A., Uchilkawa, Y.: Evolutionary construction of behavior arbitration mechanisms based on dynamically-rearranging neural networks. In Proceedings of Congress on Evolutionary Computation (CEC), pp 158–165. IEEE Press, Piscataway (2000)

  47. Nelson, A.L., Barlow, G.J., Doitsidis, L.: Fitness functions in evolutionary robotics: a survey and analysis. Robot. Auton. Syst. 57 (4), 345–370 (2009)

    Article  Google Scholar 

  48. Nolfi, S., Floreano, D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. MIT press, Cambridge (2000)

    Google Scholar 

  49. Nolfi, S., Parisi, D.: Evolving non-trivial behaviors on real robots: an autonomous robot that picks up objects. In Proceedings of the Congress of the Italian Association for Artificial Intelligence (AI*IA), pp 187–198. Springer, Berlin (1995)

    Google Scholar 

  50. Nolfi, S., Floreano, D., Miglino, O., Mondada, F.: How to evolve autonomous robots: Different approaches in evolutionary robotics. In Proceedings of the International Workshop on Artificial Life (ALIFE), pp 190–197. MIT Press, Cambridge (1994)

    Google Scholar 

  51. O’Grady, R., Christensen, A.L., Dorigo, M.: SWARMORPH: multirobot morphogenesis using directional self-assembly. IEEE Trans. Robot. 25 (3), 738–743 (2009)

    Article  Google Scholar 

  52. Okamura, A.M., Smaby, N., Cutkosky, M.R.: An overview of dexterous manipulation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), vol. 1, pp. 255–263. IEEE Press, Piscataway (2000)

    Google Scholar 

  53. Reynolds, C.W.: Evolution of corridor following behavior in a noisy world. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 402–410. MIT Press, Cambridge (1994)

    Google Scholar 

  54. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10 (2), 99–127 (2002)

    Article  Google Scholar 

  55. Stanley, K.O., Miikkulainen, R.P.: Efficient evolution of neural networks through complexification. Computer Science Department, University of Texas at Austin (2004)

  56. Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15 (2), 185–212 (2009)

    Article  Google Scholar 

  57. Tolman, E.C., Honzik, C.H.: Introduction and removal of reward, and maze performance in rats. Univ. Calif. Publ. Psychol. 4, 257–275 (1930)

    Google Scholar 

  58. Torta, A.B. L., Kramer, M.A., Thorn, C., Gibson, D.J., Kubota, Y., Graybiel, A.M., Kopell, N.J.: Dynamic cross-frequency couplings of local field potential oscillations in rat striatum and hippocampus during performance of a T-maze task. Proc. Natl. Acad. Sci. 105 (51), 20517–20522 (2008)

    Article  Google Scholar 

  59. Tuci, E., Trianni, V., M. Dorigo: ‘Feeling’ the flow of time through sensorimotor co-ordination. Connect. Sci. 16 (4), 301–324 (2004)

    Article  Google Scholar 

  60. Tunstel, E.: Mobile robot autonomy via hierarchical fuzzy behavior control. In: Proceedings of the International Symposium on Robotics and Manufacturing (WAC), pp 837–842. ASME Press, New York (1996)

    Google Scholar 

  61. Watson, R.A., Ficici, S.G., Pollack, J.B.: Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robot. Auton. Syst. 39 (1), 1–18 (2002)

    Article  Google Scholar 

  62. Whitley, L.D.: Fundamental principles of deception in genetic search. In Foundations of Genetic Algorithms, pp. 221–241. Morgan Kaufmann (1991)

Download references

Author information

Authors and Affiliations

  1. Instituto Universitário de Lisboa (ISCTE-IUL) and Instituto de Telecomunicações, Avenida das Forças Armadas, 1649-026, Lisboa, Portugal

    Miguel Duarte, Sancho Moura Oliveira & Anders Lyhne Christensen

Authors
  1. Miguel Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sancho Moura Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anders Lyhne Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miguel Duarte.

Additional information

This work was partly supported by FCT – Fundação Ciência e Tecnologia (FCT) Portugal under grants SFRH/BD/76438/ 2011, PEst-OE/EEI/LA0008/2013 and EXPL/EEI-AUT/ 0329/2013.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duarte, M., Oliveira, S.M. & Christensen, A.L. Evolution of Hybrid Robotic Controllers for Complex Tasks. J Intell Robot Syst 78, 463–484 (2015). https://doi.org/10.1007/s10846-014-0086-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-014-0086-x

Keywords

Search

Navigation