Abstract
We propose an approach to the synthesis of hierarchical control systems comprising both evolved and manually programmed control for autonomous robots. We recursively divide the goal task into sub-tasks until a solution can be evolved or until a solution can easily be programmed by hand. Hierarchical composition of behavior allows us to overcome the fundamental challenges that typically prevent evolutionary robotics from being applied to complex tasks: bootstrapping the evolutionary process, avoiding deception, and successfully transferring control evolved in simulation to real robotic hardware. We demonstrate the proposed approach by synthesizing control systems for two tasks whose complexity is beyond state of the art in evolutionary robotics. The first task is a rescue task in which all behaviors are evolved. The second task is a cleaning task in which evolved behaviors are combined with a manually programmed behavior that enables the robot to open doors in the environment. We demonstrate incremental transfer of evolved control from simulation to real robotic hardware, and we show how our approach allows for the reuse of behaviors in different tasks.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Becerra, J.A., Bellas, F., Reyes, J.S., Duro, R.J.: Complex behaviours through modulation in autonomous robot control. In Proceedings of the International Work-Conference on Artificial Neural Networks (IWANN), volume 3512 of Lecture Notes in Computer Science, pp 717–724. Springer, Berlin (2005)
Beer, R.D., Gallagher, J.C.: Evolving dynamical neural networks for adaptive behavior. Adapt. Behav. 1 (1), 91–122 (1992)
Blynel, J., Floreano, D.: Exploring the T-Maze: Evolving learning-like robot behaviors using CTRNNs. In Applications of Evolutionary Computing, pp 593–604. Springer, Berlin (2003)
Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools 25, 11 (2000)
Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2 (1), 14–23 (1986)
Celis, S., Hornby, G.S, Bongard, J.: Avoiding local optima with user demonstrations and low-level control. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 3403–3410. IEEE Press, Piscataway (2013)
Cheney, N., MacCurdy, R., Clune, J., Lipson, H.: Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 167–174. ACM Press, New York (2013)
Christensen, A.L., Dorigo, M.: Incremental evolution of robot controllers for a highly integrated task. In From Animals to Animats 9: Proceedings of the International Conference on the Simulation of Adaptive Behavior (SAB), pp 473–484. Springer, Berlin (2006a)
Christensen, A.L., Dorigo, M.: Evolving an integrated phototaxis and hole avoidance behavior for a swarm-bot. In Proceedings of the International Conference on the Simulation & Synthesis of Living Systems (ALIFE), pp 248–254. MIT Press, Cambridge (2006b)
Christensen, A.L., Oliveira, S., Duartem, M.: JBotEvolver (2014). http://code.google.com/p/jbotevolver
de Nardi, R., Togelius, J., Holland, O.E., Lucas, S.M.: Evolution of neural networks for helicopter control, Why modularity matters. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 1799–1806. IEEE Press, Piscataway (2006)
Duarte, M., Oliveira, S., Christensen, A.L.: Towards artificial evolution of complex behavior observed in insect colonies. In Proceedings of the Portuguese Conference on Artificial Intelligence (EPIA), pp 153–167. Springer, Berlin (2011)
Duarte, M., Oliveira, S., Christensen, A.L.: Hierarchical evolution of robotic controllers for complex tasks. In IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), pp 1–6. IEEE Press, Piscataway (2012)
Duarte, M., Oliveira, S., Christensen, A.L.: Automatic synthesis of controllers for real robots based on preprogrammed behaviors. In From Animals to Animats 12: Proceedings of the International Conference on Adaptive Behaviour (SAB), pp 249–258. Springer, Berlin (2012)
Duarte, M., Oliveira, S., Christensen, A.L.: Videos and source code from real robot experiments (2014). http://home.iscte-iul.pt/~alcen/hybridcontrollers/
Duarte, M., Silva, F., Rodrigues, T., Oliveira, S.M., Christensen, A.L.: JBotEvolver: A versatile simulation platform for evolutionary robotics. In: Proceedings of the International Conference on the Simulation & Synthesis of Living Systems (ALIFE), pp 210–211. MIT Press, Cambridge (2014)
Er, M.J., Kee, B.H., Tan, C.C.: Design and development of an intelligent controller for a pole-balancing robot. Microprocess. Microsyst. 26 (9-10), 433–448 (2002)
Floreano, D., Keller, L.: Evolution of adaptive behaviour in robots by means of Darwinian selection. PLoS Biol. 8 (1), 1–8 (2010)
Floreano, D., Mondada, F.: Automatic creation of an autonomous agent, Genetic evolution of a neural-network driven robot. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 421–430. MIT Press, Cambridge (1994). USA
Floreano, D., Mondada, F.: Evolution of homing navigation in a real mobile robot. IEEE Trans. Syst. Man Cybern.–Part B: Cybern. 26 (3), 396–407 (1996)
Floreano, D., Urzelai, J.: Evolutionary robots with on-line self-organization and behavioral fitness. Neural Netw. 13 (4–5), 431–443 (2000)
Fogel, D.B., Fogel, L.J., Porto, V.W.: Evolving neural networks. Biol. Cybern. 63 (6), 487–493 (1990)
Goldberg, D.E.: Genetic algorithms in search, optimization and machine learning. Addison-Wesley, MA (1989)
Gomes, J., Urbano, P., Christensen, A.L.: Evolution of swarm robotics systems with novelty search. Swarm Intell. 7 (2-3), 115–144 (2013)
Gomes, J., Urbano, P., Christensen, A.L.: PMCNS: using a progressively stricter fitness criterion to guide novelty search. In International Journal of Natural Computing Research, in press (2014)
Gomez, F., Miikkulainen, R.: Incremental evolution of complex general behavior. Adapt. Behav. 3-4 (5), 317–342 (1997)
Groß, R., Bonani, M., Mondada, F., Dorigo, M.: Autonomous self-assembly in swarmbots. IEEE Trans. Robot. 22 (6), 1115–1130 (2006)
Gutiérrez, A., Campo, A., Dorigo, M., Amor, D., Magdalena, L., Monasterio-Huelin, F.: An open localization and local communication embodied sensor. Sensors 8 (11), 7545–7563 (2008)
Hamann, H., Schmickl, T., Crailsheim, K.: A hormone-based controller for evaluation-minimal evolution in decentrally controlled systems. Artif. Life 18 (2), 165–198 (2012)
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12 (10), 993–1001 (1990)
Harvey, I., Husbands, P., Cliff, D.: Seeing the light: artificial evolution, real vision. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 392–401. MIT Press, Cambridge (1994)
Heaton, J.: Encog machine learning framework (2008). http://www.heatonresearch.com/encog
Hehn, M., D’Andrea, R.: A flying inverted pendulum. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp 763–770. IEEE Press, Piscataway (2011)
Hiller, J., Lipson, H.: Automatic design and manufacture of soft robots. IEEE Trans. Robot. 28 (2), 457–466 (2012)
Hornby, G.S., Takamura, S., Yamamoto, T., Fujita, M.: Autonomous evolution of dynamic gaits with two quadruped robots. IEEE Trans. Robot. 21 (3), 402–410 (2005)
Husbands, P.: Evolving robot behaviours with diffusing gas networks. In Proceedigs of the European Workshop Evolutionary Robotics (EvoRobot), pp 71–86. Springer, Berlin (1998)
Jakobi, N.: Evolutionary robotics and the radical envelope-of-noise hypothesis. Adapt. Behav. 6 (2), 325–368 (1997)
Koos, S., Mouret, J.-B., Doncieux, S.: The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Trans. Evol. Comput. 17 (1), 122–145 (2013)
Larsen, T., Hansen, S.T.: Evolving composite robot behaviour - a modular architecture. In Proceedings of the International Workshop on Robot Motion and Control (RoMoCo), pp 271–276. IEEE Press, Piscataway (2005)
Lee, W.-P.: Evolving complex robot behaviors. Inf. Sci. 121 (1-2), 1–25 (1999)
Lehman, J., Stanley, K.O.: Abandoning objectives: Evolution through the search for novelty alone. Evol. Comput. 19 (2), 189–223 (2011)
Meyer, J.-A., Husbands, P., Harvey, I.: Evolutionary robotics: a survey of applications and problems. In Proceedings of the European Workshop on Evolutionary Robotics (EvoRobot), pp 1–21. Springer, Berlin (1998)
Miglino, O., Lund, H.H., Nolfi, S.: Evolving mobile robots in simulated and real environments. Artif. Life 2 (4), 417–434 (1996)
Moioli, R.C., Vargas, P.A., Von Zuben, F.J., Husbands, P.: Towards the evolution of an artificial homeostatic system. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 4023–4030. IEEE Press, Piscataway (2008)
Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, J.-C., Floreano, D., Martinoli, A.: The e-puck, a robot designed for education in engineering. In Proceedings of the Conference on Autonomous Robot Systems and Competitions (ROBOTICA), pp 59–65 (2009). Instituto Politecnico de Castelo Branco, Castelo Branco, Portugal
Nakamura, H., Ishiguro, A., Uchilkawa, Y.: Evolutionary construction of behavior arbitration mechanisms based on dynamically-rearranging neural networks. In Proceedings of Congress on Evolutionary Computation (CEC), pp 158–165. IEEE Press, Piscataway (2000)
Nelson, A.L., Barlow, G.J., Doitsidis, L.: Fitness functions in evolutionary robotics: a survey and analysis. Robot. Auton. Syst. 57 (4), 345–370 (2009)
Nolfi, S., Floreano, D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. MIT press, Cambridge (2000)
Nolfi, S., Parisi, D.: Evolving non-trivial behaviors on real robots: an autonomous robot that picks up objects. In Proceedings of the Congress of the Italian Association for Artificial Intelligence (AI*IA), pp 187–198. Springer, Berlin (1995)
Nolfi, S., Floreano, D., Miglino, O., Mondada, F.: How to evolve autonomous robots: Different approaches in evolutionary robotics. In Proceedings of the International Workshop on Artificial Life (ALIFE), pp 190–197. MIT Press, Cambridge (1994)
O’Grady, R., Christensen, A.L., Dorigo, M.: SWARMORPH: multirobot morphogenesis using directional self-assembly. IEEE Trans. Robot. 25 (3), 738–743 (2009)
Okamura, A.M., Smaby, N., Cutkosky, M.R.: An overview of dexterous manipulation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), vol. 1, pp. 255–263. IEEE Press, Piscataway (2000)
Reynolds, C.W.: Evolution of corridor following behavior in a noisy world. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 402–410. MIT Press, Cambridge (1994)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10 (2), 99–127 (2002)
Stanley, K.O., Miikkulainen, R.P.: Efficient evolution of neural networks through complexification. Computer Science Department, University of Texas at Austin (2004)
Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15 (2), 185–212 (2009)
Tolman, E.C., Honzik, C.H.: Introduction and removal of reward, and maze performance in rats. Univ. Calif. Publ. Psychol. 4, 257–275 (1930)
Torta, A.B. L., Kramer, M.A., Thorn, C., Gibson, D.J., Kubota, Y., Graybiel, A.M., Kopell, N.J.: Dynamic cross-frequency couplings of local field potential oscillations in rat striatum and hippocampus during performance of a T-maze task. Proc. Natl. Acad. Sci. 105 (51), 20517–20522 (2008)
Tuci, E., Trianni, V., M. Dorigo: ‘Feeling’ the flow of time through sensorimotor co-ordination. Connect. Sci. 16 (4), 301–324 (2004)
Tunstel, E.: Mobile robot autonomy via hierarchical fuzzy behavior control. In: Proceedings of the International Symposium on Robotics and Manufacturing (WAC), pp 837–842. ASME Press, New York (1996)
Watson, R.A., Ficici, S.G., Pollack, J.B.: Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robot. Auton. Syst. 39 (1), 1–18 (2002)
Whitley, L.D.: Fundamental principles of deception in genetic search. In Foundations of Genetic Algorithms, pp. 221–241. Morgan Kaufmann (1991)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was partly supported by FCT – Fundação Ciência e Tecnologia (FCT) Portugal under grants SFRH/BD/76438/ 2011, PEst-OE/EEI/LA0008/2013 and EXPL/EEI-AUT/ 0329/2013.
Rights and permissions
About this article
Cite this article
Duarte, M., Oliveira, S.M. & Christensen, A.L. Evolution of Hybrid Robotic Controllers for Complex Tasks. J Intell Robot Syst 78, 463–484 (2015). https://doi.org/10.1007/s10846-014-0086-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-014-0086-x