×

Envisioning the qualitative effects of robot manipulation actions using simulation-based projections. (English) Zbl 1420.68214

Summary: Autonomous robots that are to perform complex everyday tasks such as making pancakes have to understand how the effects of an action depend on the way the action is executed. Within Artificial Intelligence, classical planning reasons about whether actions are executable, but makes the assumption that the actions will succeed (with some probability). In this work, we have designed, implemented, and analyzed a framework that allows us to envision the physical effects of robot manipulation actions. We consider envisioning to be a qualitative reasoning method that reasons about actions and their effects based on simulation-based projections. Thereby it allows a robot to infer what could happen when it performs a task in a certain way. This is achieved by translating a qualitative physics problem into a parameterized simulation problem; performing a detailed physics-based simulation of a robot plan; logging the state evolution into appropriate data structures; and then translating these sub-symbolic data structures into interval-based first-order symbolic, qualitative representations, called timelines. The result of the envisioning is a set of detailed narratives represented by timelines which are then used to infer answers to qualitative reasoning problems. By envisioning the outcome of actions before committing to them, a robot is able to reason about physical phenomena and can therefore prevent itself from ending up in unwanted situations. Using this approach, robots can perform manipulation tasks more efficiently, robustly, and flexibly, and they can even successfully accomplish previously unknown variations of tasks.

MSC:

68T40 Artificial intelligence for robotics
68U20 Simulation (MSC2010)

Software:

KnowRob; IsisWorld; WEKA

References:

[1] Allen, J., Maintaining knowledge about temporal intervals, Commun. ACM, 26, 832-843 (1983) · Zbl 0519.68079
[2] Beetz, M.; Klank, U.; Kresse, I.; Maldonado, A.; Mösenlechner, L.; Pangercic, D.; Rühr, T.; Tenorth, M., Robotic roommates making pancakes, (11th IEEE-RAS International Conference on Humanoid Robots. 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia (2011)), 529-536
[3] Beetz, M.; Mösenlechner, L.; Tenorth, M., CRAM—a cognitive robot abstract machine for everyday manipulation in human environments, (IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan (2010)), 1012-1017
[4] Beetz, M.; Tenorth, M.; Jain, D.; Bandouch, J., Towards automated models of activities of daily life, Technol. Disabil., 22, 27-40 (2010)
[5] Berleant, D.; Kuipers, B., Qualitative-numeric simulation with Q3, (Recent Advances in Qualitative Physics (1992), The MIT Press), 3-16
[6] Brown, J.; Sorkin, S.; Bruyns, C.; Latombe, J. C.; Montgomery, K.; Stephanides, M., Real-time simulation of deformable objects: tools and application, (Computer Animation, 2001: The Fourteenth Conference on Computer Animation (2001)), 228-236
[7] Cho, J. H.; Xenakis, A.; Gronsky, S.; Shah, A., Anyone can cook—inside Ratatouille’s kitchen, (SIGGRAPH ’07: ACM SIGGRAPH 2007 Courses (2007))
[8] Davis, E., Representations of Commonsense Knowledge, Morgan Kaufmann Series in Representation and Reasoning (1990), Morgan Kaufmann
[9] Davis, E., Pouring liquids: a study in commonsense physical reasoning, Artif. Intell., 172, 1540-1578 (2008) · Zbl 1183.68594
[10] Davis, E., Ontologies and representations of matter, (Fox, M.; Poole, D., Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence. Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2010, Atlanta, Georgia, USA, July 11-15 (2010), AAAI Press)
[11] Dean, T., Using temporal hierarchies to efficiently maintain large temporal databases, J. ACM, 36, 687-718 (1989) · Zbl 0698.68091
[12] Dogar, M.; Srinivasa, S., A framework for push-grasping in clutter, (Proceedings of Robotics: Science and Systems. Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA (2011))
[13] Faloutsos, P.; van de Panne, M.; Terzopoulos, D., Composable controllers for physics-based character animation, (Proc. of the 28th Annual Conf. on Computer Graphics and Interactive Techniques (2001))
[14] Forbus, K. D., Qualitative process theory, Artif. Intell., 24, 85-168 (1984)
[15] Forbus, K. D.; Falkenhainer, B., Self-explanatory simulations: an integration of qualitative and quantitative knowledge, (AAAI (1990)), 380-387
[16] Frank, B.; Stachniss, C.; Schmedding, R.; Teschner, M.; Burgard, W., Real-world robot navigation amongst deformable obstacles, (ICRA’09: Proc. of the 2009 IEEE Int. Conf. on Robotics and Automation (2009))
[17] Frenkel, D.; Smit, B., Understanding Molecular Simulation, Second Edition: From Algorithms to Applications (Computational Science) (2001), Academic Press
[18] Ghallab, M., On chronicles: representation, on-line recognition and learning, (Principles of Knowledge Representation and Reasoning—International Conference (1996), Morgan Kaufmann Publishers), 597-607
[19] Gould, H.; Tobochnik, J.; Wolfgang, C., An Introduction to Computer Simulation Methods: Applications to Physical Systems (2005), Addison-Wesley Longman Publishing Co., Inc.: Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA
[20] Griebel, M.; Knapek, S.; Zumbusch, G., Numerical Simulation in Molecular Dynamics: Numerics, Algorithms, Parallelization, Applications (2007), Springer Publishing Company, Incorporated · Zbl 1131.76001
[21] Haazebroek, P.; Hommel, B., Anticipative control of voluntary action: towards a computational model, (Pezzulo, G.; Butz, M.; Sigaud, O.; Baldassarre, G., Anticipatory Behavior in Adaptive Learning Systems. Anticipatory Behavior in Adaptive Learning Systems, Lecture Notes in Computer Science, vol. 5499 (2009), Springer: Springer Berlin/Heidelberg), 31-47
[22] Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I. H., The WEKA data mining software: an update, ACM SIGKDD Explor. Newsl., 11, 10-18 (2009)
[23] Hayes, P., The naive physics manifesto, (Michie, D., Expert Systems in the Micro Electronic Age (1979), Edinburgh University Press), 242-270
[24] Hayes, P., The second naive physics manifesto, (Hobbs, J. R.; Moore, R. C., Formal Theories of the Commonsense World (1985), Ablex: Ablex Norwood, NJ), 1-36
[25] Hesslow, G., The current status of the simulation theory of cognition, Brains Res. Rev., 1428, 71-79 (2012)
[26] Ingram, J.; Howard, I.; Flanagan, J.; Wolpert, D., Multiple grasp-specific representations of tool dynamics mediate skillful manipulation, Curr. Biol., 20, 7, 618-623 (2010)
[27] Johnston, B.; Williams, M., Comirit: commonsense reasoning by integrating simulation and logic, (Artificial General Intelligence 2008: Proceedings of the First AGI Conference (2008), IOS Press), 200
[28] Johnston, B.; Williams, M., Autonomous learning of commonsense simulations, (Int. Symposium on Logical Formalizations of Commonsense Reasoning (2009))
[29] Kato, F.; Hanaoka, Y.; Ngoc, T. N.; Keoki, D.; Mitake, H.; Aoki, T.; Hasegawa, S., Interactive cooking simulator: to understand cooking operation deeply, (ACM SIGGRAPH 2009 Emerging Technologies (2009))
[30] Kemp, C.; Edsinger, A.; Torres-Jara, E., Challenges for robot manipulation in human environments, IEEE Robot. Autom. Mag., 14, 20-29 (2007)
[31] Klapfer, R.; Kunze, L.; Beetz, M., Pouring and mixing liquids—understanding the physical effects of everyday robot manipulation actions, (35th German Conference on Artificial Intelligence (KI-2012), Workshop on Human Reasoning and Automated Deduction. 35th German Conference on Artificial Intelligence (KI-2012), Workshop on Human Reasoning and Automated Deduction, Saarbrücken, Germany (2012))
[32] de Kleer, J., Multiples representations of knowledge in a mechanics problem-solver, (Proceedings of the 5th International Joint Conference on Artificial Intelligence, vol. 11 (1977), Morgan Kaufmann Publishers Inc.: Morgan Kaufmann Publishers Inc. San Francisco, CA, USA), 299-304
[33] de Kleer, J.; Brown, J. S., Foundations of envisioning, (Proc. of AAAI-82. Proc. of AAAI-82, Pittsburgh, PA (1982)), 434-437
[34] Kowalski, R.; Sergot, M., A logic-based calculus of events, New Gener. Comput., 4, 67-95 (1986) · Zbl 1356.68221
[35] Kunze, L., Naïve physics and commonsense reasoning for everyday robot manipulation (2014), Technische Universität München: Technische Universität München München, Germany, Dissertation
[36] Kunze, L.; Dolha, M. E.; Beetz, M., Logic programming with simulation-based temporal projection for everyday robot object manipulation, (2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA (2011))
[37] Kunze, L.; Dolha, M. E.; Guzman, E.; Beetz, M., Simulation-based Temporal Projection of Everyday Robot Object Manipulation, (Sonenberg, L.; Stone, P.; Tumer, K.; Yolum, P., Proc. of the 10th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2011) (2011), IFAAMAS: IFAAMAS Taipei, Taiwan)
[38] Kunze, L.; Haidu, A.; Beetz, M., Acquiring task models for imitation learning through games with a purpose, (IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo Big Sight, Japan (2013))
[39] Kunze, L.; Roehm, T.; Beetz, M., Towards semantic robot description languages, (IEEE International Conference on Robotics and Automation (ICRA). IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China (2011)), 5589-5595
[40] Lakshmanan, K.; Sachdev, A.; Xie, Z.; Berenson, D.; Goldberg, K.; Abbeel, P., A constraint-aware motion planning algorithm for robotic folding of clothes, (Desai, J. P.; Dudek, G.; Khatib, O.; Kumar, V., Experimental Robotics—The 13th International Symposium on Experimental Robotics. Experimental Robotics—The 13th International Symposium on Experimental Robotics, ISER 2012, June 18-21, 2012, Québec City, Canada (2012), Springer), 547-562
[42] Lugrin, J. L.; Cavazza, M., Making sense of virtual environments: action representation, grounding and common sense, (IUI ’07: Proceedings of the 12th International Conference on Intelligent User Interfaces (2007), ACM: ACM New York, NY, USA), 225-234
[43] Majtey, A. P.; Lamberti, P. W.; Prato, D. P., Jensen-Shannon divergence as a measure of distinguishability between mixed quantum states, Phys. Rev. A, 72, 052310 (2005)
[44] McDermott, D., Robot planning, AI Mag., 13, 55-79 (1992)
[45] Mericli, T. A.; Veloso, M.; Akin, H. L., Achievable push-manipulation for complex passive mobile objects using past experience, (Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems (2013), International Foundation for Autonomous Agents and Multiagent Systems), 71-78
[46] Miller, R.; Morgenstern, L., Common sense problem page (2009)
[47] Morgenstern, L., Mid-sized axiomatizations of commonsense problems: a case study in egg cracking, Stud. Log., 67, 333-384 (2001) · Zbl 0985.68063
[48] Mösenlechner, L.; Beetz, M., Using physics- and sensor-based simulation for high-fidelity temporal projection of realistic robot behavior, (ICAPS 2009 (2009))
[49] Munro, J. I.; Papadakis, T.; Sedgewick, R., Deterministic skip lists, (SODA (1992)), 367-375 · Zbl 0829.68033
[50] Nyga, D.; Beetz, M., Everything robots always wanted to know about housework (but were afraid to ask), (2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal (2012))
[51] Oztop, E.; Kawato, M.; Arbib, M. A., Mirror neurons and imitation: a computationally guided review, Neural Netw., 19, 254-271 (2006) · Zbl 1095.68649
[52] Siskind, J. M., Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic, J. Artif. Intell. Res., 15, 31-90 (2001) · Zbl 0994.68087
[53] Smith, D.; Morgan, B., IsisWorld: an open source commonsense simulator for AI researchers, (AAAI Workshops (2010))
[54] Svensson, H.; Ziemke, T., Making sense of embodiment: simulation theories and the sharing of neural circuitry between sensorimotor and cognitive processes, Learning, 1309-1314 (1999)
[55] Tenorth, M.; Kunze, L.; Jain, D.; Beetz, M., KNOWROB-MAP—knowledge-linked semantic object maps, (10th IEEE-RAS International Conference on Humanoid Robots. 10th IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA (2010)), 430-435
[56] Tenorth, M.; Nyga, D.; Beetz, M., Understanding and executing instructions for everyday manipulation tasks from the World Wide Web, (IEEE International Conference on Robotics and Automation (ICRA). IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK, USA (2010)), 1486-1491
[57] Ueda, R.; Ogura, T.; Okada, K.; Inaba, M., Design and implementation of humanoid programming system powered by deformable objects simulation, (Proceedings of the 10th International Conference on Intelligent Autonomous Systems (2008)), 374-381
[58] Vondrak, M.; Sigal, L.; Jenkins, O. C., Physical simulation for probabilistic motion tracking, (CVPR (2008))
[59] Weigelt, M.; Kunde, W.; Prinz, W., End-state comfort in bimanual object manipulation, Exp. Psychol., 53, 143-148 (2006)
[60] Weitnauer, E.; Haschke, R.; Ritter, H., Evaluating a physics engine as an ingredient for physical reasoning, (Proceedings of the Second International Conference on Simulation, Modeling, and Programming for Autonomous Robots (2010), Springer-Verlag: Springer-Verlag Berlin, Heidelberg), 144-155
[61] (Weld, D. S.; Kleer, J.d., Readings in Qualitative Reasoning About Physical Systems (1990), Morgan Kaufmann Publishers Inc.: Morgan Kaufmann Publishers Inc. San Francisco, CA, USA)
[62] Zickler, S.; Veloso, M., Efficient physics-based planning: sampling search via non-deterministic tactics and skills, (AAMAS ’09: Proc. of the 8th Int. Conf. on Autonomous Agents and Multiagent Systems (2009))
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.