×

Online relaxation refinement for satisficing planning: on partial delete relaxation, complete hill-climbing, and novelty pruning. (English) Zbl 1522.68515

Summary: In classical AI planning, heuristic functions typically base their estimates on a relaxation of the input task. Such relaxations can be more or less precise, and many heuristic functions have a refinement procedure that can be iteratively applied until the desired degree of precision is reached. Traditionally, such refinement is performed offline to instantiate the heuristic for the search. However, a natural idea is to perform such refinement online instead, in situations where the heuristic is not sufficiently accurate. We introduce several online-refinement search algorithms, based on hill-climbing and greedy best-first search. Our hill-climbing algorithms perform a bounded lookahead, proceeding to a state with lower heuristic value than the root state of the lookahead if such a state exists, or refining the heuristic otherwise to remove such a local minimum from the search space surface. These algorithms are complete if the refinement procedure satisfies a suitable convergence property. We transfer the idea of bounded lookaheads to greedy best-first search with a lightweight lookahead after each expansion, serving both as a method to boost search progress and to detect when the heuristic is inaccurate, identifying an opportunity for online refinement. We evaluate our algorithms with the partial delete relaxation heuristic \(h^{C\mathrm{FF}}\), which can be refined by treating additional conjunctions of facts as atomic, and whose refinement operation satisfies the convergence property required for completeness. On both the IPC domains as well as on the recently published Autoscale benchmarks, our online-refinement search algorithms significantly beat state-of-the-art satisficing planners, and are competitive even with complex portfolios.

MSC:

68T20 Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.)
Full Text: DOI

References:

[1] Alc´azar, V., & Torralba, ´A. (2015). A reminder about the importance of computing and exploiting invariants in planning.In Brafman, R., Domshlak, C., Haslum, P., & Zilberstein, S. (Eds.),Proceedings of the 25th International Conference on Automated Planning and Scheduling (ICAPS’15), pp. 2-6. AAAI Press.
[2] Arfaee, S. J., Zilles, S., & Holte, R. C. (2011). Learning heuristic functions for large state spaces.Artificial Intelligence,175(16-17), 2075-2098. · Zbl 1238.68149
[3] Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995). Learning to act using real-time dynamic programming.Artificial Intelligence,72(1-2), 81-138.
[4] Bonet, B., & Geffner, H. (2001). Planning as heuristic search.Artificial Intelligence,129(1- 2), 5-33. · Zbl 0971.68146
[5] Bonet, B., & Geffner, H. (2003). Labeled RTDP: Improving the convergence of real-time dynamic programming. In Giunchiglia, E., Muscettola, N., & Nau, D. (Eds.),Proceedings of the 13th International Conference on Automated Planning and Scheduling (ICAPS’03), pp. 12-21, Trento, Italy. AAAI Press.
[6] Clarke, E., Grumberg, O., Jha, S., Lu, Y., & Veith, H. (2003). Counterexample-guided abstraction refinement for symbolic model checking.Journal of the Association for Computing Machinery,50(5), 752-794. · Zbl 1325.68145
[7] Clarke, E. M., Grumberg, O., & Long, D. E. (1994). Model checking and abstraction.ACM Transactions on Programming Languages and Systems,16(5), 1512-1542.
[8] Culberson, J. C., & Schaeffer, J. (1998). Pattern databases.Computational Intelligence, 14(3), 318-334.
[9] Domshlak, C., Hoffmann, J., & Katz, M. (2015). Red-black planning: A new systematic approach to partial delete relaxation.Artificial Intelligence,221, 73-114. · Zbl 1328.68194
[10] Domshlak, C., Karpas, E., & Markovitch, S. (2012). Online speedup learning for optimal planning.Journal of Artificial Intelligence Research,44, 709-755. · Zbl 1280.68238
[11] Edelkamp, S. (2001).Planning with pattern databases.In Cesta, A., & Borrajo, D. (Eds.),Proceedings of the 6th European Conference on Planning (ECP’01), pp. 13-24. Springer-Verlag.
[12] Eifler, R., & Fickert, M. (2018). Online refinement of Cartesian abstraction heuristics. In Bulitko, V., & Storandt, S. (Eds.),Proceedings of the 11th Annual Symposium on Combinatorial Search (SOCS’18). AAAI Press.
[13] Eifler, R., Fickert, M., Hoffmann, J., & Ruml, W. (2019). Refining abstraction heuristics during real-time planning. In Hentenryck, P. V., & Zhou, Z.-H. (Eds.),Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19). AAAI Press.
[14] Felner, A., Korf, R., & Hanan, S. (2004). Additive pattern database heuristics.Journal of Artificial Intelligence Research,22, 279-318. · Zbl 1080.68662
[15] Fickert, M. (2018). Making hill-climbing great again through online relaxation refinement and novelty pruning. In Bulitko, V., & Storandt, S. (Eds.),Proceedings of the 11th Annual Symposium on Combinatorial Search (SOCS’18), pp. 158-162. AAAI Press.
[16] Fickert, M. (2020). A novel lookahead strategy for delete relaxation heuristics in greedy best-first search. InProceedings of the 30th International Conference on Automated Planning and Scheduling (ICAPS’20), pp. 119-123. AAAI Press.
[17] Fickert, M., Gnad, D., Speicher, P., & Hoffmann, J. (2018). Saarplan: Combining Saarland’s greatest planning techniques. InIPC 2018 planner abstracts.
[18] Fickert, M., & Hoffmann, J. (2017a). Complete local search: Boosting hill-climbing through online heuristic-function refinement. InProceedings of the 27th International Conference on Automated Planning and Scheduling (ICAPS’17), pp. 107-115. AAAI Press.
[19] Fickert, M., & Hoffmann, J. (2017b). Ranking conjunctions for partial delete relaxation heuristics in planning. In Fukunaga, A., & Kishimoto, A. (Eds.),Proceedings of the 10th Annual Symposium on Combinatorial Search (SOCS’17), pp. 38-46. AAAI Press.
[20] Fickert, M., Hoffmann, J., & Steinmetz, M. (2016). Combining the delete relaxation with critical-path heuristics: A direct characterization.Journal of Artificial Intelligence Research,56(1), 269-327. · Zbl 1362.68258
[21] Fikes, R. E., & Nilsson, N. (1971). STRIPS: A new approach to the application of theorem proving to problem solving.Artificial Intelligence,2, 189-208. · Zbl 0234.68036
[22] Fink, M. (2007). Online learning of search heuristics. InProceedings of the 11th International Conference on Artificial Intelligence and Statistics (AISTATS’07), pp. 114-122.
[23] Franc‘es, G., Lipovetzky, N., Geffner, H., & Ram´ırez, M. (2018). Best-first width search in the IPC 2018: Complete, simulated, and polynomial variants. InIPC 2018 planner abstracts.
[24] Ghallab, M., Nau, D., & Traverso, P. (2004).Automated Planning: Theory and Practice. Morgan Kaufmann. · Zbl 1074.68613
[25] Gnad, D., & Hoffmann, J. (2018). Star-topology decoupled state space search.Artificial Intelligence,257, 24 - 60. · Zbl 1444.68172
[26] Haslum, P. (2006). Improving heuristics through relaxed search - an analysis of TP4 and HSP∗ain the 2004 planning competition.Journal of Artificial Intelligence Research, 25, 233-267. · Zbl 1182.68062
[27] Haslum, P. (2012).Incremental lower bounds for additive cost planning problems.In Bonet, B., McCluskey, L., Silva, J. R., & Williams, B. (Eds.),Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS’12), pp. 74-82. AAAI Press.
[28] Haslum, P., & Geffner, H. (2000). Admissible heuristics for optimal planning. In Chien, S., Kambhampati, R., & Knoblock, C. (Eds.),Proceedings of the 5th International Conference on Artificial Intelligence Planning Systems (AIPS’00), pp. 140-149, Breckenridge, CO. AAAI Press, Menlo Park.
[29] Helmert, M. (2006). The Fast Downward planning system.Journal of Artificial Intelligence Research,26, 191-246. · Zbl 1182.68245
[30] Helmert, M., Haslum, P., & Hoffmann, J. (2007). Flexible abstraction heuristics for optimal sequential planning. In Boddy, M., Fox, M., & Thiebaux, S. (Eds.),Proceedings of the 17th International Conference on Automated Planning and Scheduling (ICAPS’07), pp. 176-183, Providence, Rhode Island, USA. Morgan Kaufmann.
[31] Helmert, M., Haslum, P., Hoffmann, J., & Nissim, R. (2014). Merge & shrink abstraction: A method for generating lower bounds in factored state spaces.Journal of the Association for Computing Machinery,61(3), 16:1-16:63. · Zbl 1295.68189
[32] Helmert, M., R¨oger, G., & Karpas, E. (2011). Fast Downward Stone Soup: A baseline for building planner portfolios. InProceedings of the ICAPS Workshop on Planning and Learning (PAL’11).
[33] Hoffmann, J., & Fickert, M. (2015). Explicit conjunctions w/o compilation: Computing hFF(ΠC) in polynomial time. In Brafman, R., Domshlak, C., Haslum, P., & Zilberstein, S. (Eds.),Proceedings of the 25th International Conference on Automated Planning and Scheduling (ICAPS’15). AAAI Press.
[34] Hoffmann, J., & Nebel, B. (2001). The FF planning system: Fast plan generation through heuristic search.Journal of Artificial Intelligence Research,14, 253-302. · Zbl 0970.68044
[35] Karpas, E., Katz, M., & Markovitch, S. (2011). When optimal is just not good enough: Learning fast informative action cost-partitionings. In Bacchus, F., Domshlak, C., Edelkamp, S., & Helmert, M. (Eds.),Proceedings of the 21st International Conference on Automated Planning and Scheduling (ICAPS’11), pp. 122-129. AAAI Press.
[36] Katz, M., & Domshlak, C. (2010). Optimal admissible composition of abstraction heuristics. Artificial Intelligence,174(12-13), 767-798. · Zbl 1205.68376
[37] Katz, M., & Hoffmann, J. (2014). Mercury planner: Pushing the limits of partial delete relaxation. InIPC 2014 planner abstracts, pp. 43-47.
[38] Katz, M., Lipovetzky, N., Moshkovich, D., & Tuisov, A. (2017). Adapting novelty to classical planning as heuristic search. InProceedings of the 27th International Conference on Automated Planning and Scheduling (ICAPS’17), pp. 172-180. AAAI Press.
[39] Katz, M., Lipovetzky, N., Moshkovich, D., & Tuisov, A. (2018). MERWIN planner: Mercury enchanced with novelty heuristic. InIPC 2018 planner abstracts, pp. 53-56.
[40] Keyder, E., Hoffmann, J., & Haslum, P. (2012). Semi-relaxed plan heuristics. In Bonet, B., McCluskey, L., Silva, J. R., & Williams, B. (Eds.),Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS’12), pp. 128-136. AAAI Press.
[41] Keyder, E., Hoffmann, J., & Haslum, P. (2014). Improving delete relaxation heuristics through explicitly represented conjunctions.Journal of Artificial Intelligence Research,50, 487-533. · Zbl 1367.68265
[42] Koenig, S., & Sun, X. (2009). Comparing real-time and incremental heuristic search for real-time situated agents.Auton. Agents Multi Agent Syst.,18(3), 313-341.
[43] Korf, R. E. (1990). Real-time heuristic search.Artificial Intelligence,42(2-3), 189-211. · Zbl 0718.68082
[44] Lipovetzky, N., & Geffner, H. (2012). Width and serialization of classical planning problems. In Raedt, L. D. (Ed.),Proceedings of the 20th European Conference on Artificial Intelligence (ECAI’12), pp. 540-545, Montpellier, France. IOS Press. · Zbl 1327.68223
[45] Lipovetzky, N., & Geffner, H. (2014). Width-based algorithms for classical planning: New results. In Schaub, T. (Ed.),Proceedings of the 21st European Conference on Artificial Intelligence (ECAI’14), pp. 1059-1060, Prague, Czech Republic. IOS Press.
[46] Lipovetzky, N., & Geffner, H. (2017). Best-first width search: Exploration and exploitation in classical planning. In Singh, S., & Markovitch, S. (Eds.),Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI’17), pp. 3590-3596. AAAI Press.
[47] Nakhost, H., & M¨uller, M. (2009). Monte-carlo exploration for deterministic planning. In Boutilier, C. (Ed.),Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI’09), pp. 1766-1771, Pasadena, California, USA. Morgan Kaufmann.
[48] Richter, S., Helmert, M., & Westphal, M. (2008). Landmarks revisited. In Fox, D., & Gomes, C. (Eds.),Proceedings of the 23rd National Conference of the American Association for Artificial Intelligence (AAAI’08), pp. 975-982, Chicago, Illinois, USA. AAAI Press.
[49] Richter, S., & Westphal, M. (2010).The LAMA planner: Guiding cost-based anytime planning with landmarks.Journal of Artificial Intelligence Research,39, 127-177. · Zbl 1205.68383
[50] Rovner, A., Sievers, S., & Helmert, M. (2019). Counterexample-guided abstraction refinement for pattern selection in optimal classical planning. InProceedings of the 29th International Conference on Automated Planning and Scheduling (ICAPS’19), pp. 362-367. AAAI Press.
[51] Seipp, J. (2021). Online saturated cost partitioning for classical planning. InProceedings of the 31st International Conference on Automated Planning and Scheduling (ICAPS’21), pp. 317-321. AAAI Press.
[52] Seipp, J., & Helmert, M. (2013). Counterexample-guided Cartesian abstraction refinement. In Borrajo, D., Fratini, S., Kambhampati, S., & Oddi, A. (Eds.),Proceedings of the · Zbl 1448.68392
[53] Seipp, J., & Helmert, M. (2014). Diverse and additive Cartesian abstraction heuristics. In Chien, S., Do, M., Fern, A., & Ruml, W. (Eds.),Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS’14). AAAI Press.
[54] Seipp, J., & Helmert, M. (2018). Counterexample-guided Cartesian abstraction refinement for classical planning.Journal of Artificial Intelligence Research,62, 535-577. · Zbl 1448.68392
[55] Seipp, J., Keller, T., & Helmert, M. (2020). Saturated cost partitioning for optimal classical planning.Journal of Artificial Intelligence Research,67, 129-167. · Zbl 1442.68220
[56] Seipp, J., Pommerening, F., Sievers, S., & Helmert, M. (2017). Downward Lab.https: //doi.org/10.5281/zenodo.790461.
[57] Seipp, J., & R¨oger, G. (2018). Fast Downward Stone Soup 2018. InIPC 2018 planner abstracts.
[58] Speicher, P., Steinmetz, M., Gnad, D., Hoffmann, J., & Gerevini, A. (2017). Beyond redblack planning: Limited-memory state variables. InProceedings of the 27th International Conference on Automated Planning and Scheduling (ICAPS’17), pp. 269-273. AAAI Press.
[59] Steinmetz, M., & Hoffmann, J. (2017a). Critical-path dead-end detection vs. nogoods: Offline equivalence and online learning. InProceedings of the 27th International Conference on Automated Planning and Scheduling (ICAPS’17), pp. 283-287. AAAI Press.
[60] Steinmetz, M., & Hoffmann, J. (2017b). State space search nogood learning: Online refinement of critical-path dead-end detectors in planning.Artificial Intelligence,245, 1-37. · Zbl 1402.68165
[61] Steinmetz, M., & Hoffmann, J. (2018). LP heuristics over conjunctions: Compilation, convergence, nogood learning. In Lang, J. (Ed.),Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18), pp. 4837-4843.
[62] Thayer, J. T., Dionne, A. J., & Ruml, W. (2011). Learning inadmissible heuristics during search. In Bacchus, F., Domshlak, C., Edelkamp, S., & Helmert, M. (Eds.),Proceedings of the 21st International Conference on Automated Planning and Scheduling (ICAPS’11). AAAI Press.
[63] Torralba, ´A., Seipp, J., & Sievers, S. (2021). Automatic instance generation for classical planning. InProceedings of the 31st International Conference on Automated Planning and Scheduling (ICAPS’21), pp. 376-384. AAAI Press.
[64] Vidal, V. (2004). A lookahead strategy for heuristic search planning. In Koenig, S., Zilberstein, S., & Koehler, J. (Eds.),Proceedings of the 14th International Conference on Automated Planning and Scheduling (ICAPS’04), pp. 150-160, Whistler, Canada. AAAI Press.
[65] Vidal, V. (2011). YAHSP2: Keep it simple, stupid. InIPC 2011 planner abstracts, pp. 83-90.
[66] Wilt, C. M., & Ruml, W. (2013). Robust bidirectional search via heuristic improvement. In desJardins, M., & Littman, M. (Eds.),Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI’13), Bellevue, WA, USA. AAAI Press. · Zbl 1401.68296
[67] Wilt, C. M., & Ruml, W. (2016).Effective heuristics for suboptimal best-first search. Journal of Artificial Intelligence Research,57, 273-306. · Zbl 1401.68296
[68] Xie, F., M¨uller, M., & Holte, R. (2014). Adding local exploration to greedy best-first search in satisficing planning. In Brodley, C. E., & Stone, P. (Eds.),Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI’14), pp. 2388-2394, Austin, Texas, USA. AAAI Press
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.