×
Hascoët, Laurent

Data-flow reversal and garbage collection. (English) Zbl 07912540

ACM Trans. Math. Softw. 50, No. 1, Article No. 3, 20 p. (2023).

MSC:

65-XX Numerical analysis

Keywords:

automatic differentiation; data-flow reversal; dynamic memory; garbage collection

Software:

PyTorch; TAPENADE; ADOL-C; JAX; Tangent; ADiJaC
Cite Review PDF
Full Text: DOI

References:

[1] Baydin, A., Pearlmutter, B., Radul, A., and Siskind, J.. 2018. Automatic differentiation in machine learning: A survey. J. Mach. Learn. Res.18 (2018), 1-43. · Zbl 06982909
[2] Berz, Martin, Bischof, Christian, Corliss, George, and Griewank, Andreas (Eds.). 1996. Computational Differentiation: Techniques, Applications and Tools. SIAM, Philadelphia, PA. · Zbl 0857.00033
[3] Bischof, Christian H., Bücker, H. Martin, Hovland, Paul D., Naumann, Uwe, and Utke, Jean (Eds.). 2008. Advances in Automatic Differentiation. , Vol. 64. Springer, Berlin. DOI: · Zbl 1143.65003
[4] Bradbury, James, Frostig, Roy, Hawkins, Peter, Johnson, Matthew James, Leary, Chris, Maclaurin, Dougal, Necula, George, Paszke, Adam, VanderPlas, Jake, Wanderman-Milne, Skye, and Zhang, Qiao. 2018. JAX: Composable Transformations of Python+NumPy Programs. Retrieved from http://github.com/google/jax
[5] Bücker, M., Corliss, G., Hovland, P., Naumann, U., and Norris, B.. 2005. Automatic Differentiation: Applications, Theory, and Tools. Springer.
[6] Christianson, Bruce, Forth, Shaun A., and Griewank, Andreas (Eds.). 2018. Special Issue of Optimization Methods & Software: Advances in Algorithmic Differentiation. Taylor & Francis, vol. 33. · Zbl 1397.00042
[7] Corliss, George, Faure, Christèle, Griewank, Andreas, Hascoët, Laurent, and Naumann, Uwe (Eds.). 2002. Automatic Differentiation of Algorithms: From Simulation to Optimization. Springer, New York, NY. DOI:
[8] Fagan, M., Hascoët, L., and Utke, J.. 2006. Data representation alternatives in semantically augmented numerical models. In 6th IEEE International Workshop on Source Code Analysis and Manipulation.
[9] Forth, Shaun, Hovland, Paul, Phipps, Eric, Utke, Jean, and Walther, Andrea (Eds.). 2012. Recent Advances in Algorithmic Differentiation. , Vol. 87. Springer, Berlin. DOI: · Zbl 1247.65002
[10] Griewank, Andreas and Corliss, George F. (Eds.). 1991. Automatic Differentiation of Algorithms: Theory, Implementation, and Application. SIAM, Philadelphia, PA. · Zbl 0747.00030
[11] Griewank, A. and Walther, A.. 2008. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation (2nd ed.). Vol. 105. SIAM, Philadelphia, PA. Retrieved from http://www.ec-securehost.com/SIAM/OT105.html · Zbl 1159.65026
[12] Hascoët, L. and Morlighem, M.. 2017. Source-to-source adjoint algorithmic differentiation of an ice sheet model written in C. Optim. Meth. Softw. 33, 4-6 (2017), 829-843. DOI: · Zbl 1453.86048
[13] Hascoët, L., Naumann, U., and Pascual, V.. 2005. “To be recorded” analysis in reverse-mode automatic differentiation. Fut. Gen. Comput. Syst.21, 8 (2005), 1401-1417. DOI:
[14] Hascoët, Laurent and Utke, Jean. 2016. Programming language features, usage patterns, and the efficiency of generated adjoint code. Optim. Meth. Softw.31 (2016), 885-903. DOI: · Zbl 1365.65054
[15] Innes, M.. 2019. Don’t unroll adjoint: Differentiating SSA-form programs. Retrieved from https://fluxml.ai/Zygote.jl/latest/
[16] Naumann, U.. 2012. The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation. Vol. 24. SIAM, Philadelphia, PA. Retrieved from http://www.ec-securehost.com/SIAM/SE24.html · Zbl 1275.65015
[17] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Demaison, A., Antiga, L., and Lerer, A.. 2017. Automatic differentiation in PyTorch. In NIPS 2017 Workshop Autodiff. Retrieved from https://pytorch.org/
[18] Sagebaum, M., Albring, T., and Gauger, N. R.. 2019. High-performance derivative computations using CoDiPack. ACM Trans. Math. Softw.45, 4, Article 38 (2019), 26 pages. DOI: · Zbl 1486.65029
[19] Slusanschi, Emil I. and Dumitrel, Vlad. 2016. ADiJaC—Automatic differentiation of Java classfiles. ACM Trans. Math. Softw.43, 2, Article 9 (Sep.2016), 33 pages. DOI: · Zbl 1391.65045
[20] Merriënboer, Bart van, Moldovan, Dan, and Wiltschko, Alex B.. 2018. Tangent: Automatic differentiation using source-code transformation for dynamically typed array programming. In NeurIPS’18. Retrieved from: https://github.com/google/tangent
[21] Walther, A. and Griewank, A.. 2012. Getting started with ADOL-C. In Combinatorial Scientific Computing, Naumann, U. and Schenk, O. (Eds.). Chapman-Hall CRC Computational Science, 181-202.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.