Christian H. Bischof
Abstract
Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its derivatives. Derivatives play an important role in a wide variety of scientific computing applications, including optimization, solution of nonlinear equations, sensitivity analysis, and nonlinear inverse problems. We describe a simple component architecture for developing tools for automatic differentiation and other mathematically oriented semantic transformations of scientific software. This architecture consists of a compiler-based, language-specific front-end for source transformation, loosely coupled with one or more language-independent "plug-in" transformation modules. The coupling mechanism between the front-end and transformation modules is provided by the XML Abstract Interface Form (XAIF). XAIF provides an abstract, language-independent representation of language constructs common in imperative languages, such as C and Fortran. We describe the use of this architecture in constructing tools for automatic differentiation of Fortran 77 and ANSI C, and we discuss how access to compiler optimization techniques can enable more efficient derivative augmentation.
- 1 ABATE, J., BISCHOF, C., CARLE, A., AND ROH, L. Algorithms and design for a second-order automatic differentiation module. In Proc. Int. Symposium on Symbolic and Algebraic Computing (ISSAC) '97 (New York, 1997), Association of Computing Machinery, pp. 149-155. Google Scholar
Digital Library
- 2 AIF Developer's Page. www-unix.mcs.anl.gov/autodiff/AIF.Google Scholar
- 3 BARTHOLOMEW-BIGGS, M. OPFAD - a users guide to the OPtima Forward Automatic Differentiation tool. Tech. rep., Numerical Optimization Centre, University of Hertfordsshire, 1995.Google Scholar
- 4 BENDTSEN, C., AND STAUNING, O. FADBAD, A Flexible C++ Package for Automatic Differentiation, Using the Forward and Backward Methods. Tech. Rep. IMM-REP-1996-17, Department of Mathematical Modelling, Technical University of Denmark, Aug. 1996.Google Scholar
- 5 BERZ, M. High-order computation and normal form analysis of repetitive systems. In Physics of Particle Accelerators, M. Month, Ed., vol. AIP 249. American Institute of Physics, Woodbury, NY, 1991, p. 456.Google Scholar
- 6 BERZ, M., BISCHOF, C., CORLISS, G., AND GRIEWANK, A. Computational Differentiation: Techniques, Applications, and Tools. SIAM, Philadelphia, 1996.Google Scholar
- 7 BISCHOF, C. Automatic differentiation and numerical software design. In The Quality of Numerical Software: Assessment and Enhancement (London, 1997), R. Boisvert, Ed., Chapman & Hall, pp. 287-299. Google ScholarDigital Library
- 8 BISCHOF, C., CARLE, A., CORLISS, G., GRIEWANK, A., AND HOVLAND, P. ADIFOR: Generating derivative codes from Fortran programs. Scientific Programming 1, 1 (1992), 11-29.Google ScholarDigital Library
- 9 BISCHOF, C., CARLE, A., KHADEMI, P., AND MAUER, A. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computational Science & Engineering 3, 3 (1996), 18-32. Google ScholarDigital Library
- 10 BISCHOF, C., CARLE, A., KHADEMI, P., MAUER, A., AND HOVLAND, P. ADIFOR 2.0 user's guide (Revision C). Tech. Rep. ANL/MCS-TM-192, Mathematics and Computer Science Division, Argonne National Laboratory, 1995.Google Scholar
- 11 BISCHOF, C., CORLISS, G., AND GRIEWANK, A. ADIFOR exception handling. Tech. Rep. ANL/MCS-TM-159, Mathematics and Computer Science Division, Argonne National Laboratory, 1991.Google Scholar
- 12 BISCHOF, C., CORLISS, G., AND GRIEWANK, A. Structured second- and higher-order derivatives through univariate Taylor series. Optimization Methods and Software 2 (1993), 211-232.Google ScholarCross Ref
- 13 BISCHOF, C., AND ROH, L. The automatic differentiation intermediate form (AIF), 1996. Unpublished Information.Google Scholar
- 14 BISCHOF, C., ROH, L., AND MAUER, A. ADIC - An extensible automatic differentiation tool for ANSI-C. Software-Practice and Experience 27, 12 (1997), 1427-1456. Google ScholarDigital Library
- 15 BISCHOF, C. H., AND HAGHIGHAT, M. R. On hierarchical differentiation. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 83-94.Google Scholar
- 16 BROWN, S. OPRAD - a users guide to the OPtima Reverse Automatic Differentiation tool. Tech. rep., Numerical Optimization Centre, University of Hertfordsshire, 1995.Google Scholar
- 17 GIERING, R. Tangent linear and adjoint model compiler, users manual. Unpublished Information, Max-Planck Institut f~r Meteorologie, Hamburg, Germany, 1996.Google Scholar
- 18 GILL, P. E., MURRAY, W., AND WRIGHT, M. H. Practical Optimization. Academic Press, London, 1981.Google Scholar
- 19 GOLDMAN, V. V., AND CATS, G. Automatic adjoint modeling within a program generation framework: A case study for a weather forecasting grid-point model. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 185-194.Google Scholar
- 20 GRIEWANK, A. On automatic differentiation. In Mathematical Programming: Recent Developments and Applications (Amsterdam, 1989), Kluwer Academic Publishers, pp. 83-108.Google Scholar
- 21 GRIEWANK, A. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM, Philadelphia, 2000. Google ScholarDigital Library
- 22 GRIEWANK, A., JUEDES, D., AND UTKE, J. ADOL-C, a package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical Software 22, 2 (1996), 131-167. Google ScholarDigital Library
- 23 GRIEWANK, A., AND REESE, S. On the calculation of Jacobian matrices by the Markowitz rule. In Automatic Differentiation of Algorithms: Theory, Implementation, and Application, A. Griewank and G. F. Corliss, Eds. SIAM, Philadelphia, 1991, pp. 126-135.Google Scholar
- 24 MONAGAN, M., AND RODONI, R. R. An implementation of the forward and reverse mode of automatic differentiation in Maple. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 353-362.Google Scholar
- 25 PRESS, W. H., AND TEUKOLSKY, S. A. Numerical calculation of derivatives. Computers in Physics 5, 1 (Jan./Feb. 1991), 88-89.Google Scholar
- 26 PRYCE, J. D., AND REID, J. K. AD01 - a Fortran 90 code for automatic differentiation. Unpublished information, Rutherford Appleton Laboratory, Oxon, U.K., 1996.Google Scholar
- 27 RALL, L. B. Automatic Differentiation: Techniques and Applications, vol. 120 of Lecture Notes in Computer Science. Springer Verlag, Berlin, 1981.Google Scholar
- 28 ROH, L., HOVLAND, P. D., ABATE, J., AND BISCHOF, C. AIF component system. Unpublished Information.Google Scholar
- 29 ROSTAING, N., DALMAS, S., AND GALLIGO, A. Automatic differentiation in Odyssee. Tellus 45a, 5 (October 1993), 558-568.Google Scholar
- 30 SHIRIAEV, D., AND GRIEWANK, A. ADOL-F: Automatic differentiation of Fortran codes. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 375-384.Google Scholar
Index Terms
- Implementation of automatic differentiation tools
Recommendations
Implementation of automatic differentiation tools
PEPM '02: Proceedings of the 2002 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulationAutomatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its ...
On the implementation of automatic differentiation tools
Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its ...
A methodology for the development of discrete adjoint solvers using automatic differentiation tools
A methodology for the rapid development of adjoint solvers for computational fluid dynamics (CFD) models is presented. The approach relies on the use of automatic differentiation (AD) tools to almost completely automate the process of development of ...
Comments