Recipes for adjoint code construction

ABSTRACT

Adjoint models are increasingly being developed for use in meteorology and oceanography. Typical applications are data assimilation, model tuning, sensitivity analysis, and determination of singular vectors. The adjoint model computes the gradient of a cost function with respect to control variables. Generation of adjoint code may be seen as the special case of differentiation of algorithms in reverse mode, where the dependent function is a scalar. The described method for adjoint code generation is based on a few basic principles, which permits the establishment of simple construction rules for adjoint statements and complete adjoint subprograms. These rules are presented and illustrated with some examples. Conflicts that occur due to loops and redefinition of variables are also discussed. Direct coding of the adjoint of a more sophisticated model is extremely time consuming and subject to errors. Hence, automatic generation of adjoint code represents a distinct advantage. An implementation of the method, described in this article, is the tangent linear and adjoint model compiler (TAMC).

REFERENCES

• 1: Utpal K. Banerjee, Dependence Analysis for Supercomputing, Kluwer Academic Publishers, Norwell, MA, 1988
• 2: BAUR, W. AND STRASSEN, V. 1983. The complexity of partial derivatives. Theoretical Computer Science 22, 317-330.
• 3: BENNET, W. 1989. The Kalman smoother for a linear quasi-geostrophic model of ocean circulation. Dynamics of Atmospheres and Oceans 13, 219-267.
• 4: Christian Bischof , Peyvand Khademi , Andrew Mauer , Alan Carle, Adifor 2.0: Automatic Differentiation of Fortran 77 Programs, IEEE Computational Science & Engineering, v.3 n.3, p.18-32, September 1996 [doi>10.1109/99.537089]
• 5: CACUCCI, D. 1981. Sensitivity theory for nonlinear systems. Part I: Nonlinear functional analysis approach. Journal of Mathematical Physics 22, 2794-2812.
• 6: CHRISTIANSON, B. 1993. Reverse accumulation and attractive fixed points. Internal report, School of Information Sciences, University of Hertfordshire, England.
• 7: COURTIER, P. AND TALAGRAND, O. 1987. Variational assimilation of meteorological observations with the adjoint vorticity equation Part II: Numerical results. Quarterly Journal of the Royal Meteorological Society 113, 1329-1347.
• 8: GHIL, M. 1989. Meteorological data assimilation for oceanographers. Part I: Description and theoretical framework. Dynamics of Atmospheres and Oceans 13, 171-218.
• 9: GIERING, R. 1997. Tangent linear and Adjoint Model Compiler, Users manual. Max-Planck- Institut ffir Meteorologie.
• 10: GIERING, R. AND MAIER-REIMER, E. 1997. Data assimilation into the Hamburg LSG OGCM with its adjoint model, in prep.
• 11: GILBERT, J. 1992. Automatic Differentiation and Iterative Processes. Optimization Methods and Software 1, 13-21.
• 12: GRIEWANK, A. 1989. On automatic differentiation. In M. Iri and K. Tanabe Eds., Mathematical Programming: Recent Developments and Applications. Kluwer Akedemic Publishers, Dordrecht, 83-108.
• 13: GRIEWANK, A. 1992. Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation. Optimization Methods and Software 1, 35-54.
• 14: KAMINSKI, T., GIERING, R., AND HEIMANN, M. 1996. Sensitivity of the seasonal cycle of CO2 at remote monitoring stations with respect to seasonal surface exchange fluxes determined with the adjoint of an atmospheric transport model. Physics and Chemistry of the Earth 21, 5-6, 457-462.
• 15: KEARFOTT, R. 1996. Automatic Differentiation of Conditional Branches in an Operator Overloading Context. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank Eds., pp. 75-81. Philadelphia, Penn.: SIAM.
• 16: Steven S. Muchnick , Neil D. Jones, Program Flow Analysis: Theory and Application, Prentice Hall Professional Technical Reference, 1981
• 17: LONG, R. AND THACKER, W. 1989a. Data assimilation into a numerical equatorial ocean model. Part I: The model and assimilation algorithm. Dynamics of Atmospheres and Oceans 13, 379-412.
• 18: LONG, R. AND THACKER, W. 1989b. Data assimilation into a numerical equatorial ocean model. Part II: Assimilation experiments. Dynamics of Atmospheres and Oceans 13, 413- 440.
• 19: Louis, J.-F. 1991. Use of the adjoint to optimize model parameters. In Supplement to vol. 9, EGS 16th General Assembly (Wiesbaden, Germany, 1991), pp. C107.
• 20: MAROTZKE, J., ZHANG, Q., GIERING, R., STAMMER, D., HILL, C., AND LEE, T. 1998. The Linearization and Adjoint of the MIT Ocean General Circulation Model. in prep.
• 21: NAG. 1987. Fortran Library Manual--Mark 13. Numerical Algorithms Group.
• 22: OLDENBORGH, G., BURGERS, G., VENZKE, S., ECKERT, C., AND GIERING, R. 1997. Tracking down the delayed ENSO oscillator with an adjoint OGCM. Technical Report 97-23, Royal Netherlands Meteorological Institute, P.O. Box 201, 3730 AE De Bilt, The Netherlands. submitted to Monthly Weather Review.
• 23: ROSTAING, N., DALMAS, S., AND GALLIGO, A. 1993. Automatic differentiation in Odyss~e. Tellus 45A, 558-568.
• 24: SCHR(~TER, J. 1989. Driving of non-linear time dependent ocean models by observations of transient tracer--A problem of constrained optimization. In D. Anderson and J. Willebrand Eds., Ocean Circulation Models: Combining Data and Dynamics, pp. 257-285. Kluwer Academic Publishers.
• 25: SCHR(~TER, J. 1992. Variational assimilation of GEOSAT data into an eddy-resolving model of the gulf-stream extension area. Journal of Physical Oceanography 23, 925-953.
• 26: STAMMER, D., WUNSCH, C., GIERING, R., ZHANG, Q., MAROTZKE, J., MARSHALL, J., AND HILL, C. 1997. The Global Ocean Circulation estimated from TOPEX/POSEIDON Altimetry and a General Circulation Model. Technical Report 49, Center for Global Change Science, Massachusetts Institute of Technology.
• 27: TALAGRAND, O. 1991. The use of adjoint equations in numerical modeling of the atmospheric circulation. In Automatic Differentiation of Algorithms: Theory, Implementation and Application, A. Griewank and G. Corliess Eds., pp. 169-180. Philadelphia, Penn: SIAM.
• 28: TALAGRAND, O. AND COURTIER, P. 1987. Variational assimilation of meteorological observations with the adjoint vorticity equation Part I: Theory. Quarterly Journal of the Royal Meteorological Society 113, 1311-1328.
• 29: THACKER, W. 1987. Three lectures on fitting numerical models to observations. Technical report, GKSS Forschungszentrum Geesthacht GmbH, Geesthacht, Federal Republic of Germany.
• 30: THACKER, W. 1991. Automatic differentiation from an oceanographer's perspective. In Automatic Differentiation of Algorithms: Theory, Implementation and Application, A. Griewank and G. Corliess Eds., pp. 191-201. Philadelphia, Penn: SIAM.
• 31: TZIPERMAN, E. AND THACKER, W. 1989. An optimal control/adjoint equation approach to studying the ocean general circulation. Journal of Physical Oceanography 19, 1471-1485.
• 32: TZIPERMAN, E., THACKER, W., LONG, R., AND HWANG, S.-M. 1992. Ocean data analysis using a general circulation model, I, simulations. Journal of Physical Oceanography 22, 1434-1457.
• 33: TZIPERMAN, E., THACKER, W., LONG, R., HWANG, S.-M., AND RINTOUL, S. 1992. Ocean data analysis using a general circulation model, II, North Atlantic model. Journal of Physical Oceanography 22, 1458-1485.
• 34: WEBSTER, S. AND HOPKINS, B. 1994. Adjoints and singular vectors in a barotropic model. In Workshop on Adjoint Applications in Dynamic Meterology (1994).
• 35: Xu, Q. 1996a. Generalized adjoint for physical processes with parameterized discontinuities: Part i: Basic issues and heuristic examples. J. Atmospheric Sciences 53, 8, 1123-1142.
• 36: Xu, Q. 1996b. Generalized adjoint for physical processes with parameterized discontinuities: Part ii: Vector formulations and matching conditions. J. Atmospheric Sciences 53, 8, 1143-1155.

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Preprocessors

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.4 Quadrature and Numerical Differentiation
          Subjects: Automatic differentiation
      G.1.6 Optimization
          Subjects: Gradient methods

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.2 Automatic Programming
          Subjects: Program transformation

General Terms:
Algorithms, Theory

Keywords:
adjoint model, adjoint operator, automatic differentiation, computational differentiation, data assimilation, differentiation of algorithms, implicit functions, inverse modeling, optimization, reverse mode