A Beginner’s Guide to Dynamic Optimization in Economics

Table of Contents

1. Frontmatter

2. 1. Basics

   Gerald Shively

   Abstract

   Chapter 1 outlines many of the basic terms and concepts fundamental to dynamic optimization. The chapter contains several examples, starting with the “cake eating” problem, which is the most basic of all dynamic optimization problems. The interconnectedness of states across stages is introduced as an essential feature of dynamic optimization. The chapter ends with a thought experiment to help the reader relate the concepts introduced to one’s daily life.

3. 2. Mr. Kuchenfresser

   Gerald Shively

   Abstract

   Chapter 2 introduces Mr. Kuchenfresser, an imaginary economic agent through whom we develop some mathematical examples of simple dynamic optimization problems. The goal is to develop intuition about how the mathematics of dynamic optimization works. This provides a foundation which allows us to expand our model in useful ways. The chapter also formally introduces the concept of an equation of motion, which will prove central to the problems encountered in subsequent chapters.

4. 3. Impatience

   Gerald Shively

   Abstract

   Chapter 3 introduces the twin concepts of impatience and discounting. The optimal path of consumption is derived for a three-period model in which future consumption is discounted.

5. 4. Mr. Kuchenfresser Meets Ms. Banker

   Gerald Shively

   Abstract

   Chapter 4 pushes the dynamic optimization example a bit further by adding production and savings alongside consumption. The aim is to generalize the model in preparation for the treatments in later chapters. A simple three-period numeric example is provided.

6. 5. Euler, Euler, Master of Us All

   Gerald Shively

   Abstract

   The main difference between static optimization and dynamic optimization is that, instead of dealing with a differential dx, which measures changes in the value of \(y = f\left( x \right)\), we must instead deal with a shift or variation of the curve \(y\left( t \right)\), which consists of many contributing pieces. For example, in solving Mr. Kuchenfresser’s various consumption problems in previous chapters, meeting the goal of maximizing his aggregate utility of consumption (a single value) required choosing an entire set of consumption values along a path. The optimization took place with respect to the entire set of consumption choices. We couldn’t choose one in isolation without understanding its impact on those who followed.

7. 6. Equations of Motion

   Gerald Shively

   Abstract

   Chapter 6 focuses on equations of motion. These are the equations in a dynamic problem that describe the evolution of the state variables. Both difference equations and differential equations are presented and discussed. Several examples are provided, including the cobweb model of market equilibrium and a simple model of a renewable natural resource.

8. 7. Inventory and Stock Adjustment Models

   Gerald Shively

   Abstract

   Chapter 7 explores dynamic inventory and stock adjustment models in greater detail. Two examples are provided—one of production and inventory control and another of labor adjustment. Uncertainty is introduced in the context of an application to vaccine inventories.

9. 8. Optimal Control

   Gerald Shively

   Abstract

   Chapter 8 presents optimal control theory and outlines the necessary conditions and steps to solving an optimal control problem. Two examples with a single control variable are solved, and several special cases, including a so-called “bang-bang” control problem, are described.

10. 9. Further Refinements in Optimal Control

    Gerald Shively

    Abstract

    Chapter 9 introduces several considerations that sometimes arise in optimal control problems. These include the specification of transversality conditions, the presence of control parameters, and the situation of blocked intervals. The chapter also details the mathematical correspondence between present-value and current value Hamiltonians.

11. 10. Dynamic Stability

    Gerald Shively

    Abstract

    Chapter 10 revisits the topic of dynamic stability, and explores in greater detail the construction and interpretation of phase diagrams. The concept of a steady state is explored mathematically and diagrammatically.

12. 11. Dynamic Programming

    Gerald Shively

    Abstract

    Chapter 11 covers dynamic programming, beginning with the simplest of such problems, so-called knapsack problems. Bellman’s principle of optimality is introduced and the mathematical details of Bellman’s equation are outlined, along with the concept of recursion. Examples provided include a least-cost travel model and an optimal stopping problem.

13. 12. A World of Uncertainty

    Gerald Shively

    Abstract

    Chapter 12 explores the role of uncertainty in dynamic optimization. The game Monopoly is used to illustrate some basic concepts, include Markov processes and Markov chains. Several examples are provided to illustrate these concepts, including product marketing and valuing a sports team.

14. 13. To Infinity … and Beyond

    Gerald Shively

    Abstract

    Chapter 13 focuses on infinite-horizon dynamic problems and the shift in perspective needed for solving them. Intuition is developed through the example of a commercial forest rotation. The chapter closes with the solution of a numeric infinite-horizon stochastic dynamic programming problem via the technique of policy iteration.