Representing and Decomposing Marginal Valuations

12. Representing and Decomposing Marginal Valuations#

Authors: Lars Peter Hansen (University of Chicago) and Thomas J. Sargent (NYU)

\(\newcommand{\eqdef}{\stackrel{\text{def}}{=}}\)

Painting by Pissarro, The Factory at Pontoise, 1873

“It thus becomes important to inquire in what conditions the values of the social net product and the private net product of any given increment … of investment … are liable to diverge from one another….” —Pigou (1920), The Economics of Welfare, Chapter IX

“If man is not to do more harm than good in his efforts to improve the social order, he will have to learn that in this, as in all other fields where essential complexity of an organized kind prevails, he cannot acquire the full knowledge which would make mastery of the events possible.” —Hayek (1974), The Pretense of Knowledge

12.1. Introduction#

Partial derivatives of value functions appear in first-order conditions for controls in Markov decision problems. They measure marginal valuations. Such marginal valuations indicate directions for improving values and also losses that arise from making choices that are suboptimal from the perspective of a baseline controlled Markov process used to pose a Markov decision problem. Consequently, these marginal valuations also play an important role in max–min formulations of robust control. They are relevant for both individual decision problems and evaluations of social policies involving taxation and regulations affecting both physical and social environments. Within macroeconomics, robust-control theories have been used by [Hansen et al., 1999] to assess impacts of uncertainty on investment and equilibrium prices and quantities, and by [Alvarez and Jermann, 2004] to evaluate the welfare consequences of uncertainty, extending [Lucas, 1987]. In public economics, [Dasgupta and Mäler, 2001] apply marginal valuation to assess environmental capital and to ascertain directions of potential policy improvements. In environmental economics, there is an extensive literature measuring the social cost of carbon with different approaches. See, for instance, [Cai et al., 2017], [Nordhaus, 2017], [Rennert et al., 2022], and [Barnett et al., 2020][1].

This chapter builds on the stochastic responses of Chapter 5. Chapter 9 put those responses to a different use: a shock elasticity differentiates with respect to an exposure to a shock, a marginal valuation with respect to a state.
By embracing an asset-pricing perspective, we frame marginal valuations in terms of state-dependent discounting of stochastic payoff flows. We decompose a partial derivative into the forces that produce it. We design this approach to expand our understanding of how uncertainty concerns impact valuation. The resulting representations allow us to quantify economically interpretable stochastic flows that contribute to marginal valuations including ones that are equilibrium outcomes of dynamic stochastic models. Since dynamic stochastic equilibrium models typically involve many interacting components, decomposing their sources of influence helps “open black boxes” and provides plausible explanations for model outcomes. The decomposition isolates where uncertainty in modeling and measurement has its biggest impact.

12.2. Discrete time#

Since these responses are central inputs to our analysis, we start by reviewing our stochastic-response construction from Chapter 5. Consider a Markov process:

(12.1)#\[\begin{split}X_{t+1} = \psi(X_t, W_{t+1}) \\ Y_{t+1} - Y_t = \kappa(X_t, W_{t+1}),\end{split}\]

where \(X\) is \(n\)-dimensional, \(W\) is \(k\)-dimensional, and \(Y\) is a scalar. The \(Y\) process captures stochastic growth along a balanced path in the underlying dynamical system. We also study the associated variational processes

(12.2)#\[\begin{split}\Lambda_{t+1} = \frac {\partial \psi}{\partial x^{\top}} (X_t, W_{t+1}) \Lambda_t \\ \Theta_{t+1} - \Theta_t = \frac {\partial \kappa}{\partial x^{\top}}(X_t, W_{t+1}) \Lambda_t \end{split}\]

that capture marginal responses to small changes in initial states. As in Chapter 5, we refer to these as stochastic responses.

We use stochastic responses to provide an “asset-pricing” representation of partial derivatives of a value function with respect to a component of \(X_0\). Consistent with Chapter 11, consider a separable value function of the form \(V(X_t) + Y_t\) that satisfies:

(12.3)#\[\begin{split}\begin{align} & V(X_t) + Y_t = \\ & \hspace{1cm} \frac 1 {1 - \rho} \log \left( [1-\exp(-\delta)] \exp \left[\phi(X_t) + Y_t \right]^{1-\rho} + \exp(-\delta) \exp\left[ R(X_t) + Y_t \right]^{1 - \rho} \right)\\ & R(X_t) + Y_t = \frac 1 {1 - \gamma} \log {\mathbb E} \left( \exp \left[(1-\gamma)\left[ V(X_{t+1}) + Y_{t+1} \right] \right] \mid X_t, Y_t \right) \end{align}\end{split}\]

where the logarithm of consumption is given by:

(12.4)#\[\log C_t = \phi(X_t) + Y_t\]

and \(R(X_t) + Y_t\) is the uncertainty-adjusted continuation value. The parameters \(\delta > 0\), \(\rho > 0\), and \(\gamma \ge 1\) govern the subjective discount rate, the reciprocal of the intertemporal elasticity of substitution, and uncertainty aversion, respectively. The additive \(Y_t\) term appears because we assume the dynamical system evolves along a balanced-growth path; the state vector \(X\) is scaled to induce (asymptotically) stationary dynamics. More generally, \(\phi(X_t) + Y_t\) is the logarithm of the current-period contribution to preferences.[2]

Notice that \(Y_t\) cancels from this recursion, so we may write:

(12.5)#\[\begin{split}\begin{align} V(X_t) = & \frac 1 {1 - \rho} \log \left( [1-\exp(-\delta)] \exp\left[ (1 - \rho) \phi(X_t) \right] + \exp(-\delta) \exp\left[ (1 - \rho) R(X_t) \right] \right) \\ R(X_t) = & \frac 1 {1 - \gamma} \log {\mathbb E} \left( \exp\left[ (1 - \gamma) \left[ V(X_{t+1}) + Y_{t+1} - Y_{t} \right] \right] \mid X_t\right). \end{align}\end{split}\]

The state dynamics and consequent value function can reflect an arbitrary collection of decision rules, not necessarily socially optimal ones. To perform a local policy analysis, we compute marginal valuations for this value function.

Differentiate both sides of (12.3) with respect to \(X_t\) and \(Y_t\), and form dot products with the corresponding variational processes:

(12.6)#\[\begin{split}\begin{align} \frac{\partial V}{\partial x}(X_t) \cdot \Lambda_t = & \left[ \frac {[1-\exp(-\delta)] \exp\left[ (1 - \rho) \phi(X_t)\right]}{ \exp\left[ (1 - \rho) V(X_t) \right] } \right] \frac {\partial \phi}{\partial x} (X_t) \cdot \Lambda_t \cr & + \left[\frac {\exp(-\delta)\exp\left[ (1 - \rho) R(X_t) \right]}{ \exp\left[ (1 - \rho) V(X_t) \right]} \right]\frac {\partial R}{\partial x} (X_t) \cdot \Lambda_t \cr \frac {\partial R}{\partial x} (X_t) \cdot \Lambda_t = & {\mathbb E}\left( N_{t+1} \left[\frac{\partial V}{\partial x}(X_{t+1}) \cdot \Lambda_{t+1} + \Theta_{t+1} - \Theta_t \right] \mid X_t, \Lambda_t \right) \\ N_{t+1} = & \frac {\exp\left[ (1 - \gamma) \left[ V(X_{t+1}) + Y_{t+1} - Y_{t} \right] \right]}{ {\mathbb E} \left( \exp\left[ (1 - \gamma) \left[ V(X_{t+1}) + Y_{t+1} - Y_{t} \right] \right] \mid X_t \right)}. \end{align}\end{split}\]

Substituting from (12.1) and (12.2) yields

\[\begin{split}\begin{align} \frac {\partial R}{\partial x} (X_t) \cdot \Lambda_t = & {\mathbb E}\left( N_{t+1} \left[\frac{\partial V}{\partial x}(X_{t+1}) \cdot \Lambda_{t+1} + \frac{\partial \kappa}{\partial x^{\top}}(X_t, W_{t+1}) \Lambda_{t} \right] \mid X_t , \Lambda_t \right) \\ N_{t+1} = & \frac {\exp\left[ (1 - \gamma) \left[ V(X_{t+1}) + \kappa(X_t, W_{t+1} )\right] \right]}{ {\mathbb E} \left( \exp\left[ (1 - \gamma) \left[ V(X_{t+1}) + \kappa(X_t, W_{t+1} )\right] \right] \mid X_t \right)}. \end{align}\end{split}\]

As argued in Chapter 11, \(N_{t+1}\) induces an uncertainty-adjusted change of probability measure, with an implied expectation operator that we denote by \({\widetilde {\mathbb E}}\). We motivated this construction by robustness considerations. Although the approach applies for general \(\rho > 0\), to simplify the formulas we set \(\rho = 1\) (unitary intertemporal elasticity of substitution). The resulting equation becomes:

\[\begin{split}\begin{align} \frac{\partial V}{\partial x}(X_t) \cdot \Lambda_t = & [1 - \exp(-\delta)] \frac {\partial \phi}{\partial x} (X_t) \cdot \Lambda_t + \exp(-\delta) {\widetilde {\mathbb E}} \left[\frac{\partial \kappa}{\partial x^{\top}}(X_t, W_{t+1}) \Lambda_{t} \mid X_t, \Lambda_t \right] \\ & + \exp(-\delta) {\widetilde {\mathbb E}}\left[ \frac{\partial V}{\partial x}(X_{t+1}) \cdot \Lambda_{t+1} \mid X_t, \Lambda_t \right]. \end{align}\end{split}\]

Solving forward from date zero gives

(12.7)#\[\begin{align} \frac{\partial V}{\partial x}(X_0) \cdot \Lambda_0 = & [1 - \exp(-\delta)] \sum_{t=0}^\infty \exp(-\delta t) {\widetilde {\mathbb E}} \left[\frac {\partial \phi}{\partial x} (X_{t}) \cdot \Lambda_{t} \mid X_0, \Lambda_0 \right] \cr & + \exp(-\delta) \sum_{t=0}^\infty \exp(-\delta t) {\widetilde {\mathbb E}} \left[\frac {\partial \kappa}{\partial x^{\top}} (X_{t}, W_{t+1}) \Lambda_{t} \mid X_0, \Lambda_0 \right]. \end{align}\]

A summation-by-parts argument together with the evolution equation for \(\Theta\) (second equation in (12.2)) implies

(12.8)#\[\begin{split} & \frac{\partial V}{\partial x}(X_0) \cdot \Lambda_0 + \Theta_0 \cr & = [1 - \exp(-\delta)] \sum_{t=0}^\infty \exp(-\delta t) {\widetilde {\mathbb E}} \left( \left[\frac {\partial \phi}{\partial x} (X_{t})\cdot \Lambda_t + \Theta_t \right] \mid X_0, \Lambda_0, \Theta_0 \right) . \end{split}\]

Formula (12.8) gives our basic discrete-time asset-pricing representation. The stochastic-response process \(\Lambda\) appears in it. The representation applies to marginal changes in any of the \(n\) state variables. It differs from the direct representation obtained by solving a vector co-state system forward, a comparison we make later in this chapter. Also, it is expressed in terms of an uncertainty-adjusted probability measure reflected in our use of the \({\widetilde \cdot}\) notation.

Representation (12.8) supports two decompositions, each with antecedents in innovation-accounting methods used in empirical macroeconomics. The first decomposes marginal valuations by payoff horizon:

\[[1 - \exp(-\delta)] \exp(-\delta t) {\widetilde {\mathbb E}}\left[ \frac {\partial \phi}{\partial x} (X_{t})\cdot \Lambda_{t} + \Theta_t \mid X_0, \Lambda_0, \Theta_0 \right].\]

The second decomposition exploits the additive structure of the impacted states as given by the dot product in the stochastic flow:

\[[1 - \exp(-\delta)] \exp(- \delta t) \frac {\partial \phi}{\partial x} (X_{t}) \cdot \Lambda_t = [1 - \exp(-\delta)] \exp(- \delta t) \sum_{j=1}^n \frac {\partial \phi}{\partial x_j} (X_{t}) \Lambda_{j,t}. \]

This decomposition measures the importance of state interactions for valuation, since a marginal change in one initial state can affect other states at future dates.

We next explore formulations and applications in continuous time with both Brownian and jump risk.

12.3. Continuous time#

A continuous-time formulation lets us distinguish small shocks (Brownian increments) from large shocks (Poisson jumps). We begin with a specification driven by Brownian motion, i.e., diffusion dynamics. Jumps are treated as terminal conditions for which we impose continuation values conditioned on a jump occurring; the possibility of a jump contributes to the value function. After developing this approach, we extend it to include valuations that reflect concerns about model misspecification, i.e., “robust valuations.” [Hansen and Souganidis, 2025] establishes the Feynman-Kac representation that this section uses.

12.3.1. Diffusion dynamics#

We start with a Markov diffusion that governs state dynamics

\[\begin{split}\begin{align*} dX_t & = \mu(X_t) dt + \sigma(X_t) dW_t \\ dY_t & = \nu(X_t) dt + \varsigma(X_t) dW_t. \end{align*}\end{split}\]

that need not be the outcome of an optimization problem. We treat \(\varsigma(x)\) as a \(k\)-dimensional vector, conformable with \(\sigma(x)^{\top}V_x(x)\), and write \(\varsigma(x) \cdot h\) for its inner product with a drift distortion.

Using the variational-process construction of Section Constructing responses of Chapter 5, recall that

\[d\Lambda_{t}^i = \left(\Lambda_t\right)^\top\frac{\partial \mu_i}{\partial x}(X_t) dt + \left({\Lambda_t}\right)^\top\frac{\partial \sigma_i}{\partial x}(X_t) dW_t , \]

where we follow [Kunita, 1990] (Chapter 4) by viewing these as pathwise derivatives rationalized in the context of stochastic flows. With the appropriate stacking, the drift for the composite process \((X,\Lambda)\) is:

(12.9)#\[\begin{split}\mu^a(x,\lambda) \eqdef \begin{bmatrix} \mu(x) \\ \lambda^\top{\frac {\partial \mu_1} {\partial x} }(x) \\ ... \\ \lambda^\top{\frac {\partial \mu_n} {\partial x} }(x) \end{bmatrix},\end{split}\]

and the composite matrix coefficient on \(dW_t\) is given by

(12.10)#\[\begin{split}\sigma^a(x,\lambda) \eqdef \begin{bmatrix} \sigma(x) \\ \lambda^\top\frac {\partial \sigma_1 }{\partial x}(x)\\ ... \\ \lambda^\top \frac {\partial \sigma_n }{\partial x}(x) \end{bmatrix}.\end{split}\]

Similarly, \(\Theta\) is the scalar variational process associated with \(Y\), with evolution

\[d \Theta_t = \left( \Lambda_t \right)^\top \frac {\partial \nu}{\partial x} (X_t)dt + \left({\Lambda_t}\right)^\top \frac {\partial \varsigma}{\partial x} (X_t)dW_t \]

12.3.2. An initial representation of a partial derivative#

Consider the evaluation of discounted utility whose instantaneous contribution is the logarithm of consumption, \(\phi(x) + y\), where \(x\) is a realization of the state vector \(X_t\). For the moment, we abstract from robustness considerations. The function \(V\) satisfies a Feynman-Kac (FK) equation:

(12.11)#\[\begin{align} 0 = & \delta \left[\phi(x) + y\right] - \delta \left[V(x) + y \right] + \mu(x) \cdot \frac {\partial V}{\partial x}(x) + \nu(x) \cr &+ {\frac 1 2 }{\rm trace}\left[\sigma(x)^{\top} \frac {\partial^2 V}{\partial x \partial x^{\top}}(x) \sigma(x) \right]. \end{align}\]

The variable \(y\) could be dropped here. We keep it because the derivation is easier with it. As in the discrete-time example, we want to represent

\[ {\frac {\partial V}{\partial x}}(x)\cdot \lambda + \theta \]

where \((\lambda, \theta)\) is a potential realization of \((\Lambda_t, \Theta_t)\) for any \(t \ge 0\). We could set \(\Lambda_0\) to a coordinate vector and \(\Theta_0 = 0\) to represent the marginal valuation of the corresponding state. Later we allow more flexibility in specifying \((\Lambda_0, \Theta_0)\) for reasons we will make clear. We seek a representation as an expected discounted value of a marginal stochastic flow that we construct below.

Differentiating the Feynman-Kac equation (12.11) with respect to each of the individual states gives a vector of equations, one for each state variable. Forming the dot product of this system with \((\lambda, \theta)\) produces the scalar equation of particular interest. The resulting equation is itself a Feynman-Kac equation, but for the scalar function:

\[\lambda \cdot \frac {\partial V}{\partial x} + \theta.\]

See [Hansen and Souganidis, 2025] for details. Because the equation involves both \((\lambda, \theta)\) and \(x\), the solution is expressed using the diffusion dynamics of the joint process \((X,Y,\Lambda,\Theta)\) and takes the form of a discounted expected value:

(12.12)#\[\begin{split} & \frac {\partial V}{\partial x}(X_0) \cdot \Lambda_0 + \Theta_0 \cr & = \int_0^\infty \exp( - \delta t ) {\mathbb E} \left( \delta \left[\frac {\partial \phi }{\partial x} (X_{t})\cdot \Lambda_{t} + \Theta_t\right] \mid X_0, Y_0, \Lambda_0, \Theta_0 \right) dt . \end{split} \]

By initializing \(\Lambda_0\) to a coordinate vector with zeros in all entries except position \(i\), we obtain the partial derivative with respect to the \(i^{th}\) state as a discounted present value with discount rate \(\delta\). Here \((\Lambda_{t},\Theta_t)\) is the stochastic-response process studied in Chapter 5, giving the marginal response of the date-\(t\) state vector to a marginal change in the \(i^{th}\) component of the initial state vector. This marginal change induces a marginal reward at date \(t\):

\[\frac {\partial \phi }{\partial x} (X_{t})\cdot \Lambda_t + \Theta_t \]

which we interpret as a marginal stochastic flow — the counterpart of a utility-weighted cash flow in standard equity-pricing theory. This is the marginal valuation counterpart to an impulse response function.

Remark 12.1

There are related results in prior research investigating an objective with only a terminal contribution. For instance, see [Bismut, 1978]’s (Section 3.2) treatment of the maximum principle, although the representation he derives imposes the envelope condition implied by optimization. We deliberately avoid imposing an envelope condition to allow for suboptimal decision rules and to enhance interpretability. See also [Fournie et al., 1999]’s formula for the sensitivity of a derivative claim to an initial change in a state variable.[3]

Decomposition I

An immediate application of formula (12.12) is a decomposition of marginal valuations by payoff horizon:

\[\delta \exp(-\delta t) {\mathbb E} \left[ \frac {\partial \phi}{\partial x} (X_{t}) \cdot \Lambda_t + \Theta_t \mid X_0, Y_0, \Lambda_0, \Theta_0 \right] \]

for \(t\ge 0.\)

Decomposition II

A second decomposition based on (12.12) additively decomposes the marginal valuation of a state variable (as determined by the initialization of \(\Lambda_0\) and \(\Theta_0\)) into contributions of each of the future state variables. Write:

\[ \delta \frac {\partial \phi}{\partial x} (X_{t}) \cdot \Lambda_{t} = \delta \sum_{j=1}^n \frac {\partial \phi}{\partial x_j} (X_{t}) \Lambda_{j,t} .\]

Then

\[\begin{align} & \frac {\partial V}{\partial x}(X_0) \cdot \Lambda_0 + \Theta_0 \cr = & \delta \sum_{j=1}^n \int_0^\infty \exp( - \delta t ) {\mathbb E} \left[ \frac {\partial \phi}{\partial x_j} (X_{t}) \Lambda_{j,t} \mid X_0, \Lambda_0 \right] dt \cr & + \delta \int_0^\infty \exp( - \delta t ) {\mathbb E} \left( \Theta_t \mid X_0, Y_0, \Lambda_0, \Theta_0 \right) dt \end{align}\]

provides \(n+1\) different contributions to the marginal valuation. This decomposition reveals the importance of state variable interactions in the valuation of each state variable.

This decomposition can be used in conjunction with Decomposition I to obtain a horizon-based representation for each state variable contribution to valuation.

While setting \(\Lambda_0\) equal to a coordinate vector provides the building blocks for measuring marginal contributions to valuation, the marginal assessment of policy rules opens the door to other initializations.

Remark 12.2

Consider the local assessment of a possibly suboptimal policy rule in the initial time period. Let \({\widehat \Gamma}\) be a policy rule where

\[\begin{align*} \mu(x) & = {\hat \mu}\left[x, {\hat \Gamma}(x) \right] \cr \nu(x) & = {\hat \nu}\left[x, {\hat \Gamma}(x) \right] \cr \phi(x) & = {\hat \phi} \left[x, {\hat \Gamma}(x) \right]. \end{align*}\]

Consider a small initial change in the policy rule \({\widehat \Gamma} + \epsilon \Gamma(x)\) in the initial time period where \(\Gamma\) is a perturbation direction from the potentially suboptimal policy rule \({\widehat \Gamma}(x)\). The terms of interest are:

\[\begin{align*} & \delta \frac {\partial {\hat \phi}}{\partial \gamma^{\top}} \left[X_0, {\widehat \Gamma}(X_0)\right] \Gamma (X_0) \cr &+ \frac {\partial {\hat \nu}}{\partial \gamma^{\top}}\left[X_0, {\widehat \Gamma}(X_0)\right] \Gamma (X_{0}) + \frac {\partial V}{\partial x} (X_{0}) \cdot \left( \frac {\partial {\hat \mu}}{\partial \gamma^{\top}}\left[X_0, {\widehat \Gamma}(X_0)\right]\Gamma (X_{0}) \right), \end{align*}\]

where \(\gamma\) is placeholder notation for the decision argument. Optimality, which we do not need to impose, implies that this expression is zero for any admissible choice of \(\Gamma\). The first term is a “static” contribution and the remaining two are “dynamic” contributions. For the dynamic contributions, set

\[\begin{align*} \Lambda_0 & = \frac {\partial {\hat \mu}}{\partial \gamma^{\top}}\left[X_0, {\widehat \Gamma}(X_0)\right]\Gamma (X_{0}) \cr \Theta_0 & = \frac {\partial {\hat \nu}}{\partial \gamma^{\top}}\left[X_0, {\widehat \Gamma}(X_0)\right] \Gamma (X_{0}). \end{align*}\]

12.3.3. Relation to co-states#

We start with an example that is scalar and thus omits the important consideration of state interaction. The scalar specification is meant only to simplify the computations, as there are direct extensions to the multivariate case.

12.3.3.1. Setup#

Start with the state equation:

\[dX_t = \mu(X_t) dt + \sigma(X_t) dW_t.\]

Construct the dynamic evolution for stochastic responses:

\[d\Lambda_t = \Lambda_t \mu'(X_t) dt + \Lambda_t \sigma'(X_t) dW_t.\]

Consistent with our previous analysis, the random variable \(\Lambda_t\) gives the date \(t\) local stochastic response to a marginal change in the initial state scaled by \(\Lambda_0.\)

We again evaluate discounted expected utility, \(V,\) by solving a Feynman-Kac equation where \(\delta\) is the subjective rate of discount and \(\delta U\) is the instantaneous contribution to utility:

\[\delta U(x) - \delta V(x) + \mu(x) V'(x) + {\frac 1 2 } \sigma^2(x) V''(x) = 0 .\]

Differentiate the terms in this equation with respect to the scalar state:

(12.13)#\[\delta U'(x) - \delta V'(x) + \mu(x) V''(x) + \mu'(x) V'(x) + \sigma'(x) \sigma(x) V''(x) + {\frac 1 2} \sigma^2(x) V'''(x) = 0 .\]

Let \({\mathcal A}\) be the generator of \(X\) and apply it to \(V'\):

\[{\mathcal A} V' = \mu V'' + {\frac 1 2 } \sigma^2 V'''.\]

Rewrite (12.13) as:

(12.14)#\[{\mathcal A} V' = - \delta U' + \delta V' - \mu' V' - \sigma' \sigma V''\]

Remark 12.3

From (12.14) we may construct a generator \({\mathcal A}^d\) with a level term given by \(- \delta + \mu',\) and a drift term: \(\mu + \sigma' \sigma .\) The level changes the instantaneous discounting and the drift term modifies the distribution of the Brownian increment, \(dW_t\), by including a drift term \(\sigma'(X_t) dt.\) The solution to this equation is

\[V'(x) = \delta {\mathbb E}^d \left( \int_0^\infty \exp\left( - \int_0^t\left[ \delta - \mu'(X_u) \right] du \right) U'(X_t) dt \mid X_0 = x\right)\]

provided that the right-hand side integral is well defined including the conditional expectation \({\mathbb E}^d\left( \cdot \mid X_0=x\right)\) where \({\mathbb E}^d\) uses the proposed drift distortion.[4] We find this forward-looking solution to be less interpretable than the one we derived using stochastic responses. The direct co-state solution becomes all the more difficult to interpret in the multivariate case since a vector of equations is solved forward.

We next remind readers of the commonly used Hamiltonian formulation.

12.3.3.2. Hamiltonian#

Consider the Hamiltonian with the control law imposed:

\[{\rm Ham}(x,t, p,q) \eqdef \exp(-\delta t) \delta U(x) + p\mu(x) + q \sigma(x) .\]

Then the state evolution can be expressed as:

\[dX_t = \frac {\partial {\rm Ham}}{\partial p}(X_t,t, P_t, Q_t) dt + \frac {\partial {\rm Ham}}{\partial q} (X_t,t, P_t, Q_t) dW_t,\]

and the co-state evolution as a backward stochastic differential equation:

\[dP_t = - \frac {\partial {\rm Ham}}{\partial x}(X_t,t, P_t, Q_t) dt + Q_t dW_t\]

where

\[\frac {\partial {\rm Ham}}{\partial x}(x,t, p, q) = \exp(-\delta t) \delta U'(x) + p \mu'(x) + q \sigma'(x) .\]

Given our discounted formulation, it is convenient to transform the co-state so that \(P^d_t \eqdef \exp(\delta t) P_t\) implying that

(12.15)#\[dP_t^d = \left[ - \delta U'(X_t) + \delta P_t^d - P_t^d \mu'(X_t) - Q_t^d \sigma'(X_t)\right] dt + Q_t^d dW_t\]

where \(Q_t^d = \exp(\delta t) Q_t\). The corresponding discounted Hamiltonian is:

\[{\rm Ham}^d(x, p^d,q^d) \eqdef \delta U(x) + p^d\mu(x) + q^d \sigma(x) .\]

The well-known link to dynamic programming implies:

\[\begin{split}\begin{split} p^d & = V' \\ q^d & = \sigma V''. \end{split}\end{split}\]

With these equalities, evolution (12.15) is consistent with the generator formulation (12.14).

12.3.4. Robustness#

We next consider a general class of drift distortions that can help us study model misspecification concerns. We first explore the consequences of exogenously specified drift distortions. After that, we show how such a distortion can emerge endogenously as a decision maker’s response to concerns about model misspecifications.

For diffusions, we modify the Brownian increment: instead of \(W\) being a multivariate Brownian motion, we allow it to have a drift \(H\) under a change in the probability distribution. We index alternative probability specifications by their corresponding drift processes \(H\). Locally,

\[dW_t = H_t dt + dW^H_t\]

where \(W^H\) is a Brownian motion under the \(H\) probability. Given that both the distribution parameterized by \(H\) and the baseline distribution for the increment are normals with an identity matrix as the local covariance matrix, the local measure of relative entropy is given by the quadratic term:

\[{\frac 1 2} H_t \cdot H_t .\]

See [James, 1992], [Anderson et al., 2003], and [Hansen et al., 2006] for further discussions of this continuous-time formulation. Chapter 10 provides the decision-theoretic foundations for the entropy penalty used here, and [Cerreia-Vioglio et al., 2025] provide an axiomatic foundation for misspecification aversion.

We again suppose that any decision or policy rules are embedded in the baseline state dynamics. To make a robustness adjustment, we introduce a minimizing or adversarial decision maker who minimizes the discounted expected utility by choice of the drift distortion. Consider a value function, \(V,\) that solves:

(12.16)#\[\begin{align} 0 &= \min_h\hspace{.2cm} \delta \phi(x) - \delta V(x) + {\frac \xi 2}|h|^2 \cr & + \left[\mu(x) +\sigma(x)h \right] \cdot V_x(x) + \nu(x) + \varsigma(x) \cdot h \cr & + {\frac 1 2} {\rm trace} \left[ \sigma (x)^{\top} V_{xx}(x) \sigma(x) \right]. \end{align}\]

The minimizing \(h\) in (12.16) expressed as a function of \(x\) satisfies:

(12.17)#\[h^*(x) = - \frac 1 \xi \left[ \sigma(x)^{\top} V_x(x) + \varsigma(x) \right].\]

We use this solution to provide an alternative perspective on the implications of robustness. Define the drift distortion:

\[H_t^* \eqdef h^*\left({\overline X}_t \right).\]

We alter the stochastic dynamics for the original state vector to be:

\[\begin{align} d X_t & = \mu(X_t)dt + \sigma(X_t) h^* \left( \overline{X}_t \right) dt + \sigma(X_t) dW_t^{H^*}, \cr dY_t & = \nu(X_t)dt + \varsigma(X_t) h^* \left( \overline{X}_t \right) dt + \varsigma(X_t) dW_t^{H^*} \end{align}\]

where \(\overline{X}\) satisfies:

\[d \overline{X}_t = {\bar \mu}\left( \overline{X}_t \right) dt + {\bar \sigma} \left({\overline X}_t\right) dW_t^{H^*}\]

and

\[\begin{align*} {\bar \mu}\left( \overline{X}_t \right) \eqdef & \mu\left( {\overline X}_t \right) + {\bar \sigma} \left({\overline X}_t\right) h^*\left( {\overline X}_t\right) \cr {\bar \sigma} \left({\overline X}_t\right) \eqdef & \sigma\left( {\overline X}_t\right) \end{align*}\]

for the initialization \({\overline X}_0 = X_0\). Given this initial condition, by design \(X_t = {\overline X}_t\) for \(t \ge 0.\) We use the constructed process \(\{ {\overline X}_t : t \ge 0 \}\) solely to represent the minimizing drift distortion.

Note that the stochastic-response processes for \(X\) and \(Y\) satisfy the recursions:

(12.18)#\[\begin{align} d\Lambda_{t}^i & = \left(\Lambda_t\right)^{\top}\frac{\partial \mu_i}{\partial x}(X_t) dt + \left({\Lambda_t}\right)^{\top}\frac{\partial \sigma_i}{\partial x}(X_t) \left[ h^*\left( {\overline X}_t \right) dt + dW_t^{H^*}\right] \cr d \Theta_t & = \left(\Lambda_t \right)^{\top} \frac{\partial \nu}{\partial x}(X_t) dt + \left({\Lambda_t}\right)^{\top} \frac {\partial \varsigma}{\partial x}\left[ h^*\left( {\overline X}_t \right) dt + dW_t^{H^*}\right]. \end{align}\]

The value function of interest is \({\overline V}(x, {\bar x})\) satisfying:

(12.19)#\[\begin{align} 0 = \hspace{.2cm} & \delta \left[\phi(x) \right] - \delta \left[ {\overline V}(x,{\bar x}) \right] + {\frac \xi 2}|h^*(\bar x)|^2 \cr & + {\overline V}_x(x, {\bar x} ) \cdot \left[ \mu(x) +\sigma(x)h^*(\bar x) \right] \cr & + \nu(x) + \varsigma(x) h^*(\bar x) + {\overline V}_{\bar x}(x, {\bar x} ) \cdot {\bar \mu}({\bar x}) \cr & + {\frac 1 2} {\rm trace} \begin{bmatrix} \sigma (x)^{\top} & {\bar \sigma}({\bar x})^{\top} \end{bmatrix} \begin{bmatrix} {\overline V}_{xx^{\top}}(x,{\bar x}) & {\overline V}_{x{\bar x}^{\top}}(x,{\bar x}) \cr {\overline V}_{{\bar x} x^{\top}}(x,{\bar x}) & {\overline V}_{{\bar x} {\bar x}{^\top}}(x,{\bar x}) \end{bmatrix}\begin{bmatrix} \sigma(x) \cr {\bar \sigma}({\bar x}) \end{bmatrix}. \end{align}\]

By design:

\[{\overline V}(x,x) = V(x)\]

for \(V\) that satisfies (12.16). Note that the HJB equation, as posed, allows \(\bar x \ne x\). Importantly, differentiating \(H\) with respect to \(x\) contributes nothing because \(H\) depends only on the \(\bar{X}_t\) process. Given the value function \({\overline V}(x,{\bar x})\), suppose a minimizing decision maker solves:

\[\min_{{\bar x}} {\overline V}(x, {\bar x}).\]

Given the original minimization problem, the solution is necessarily \(\bar x = x\), implying that

\[{\overline V}_{\bar x}(x,x) = 0. \]

This solution follows because it lies among the minimizing options of the original problem. The choice \(x = {\bar x}\) implements the previously derived solution. As a consequence:

\[\begin{split}{\overline V}_x(x,x) & = V_x(x) \\ {\overline V}_{xx^{\top}}(x,x) + {\overline V}_{x{\bar x}^{\top}}(x,x) & = V_{xx^{\top}}(x). \end{split}\]

Treating \((X_t, {\overline X}_t)\) as a composite state vector and imitating our earlier argument that abstracted from robustness,

\[\begin{align*} & \frac {\partial V}{ \partial x}(X_0) \cdot \Lambda_0 + \Theta_0\cr &= \delta \int_0^\infty \exp(-\delta t) \widetilde{\mathbb{E}} \left( \frac \partial {\partial x} \phi\left(X_{t}\right) \cdot \Lambda_{t} + \Theta_t \mid X_0, \Lambda_0, \Theta_0 \right) dt . \end{align*}\]

We use the \({\widetilde {\mathbb E}}\) notation because stochastic responses are computed under the uncertainty-adjusted state evolution implied by the drift distortion \(\{ H_t^* : t \ge 0 \}\), where

\[H_t^* \eqdef h^*\left({\overline X}_t \right).\]

Armed with this change of probability measure, we may apply Decomposition I and Decomposition II.

Remark 12.4

Robust control theory goes further by exploring ramifications for the decision rule itself. The construction we described for valuation extends to the control framework as well. We explore both a recursive representation of a two-player game and a Stackelberg formulation solved from a date-zero perspective, following insights in [Fleming and Souganidis, 1989]. Consider first a recursive formulation in which we find a value function, \(V,\) that solves:

(12.20)#\[\begin{align} 0 &= \max_{\gamma \in {\mathcal \Gamma} } \min_h \hspace{.2cm} \delta\left[ U(x,\gamma) + y\right] - \delta \left[ V(x) + y \right]+ {\frac \xi 2}|h|^2 \cr & + \left[\mu(x,\gamma) +\sigma(x,\gamma)h \right] \cdot V_x(x) + \nu(x) + \varsigma(x,\gamma) \cdot h \cr & + {\frac 1 2} {\rm trace} \left[ \sigma (x,\gamma)^{\top} V_{xx}(x) \sigma(x,\gamma) \right]. \end{align}\]

This value function is constructed by solving a recursive version of the zero-sum game. One condition that [Fleming and Souganidis, 1989] impose is the Bellman-Isaacs condition, requiring that exchanging the orders of \(\min\) and \(\max\) does not alter the value function of the recursive game. In effect, [Fleming and Souganidis, 1989] show that coupled dynamic programs characterize the two-player, zero-sum game that interests us, as well as certain other zero-sum games. Following [Hansen et al., 2006], this approach yields the analogous recipe for constructing a minimizing drift-distortion process \(\{H_t : t \ge 0\}\) used for robust valuation.

Remark 12.5

While we demonstrated that we can treat a drift distortion as exogenous to the original state dynamics, for some applications we will want to view it as a change in the endogenous dynamics that are reflected in (12.17).

12.3.5. Allowing the IES to differ from unity#

For notational and conceptual simplicity, we have focused on the special case in which \(\rho = 1,\) where \(\rho\) is the reciprocal of the intertemporal elasticity of substitution (IES). We now briefly sketch an extension that allows \(\rho\) to differ from unity in a recursive utility specification. Consider the utility recursion:

\[\begin{split}& \left(\frac{\delta}{1-\rho}\right)\left(\exp\left[(1-\rho)\left[\phi(X_t)+Y_t - {V}(X_t)- Y_t\right]\right]-1\right) \\ & + \mu_{v,t} = 0. \end{split}\]

where \({\mu}_{v,t}\) is the local mean of \({V}(X) + Y.\) With the robust adjustment discussed in the previous subsection, compute:

\[\begin{align} & \frac{\partial}{\partial x} \left(\frac{\delta}{1-\rho}\right) \left( \exp\left[(1-\rho)\left[\phi(x) - V(x)\right]\right]-1 \right) \cr &= \delta \exp\left[(1-\rho)\left[\phi(x) - {V}(x)\right]\right] \left[\frac {\partial \phi} {\partial x} (x) -\frac{\partial{V}}{\partial x}(x)\right]. \end{align} \]

Using this calculation, we modify the previous formulas by replacing the subjective discount factor \(\exp(-\delta t)\) with

\[{\rm Dis}_t \eqdef \exp\left(-\int_0^t \delta \exp\left[(1-\rho)\left[\phi\left(X_\tau\right) -V(X_\tau)\right]\right] d\tau \right).\]

The instantaneous discount rate is now state-dependent, reflecting both how current utility compares to the continuation value and whether \(\rho\) is greater or less than one. When current utility exceeds the continuation value, the discount rate is scaled down if \(\rho > 1\) and scaled up if \(\rho < 1\).

We replace the instantaneous contribution to the flow term, \(\delta \frac {\partial \phi} {\partial x} (X_t),\) with:

\[ \delta\exp\left[(1-\rho)\left[\phi(X_t)- V(X_t)\right]\right]\frac {\partial \phi}{\partial x} (X_t)\]

Combining these contributions gives:

\[\begin{split}& \frac {\partial V}{\partial x}\left( X_0 \right) \cdot \Lambda_0 + \Theta_0 = \\ & \delta \widetilde{\mathbb E}\left( \int_0^\infty {\rm Dis}_t \exp\left[(1-\rho)\left[\phi\left( X_t \right)- V(X_t)\right]\right] \right. \cr & \hspace{1cm} \left. \times \left(\frac {\partial \phi} {\partial x} (X_t) \cdot \Lambda_t + \Theta_t \right) dt \mid X_0, \Lambda_0, \Theta_0 \right) .\end{split}\]

12.3.6. Jumps#

We now incorporate Poisson jumps into the analysis as a way to capture big events. We are particularly interested in cases in which jump intensities depend on endogenous state variables. To leverage and extend our previous analysis, we adopt a pre-jump perspective, viewing the first jump as a terminal value with an endogenous payoff formally modeled as a continuation value conditioned on a jump occurring. From an ex ante perspective, any of \(L\) possible jumps could occur first, each with an intensity denoted \(\mathcal{J}^\ell(x)\) for \(\ell = 1, 2, \ldots, L\). We denote by \(V^\ell(x)+y\) the corresponding continuation value function after a jump of type \(\ell\) has occurred. As is typical of backward-induction approaches to dynamic programming, we first compute the final jump continuation values and feed them into pre-jump counterparts. To simplify the notation, we impose that \(\rho = 1,\) but it is straightforward to incorporate the \(\rho \ne 1\) extension we discussed in the previous subsection.

As in [Anderson et al., 2003], an HJB equation that adds concerns about robustness to misspecified jump intensities includes a robust adjustment to those intensities. The penalty parameter \(\xi\) carries a different meaning here than it did for the diffusion. For Brownian increments it penalizes the quadratic \({\frac 1 2}|h|^2\), the local relative entropy of a drift distortion; for jumps it penalizes the relative entropy of a distorted intensity, given below. We use the same symbol because a single \(\xi\) restrains both explorations in the applications that follow, but the two penalties are distinct objects. The minimizing objective and constraints are separable across jumps, so we solve:

\[\min_{g^\ell} \mathcal{J}^\ell \left[ g^\ell \left(V^\ell - V\right) + \xi \left( 1 - g^\ell + g^\ell \log g^\ell \right)\right]\]

for \(\ell = 1,2, ..., L\), where \(g^\ell \ge 0\) alters the intensity of type \(\ell,\) and the term

\[\mathcal{J}^\ell\left[1 - g^\ell + g^{\ell}\log g^\ell\right]\]

measures the relative entropy of jump intensity specifications. Notice that \(g^{\ell}\log g^\ell\) is convex and thus a gradient inequality implies that

\[g^{\ell}\log g^\ell \ge g^{\ell} - 1, \]

verifying that the relative entropy measure is positive unless \(g^{\ell} = 1\). The minimizing \(g^{\ell}\) is

\[g^{\ell*} = \exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right]\]

with a minimized objective given by the intensity \({\mathcal J}^\ell\) multiplied by:

(12.21)#\[\begin{align} & \exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right] \left(V^\ell - V\right) + \xi - \xi \exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right] \cr & - \left(V^\ell - V\right)\exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right] \cr & = \xi \left(1- \exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right]\right). \end{align}\]

The minimized objective is increasing and concave in the value function difference: \(V^\ell - V\).

Remark 12.6

To deduce the relative-entropy formula for jumps, consider a discrete-time approximation in which the probability of a type-\(\ell\) jump over a time interval \(\epsilon\) is approximately \(\epsilon{\mathcal J}^\ell g^\ell\), and the probability of no jump is \(1 - \epsilon{\mathcal J}^\ell g^\ell\), where \(g^\ell = 1\) under the baseline specification. The approximation improves as \(\epsilon\) shrinks to zero. The corresponding (approximate) relative entropy is

\[\begin{aligned} & \left(\log \epsilon + \log {\mathcal J}^\ell + \log g^\ell - \log \epsilon - \log {\mathcal J}^\ell \right) \epsilon {\mathcal J}^\ell g^\ell \cr & + \left[ \log \left( 1 - \epsilon {\mathcal J}^\ell g^\ell \right) - \log \left( 1 - \epsilon {\mathcal J}^\ell \right) \right] \left( 1 - \epsilon g^\ell {\mathcal J}^\ell \right) \end{aligned}\]

Differentiate this expression with respect to \(\epsilon\) to obtain:

\[\log g^\ell {\mathcal J}^\ell g^\ell - {\mathcal J}^\ell g^\ell + {\mathcal J}^\ell = {\mathcal J}^\ell \left( g^\ell \log g^\ell - g^\ell +1\right). \]

We will also be interested in the partial derivative of the function in (12.21) with respect to the state vector:

\[g^{\ell*} \left(\frac {\partial V^\ell}{\partial x} - \frac {\partial V}{\partial x} \right)\]

where \(g^{\ell*}\) is the minimizer used to alter the jump intensity.

When constructing the HJB equation, we retain the diffusion dynamics and now incorporate the \(L\) possible jumps. The usual term:

\[ \sum_{\ell=1}^L \mathcal{J}^\ell \left (V^\ell - V\right) .\]

is replaced by

\[\xi \sum_{\ell=1}^L \mathcal{J}^\ell \left(1- \exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right]\right) = \sum_{\ell=1}^L g^{\ell*} \mathcal{J}^\ell \left (V^\ell - V\right) + \xi \sum_{\ell = 1}^L {\mathcal J}^\ell\left(1 - g^{\ell*} + g^{\ell*} \log g^{\ell*}\right)\]

as an adjustment for robustness in the jump intensities. The resulting HJB equation is:

\[\begin{split}\begin{align} 0 = \min_{h} & - \delta V + \delta U + {\frac{\xi}{2}}|h|^2 +\left[\mu +\sigma h\right]\cdot \frac {\partial V}{\partial x} + \nu + \varsigma \cdot h\\ & + {\frac{1}{2}}{\rm trace}\left[\sigma^{\top}\frac {\partial^2 V }{\partial x \partial x^{\top}}\sigma\right] \\ & + \xi \sum_{\ell=1}^L \mathcal{J}^\ell \left(1- \exp \left[ - \frac 1 \xi \left (V^\ell - V\right) \right]\right) \end{align}\end{split}\]

Given this modified HJB equation, differentiation proceeds as before, but with two new stochastic-flow terms and modified exponential discounting that accounts for the jump possibilities.

(12.22)#\[\begin{split}\begin{align} \Upsilon_t^1 \eqdef & \xi \sum_{\ell = 1}^L \mathcal{J}^{\ell}(X_t) \left(1 - \exp \left[- \frac 1 \xi \left[V^\ell(X_t) - V(X_t) \right]\right]\right) \frac {\partial \log \mathcal{J}^{\ell}}{\partial x} (X_t) \cdot \Lambda_t & \text{(i)}\\ \Upsilon_t^2 \eqdef & \sum_{\ell=1}^L\mathcal{J}^{\ell}(X_t)g^{\ell*}(X_t) \frac {\partial V^\ell}{\partial x} (X_t)\cdot \Lambda_t & \text{(ii)} \\ \end{align}\end{split}\]

Term (i) prices the response of the jump intensity to the state. The factor multiplying \(\xi\) in term (i) is \(1 - g^{\ell *}\). Term (ii) prices the post-jump marginal valuation. In some examples the jump intensities are constant or depend only on an exogenous state; in such cases term (i) drops out and only term (ii) remains. In the applications that follow, the intensities depend on endogenous state variables, which is what makes term (i) matter. The post-jump marginal valuations in term (ii) are themselves forward-looking and conditioned on the respective jump. Significantly, these terms do not include derivatives of the \(g^{\ell*}\)’s with respect to \(x\); the \(g^{\ell*}\)’s can be viewed as exogenous inputs, much like the \(H^*\) distortions.

Note that \(V(X_t)\), \(V^\ell(X_t)\), \(\ell =1,\ldots,L\), and \(\frac {\partial V^\ell}{\partial x}(X_t)\), \(\ell = 1, \ldots, L\), are inputs to these two terms.

The counterpart of the discounting term now includes a probability adjustment for no jump occurring. This term inherits state dependence from the jump intensities, giving rise to:

\[D_t \eqdef \exp\left( - \int_0^t\left[\delta + \sum_{\ell=1}^L\mathcal{J}^{\ell}(X_u)g^{\ell*}(X_u)\right]du\right).\]

We include this probability adjustment because of our pre-jump perspective. This gives the marginal valuation formula:

(12.23)#\[\frac {\partial V}{\partial x}(X_0) \cdot \Lambda_0 + \Theta_0 = \int_0^\infty {\mathbb E} \left(D_t \left[\delta \frac {\partial \phi }{\partial x} (X_{t})\cdot \Lambda_t + \delta \Theta_t + \Upsilon^1_t + \Upsilon^2_t \right] \mid X_0, \Lambda_0, \Theta_0 \right) dt .\]

To extend Decomposition II, we rewrite the composite stochastic flow as the sum of three terms:

\[\begin{align} & \delta \frac {\partial \phi }{\partial x} (X_{t})\cdot \Lambda_t + \delta \Theta_t \cr & + \xi \sum_{\ell = 1}^L \mathcal{J}^{\ell}(X_t) \left(1 - \exp \left[ - \frac 1 \xi \left[V^\ell(X_t) - V(X_t) \right]\right] \right) \frac {\partial \log \mathcal{J}^{\ell}}{\partial x} (X_t) \cdot \Lambda_t \cr & + \sum_{\ell=1}^L\mathcal{J}^{\ell}(X_t)g^{\ell*}(X_t) \frac {\partial V^\ell}{\partial x} (X_t) \cdot \Lambda_t . \end{align}\]

which again allows us to decompose by the impacted future states. Moreover, we can deduce the contributions of different sources to the stochastic flows and assess the impacts of alternative jump types, \(\ell = 1,2,\ldots, L\).

Simulation-based methods can be used to compute marginal-value decompositions based on the extended versions of Decomposition I and Decomposition II that accommodate robustness and infrequent jumps. The simulations should be conducted under the implied worst-case diffusion dynamics.

12.4. Summary#

A partial derivative of a value function is a marginal valuation, and it has an asset-pricing representation. The stochastic responses \(\Lambda\) of Chapter 5 propagate a marginal change in an initial state through the system, and an uncertainty-adjusted expectation discounts the resulting flow of marginal contributions. Two decompositions follow, one by time horizon and one by state variable. Reporting both is what makes a marginal valuation legible: a single number conceals which state and which horizon produced it.

The uncertainty adjustment is the change of measure of Chapter 11, so robustness concerns enter valuation through the probabilities used in discounting rather than through a discount rate. The climate application shows what the decompositions buy. The social value of research and development and the social cost of global warming each turn out to be near-cancellations of larger contributions of opposite sign, a structure that no aggregate reports and that determines how each responds to a change in the aversion parameter.

12.5. Exercises#

Exercise 12.1 (Solving the marginal valuation forward)

The chapter obtains representation (12.7) by solving a recursion forward. Verify it, and then check it against a case where the value function can be computed directly.

(a) Starting from the \(\rho = 1\) version of the differentiated recursion, iterate forward and confirm (12.7). What condition on the discount rate and on the growth of \(\left\|\Lambda_t\right\|\) is needed for the sum to converge?

(b) Apply the summation-by-parts argument together with the evolution of \(\Theta\) to obtain the compact form in which \(\frac{\partial V}{\partial x}(X_0)\cdot\Lambda_0 + \Theta_0\) is a discounted expected value of \(\frac{\partial \phi}{\partial x}(X_t)\cdot\Lambda_t + \Theta_t\).

(c) Now take the continuous-time diffusion \(dX_t = {\mathbb A}X_t\,dt + {\mathbb B}\,dW_t\) with \(\phi(x) = \varphi\cdot x\) and \(\nu = \varsigma = 0\), abstracting from robustness. Solve the Feynman-Kac equation (12.11) for \(V\) by guessing \(V(x) = {\sf v}\cdot x\), and exhibit \({\sf v}\).

(d) Verify representation (12.12) for this example by computing the integral directly, using \(\Lambda_t = \exp\left({\mathbb A}t\right)\Lambda_0\) from Example 5.4 of Chapter 5. Confirm that the two answers agree.

Exercise 12.2 (Two decompositions of a marginal valuation)

Continue with the linear diffusion of Exercise 12.1(c), now with \(n = 2\) states.

(a) Write out Decomposition I explicitly. What shape does the horizon profile take, and which feature of \({\mathbb A}\) controls how fast it decays?

(b) Write out Decomposition II explicitly. Show that when \({\mathbb A}\) is diagonal, the cross terms vanish and the marginal valuation of state \(i\) involves only state \(i\).

(c) Now let \({\mathbb A}\) be upper triangular with a nonzero off-diagonal entry. Initialize \(\Lambda_0\) to the second coordinate vector and show that the valuation loads on both states. Explain the direction of the dependence in terms of the dynamics.

Exercise 12.3 (Robust adjustment of a jump intensity)

The section on jumps replaces the usual term \(\sum_\ell {\mathcal J}^\ell\left(V^\ell - V\right)\) in the HJB equation by a robust counterpart. Derive it.

For a single jump type, the minimization is

\[\min_{g \ge 0} \ {\mathcal J}\left[g\left(V^\ell - V\right) + \xi\left(1 - g + g\log g\right)\right] .\]

(a) Verify that \({\mathcal J}\left[1 - g + g\log g\right] \ge 0\) with equality only at \(g = 1\), so that it is a legitimate divergence. Which gradient inequality is being used?

(b) Solve the minimization for \(g^{\ell *}\) and show that the minimized objective equals \({\mathcal J}^\ell \xi\left(1 - g^{\ell *}\right)\), matching (12.21).

(c) Show that the minimized objective is strictly increasing in \(V^\ell - V\), and interpret: which jumps does the adversary make more likely, and which less?

(d) Take the limits \(\xi \rightarrow \infty\) and \(\xi \downarrow 0\). In each case say what happens to \(g^{\ell *}\) and to the minimized objective, and describe the decision maker being represented. Why does the chapter emphasize that derivatives of \(g^{\ell *}\) with respect to \(x\) do not appear in the stochastic flows (12.22)?

Exercise 12.4 (The auxiliary state that holds the distortion fixed)

To represent the robust marginal valuation, the chapter introduces a second state \({\overline X}\) that carries the drift distortion, and works with \({\overline V}(x,{\bar x})\) solving (12.19). This exercise explains why the device is needed and what it buys.

(a) Explain why differentiating the minimized HJB equation (12.16) directly, treating \(h^*(x)\) as a function of \(x\), would require terms in \(\frac{\partial h^*}{\partial x}\). Why is it legitimate to omit them?

(b) Show that \({\overline V}(x,x) = V(x)\), that \({\bar x} = x\) solves \(\min_{\bar x}{\overline V}(x,{\bar x})\), and hence that \({\overline V}_{\bar x}(x,x) = 0\).

(c) Deduce the two relations \({\overline V}_x(x,x) = V_x(x)\) and \({\overline V}_{xx'}(x,x) + {\overline V}_{x{\bar x}'}(x,x) = V_{xx'}(x)\). Which of the two is used when the marginal valuation is expressed as a discounted stochastic flow?

(d) Remark 12.5 notes that for some applications the drift distortion should be viewed as endogenous rather than exogenous. Using (a) and (b), explain precisely what the \({\overline X}\) construction is assuming away about the response of the adversary, and why that assumption is harmless for computing a derivative but would not be harmless for computing a level.

12.6. Answers#

Solution to Exercise 12.1 (Solving the marginal valuation forward)

(a) The \(\rho = 1\) recursion is

\[\frac{\partial V}{\partial x}(X_t)\cdot\Lambda_t = \left[1-\exp(-\delta)\right]\frac{\partial\phi}{\partial x}(X_t)\cdot\Lambda_t + \exp(-\delta){\widetilde{\mathbb E}}\left[\frac{\partial\kappa}{\partial x^\top}(X_t,W_{t+1})\Lambda_t\Big|\cdot\right] + \exp(-\delta){\widetilde{\mathbb E}}\left[\frac{\partial V}{\partial x}(X_{t+1})\cdot\Lambda_{t+1}\Big|\cdot\right].\]

Substituting the same expression for the date \(t+1\) term, iterating, and using the Law of Iterated Expectations produces (12.7). Convergence requires that the remainder \(\exp(-\delta T){\widetilde{\mathbb E}}\left[\frac{\partial V}{\partial x}(X_T)\cdot\Lambda_T\right]\) vanish, for which it suffices that \({\widetilde{\mathbb E}}\left\|\Lambda_T\right\|\) grow at a rate strictly below \(\delta\). Discounting must dominate the growth of the stochastic response, which is the same race that governs finite valuation in Chapter 8.

(b) The evolution \(\Theta_{t+1}-\Theta_t = \frac{\partial\kappa}{\partial x^\top}(X_t,W_{t+1})\Lambda_t\) lets the second sum in (12.7) be rewritten by changing the order of summation:

\[\exp(-\delta)\sum_{t=0}^\infty \exp(-\delta t){\widetilde{\mathbb E}}\left[\Theta_{t+1}-\Theta_t\right] = \left[1-\exp(-\delta)\right]\sum_{t=0}^\infty\exp(-\delta t){\widetilde{\mathbb E}}\left[\Theta_t\right] - \Theta_0 ,\]

so moving \(\Theta_0\) to the left gives

\[\frac{\partial V}{\partial x}(X_0)\cdot\Lambda_0 + \Theta_0 = \left[1-\exp(-\delta)\right]\sum_{t=0}^\infty\exp(-\delta t){\widetilde{\mathbb E}}\left[\frac{\partial\phi}{\partial x}(X_t)\cdot\Lambda_t + \Theta_t\right] .\]

The two pieces of the stochastic flow, the marginal reward and the accumulated response of \(Y\), travel together.

(c) With \(\nu = \varsigma = 0\) there is no \(Y\) and (12.11) reads

\[0 = \delta\varphi\cdot x - \delta V(x) + {\mathbb A}x\cdot\frac{\partial V}{\partial x}(x) + {\frac 1 2}{\rm trace}\left[{\mathbb B}^\top V_{xx}{\mathbb B}\right] .\]

Guessing \(V(x) = {\sf v}\cdot x\) kills the second-derivative term and leaves \(\delta\varphi^\top - \delta {\sf v}^\top + {\sf v}^\top{\mathbb A} = 0\), so

\[{\sf v}^\top = \delta\,\varphi^\top\left(\delta{\mathbb I}-{\mathbb A}\right)^{-1} ,\]

which is well defined because \({\mathbb A}\) is stable and \(\delta > 0\).

(d) Since \(\frac{\partial\phi}{\partial x} = \varphi^\top\) is constant and \(\Lambda_t = \exp\left({\mathbb A}t\right)\Lambda_0\) is nonstochastic here, (12.12) gives

\[\frac{\partial V}{\partial x}(X_0)\cdot\Lambda_0 = \delta\int_0^\infty \exp(-\delta t)\,\varphi^\top\exp\left({\mathbb A}t\right)\Lambda_0\, dt = \delta\,\varphi^\top\left[\int_0^\infty \exp\left(\left({\mathbb A}-\delta{\mathbb I}\right)t\right)dt\right]\Lambda_0 .\]

The matrix \({\mathbb A}-\delta{\mathbb I}\) is stable, so the integral equals \(\left(\delta{\mathbb I}-{\mathbb A}\right)^{-1}\) and the whole expression is \(\delta\varphi^\top\left(\delta{\mathbb I}-{\mathbb A}\right)^{-1}\Lambda_0 = {\sf v}^\top\Lambda_0\), which agrees with (c) since \(\frac{\partial V}{\partial x} = {\sf v}\).

We obtained the left side by solving a partial differential equation, the right side by simulating a response process forward and discounting. The representation converts a boundary-value problem into an asset-pricing calculation, and that is what makes the decompositions of the next exercise possible.

Solution to Exercise 12.2 (Two decompositions of a marginal valuation)

(a) Decomposition I reports, for each horizon,

\[\delta\exp(-\delta t)\,\varphi^\top \exp\left({\mathbb A}t\right)\Lambda_0 .\]

The profile is a product of an exponential discount factor and a matrix exponential, so it is a sum of terms of the form \(\exp\left(\left({\sf z}_i - \delta\right)t\right)\) over the eigenvalues \({\sf z}_i\) of \({\mathbb A}\). The decay rate is \(\delta - \max_i {\rm Re}({\sf z}_i)\). A slow eigenvalue moves the valuation toward distant horizons and flattens the profile. Complex eigenvalues produce a damped oscillation rather than monotone decay.

(b) Decomposition II splits the same object by which future state is being moved:

\[\delta\sum_{j=1}^n \int_0^\infty \exp(-\delta t)\,\varphi_j \left[\exp\left({\mathbb A}t\right)\Lambda_0\right]_j dt .\]

If \({\mathbb A} = {\rm diag}\left({\sf z}_1,{\sf z}_2\right)\) then \(\exp\left({\mathbb A}t\right)\) is diagonal, so \(\left[\exp\left({\mathbb A}t\right)\Lambda_0\right]_j = e^{{\sf z}_jt}\Lambda_{0,j}\). Initializing \(\Lambda_0\) to the \(i^{th}\) coordinate vector makes every component except the \(i^{th}\) identically zero, and the marginal valuation of state \(i\) is \(\delta\varphi_i/\left(\delta - {\sf z}_i\right)\), involving state \(i\) alone.

(c) Take

\[\begin{split}{\mathbb A} = \begin{bmatrix} {\sf z}_1 & {\sf c} \\ 0 & {\sf z}_2\end{bmatrix}, \qquad {\sf c}\ne 0, \qquad \Lambda_0 = \begin{bmatrix}0\\1\end{bmatrix}.\end{split}\]

Then \(\exp\left({\mathbb A}t\right)\Lambda_0\) has second component \(e^{{\sf z}_2t}\) and first component \({\sf c}\,\frac{e^{{\sf z}_1t}-e^{{\sf z}_2t}}{{\sf z}_1-{\sf z}_2}\), which is nonzero. So the valuation loads on both states, with the state-one contribution proportional to \({\sf c}\).

Perturbing state two at date zero feeds into state one at later dates through the off-diagonal entry \({\sf c}\), and once state one has moved it contributes \(\varphi_1\) per unit to the flow. The sign of the extra contribution is the sign of \({\sf c}\varphi_1\). Nothing about state two’s own reward \(\varphi_2\) is involved.

Solution to Exercise 12.3 (Robust adjustment of a jump intensity)

(a) The function \(g\log g\) is strictly convex on \((0,\infty)\) with derivative \(1 + \log g\), equal to \(1\) at \(g = 1\). The gradient inequality at \(g=1\) therefore gives \(g\log g \ge 0 + 1\cdot(g-1) = g - 1\), that is \(1 - g + g\log g \ge 0\), with equality only at \(g = 1\) by strictness. Multiplying by \({\mathcal J} > 0\) preserves the sign. This is the same convexity used for relative entropy throughout Chapter 7, applied here to intensities instead of densities.

(b) Differentiating the bracket with respect to \(g\),

\[\left(V^\ell - V\right) + \xi\left(-1 + \log g + 1\right) = \left(V^\ell-V\right) + \xi\log g = 0 \quad\Longrightarrow\quad g^{\ell*} = \exp\left[-{\frac 1 \xi}\left(V^\ell-V\right)\right] .\]

Substituting back, and using \(\log g^{\ell*} = -\left(V^\ell-V\right)/\xi\),

\[g^{\ell*}\left(V^\ell-V\right) + \xi - \xi g^{\ell*} + \xi g^{\ell*}\left[-{\frac 1 \xi}\left(V^\ell-V\right)\right] = \xi\left(1-g^{\ell*}\right) ,\]

the first two terms cancelling against the last. Multiplying by \({\mathcal J}^\ell\) gives (12.21). The third form in that display follows from \(g^{\ell*}\exp\left[{\frac 1 \xi}\left(V^\ell-V\right)\right] = 1\).

(c) Writing \({\sf d} = V^\ell - V\), the minimized objective per unit intensity is \(\xi\left(1 - e^{-{\sf d}/\xi}\right)\), whose derivative is \(e^{-{\sf d}/\xi} > 0\). So it is strictly increasing in \({\sf d}\).

The adversary raises the intensity of jumps the decision maker dislikes. If \(V^\ell < V\), so the post-jump value is lower, then \(g^{\ell*} = \exp\left[-\left(V^\ell-V\right)/\xi\right] > 1\) and the jump is made more likely; if \(V^\ell > V\) it is made less likely. A decision maker who fears misspecification of a hazard rate acts as though bad events arrive faster and good events slower than the baseline model says.

(d) As \(\xi \rightarrow \infty\), \(g^{\ell*}\rightarrow 1\) and \(\xi\left(1-g^{\ell*}\right)\rightarrow V^\ell - V\): the distortion vanishes and the usual term \(\sum_\ell{\mathcal J}^\ell\left(V^\ell-V\right)\) is recovered. As \(\xi\downarrow 0\), \(g^{\ell*}\rightarrow \infty\) when \(V^\ell < V\) and \(\rightarrow 0\) when \(V^\ell > V\), while \(\xi\left(1-g^{\ell*}\right)\) tends to \(-\infty\) in the first case and to \(0\) in the second. The decision maker becomes one who treats any adverse jump as certain to arrive immediately, and is entirely unwilling to count on favorable ones.

The chapter emphasizes the absence of \(\frac{\partial g^{\ell*}}{\partial x}\) for the same reason that the \({\overline X}\) device of Exercise 12.4 exists. Because \(g^{\ell*}\) minimizes the objective, the envelope theorem applies: at the optimum the derivative of the objective with respect to \(g\) is zero, so the indirect effect of \(x\) on the objective through \(g^{\ell*}\) vanishes to first order. Only the direct effect survives. This is what lets the \(g^{\ell*}\)’s be treated as exogenous inputs in (12.22), exactly as the \(H^*\) distortions are treated for the diffusion, and it is what keeps the marginal valuation formula (12.23) a discounted expectation rather than something involving derivatives of the adversary’s policy.

Solution to Exercise 12.4 (The auxiliary state that holds the distortion fixed)

(a) Once the minimization in (12.16) has been solved, \(h^*\) is a function of \(x\) through (12.17). Differentiating the resulting equation with respect to \(x\) by the chain rule would generate terms multiplying \(\frac{\partial h^*}{\partial x}\) wherever \(h^*\) appears, namely in \(\sigma(x)h^*\), in \(\varsigma(x)\cdot h^*\), and in \(\frac{\xi}{2}\left|h^*\right|^2\).

They may be omitted for the envelope reason. Collecting the coefficient of \(\frac{\partial h^*}{\partial x}\) gives exactly the derivative of the minimand with respect to \(h\), evaluated at the minimizer, which is zero by the first-order condition. The indirect effect through the adversary’s response cancels.

(b) Setting \({\bar x} = x\) in (12.19) reproduces (12.16) evaluated at the minimizing \(h^*(x)\), and both are solved by the same function; hence \({\overline V}(x,x) = V(x)\). For the minimization, note that \({\overline V}(x,{\bar x})\) is the value delivered when the distortion is held at \(h^*({\bar x})\) while the state is at \(x\). Since \(h^*(x)\) is by construction the minimizing choice at \(x\), no other \({\bar x}\) can do better, so \({\bar x} = x\) attains \(\min_{\bar x}{\overline V}(x,{\bar x})\). The first-order condition for an interior minimum is \({\overline V}_{\bar x}(x,x) = 0\).

(c) Differentiate the identity \({\overline V}(x,x) = V(x)\) with respect to \(x\), using the chain rule through both arguments:

\[{\overline V}_x(x,x) + {\overline V}_{\bar x}(x,x) = V_x(x) ,\]

and the second term vanishes by (b), giving \({\overline V}_x(x,x) = V_x(x)\). Differentiating once more,

\[{\overline V}_{xx^{\top}}(x,x) + {\overline V}_{x{\bar x}^{\top}}(x,x) = V_{xx^{\top}}(x) ,\]

where the terms involving \({\overline V}_{{\bar x}x^{\top}}\) and \({\overline V}_{{\bar x}{\bar x}^{\top}}\) drop out because \({\overline V}_{\bar x}(x,x) \equiv 0\) as a function of \(x\).

The marginal valuation uses the first relation. \(V_x\) coincides with \({\overline V}_x\), the derivative taken holding the distortion fixed; \({\overline V}_x\) satisfies a Feynman-Kac equation with no derivatives of \(h^*\) in it, so it admits the discounted-flow representation.

(d) The construction assumes that the adversary does not re-optimize in response to the perturbation being priced. \({\overline X}\) is initialized at \(X_0\) and thereafter evolves under \({\bar \mu}\), carrying the distortion \(h^*\left({\overline X}_t\right)\) along a path that is not revised when we tilt \(\Lambda_0\).

For a derivative this is harmless, and (b) is why: the adversary’s choice is at an interior optimum, so a marginal change in the state produces only a second-order change in the value through his response. For a level it would not be harmless. Comparing the value under two non-infinitesimally different policies requires the adversary to re-optimize at every point along the way, and the difference between his old and new choices is then first order. This is the distinction Remark 12.5 draws when it says that for some applications the drift distortion should be treated as a change in the endogenous dynamics rather than as exogenous: local policy assessment can hold it fixed, global comparison cannot.