4. Processes with Markovian increments#
In this chapter, we use a stationary Markov process to construct a process that displays stochastic arithmetic growth, then show how to extract a linear time trend and a martingale. Eventually, we will explore the implications of exponentiating this process to transform an arithmetically growing process, like those described in this chapter, to construct a process that displays geometric growth.
4.1. Definition of additive functional#
Let \(\{W_{t+1} : t \ge 0\}\) be a \(k\)-dimensional stochastic process of unanticipated economic shocks. Let \(\{ X_t : t \ge 0 \}\) be a discrete-time stationary Markov process that is generated by initial distribution \(Q\) for \(X_0\) and transition equation
where \(\phi\) is a Borel measurable function. Let \(\left\{ \mathfrak{A}_t : t=0,1,... \right\}\) be the filtration generated by histories of \(W\) and \(X\); \(\mathfrak{A}_t\) serves as the information set (sigma algebra) generated by \(X_0, W_1, \ldots , W_t\). We presume that the conditional probability distribution for \(W_{t+1}\) conditioned on \(\mathfrak{A}_t\) depends only on \(X_t\). To assure that the process \(\{W_{t+1} : t \ge 0 \}\) represents unanticipated shocks, we restrict it to satisfy
We condition on a statistical model in the sense of Section Limiting Empirical Measures in Chapter Stochastic Processes and Laws of Large Numbers. Given this, for notational simplicity, we simply assume that the stationary \(\{X_t : t \ge 0\}\) process is ergodic.[1] The Markov structure of \(\{ X_t \}\) makes the distribution of \((X_{t+1}, W_{t+1}) \) conditioned on \(\mathfrak{A}_t\) depend only on \(X_t\).[2]
Definition 4.1
A process \(\{ Y_{t} \}\) is said to be an additive functional if it can be represented as
for a (Borel measurable) function \(\kappa: {\mathbb R}^n \times {\mathbb R}^k \rightarrow {\mathbb R}\), or equivalently
where we initialize \(Y_0\) at some arbitrary random variable depending on date zero information, although such a restriction is not needed for what follows.
When \(Y_0\) is a function of \(X_0\), we can construct \(Y_t\) as a function of the underlying Markov process between dates zero and \(t\).
Definition 4.2
An additive functional \(\{ Y_t : t=0,1,...\}\) is said to be an additive martingale if \(E\left[ \kappa(X_{t}, W_{t+1}) \vert X_t \right] = 0.\)
It is easy to see that a linear combination of two additive functionals \(\{Y_{t}^{[1]}\}\) and \(\{Y_{t}^{[2]}\}\) is itself an additive functional. If \(\kappa_1\) is used to construct the first process and \(\kappa_2\) the second process, then \(\kappa = \kappa_1 + \kappa_2\) can be used to construct the sum of the two processes. –>
Example 4.1
(Stochastic Volatility) Suppose that
where \(\{ W_{t+1} : t\ge 0 \}\) is an i.i.d.~sequence of standardized multivariate normally distributed random vectors, \({\mathbb A}\) is a stable matrix, and \({\mathbb B}\) has full column rank, and the random vector \(X_0\) is generated by initial distribution \(Q\) associated with the stationary distribution for the \(\{ X_t \}\) process. Here \(\mu(X_t)\) is the conditional mean of \(Y_{t+1} - Y_t\) and \(|\sigma(X_t)|^2\) is its conditional variance. When \(\sigma\) depends on \(X_t\), This is called a stochastic volatility model because \(|\sigma(X_t)|^2\) is a stochastic process.
In Example 4.1, when the conditional mean \(\mu(X_t) = 0\), the process \(\{Y_t \}\) is a martingale. Note that \(E\left[ \kappa( X_t, W_{t+1} ) \vert X_t \right] = 0\) implies the usual martingale restriction
4.2. Extracting Martingales#
We can decompose an additive functional into a sum of components, one of which is an additive martingale that encapsulates all long-run stochastic variation as in Proposition 3.1. In this section, we show how to extract the martingale component. We adopt a construction like that used to establish Proposition 3.1 and proceed in four steps.
Construct the trend coefficient as the unconditional expectation:
Form the random variable \(H_t\) by computing multiperiod forecasts for each horizon and summing these forecasts over all horizons. Start by constructing
Thus
Summing the terms, construct
where
where \({\mathbb T}\) is the operator defined in (2.1). The right side becomes a function of only \((X_{t-1},W_t)\) once we substitute for \(\phi(X_{t-1},W_t)\) for \(X_t\) as implied by (4.1).
This construction requires that the infinite sum
converges in mean square relative to the stationary distribution for \(\{X_t: t\ge 0\}\). A sufficient condition for this is that \({\mathbb T}^m\) is a strong contraction for some integer \(m \geq 1\) and \(\overline{\kappa} \in {\mathcal N}\) where \({\mathcal N}\) is defined in (2.9).
Compute
where[3]
Build the martingale increment:
where
By construction, the expectation of \(\kappa_m(X_t, W_{t+1})\) conditioned on \(X_t\) is zero.
Armed with these calculations, we now report a Markov counterpart to Proposition 3.1.
Proposition 4.1
Suppose that \(\{Y_{t} : t\ge 0\}\) is an additive functional, that \({\mathbb T}^m\) is a strong contraction on \({\mathcal N}\) for some \(m\), and that \(E[\kappa(X_{t},W_{t+1})^2] < \infty\). Then
Notice that the martingale component is itself an additive functional. The first is a linear time trend, the second an additive martingale, the third a stationary process with mean zero, and the fourth a time-invariant constant. If we happen to impose the initialization: \(Y_0 = - \kappa_+(X_0)\), then the fourth term is zero. We use a Proposition 4.1 decomposition as a way to associate a ‘’permanent shock’’ with an additive functional. The permanent shock is the increment to the martingale.
4.3. Applications#
We now compute martingale increments for two models of economic time series.
4.3.1. Vector autoregression#
We apply the four-step construction in Algorithm Martingale construction when the Markov state \(\{ X_t \}\) follows a first-order VAR
where \({\mathbb A}\) is a stable matrix and \(\{ W_{t+1} : t\ge 0 \}\) is a sequence of independent and identically normally distributed random variables with mean vector zero and identity covariance matrix. The one-step ahead conditional covariance matrix of the time \(t+1\) shocks \(\mathbb{B} W_{t+1}\) to \(X_{t+1}\) equals \(\mathbb{B} \mathbb{B}'\). Let
where \({\mathbb D}\) and \({\mathbb F}\) are row vectors with the same dimensions as \(X_t\) and \(W_{t+1}\), respectively. For this example, the four steps of Algorithm Martingale construction become:
The trend growth rate is \(\nu\) as specified.
- \[\kappa_h(X_{t-1}, W_t, X_t ) = {\mathbb D} X_{t-1} + {\mathbb F} W_{t} + {\mathbb D}({\mathbb I} - {\mathbb A} )^{-1} X_t \]
- \[\kappa_+(X_t) = {\mathbb D} X_t + {\mathbb D} ({\mathbb I} - {\mathbb A} )^{-1}{\mathbb A} X_t\]
- (4.5)#\[\kappa_m(X_{t-1}, W_t) = {\mathbb F} W_{t} + {\mathbb D} ({\mathbb I} - {\mathbb A} )^{-1} (X_t - {\mathbb A} X_{t-1} ) = \left[{\mathbb F} + {\mathbb D} ({\mathbb I} - {\mathbb A} )^{-1} {\mathbb B} \right] W_t. \]
From Example 1.7, we expect the coefficient of martingale increment to be the sum of impulse responses for the increment process \(\{ {\mathbb D} X_t + {\mathbb F} W_{t+1} : t\ge 0\}\). The impulse response function is the sequence of vectors:
Summing these vectors gives
as anticipated.
4.3.2. An economic rationale#
A special case of what is called recursive utility, represents preferences using recursively constructed continuation values. As an illustration, let \(V_t\) be the date \(t\) continuation value. Suppose that for \({\widehat C}_t = \log C_t\),
where the first two equations are backward looking and third and fourth ones are forward looking. In particular, current \({\widehat V}_t\) depends on the future \({\widehat V}_{t+1}\) through the construction of \({\widehat R}_t.\) We restrict \(0 < \beta < 1,\) \(\gamma \ge 1\), and \({\mathbb A}\) is a stable matrix. Instead of using the conditional expectation of the future continuation value, these preferences use what is called a certainly equivalent in conjunction with an exponential risk adjustment as reflected in formula for \({\widehat R}_t\). The limiting case as \(\gamma \downarrow 1\) is
which can be solved forward to obtain:
Thus this limiting case gives discounted expected logarithmic utility as way to assess alternative consumption processes.
When the process \(\{ {\mathbb D} X_t\}\) is highly persistent, there is said to be substantial “long-run risk” in consumption. [Bansal and Yaron, 2004] also consider a process governing stochastic volatility that we abstract from in the computations that follow. While the illustrative calculations in what follows use the VAR-type application of [Hansen et al., 2008], some of the basic insights extend much more generally.
To provide a Markov characterization of the continuation value process, it is convenient to subtract \({\widehat C}_t\) from both sides of the second equation:
Given the assumed dynamics for the growth rate in consumption, \({\widehat C}\) and \({\widehat V}\) are co-integrated. We use ``guess and verify’’ to seek a solution of the form:
The coefficient vector \(\upsilon\) satisfies:
with a solution:
The scalar \({\sf v}\) satisfies:
Solving this forward gives:
Notice, in particular, the term in the square brackets of (4.7). The first contribution is average growth rate expressed in logarithms. For the second contribution, it is revealing to inspect the \(\beta = 1\) limit:
This gives a variance adjustment to the continuation value that the product of \((1-\gamma)/2\) and variance of the increment to the martingale component of the logarithm of consumption as reported in (4.5). When \(\gamma >1,\) this variance coincides with the continuation value relative to the case of logarithmic utility. Larger \(\gamma\) increases this adjustment. The \(\beta = 1\) limit is pertinent because this parameter is often set close to unity in the macro-finance literature.
In Chapter Exploring Recursive Utility, we investigate more generally the implications of a class of recursive utility preferences which nests this specification as a special case. Among other results, we obtain analogous formulas as first-order approximations to more general consumption dynamics.
4.3.3. Growth-Rate Regimes#
We construct a Proposition 4.1 decomposition for a model with persistent switches in the conditional mean and volatility of the growth rate \(Y_{t+1}- Y_t\).
Suppose that \(\{X_t : t \ge 0\}\) evolves according to an \(n\)-state Markov chain with transition matrix \({\mathbb P}\). Realized values of \(X_t\) are coordinate vectors in \({\mathbb R}^n\). Suppose that \({\mathbb P}\) has only one unit eigenvalue. Let \({\bf q}\) be the row eigenvector associated with that unit eigenvalue normalized so that \({\bf q} \cdot {\bf 1}_n = 1\) and
Consider an additive functional satisfying
where \(\{ W_{1,t} \}\) is an i.i.d. sequence of multivariate standard normally distributed random vectors. Evidently, the stationary Markov \(\{X_t : t \ge 0 \}\) process induces discrete changes in both the conditional mean and the conditional volatility of the growth rate process \(\{ Y_{t+1} - Y_t \}\).
Observe that \( E (X_{t+1} | X_t ) ={\mathbb P} X_t \) and let
Thus we can represent the evolution of the Markov chain as
\(\{W_{2,t+1} : t \ge 0 \}\) is an \(n \times 1\) discrete-valued vector process that satisfies \(E ( W_{2,t+1} | X_t) = 0 \), which is therefore a martingale increment sequence adapted to \(X_t, X_{t-1}, ..., X_0\).
We again apply the four-step construction in algorithm.[4]
- \[\nu = {\mathbb D} {\bf q} \]
- \[H_t = {\mathbb D} (X_{t-1} - {\bf q}) + {X_{t-1}}'{\mathbb F} W_{1,t} + {\mathbb D}\left({\mathbb I} - {\mathbb P}\right)^{-1} X_t \]
- \[H_t^+ = E\left( H_{t+1} \mid X_t \right) = {\mathbb D} \left( X_{t} - {\bf q} \right) + {\mathbb D}\left({\mathbb I} - {\mathbb P}\right)^{-1} {\mathbb P}X_t\]
which implies that
\[\kappa_+(x) = {\mathbb D} \left( X_{t} - {\bf q} \right) + {\mathbb D}\left({\mathbb I} - {\mathbb P}\right)^{-1} {\mathbb P}x\] - \[G_t = H_t - H_{t-1}^+ = {X_{t-1}}'{\mathbb F} W_{1,t} + {\mathbb D}\left({\mathbb I} - {\mathbb P}\right)^{-1} W_{2,t}\]
where we have substituted from equation (4.8).
The martingale increment has both continuous and discrete components: