Markov Processes. Stopping Times

WarningDraft

This lecture note is still being edited: the text may contain typos, inaccuracies, and locally unpolished passages. If you find something, please let the author know (contact information on the course page).

In this section, either

• \(T = \mathbb {Z}_+\) or \(T = \mathbb {Z}_+ \cup \left\{ +\infty \right\} = \left\{ 0,1,2,3,\ldots , +\infty \right\}\) (discrete case).

• \(T = [0,1]\) or \(T = \mathbb {R}_+\) or \(T = \mathbb {R}_+ \cup \left\{ +\infty \right\} = [0,+\infty ]\) (continuous case).

Recall that a union of sigma-algebras is not always a sigma-algebra. For the minimal sigma-algebra on \(\Omega\) generated by the union of some sub-sigma-algebras \(\mathcal{F}_1, \mathcal{F}_2 \subset 2^{\Omega }\), we use the notation \(\mathcal{F}_1 \vee \mathcal{F}_2\). Briefly:

\[ \begin{align} \mathcal{F}_1 \vee \mathcal{F}_2 & := \sigma \left(\mathcal{F}_1 \cup \mathcal{F}_2 \right) \\ & = \sigma \left(\left\{ A \; : \; A \in \mathcal{F}_1 \text{ or } A \in \mathcal{F}_2\right\} \right) \end{align} \]

Note that the intersection of any collection of sigma-algebras is a sigma-algebra. So there is no need to introduce a notation \(\mathcal{F}_1 \wedge \mathcal{F}_2\). Namely,

\[ \mathcal{F}_1 \cap \mathcal{F}_2 = \sigma \left\{ \mathcal{F}_1 \cap \mathcal{F}_2\right\} \]

1 Filtration

1.1 The flow of information: filtration

Definition 1 Let \((\Omega , \mathcal{F}, \mathbb {P})\) be a probability space. A collection of \(\sigma\)-algebras \(\mathbb {F}=\left(\mathcal{F}_{t}, t \in T\right)\), where \(T \subset \mathbb {R}\), is called a filtration (or a flow of \(\sigma\)-algebras) on \((\Omega , \mathcal{F}, \mathbb {P})\), if

\[ \quad \mathcal{F}_{s} \subset \mathcal{F}_{t} \subset \mathcal{F} \qquad \text{for any } s < t \]

The quadruple \((\Omega , \mathcal{F}, \mathbb {F}, \mathbb {P})\) is called a filtered probability space or a stochastic basis.

A filtration can be understood as a flow of information that increases over time.

Definition 2 A random process \(\left(X_{t}, t \in T\right)\) is called adapted to the filtration \(\mathbb {F}=\left(\mathcal{F}_{t}, t \in T\right)\), if for all \(t \in T\) the random variable \(X_{t}\) is \(\mathcal{F}_{t}\)-measurable, i.e. \[ \sigma \left(X_{t}\right)=\mathcal{F}_{X_{t}} \subset \mathcal{F}_{t} \]

Where might a filtration on a probability space come from?

Definition 3 The natural filtration of the process \(\left(X_{t}, t \in T\right), T \subset \mathbb {R}\), is the filtration \(\mathbb {F}^{X}=\left(\mathcal{F}_{t}^{X}, t \in T\right)\), where \[ \mathcal{F}_{t}^{X}:= \sigma \left(X_{s} \; : \; s \leqslant t, s \in T\right). \] That is, \(\mathcal{F}_{t}^{X}\) is the minimal sigma-algebra with respect to which the entire process from \(0\) to \(t\) is measurable.

Any process is adapted to its own natural filtration.

ExampleExample 1

Let \[ \Omega = \left\{ 0,1\right\} ^{\mathbb {Z}_+} = \left\{ (i_1, i_2, \ldots ) \; : \; i_1, i_2, \ldots \in \left\{ 0,1\right\} \right\} , \] let \(\mathcal{F} = 2^{\Omega }\), let \(\mathbb {P}\) reflect a Bernoulli scheme with parameter \(p \in [0,1]\). Let \(X_{n}(\omega ) := \omega_n\). We have \(X_{1}, X_{2}, \ldots\) — a discrete random process of independent components with distribution \(\operatorname {Ber}(p)\). Characterize the natural filtration.

SolutionSolution

\(\mathcal{F}_{n}^{X}\) is the minimal sigma-algebra with respect to which all projections \(X_k\) for \(k = 1, \ldots , n\) are measurable. That is, \(\mathcal{F}_{n}^{X}\) is generated by events of the form \[ A_{i_1, \ldots , i_{n}} = \left\{ (i_1, i_2, \ldots , i_n, i_{n+1}, i_{n+2}, \ldots ) \; : \; i_{n+1}, i_{n+2}, \ldots \in \left\{ 0,1\right\} \right\} \] for arbitrary fixed \(i_1, \ldots , i_{n} \in \left\{ 0,1\right\}\). \(\mathcal{F}_{n}^{X}\) reflects knowledge of the outcomes of coin tosses \(1\) through \(n\), and no more. For example, \(\mathcal{F}^{X}_1\) is generated by \(2\) events: \[ \begin{align} A_{0} & = \left\{ (0,i_2,i_3, \ldots ) \; : \; i_{2},i_{3},\ldots \in \left\{ 0,1\right\} \right\} \quad \text{and} \\ & \quad A_{1} = \left\{ (1,i_2,i_3, \ldots ) \; : \; i_{2},i_{3},\ldots \in \left\{ 0,1\right\} \right\} \end{align} \] and consists, respectively, of \(4\) events: \[ \mathcal{F}^{X}_{1} = \left\{ \varnothing , A_{0}, A_{1}, \Omega \right\} \]

1.2 Standard conditions

Denote by \(\mathcal{N}_{\mathbb {P}} := \left\{ B \subset \omega : \exists A \in \mathcal{F} \hookrightarrow B \subset A, \; \mathbb {P}\left(A\right) = 0\right\}\) all the negligible subsets of \(\Omega\), i.e. all possible subsets of all sets of \(\mathbb {P}\)-measure zero. An equivalent formula:

\[ \mathcal{N}_{\mathbb {P}} = \bigcup _{\substack {A \in \mathcal{F} \\ \mathbb {P}\left(A\right) = 0}} 2^A. \]

Recall that a probability space \((\Omega , \mathcal{F}, \mathbb {P})\) is called complete, if \(\mathcal{N}_{\mathbb {P}} \subset \mathcal{F}\).

Definition 4 A filtration \(\mathbb {F} = (\mathcal{F}_{t}, t \in T)\) is called complete, if every sigma-algebra in \(\mathbb {F}\) contains the \(\mathbb {P}\)-negligible sets: \[ \forall t \in T: \; \mathcal{N}_{\mathbb {P}} \subset \mathcal{F}_t \] A filtered probability space is called complete, if its filtration is complete and \((\Omega , \mathcal{F}, \mathbb {P})\) itself is complete: \(\mathcal{N}_{\mathbb {P}} \subset \mathcal{F}\).

In fact, for the completeness of a filtered space it suffices to require that all \(\mathbb {P}\)-negligible sets \(\mathcal{N}_{\mathbb {P}}\) be contained in \(\mathcal{F}_0\). Indeed, all other sigma-algebras of the filtration contain \(\mathcal{F}_0\).

Why is completeness of the filtration needed? We have repeatedly passed to modifications of processes, treating them as practically indistinguishable from one another. However, this intuitive picture conflicts with the notion of being adapted. Recall: \(Y_t\) is a modification of \(X_t\), if for every \(t \in T\) we have \(X_t \stackrel{\text{a.s.}}{=} Y_t\), i.e. there exists \(\Omega_t \subset \Omega\) with \(\mathbb {P}\left(\Omega_t\right) = 1\), on which \(X_t(\omega ) = Y_t(\omega )\). Note that the filtration plays no role at all in this definition. This gives rise to a pathology: a modification of an adapted process need not be adapted. The simplest example: let \(X_0 \equiv 1\) and \(Y_0 = \; \mathbb {1}_{\Omega '}\) for some \(\Omega ' \in \mathcal{F}\) with \(\mathbb {P}\left(\Omega '\right) = 1\). Then \(Y_0 \stackrel{\text{a.s.}}{=} X_0\), but \(X_0\) is measurable with respect to the trivial \(\sigma\)-algebra \(\{ \varnothing , \Omega \}\), whereas \(Y_0\) is not. Completeness of the filtration eliminates this defect: if \(\mathcal{F}_0\) contains all sets of measure zero, then the event \(\Omega '\) automatically belongs to \(\mathcal{F}_0\), and \(Y_0\) turns out to be measurable. More generally, any modification of an adapted process remains adapted with respect to a complete filtration — it is precisely this property that makes completeness a standard requirement in the theory of random processes.

In the continuous case, a useful requirement is right-continuity of the filtration:

\[ \mathcal{F}_t = \mathcal{F}_{t+},\quad \text{ for all }t\in T \]

where

\[ \mathcal{F}_{t+} := \bigcap _{\varepsilon > 0} \mathcal{F}_{t + \varepsilon } \]

Definition 5 Let \((\Omega , \mathcal{F}, \mathbb {F}, \mathbb {P})\) be a filtered probability space. It is said to satisfy the standard conditions (usual hypotheses), if

• The filtration \(\mathbb {F}\) is complete: \(\mathcal{N}_{\mathbb {P}} \subset \mathcal{F}_0\) (as a consequence, the probability space itself is complete, since \(\mathcal{F}_0 \subset \mathcal{F}\));

• The filtration is right-continuous: \(\mathcal{F}_t = \mathcal{F}_{t+}\) for all \(t \in T\).

In the discrete-time case, the right-continuity requirement on the filtration is superfluous.

Any filtered space can be minimally extended so that it satisfies the standard conditions. Namely,

\[ \mathcal{F}'_{t} := \mathcal{F} \vee \mathcal{N}_{\mathbb {P}}, \qquad \mathcal{F}''_t := \mathcal{F}'_{t+} \]

In doing so (getting ahead of ourselves) the Markov property and the martingale property of adapted processes are preserved.

2 The Markov property: the general case

2.1 Definition via filtration

Definition 6 A process \(\left(X_{t}, t \in T\right)\), taking values in a space \((\mathsf{S}, \mathscr {B})\), is called Markov with respect to a filtration \(\mathbb {F}=\left(\mathcal{F}_{t}, \in T \right)\), if \(X_{t}\) is adapted to \(\mathbb {F}\) and the Markov property holds: for all \(s \leqslant t\) (\(s, t \in T\)), for all \(B \in \mathscr {B}\) we have \[ \mathbb {P}\left(X_t \in B \mid \mathcal{F}_s\right) \; \stackrel{\text{a.s.}}{=} \; \mathbb {P}\left(X_t \in B \mid X_s\right) \] A Markov process without reference to a filtration means a Markov process with respect to its own natural filtration.

Up to now we have been considering the Markov property precisely with respect to the natural filtration of the process.

Theorem 1 (Equivalent formulations of the Markov property) Let \((X_t, \; t \in T)\) be a process adapted to the filtration \(\mathbb {F} = (\mathcal{F}_t, \; t \in T)\). The following conditions are equivalent:

1. \((X_t, \mathcal{F}_t)\) is Markov, i.e.

\[ \mathbb {P}\left(X_t \in B \mid \mathcal{F}_s\right) \; \stackrel{\text{a.s.}}{=} \; \mathbb {P}\left(X_t \in B \mid X_s\right) \]

for all \(s \leq t\), \(B \in \mathscr {B}\).

2. we have

\[ \mathbb {E}\left[f(X_t) \mid \mathcal{F}_s\right] \; \stackrel{\text{a.s.}}{=} \; \mathbb {E}\left[f(X_t) \mid X_s\right] \]

for all \(s \leq t\), for any \(f: \mathbb {R} \to \mathbb {R}\) — a bounded, \(\mathscr {B}\)-measurable function.

3. we have

\[ \mathbb {P}\left(C \mid \mathcal{F}_s\right) \; \stackrel{\text{a.s.}}{=} \; \mathbb {P}\left(C \mid X_s\right) \]

for all \(s\), for any \(C \in \mathcal{F}^{X}_{\geq s} := \sigma \left(X_{u} \; : \; u \geq s\right)\). \(C\) is an event from the “future” of the process.

This condition can be rephrased as follows: the future of the process \(\mathcal{F}^{X}_{\geq s}\) and the current knowledge \(\mathcal{F}_s\) are conditionally (given \(X_s\)) independent of one another.

4. we have

\[ \mathbb {E}\left[Y \mid \mathcal{F}_s\right] \; \stackrel{\text{a.s.}}{=} \; \mathbb {E}\left[Y \mid X_s\right] \]

for all \(s\), for any \(Y\) — a \(\mathcal{F}^X_{\geq s}\)-measurable bounded random variable.

If \(\mathbb {F}\) is the natural filtration, then the additional equivalent conditions are as follows:

1. it holds that

\[ \mathbb {P}\left(X_t \in B \mid X_s, X_{s_m}, X_{s_{m-1}}, \ldots , X_{s_1}\right) = \mathbb {P}\left(X_t \in B \mid X_s\right) \]

for all \(\forall m, \forall s_{1}<\cdots <s_{m} < s < t, \; \forall B \in \mathscr {B}\)

2. it holds that

\[ \mathbb {E}\left[f(X_t) \mid X_s, X_{s_m}, X_{s_{m-1}}, \ldots , X_{s_1}\right] = \mathbb {E}\left[f(X_t) \mid X_s\right] \]

for all \(\forall m, \forall s_{1}<\cdots <s_{m}<s<t, \forall f\) — a bounded Borel function

The meaning of the definition of a Markov process is that the distribution of the “future” \(\left(X_{t}\right)\) depends on the “past” \(\left(\mathcal{F}_{s}\right)\) only through the “present” \(X_{s}\).

2.2 Some examples

The first class of processes that falls entirely within the class of Markov processes consists of processes with independent increments. Let us state their definition in general form, relative to a filtration. A random process \((X_t, t \in T)\), adapted to a filtration \(\mathbb {F} = (\mathcal{F}_t, t \in T)\), is called a process with independent increments (with respect to the filtration \((\mathcal{F}_t, t \in T)\)), if

\[ X_{t} - X_{s} \quad \text{ and } \quad \mathcal{F}_s \quad \text{ are independent for any } s < t. \]

Earlier we stated the definition of processes with independent increments, essentially, for the natural filtration \(\mathbb {F}^{X} = (\mathcal{F}^X)\).

Theorem 2 Processes with independent increments (with respect to some filtration) are Markov (with respect to the same filtration).

This theorem immediately gives us several good examples of Markov processes:

• the Wiener process \(\left(W_{t}, t \geqslant 0\right)\);

• the Poisson process \(\left(N_{t}, t \geqslant 0\right)\);

• the random walk.

Unlike processes with independent increments, Gaussian processes are by no means always Markov. As we recall, the distribution of a Gaussian process is determined by its covariance function. It is natural to expect that the Markov property should be expressible as some condition on the covariance function. This condition is stated in the following theorem.

Theorem 3 Let \(\left(X_{t}, t \in T\right), T \subseteq \mathbb {R}\) be a Gaussian process with covariance function \(R(s, t)\). Then \(X_{t}\) is Markov if and only if for all \(s \leqslant u \leqslant t\), the equality holds \[ R(s, u) \cdot R(u, t) = R(s, t) \cdot R(u, u) \]

2.3 Transition functions of a Markov process

Extensive research into Markov processes is concerned with the study of transition functions.

Definition 7 Let \(T \subset \mathbb {R}\). A function \(P_{s,t}(x, B)\), where \(s, t \in T, s \leqslant t, x \in \mathbb {R}, B \in \mathcal{B}(\mathbb {R})\), is called a transition function, if

1. For fixed \(s, x, t\), \(\; P_{s,t}(x, \cdot )\) is a probability measure on \(\mathbb {R}\);

2. For fixed \(s, t, B\), the function \(P_{s,t}(\cdot , B)\) is Borel;

3. For \(t=s\), we have \(P_{s,s}(x, B) = \delta_{x}(B)= \; \mathbb {1}_{B}(x)\);

4. For \(s \leqslant u \leqslant t, s, u, t \in T, x \in \mathbb {R}, B \in \mathcal{B}(\mathbb {R})\), the Chapman-Kolmogorov equation holds:

   \[ P_{s,t}(x, B) =\int _{\mathbb {R}} P_{s,u}(x, dy) \cdot P_{u,t}(y, B) . \]

Definition 8 A Markov process \(\left(X_{t}, t \in T\right)\) has transition function \(P_{s,t}(x, B)\), if for any \(s < t\) and \(B \in \mathscr {B}\) we have \[ \mathbb {P}\left(X_{t} \in B \mid X_{s}=x\right) \; = \; P_{s,t}(x, B) \qquad \mathbb {P}_{X_{s}}\text{-almost surely with respect to } x \] (i.e. for almost every \(x\) with respect to the distribution \(\mathbb {P}_{X_{s}}\) on \(\mathbb {R}\)). That is, \(P_{s,t}(x,B)\) is the probability that a Markov process which at time \(s\) is at the point \(x\) will, by time \(t\), land in the set \(B\).

Definition 9 A Markov process with transition function \(P_{s,t}(x, B)\) is called homogeneous, if \[ P_{0,t}(x, B) = P_{u,t + u}(x, B) \qquad \text{ for all } t,u \] In this case one can introduce the notation \[ P_{t}(x, B) := P_{0,t}(x, B) \]

2.3.1 A few words on the necessity of a transition function in the case of an uncountable state space

Upon reading definition Definition 8, a question may arise: why can we not simply take, as \(P_{s,t}(x,B)\),

\[ \tilde{P}_{s,t}(x,B) := \mathbb {P}\left(X_{t} \in B \mid X_{s}=x\right) ? \]

Why introduce the additional notion of a transition function at all?

• Let, to begin with, \(s<t\) be fixed. For simplicity we may take \(T = \left\{ 0,1\right\}\), i.e. the process \(X = (X_n, n \in \left\{ 0,1\right\} )\) has only 2 time instants (note that any such process will be Markov), and \(s=0, t=1\). In essence we will be studying the bivariate distribution \((X_0, X_1)\). In the expression \(\mathbb {P}\left(X_{t} \in B \mid X_{s}=x\right)\) we will vary \(B \in \mathscr {B}\left(\mathbb {R}\right)\) and \(x \in \mathbb {R}\) and check whether properties 1, 2 of Def. Definition 7 hold.

  Recall how the conditional probability \(\mathbb {P}\left(X_{t} \in B \mid X_{s}=x\right)\) is defined in the general case, i.e. when \(\mathbb {P}\left(X_{s}=x\right) = 0\). This is done in 2 steps:

  1. For a fixed set \(B\), the random probability \(\mathbb {P}\left(X_{t} \in B \mid X_{s}\right)\) is defined, i.e. the random variable

     \[ \mathbb {P}\left(X_{t} \in B \mid X_{s}\right) = \mathbb {E}\left[\; \mathbb {1}_{B}(X_t) \mid X_{s}\right] = \mathbb {E}\left[\; \mathbb {1}_{B}(X_t) \mid \sigma (X_s)\right] \]

     via conditional expectation. It is defined up to almost-sure equivalence, like any conditional expectation;

  2. Next, one must recall that, by definition, the conditional expectation \(\mathbb {P}\left(X_{t} \in B \mid X_{s}\right)\) is a random variable measurable with respect to \(X_s\), i.e. with respect to the generated sigma-algebra \(\sigma (X_s)\). It is known that a random variable measurable with respect to \(X_s\) is simply some Borel function of \(X_s\). Consequently, in our case there exists a Borel function \(\varphi_B\) such that \(\mathbb {P}\left(X_{t} \in B \mid X_{s}\right) = \varphi_B(X_s)\). In this case one sets

     \[ \mathbb {P}\left(X_{t} \in B \mid X_{s} = x\right) := \varphi _B(x) \]

  From this definition, for a fixed \(B \in \mathscr {B}\left(\mathbb {R}\right)\) the function \(\mathbb {P}\left(X_{t} \in B \mid X_{s} = \cdot \right): \mathbb {R} \to [0,1]\) is Borel, i.e. property 2 of Def. Definition 7 holds. But does property 1 hold, i.e. is the function \(\mathbb {P}\left(X_{t} \in \cdot \mid X_{s} = x\right)\) a probability measure for all \(x \in \mathbb {R}\)?

  Note that the conditional probability can always be chosen so that it is nonnegative for every elementary outcome \(\omega \in \Omega\). Consequently, we may assume that for all \(x \in \mathbb {R}\) the set function \(\mathbb {P}\left(X_t \in \cdot \mid X_s = x\right)\) is nonnegative.

  It remains to check countable additivity. Let \(B_1, B_2, \ldots \in \mathscr {B}\left(\mathbb {R}\right)\) be disjoint sets. Then we should have

  \[ \mathbb {P}\left(X_t \in \bigsqcup _{i=1}^\infty B_i \mid X_s = x\right) = \sum _{i=1}^\infty \mathbb {P}\left(X_t \in B_i \mid X_s = x\right), \]

  for all \(x \in \mathbb {R}\). At the same time we have

  \[ \begin{split} \mathbb {P}\left(X_t \in \bigsqcup _{i=1}^\infty B_i\mid X_s\right) & = \mathbb {E}\left[\sum _{i=1}^\infty \; \mathbb {1}_{X_t \in B_i}\mid X_s\right] \stackrel{\text{a.s.}}{=} \\ & \stackrel{\text{a.s.}}{=} \sum _{i=1}^\infty \mathbb {E}\left[\; \mathbb {1}_{X_t \in B_i} \mid X_s\right] = \\ & =\sum _{i=1}^\infty \mathbb {P}\left(X_t \in B_i \mid X_s\right) \end{split} \tag{1}\]

  This equality holds almost surely, as does almost every equality involving conditional expectations. Denote by \(\mathcal{N}(B_1,B_2,\ldots ) \subset \Omega\) the set of those elementary outcomes where equality Equation 1 fails. Note that it has measure zero, but it depends on \(B_1,B_2,\ldots\). That is, to obtain the entire set of elementary outcomes \(\mathcal{N}\) where countable additivity of the set function \(\mathbb {P}\left(X_t \in \cdot \mid X_s\right)(\omega )\) is violated, one must take the union of \(\mathcal{N}(B_1,B_2,\ldots )\) over all disjoint \(B_1, B_2,\ldots \in \mathscr {B}\left(\mathbb {R}\right)\):

  \[ \begin{align} \mathcal{N} & = \bigcup _{\substack {B_1, B_2,\ldots \in \mathscr {B}\left(\mathbb {R}\right) \\ B_i \cap B_j = \varnothing , \; i \neq j}} \\ & \quad \mathcal{N}(B_1,B_2,\ldots ) \end{align} \]

  Even though every set in the union has measure zero, the union is taken over an uncountable collection. Consequently, it may have nonzero measure¹. It could hypothetically happen that \(\mathcal{N} = \Omega\), i.e. countable additivity of \(\mathbb {P}\left(X_t \in \cdot \mid X_s = x\right)\) fails for absolutely every \(x \in X_s(\Omega )\).

  Nevertheless, it turns out that if \(X_t, X_s\) take values in Borel spaces, then for each individual \(B \in \mathscr {B}\left(\mathbb {R}\right)\) one can choose a version of \(\mathbb {P}\left(X_t \in B \mid X_s \right)\) such that, in the end, for every \(x \in \mathbb {R}\) the set function \(\mathbb {P}\left(X_t \in \cdot \mid X_s = x\right)\) is a probability measure. For more detail see Markov kernels.

• Now suppose that for each \(s < t\) we have chosen \(\tilde{P}_{s,t}(\cdot , \cdot ) = \mathbb {P}\left(X_t \in \cdot \mid X_s \in \cdot \right)\) so that it satisfies properties 1-2 of Def. Definition 7. But even in this case problems can arise:

  For \(s < u < t\) denote by \(\mathcal{N}(s,u,t) \subset \mathbb {R}\) the set of those \(x\) where the Chapman-Kolmogorov equations fail. To obtain the set \(\mathcal{N}\) of all \(x\) where the C-K equations fail, one must take the union of \(\mathcal{N}(s,u,t)\) over all \(s < u < t\):

  \[ \mathcal{N} = \bigcup _{\substack {s,u,t \in T \\ s < u < t } }\mathcal{N}(s,u,t) \]

  If \(T\) is a continuum (for example, \(T = \mathbb {R}_+\)), then once again we have a continuum union of sets of measure zero. That is, it could hypothetically happen that \(\mathcal{N} = \mathbb {R}\), and the C-K equations would fail to hold for any \(x \in \mathbb {R}\).

ProblemProblem 1

Prove that the function \[ \tilde{P}_{s,t}(x,B) := \mathbb {P}\left(X_{t} \in B \mid X_{s}=x\right) \] of an arbitrary Markov process \(X_{t}\) satisfies the Chapman-Kolmogorov equations not for all \(x \in \mathbb {R}\), but only for almost every \(x\) with respect to the measure \(\mathbf{P}_{X_{s}}\).

Thus, the point of definition Definition 8 is that a Markov process with a transition function has a “good” version of the conditional distribution appearing on the left-hand side of the equality.

The question of whether a Markov process has a transition function is very difficult and has not been fully resolved. Problem MarkovProcesses:WeakTransFunc means that for arbitrary Markov processes only a weakened version of property 4 holds, valid not for all \(x\), but only for almost all \(x\) with respect to the measure \(\mathbb {P}_{X_{s}}\).

In conclusion, let us give the definition of the transition density.

Definition 10 If the transition function \(P_{s,t}(x, B)\) of a Markov process \(X_{t}\) has the form \[ P_{s,t}(x, B)=\int _{B} p_{s,t}(x,y) d y \] then \(p_{s,t}(x,y)\) is called the transition density of the process \((X_{t})\).

ExampleExample 2

Is Brownian motion a Markov process? If it is — try to construct a transition function for it. If you succeed — determine whether this transition function has a transition density.

SolutionSolution

\(R(s,t) = \min (s,t)\). By the criterion Theorem 3 for the Markov property of Gaussian processes (and Brownian motion is a Gaussian process), we need to check the equality \(R(s,u) \cdot R(u,t) = R(s,t) \cdot R(u,u)\) for \(s \leq u \leq t\). We have

\[ R(s,u) \cdot R(u,t) = s \cdot u \]

On the other hand

\[ R(s,t) \cdot R(u,u) = s \cdot u \]

So Brownian motion is a Markov process.

Transition probability:

\[ \begin{align} \mathbb {P}\left(W_{t} \in B \mid W_{s} = x \right) & \stackrel{\text{a.s.}}{=} \mathbb {P}\left(W_{t} - W_{s} \in B - x \mid W_{s} = x \right) \\ & \stackrel{\text{a.s.}}{=} \mathbb {P}\left(W_t - W_{s} \in B - x\right) = \\ & = \mathbb {P}\left(\underbrace{W_t - W_{s} + x}_{\sim \mathscr {N}\left(x, t-s\right)} \in B \right) \\ & = \int _{B} \frac{1}{\sqrt{2 \pi (t-s)}}\exp \left(-\frac{1}{2(t-s)}\left(y-x\right)^2\right) \; dy \end{align} \]

So, as a transition probability we may try \(P_{s,t}(x, \cdot ) := \mathscr {N}\left(x, t-s\right)\), i.e.

\[ \begin{align} P_{s,t}(x,B) & := \int _{B} p_{s,t}(x,y) \; dy \qquad \text{ where } \\ & \qquad p_{s,t}(x,y) := \frac{1}{\sqrt{2 \pi (t-s)}}\exp \left(-\frac{1}{2(t-s)}\left(y-x\right)^2\right) \end{align} \]

Let us check that the properties of a transition probability hold.

Enum-item1.

For fixed \(s, x, t\), \(\; P_{s,t}(x, \cdot )\) is a probability measure on \(\mathbb {R}\)? Given how it is defined, \(P_{s,t}(x, \cdot ) = \mathscr {N}\left(x, t-s\right)\), which is indeed a probability measure.

Enum-item2.

For fixed \(s, t, B\), is the function \(P_{s,t}(\cdot , B)\) Borel? For fixed \(s,t\), the function \(p_{s,t}: \mathbb {R}^{2} \to \mathbb {R}\) is Borel. By the properties of the Lebesgue integral with respect to a product measure, measurability in the second argument is preserved when integrating with respect to one of the arguments.

Enum-item3.

For \(t=s\), does \(P_{s,s}(x, B) = \delta_{x}(B)= \; \mathbb {1}_{x \in B}\) hold? It holds, since \(P_{s,s}(x, B) = \mathscr {N}\left(x, 0\right) = \delta_0\).

Enum-item4.

For \(s \leqslant u \leqslant t, s, u, t \in T, x \in \mathbb {R}, B \in \mathcal{B}(\mathbb {R})\), does the Chapman-Kolmogorov equation hold? Let’s check. \(\int_{\mathbb {R}} P_{s,u}(x, dy) \cdot P_{u,t}(y, \cdot )\) is simply the convolution of the normal distributions \(\mathscr {N}\left(x, u-s\right)\) and \(\mathscr {N}\left(0, t-u\right)\), i.e. the distribution of the sum of independent random variables with these distributions. Since the increments of the Wiener process are independent, this is the distribution of the sum \((W_u - W_s + x) + (W_{t} - W_{u})\). It equals \(W_{t} - W_{s} + x\) and has distribution \(\mathscr {N}\left(x, t-s\right)\), i.e. exactly \(P_{s,t}(x, \cdot )\).

Thus \(P_{s,t}\) (as defined above) are indeed the transition probabilities for Brownian motion. The transition densities are given above.

AnswerAnswer

Yes.

ProblemProblem 2

Is the Brownian bridge a Markov process? If it is — try to construct a transition function for it. If you succeed — determine whether this transition function has a transition density.

3 Stopping times

From the point of view of applications, it is useful to analyze the behavior of a random process at random time instants. In this section we consider the most important such class of random times, called stopping times. To do this, we need to introduce a number of definitions.

3.1 Definitions

Definition 11  

• A mapping

  \[ \tau : \Omega \to T \cup \{ +\infty \} \]

  that is \(\mathcal{F} \mid \mathscr {B}\left(T \cup \{ +\infty \} \right)\)-measurable is called a random time.

• A mapping \(\tau : \Omega \rightarrow T \cup \{ +\infty \}\) is called a stopping time (Markov time) with respect to the filtration \(\mathbb {F}=\left(\mathcal{F}_{t}, t \in T\right)\) on \((\Omega , \mathcal{F}, \mathbb {F}, \mathbb {P})\), if at every moment in time \(t\) one can determine, from the information contained in the filtration, whether \(\tau\) has already occurred or not. Formally: for any \(t \in T\) we have

  \[ \{ \tau \leqslant t\} \in \mathcal{F}_{t}. \]

• If \(\tau\) is a stopping time and \(\mathbb {P}\left(\tau < +\infty \right) = 1\) holds, i.e. \(\tau\) is almost surely finite, then \(\tau\) is called an almost surely finite stopping time² with respect to \(\mathbb {F}\).

Let us discuss the informal meaning of this definition. Let \(\tau\) be the random time at which some event occurs in the process \(\left(X_{t}, t \in T\right)\). Then \(\tau\) will be a stopping time with respect to the natural filtration of the process \(X_{t}\), if for any \(t_{0} \in T\) one can say unambiguously whether \(\tau\) has already occurred by time \(t_{0}\) or not (that is, whether \(\tau \leqslant t_{0}\) or \(\tau > t_{0}\)?), knowing only the values of the process \(X_{t}\) up to and including time \(t_{0}\).

ProblemProblem 3

Prove that any stopping time is a random time.

ProblemProblem 4

Discrete case. Let \(T = \mathbb {N}\), let a filtration \(\mathbb {F}=\left(\mathcal{F}_{n}, n \in \mathbb {N}\right)\) be given, and let \(\tau_{1}, \tau_{2}, \ldots\) be stopping times with respect to \(\mathbb {F}\). Prove that the random variables \[ \sum _{k=1}^{m} \tau _{k}, \quad \prod _{k=1}^{m} \tau _{k}, \quad \sup _{k} \tau _{k}, \quad \inf _{k} \tau _{k} \] are also stopping times with respect to \(\mathbb {F}\).

ProblemProblem 5

Continuous case. Let \(T = \mathbb {R}_+\), let a filtration \(\mathbb {F}=\left(\mathcal{F}_{t}, t \geq 0\right)\) be given, and let \(\tau_{1}, \tau_{2}, \ldots , \tau_m\) be stopping times with respect to \(\mathbb {F}\). Prove that \[ \sum _{k=1}^{m} \tau _{k}, \qquad \max _{k=1, \ldots , m} \tau _{k}, \qquad \min _{k=1, \ldots , m} \tau _{k} \] are also stopping times with respect to \(\mathbb {F}\).

Where might difficulties arise in proving that these are stopping times, if there had originally been infinitely many \(\tau_1, \tau_2, \ldots\)?

3.2 The first hitting time

Definition 12 Let \(X = (X_t, t \in T)\) be a random process defined on \((\Omega , \mathcal{F}, \mathbb {P})\), taking values in \((\mathsf{S}, \mathscr {B})\). Let \(B \in \mathscr {B}\) be an arbitrary measurable subset of \(\mathsf{S}\). Then the mapping \[ \begin{align} \tau _B := \inf \left\{ t \in T \; : \; X_t \in B\right\} \end{align} \] from \(\Omega\) to \(T \cup \left\{ +\infty \right\}\)³ is called the (first) hitting time of the process \((X_t)\) in \(B\).

It turns out that for most random processes encountered in practice, the hitting time of a measurable set is a stopping time. A bit later we will state the precise conditions under which this holds (and partially prove them), but for now let us consider a few practical examples. Suppose 2 teams play a football match, and let \(X_t\) be the total number of goals scored by time \(t\) min., \(t \in [0,90]\) (for simplicity we assume the referee adds no stoppage time). Let \(B = [1, +\infty )\). Then \(\tau_B = \inf \left\{ t \in T: X_t(\omega ) \geq 1\right\}\) is the time of the first goal scored in the match. It is intuitively clear that, having, at any moment \(t_0\), only information about the goals scored by time \(t_0\) (i.e. \(\mathcal{F}^X_{t_0}\)), one can determine whether the first goal of the match has already been scored (by time \(t_0\)) (i.e. \(\tau_B \leq t_0\)) or not. That is, \(\left\{ \tau_B \leq t_0\right\} \in \mathcal{F}^X_{t_0}\).

In contrast, if we consider the random time \(\tau '_B := \inf \left\{ t \in T: X_{t+1}(\omega ) \geq 1\right\}\), i.e. the moment one minute before the first goal is scored, then, knowing only \(\mathcal{F}^X_t\) by time \(t\), we cannot always say for certain whether \(\tau '_B\) has already occurred or not, since we do not know the future. That is, in general \(\tau '_B\) is not a stopping time.

Another example of a random time that is not Markov is the time of the last goal scored: \(\tau '' := \inf \left\{ t \in T: X_t = X_{90}\right\}\) (for definiteness we assume that at least one goal was scored in the match, otherwise \(\tau '' = 0\)). To say whether \(\tau ''\) has occurred by time \(t_0\), one needs to know the value of \(X_{90}\), i.e. the future, which we do not know at an arbitrary time \(t_0 \in (0,90)\).

Theorem 4 (The début theorem, the theorem on the first hitting time⁴) Let \(X = \left(X_{t}, t \in T\right)\) be a random process taking values in \((\mathsf{S}, \mathscr {B})\), defined on \((\Omega , \mathcal{F}, \mathbb {P})\) and adapted to the filtration \(\mathbb {F} = (\mathcal{F}_t, t \in T)\). Let \(B \in \mathscr {B}\) be an arbitrary measurable set.

If time is discrete (\(T = \mathbb {N}\) or \(T = \mathbb {Z}_+\)), then the hitting time \(\tau_B\) is always (i.e. for any measurable \(B\)) a stopping time with respect to \(\mathbb {F}\).

Now let \(T = \mathbb {R}_+\), i.e. time is continuous, and suppose additionally that \(\mathsf{S}\) is a metric space with metric \(\rho\) (for example, \(\mathbb {R}^d\)), and \(\mathscr {B}\) is its Borel sigma-algebra. Then \(\tau_B\) will be a stopping time with respect to \(\mathbb {F}\), if any of the following conditions holds:

1. \(B\) is closed, \((X_t)\) has all continuous sample paths (if the filtration were complete, it would suffice to require that only almost all sample paths be continuous);

2. \(B\) is open, the filtration is right-continuous, \((X_t)\) has all right-continuous sample paths (if the filtration were complete, it would suffice to require that only almost all sample paths be right-continuous);

3. \(B\) is an arbitrary Borel set, the filtered space satisfies the standard conditions (i.e. the filtration is complete and right-continuous), and \((X_t)\) has almost surely all right-continuous sample paths.

We will prove items (i) and (ii) in the problems below. Item (iii) is the general début theorem (P. A. Meyer): under the standard conditions, the Markov property of \(\tau_B\) is preserved for an arbitrary Borel \(B\). Its proof relies on the measurable projection theorem and lies far beyond the scope of our course⁵. The general début theorem is one of the main reasons why filtered spaces are standardly extended: for example, for Brownian motion with the completed, right-continuous natural filtration, the hitting time of any Borel \(B \subset \mathbb {R}\) turns out to be a stopping time.

Mn-remarkRemark

Two useful additions to the theorem.

• Right-continuity of the filtration in item (ii) is needed only at the very end of the proof: without it, for open \(B\) and right-continuous sample paths, it is still true that \(\left\{ \tau_B < t\right\} \in \mathcal{F}_t\) for all \(t\), i.e. \(\tau_B\) is a stopping time with respect to the filtration \(\mathbb {F}_+ := (\mathcal{F}_{t+}, \; t \in T)\).

• For closed \(B\) and a process all of whose sample paths are right-continuous with left limits (càdlàg), one can prove by elementary means the Markov property not of \(\tau_B\) itself, but of the time

  \[ \widehat{\tau }_B := \inf \left\{ t \in T \; : \; X_t \in B \text{ or } X_{t-} \in B\right\} . \]

As we can see, for \(\tau_B\) to be a stopping time, in the continuous case one must impose some restrictions

• on the set \(B\),

• on the sample paths of the process \(X_t\),

• on the filtration \(\mathbb {F}\).

Without this, pathological cases arise.

ExampleExample 3

Give an example of a process with continuous sample paths and an open set \(B\), for which the hitting time \(\tau_B\) is not a stopping time with respect to the natural filtration of the process.

SolutionSolution

Idea: two elementary outcomes, two sample paths that coincide up to time \(t = 1\) and diverge afterward.

Let \(\Omega = \left\{ \omega_1, \omega_2\right\}\), \(\mathcal{F} = 2^{\Omega } = \left\{ \varnothing , \Omega , \left\{ \omega_1\right\} , \left\{ \omega_2\right\} \right\}\), let \(\mathbb {P}\left(\omega_i\right) = \frac{1}{2}\). Let \(T = \mathbb {R}_+\), let

\[ X_t(\omega _1 ) = t, \qquad X_{t}(\omega _2) = \begin{cases} t, & t \leq 1 \\ 2 - t, & t > 1 \end{cases} \]

Note that the natural filtration is:

\[ \mathcal{F}^{X}_t = \begin{cases} \left\{ \varnothing , \Omega \right\} , & t \leq 1 \\ \left\{ \varnothing , \Omega , \left\{ \omega _1\right\} , \left\{ \omega _2\right\} \right\} = \mathcal{F}, & t > 1 \end{cases} \]

Let \(B = (1,2)\). Then \(\tau_{B}(\omega_1) = 1\) (note: the infimum is not attained, since \(X_1(\omega_1) = 1 \notin B\)), while \(\tau_{B}(\omega_2) = +\infty\), since the second sample path never exceeds \(1\). But at the same time

\[ \underbrace{\left\{ \tau _{B} \leq 1\right\} }_{=\left\{ \omega _1\right\} } \not\in \mathcal{F}^{X}_{1} = \left\{ \varnothing , \Omega \right\} , \]

i.e. \(\tau_B\) is not a stopping time with respect to \(\mathbb {F}^X\).

Let us relate this example to the début theorem: the sample paths are continuous, but \(B\) is not closed — item (i) does not apply. The filtration \(\mathbb {F}^X\) here is even complete (there are no nonempty negligible sets at all), but it is not right-continuous: \(\mathcal{F}^{X}_{1+} = \mathcal{F} \neq \mathcal{F}^{X}_{1}\) — so items (ii), (iii) do not apply either. At the same time \(\left\{ \tau_B < t\right\} \in \mathcal{F}^{X}_t\) for all \(t\) (check this!), i.e. \(\tau_B\) is a stopping time with respect to \(\mathbb {F}^X_+ = (\mathcal{F}^{X}_{t+}, \; t \geq 0)\).

ProblemProblem 6

Prove the début theorem in the case \(T = \mathbb {N}\).

ProblemProblem 7

Prove the début theorem in the continuous case (items (i) and (ii)).

HintHint

Start with item (ii): describe the event \(\left\{ \tau_B < t\right\}\) in terms of the values of the process at rational times. In item (i) consider \(\inf_{s \in [0,t]} \rho (X_s, B)\) and the open \(\frac{1}{n}\)-neighborhoods of the set \(B\).

3.3 Events , that have occurred by the stopping time

With a stopping time one can associate a sigma-algebra of events “having occurred by time \(\tau\)”. Let us give a formal definition.

Definition 13 Let \(\tau\) be a stopping time with respect to \(\mathbb {F}=\left(\mathcal{F}_{t}, t \in T\right)\). Then the sigma-algebra \(\mathcal{F}_{\tau }\) is defined as \[ \mathcal{F}_{\tau }:=\left\{ A \in \mathcal{F} \; : \; \{ \tau \leqslant t\} \cap A \in \mathcal{F}_{t} \text{ for all } t\in T \right\} . \]

\(\mathcal{F}_{\tau }\) contains all possible events (from the overall sigma-algebra \(\mathcal{F}\)) such that at every time \(t\) one can determine whether this event has occurred, given that the stopping time \(\tau\) has already occurred.

Theorem 5 (Basic properties of \(\mathcal{F}_{\tau }\)) Let \(\tau\) be a stopping time.

1. \(\mathcal{F}_{\tau }\) is indeed a \(\sigma\)-algebra, and moreover \(\mathcal{F}_{\tau } \subset \mathcal{F}\).

2. The random variable \(\tau\) (which, in general, is extended-valued, since it may take the value \(+\infty )\) is measurable with respect to \(\mathcal{F}_{\tau }\).

3. If \(\tau =t= \operatorname {const}\), then \(\mathcal{F}_{\tau }=\mathcal{F}_{t}\).

4. If \(\tau\) is an almost surely finite stopping time, then \(\mathcal{F}_{\tau }\) is the smallest sigma-algebra generated by adapted càdlàg processes, sampled at \(\tau\):

   \[ \mathcal{F}_{\tau } = \sigma \left\{ X_{\tau } \; : \; (X_{t}, t \in T) \text{ -- is an adapted càdlàg process}\right\} \]

5. If \(\tau_1 \stackrel{\text{a.s.}}{\leq } \tau_2\), then \(\mathcal{F}_{\tau_1} \subset \mathcal{F}_{\tau_2}\);

6. \(\mathcal{F}_{\tau_1} \cap \mathcal{F}_{\tau_2} = \mathcal{F}_{\tau_1 \wedge \tau_2}\)

ProblemProblem 8

Prove property 2: \(\tau\) is an \(\mathcal{F}_{\tau }|\mathscr {B}\left(T \cup \left\{ +\infty \right\} \right)\)-measurable random variable.

ProblemProblem 9

Let a filtration \(\mathbb {F}=\left(\mathcal{F}_{t}, t \in T\right), T \subset \mathbb {R}\), be given on \((\Omega , \mathcal{F}, \mathbb {P})\). Prove that for stopping times \(\sigma , \tau\) with respect to \(\mathbb {F}\) and for any \(A \in \mathcal{F}_{\tau }\) we have \[ A \cap \{ \tau \leq \sigma \} \in \mathcal{F}_{\tau } \cap \mathcal{F}_{\sigma } \]

As a consequence we obtain that if \(\tau \leq\) \(\sigma\), then \(\mathcal{F}_{\tau } \subset \mathcal{F}_{\sigma }\).

ProblemProblem 10

Let \(\tau\) be a stopping time with respect to the filtration \(\mathbb {F}=\left(\mathcal{F}_{t}, t \in T\right)\), and let the random process \(\left(X_{t}, t \in T\right)\) be adapted to \(\mathbb {F}\). Prove that \(X_{\tau }\) is \(\mathcal{F}_{\tau }\)-measurable (we take \(X_{\tau }=+\infty\), if \(\tau =+\infty\)), if

1. \(T=\mathbb {N}\);

2. \(T=\mathbb {R}_{+}\) and the sample paths of \(\left(X_{t}, t \in T\right)\) are right-continuous.

4 Lévy processes and applications

4.1 Lévy processes: continuity in probability and independent stationary increments

Let us turn to applications of stopping times to the Wiener process. First we present a simple statement (which was already partly included in the independent work for the previous chapter): if we stop the Wiener process at any fixed time and restart the clock and the position from scratch, then the new process will again be a Wiener process.

Lemma 1 (An equivalent Markov property for Brownian motion) Let \(\left(W_{t}, t \geqslant 0\right)\) be a Brownian motion. Then for all \(a>0\) the process \[ X_{t} := W_{t+a}-W_{a}, \qquad t \geqslant 0, \] is also a Brownian motion and is independent of the sigma-algebra \(\mathcal{F}_{a}^{W}\) (i.e. \(\sigma \left(X_{t}, t \geqslant 0\right)\) is independent of \(\mathcal{F}_{a}^{W}\)).

A natural question: can \(a\) be replaced by a random variable? It turns out that the statement remains true if \(a\) is replaced by a stopping time with respect to the natural filtration of the Wiener process. Moreover, a similar fact holds for an entire class of processes called Lévy processes.

Lévy processes, which include the Poisson process and Brownian motion as special cases, were the first class of stochastic processes to be studied in the modern sense (by the French mathematician Paul Lévy). They still serve as prototypical examples for Markov processes, as well as for semimartingales.

We assume the presence of a filtered probability space \((\Omega , \mathcal{F}, \mathbb {F}, \mathbb {P})\) satisfying the standard conditions.

Definition 14 A process \(\left(X_{t}, t \geqslant 0\right)\), such that \(X_0 \stackrel{\text{a.s.}}{=} 0\), adapted to the filtration \(\mathbb {F} = \left(\mathcal{F}_t\right)_{t \geq 0}\), is called a Lévy process, if

1. it has independent increments: \(X_t - X_s\) is independent of \(\mathcal{F}_t\), \(\forall 0 \leq s < t < \infty\);

2. it has stationary increments: for any \(t, s, h \in \mathbb {R}_{+}\) we have

   \[ \operatorname {Law}\left(X_{t}-X_{s}\right) = \operatorname {Law}\left(X_{t+h}-X_{s+h}\right). \]

3. \(X_t\) is continuous in probability: for all \(t \geq 0\) we have

   \[ X_s \xrightarrow [s \to t]{\mathbb {P}} X_t \]

Definition 15 A process \(\left(X_{t}, t \geqslant 0\right)\), such that \(X_0 \stackrel{\text{a.s.}}{=} 0\), is called an intrinsic Lévy process, if it is a Lévy process with respect to its own minimal (completed) filtration.

The following theorem shows that a Lévy process “renews itself” at stopping times.

Theorem 6 (The strong Markov property of Lévy processes) Let \(X = (X_t, t \geq 0)\) be a Lévy process with almost surely right-continuous sample paths, and let \(\tau\) be a stopping time with respect to \(\mathbb {F}^X\) (\(\tau < \infty\) a.s.). On the set \(\tau < \infty\) define the process \(Y = (Y_t, t \geq 0)\) by \(Y_t = X_{\tau + t} - X_{\tau }, \; t \geq 0\). Then

1. \(Y\) is a Lévy process, adapted to the filtration \(\left\{ \mathcal{F}_{\tau + t}, \; t \geq 0\right\}\);

2. \(Y\) is independent of \(\mathcal{F}_{\tau }\);

3. The processes \(X = (X_t, \; t \geq 0)\) and \(Y = (Y_{t}, \; t \geq 0)\) have the same distribution in the path space \(\left(\mathbb {R}^{\mathbb {R}_+}, \mathscr {B}\left(\mathbb {R}\right)^{\mathbb {R}_+}\right)\).

Since standard Brownian motion is a Lévy process, the theorem in particular confirms the strong Markov property of Brownian motion. This allows us to establish a rather beautiful result for Brownian motion, known as the reflection principle.

4.2 The reflection principle for the Wiener process

Theorem 7 (The reflection principle for the Wiener process) Let \(\left(W_{t}, t \geqslant 0\right)\) be a Wiener process, and let \(\tau\) be a stopping time with respect to \(\mathbb {F}^{W}\). Then the process \(\left(W^{\tau }_{t}, t \geqslant 0\right)\), where \[ W^{\tau }_{t} := \begin{cases} W_{t}, & t \leqslant \tau \\ 2 W_{\tau }-W_{t}, & t>\tau \end{cases} \] is a Wiener process, independent of \(\mathcal{F}^{W}_{\tau }\).

The black curve is the trajectory of \(W_t\). The dashed red curve is the trajectory of the reflected process \(W^{\tau}_t\) for the stopping time \(\tau = \tau_{[a, +\infty)}\) — the time at which the original process \(W_t\) first reaches level \(a\).

ProblemProblem 11

Prove the reflection principle.

Which stopping times would it be interesting to consider in connection with the Wiener process? We will consider the first time the Wiener process reaches a level \(x \in \mathbb {R}\):

\[ \tau _{x} := \inf \left\{ t \geq 0 \; : \; W_t = x\right\} \]

ProblemProblem 12

Prove that \(\tau_{x}\) is an almost surely finite stopping time (i.e. an almost surely finite stopping time) with respect to the natural filtration of the Wiener process.

4.3 Distributions related to the maximum of the Wiener process on an interval

Using the random variable \(\tau_{x}\), we can find the distribution of the maximum value of the Wiener process on the interval \([0, t]\). Denote \(M_{t} := \max_{s \leqslant t} W_{s}\). Note the simple relation between \(M_{t}\) and \(\tau_{x}\): for any \(x>0\)

\[ \left\{ M_{t} \geqslant x\right\} =\left\{ \tau _{x} \leqslant t\right\} . \tag{2}\]

As a consequence we obtain that the process \(M_{t}\) is adapted to the natural filtration of the Wiener process.

Since \(W\) is a continuous process, \(M_t=\sup_{0 \leq u \leq t, u \in \mathbb {Q}} W_u\), where \(\mathbb {Q}\) denotes the rational numbers; consequently, \(M_t\) is an adapted process (with respect to the natural filtration of \(W_t\)) with nondecreasing sample paths.

A simulated (not authentic) trajectory of \(W_t\) and its running maximum \(M_t = \max_{s \leq t} W_s\). Drag the level slider \(x\), click “New trajectory” to regenerate the path, or “Animate” to watch \(M_t\) “catch up” to level \(x\) exactly at time \(\tau_x\) — this is precisely the identity \(\{M_t \geq x\} = \{\tau_x \leq t\}\).

Level x = 0.80

Using the reflection principle, we can find the joint distribution of \(M_{t}\) and \(W_{t}\). Suppose \(W_t\) is a Brownian motion, and for some sample path Brownian motion rises above the level \(y\) before time \(t\), but at time \(t\) the value \(W_t\) is less than \(y-x\), where \(x > 0\). We can take the graph of this path and reflect it about the horizontal line at level \(y\), starting from the first time the path crosses level \(y\). This gives us a new path that ends above \(y + x\). Thus, there is a one-to-one correspondence between paths where the maximum up to time \(t\) is above \(y\) and \(W_t\) is below \(y-x\), and paths where \(W_t\) is above \(y + x\). We have

Lemma 2 For any \(x, y, t \geqslant 0\) we have: \[ \mathbb {P}\left(W_{t}< y-x, M_{t} \geqslant y\right) = \mathbb {P}\left(W_{t} > y+x\right) \]

The reflection principle for \(W_t\): a simulated (not authentic) trajectory that hits level \(y\), together with its reflection \(2y - W_t\) after time \(\tau_y\). Drag the sliders \(y\) and \(x\), click “New trajectory” to regenerate the path, or “Animate” to watch the tail of the trajectory smoothly “pivot” around level \(y\), illustrating the identity \(\mathbb{P}\left(W_t < y-x,\, M_t \geq y\right) = \mathbb{P}\left(W_t > y+x\right)\).

y = 0.50 x = 0.35

ExampleExample 4

Prove the previous lemma.

SolutionSolution

\(x, y, t \geqslant 0\) are fixed. We have \[ \begin{align} \mathbb {P}\left(W_t < y - x, M_t \geq y\right) = \mathbb {P}\left(W_t < y - x, \tau _y \leq t\right) \enclose {circle}{=} \end{align} \] Let \(Z_t\) be the process \(W_t\), reflected at time \(\tau_y = \inf \left\{ t \geq 0: W_t = y\right\}\). Note that if \(t \geq \tau_y\), as in the probability above, then \(Z_t = 2W_{\tau_y} - W_t = 2y - W_t\), i.e. \(W_t = 2y - Z_t\). Consequently, \[ \enclose {circle}{=} \; \mathbb {P}\left(2y - Z_t < y - x, \tau _y \leq t\right) = \mathbb {P}\left(Z_t > y + x, \tau _y \leq t\right) \enclose {circle}{=} \] Next, note that if \(Z_t > y + x\), then the time of reflection \(\tau_y\) has already occurred (\(\tau_y \leq t\)), since \(\tau_y > t \; \Rightarrow \; Z_t = X_t, \; X_t < y \; \Rightarrow \; Z_t < y\). That is, \(\left\{ Z_t > y + x\right\} \subset \left\{ \tau_y \leq t\right\}\) and \[ \enclose {circle}{=} \; \mathbb {P}\left(Z_t > y + x\right) = \mathbb {P}\left(X_t > y + x\right) \] The last equality is obtained since the processes \(Z\) and \(X\) have the same distribution.

The joint distribution of \((W_1, M_1)\): a cloud of \(2200\) simulated pairs. The support is the region \(\{(w,m) : m \geq \max(w,0)\}\) (boundary shown in red); no points fall exactly on the boundary itself, since \(\mathbb{P}\left(M_1 = W_1\right) = \mathbb{P}\left(M_1 = 0\right) = 0\). Among the points, \(18\) pairs are highlighted in orange for which the full trajectory was retained — hover over (or click) such a point to see it on the right.

Hover over a highlighted (orange) point on the left to see its trajectory

From this lemma we can find the distribution of the random variable \(M_{t}=\max_{s \in [0, t]} W_{s}\). Clearly \(\mathbb {P}\left(M_t \geq 0\right) = 1\). Next, let \(y \geq 0\). We have

\[ \begin{align} \mathbb {P}\left(M_t \geq y\right) & = \mathbb {P}\left(M_t \geq y; W_t < y\right) + \mathbb {P}\left(M_t \geq y; W_t \geq y\right) = \\ & = \mathbb {P}\left(W_t > y\right) + \mathbb {P}\left(W_t \geq y\right) = \\ & = 2\mathbb {P}\left(W_t \geq y\right), \end{align} \]

where the previous lemma is used to obtain the 2nd equality, together with the observation that \(W_t \geq y \; \Rightarrow \; M_t \geq y\), i.e. \(\left\{ M_t \geq y; W_t \geq y\right\} = \left\{ W_t \geq y\right\}\). Thus we have

Theorem 8 (Bachelier) We have \(M_t \stackrel{d}{=} \left|W_t\right|\).

Definition 16 If \(X \sim \mathscr {N}\left(0, \sigma^2\right)\), then the distribution of the random variable \(\left|X\right|\) is called the half-normal distribution with parameter \(\sigma^2\). It has the following density: \[ f(x) = 2 \cdot \frac{1}{\sqrt{2\pi }\sigma }e^{-\frac{x^2}{2\sigma ^2}} \; \mathbb {1}_{[0, +\infty )}(x) \]

Bachelier’s theorem can be restated as follows: the maximum of the Wiener process \(W_t\) on the interval \([0,1]\) (i.e. \(M_t\)) has a half-normal distribution with parameter \(t\).

Bachelier’s theorem, empirically: a live histogram of the values of \(M_1\) and \(\left|W_1\right|\), accumulating in batches as they are generated, plotted against the density of the half-normal distribution. The slider \(t\) changes the scale parameter of the distribution, the “number of simulations” slider sets the target sample size, and the “Restart” button resets the accumulation to zero. To avoid running \(2 \cdot 10^4\) full trajectories of 3000 steps each in the browser, the values \(M_1, \left|W_1\right|\) are drawn directly from their shared half-normal distribution, which is exactly what Bachelier’s theorem says they follow.

t = 1.00 number of simulations (target) = 5000 accumulated: 0

ProblemProblem 13

(Exercise). Let \(X \sim \mathscr {N}\left(0, \sigma^2\right)\). Find \(\mathbb {E}\left[\left|X\right|\right], \operatorname {Var}\left[\left|X\right|\right]\).

Let \(W = (W_t, t \geq 0)\) be a Wiener process, \(M_t\) its maximum on the interval \([0,t]\). Find \(\mathbb {E}\left[M_t\right], \operatorname {Var}\left[M_t\right]\).

ProblemProblem 14

Let \(\left(W_{t}, t \geq 0\right)\) be a Wiener process, \(x > 0\), \(\tau_{x}=\inf \left\{ t \geq 0: W_{t}=x\right\}\) the first time level \(x\) is reached. Compute the density of \(\tau_{x}\) and find \(\mathbb {E}\left[\tau_{x}\right]\).

HintHint

Use formula Equation 2.

Densities \(f_{\tau_x}(t) = \dfrac{x}{\sqrt{2\pi t^3}}\, e^{-x^2/(2t)}\) of the first hitting time of level \(x\) (the Lévy distribution, \(IG(1/2, x^2/2)\)) for three fixed levels (gray curves) and for the level \(x\) set by the slider (orange curve). As \(x\) increases, the peak shifts right and lowers; the heavy tail of order \(t^{-3/2}\) gives \(\mathbb{E}\left[\tau_x\right] = +\infty\).

Level x = 1.20

A simulation of \(\tau_1 \overset{d}{=} 1/Z^2\), \(Z \sim \mathcal{N}(0,1)\), accumulating in real time: the running mean \(\frac{1}{n}\sum_{i \leq n} \tau_1^{(i)}\) (red line) does not converge — rare, enormous values of \(\tau_1\) (occurring when \(Z\) lands very close to zero) repeatedly jolt it upward. The running median (blue line), by contrast, quickly stabilizes near the theoretical value \(\approx 2.2\) (gray dashed line). Both axes are logarithmic. Click “Restart” to see a new set of jumps.

n = 0

ProblemProblem 15

Let \(\left(W_t, t \geq 0\right)\) be a Wiener process. Find the distribution of the random variable \[ Y_{a}=\sup _{t \in [\tau _x, \tau _x+a]} W_{t}. \]

HintHint

Use the strong Markov property of the Wiener process and Bachelier’s theorem.

The strong Markov property “live”: after the time \(\tau_x\) at which the Wiener process \(W_t\) first reaches level \(x\), the process \(X_a := W_{\tau_x + a} - x\) is a freshly started Wiener process, independent of \(\mathcal{F}_{\tau_x}\) (green “new axes” originating at the point \((\tau_x, x)\)). Drag the slider \(a\) to see the window \([\tau_x, \tau_x+a]\) (light blue shading) grow, along with its supremum \(Y_a = \sup_{t \in [\tau_x, \tau_x+a]} W_t\) (red dot), and click the button to generate a new (simulated) trajectory.

a = 0.00

Let us now investigate at what argument the Wiener process on an arbitrary interval \([a,b]\) attains its maximum. Introduce the random variable

\[ T_{[a, b]} := \operatorname *{arg\, max}_{s \in [a, b]} W_{s}, \quad 0 \leq a < b < +\infty \]

For this quantity to be well-defined, it is necessary that almost surely all sample paths of the Wiener process attain their maximum on \([a,b]\) at exactly one point. To show this, let us solve the following auxiliary problem.

ProblemProblem 16

Let \(\left(W_{t}, t \geq 0\right)\) be a Wiener process. Prove that for any \(0 \leq a<b \leq c<d\), with probability 1 we have \[ \max _{s \in [a, b]} W_{s} \neq \max _{s \in [c, d]} W_{s}. \]

This shows that the quantity \(T_{[a,b]} = \operatorname *{arg\, max}_{s \in [a, b]} W_{s}\) is well-defined.

Let us now turn to finding the distribution of \(T_{[a,b]}\).

ProblemProblem 17

Prove that the random variable \(T=T_{[0,1]} = \operatorname *{arg\, max}_{s \in [0, 1]} W_{s}\) has the arcsine distribution. Namely, prove that for \(t \in (0,1)\) the distribution function of \(T\) is as follows: \[ F_T(t) = \mathbb {P}\left(T \leq t\right) = \frac{2}{\pi } \arcsin \sqrt{t} \]

Definition 17 The distribution of the time at which the Wiener process attains its maximum on the interval \([0,1]\), i.e. the distribution with distribution function \(F(x) = \frac{2}{\pi } \arcsin (\sqrt{x}), \; x \in [0,1]\), is called the arcsine distribution.

The arcsine law in real time: on the left, a single freshly simulated trajectory \(W_s, s \in [0,1]\) with the point \(T = \argmax_{s \in [0,1]} W_s\) highlighted; on the right, a histogram of \(2 \cdot 10^4\) accumulating values of \(T\) and (once enough data has accumulated) the theoretical arcsine density \(\frac{1}{\pi\sqrt{t(1-t)}}\). Counterintuitively, the maximum is attained most often near the endpoints of the interval, not in the middle. Click “Restart” to start over.

a fresh trajectory of Wₛ and its argmax

n = 0 / 20000

ProblemProblem 18

Let \(\left(W_{t}, t \geqslant 0\right)\) be a Wiener process, and let \(s < u < 0\). Find \[ \mathbb {P}\left(W_t \text{ has no zeros on the interval }[s, u]\right). \]

Seven simulated trajectories of \(W_t\) per click: the blue ones have no zeros on the interval \([u,v]\) (gray band), the red ones do. Here \(\mathbb{P}\left(W \text{ has no zeros on } [u,v]\right) = \frac{2}{\pi}\arcsin\sqrt{u/v}\), which equals \(\frac12\) when \(u=1, v=2\). Drag the sliders \(u\) and \(v\) (the counter resets when you do, since the target probability changes) and click “Generate more” to accumulate the empirical fraction of zero-free trajectories and watch it converge to the theoretical value.

u = 1.00 v = 2.00

theoretical P(no zeros) = 0.500  |  empirical fraction: 0 / 0

Footnotes

1. For example, if \(\left(\Omega , \mathcal{F}, \mathbb {P}\right) = \left([0,1], \mathscr {B}\left([0,1]\right), \lambda \right)\), then for all \(\omega \in \Omega\), \(\; \mathbb {P}\left(\left\{ \omega \right\} \right) = 0\). At the same time \(\mathbb {P}\left(\bigcup_{\omega \in \Omega }\left\{ \omega \right\} \right) = \mathbb {P}\left(\Omega \right) = 1\).

2. In the literature, almost surely finite stopping times are often not distinguished, and the terms “stopping time” and “almost surely finite stopping time” are used as synonyms.

3. We take \(\inf\) over the empty set to be \(+\infty\).

4. In the English-language literature, the first hitting time, besides hitting time, is also called the début; hence the second name — the début theorem.

5. See, for example, C. Dellacherie, P.-A. Meyer. Probabilities and Potential, Ch. III (section and projection theorems).