Distributions on the space of trajectories

WarningDraft

This lecture note is still being edited: the text may contain typos, inaccuracies, and locally unpolished passages. If you find something, please let the author know (contact information on the course page).

As already mentioned, a random process \(X=\left(X(t,\omega ), t \in T\right)\) can be viewed from three different perspectives.

• As in the definition: fix \(t=t_0\) and consider \(X_t(\cdot ) = X_{t_0}(\cdot )\), a random element. In this case we obtain a family of random elements \(X_t: (\Omega ,\mathcal{F}) \to (\mathsf{S}, \mathscr {B})\), indexed by \(t \in T\). If we denote by \(L^0\) the set of all measurable maps from \((\Omega ,\mathcal{F})\) to \((\mathsf{S}, \mathscr {B})\), then the random process in this case is represented as

  \[ X: T \to L^0, \qquad t \mapsto X_{t} = X(t,\cdot ) \]

• Fix \(\omega =\omega_0\) and consider \(X(\cdot , \omega ) = X(\cdot , \omega_0) = X_{\cdot }(\omega_0)\). Recall that this map on \(T\) is called a trajectory of the random function \(X=\left(X(t,\omega ), t \in T\right)\). The random process in this case is represented as

  \[ X: \Omega \to \mathsf{S}^T, \qquad \omega \mapsto X(\cdot , \omega ) \]

  where \(\mathsf{S}^T\) is the space of trajectories, i.e., the set of all possible maps from \(T\) to \(\mathsf{S}\).

  Given that \(\Omega\) carries a sigma-algebra \(\mathcal{F}\), it is natural to introduce some sigma-algebra on \(\mathsf{S}^T\) as well, so as to view \(X\) as a single random element with values in the space of trajectories (functions). In that case one could say that \(X\) is a random function or a random trajectory.

  Today’s seminar is devoted to this question.

• Let us not fix any variable at all. In this case the random process is represented as

  \[ X: T \times \Omega \to \mathsf{S}, \qquad (t, \omega ) \mapsto X(t,\omega ) \]

  If some sigma-algebra \(\mathcal{O}\) is given on the space \(T \times \Omega\), it is natural to ask whether \(X\) is measurable as a map from \((T \times \Omega , \mathcal{O})\) to \((\mathsf{S},\mathcal{F})\). The study of random functions from this point of view is closely tied to stochastic calculus. We will not touch on these topics in our course. For more detail, see, e.g., (Gushchin 2015).

Going forward, \(X\) without a subscript will denote the map \(\Omega \to \mathsf{}^T\).

1 The random function. The cylindrical sigma-algebra

With respect to which sigma-algebra on \(\mathsf{S}^T\) is the map \(X: \Omega \to \mathsf{S}^T\) guaranteed to be measurable? Let us introduce the projection map: for arbitrary \(t\) let

\[ \pi _{t}: \mathsf{S}^T \to \mathsf{S}, \qquad f \mapsto f(t) \]

Then \(X_{t} = \pi_t \circ X\).

Diagram of measurable spaces and maps: the random function \(X\), its section \(X_t\) at time \(t\), and the projection \(\pi_t\). Hover over a node or an arrow to see an explanation.

From the definition we know that \(X_t = \pi_t \circ X\) is measurable for every \(t \in T\):

\[ \begin{align} \left\{ X_t \in B\right\} & = X_t^{-1}(B) \\ & = ( \pi _t \circ X)^{-1}(B) \\ & = X^{-1}\left(\pi _t^{-1}(B)\right) \in \mathcal{F}, \qquad \forall B \in \mathscr {B} \end{align} \]

\(\pi_t^{-1}(B)\) is the set of all trajectories that pass through the “gate” (set) \(B\) at time \(t\). Such a set is called an elementary cylinder.

Move the sliders for the center and width of the “gate” (B), as well as for the point (t_0) — trajectories are colored blue if they pass through the gate at time (t_0) (i.e., they lie in the cylinder ({t_0}^{-1}(B))), and dashed red otherwise. Turn on the “second gate” toggle to see the intersection of two elementary cylinders ({t_0}^{-1}(B) _{t_1}^{-1}(B_1)).

t₀ = 2.0 gate center B = 1.46 gate width B = 0.45 second gate (intersection)

t₁ = 3.0 gate center B₁ = 1.35 gate width B₁ = 0.12

Denote by \(\mathcal{M}\) the set of all elementary cylinders:

\[ \mathcal{M} = \left\{ \pi _t^{-1}(B) \; : \; t \in T, \; B \in \mathscr {B}\right\} \]

In this case we have \(X^{-1}(\mathcal{M}) \subset \mathcal{F}\). By the properties of sigma-algebras we get \(X^{-1}(\sigma (\mathcal{M})) \subset \mathcal{F}\). The sigma-algebra \(\sigma (\mathcal{M})\) is called the cylindrical sigma-algebra on \(\mathsf{S}^T\) and is denoted \(\mathscr {B}^T\).

We have obtained the following statement: if \((X_{t})_{t \in T}\) is a random process, then \(X\) is a measurable map from \((\Omega , \mathcal{F})\) to the space of trajectories \(\mathsf{}^T\), equipped with the cylindrical sigma-algebra \(\mathscr {B}^T\).

It turns out that the converse statement is also true.

Lemma 1 (Equivalent definition of a random process) colback=black!5!white,colframe=red!75!black,size=small, fonttitle=,width=0.4box align=top, nobeforeafter Let \(X = X_{t} = X_t(\omega ) = X(t,\omega )\) be some map from \(T \times \Omega\) to \(\mathbb {R}\). Then

\((X_t)_{t \in T}\) is a stochastic process, i.e., \(\forall t \in T\) the map \(X_t: (\Omega,\mathcal{F}) \to (S, \mathcal{B}(S))\) is measurable.

\(\iff\)

\(X: \omega \mapsto X(\cdot, \omega)\) is a measurable map from \((\Omega, \mathcal{F})\) to \((S^T, \mathcal{B}(S)^T)\). In other words, \(X\) is a random function, i.e., a random element with values in the space \(\mathbb{R}^T\) equipped with the cylindrical sigma-algebra.

Diagram of the equivalence between two definitions of a stochastic process: as an indexed family of measurable maps \(X_t\), and as a single measurable map \(X\) into a space of functions.

2 Distribution of the random function

Let a random function \(X: (\Omega , \mathcal{F}) \to (\mathsf{S}^T, \mathscr {B}^T)\) be given. If a probability measure \(\mathbb {P}\) is given on the space \((\Omega , \mathcal{F})\), then \(X\) acquires a distribution — a probability measure on \((\mathsf{S}^T, \mathscr {B}^T)\), denoted \(\mathbf{P}_{X}\) or \(\operatorname {Law}\left(X\right)\):

\[ \mathbf{P}_{X}\left(A\right) := \mathbb {P}\left(X \in A\right), \quad A \in \mathscr {B}^T \]

Since the space \(\mathsf{S}^T\) is, as a rule, infinite-dimensional, it is convenient to reduce the study of distributions on \((\mathsf{S}^T, \mathscr {B}^T)\) to the study of their finite-dimensional projections.

Definition 1 For arbitrary \(n \in \mathbb {N}\), for an arbitrary collection of distinct points \(t_1, \ldots , t_n \in T\), denote \[ \begin{align} \pi _{t_1, \ldots , t_n}& : \mathsf{S}^T \to \mathsf{S}^n, \qquad y \mapsto (y(t_1), \ldots , y(t_n)) \\ \end{align} \] the projection of the space \(\mathsf{S}^T\) onto \(\mathsf{S}^n\) at the points \(t_1, \ldots , t_n\).

Diagram of measurable spaces and maps: the random function \(X\), its finite-dimensional section \((X_{t_1}, \ldots, X_{t_n})\), and the projection \(\pi_{t_1, \ldots, t_n}\). Hover over a node or an arrow to see an explanation.

Since \(X: (\Omega , \mathcal{F}) \to (\mathsf{S}^T, \mathscr {B}^T)\) and \(\pi_{t_1, \ldots , t_n}: (\mathsf{S}^T, \mathscr {B}^T) \to (\mathsf{S}^n, \mathscr {B}^n)\) are measurable, their composition

\[ (X_{t_1}, \ldots , X_{t_n}) = \pi _{t_1, \ldots , t_n} \circ X : (\Omega , \mathcal{F}) \to (\mathsf{S}^n, \mathscr {B}^n) \]

is also measurable, i.e., is a random element. The distribution of this random element is called the finite-dimensional distribution of the random function \(X\) (corresponding to \(t_1, \ldots , t_n\)) and is denoted \(\mathbf{P}_{X,t_1, \ldots , t_n}\) or \(\operatorname {Law}\left(X_{t_1}, \ldots , X_{t_n}\right)\), or, when it is clear from context which random function is meant, simply \(\mathbf{P}_{t_1, \ldots , t_n}\).

ProblemProblem 1

For brevity, use the notation \[ \begin{align} \left\{ \pi ^{-1}_{t_1, \ldots , t_n}(\mathscr {B}^n)\right\} & := \\ & \quad \left\{ \pi ^{-1}_{t_1, \ldots , t_n}(B) \; : \; n \in \mathbb {N}, \; t_1, \ldots , t_n \in T, \; t_i \neq t_j, \; B \in \mathscr {B}^n\right\} \end{align} \] Prove that

1. \(\mathscr {B}^T = \sigma \left\{ \pi^{-1}_{t_1, \ldots , t_n}(\mathscr {B}^n)\right\}\);

2. \(\left\{ \pi^{-1}_{t_1, \ldots , t_n}(\mathscr {B}^n)\right\}\) is a \(\pi\)-system ¹.

Recall that if 2 probability measures agree on some \(\pi\)-system, then they agree on the sigma-algebra it generates as well. Moreover, if \(\mathbf{P}_{X}\) is the distribution of the random function \(X\) on \((\mathsf{S}^T, \mathscr {B}^T)\), then the restriction of this probability measure to the sigma-algebra \(\pi^{-1}_{t_1, \ldots , t_n}(\mathscr {B}^n) \subset \mathscr {B}^T\) is, in essence, the finite-dimensional distribution, since \(\forall B \in \mathscr {B}^n\) we have

\[ \mathbf{P}_{X}\left(\pi ^{-1}_{t_1, \ldots , t_n}(B)\right) = \mathbf{P}_{t_1, \ldots , t_n}(B) \]

Thus, the distribution \(\mathbf{P}_{X}\) of an arbitrary random function \(X\) on \((\mathsf{S}^T, \mathscr {B}^T)\) is uniquely determined by the finite-dimensional distributions \(\left\{ \mathbf{P}_{t_1, \ldots , t_n}\right\}\).

2.1 Constructing a distribution on the space of functions from finite-dimensional distributions

We have established that the distribution \(\mathbf{P}\) (i.e., a probability measure on \((\mathsf{S}^T, \mathscr {B}^T)\)) is uniquely determined by the finite-dimensional distributions

\[ \mathbf{P}_{t_1,\ldots ,t_n} := \mathbf{P} \circ \pi _{t_1,\ldots ,t_n}, \qquad t_1,\ldots ,t_n \in T \]

A natural question arises: under what conditions can the distribution \(\mathbf{P}\) on the whole (typically infinite-dimensional) \((\mathsf{S}^T, \mathscr {B}^T)\) be recovered from given finite-dimensional distributions \(\mathbf{P}_{t_1,\ldots ,t_n}\)?

If \(\mathbf{P} = \mathbf{P}_X\), i.e., \(\mathbf{P}\) is the distribution of some random function \(X\), then note that

\[ \mathbf{P}_{t_1, \ldots , t_n}\left(B_1 \times \dots \times B_n\right) = \mathbb {P}\left(\bigcap _{k=1}^n X_{t_k}^{-1}(B_k)\right), \]

where \(B_1, \ldots , B_n \in \mathscr {B}\). From this relation follow the symmetry and consistency conditions for finite-dimensional distributions: \(\forall n \geq 2, \; \forall t_1, \ldots , t_n\) a collection of distinct points from \(T\), \(\forall B_1, \ldots , B_n \in \mathscr {B}\)

1. \(\mathbf{P}_{t_1, \ldots , t_n}\left(B_1 \times \dots \times B_n\right) = \mathbf{P}_{t_{i_1}, \ldots , t_{i_n}}\left(B_{i_1} \times \dots \times B_{i_n}\right)\) for any permutation \((i_1, \ldots , i_n)\) of the numbers \((1, \ldots , n)\);

2. \(\mathbf{P}_{t_1, \ldots , t_n}\left(B_1 \times \dots \times B_{n-1} \times \mathsf{S}\right) = \mathbf{P}_{t_1, \ldots , t_{n-1}}\left(B_1 \times \dots \times B_{n-1}\right)\).

It turns out that, under certain topological restrictions on \((\mathsf{S}, \mathscr {B})\), conditions , are sufficient for \(\mathbf{P}\) to exist.

Theorem 1 (Kolmogorov) Let a collection of distributions be given \[ \left\{ \mathbf{P}_{t_1,\ldots ,t_n}\text{ -- a distribution on } (\mathsf{S}^n,\mathscr {B}^n) \; : \; n \in \mathbb {N}, t_1, \ldots , t_n \in T, \; t_i \neq t_j\right\} \] satisfying the symmetry and consistency conditions , . Then there exists a distribution \(\mathbf{P}\) on \((\mathsf{}^T, \mathscr {B}^T)\) whose finite-dimensional distributions are exactly the distributions given above.

We have obtained a probability measure \(\mathbf{P}\) on \((\mathsf{}^T, \mathscr {B}^T)\). But how do we now obtain a random element (random function) \(X\) with this distribution? Recall the canonical model:

\[ (\Omega , \mathcal{F}) := \left(\mathsf{}^T, \mathscr {B}^T\right), \qquad X := \operatorname {Id} \]

That is, we define \(X\) as the identity map on the space of trajectories. In this case

\[ (X_t)_{t \in T} = (\pi _t)_{t \in T}, \]

since \(X_t = \pi_t \circ X = \pi_t \circ \operatorname {Id}\). In other words, the (one-dimensional) projections of the space of trajectories themselves play the role of the random process. This construction is called the canonical model.

2.2 Consistency conditions via characteristic functions for random functions with values in

Recall that there is a one-to-one correspondence between probability measures on \((\mathbb {R}^{n}, \mathscr {B}\left(\mathbb {R}^{n}\right))\) and their characteristic functions. Let us reformulate conditions , in terms of characteristic functions.

Theorem 2 Let a collection of distributions \(\mathbb {P}_{t_1, \dots , t_n}\) on \((\mathbb {R}^{n}, \mathscr {B}\left(\mathbb {R}^{n}\right))\) be given, \(n \in \mathbb {N}, \; t_1, \ldots , t_n \in T, t_i \neq t_j\) for \(i \neq j\). Denote by \(\varphi_{t_1,\dots ,t_n}\) the characteristic function of \(\mathbb {P}_{t_1, \dots , t_n}\). Then the symmetry and consistency conditions , are equivalent to the following conditions:

1. \(\varphi_{t_{i_1}, \ldots , t_{i_n}}(\lambda_{i_1}, \ldots , \lambda_{i_n}) = \varphi_{t_1, \ldots , t_n}(\lambda_1, \ldots , \lambda_n)\) for any permutation \((i_1, \ldots , i_n)\) of the numbers \((1, \ldots , n)\);

2. \(\varphi_{t_1, \ldots , t_{n-1}, t_n}(\lambda_1, \ldots , \lambda_{n-1}, 0) = \varphi_{t_1, \ldots , t_{n-1}}(\lambda_1, \ldots , \lambda_{n-1})\).

2.3 A special case of constructing a distribution: independent increments

Let us recall the definition.

Definition 2 A process \(\left(X_{t}, t \geqslant 0\right)\) is called a process with independent increments if \(\forall n \in \mathbb {N} \; \forall 0 \leqslant t_{1}<\ldots <t_{n}\) the random variables \(X_{t_{n}}-X_{t_{n-1}}, \ldots , X_{t_{2}}-X_{t_{1}}, X_{t_{1}}\) are jointly independent.

Note that in discrete time, a process with independent increments is nothing but a random walk. Thus, it is natural to view processes with independent increments as the continuous-time generalization of random walks.

Theorem 3 (Kolmogorov’s existence theorem for processes with independent increments) Let the initial (one-dimensional) distribution of the process \(\mathbb {Q}_0\) and a collection of (one-dimensional) probability distributions of the increments of the process \(\left\{ \mathbb {Q}_{s,t}: \; 0 \leq s < t < +\infty \right\}\) with characteristic functions \(\left\{ \varphi_{s,t}\right\}\) be given. A process \((X_t, t \geq 0)\) with independent increments satisfying \(X_0 \sim \mathbb {Q}_0, \; X_{t}-X_{s} \sim \mathbb {Q}_{s, t}, 0 \leq s < t\), exists if and only if \[ \varphi _{s,t} = \varphi _{s,u} \cdot \varphi _{u,t}, \quad \forall 0\leq s < u < t. \]

ExampleExample 1

Prove Theorem Theorem 3.

SolutionSolution

In one direction: obvious.

In the other direction: without loss of generality, let \(\mathbb {Q}_0 = \delta_0\). Let \(\varphi_{s,t} = \varphi_{s,u} \cdot \varphi_{u,t}\). Let us prove that the process exists. Let \(\varphi_{t_1, \ldots , t_n} = \varphi_{X_{t_1}, \ldots , X_{t_n}}\), \(t_1 < \dots < t_n\), be the characteristic function of its finite-dimensional distribution. Let us check that the consistency condition holds. Let \(\lambda = (\lambda_1, \dots , \lambda_n)^T \in \mathbb {R}^{n}\). We have

\[ \begin{align} \varphi _{t_1, \ldots , t_n}(\lambda ) & = \mathbb {E}\left[e^{i \left\langle (X_{t_1}, \ldots , X_{t_n})^T, \lambda \right\rangle }\right] = \\ & = \mathbb {E}\left[e^{i \left\langle A^{-1}A(X_{t_1}, \ldots , X_{t_n})^T, \lambda \right\rangle }\right] = \\ & = \mathbb {E}\left[e^{i \left\langle A(X_{t_1}, \ldots , X_{t_n})^T, (A^{-1})^T\lambda \right\rangle }\right] = \\ & = \varphi _{A(X_{t_1}, \ldots , X_{t_n})^T}((A^{-1})^T\lambda ), \end{align} \]

where

\[ \begin{align} A & = \begin{pmatrix} 1 & 0 & 0 & 0 & 0 & \dots & 0 & 0 \\ -1 & 1 & 0 & 0 & 0 & \dots & 0 & 0 \\ 0 & -1 & 1 & 0 & 0 & \dots & 0 & 0 \\ 0 & 0 & -1 & 1 & 0 & \dots & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & 0 & \dots & -1 & 1 \\ \end{pmatrix}, \\ A^{-1} & = \begin{pmatrix} 1 & 0 & 0 & 0 & 0 & \dots & 0 & 0 \\ 1 & 1 & 0 & 0 & 0 & \dots & 0 & 0 \\ 1 & 1 & 1 & 0 & 0 & \dots & 0 & 0 \\ 1 & 1 & 1 & 1 & 0 & \dots & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 1 & 1 & 1 & 1 & 1 & \dots & 1 & 1 \\ \end{pmatrix}, \\ (A^{-1})^T & = \begin{pmatrix} 1 & 1 & 1 & 1 & 1 & \dots & 1 & 1 \\ 0 & 1 & 1 & 1 & 1 & \dots & 1 & 1 \\ 0 & 0 & 1 & 1 & 1 & \dots & 1 & 1 \\ 0 & 0 & 0 & 1 & 1 & \dots & 1 & 1 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & 0 & \dots & 0 & 1 \\ \end{pmatrix} \end{align} \]

Therefore, denoting \(\lambda ' := (A^{-1})^T\lambda\), i.e., \(\lambda '_k = \lambda_k + \ldots + \lambda_n\), and using independence of the increments, we have

\[ \begin{align} \varphi _{t_1, \ldots , t_n}(\lambda ) & = \varphi _{X_{t_1}, X_{t_2}-X_{t_1}, \ldots , X_{t_n} - X_{t_{n-1}}}(\lambda ^\prime ) = \\ & = \varphi _{0, t_1}\left(\sum _{i=1}^n\lambda _i\right) \cdot \prod _{k=2}^{n} \varphi _{t_{k-1}, t_k}\left(\sum _{i=k}^n\lambda _i\right) \end{align} \]

This calculation was carried out for a hypothetical process \(X\) (which does not yet exist); let us now take its right-hand side as the definition of the characteristic functions \(\varphi_{t_1, \ldots , t_n}\) for ordered collections \(t_1 < \ldots < t_n\) (extend to unordered collections by symmetry, so that condition holds automatically), and check the consistency condition from Theorem Theorem 2, doing so directly in the strengthened form: with any coordinate \(\lambda_k\) set to zero, not just the last. For \(\lambda_k = 0\) we have \(\lambda '_k = \lambda '_{k+1}\), and two adjacent factors merge into one exactly by the condition of the theorem:

\[ \varphi _{t_{k-1}, t_k}(\lambda '_{k+1}) \cdot \varphi _{t_k, t_{k+1}}(\lambda '_{k+1}) = \varphi _{t_{k-1}, t_{k+1}}(\lambda '_{k+1}), \]

and we obtain exactly the characteristic function of the collection without the point \(t_k\) (at \(k = n\) the last factor simply becomes \(\varphi_{t_{n-1}, t_n}(0) = 1\)). It is precisely here that the condition \(\varphi_{s,t} = \varphi_{s,u} \cdot \varphi_{u,t}\) is used. The consistency conditions have been checked, and by Kolmogorov’s existence theorem (Theorem Theorem 1) the desired process exists.

Definition 3 Let \(\Lambda\) be some measure (i.e., a nonnegative countably additive set function) on \((\mathbb {R}_+, \mathscr {B}\left(\mathbb {R}_+\right))\) with density \(\lambda (t)\) with respect to Lebesgue measure \(\mu\): \[ \Lambda (B) = \int _{B} \lambda (t) \; \mu (dt), \qquad B \in \mathscr {B}\left(\mathbb {R}_+\right) \] A random process \(\left(N_{t}, t \geqslant 0\right)\) is called an (inhomogeneous) Poisson process with intensity function \(\lambda (t)\) if the following three conditions hold:

1. \(N_{0}=0\) a.s.;

2. \(N_{t}\) has independent increments;

3. \(N_{t}-N_{s}\) has a Poisson distribution with parameter \(\Lambda (s,t]\) for all \(0 \leq s \leq t < \infty\).

Note that these conditions are imposed only on the distribution of the process on the space of trajectories. In this sense, this definition is really a definition of the distribution of the Poisson random function on the space of trajectories. However, in most sources the notions of a random process and its distribution are used interchangeably.

If \(\lambda (t) \equiv \operatorname {const}\), then the Poisson process with such intensity is called homogeneous. We have already encountered precisely the homogeneous Poisson process earlier. Note, moreover, that we constructed the homogeneous Poisson process explicitly, as a renewal process. And it immediately had nice trajectories: they were all right-continuous with finite left limits.

ProblemProblem 2

Prove that the inhomogeneous Poisson process exists.

This problem only requires proving the existence of a distribution on \((\mathbb {R}^T, \mathscr {B}^T)\) with the stated properties. For practical purposes we would like more: that, as in the homogeneous case, the inhomogeneous process has trajectories that are right-continuous with finite left limits. This is exactly how a store’s customer counter behaves in practice. For more on obtaining nice trajectories, see below.

HintHint

Characteristic function of the Poisson distribution: \(\varphi_{\operatorname {Pois}\left(\alpha \right)}(t) = \mathrm{exp}\left(\alpha (e^{it} - 1)\right)\)

The top panel shows the intensity ((t) = 0 + A(1.5t-1)); the bottom panel shows a trajectory of (N_t) simulated by thinning, with red marks at the jump times: a homogeneous Poisson process of rate ({}=0+A) is generated, and each point is independently retained with probability ((t)/{}). Move the sliders (_0) and (A) to see how the density of jumps tracks the intensity, and click “New realization” to regenerate the trajectory for the same ((t)).

λ₀ = 2.0 A = 1.8

3 Equivalent random functions

Let us turn to the question of which random functions, and in what sense, can be considered “indistinguishable.”

Definition 4 Let \(X, Y\) be two random functions defined on the same probability space \((\Omega , \mathcal{F}, \mathbb {P})\), parametrized by the same set \(T\), and taking values in the same space \((\mathsf{S}, \mathscr {B})\).

• \(X, Y\) are called stochastically equivalent in the wide sense if their distributions on the space of trajectories \((\mathsf{S}^T, \mathscr {B}^T)\) coincide: \(\mathbf{P}_{X} = \mathbf{P}_{Y}\).

  Recall that the distribution of a random function is uniquely determined by its finite-dimensional distributions. Therefore, for \(X,Y\) to be stochastically equivalent in the wide sense, it suffices that \(\forall n \in \mathbb {N}, \; t_1, \ldots , t_n \in T, \; t_i \neq t_j\) for \(i \neq j\),

  \[ \operatorname {Law}\left(X_{t_1}, \ldots , X_{t_n}\right) = \operatorname {Law}\left(Y_{t_1}, \ldots , Y_{t_n}\right). \]

  Note that for this definition, in general it is not necessary that \(X, Y\) be defined on the same probability space \((\Omega , \mathcal{F}, \mathbb {P})\).

• \(X, Y\) are called stochastically equivalent in the narrow sense if

  \[ \mathbb {P}\left(X_t = Y_t\right)= 1, \; \forall t \in T \]

  That is, at every time point \(t\) the values of the processes coincide a.s. In this case \(Y\) is called a modification or version of \(X\) (\(X\) is likewise called a modification or version of \(Y\)).

• \(X, Y\) are called indistinguishable (stochastically indistinguishable) if they are a.s. equal:

  \[ \mathbb {P}\left(X = Y\right) = 1 \]

  Let us spell out the event \(\left\{ X = Y\right\}\):

  \[ \begin{align} \left\{ X = Y\right\} & = \left\{ \omega \; : X(\omega ) = Y(\omega )\right\} \\ & = \left\{ \omega \; : \; X_{t}(\omega ) = Y_t(\omega ), \; \forall t \in T\right\} \\ & = \left\{ X_t = Y_t \quad \forall t \in T\right\} \end{align} \]

  Stochastic indistinguishability means that \(X\) and \(Y\) have a.s. identical trajectories:

  \[ \mathbb {P}\left(X = Y\right) = \mathbb {P}\left(X_t = Y_t \quad \forall t \in T\right) = 1 \]

  Compare this to stochastic equivalence in the narrow sense: \(\mathbb {P}\left(X_t = Y_t \right) = 1, \; \forall t \in T\). The definitions differ in where “\(\forall t \in T\)” is placed: inside the scope of \(\mathbb {P}\) or outside it.

Remark: it is assumed that all events appearing above under the sign \(\mathbb {P}\) are measurable, i.e., lie in \(\mathcal{F}\).

ProblemProblem 3

1. Give an example of processes \(X = (X_t), Y = (Y_t)\) such that \(\operatorname {Law}\left(X_t\right) = \operatorname {Law}\left(Y_t\right), \; \forall t \in T\), but which are not stochastically equivalent in the wide sense. That is, the processes should have the same distribution at each point, but some finite-dimensional distributions should differ.

2. Give an example of real-valued processes \(X = (X_t), \; Y = (Y_t)\), defined on the same probability space \((\Omega , \mathcal{F}, \mathbb {P})\), that are stochastically equivalent in the wide sense but not in the narrow sense.

3. Prove that stochastic equivalence in the narrow sense implies stochastic equivalence in the wide sense.

The definitions of processes equivalent in the narrow sense and of indistinguishable processes are very similar. Let us try to understand the difference between them. Note that if \(X,Y\) are indistinguishable, then

\[ \begin{align} 1 & = \mathbb {P}\left(\forall t \in T \; X_t = Y_t\right) \\ & = \mathbb {P}\left(\bigcap _{t \in T}\left\{ X_t = Y_t\right\} \right) \\ & \overset {*}{\leq } \mathbb {P}\left(X_{t'} = Y_{t'}\right), \; \forall t' \in T \end{align} \]

The inequality marked with an asterisk holds only if a technical condition holds: the event \(X_{t'} = Y_{t'}\) is measurable to begin with, i.e., \(\left\{ X_{t'} = Y_{t'}\right\} \in \mathcal{F}\). Note that if \((\mathsf{S}, \mathscr {B}) = (\mathbb {R}^{n}, \mathscr {B}\left(\mathbb {R}^{n}\right))\) (as in most examples in our course) or if the probability space \((\Omega , \mathcal{F}, \mathbb {P})\) is complete, this condition holds for every \(t' \in T\). Thus, in the overwhelming majority of examples in the course, if \(X,Y\) are indistinguishable, then they are stochastically equivalent in the narrow sense².

Let us give an example of stochastically equivalent processes that are not indistinguishable.

ExampleExample 2

Let \((\Omega , \mathcal{F}, \mathbb {P}) = ([0,1], \mathscr {B}\left([0,1]\right), \lambda )\), where \(\lambda\) is Borel (Lebesgue) measure. Let \(T = [0,1]\). Let \[ X(t,\omega ) = 0, \; Y(t, \omega ) \equiv \; \mathbb {1}_{\left\{ t = \omega \right\} }, \] \(\forall t \in [0,1], \omega \in [0,1]\). Prove that \(X,Y\) are stochastically equivalent (in the narrow sense) random processes. Prove that, at the same time, they are not indistinguishable, i.e., their trajectories do not coincide with probability \(1\).

SolutionSolution

Stochastic equivalence in the narrow sense: for any fixed \(t \in [0,1]\)

\[ \mathbb {P}\left(X_t = Y_t\right) = \mathbb {P}\left(\omega \; : \; \omega \neq t\right) = 1 - \lambda \left(\left\{ t\right\} \right) = 1 \]

There is no indistinguishability, however:

\[ \left\{ X = Y\right\} = \left\{ \omega \; : \; Y(t, \omega ) = 0 \; \; \forall t \in [0,1]\right\} = \varnothing , \]

since for each \(\omega\) the trajectory \(Y(\cdot , \omega )\) equals \(1\) at the point \(t = \omega\). Altogether \(\mathbb {P}\left(X = Y\right) = 0 \neq 1\).

Note that for these processes, not a single pair of trajectories coincides.

An example of a modification without indistinguishability: \(X \equiv 0\) (blue trajectories) and \(Y(t,\omega) = \mathbb{1}\{t=\omega\}\) (red) as functions of two variables \((\omega, t)\). The gray semi-transparent plane is the level \(z=0\). For each red trajectory, an open circle marks the punctured point \(t=\omega\) at level \(0\), and a filled circle marks the value \(1\) at the same point; the points equal to \(1\) lie on the diagonal \(t=\omega\) (dashed). For fixed \(t\) the processes coincide a.s., but no pair of trajectories coincides entirely.

4 Shortcomings of the cylindrical sigma-algebra

If \(T\) is infinite, then whether a map \(y \in \mathsf{S}^T\) belongs to a set \(B \in \mathscr {B}^{T}\) is determined only by its values on some countable subset \(U \subset T\). That is, for any fixed \(B \in \mathscr {B}^{T}\) there exists an at most countable collection of points \(\left\{ t_i\right\} = U \subset T\) such that if \(f \in B\) and another function \(g\) agrees with \(f\) at these points

\[ f(t_i) = g(t_i), \quad \forall t_i \in U, \]

then \(g\) is also in \(B\): \(g \in B\).

ExampleExample 3

Let \(T=[0,1],\left(\mathsf{S}, \mathscr {B}\right)=(\mathbb {R}, \mathscr {B}\left(\mathbb {R}\right))\). Then the space of trajectories \(\mathsf{S}^T= \mathbb {R}^{[0,1]}\) is the space of real-valued functions on \([0,1]\). Let \(\alpha \in \mathbb {R}\) be arbitrary, let \[ M_\alpha = \left\{ y \in \mathbb {R}^{[0,1]}: \sup _{t \in [0,1]}y(t) \leq \alpha \right\} . \] Prove that \(M_\alpha \not\in \mathcal{B}^{[0,1]}\).

SolutionSolution

Suppose that \(M_\alpha \in \mathscr {B}^{[0,1]}\). Then, by the above, there is a countable set \(U \subset [0,1]\) such that whether a function belongs to \(M_\alpha\) is determined only by its values on \(U\). Take \(f \equiv \alpha\) (clearly, \(f \in M_\alpha\)) and a point \(t_0 \in [0,1] \setminus U\) (it exists since \(U\) is countable while \([0,1]\) is uncountable). The function \[ g(t) = \begin{cases} \alpha , & t \neq t_0 \\ \alpha + 1, & t = t_0 \end{cases} \] agrees with \(f\) on \(U\), and therefore must also lie in \(M_\alpha\). But \(\sup_{t} g(t) = \alpha + 1 > \alpha\) — a contradiction.

ExampleExample 4

Exercise. Let \(T=[0,1],\left(\mathsf{S}, \mathscr {B}\right)=(\mathbb {R}, \mathscr {B}\left(\mathbb {R}\right))\). Then the space of trajectories \(\mathsf{S}^T= \mathbb {R}^{[0,1]}\) is the space of real-valued functions on \([0,1]\). Prove that \(C[0,1] \not\in \mathcal{B}^{T}\).

SolutionSolution

Similar to the previous example. Let \(C[0,1] \in \mathscr {B}^{[0,1]}\), and let \(U\) be the corresponding countable set of points. Take \(f \equiv 0 \in C[0,1]\), a point \(t_0 \in [0,1] \setminus U\), and a function \(g = \; \mathbb {1}_{\left\{ t_0\right\} }\), which agrees with \(f\) on \(U\). Then \(g\) would also have to lie in \(C[0,1]\), but \(g\) is discontinuous at \(t_0\) — a contradiction.

The 2 previous examples tell us that if we know absolutely nothing about the trajectories of a real-valued process \((X_t: t \in T)\), \(T = [0,1]\) (or \(\mathbb {R}\)), then we cannot always determine the probability \(\mathbb {P}\left(\sup_t X \leq \alpha \right) = \mathbb {P}\left(X^{-1}(M_\alpha )\right)\) that the trajectories are bounded above by some number \(\alpha \in \mathbb {R}\), or the probability \(\mathbb {P}\left(X \in C[0,1]\right) = \mathbb {P}\left(X^{-1}(C[0,1])\right)\) that the trajectories are continuous. This happens because the sets \(M_\alpha , C[0,1]\) do not belong to the cylindrical sigma-algebra \(\mathscr {B}^{[0,1]}\), and hence one cannot assert with certainty that \(X^{-1}(M_\alpha )\) or \(X^{-1}(C[0,1])\) belong to \(\mathcal{F}\), despite the fact that the map \(X: (\Omega , \mathcal{F}, \mathbb {P}) \to (\mathbb {R}^{[0,1]}, \mathscr {B}^{[0,1]})\) itself is measurable. The next 2 problems illuminate this point.

ProblemProblem 4

1. Show that for one and the same random function, in some cases we can measure its landing in a set outside the cylindrical sigma-algebra, and in other cases we cannot. In other words, give an example of a probability space \((\Omega , \mathcal{F}, \mathbb {P})\), a random function \(X\) on it, and subsets of trajectories \(G_1,G_2 \subset \mathbb {R}^{T}\), such that

\[ \begin{array}{ll} G_1 \not\in \mathscr {B}^T \qquad & G_2 \not\in \mathscr {B}^T \\ \left\{ X \in G_1\right\} \in \mathcal{F}\qquad & \left\{ X \in G_2\right\} \notin \mathcal{F} \end{array} \]

2. Show that for one and the same set of trajectories outside the cylindrical sigma-algebra, for some random function we can measure the probability of it landing in this set, while for another random function we cannot. In other words, give an example of a probability space \((\Omega , \mathcal{F}, \mathbb {P})\), two random functions \(X\) and \(Y\) on it, and a set \(G \notin \mathcal{B}^{T}\) such that \(\{ X \in G\} \in \mathcal{F}, \; \{ Y \in G\} \notin \mathcal{F}\).

HintHint

In both parts take \(\Omega = T = [0,1]\), \((\mathsf{S}, \mathscr {B}) = (\mathbb {R}, \mathscr {B}\left(\mathbb {R}\right))\), and as \(\mathcal{F}\) take the sigma-algebra of countable and co-countable subsets of \([0,1]\).

Enum-item(1)

\(G_1 = M_1, \; G_2 = C[0,1]\).

Enum-item(2)

\(G = C[0,1], \; X \equiv 0\).

Thus, it is not always possible to speak of the probability that a process lands in a set that does not belong to the cylindrical \(\sigma\)-algebra. Moreover, it turns out that even when this is possible, the probability of landing in such a set need not be determined by the finite-dimensional distributions of the process:

ProblemProblem 5

Give an example of a probability space \((\Omega , \mathcal{F}, \mathbb {P})\), two random functions \(X\) and \(Y\) on it that are stochastically equivalent in the narrow sense, and a set \(G \notin \mathcal{B}^{T}\) such that \(\left\{ X \in G\right\} , \left\{ Y \in G\right\} \in \mathcal{F}\), but nevertheless \(\mathbb {P}\left(X \in G\right) \neq \mathbb {P}\left(Y \in G\right)\).

HintHint

Take \((\Omega , \mathcal{F}, \mathbb {P}) = ([0,1], \mathscr {B}\left([0,1]\right), \lambda )\), \((\mathsf{S}, \mathscr {B}) = (\mathbb {R}, \mathscr {B}\left(\mathbb {R}\right))\), \(T = [0,1]\), \(G = C[0,1]\), \(X \equiv 0\).

5 Constructing a random function with a given distribution and nice trajectories

As can be seen from the discussion and problems above, processes with the same distribution on the space \((\mathsf{}^T, \mathscr {B}^T)\) can have drastically different trajectories. In fact, Kolmogorov’s theorem does not say much of anything about trajectories — it only constructs a distribution from finite-dimensional projections. In practice, however, we would still like to obtain a process with nice trajectories.

5.1 Continuous trajectories

For simplicity let \(T = [0,1]\) (one could take \(\mathbb {R}_+\) or \(\mathbb {R}\); the computations in this section would not change).

Suppose that all (or almost all) trajectories of the process \(X(\omega ) = X(t,\omega )\) are continuous, i.e., \(\mathbb {P}\left(\omega \in \Omega : X(\omega ) \in C[0,1]\right) = 1\). Denote the cylindrical \(\sigma\)-algebra on the space \(C[0,1]\) by \(C[0,1] \cap \mathscr {B}^{[0,1]}\) and define it as the intersection of elements of the cylindrical sigma-algebra \(\mathscr {B}^{[0,1]}\) on \(\mathbb {R}^{[0,1]}\) with \(C[0,1]\):

\[ C[0,1] \cap \mathscr {B}^{[0,1]} := \left\{ C[0,1] \cap B: \; B \in \mathscr {B}^{[0,1]}\right\} \]

It is not hard to prove that this is indeed a sigma-algebra, and moreover it is generated by the intersections of cylinders with \(C[0,1]\):

\[ \begin{align} C[0,1] \cap \mathscr {B}^{[0,1]} & = \\ & \quad \sigma \left(\left\{ C[0,1] \cap \pi ^{-1}_{t_1,\ldots ,t_n}(B) \; : \; n \in \mathbb {N}, \; 0 \leq t_1 < \ldots < t_n \leq 1, \; B \in \mathscr {B}^n\right\} \right) \end{align} \tag{1}\]

Recall that \(C([0,1])\) carries yet another sigma-algebra — the Borel sigma-algebra \(\mathscr {B}\left(C[0,1]\right)\), generated by the open balls in the supremum metric

\[ \rho (f', f'') = \sup \limits _{t \in [0,1]} \left|f'(t) - f''(t)\right|. \]

It turns out that the following lemma holds.

Lemma 2 On \(C[0,1]\), the Borel sigma-algebra \(\mathscr {B}\left(C[0,1]\right)\) and the cylindrical sigma-algebra coincide: \[ \mathscr {B}\left(C[0,1]\right) = C[0,1] \cap \mathscr {B}^{[0,1]} \]

ProblemProblem 6

Prove Lemma Lemma 2.

We obtain the following result.

Theorem 4 Let \(X = (X_t, t \in [0,1])\) be a real-valued random process such that almost all its trajectories are continuous, i.e., \(\mathbb {P}\left(X \in C[0,1]\right) = 1\). Then the process \(X\) is \(\mathcal{F}|\mathscr {B}\left(C[0,1]\right)\)-measurable as a map \[ X: (\Omega , \mathcal{F}) \to (C[0,1], \mathscr {B}\left(C[0,1]\right)). \]

ProblemProblem 7

Let \(X = (X_t, t \in [0,1])\) be a real-valued random process with a.s. continuous trajectories. Prove that in this case \[ \left\{ \sup _{t \in [0,1]} X(t) \leq \alpha \right\} = \left\{ X \in M_\alpha \right\} = X^{-1}(M_\alpha ) \in \mathcal{F}, \; \forall \alpha \in \mathbb {R}. \]

This problem shows that if we have an a.s. continuous process \(X\), we may freely find the probability that its trajectories do not exceed an arbitrary level \(\alpha\).

Recall that Kolmogorov’s existence theorem allowed us to construct a process with prescribed consistent finite-dimensional distributions, with values in an arbitrary Borel space, for example, in the space \((\mathbb {R},\mathscr {B}\left(\mathbb {R}\right))\). However, this theorem said nothing about the trajectories of this process. Obtaining continuous trajectories for a process is aided by the following theorem, also proved by Kolmogorov.

Theorem 5 (Kolmogorov’s continuous modification theorem) Let \(X = (X_t, t \in T)\), where \(T = [0,1]\) or \(\mathbb {R}_+\) or \(\mathbb {R}\), be a real-valued random process such that there exist \(\alpha , \varepsilon > 0\), \(C = C(\alpha , \varepsilon ) > 0\) such that \[ \mathbb {E}\left[\left|X_t - X_s\right|^\alpha \right] \leq C \left|t-s\right|^{1 + \varepsilon }, \quad \forall t,s \in T. \tag{2}\] Then the process \(X\) has a continuous version (modification).

Recall that \(Y = (Y_t)\) is a modification of the process \(X = (X_t)\) if they coincide a.s. at each point \(t \in T\): \(\mathbb {P}\left(X_t = Y_t\right) = 1\), \(\forall t \in T\). This theorem says that if the distribution of the process \(X = (X_t)\) satisfies condition Equation 2 (i.e., the expectations of the increments of the process are, in some sense, bounded by the increments of the argument), then this process can always be “slightly” redefined at each point \(t\) (i.e., redefined on a set of measure zero) so that the new version of the process has a.s. continuous trajectories.

ProblemProblem 8

1. Prove that a stationary Poisson process \(\left(X_{t}, t \geq 0\right)\) of intensity \(\lambda >0\) has no continuous modification, i.e., no modification all of whose trajectories are continuous.

2. Using the previous part, show that if we set \(\varepsilon = 0\) in condition (Equation 2), the theorem need not hold in general.

5.2 Càdlàg trajectories

Càdlàg is an abbreviation of the French continue à droite, limite à gauche, right-continuous with finite left limits. Processes with such trajectories are very common in the theory of random processes and in stochastic calculus. The theory of random processes was actively developed in France in the second half of the 20th century, which is why this French abbreviation is conventionally used to refer to such trajectories concisely. All possible càdlàg trajectories on \(T\) are called the Skorokhod space on \(T\) and are denoted

\[ D(T) \]

The space \(D(T)\), like \(C(T)\), has a wealth of nice properties. To avoid overloading the exposition, we will not present them here. For more detail, see (Биллингсли 1977). Here we will give a certain analogue of Kolmogorov’s continuous modification theorem.

Theorem 6 (On the nondecreasing càdlàg modification) Let \(X=\left(X_{t}, t \geq 0\right)\) be a random process that is

1. stochastically nondecreasing, i.e., \(\mathbb {P}\left(X_{s} \leq X_{t}\right) = 1\) for all \(0 \leq s \leq t<\infty\);

2. right-continuous in probability: at every point \(s\) of the ray \([0, \infty )\) we have

\[ X_t \xrightarrow [t \to s+0]{\mathbb {P}} X_s, \quad \forall s \geq 0. \] Then \(X\) has a modification with a.s. nondecreasing càdlàg trajectories.

ProblemProblem 9

Show that for the inhomogeneous Poisson process with intensity function \(\lambda (t)\), one can obtain a version with nondecreasing càdlàg trajectories.

Lemma 3 Let \(f(x)\) be a function on \([0,1]\) that is nondecreasing along rational points. Then the function \(\tilde{f}(x)\), defined as \[ \tilde{f}(x)=\lim _{r \rightarrow x+0, r \in \mathbb {Q}} f(r), \quad x \in [0,1), \qquad \tilde{f}(1) := f(1), \] is right-continuous and nondecreasing.

ProblemProblem 10

1. Prove Lemma Lemma 3;

2. Using Lemma Lemma 3, prove Theorem Theorem 6.

References

Gushchin, Alexander A. 2015. Stochastic Calculus for Quantitative Finance. Elsevier.

Биллингсли, П. 1977. Сходимость Вероятностных Мер. Наука.

Footnotes

1. A \(\pi\)-system is a nonempty collection of subsets closed under pairwise intersections.

2. In general one can construct an incomplete probability space \((\Omega , \mathcal{F}, \mathbb {P})\), a non-Euclidean value space \((\mathsf{S}, \mathscr {B})\), and 2 random functions \(X= (X_t),Y = (Y_t)\) that would be indistinguishable but not stochastically equivalent.