The Wiener Process, Gaussian Processes

WarningDraft

This lecture note is still being edited: the text may contain typos, inaccuracies, and locally unpolished spots. If you find something, please let the author know (contact information on the course page).

1 Gaussian processes

Definition 1 A random process \(\left(X_{t}, t \in T\right)\) is called Gaussian if all its finite-dimensional distributions are Gaussian, i.e., for any \(n \in \mathbb {N}\) and any \(t_{1}, \ldots , t_{n} \in T\) the random vector \(\left(X_{t_{1}}, \ldots , X_{t_{n}}\right)\) is Gaussian.

Recall that the distribution of a Gaussian vector is uniquely determined by its mean and covariance matrix. Hence the finite-dimensional distributions of a Gaussian random function are uniquely determined by its mean function and covariance function. What properties must these functions have for such a random function to exist? The mean function can be arbitrary, while the covariance function must have the property of nonnegative definiteness.

Definition 2 A real-valued function \(f(t, s), t, s \in T\), is called nonnegative definite on \(T \times T\) if \(\forall n, \; \forall t_{1}, \ldots , t_{n} \in T\) the matrix \[ \bigg(f(t_i, t_j)\bigg)_{i,j=1}^n = \begin{pmatrix} f(t_1, t_1) & f(t_1, t_2) & \cdots & f(t_1, t_n) \\ f(t_2, t_1) & f(t_2, t_2) & \cdots & f(t_2, t_n) \\ \vdots & \vdots & \vdots & \vdots \\ f(t_n, t_1) & f(t_n, t_2) & \cdots & f(t_n, t_n) \\ \end{pmatrix} \] is nonnegative definite, i.e., \(\forall \overrightarrow {x} = (x_{1}, \ldots , x_{n})^T \in \mathbb {R}^{n}\) we have \[ (x_{1}, \ldots , x_{n}) \cdot \begin{pmatrix} f(t_1, t_1) & f(t_1, t_2) & \cdots & f(t_1, t_n) \\ f(t_2, t_1) & f(t_2, t_2) & \cdots & f(t_2, t_n) \\ \vdots & \vdots & \vdots & \vdots \\ f(t_n, t_1) & f(t_n, t_2) & \cdots & f(t_n, t_n) \\ \end{pmatrix} \cdot \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix} \quad \geq \quad 0 \]

The expression on the left can be rewritten as

\[ \overrightarrow {x}^T \cdot \left(f(t_i,t_j)\right) \cdot \overrightarrow {x} \qquad \text{ or } \qquad \sum _{i,j = 1}^{n} f(t_i, t_j)x_ix_j \]

Lemma 1 The covariance function \(R(s, t)\) of any random function is nonnegative definite and symmetric.

ExampleExample 1

Prove the preceding lemma.

SolutionSolution

Let \(X = (X_t, t \in T)\) be a random process for which \(R(s,t)\) is the covariance function. Let \(t_1, \ldots , t_n \in T\) be arbitrary fixed elements of \(T\), and let \(\overrightarrow {x} \in \mathbb {R}^{n}\) be an arbitrary vector in \(\mathbb {R}^{n}\). Note that the matrix \(\left(R(t_i, t_j)\right)_{i,j=1}^n \in \mathbb {R}^{n \times n}\) is the covariance matrix of the random vector \((X_{t_1}, \ldots , X_{t_n})^T =: X_{t_1 \ldots t_n}\):

\[ \begin{align} \operatorname {Var}\left[X_{t_1 \ldots t_n}\right] = \operatorname {Var}\left[\begin{pmatrix} X_{t_1} \\ X_{t_2} \\ \vdots \\ X_{t_n} \end{pmatrix}\right] & = \begin{bmatrix} \operatorname {Cov}\left[ X_{t_1}, X_{t_1} \right] & \operatorname {Cov}\left[ X_{t_1}, X_{t_2} \right] & \cdots & \operatorname {Cov}\left[ X_{t_1}, X_{t_n} \right] \\ \operatorname {Cov}\left[ X_{t_2}, X_{t_1} \right] & \operatorname {Cov}\left[ X_{t_2}, X_{t_2} \right] & \cdots & \operatorname {Cov}\left[ X_{t_2}, X_{t_n} \right] \\ \vdots & \vdots & \vdots & \vdots \\ \operatorname {Cov}\left[ X_{t_n}, X_{t_1} \right] & \operatorname {Cov}\left[ X_{t_n}, X_{t_2} \right] & \cdots & \operatorname {Cov}\left[ X_{t_n}, X_{t_n} \right] \\ \end{bmatrix} \\ & = \begin{bmatrix} R(t_1, t_1) & R(t_1, t_2) & \cdots & R(t_1, t_n) \\ R(t_2, t_1) & R(t_2, t_2) & \cdots & R(t_2, t_n) \\ \vdots & \vdots & \vdots & \vdots \\ R(t_n, t_1) & R(t_n, t_2) & \cdots & R(t_n, t_n) \end{bmatrix} = \left(R(t_i, t_j)\right)_{i,j=1}^n \end{align} \]

Next, what is the expression \(\overrightarrow {x}^T \cdot \left(R(t_i, t_j)\right) \cdot \overrightarrow {x}\) from the point of view of probability theory? Recall that if \(Y\) is a random vector of dimension \(n\) and \(A \in \mathbb {R}^{m \times n}\) is an arbitrary matrix of size \(m \times n\), then

\[ \operatorname {Var}\left[A Y\right] = A \operatorname {Var}\left[Y\right] A^T \]

Consequently,

\[ \begin{align} \overrightarrow {x}^T \cdot \underbrace{\left(R(t_i, t_j)\right)}_{= \operatorname {Var}\left[X_{t_1 \ldots t_n}\right]} \cdot \overrightarrow {x} & = \operatorname {Var}\left[\overrightarrow {x}^T \cdot X_{t_1 \ldots t_n}\right] = \\ & = \operatorname {Var}\left[x_1X_{t_1} + \ldots + x_nX_{t_n}\right] \geqslant 0, \end{align} \]

since the variance of a random variable is always nonnegative.

It turns out that nonnegative definiteness and symmetry are not only necessary but also sufficient for a function to be the covariance function of some random process. The following theorem states that such a process can even be chosen to be Gaussian.

Theorem 1 Let \(T\) be some set, let \(a: T \rightarrow \mathbb {R}\) be an arbitrary function, and let \(R: T \times T \rightarrow \mathbb {R}\) be a symmetric and nonnegative definite function. Then there exists a probability space \((\Omega , \mathcal{F}, \mathbb {P})\) and a Gaussian process \(\left(X_{t}, t \in T\right)\) on it such that \(\forall s, t \in T\) \[ \mathbb {E}\left[X_{t}\right] = a(t), \quad \operatorname {Cov}\left[ X_s, X_t \right] = R(s,t). \]

ExampleExample 2

Prove that there exists a Gaussian process \(\left(X_{t}, t \geqslant 0\right)\) with zero mean function and covariance function \(r(s, t)=\min (s, t)\).

SolutionSolution

By theorem Theorem 1, it suffices to check that the function \(r(s, t)=\min (s, t)\) is nonnegative definite. To this end, consider the following integral representation: \[ \min (s, t)=\int _{0}^{+\infty } I_{[0, t]}(x) I_{[0, s]}(x) d x . \] Then for any \(t_{1}, \ldots , t_{n} \in \mathbb {R}_{+}, z_{1}, \ldots , z_{n} \in \mathbb {R}\) we have \[ \begin{align} \sum _{i, j=1}^{n} \min \left(t_{i}, t_{j}\right) z_{i} z_{j} & =\sum _{i, j=1}^{n}\left(\int _{0}^{\infty } I_{\left[0, t_{i}\right]}(x) I_{\left[0, t_{j}\right]}(x) d x\right) \\ & \quad z_{i} z_{j}= \\ & =\int _{0}^{\infty }\left(\sum _{i, j=1}^{n}\left(z_{i} I_{\left[0, t_{i}\right]}(x)\right)\left(z_{j} I_{\left[0, t_{j}\right]}(x)\right)\right) \\ & \quad d x= \\ & =\int _{0}^{\infty }\left(\sum _{i=1}^{n} z_{i} I_{\left[0, t_{i}\right]}(x)\right)^{2} d x \geqslant 0 . \end{align} \]

Let us note that this example can also be solved another way, by explicitly exhibiting a random process with covariance function \(\min (s, t)\). One such process is the Poisson process of intensity 1. A second suitable process (which, as we shall see below, is Gaussian, unlike the Poisson process) is the Wiener process, also called the process of Brownian motion.

2 The Wiener process: introduction

2.1 The Wiener process as a process with independent increments

The Wiener process has several definitions. One of them is as follows: it is a process started at zero with independent increments, distributed as \(\mathscr {N}\left(0, t-s\right)\). As a rule, continuity of the paths is also included in the definition. Formally:

Definition 3 A random process \(\left(W_{t}, t \geqslant 0\right)\) is called a Wiener process or Brownian motion (abbreviated WP or BM) if

1. \(W_{0}=0\) a.s.;

2. \(W_{t}\) has independent increments;

3. \(W_{t}-W_{s} \sim \mathcal{N}(0, t-s), \forall t \geqslant s \geqslant 0\).

4. (*) \(W_t\) has almost surely continuous paths.

The model of the Wiener process was first proposed by L. Bachelier in 1903 to model fluctuations of stock prices on an exchange. Since then this process has become one of the central objects in probability theory, and it is actively used to model chaotic motion.

Typical trajectories of Brownian motion: continuous, nowhere differentiable functions (see path properties further down in the notes)

200 simulated trajectories of Brownian motion on [0,1]; each trajectory’s color corresponds to its terminal value W₁. On the right — a histogram of the distribution of terminal values over the same color bins.

Theorem 2 The Wiener process exists.

Some ways of proving this theorem, i.e., of constructing a Wiener process, can be found in section Section 4.

2.2 The Wiener process is a Gaussian process

ExampleExample 3

Prove that the Wiener process is Gaussian.

SolutionSolution

Let \(t_{1}, \ldots , t_{n} \in \mathbb {R}_{+}\). We need to show that the vector \(\left(W_{t_{1}}, \ldots , W_{t_{n}}\right)\) is Gaussian. We may assume \(t_{1} \leqslant \ldots \leqslant t_{n}\). Note that by the definition of the Wiener process, the random variables \(W_{t_{n}}-\) \(W_{t_{n-1}}, \ldots , W_{t_{2}}-W_{t_{1}}, W_{t_{1}}\) are independent and normally distributed. Hence the vector \(\left(W_{t_{n}}-W_{t_{n-1}}, \ldots , W_{t_{2}}-\right.\) \(\left.W_{t_{1}}, W_{t_{1}}\right)\) is Gaussian. The vector \(\left(W_{t_{1}}, \ldots , W_{t_{n}}\right)\) is a linear transformation of it, so it too is Gaussian, and the process \(W_{t}\) is Gaussian.

Since the Wiener process is Gaussian, it is important to compute its covariance function and mean function.

ExampleExample 4

Let \(\left(W_{t}, t \geqslant 0\right)\) be a Wiener process. Show that \[ \mathbb {E}\left[W_t\right] = 0, \; \forall t \geqslant 0, \qquad \operatorname {Cov}\left[ W_t, W_s \right] = \min (t,s), \; \forall t,s \geqslant 0 . \]

SolutionSolution

Covariance: \[ \begin{align} \operatorname {Cov}\left[ W_{t}, W_{s} \right] & = \operatorname {Cov}\left[ W_{t}-W_{s}, W_{s} \right] + \operatorname {Cov}\left[ W_{s}, W_{s} \right] = \quad \text{since $W_{t}-W_{s}$ and $W_s$ are independent} \\ & = 0 + \operatorname {Cov}\left[ W_{s}, W_{s} \right] =\operatorname {Var}\left[W_s\right] = s . \end{align} \] In the general case we obtain \[ \operatorname {Cov}\left[ W_t, W_s \right] = \min (t,s) \]

The Gaussianity of the Wiener process allows us to state its second, equivalent, definition.

Definition 4 A random process \(\left(W_{t}, t \geqslant 0\right)\) is called a Wiener process or Brownian motion if:

1. \(W_{t}\) is a Gaussian process;

2. \(\mathbb {E}\left[W_{t}\right]=0 \; \forall t \geqslant 0\);

3. \(\operatorname {Cov}\left[ W_s, W_t \right]=\min (s, t), \; \forall s, t \geqslant 0\).

4. (*) \(W_t\) has almost surely continuous paths.

Theorem 3 The first 3 conditions in definition Definition 3 and in definition Definition 4 are equivalent and uniquely characterize the finite-dimensional distributions of the process. As a consequence, definitions Definition 3 and Definition 4 (together with the condition of almost sure continuity of paths) are equivalent.

The second definition, via Gaussianity, is often more convenient for checking that a given process is a Wiener process. Checking Gaussianity is often easier than checking independence of increments.

ExampleExample 5

Let \(\left(W_{t}, t \geqslant 0\right)\) be a Wiener process. Prove that the process \[ B_{t}= \begin{cases} t \cdot W_{\frac{1}{t}}, & t > 0 \\ 0, & t = 0 \end{cases} \] is also a Wiener process.

SolutionSolution

Let us check the second definition, via Gaussianity.

Enum-item1.

\(\forall t_{1}, \ldots , t_{n} > 0\) the vector \(\left(B_0, B_{t_{1}}, \ldots , B_{t_{n}}\right)^T\) is a linear transformation of the vector

\[ \left(\underbrace{W_{0}}_{=0 \text{ a.s.}}, W_{\frac{1}{t_{1}}}, \ldots , W_{\frac{1}{t_{n}}}\right)^T \]

This vector is Gaussian by the definition of \(W_t\). Hence \(\left(B_0, B_{t_{1}}, \ldots , B_{t_{n}}\right)^T\) is also a Gaussian vector (a linear transformation of a Gaussian vector is always a Gaussian vector).

We have obtained that \((B_{t})\) is a Gaussian process.

Enum-item2.

\(\mathbb {E}\left[ B_{t} \right] = 0, \; \forall t \geqslant 0\).

Enum-item3.

\(\operatorname {Cov}\left[ B_{t}, B_{s} \right] = \operatorname {Cov}\left[ t W_{\frac{1}{t}}, s W_{\frac{1}{s}} \right] = t s \min \left(\frac{1}{t}, \frac{1}{s}\right)=\) \(\frac{t s}{\max (t, s)}=\min (t, s), \; \forall t, s > 0\).

Enum-item4.

Continuity of paths? Let us spell this out in detail. We know that the paths of \(W_t\) itself are almost surely continuous, i.e., there is \(\Omega ' \subset \Omega\) such that \(\mathbb {P}\left(\Omega '\right) = 1\) and \(W_t(\omega )\) is continuous in \(t\) for every fixed \(\omega \in \Omega '\). If for a fixed \(\omega\) the function \(W_t(\omega )\), \(t \geq 0\), is continuous at every point (and right-continuous at zero), then \(t \cdot W_{1/t}(\omega )\) will be continuous for all \(t \in (0,+\infty )\). What remains is the question of right-continuity at zero. Here we need to jump ahead slightly and use the result of problem Lemma 2:

\[ t \cdot W_{1/t} \xrightarrow [t \to 0+]{\text{a.s.}} 0 \]

That is, there exists \(\Omega '' \subset \Omega\) such that \(\mathbb {P}\left(\Omega ''\right) = 1\) and \(t \cdot W_{1/t}(\omega ) \to 0\) as \(t \to 0+\) for every fixed \(\omega \in \Omega ''\). As a result, for every \(\omega \in \Omega ' \cap \Omega ''\) the path \(B_t(\omega )\) is continuous at every point of \([0,+\infty )\). Moreover \(\mathbb {P}\left(\Omega ' \cap \Omega ''\right) = 1\). We have obtained almost sure continuity of the paths of \(B_t\).

Hence the process \((B_{t}, \; t \geq 0)\) is a Wiener process.

ProblemProblem 1

Let \(\left(W_{t}, t \geqslant 0\right)\) be a Wiener process. Prove that the following processes are also Wiener processes:

1. the Wiener process reflected about the \(x\)-axis: \(X_{t}:=-W_{t}\);

2. the Wiener process with rescaled time: \(X_{t}:=\sqrt{c} W_{t / c}, c>0\);

3. the Wiener process started at time \(t_0 > 0\): \(X_{t}:=W_{t+t_0}-W_{t_0}\), where \(t_0 > 0\) is an arbitrary fixed moment of time;

4. the Wiener process reflected at the point \(t_0 > 0\):

\[ X_t := \begin{cases} W_t, & t < t_0 \\ 2W_{t_0} - W_t, & t \geqslant t_0 \end{cases}, \]

where \(t_0 > 0\) is an arbitrary fixed moment of time.

ProblemProblem 2

Let \(\left(B_{t}, t \in [0,1]\right)\) be an almost surely continuous Gaussian process with zero mean function and covariance function \(\operatorname {Cov}\left[ B_t, B_s \right]=\min (s, t)-st\).

1. Prove that such a process exists;

2. Prove that the process

   \[ \tilde{W}_s := (s+1)B_{\frac{s}{s+1}}, \qquad s \geq 0 \]

   is a Wiener process.

3. Is \(B_t\) a process with independent increments?

The process \((B_t, t \in [0,1])\) is called a Brownian bridge, since its variance is largest at the point \(\frac{1}{2}\), while at the endpoints of \([0,1]\) it is almost surely equal to zero.

HintHint

You know that the Wiener process exists. Use this. Part (2) gives a hint as to how to express the Brownian bridge in terms of the Wiener process.

ProblemProblem 3

1. Let \(\left(W_{t}, t \in [0,1]\right)\) be a Wiener process on \([0,1]\). Prove that \(B_t = W_t - tW_1\) is a Brownian bridge;

2. Let \(\left(B_{t}, t \in [0,1]\right)\) be a Brownian bridge, and let \(Z \sim \mathscr {N}\left(0, 1\right)\) be a standard normal random variable independent of \((B_t)\). Prove that \(W_t := B_t + tZ\) is a Wiener process on \([0,1]\).

3 The Wiener process: path properties

Rigorous proofs of the main facts of this section can be found in (Булинский and Ширяев 2005, III).

3.1 Local properties of paths

By definition, the Wiener process has almost surely continuous paths. On the outcomes where the paths are nonetheless discontinuous, the process can be redefined to be zero. This does not change the finite-dimensional distributions, but now all paths are continuous. When all paths are continuous, this somewhat simplifies our understanding.

Theorem 4 (Paley–Wiener–Zygmund) With probability 1 the path \(\left(W_{t}, t \geqslant 0\right)\) is nowhere differentiable on \(\mathbb {R}_{+}\).

The proof of this theorem is not easy; it can be found in (Булинский and Ширяев 2005, III, § 1).

Thus the Wiener process does indeed model chaotic motion: its typical path is continuous but not at all smooth (similar to the Weierstrass function).

Mn-extraAdditional material

Let us deepen our understanding of the continuity of paths.

Theorem 5 Let \(W_t\) be Brownian motion.

• For any \(\gamma \in (0,\frac{1}{2})\), almost surely all paths of Brownian motion are Hölder continuous with exponent \(\gamma\) on any compact set.

• For any \(\gamma \in [\frac{1}{2},1]\), almost surely all paths of Brownian motion are not Hölder continuous with exponent \(\gamma\) on any compact set.

Recall that a function \(f: \mathbb {R}_+ \to \mathbb {R}\) is called Hölder continuous on a set \(A \subset \mathbb {R}_+\) with exponent \(\gamma > 0\) if there exists a constant \(C > 0\) such that

\[ \left|f(x) - f(y)\right| \leq C \cdot \left|x - y\right|^{\gamma } \]

for all \(x,y \in A\).

3.2 Variation of paths

For details see (Schilling 2021, chap. 9).

Let us recall the definition of \(p\)-variation: let \(f:[0, \infty ) \rightarrow \mathbb {R}^d\) be a (nonrandom) function, and let

\[ \Delta = \left\{ t_0, t_1, \ldots , t_n\right\} , \qquad \text{where } \qquad 0 = t_0 < t_1 < \ldots < t_{n} = t \]

be a finite partition of the interval \([0, t]\). By \(\left|\Delta \right|:=\max_{1 \leqslant j \leqslant n}\left(t_j-t_{j-1}\right)\) we denote the mesh of the partition. For \(p>0\) let

\[ Q_p(f, [0,t], \Delta ) := \sum _{i=1}^{n}\left|f(t_i) - f(t_{i-1})\right|^p \]

be the sum of \(p\)-variations over the partition \(\Delta\). The supremum over all finite partitions is the strong or total \(p\)-variation of the function \(f\) on the interval \([0,t]\):

\[ \operatorname {VAR}_p(f, \; [0,t]) := \sup _{\Delta } Q_p(f, [0,t], \Delta ) \]

If \(\operatorname {VAR}_1(f ; [0, t])<\infty\), then \(f\) is said to be a function of bounded or finite (total) variation.

The notion of \(p\)-variation along a given sequence of finite partitions \(\left(\Delta_k\right)_{k \geqslant 1}\) of the interval \([0, t]\) with \(\lim_{k \rightarrow \infty }\left|\Delta_k\right|=0\) is often used:

\[ \operatorname {var}^{\left\{ \Delta _k\right\} }_p(f ;[0, t]):=\lim _{\left|\Delta _k\right| \rightarrow 0} Q_p(f ;[0, t] ; \; \Delta _k) . \]

Note that \(\operatorname {VAR}_p(f ;[0, t])\) is always defined (i.e., for any function, for any \(p > 0\), and for any interval), unlike \(\operatorname {var}^{\left\{ \Delta_k\right\} }\), which may diverge. Although, of course, \(\operatorname {VAR}_p(f ;[0, t])\) may take the value \(+\infty\). If \(\operatorname {var}^{\left\{ \Delta_k\right\} }_p(f ;[0, t])\) does exist (along some sequence of partitions \(\left\{ \Delta_k\right\}_{k \in \mathbb {N}}\) of the interval \([0,t]\)), then

\[ \operatorname {VAR}_p(f ;[0, t]) \; \geqslant \; \operatorname {var}^{\left\{ \Delta _k\right\} }_p(f ;[0, t]) \]

When instead of \(f(s)\) we use a random function \(X(s, \omega )\), the quantity \(\operatorname {var}_p(X(\cdot , \omega ) ;[0, t])\) may be a limit in distribution, in probability, in \(L^q\), or almost surely. Here the sequence of partitions \(\left(\Delta_k\right)_{k \geqslant 1}\) must not depend on \(\omega\).

3.2.1 1-variation

A well-known result from analysis states that a function of bounded variation is differentiable almost everywhere. Since we know from the previous section that almost no Brownian path is differentiable anywhere, we can conclude that the paths of BM do not have bounded variation almost surely.

Theorem 6 Almost surely all paths of BM have the following property: on any bounded interval \([0,t]\) they have infinite \(1\)-variation.

Finiteness of \(1\)-variation means that the path is rectifiable. Consequently, almost no path of BM is rectifiable; that is, the graph has infinite length on any bounded interval.

Corollary 1 Moreover, almost surely all paths of BM have infinite \(p\)-variation on any bounded interval for any \(p < 2\): \[ \operatorname {VAR}_p(W;[0,t]) = +\infty \quad \text{a.s.}, \qquad \forall p < 2, \; \forall t > 0. \]

Thus the theorem above is merely the special case \(p=1\). This is no longer proved via non-differentiability but directly via the quadratic variation (see below): the maximal increment over the partition tends to \(0\) almost surely by uniform continuity of the path on \([0,t]\), and the sum of \(p\)-variations for \(p=2-\delta\) is bounded above by (this maximum)\(^\delta\) times the sum of the squared increments — which remains bounded (see (Schilling 2021, Corollary 9.2)).

3.2.2 Quadratic variation

We know that Brownian paths are continuous functions with poor behavior. They are not differentiable and not rectifiable. If we try to compute the length of the path \(W(t), 0 \leq t \leq 1\),

\[ Q_1(W_t, [0,t], \Delta ) := \sum _{i=1}^{n}\left|W_{t_i} - W_{t_{i-1}}\right| \]

If \(\left|\Delta \right|\) is small, each individual difference

\[ \left|W_{t_i} - W_{t_{i-1}}\right| = \left|W_{t_i}(\omega ) - W_{t_{i-1}}(\omega )\right| \]

is small by continuity, but not small enough for the sums to converge. If we shrink the individual differences, say by squaring each one, we might hope that the limit will be finite.

Theorem 7 Let \((W_t, t\geq 0)\) be a WP on some probability space \((\Omega , \mathcal{F}, \mathbb {P})\). Let \(t > 0\) be fixed. Then the following convergence holds: \[ Q_{2}(W_t; [0,t]; \Delta ) \quad \xrightarrow [\left|\Delta \right| \to 0]{L^2(\Omega )} t \] Namely, for any \(\varepsilon > 0\) there exists \(\delta > 0\) such that for any partition \(\Delta\) of \([0,t]\) with \(\left|\Delta \right| < \delta\) we have \[ \left\| \; Q_{2}(W_t; [0,t]; \Delta ) \; \; - \; \; t\; \right\| _{L^2} \quad < \quad \varepsilon \]

Mn-proofProof

Let \(\Delta = \left\{ 0=t_0 < t_1 < \ldots < t_n = t\right\}\) be a partition of \([0,t]\). Note that \(t = \sum_{i=1}^{n}(t_i - t_{i-1})\). Then

\[ \begin{align} \mathbb {E}\left[\left(Q_2(W_t;[0,t];\Delta ) - t\right)^2\right] & = \\ & \quad \mathbb {E}\left[\left(\sum _{i=1}^{n} \left((W_{t_i} - W_{t_{i-1}})^2 - (t_i - t_{i-1})\right)\right)^2\right] = \\ & = \sum _{i=1}^{n} \mathbb {E}\left[\left((W_{t_i} - W_{t_{i-1}})^2 - (t_i - t_{i-1})\right)^2\right], \end{align} \]

where in the second equality we used the fact that the terms \((W_{t_i} - W_{t_{i-1}})^2 - (t_i - t_{i-1})\) are independent (since the increments of WP on disjoint intervals are independent) and have zero mean, so all cross terms vanish when we expand the square of the sum.

If \(X \sim \mathscr {N}\left(0, \sigma^2\right)\), then

\[ \mathbb {E}\left[(X^2 - \sigma ^2)^2\right] = \mathbb {E}\left[X^4\right] - 2 \sigma ^2 \mathbb {E}\left[X^2\right] + \sigma ^4 = 3\sigma ^4 - 2\sigma ^4 + \sigma ^4 = 2 \sigma ^4, \]

since \(\mathbb {E}\left[X^4\right] = 3\sigma^4\). Applying this to \(X = W_{t_i} - W_{t_{i-1}} \sim \mathscr {N}\left(0, t_i - t_{i-1}\right)\), we get

\[ \begin{align} \mathbb {E}\left[\left(Q_2(W_t;[0,t];\Delta ) - t\right)^2\right] & = \sum _{i=1}^{n} 2 (t_i - t_{i-1})^2 \leq \\ & \leq 2\left|\Delta \right| \sum _{i=1}^{n} (t_i - t_{i-1}) = 2\left|\Delta \right| \cdot t \xrightarrow [\left|\Delta \right| \to 0]{} 0. \end{align} \]

Mn-remarkRemark

The theorem above gives convergence \(Q_2(W_t;[0,t];\Delta ) \to t\) in \(L^2\) along an arbitrary sequence of partitions with \(\left|\Delta_k\right|\to 0\). For almost sure convergence, an arbitrary sequence is no longer enough — additional conditions on how the partitions are structured are needed. Below are three sufficient conditions of a different nature (without proofs, see (Schilling 2021, 9.2)).

Theorem 8 (Lévy, 1948) Let \((\Delta_k)_{k\geq 1}\) be a sequence of partitions of \([0,t]\) such that \(\sum_{k=1}^{\infty }\left|\Delta_k\right| < \infty\). Then \[ Q_2(W_t;[0,t];\Delta _k) \xrightarrow [k\to \infty ]{\text{a.s.}} t. \]

This is proved via Chebyshev’s inequality (the same bound on \(\mathbb {P}\left(\left|Q_2(\Delta )-t\right|>\varepsilon \right)\) as in the proof of the theorem above) and the Borel–Cantelli lemma: since \(\sum_k \left|\Delta_k\right|<\infty\), we also have \(\sum_k \mathbb {P}\left(\left|Q_2(\Delta_k)-t\right|>\varepsilon \right) < \infty\) for every \(\varepsilon >0\).

Theorem 9 (Dudley, 1973) The summability condition can be weakened: if \(\left|\Delta_k\right| = o\left(1/\log k\right)\), then we still have \[ Q_2(W_t;[0,t];\Delta _k) \xrightarrow [k\to \infty ]{\text{a.s.}} t. \]

The proof is harder (exponential moments and choosing a parameter in the spirit of Chernoff’s inequality), but in essence it is the same method: bound the deviation probabilities and sum via Borel–Cantelli.

Theorem 10 (Lévy, 1940) If \((\Delta_k)_{k\geq 1}\) is a nested sequence of partitions (\(\Delta_k \subset \Delta_{k+1}\)) with \(\left|\Delta_k\right|\to 0\), then \[ Q_2(W_t;[0,t];\Delta _k) \xrightarrow [k\to \infty ]{\text{a.s.}} t \] with no conditions whatsoever on the rate of decay of \(\left|\Delta_k\right|\).

Here the idea is different: the quantities \(Q_2(\Delta_k)\), when viewed in reverse order (\(k \to \infty\) instead of \(k \to 1\)), form a reverse martingale, and reverse martingales have their own theorem on almost sure convergence.

Mn-remarkRemark

It is important not to confuse \(\operatorname {var}_2^{\{ \Delta_k\} }\) along a specific sequence of partitions (which is what was discussed above — the limit is always equal to \(t\)) with the strong variation \(\operatorname {VAR}_2\) — the supremum over all partitions, including partitions depending on \(\omega\). These are fundamentally different things:

Theorem 11 (Lévy, 1948) For almost every \(\omega\) there exists its own (depending on \(\omega\)) nested sequence of partitions \((\Delta_k(\omega ))_{k\geq 1}\) of \([0,1]\) with \(\left|\Delta_k(\omega )\right| \to 0\), along which \[ Q_2(W(\cdot ,\omega ); [0,1]; \Delta _k(\omega )) \xrightarrow [k\to \infty ]{} +\infty . \]

Corollary 2 Consequently, the strong quadratic variation of BM is infinite almost surely: \[ \operatorname {VAR}_2(W(\cdot ,\omega );[0,1]) = +\infty \qquad \text{for almost every } \omega \in \Omega . \]

Here is the promised example of the strict inequality \(\operatorname {VAR}_p > \operatorname {var}_p^{\{ \Delta_k\} }\) from the beginning of the section: along “good” sequences \(\operatorname {var}_2 = t < \infty\), but the supremum over all partitions equals \(+\infty\). The meaning: if a partition is chosen to fit a specific path (rather than fixed in advance), there will always be “fast” segments where the increment accumulates faster than \(\sqrt{\text{length}}\).

Theorem 12 (Friedman, 1971) Symmetrically, for almost every \(\omega\) there also exists its own nested sequence of partitions \((\Delta_k'(\omega ))_{k\geq 1}\) of \([0,1]\) with \(\left|\Delta_k'(\omega )\right|\to 0\), along which \[ Q_2(W(\cdot ,\omega );[0,1];\Delta _k'(\omega )) \xrightarrow [k\to \infty ]{} 0. \]

That is, by playing with partitions, the quadratic variation of one and the same path can be “adjusted” to be anything — either to \(0\) or to \(+\infty\). Only for partitions fixed in advance (not tailored to \(\omega\)) is the answer always the same — \(t\) (the three theorems above).

Corollary 3 The theorem on quadratic variation holds not only for WP but for any continuous martingale \((X_t, \mathcal{F}_t)_{t\geq 0}\), \(X_0=0\), such that \((X_t^2 - t, \mathcal{F}_t)_{t\geq 0}\) is also a martingale: for any sequence of partitions \(\Delta_k\) of \([0,t]\) with \(\left|\Delta_k\right|\to 0\) \[ Q_2(X;[0,t];\Delta _k) \xrightarrow [k\to \infty ]{L^2(\Omega )} t. \]

The proof repeats almost verbatim the proof of the theorem for WP: instead of independence of increments one uses the martingale property (the conditional expectation of the product of “disjoint” increments still vanishes by the tower property).

Theorem 13 (Lévy’s martingale characterization, 1948) Let \((X_t, \mathcal{F}_t)_{t\geq 0}\), \(X_0=0\), be a continuous martingale such that \((X_t^2-t,\mathcal{F}_t)_{t\geq 0}\) is also a martingale. Then \((X_t)_{t\geq 0}\) is a Wiener process.

This is the converse of the fact from corollary Corollary 3: the property “quadratic variation is linear and equal to \(t\)” is not merely a consequence of the process being a WP, but its complete characterization among continuous martingales. This very theorem underlies many constructions in stochastic analysis — for example, proofs that some stochastic integral or limit of processes is a Wiener process.

ProblemProblem 4

Let \(\left(W_{t}, t \geq 0\right)\) be a Wiener process, \([a,b] \subset \mathbb {R}_+\). Let \((\Delta_n)_{n \geq 1}\) be a sequence of partitions of \([a,b]\) with \(\left|\Delta_n\right| \xrightarrow [n \to +\infty ]{} 0\). Set \[ U_{n} := Q_2(W, [a,b], \Delta _n) \] Find the limit in mean square (in \(L^2\)) of the quantities \(U_{n}\) as \(n \rightarrow +\infty\).

3.3 Limiting properties of paths

3.3.1 LLN

Let us give an analogue of the law of large numbers (LLN) for the Wiener process: for \(t > 0\) we have

\[ \frac{W_t}{t} \sim \mathscr {N}\left(0, \frac{t}{t^2}\right) = \mathscr {N}\left(0, \frac{1}{t}\right) \xrightarrow [t \to \infty ]{} 0 \]

Hence

\[ \frac{W_t}{t} \xrightarrow [t \to \infty ]{\mathbb {P}, d} 0 \]

Moreover, a strengthened (path-wise) LLN turns out to hold:

Lemma 2 Let \((W_t, t \geq 0)\) be a Wiener process. Then \[ \frac{W_t}{t} \xrightarrow [t \to \infty ]{\text{a.s.}} 0 \]

3.3.2 CLT

On the other hand, for the Wiener process one can state an analogue of the central limit theorem (CLT): for \(t > 0\) we have \(\frac{W_t}{\sqrt{t}} \sim \mathscr {N}\left(0, 1\right)\). That is, we can write \(\frac{W_t}{\sqrt{t}} \xrightarrow [t \to \infty ]{d} \mathscr {N}\left(0, 1\right)\). But what happens to the paths of \(\frac{W_t}{\sqrt{t}}\)?

Lemma 3 The paths of \(\frac{W_t}{\sqrt{t}}\) almost surely have set of limit points equal to \(\mathbb {R}\). Equivalently: \[ \mathbb {P}\left(\varlimsup _{n \to \infty } \frac{W_t}{\sqrt{t}} = +\infty \right) = \mathbb {P}\left(\varliminf _{n \to \infty } \frac{W_t}{\sqrt{t}} = -\infty \right) = 1 \]

As a consequence we obtain that \(\frac{W_t}{\sqrt{t}}\) cannot converge almost surely, i.e., there is no random variable \(\xi\) such that \(\frac{W_t}{\sqrt{t}} \xrightarrow [t \to +\infty ]{\text{a.s.}} \xi\). It turns out that this also holds for convergence in probability.

Lemma 4 There is no random variable \(\xi\) such that \[ \frac{W_{t}}{\sqrt{t}} \; \xrightarrow [t \to +\infty ]{\mathbb {P}}\; \xi \]

3.3.3 The law of the iterated logarithm

The law of the iterated logarithm is a statement operating “between” the analogue of the law of large numbers (LLN) for the Wiener process and the analogue of the central limit theorem (CLT) for the Wiener process.

Theorem 14 The law of the iterated logarithm holds: \[ \begin{align} \mathbb {P}\left(\varlimsup _{t \to +\infty } \frac{W_{t}}{\sqrt{2 t \ln \ln t}} = 1\right) & = \mathbb {P}\left(\varliminf _{t \to +\infty } \frac{W_{t}}{\sqrt{2 t \ln \ln t}} = -1\right) \\ & = 1 \end{align} \] That is, almost surely, for all elementary outcomes \(\omega \in \Omega\), the set of limit points of the continuous function \(\frac{W_t(\omega )}{\sqrt{2t \ln \ln (t)}}\) is equal to \([-1,1]\).

The proof of the law of the iterated logarithm is rather difficult; it can be found in (Булинский and Ширяев 2005, III, § 11). Here we shall instead discuss its meaning.

According to this theorem, almost all paths of the Wiener process remain inside a widening “tube” between the curves \(\pm (1 + \varepsilon )\sqrt{2t \ln \ln (t)}\) (for any \(\varepsilon > 0\), starting from some \(t_0(\varepsilon , \omega )\)). At the same time, with probability 1 they infinitely often “escape” the “tube” with boundary \(\pm (1 - \varepsilon )\sqrt{2t \ln \ln (t)}\).

A single trajectory of the Wiener process (log scale in t) and the tube ±(1±ε)√(2t ln ln t). By the law of the iterated logarithm, starting from some t₀(ε,ω), the trajectory stays inside the outer tube (solid) but exits the inner one (dashed) infinitely often — move the slider ε to see this for different ε > 0.

Let us try to gather all the limiting properties of the Wiener process we have obtained into a single table.

		Type of convergence (in decreasing order of strength)
		Pathwise convergence, i.e. a.s.	Convergence in probability	Convergence in distribution
Scaling functions (in decreasing order of dominance)	\(\displaystyle t\)	\(\displaystyle \overset{\text{a.s.}}{\lim_{t\to+\infty}}\ \frac{W_t}{t}=0\)	\(\displaystyle \overset{\mathbb P}{\lim_{t\to+\infty}}\ \frac{W_t}{t}=0\)	\(\displaystyle \overset{d}{\lim_{t\to+\infty}}\ \frac{W_t}{t}=0\)
	\(\displaystyle \sqrt{t}\times\sqrt{2\ln\ln(t)}\)	\(\displaystyle \overset{\text{a.s.}}{\lim_{t\to+\infty}}\ \frac{W_t}{\sqrt{2t\ln\ln(t)}}=[-1,1]\)	\(\displaystyle \overset{\mathbb P}{\lim_{t\to+\infty}}\ \frac{W_t}{\sqrt{2t\ln\ln(t)}}=0\)	\(\displaystyle \overset{d}{\lim_{t\to+\infty}}\ \frac{W_t}{\sqrt{2t\ln\ln(t)}}=0\)
	\(\displaystyle \sqrt{t}\)	\(\displaystyle \nexists\,\xi:\ \overset{\text{a.s.}}{\lim_{t\to+\infty}}\ \frac{W_t}{\sqrt{t}}=\xi\)	\(\displaystyle \nexists\,\xi:\ \overset{\mathbb P}{\lim_{t\to+\infty}}\ \frac{W_t}{\sqrt{t}}=\xi\)	\(\displaystyle \overset{d}{\lim_{t\to+\infty}}\ \frac{W_t}{\sqrt{t}}=\mathcal N(0,1)\)

Table 1.1: Limiting properties of the Wiener process. WARNING! Nonstandard notation! The expression \(\lim_{t\to+\infty} g(t) = A\) for a function \(g:\mathbb{R}_+\to\mathbb{R}\) and a set \(A\subset\mathbb{R}\) means that \(A\) is the set of all limit points of \(g(t)\) as \(t\to+\infty\).

Let us discuss the meaning of the law of the iterated logarithm from a different angle. If in the LLN we scaled the Wiener process by the function \(t\), and in the CLT by the function \(\sqrt{t}\), then here we divide the Wiener process by the function \(\sqrt{2t \ln \ln (t)}\). Note that \(t \gg \sqrt{2t \ln \ln (t)} \gg \sqrt{t}\) as \(t \to +\infty\), i.e.,

\[ \sqrt{2t \ln \ln (t)} = o\left(t\right), \quad \sqrt{t} = o\left(\sqrt{2t \ln \ln (t)}\right) \quad \text{as } \; t \to +\infty . \]

20 trajectories of the Wiener process and three scaling functions: t (black, dash-dot), √(2t ln ln t) (orange), and √t (gray). Move the slider T: the longer the horizon, the more apparent that t is big-O of the bundle, while the bundle itself grows like √t, tracking √(2t ln ln t).

Next, note that if \(f(t) = o\left(\sqrt{2t \ln \ln (t)}\right)\) is some positive function dominated by \(\sqrt{2t \ln \ln (t)}\), then the law of the iterated logarithm implies

\[ \begin{align} \frac{W_t}{f(t)} & = \underbrace{\frac{\sqrt{2t \ln \ln (t)}}{f(t)}}_{\to +\infty \text{ a.s.}} \underbrace{\frac{W_t}{\sqrt{2t \ln \ln (t)}}}_{\to [-1,1] \text{ a.s.}} \\ & \quad \xrightarrow [t \to +\infty ]{\text{a.s.}} \mathbb {R} \end{align} \]

If, on the other hand, \(g(t)\) is some positive function dominating \(\sqrt{2t \ln \ln (t)}\), i.e., \(\sqrt{2t \ln \ln (t)} = o\left(g(t)\right)\), then

\[ \begin{align} \frac{W_t}{g(t)} & = \underbrace{\frac{\sqrt{2t \ln \ln (t)}}{g(t)}}_{\to 0 \text{ a.s.}} \underbrace{\frac{W_t}{\sqrt{2t \ln \ln (t)}}}_{\to [-1,1] \text{ a.s.}} \\ & \quad \xrightarrow [t \to +\infty ]{\text{a.s.}} 0. \end{align} \]

Thus, as \(t \to +\infty\), the function \(\sqrt{2t \ln \ln (t)}\) is precisely the function between \(\sqrt{t}\) and \(t\) such that, when scaling by it, the paths of the Wiener process almost surely have a nontrivial set of limit points: neither \(\left\{ 0\right\}\) nor \(\mathbb {R}\).

The same trajectory of the Wiener process under three normalizations (log scale in t, the start is cut off at t < 0.4 — there all three ratios blow up). Dividing by t (black) forces the trajectory to zero — this is the SLLN. Dividing by √t (blue) — nothing forces it, the oscillations remain of order 1. The red curve W_t/√(2t ln ln t) is the intermediate LIL regime: on an infinite horizon it is precisely this one whose set of limit points is [-1,1]. Move the slider T.

On the other hand, if a positive function \(f(t)\) dominates \(\sqrt{t}\), i.e., \(\sqrt{t} = o\left(f(t)\right)\), then

\[ \begin{align} \frac{W_t}{f(t)} & = \underbrace{\frac{\sqrt{t}}{f(t)}}_{\to 0} \cdot \underbrace{\frac{W_t}{\sqrt{t}}}_{ \sim \mathscr {N}\left(0, 1\right)} \\ & \quad \xrightarrow [t \to +\infty ]{d,\mathbb {P}} 0 \end{align} \]

Consequently, as \(t \to +\infty\), the function \(\sqrt{2t \ln \ln (t)}\) is precisely the function \(f(t)\) between \(\sqrt{t}\) and \(t\) such that, when scaling by it,

\[ \begin{align} \frac{W_t}{f(t)} & \; \xrightarrow [t \to +\infty ]{d,\mathbb {P}} \; 0, \\ & \quad \text{ but } \quad \frac{W_t}{f(t)} \; \stackrel{\text{a.s.}}{\not\to } \; 0 \quad \text{ as } \quad t \to +\infty \end{align} \]

The same bundle of 20 trajectories of the Wiener process under three normalizations (one trajectory is highlighted in all panels; log scale in t, the start t < 0.4 is cut off). Left: W_t/√t — the spread neither shrinks nor grows (the CLT regime). Center: W_t/√(2t ln ln t) — the bundle gradually presses against the band [-1,1] (dashed) — this is the LIL. Right: W_t/t — the bundle collapses to zero (the SLLN). Move the shared horizon slider T.

ProblemProblem 5

Prove that \(\frac{W_t}{\sqrt{t}} \xrightarrow [t \to +\infty ]{\text{a.s.}} \mathbb {R}\), i.e., almost surely the set of limit points of the path \(\frac{1}{\sqrt{t}}W_{t}(\omega )\), \(t\in \mathbb {R}_+\), is \(\mathbb {R}\).

1. You may use the law of the iterated logarithm;

2. You may not use the law of the iterated logarithm.

ProblemProblem 6

Prove that \(\frac{W_t}{t} \xrightarrow [t \to +\infty ]{\text{a.s.}} 0\),

1. using the law of the iterated logarithm;

2. without using the law of the iterated logarithm.

ProblemProblem 7

Let \((W_t, t \geq 0)\) be a Wiener process. Prove that there is no random variable \(\xi\) such that \[ \frac{W_t}{\sqrt{t}} \; \xrightarrow [t \to +\infty ]{\mathbb {P}} \; \xi \]

HintHint

Consider \(\frac{W_{2t}}{\sqrt{2t}} - \frac{W_t}{\sqrt{t}}\) as \(t \to +\infty\).

ProblemProblem 8

Prove that \(\frac{W_t}{\sqrt{2t \ln \ln (t)}} \xrightarrow [t \to +\infty ]{\mathbb {P}, d} 0\).

ProblemProblem 9

Using the law of the iterated logarithm, prove the local law of the iterated logarithm for the Wiener process: \[ \frac{W_t}{\sqrt{2t \ln \ln (1/t)}} \; \xrightarrow [t \to 0+]{\text{a.s.}} \; [-1,1] \]

4 Construction of the Wiener process

Below we shall mostly construct the Wiener process on the interval \([0,1]\). But from a Wiener process on \([0,1]\) one can obtain a Wiener process on \(\mathbb {R}_+\).

ProblemProblem 10

Let \((W_t, t \in [0,1])\) be a Wiener process on \([0,1]\). How can one obtain from \(W_t\) a Wiener process \((W'(t), t \in \mathbb {R}_+)\) on all of \(\mathbb {R}_+\)?

4.1 Kolmogorov’s construction

Let us take a closer look at the definition of WP Definition 3. The first 3 conditions uniquely (you will need to prove this) determine the finite-dimensional distributions of the process, and hence the distribution of the process itself. The 4th condition imposes restrictions on the paths. We have a theorem on the existence of processes with independent increments that allows us to construct processes with independent increments, as well as Kolmogorov’s theorem on continuous modification, which allows us to obtain a modification (i.e., preserving the distribution) of a process with continuous paths. Both were proved by Kolmogorov. This is precisely Kolmogorov’s construction of the Wiener process.

ProblemProblem 11

1. Prove that there exists a process satisfying conditions 1–3 of Definition 3.

2. Prove that the Wiener process exists.

HintHint

Enum-item(1)

Use Kolmogorov’s theorem on the existence of processes with independent increments.

Enum-item(2)

Use part (1) and Kolmogorov’s theorem on continuous modification.

4.2 Construction via a series of continuous functions with random coefficients

The construction from the previous section is difficult to visualize; it is theoretical in nature. The construction in this section is more illustrative. Let us recall some theory.

4.2.1 The Hilbert space \(L^2[0,1]\)

We assume that we are working over the field of real numbers \(\mathbb {R}\).

\(L^2[0,1] = L^2\left([0,1], \mathscr {B}\left([0,1]\right), \lambda \right)\) is the Lebesgue space, i.e., the space of functions from \([0,1]\) to \(\mathbb {R}\) (more precisely, equivalence classes of functions under \(\lambda\)-almost-everywhere equality) that are square integrable:

\[ \int f^2 \; d\lambda = \int _{[0,1]}f^2(x) \; \lambda (dx) < \infty \]

On it we introduce the form

\[ \left\langle f, g \right\rangle := \int _{[0,1]} f(x) \cdot g(x) \; \lambda (dx) \]

satisfying the axioms of an inner product. Moreover, with respect to this inner product (or rather, the metric it generates), the space \(L^2[0,1]\) is complete, i.e., a Hilbert space.

4.2.2 Orthonormal basis

Consider a family of vectors \(\left\{ e_n\right\}\), indexed by some countable set. Without loss of generality, we will assume \(n \in \mathbb {N} = \left\{ 1,2,\ldots \right\}\), although in practice it is sometimes convenient to take \(n \in \mathbb {Z}_+ = \left\{ 0,1,2,\ldots \right\}\) or \(n \in \mathbb {Z}\).

A family \(\left\{ e_n\right\}_{ n \in \mathbb {N}}\) is called an orthonormal basis (abbreviated ONB) in a Hilbert space \(V\) if

• the vectors are orthonormal:

  \[ \left\langle e_i, e_j \right\rangle = \begin{cases} 1, & i = j\\ 0, & i \neq j \end{cases} \]

• the system is complete, i.e., any vector in \(V\) can be approximated by finite linear combinations from it. In other words,

  \[ V = \overline{\operatorname {span}\left(\left\{ e_n\right\} _{n \in \mathbb {N}}\right)} \]

If \(\left\{ e_n\right\}_{n \in \mathbb {N}}\) is an ONB in \(V\), then any vector \(f \in V\) has a unique representation

\[ f = \sum _{n=1}^{+\infty } \alpha _n \cdot e_n, \]

where \(\alpha_n\) is determined as: \(\alpha_n = \left\langle f, e_n \right\rangle\). Moreover, we have the following:

Lemma 5 (Parseval’s equality) If \(\left\{ e_n\right\}_{n \in \mathbb {N}}\) is an ONB in a Hilbert space \(V\), and \(f, g \in V\) are arbitrary vectors, then the inner product of \(f\) and \(g\) can be computed as the sum of the products of their expansion coefficients with respect to the ONB \(\left\{ e_n\right\}\): \[ \left\langle f, g \right\rangle = \sum _{n=1}^{\infty } \left\langle f, e_n \right\rangle \cdot \left\langle g, e_n \right\rangle \] In particular, \[ \left\| f\right\| = \sqrt{ \left\langle f, f \right\rangle } = \sqrt{\sum _{n=1}^{\infty } \left\langle f, e_n \right\rangle ^2} \]

4.2.3 Well-known ONBs in \(L^2[0,1]\)

Let us recall some well-known orthonormal bases in \(L^2[0,1]\).

• The standard ONB for the Fourier series:

  \[ e_{n}(t) \quad =\quad \begin{cases} \sqrt{2}\sin (2n\pi t), & n \geq 1 \\ 1, & n = 0 \\ \sqrt{2}\cos (-2n\pi t), & n \leq -1 \end{cases} \qquad =\qquad \begin{cases} \cdots , & \cdots \\ \sqrt{2}\sin (6 \pi t), & n = 3 \\ \sqrt{2}\sin (4 \pi t), & n = 2 \\ \sqrt{2}\sin (2\pi t), & n = 1 \\ 1, & n = 0 \\ \sqrt{2}\cos (2\pi t), & n = -1 \\ \sqrt{2} \cos (4\pi t), & n = -2 \\ \cdots , & \cdots \end{cases} \]

  \(n \in \mathbb {Z}\);

• The cosine ONB:

  \[ e_{n}(t) \quad =\quad \begin{cases} 1, & n = 0 \\ \sqrt{2}\cos (n\pi t), & n \geq 1 \end{cases} \qquad =\qquad \begin{cases} 1, & n=0 \\ \sqrt{2}\cos \left(\pi t\right), & n =1 \\ \sqrt{2}\cos \left(2\pi t\right), & n =2 \\ \sqrt{2}\cos \left(3\pi t\right), & n =3 \\ \vdots \end{cases} \]

  \(n \in \mathbb {Z}_+\);

• The other-cosine ONB:

  \[ e_{n}(t) \quad =\quad \sqrt{2}\cos \left(\left(n - \frac{1}{2}\right) \pi t\right)\qquad =\qquad \begin{cases} \sqrt{2}\cos \left(\frac{1}{2}\pi t\right), & n=1 \\ \sqrt{2}\cos \left(\frac{3}{2}\pi t\right), & n =2 \\ \sqrt{2}\cos \left(\frac{5}{2}\pi t\right), & n =3 \\ \vdots \end{cases} \]

  \(n \in \mathbb {N}\);

• The sine ONB:

  \[ e_{n}(t) \quad =\quad \sqrt{2}\sin \left(n \pi t\right)\qquad =\qquad \begin{cases} \sqrt{2}\sin \left(\pi t\right), & n=1 \\ \sqrt{2}\sin \left(2\pi t\right), & n =2 \\ \sqrt{2}\sin \left(3\pi t\right), & n =3 \\ \vdots \end{cases} \]

  \(n \in \mathbb {N}\);

• The other-sine ONB:

  \[ e_{n}(t) \quad =\quad \sqrt{2}\sin \left(\left(n - \frac{1}{2}\right) \pi t\right)\qquad =\qquad \begin{cases} \sqrt{2}\sin \left(\frac{1}{2}\pi t\right), & n=1 \\ \sqrt{2}\sin \left(\frac{3}{2}\pi t\right), & n =2 \\ \sqrt{2}\sin \left(\frac{5}{2}\pi t\right), & n =3 \\ \vdots \end{cases} \]

  \(n \in \mathbb {N}\);

• The Haar ONB: the system of functions \(\left\{ 1\right\} \cup \left\{ \psi_{m,k} \; : \; m \in \mathbb {Z}_+, \; 0 \leq k < 2^{m}\right\}\), i.e., the system consisting of the function identically equal to \(1\) and the functions \(\psi_{m,k}\), which can be defined inductively:

  \[ \psi _{0,0}(t) = \begin{cases} 1, & 0 < t < \frac{1}{2} \\ -1 & \frac{1}{2} < t < 1 \\ 0, & \text{ otherwise} \end{cases}, \qquad \psi _{m,k}(t) = 2^{m/2}\psi _{0,0}(2^mt - k), \quad m \in \mathbb {N}, \; 0 \leq k < 2^{m} \]

  Or explicitly:

  \[ \psi _{m,k}(t) = \begin{cases} 2^{m/2}, & \frac{k}{2^{m}} < t < \frac{2k+1}{2^{m+1}} \\ -2^{m/2}, & \frac{2k+1}{2^{m+1}} < t < \frac{k+1}{2^{m}} \\ 0, & \text{ otherwise} \end{cases} \]

  The first several Haar functions, \(m=0,1,2\):

Haar functions ψₙ,ₖ(x) = ψ(2ⁿx − k), where ψ = +1 on [0,½), −1 on [½,1), 0 otherwise. The slider n sets the scale, the slider k the shift within the level (0 ≤ k < 2ⁿ).

4.2.4 The general construction of WP via a series with random coefficients

Let us return to the Wiener process. Let \(\left\{ e_n\right\}_{n \in \mathbb {N}}\) be an arbitrary ONB in \(L^2[0,1]\). Let us state right away the main idea: one can introduce the Wiener process as \[ W_{t} = W_t(\omega ) \qquad := \qquad \sum _{n=1}^{+\infty } \quad \underbrace{\xi _{n}(\omega )}_{\substack {\text{Provides}\\ \text{randomness}}} \; \cdot \; \underbrace{\tilde{e}_{n}(t)}_{\substack {\text{Provides} \\ \text{dependence}\\ \text{on time}}} \tag{1}\] where \(\xi_1, \xi_2, \ldots\) are i.i.d. random variables on some (common for all) probability space \((\Omega , \mathcal{F}, \mathbb {P})\), distributed as \(\mathscr {N}\left(0, 1\right)\), and \[ \tilde{e}_n(t) := \left\langle \; \mathbb {1}_{[0,t]}, e_n \right\rangle _{L^2[0,1]} = \int _0^t e_n(r) \; dr \] In other words, \(\left\{ \tilde{e}_n(t)\right\}\), for fixed \(t\), are the expansion coefficients of the function \(\; \mathbb {1}_{[0,t]}\) with respect to the ONB \(\left\{ e_n\right\}\).

We shall discuss formula Equation 1 from a strictly rigorous point of view a little later. For now, let us informally justify why \(\tilde{e}_n\) is exactly what appears in the formula. Suppose (without justification for now) that we can freely interchange the expectation and the infinite sum. In that case \(\mathbb {E}\left[W_t\right] = \sum_{n=1}^{+\infty }\mathbb {E}\left[\xi_n\right] \cdot e_n(t) = \sum_{n=1}^{+\infty } 0 \cdot e_n(t) = 0\) for all \(t\), i.e., this is a centered process. Next, the covariance function:

\[ \begin{align} \operatorname {Cov}\left[ W_s, W_t \right] & = \mathbb {E}\left[W_s \cdot W_t\right] = \\ & = \left\langle W_s , W_t \right\rangle _{L^2(\Omega )} = \\ & = \left\langle \sum _{n_1=1}^{+\infty }\xi _{n_1} \cdot \tilde{e}_{n_1}(s), \sum _{n_2=1}^{+\infty }\xi _{n_2} \cdot \tilde{e}_{n_2}(t) \right\rangle _{L^2(\Omega )} \stackrel{?}{=} \\ & \stackrel{?}{=} \sum _{n_1 = 1}^{+\infty } \sum _{n_2 = 1}^{+\infty } \left\langle \xi _{n_1} \cdot \tilde{e}_{n_1}(s) \; , \; \xi _{n_2} \cdot \tilde{e}_{n_2}(t) \right\rangle _{L^2(\Omega )} = \\ & = \sum _{n_1 = 1}^{+\infty } \sum _{n_2 = 1}^{+\infty } \underbrace{ \left\langle \xi _{n_1}, \xi _{n_2} \right\rangle _{L^2(\Omega )}}_{=\delta _{n_1}^{n_2}}\cdot \tilde{e}_{n_1}(s) \cdot \tilde{e}_{n_2}(t) = \\ & =\sum _{n=1}^{+\infty }\tilde{e}_n(s) \cdot \tilde{e}_{n}(t) = \\ & = \sum _{n=1}^{+\infty } \left\langle I_{[0,s]}, e_{n} \right\rangle _{L^2[0,1]} \cdot \left\langle I_{[0,t]}, e_{n} \right\rangle _{L^2[0,1]} = \text{ (Parseval's equality) } \\ & = \left\langle \; \mathbb {1}_{[0,s]}, \; \mathbb {1}_{[0,t]} \right\rangle _{L^2[0,1]} = \\ & = \int _{0}^{}\; \mathbb {1}_{[0,s]}(r) \cdot \; \mathbb {1}_{[0,t]}(r) \; dr = \int _0^{\min (s,t)} \; dr = \\ & = \min (s,t) \end{align} \]

The unjustified equality is marked with a question mark; everything else you should understand. In the end, we arrived at the fact that \(W_t\) has the required covariance function! Furthermore, \(W_t\) is a limit of linear combinations of independent Gaussian random variables, so presumably (a rigorous justification you should give below in the problem) it is also a Gaussian process.

4.2.5 Concrete examples

• Consider the cosine ONB:

  \[ e_n(t) = \begin{cases} 1, & n = 0 \\ \sqrt{2}\cos (n\pi t), & n \geq 1 \end{cases} \]

  \(n \in \mathbb {Z}_+\). Then \(\tilde{e}_0(t) = t\), and for \(n \geq 1\) we have

  \[ \tilde{e}_n(t) = \sqrt{2} \int _{0}^{t}\cos \left(n\pi r\right) \; dr = \frac{\sqrt{2}}{n\pi } \sin \left(n\pi t\right) \]

  In the end we obtain

  \[ \begin{align} W_t & = \sum _{n=1}^{+\infty } \xi _n \cdot \tilde{e}_n(t) = \\ & = \xi _0 t + \frac{\sqrt{2}}{\pi } \cdot \sum _{n=1}^{+\infty } \xi _n \cdot \frac{\sin \left(n \pi t\right)}{n} = \end{align} \]

Three classical ONB constructions of Brownian motion: partial sums \(W_t^{(N)} = \sum_n \xi_n \tilde e_n(t)\). Switch the construction in the dropdown on the left (Paley–Wiener, Kosambi–Karhunen–Loève, Lévy–Ciesielski), move the slider \(N\) to see how the sum increasingly accurately approximates the trajectory of the Wiener process, use the “Show more trajectories” slider to add several more independent realizations to it, and click the button to regenerate the random coefficients \(\xi_n\) (new trajectories). At the bottom left are the current basis functions \(e_n(t)\) and \(\tilde e_n(t)\) of the selected construction; in the bottom-left corner of the main plot is the sample \(\xi_0,\ldots,\xi_N\) itself, from which the main trajectory is built.

N = 20 Show more trajectories = 0

Paley-Wiener construction Kosambi-Karhunen-Loève construction Lévy–Ciesielski construction

.

This representation, the first of its kind, was obtained by Norbert Wiener back in 1923.

• Consider the other-cosine ONB:

  \[ e_n(t) = \sqrt{2}\cos \left(\left(n - \frac{1}{2}\right) \pi t\right), \qquad n \in \mathbb {N} = \left\{ 1,2,3,\ldots \right\} \]

  Then

  \[ \begin{align} \tilde{e}_n(t) & = \sqrt{2} \int _{0}^{t}\cos \left(\left(n - \frac{1}{2}\right) \pi r\right) \; dr = \\ & = \frac{\sqrt{2}}{\pi } \cdot \frac{1}{n - \frac{1}{2}} \sin \left(\left(n - \frac{1}{2}\right) \pi t\right) \end{align} \]

  In the end we obtain

  \[ \begin{align} W_t & = \sum _{n=1}^{+\infty } \xi _n \cdot \tilde{e}_n(t) = \\ & = \sqrt{2}\sum _{n=1}^{+\infty } \xi _n \cdot \frac{\sin \left(\left(n - \frac{1}{2}\right) \pi t\right)}{\left(n - \frac{1}{2}\right) \pi } \end{align} \]

  This expansion is, in a certain sense, optimal among all others of this type. For more details, see the Kosambi–Karhunen–Loève expansion.

• Consider the Haar ONB: \(\left\{ 1\right\} \cup \left\{ \psi_{m,k} \; : \; m \in \mathbb {Z}_+, \; 0 \leq k < 2^m\right\}\), where

  \[ \psi _{m,k}(t) = \begin{cases} 2^{m/2}, & \frac{k}{2^m} < t < \frac{2k+1}{2^{m+1}} \\ -2^{m/2}, & \frac{2k+1}{2^{m+1}} < t < \frac{k+1}{2^m} \\ 0, & \text{ otherwise} \end{cases} \]

  Then for the identical unit function: \(\tilde{1} = \int_0^t 1 \; dr = t\). For the remaining functions:

  \[ \tilde{\psi }_{m,k} = \int _{0}^t \psi _{m,k}(r) \; dr = \begin{cases} 0, & 0 \leq t < \frac{k}{2^m} \\ \text{linearly increasing from $0$ to $2^{-(m+2)/2}$}, & \frac{k}{2^m} < t < \frac{2k+1}{2^{m+1}} \\ 2^{-(m+2)/2}, & t = \frac{2k+1}{2^{m+1}} \\ \text{linearly decreasing from $2^{-(m+2)/2}$ to $0$}, & \frac{2k+1}{2^{m+1}} < t < \frac{k+1}{2^m} \\ 0, & \frac{k+1}{2^m} \leq t \leq 1 \end{cases} \]

  The system of functions \(\left\{ t\right\} \cup \left\{ \tilde{\psi }_{m,k} \; : \; m \in \mathbb {Z}_+, 0 \leq k < 2^m\right\}\) is called the Schauder functions ¹.

  In the end we obtain

  \[ W_t = \xi _0 t + \sum _{m=0}^{+\infty }\sum _{k=0}^{2^m - 1}\xi _{m,k}\tilde{\psi }_{m,k}(t) \]

  where \(\xi_0, \left\{ \xi_{m,k}\right\}\) is a countable collection of i.i.d. random variables, distributed as \(\mathscr {N}\left(0, 1\right)\). The Lévy–Ciesielski construction: partial sums over the Schauder functions \(W_t^{(N)} = \xi_0 t + \sum_{n=0}^{N}\sum_{k=0}^{2^n-1} \xi_{n,k}\tilde\psi_{n,k}(t)\), where \(\tilde\psi_{n,k}\) is the antiderivative of the Haar function \(\psi_{n,k}\) (see above). Move the slider \(N\) to see how adding progressively finer “details” — one new dyadic level at a time — brings the sum closer to a trajectory of the Wiener process, and click the button to regenerate the random coefficients \(\xi_0, \{\xi_{n,k}\}\) (a new trajectory).

N = 6

.

This construction is called the Lévy–Ciesielski construction. For more details, with references to the original papers, see (Schilling 2021, chap. 3).

4.2.6 Justification

So, we have the formula

\[ W_t = \sum _{n=1}^{\infty } \xi _n \cdot \tilde{e}_n(t) \]

where \(\xi_1,\xi_2, \ldots\) are i.i.d. random variables with distribution \(\mathscr {N}\left(0, 1\right)\) (defined on some common probability space \((\Omega , \mathcal{F}, \mathbb {P})\)), and \(\left\{ e_n\right\}_{n \in \mathbb {N}}\) is an ONB in \(L^2[0,1]\). So far it is unclear how to interpret this formula, because it is not entirely clear whether the given expression converges, and in what sense. Let us define the partial sums

\[ W^{(N)} = W^{(N)}_t := \sum _{n=1}^{N} \xi _n \cdot \tilde{e}_{n}(t) \]

Note that \(W^{(N)}\) (for fixed \(N\)) can be regarded as a random element of the Banach space \(C[0,1]\). Indeed, \(\tilde{e}_n\) is an integral with variable upper limit, i.e., a continuous function. And

\[ W^{(N)}: \omega \in \Omega \quad \mapsto \quad \sum _{n=1}^{N}\xi _n(\omega ) \cdot \tilde{e}_{n} \in C[0,1] \]

That is, \(W^{(N)}\) maps a random outcome to a linear combination of continuous functions.

Theorem 15 The series \(\sum_{n=1}^{\infty }\xi_n \cdot \tilde{e}_n\), i.e., the sequence of partial sums \(W^{(N)}\), converges almost surely in the Banach space \(C[0,1]\). In other words, there is \(\Omega ' \subseteq \Omega\) such that \(\mathbb {P}\left(\Omega '\right) = 1\) and for every fixed elementary outcome \(\omega \in \Omega '\) the sequence \(W^{(N)}(\omega ) = \sum_{n=1}^{N}\xi_n(\omega ) \cdot \tilde{e}_n \in C[0,1]\) converges in the space \(C[0,1]\) (with the supremum norm) to some element of \(C[0,1]\), which can be denoted \(W(\omega )\).

This theorem allows us to rigorously define \(W = W_t = W_t(\omega )\) for almost every \(\omega\). For the canonical proof, see (Kwapien and Woyczynski 1992, Theorem 2.5.1, p. 49): it is fairly short, but relies on the Itô–Nisio theorem and fractional integration operators (fractional antiderivatives). Anyone interested in delving deeper into random processes and functional analysis is welcome to do so. We shall give this proof in section Section 5 at the end of the chapter: we shall state the Itô–Nisio theorem and use it as a black box, and work out from scratch all the necessary properties of fractional integration operators.

So, the process \(W_t = \sum_{n=1}^{\infty } \xi_n \cdot \tilde{e}_n(t)\) has almost surely continuous paths. But what about the finite-dimensional distributions? Will they be the ones needed for a Wiener process? You are able to work this question out on your own.

ProblemProblem 12

1. Prove that for any fixed \(t \in [0,1]\) the following convergence holds almost surely and in \(L^2 = L^2(\Omega , \mathcal{F}, \mathbb {P})\):

\[ W^{(N)}_{t} \quad \xrightarrow [N \to +\infty ]{\mathbb {P}\text{-a.s.}, \; \; L^2(\Omega )} \quad W_t \]

2. Prove that the finite-dimensional distributions of \(W^{(N)}_{t}\) converge to the finite-dimensional distributions of \(W_{t}\) as \(N \to \infty\);

3. Prove that \((W_t, \; t \in [0,1])\) is a Wiener process on \([0,1]\).

Now that we know how to meaningfully sum the series \(\sum_n \xi_n \tilde{e}_n\), we can “play” with its terms: what happens if we do not sum all of them?

ExampleExample 6

Let \(A \subseteq \mathbb {N}\) be an arbitrary set of indices, and let \[ X^{A}_t := \sum _{n \in A} \xi _n \cdot \tilde{e}_n(t) \] be a “thinned” series (almost sure convergence in \(C[0,1]\) is preserved, see the remark).

1. Find the covariance function of the process \(X^A\). Show that \(W = X^{A} + X^{A^c}\) is a decomposition of the Wiener process into a sum of two independent centered continuous Gaussian processes (for example, splitting into even and odd \(n\)).

2. Take the Haar system and discard exactly one function: the constant \(e \equiv 1\) (i.e., the term \(\xi_0 t\)). What process remains?

3. What happens if we discard a finite number of functions from an arbitrary ONB (for example, the first \(5\))? Does the limit remain a Wiener process? And what about its quadratic variation?

On the convergence of \(X^A\): the proof from section Section 5 carries over verbatim to a part of the series — completeness of the ONB is used there only through Bessel’s inequality, which holds for any orthonormal system.

HintHint

For part (2): compute the covariance function and compare it to problem Problem 2.

SolutionSolution

Enum-item(1)

As with the full series, for fixed \(s,t\) the partial sums converge in \(L^2(\Omega )\), and

\[ \operatorname {Cov}\left[ X^A_s, X^A_t \right] = \sum _{n \in A} \tilde{e}_n(s) \cdot \tilde{e}_n(t) \eqqcolon K_A(s,t) \]

(the same computation as in the main text, except Parseval’s equality is not applied — the sum over \(A\) simply remains a sum over \(A\)). The process \(X^A\) is centered and Gaussian (\(L^2\)-limits of linear combinations of i.i.d. random variables). The processes \(X^A\) and \(X^{A^c}\) are built from disjoint collections of independent \(\xi_n\), so they are independent (check this on finite sums and pass to the limit), and pointwise \(X^A_t + X^{A^c}_t = W_t\) a.s., while \(K_A(s,t) + K_{A^c}(s,t) = \min (s,t)\). For example, summing separately over even and odd \(n\), we get a decomposition of \(W\) into a sum of two independent continuous centered Gaussian processes — neither of which is itself a Wiener process.

Enum-item(2)

Let us compute the covariance: the sum over the full system equals \(\min (s,t)\), and the discarded term equals \(\tilde{e}_0(s) \cdot \tilde{e}_0(t) = st\):

\[ K(s,t) = \min (s,t) - st \]

This is exactly the covariance function of the Brownian bridge (problem Problem 2)! The picture can also be seen directly: all the tent functions \(\tilde{\psi }_{m,k}\) vanish at \(t = 1\), so \(W_1 = \xi_0\), and the thinned series equals

\[ \sum _{m,k}\xi _{m,k} \cdot \tilde{\psi }_{m,k}(t) = W_t - \xi _0 t = W_t - t W_1 = B_t \]

— the classical representation of the bridge. And the decomposition \(W_t = \underbrace{t \cdot W_1}_{\text{random slope}} + \; B_t\) with independent terms is a special case of part (1).

Enum-item(3)

We get \(X_t = W_t - \sum_{n \leq 5}\xi_n \tilde{e}_n(t)\) — the Wiener process minus a random but smooth correction: the functions \(\tilde{e}_n(t) = \int_0^t e_n(r) \, dr\) are absolutely continuous. The process \(X\) is centered Gaussian with covariance \(\min (s,t) - \sum_{n\leq 5}\tilde{e}_n(s)\tilde{e}_n(t) \neq \min (s,t)\), i.e., it is not a Wiener process: for example, \(\operatorname {Var}\left[X_t\right] = t - \sum_{n \leq 5}\tilde{e}_n(t)^2 < t\) at those \(t\) for which at least one \(\tilde{e}_n(t) \neq 0\). But its “small-scale roughness” is the same: the quadratic variation is still equal to \(t\). Indeed, for \(F(t) := \sum_{n\leq 5}\xi_n\tilde{e}_n(t)\) and a partition \(\left\{ t_i\right\}\) of \([0,t]\) with mesh tending to zero,

\[ \sum _i \left(\Delta _i F\right)^2 = \sum _i \left(\int _{t_i}^{t_{i+1}} F'(r) \, dr\right)^2 \leq \max _i \left|t_{i+1} - t_i\right| \cdot \int _0^t F'(r)^2 \, dr \to 0 \]

(the Cauchy–Schwarz inequality on each interval of the partition), while the cross term \(2\sum_i \Delta_i W \Delta_i F\) is bounded via the square root of the product of the sums of squares and also goes to zero. An amusing conclusion: the quadratic variation does not feel smooth (even if random) additions and does not distinguish \(X\) from \(W\).

AnswerAnswer

2. the Brownian bridge.

Mn-extraAdditional material

What if \(\xi_n\) is not Gaussian? Suppose that in the formula \(\sum_n \xi_n \tilde{e}_n\) the coefficients \(\xi_n\) are i.i.d. with some other distribution: \(\operatorname {U}[-1,1]\), \(\operatorname {Exp}(\lambda )\), \(\operatorname {Pois}\left(\lambda \right)\). Let us briefly enumerate what happens here; precise convergence criteria for such series are a separate area, see (Kwapien and Woyczynski 1992).

• Centering matters. If \(\mu := \mathbb {E}\left[\xi_1\right] \neq 0\) (the cases \(\operatorname {Exp}\) and \(\operatorname {Pois}\)), a deterministic part \(\mu \cdot \sum_n \tilde{e}_n(t)\) splits off from the series, whose convergence is a separate question about the basis. For the Haar system it converges uniformly and absolutely by generations: within generation \(m\), at each point \(t\) at most one tent function is nonzero, and it is at most \(2^{-(m+2)/2}\). From here on we assume \(\xi_n\) is centered (we subtract \(\mu\)).

• Convergence for the Haar system is almost free. If \(\mathbb {E}\left[\xi_1\right] = 0\) and \(\mathbb {E}\left[\left|\xi_1\right|^p\right] < \infty\) for at least one \(p > 2\), then the series with respect to the Haar system converges almost surely uniformly. Sketch: by the Borel–Cantelli lemma and Markov’s inequality

  \[ \sum _m \mathbb {P}\left(\max _{0 \leq k < 2^m} \left|\xi _{m,k}\right| > \frac{2^{m/2}}{m^2}\right) \; \leq \; \sum _m 2^m \cdot \mathbb {E}\left[\left|\xi _1\right|^p\right] \cdot \frac{m^{2p}}{2^{pm/2}} < \infty \quad \text{(since $p > 2$)}, \]

  so almost surely for all large \(m\) the sup-norm of the \(m\)-th generation of the series does not exceed \(2^{-\frac{m}{2} - 1} \cdot \frac{2^{m/2}}{m^2} = \frac{1}{2m^2}\) — the series of generations is dominated by a convergent numerical series. And for bounded \(\xi_n\) (for example, \(\operatorname {U}[-1,1]\)), even Borel–Cantelli is not needed: the majorant \(2^{-(m+2)/2} \cdot \sup \left|\xi \right|\) is deterministic. The exponential and Poisson distributions have all moments finite, so after centering they fall into this case too. Note: the limit is continuous even for integer-valued Poisson coefficients.

• But the result is no longer a Wiener process. The limit is centered, its covariance function equals \(\operatorname {Var}\left[\xi_1\right] \cdot \min (s,t)\), and its increments are uncorrelated — but the finite-dimensional distributions are non-Gaussian, and the increments are, as a rule, dependent. For instance, for the Haar system and \(\xi_n \sim \operatorname {U}[-1,1]\) we get \(X_1 = \xi_0 \sim \operatorname {U}[-1,1]\) — nothing like normality.

• Moreover, the limit depends on the choice of basis! For the Haar system, \(X_1 = \xi_0\) is a copy of a single term; whereas for the Kosambi–Karhunen–Loève basis, \(X_1 = \sqrt{2}\sum_n (-1)^{n+1} \xi_n / \left(\left(n - \frac{1}{2}\right)\pi \right)\) is a weighted sum of the entire sequence, whose distribution is already “smoothed” in the spirit of the CLT. The deep reason why Gaussian \(\xi_n\) have no such dependence is that passing from one ONB to another is an “infinite-dimensional rotation” of the coefficient vector \((\xi_1, \xi_2, \ldots )\), and the standard Gaussian vector is essentially the only vector with independent identically distributed coordinates whose distribution is invariant under all rotations (Maxwell’s theorem).

4.3 Donsker’s construction (multivariate CLT)

Let \(X_1, X_2, \ldots\) be i.i.d. random variables with finite variance. Without loss of generality, we may assume \(\mathbb {E}\left[X_i\right] = 0\) and \(\operatorname {Var}\left[X_i\right] = 1\). Indeed, if this is not the case, we can always pass to the centered, normalized random variables

\[ \tilde{X}_i := \frac{X_i - \mathbb {E}\left[X_i\right]}{\sqrt{\operatorname {Var}\left[X_i\right]}} \]

.

As you know from the probability theory course, the CLT holds for such a sequence:

Theorem 16 (The central limit theorem) Under the conditions above, \(\frac{S_n}{\sqrt{n}} \xrightarrow [n \to \infty ]{d} \mathscr {N}\left(0, 1\right)\).

ProblemProblem 13

Let \(m \in \mathbb {N}\) be some fixed number. Under the conditions above, find the limit in distribution of the random vector \[ \frac{1}{\sqrt{n}} \; \cdot \; \begin{pmatrix} 0 \\ S_{\lfloor n/m \rfloor } \\ S_{\lfloor 2n/m \rfloor } \\ \cdots \\ S_{\lfloor (m-1)n/m \rfloor } \\ S_{n} \end{pmatrix} \]

Theorem 17 (Donsker’s theorem, the infinite-dimensional CLT) Let \(X_1, X_2, \ldots\) be i.i.d. random variables with \(\mathbb {E}\left[X_i\right] = 0\), \(\operatorname {Var}\left[X_{i}\right] = 1\). For a fixed natural number \(n \in \mathbb {N}\), consider the random function \[ Y^{(n)} = Y^{(n)}_t = Y^{(n)}_t(\omega ), \qquad t \in [0,1], \; \omega \in \Omega , \] which at the points \(\frac{k}{n}\), \(k \in \left\{ 0,1,\ldots , n\right\}\), equals \[ Y^{(n)}_{k/n} := \frac{1}{\sqrt{n}}S_{k}, \] and between these points is equal to the linear interpolation of these values. In other words, \[ \begin{align} Y_{t}^{(n)} & := \frac{1}{\sqrt{n}}S_{\lfloor nt \rfloor } + \left(nt - \lfloor nt \rfloor \right) \cdot \frac{1}{\sqrt{n}}S_{\lfloor nt \rfloor + 1}, \qquad t \in [0,1] \end{align} \] where \(S_{n} := X_1 + \ldots + X_{n}\). Then \(Y^{(n)}\) is a random element of the space \(C[0,1]\) for each \(n\), and \(Y^{(n)}\) converges weakly to some probability measure on \(C[0,1]\) as \(n \to \infty\). This measure is precisely the Wiener measure on \(C[0,1]\), i.e., the distribution of the Wiener process.

For a proof, see (Биллингсли 1977, Theorem 8.2).

A visualization for \(X_i \sim \frac{1}{2}\delta_{-1} + \frac{1}{2}\delta_1\), i.e., for \(S_n\) — the simplest symmetric random walk: the scaled trajectory \(Y_t^{(n)} = \frac{1}{\sqrt n} S_{\lfloor nt \rfloor} + (nt - \lfloor nt \rfloor)\cdot \frac{1}{\sqrt n} X_{\lfloor nt \rfloor + 1}\), extended to \(t \in [0,4]\) (the theorem uses \(t\in[0,1]\); here the range is extended so that more of the walk’s “history” is visible). Move the slider \(n\) to see how, as the number of steps increases, the scaled walk increasingly accurately approximates a trajectory of the Wiener process (Donsker’s invariance principle); use the “Show more trajectories” slider to add independent realizations, and click the button to regenerate the steps \(X_1,X_2,\ldots\) (new trajectories).

n = 2000 Show more trajectories = 0

.

5 Proofs

5.1 Proof of theorem Theorem 15 (a.s. convergence of the series \(\sum_n \xi_n \tilde{e}_n\) in \(C[0,1]\))

We follow the proof in (Kwapien and Woyczynski 1992, Theorem 2.5.1, pp. 49–51). It relies on two tools: the Itô–Nisio theorem on the convergence of series of independent random elements of a Banach space (we shall state it and use it as a black box) and fractional integration operators, whose properties we shall work out from scratch. The proof is essentially self-contained: the Wiener process is nowhere assumed to have been constructed in advance — on the contrary, the series itself constructs it.

First, the idea. Let \(D^1\) denote the operator of taking an antiderivative: \(\left(D^1 f\right)(t) = \int_0^t f(s) \, ds\); then \(\tilde{e}_n = D^1 e_n\), and our series is \(\sum_n \xi_n \, D^1 e_n\). Proving convergence directly in \(C[0,1]\) is hard: one would have to control the suprema of a Gaussian process. On the other hand, in the space \(L^4[0,1]\) the norm is computed as an integral, and for Gaussian terms everything reduces to moments; and it is the Itô–Nisio theorem that will carry us from mean convergence to almost sure convergence. The plan is this: we decompose \(D^1\) as a composition of two “fractional antiderivatives”

\[ D^1 = D^{3/8} \circ D^{5/8} \]

An intermediate object — the series \(\sum_n \xi_n \, D^{5/8}e_n\) — converges almost surely in \(L^4[0,1]\), and the operator \(D^{3/8}\) will turn out to be continuous from \(L^4[0,1]\) to \(C[0,1]\) and will carry that convergence over to the one we need.

5.1.1 Part 1: the Itô–Nisio theorem

Theorem 18 (Itô–Nisio; {}) Let \(V\) be a separable Banach space, and let \(X_1, X_2, \ldots\) be independent \(V\)-valued random elements having symmetric distributions: \(-X_n \overset {d}{=} X_n\) for all \(n\). Let \(S_N := \sum_{n=1}^{N} X_n\). Then the following conditions are equivalent:

1. \(S_N\) converges a.s. (in the norm of \(V\));

2. \(S_N\) converges in probability;

3. \(S_N\) converges in distribution (i.e., the distributions \(\operatorname {Law}(S_N)\) converge weakly);

4. there exist a \(V\)-valued random element \(S\) and a set \(\mathcal{D} \subseteq V'\) of continuous linear functionals separating the points of \(V\) (for any \(v \neq u\) there is \(\varphi \in \mathcal{D}\) with \(\varphi (v) \neq \varphi (u)\)) such that \(\varphi (S_N) = \sum_{n \leq N} \varphi (X_n) \to \varphi (S)\) in probability for every \(\varphi \in \mathcal{D}\).

The substantive meaning: for series of independent symmetric terms, all types of convergence — from the weakest, coordinate-wise in distribution, to the strongest, almost sure in norm — coincide. This is an infinite-dimensional generalization of the classical Lévy theorem (“for series of independent quantities, convergence in distribution implies almost sure convergence”). The proof is beautiful but would require a separate seminar — see (Kwapien and Woyczynski 1992, 2.1). We shall need the implication (ii) \(\; \Rightarrow \;\) (i).

5.1.2 Part 2: fractional integration

For \(0 < \alpha \leq 1\) set

\[ k_\alpha (s) := \frac{s^{\alpha - 1}}{\Gamma (\alpha )}, \quad s \in (0, 1], \qquad \qquad \left(D^\alpha f\right)(t) := \int _0^t k_\alpha (t - s) f(s) \, ds \]

The operator \(D^\alpha\) is called the fractional integration operator (of order \(\alpha\)). For \(\alpha = 1\) the kernel is \(k_1 \equiv 1\), and \(D^1\) is the ordinary antiderivative; for \(\alpha < 1\) the kernel has an integrable singularity at zero. We shall need three properties.

Lemma 6 (Semigroup property) Let \(\alpha , \beta \in (0,1]\), \(\alpha + \beta \leq 1\), \(f \in L^2[0,1]\). Then \(D^{\beta }\left(D^{\alpha } f\right) = D^{\alpha + \beta } f\).

Mn-proofProof

For fixed \(r\) we change the order of integration (Fubini; justification below): \[ \begin{align} \left(D^{\beta }\left(D^{\alpha }f\right)\right)(r) & = \int _0^r k_\beta (r - t) \int _0^t k_\alpha (t-s) f(s) \, ds \, dt = \\ & = \int _0^r \left(\int _s^r k_\beta (r-t) \, k_\alpha (t - s) \, dt\right) f(s) \, ds \end{align} \] The inner integral is computed by the substitution \(t = s + (r-s)u\), \(u \in [0,1]\): \[ \begin{align} \int _s^r (r-t)^{\beta - 1}(t - s)^{\alpha - 1} \, dt & = (r-s)^{\alpha + \beta - 1}\int _0^1 (1-u)^{\beta - 1} u^{\alpha - 1} \, du = \\ & = (r-s)^{\alpha + \beta - 1} \cdot B(\alpha , \beta ) \end{align} \] and, since \(B(\alpha ,\beta ) = \frac{\Gamma (\alpha )\Gamma (\beta )}{\Gamma (\alpha + \beta )}\) (a well-known property of the beta function), \[ \int _s^r k_\beta (r - t) k_\alpha (t-s) \, dt = \frac{(r-s)^{\alpha +\beta -1}}{\Gamma (\alpha + \beta )} = k_{\alpha + \beta }(r - s) \] which is exactly \(D^{\alpha +\beta }f\). Justification of Fubini: the same computation for \(\left|f\right|\) (Tonelli’s theorem, everything nonnegative) gives \(\left(D^{\alpha + \beta }\left|f\right|\right)(r) < \infty\) for almost every \(r\), i.e., absolute integrability.

Lemma 7 Let \(\alpha > \frac{1}{2}\). Then for \(f \in L^2[0,1]\) \[ \sup _{t \in [0,1]} \, \left|\left(D^{\alpha }f\right)(t)\right| \leq c_\alpha \left\| f\right\| _{L^2}, \qquad c_\alpha := \frac{1}{\Gamma (\alpha )\sqrt{2\alpha - 1}} \] In particular, \(D^{\alpha } f \in L^4[0,1]\) and \(\left\| D^\alpha f\right\|_{L^4} \leq c_\alpha \left\| f\right\|_{L^2}\).

Mn-proofProof

The Cauchy–Schwarz inequality in \(L^2(0,t)\): \[ \left|\left(D^\alpha f\right)(t)\right| \leq \left\| k_\alpha (t - \cdot )\right\| _{L^2(0,t)} \cdot \left\| f\right\| _{L^2} = \frac{1}{\Gamma (\alpha )}\left(\int _0^t u^{2\alpha - 2} \, du\right)^{1/2} \left\| f\right\| _{L^2} = \frac{t^{\alpha - \frac{1}{2}}}{\Gamma (\alpha )\sqrt{2\alpha - 1}}\left\| f\right\| _{L^2} \] and \(t \leq 1\). The exponent \(2\alpha - 2 > -1\) precisely because \(\alpha > \frac{1}{2}\).

Lemma 8 Let \(\beta > \frac{1}{4}\). Then \(D^{\beta }\) is a continuous linear operator from \(L^4[0,1]\) to \(C[0,1]\): for \(f \in L^4[0,1]\) the function \(D^\beta f\) is continuous on \([0,1]\) and \[ \left\| D^{\beta } f\right\| _{\infty } \leq c'_\beta \left\| f\right\| _{L^4}, \qquad c'_\beta := \left\| k_\beta \right\| _{L^{4/3}[0,1]} < \infty \]

Mn-proofProof

Hölder’s inequality with exponents \(4\) and \(\frac{4}{3}\):

\[ \left|\left(D^\beta f\right)(t)\right| \leq \left\| k_\beta (t - \cdot )\right\| _{L^{4/3}(0,t)} \cdot \left\| f\right\| _{L^4} \leq \left\| k_\beta \right\| _{L^{4/3}[0,1]} \cdot \left\| f\right\| _{L^4} \]

where \(\left\| k_\beta \right\|_{L^{4/3}}^{4/3} = \Gamma (\beta )^{-4/3}\int_0^1 u^{(\beta - 1)\cdot \frac{4}{3}} \, du < \infty\), since \((\beta - 1)\cdot \frac{4}{3} > -1 \iff \beta > \frac{1}{4}\).

Continuity of \(D^\beta f\) in \(t\) (this point is omitted in the book): let us write \(k_\beta (t-s)\; \mathbb {1}_{0 < s < t} = g(s - t)\), where \(g(u) := k_\beta (-u)\; \mathbb {1}_{(-1,0)}(u) \in L^{4/3}(\mathbb {R})\). Then

\[ \left|\left(D^\beta f\right)(t') - \left(D^\beta f\right)(t)\right| \leq \left\| g(\cdot - t') - g(\cdot - t)\right\| _{L^{4/3}(\mathbb {R})} \cdot \left\| f\right\| _{L^4} \xrightarrow [t' \to t]{} 0 \]

by continuity of translation in \(L^{q}(\mathbb {R})\) — a standard fact of functional analysis (continuous functions are dense in \(L^q\), and for them the statement is obvious).

5.1.3 Part 3: convergence of the “intermediate object” in \(L^4[0,1]\)

Take \(\alpha = \frac{5}{8} > \frac{1}{2}\) and \(\beta = \frac{3}{8} > \frac{1}{4}\), so that \(\alpha + \beta = 1\). Set

\[ \varphi _n := D^{5/8} e_n \in L^4[0,1], \qquad S_N := \sum _{n=1}^{N} \xi _n \varphi _n \]

— independent symmetric random elements of \(L^4[0,1]\) and their partial sums. Let us estimate, for \(M < N\) (Tonelli’s theorem — the integrand is nonnegative):

\[ \mathbb {E}\left[\left\| S_N - S_M\right\| ^4_{L^4}\right] = \int _0^1 \mathbb {E}\left[\left(\sum _{n=M+1}^{N} \xi _n \varphi _n(t)\right)^4\right] dt = 3\int _0^1 \left(\sum _{n = M+1}^{N} \varphi _n(t)^2\right)^2 dt \]

— for each \(t\), the quantity \(\sum_{n=M+1}^N \xi_n\varphi_n(t)\) is Gaussian \(\mathscr {N}\left(0, \; \sum_{M < n \leq N}\varphi_n(t)^2\right)\), and the fourth moment of \(\mathscr {N}\left(0, \sigma^2\right)\) equals \(3\sigma^4\).

Now for the key estimate. Note that \(\varphi_n(t) = \left\langle k_{5/8}(t - \cdot )\; \mathbb {1}_{(0,t)}, e_n \right\rangle_{L^2[0,1]}\) are the Fourier coefficients of the function \(h_t := k_{5/8}(t - \cdot )\; \mathbb {1}_{(0,t)} \in L^2[0,1]\). By Bessel’s inequality and lemma Lemma 7

\[ \sum _{n=1}^{\infty } \varphi _n(t)^2 \leq \left\| h_t\right\| ^2_{L^2} \leq c_{5/8}^2 \qquad \forall t \in [0,1] \]

Hence the tails \(R_M(t) := \sum_{n > M}\varphi_n(t)^2\) decrease monotonically to zero at each \(t\) (tails of a convergent numerical series) and are uniformly bounded by the constant \(c_{5/8}^2\). By the Lebesgue dominated convergence theorem

\[ \sup _{N > M} \, \mathbb {E}\left[\left\| S_N - S_M\right\| ^4_{L^4}\right] \leq 3\int _0^1 R_M(t)^2 \, dt \; \xrightarrow [M \to \infty ]{} \; 0 \]

i.e., \((S_N)\) is Cauchy in fourth mean, and hence (by Markov’s inequality) also in probability; the space \(L^4[0,1]\) is complete with respect to convergence in probability in the following sense: a sequence that is Cauchy in probability converges in probability to some limit (a standard fact: pass to an almost surely convergent subsequence). So \(S_N \to S\) in probability in \(L^4[0,1]\). The space \(L^4[0,1]\) is separable and Banach, and the terms \(\xi_n\varphi_n\) are independent and symmetric — by the Itô–Nisio theorem (implication (ii) \(\; \Rightarrow \;\) (i))

\[ S_N \; \xrightarrow [N \to \infty ]{\text{a.s.}} \; S \qquad \text{in } L^4[0,1] \]

5.1.4 Part 4: assembly

It remains to apply the operator \(D^{3/8}\). By the semigroup property (lemma Lemma 6) and linearity

\[ D^{3/8} S_N = \sum _{n=1}^{N} \xi _n \, D^{3/8}\left(D^{5/8} e_n\right) = \sum _{n=1}^{N} \xi _n \, D^{1} e_n = \sum _{n=1}^{N} \xi _n \tilde{e}_n = W^{(N)} \]

— this is exactly the original partial sums! The operator \(D^{3/8}: L^4[0,1] \to C[0,1]\) is linear and continuous (lemma Lemma 8), so on the same event of full probability where \(S_N \to S\) in \(L^4\),

\[ W^{(N)} = D^{3/8}S_N \; \xrightarrow [N \to \infty ]{} \; D^{3/8}S \qquad \text{in } C[0,1] \]

The theorem is proved; the limit \(W := D^{3/8}S\) is an almost surely continuous function.

Remarks.

• Where do the exponents \(\frac{5}{8}\) and \(\frac{3}{8}\) come from? We need \(\alpha > \frac{1}{2}\) (lemma Lemma 7), \(\beta > \frac{1}{p}\) (lemma Lemma 8 for \(L^p\)), and \(\alpha + \beta = 1\). For \(p = 2\) we would get \(1 = \alpha + \beta > \frac{1}{2} + \frac{1}{2} = 1\) — a contradiction: \(D^1\) can only be split into two suitable factors via \(L^p\) with \(p > 2\). We took \(p = 4\) (the first even exponent) so that the moment \(\mathbb {E}\left[\eta^4\right] = 3\sigma^4\) could be computed in one line; in (Kwapien and Woyczynski 1992) the same computation is carried out for arbitrary \(2 < p < \infty\) via \(c(p) := \mathbb {E}\left[\left|\xi_1\right|^p\right]\).

• Completeness of the ONB was not used in Parts 3–4: Bessel’s inequality, which holds for any orthonormal system, sufficed. Therefore the proof carries over verbatim to “thinned” series \(\sum_{n \in A}\xi_n \tilde{e}_n\) (see the problem in section Section 4) and does not depend on the order in which the basis is enumerated. Completeness is needed only at the next step — when we identify the limit with the Wiener process via Parseval’s equality (the problem after theorem Theorem 15).

• Bonus (Remark 2.5.1 in (Kwapien and Woyczynski 1992)): for \(\beta > \frac{1}{p} + \gamma\) the operator \(D^\beta\) is continuous from \(L^p[0,1]\) to the space of \(\gamma\)-Hölder functions. Choosing \(p\) large and \(\beta\) slightly less than \(\frac{1}{2}\), we get for free: the paths of the limiting process are \(\gamma\)-Hölder for any \(\gamma < \frac{1}{2}\) — in full agreement with the section on local properties of the paths of WP.

• For the Haar system there is also a completely direct proof, without fractional integration or the Itô–Nisio theorem (essentially, Lévy’s original 1940 argument): group the series by generations. The tent functions \(\tilde{\psi }_{m,k}\), \(0 \leq k < 2^m\), have disjoint supports, so

  \[ \sup _t \, \left|\; \sum _{k} \xi _{m,k} \tilde{\psi }_{m,k}(t)\right| = 2^{-\frac{m+2}{2}} \max _{k} \left|\xi _{m,k}\right|, \]

  and from the Gaussian tail bound \(\mathbb {P}\left(\left|\xi \right| > x\right) \leq e^{-x^2/2}\) (for \(x \geq 1\)) and the Borel–Cantelli lemma: \(\sum_m \mathbb {P}\left(\max_k \left|\xi_{m,k}\right| > 2\sqrt{m}\right) \leq \sum_m 2^m e^{-2m} < \infty\), i.e., almost surely for all large \(m\) the generation contributes no more than \(2^{-\frac{m+2}{2}} \cdot 2\sqrt{m}\) — a convergent numerical series. By the Weierstrass criterion the series (by generations) converges uniformly almost surely. The drawback of this route: it is tied to a specific basis; for an arbitrary ONB this does not work.

References

Kwapien, S., and W. Woyczynski. 1992. Random Series and Stochastic Integrals: Single and Multiple. Probability and Its Applications. Birkhäuser Boston.

Schilling, René L. 2021. Brownian Motion: A Guide to Random Processes and Stochastic Calculus. De Gruyter. https://doi.org/doi:10.1515/9783110741278.

Биллингсли, П. 1977. Сходимость Вероятностных Мер. Наука.

Булинский, А. В., and А. Н. Ширяев. 2005. Теория Случайных Процессов. Физматлит.

Footnotes

1. Do not confuse the Schauder functions with the Faber–Schauder system or with the Schauder basis. The Faber–Schauder system is obtained from the Schauder system by normalizing in the supremum norm and adding the function identically equal to \(1\). What a Schauder basis is (sometimes it is simply called a basis for an infinite-dimensional Banach space) you should cover in functional analysis. Interestingly, the Faber–Schauder system is a Schauder basis in the Banach space \(C[0,1]\).