This lecture note is still being edited: the text may contain typos, inaccuracies, and locally unpolished spots. If you find something, please let the author know (contact information on the course page).
The material of this chapter is abstract: \(X\) is an arbitrary set, and the outer measure and Carathéodory’s extension theorem are formulated and proved for an arbitrary space, with no connection to \(\mathbb {R}\) whatsoever (Lebesgue measure will appear only as a special case of this construction in the next chapter).
Extending premeasures from semirings to the generated sigma-algebras is needed not only when constructing measures on finite-dimensional Euclidean spaces \(\mathbb {R}^{k}\), but also, for example, in the theory of random functions (random processes), where the function spaces are typically infinite-dimensional (\(C[0,1]\), for instance). For this reason we will formulate Carathéodory’s theorem for an arbitrary space. To do this, let us first discuss what an outer measure is in the general case.
The general notion of outer measure, the Carathéodory criterion
Definition 2 (Carathéodory’s criterion) Let \(\mu^*\) be an outer measure. \(A \subset X\) is called \(\mu^*\)-measurable if \[
\mu ^*(E \cap A) + \mu ^*(E \setminus A) = \mu ^*(E), \quad \forall E \subset X
\] That is, \(A\) is a good “separator” for all other sets. The notation for the set of \(\mu^*\)-measurable subsets of \(X\) is: \(\mathcal{M}(\mu^*)\). Note right away that \(\varnothing\) and \(X\) are always \(\mu^*\)-measurable.
Note that for measurability it suffices to have
\[
\mu ^*(E \cap A) + \mu ^*(E \setminus A) \leq \mu ^*(E), \quad \forall E \subset X,
\]
since the reverse inequality follows from the subadditivity of \(\mu^*\).
Let us try to give some motivation for this rather opaque criterion. The natural idea is to form an “inner measure” \(\mu_*\) by approximating the set \(A\) from the inside with sets from \(\mathcal{S}\), and to define \(A\) as measurable if the inner and outer measures coincide, \(\mu_*(A) = \mu^*(A)\). Indeed, this is exactly what Lebesgue himself did. But, strangely enough, this condition is harder to work with in practice (for instance, when proving that the measurable sets form a sigma-algebra), and it is not well suited to generalization.
Namely, one can look at the inner measure from a somewhat different angle: taking the inner measure of \(A\) is the same as taking the outer measure of \(A^\complement = X \setminus A\):
\[
\mu _*(A) := \mu ^*(X) - \mu ^*(A^\complement )
\]
Inner measure via outer measure: μ₊(A) := μ(X) − μ(Aᶜ). The region Aᶜ = X∖A is shaded — hover over the region to see its definition.
In this case the condition that the outer and inner measures be equal reads:
\[
\mu ^*(A) = \mu ^*(X) - \mu ^*(A^\complement )
\]
Although this condition works for spaces with finite measures, we run into problems when \(\mu^*\) can take infinite values. Among other things, an indeterminate expression of the form \(\infty - \infty\) can appear on the right-hand side of the equality. The condition can be reformulated as follows:
\[
\mu ^*(A) + \mu ^*(A^\complement ) = \mu ^*(X)
\]
The indeterminacy \(\infty - \infty\) disappears. But in this case, if, for example, \(\mu^*\) is the outer Lebesgue measure on \(\mathbb {R}\) and \(A\) is a set such that both \(A\) and \(A^\complement\) are unbounded, then \(\mu^*(A) = \mu_*(A) = +\infty\), i.e., any such set would be measurable. This contradicts the axiom of choice, which can nevertheless be used to construct a nonmeasurable set of the form \(A\).
To fix the situation, we can introduce the “inner measure relative to \(E\)”, for an arbitrary \(E\) such that \(A \subset E \subset X\):
\[
\mu _E(A) = \mu ^*(E) - \mu ^*(E \setminus A),
\]
and then check the condition “inner measure = outer measure” for all such \(E\), rather than only for \(E=X\) as we did before. We get
\[
\mu ^*(A) + \mu ^*(E \setminus A) = \mu ^*(E), \qquad \forall E: \; A \subset E \subset X
\]
Inner measure relative to E: A ⊂ E ⊂ X. The region E∖A is shaded — hover over the regions to see their definitions.
In fact it is more convenient to drop the requirement that \(S\) be a superset of \(A\), and instead allow \(A\) to “cut a piece” out of \(S\):
\[
\mu ^*(E \cap A) + \mu ^*(E \setminus A) = \mu ^*(E), \qquad \forall E \subset X
\]
Carathéodory criterion: E is an arbitrary subset of X, not necessarily containing A. The dark region is E∩A, the light gray region is E∖A. Hover over the regions.
This gives us the Carathéodory measurability criterion.
Define the functions \(\mu^*_1, \mu^*_2, \mu^*_3\) on \(2^{\mathbb {R}}\) as follows: \[
\begin{align} \mu ^*_1(A) & = \begin{cases} 0, & A = \varnothing \\ 1, & A \neq \varnothing \end{cases} & \qquad \mu ^*_2(A) & = \begin{cases} 0, & A = \varnothing \\ +\infty , & A \neq \varnothing \end{cases}, & \qquad \mu ^*_3(A) & = \begin{cases} 0, & A \text{ bounded} \\ 1, & A \text{ unbounded} \end{cases} \end{align}
\]
Which of these functions are outer measures?
For each \(i \in \{ 1,\dots ,3\}\) for which \(\mu^*_i\) turned out to be an outer measure, characterize the set \(\mathcal{M}(\mu_i^*)\).
Theorem 1 (On outer measure) Let \(\mu^*\) be an outer measure on \(X\). Then
all \(\mu^*\)-measurable subsets \(\mathcal{M}(\mu^*)\) form a sigma-algebra;
\(\mu^*|_{\mathcal{M}(\mu^*)}\) (the restriction of \(\mu^*\) to \(\mathcal{M}(\mu^*)\)) is a measure on the sigma-algebra \(\mathcal{M}(\mu^*)\). That is, \(\mu^*(\varnothing ) = 0\) and \(\mu^*\) has countable additivity on \(\mathcal{M}(\mu^*)\).
Prove that sets of zero outer measure are always Carathéodory measurable. Namely, let \(\mu^*\) be an outer measure on \(X\), and let \(\mu^*(B) = 0\). Prove that \(B \in \mathcal{M}(\mu^*)\).
The extension theorem
The most important example of an outer measure is the outer Lebesgue measure, also called the outer measure via (arbitrary) countable covers. For us, in this course, it will be the only example of an outer measure, although other outer measures do exist, see the Hausdorff measure.
Let \(\mathcal{S} \subset 2^X\) be an arbitrary collection of subsets with \(\varnothing \in \mathcal{S}\). Let \(\mu : \mathcal{S} \to [0, +\infty ]\) be a nonnegative monotone set function with \(\mu (\varnothing ) = 0\). For an arbitrary \(A \subset X\) define the outer Lebesgue measure induced by \(\mu\) as
\[
\begin{align} \mu ^*(A) := \inf \left\{ \sum _{i=1}^{\infty }\mu (F_i) \; : \; A \subseteq \bigcup _{i=1}^{\infty }F_{i}, \, A_i \in \mathcal{S} \text{ for all }i\right\} \quad (\inf (\varnothing ) := +\infty ) \end{align}
\]
In this section and hereafter, by \(\mu^*\) (or \(\lambda^*\)) we will mean precisely the outer Lebesgue measure for \(\mu\) (for \(\lambda\)).
(Klenke 2020, Theorem 1.53):
Theorem 2 (Carathéodory’s extension theorem) Let \(\mathcal{S}\) be a semiring, and let \(\mu : \mathcal{S} \to [0, +\infty ]\) be a premeasure (i.e., \(\mu (\varnothing ) = 0\) and \(\mu\) is countably additive on \(\mathcal{S}\)). Then there exists an extension of \(\mu\) to a measure \(\tilde{\mu }\) on the sigma-algebra \(\sigma (\mathcal{S})\).
If, in addition, \(\mu\) is a \(\sigma\)-finite measure on \(\mathcal{S}\), then the extension above is unique (and \(\sigma\)-finite).
Proof (sketch). Outline of the construction of the extension:
From \(\mu\) we define the outer measure \(\mu^*\) via the infimum of the measures of countable covers. \(\mu^*\) is an outer measure in the sense of Def. Definition 1. From this, by Thm. Theorem 1, it follows immediately that \(\mu^*\) is a measure on the sigma-algebra of measurable sets \(\mathcal{M}(\mu^*)\).
\(\mu\) and \(\mu^*\) coincide on \(\mathcal{S}\). That is, \(\mu^*\) is an extension of \(\mu\).
\(\mathcal{S} \subset \mathcal{M}(\mu^*)\). Since \(\mathcal{M}(\mu^*)\) is a sigma-algebra, it follows that \(\sigma (\mathcal{S}) \subset \mathcal{M}(\mu^*)\).
We define \(\tilde{\mu }\) as the restriction of \(\mu^*\) to \(\sigma (\mathcal{S})\).
In the case where \(\mu\) is a sigma-finite measure, we will prove uniqueness later, using the theorem on \(\pi\)-\(\lambda\) systems.
The facts not yet proved are highlighted in red. Let us prove them one by one.
The next lemma will give us the first 2 facts: that \(\mu^*\) is an outer measure, and that \(\mu\) and \(\mu^*\) coincide on \(\mathcal{S}\).
Lemma 1 Let \(\mathcal{S} \subset 2^X\), \(\varnothing \in \mathcal{S}\), and let \(\mu : \mathcal{S} \to [0,+\infty ]\) be a monotone set function with \(\mu (\varnothing ) = 0\). Then \(\mu^*\), as defined above, is an outer measure. If, in addition, \(\mu\) is \(\sigma\)-subadditive on \(\mathcal{S}\), then \[
\mu ^*(A) = \mu (A), \quad \forall A \in \mathcal{S}
\]
Prove that, under the conditions of Lem. Lemma 1, \(\mu^*\) is indeed an outer measure.
The first 2 properties of an outer measure are obvious. Let us prove countable subadditivity. Let \((A_n)_{n=1}^\infty \subset 2^{\mathbb {R}}\) be a cover of \(A \subset \mathbb {R}\), i.e., \(A \subset \bigcup_{n=1}^\infty A_n\). For an arbitrary \(n \in \mathbb {N}\) let \((\mathcal{F}_{n,k})_{k=1}^\infty\) be a sequence of at most countable covers of \(A_n\) such that \[
\mu ^*(A_n) = \lim _{k \to \infty } \sum _{F \in \mathcal{F}_{n,k}} \mu (F)
\] Note that \(\forall k \in \mathbb {N}\) the set \(\bigcup_{n=1}^\infty \mathcal{F}_{n,k}\) covers all of \(A_1, A_2, \dots\), i.e., it covers \(A\). At the same time it is, clearly, at most countable. That is, \(\bigcup_{n=1}^\infty \mathcal{F}_{n,k} \in \mathcal{U}(A)\) and \[
\begin{align} \mu ^*(A) & \leq \sum _{F \in \bigcup _{n=1}^\infty \mathcal{F}_{n,k}} \mu (F) = \sum _{n=1}^\infty \sum _{F \in \mathcal{F}_{n,k}} \mu (F) \\ \mu ^*(A) & \leq \lim _{k \to \infty }\sum _{n=1}^\infty \sum _{F \in \mathcal{F}_{n,k}} \mu (F) = \sum _{n=1}^\infty \lim _{k \to \infty } \sum _{F \in \mathcal{F}_{n,k}} \mu (F) = \\ & = \sum _{n=1}^\infty \mu ^*(A_n) \end{align}
\]
Prove that, under the conditions of Lem. Lemma 1, if \(\mu\) is sigma-subadditive, then \(\mu^*(A) = \mu (A)\), \(\forall A \in \mathcal{S}\).
Prove that \(\mathcal{S} \subset \mathcal{M}(\mu^*)\), i.e., that every element of the semiring \(\mathcal{S}\) is \(\mu^*\)-measurable.
Let \(A \in \mathcal{S}\) be arbitrary. We need to show that \(A \in \mathcal{M}(\mu^*)\): for an arbitrary \(E \subset X\) we need to prove that \[
\mu ^*(E) \geq \mu ^*(E \cap A) + \mu ^*(E \setminus A)
\]
If \(\mu^*(E) = +\infty\), then this holds automatically. So from now on we assume that \(\mu^*(E) < \infty\).
Let \(\varepsilon > 0\) be arbitrary. Let us prove that
\[
\mu ^*(E) \geq \mu ^*(E \cap A) + \mu ^*(E \setminus A) - \varepsilon
\]
We can then let \(\varepsilon\) tend to zero and obtain what is required.
Let \(B_1, B_2, \ldots \in \mathcal{S}\) be a countable cover of \(E\) by elements of the semiring such that
\[
\mu ^*(E) + \varepsilon \geq \sum _{n=1}^{\infty } \mu \left(B_n\right)
\]
We can choose such a cover by the definition of the outer Lebesgue measure.
Note that \(B_1 \cap A, B_2 \cap A, \ldots\) is a countable cover of \(E \cap A\) by elements of the semiring \(\mathcal{S}\), hence, by the definition of the outer Lebesgue measure,
\[
\mu ^*(E \cap A) \leq \sum _{n=1}^{\infty } \mu \left(B_n \cap A\right)
\]
Similarly, for \(E \setminus A\) we can write:
\[
\mu ^*(E \setminus A) \leq \sum _{n=1}^{\infty } \mu \left(B_n \setminus A\right)
\]
Adding these inequalities, we get
\[
\begin{align} \mu ^*(E \cap A) + \mu ^*(E \setminus A) & \leq \sum _{n=1}^{\infty } \left[\underbrace{\mu \left(B_n \cap A\right) + \mu \left(B_n \setminus A\right)}_{=\mu (B_n) \text{ since } \mu \text{ is additive on } \mathcal{S}}\right] = \sum _{n=1}^{\infty } \mu \left(B_n\right) \leq \\ & \leq \mu ^*(E) + \varepsilon \end{align}
\]
Find the inaccuracy in the previous proof, i.e., in Example Example 2. How should it be corrected?
Let \(X = \mathbb {Q}, \; \mathcal{S} = \left\{ (a, b] \cap \mathbb {Q}: -\infty < a \leq b < + \infty , a,b \in \mathbb {Q}\right\}\).
Prove that \(\mathcal{S}\) is a semiring, and \(\sigma (\mathcal{S}) = 2^{\mathbb {Q}}\).
Define the counting measure on the semiring \(\mathcal{S}\):
\[
\begin{align} \mu (A) := \begin{cases} \left|A\right|, & \left|A\right| < \left|\mathbb {N}\right|, \\ +\infty , & \left|A\right| = \left|\mathbb {N}\right| \end{cases}, \qquad A \in \mathcal{S} \end{align}
\]
Prove that \(\mu\) is a premeasure on \((\mathbb {Q}, \mathcal{S})\) that is not \(\sigma\)-finite.
- Define the counting measure on the sigma-algebra \(2^{\mathbb {Q}}\):
\[
\begin{align} \# (A) := \begin{cases} \left|A\right|, & \left|A\right| < \left|\mathbb {N}\right|, \\ +\infty , & \left|A\right| = \left|\mathbb {N}\right| \end{cases}, \qquad A \in 2^{\mathbb {Q}} \end{align}
\]
Prove that \(\#\) and \(2\#\) are two distinct \(\sigma\)-finite measures on \(\sigma (\mathcal{S}) = 2^{\mathbb {Q}}\) that are both extensions of \(\mu\). Here \(2\#\) means the counting measure with coefficient \(2\) (clearly, one could in fact take any coefficient).
- Let \(\mu^*: 2^{\mathbb {Q}} \to [0, +\infty ]\) be the outer Lebesgue measure generated by the measure \(\mu\). Prove that
\[
\mu ^*(A) = \begin{cases} 0, & A =\varnothing \\ +\infty , & A \neq \varnothing \end{cases}
\]
Prove that \(\mu^*\) is not \(\sigma\)-finite.
- Prove that \(\mathcal{M}(\mu^*) = 2^{\mathbb {Q}}\). That is, you need to prove that \(\mu^*\) is not merely an outer measure on \(2^{\mathbb {Q}}\), but a genuine measure (by the theorem on outer measure, we know that an outer measure on the sigma-algebra of its measurable sets is a measure).
This problem shows that if \(\mu\) is not sigma-finite, there may be several extensions to \(\sigma (\mathcal{S})\), some of which may be sigma-finite (like \(\#\), \(2\#\), \(7\#\), ), and some of which may not be (like \(\mu^*\)).
A sketch, not fully rigorous. Let \(X = \left\{ 0,1\right\}^{\mathbb {N}}\) be the set of all infinite sequences of zeros and ones (think of \(x=(x_1,x_2,\ldots ) \in X\) as the outcomes of an infinite series of coin tosses). For \(n\in \mathbb {N}\) and \((\varepsilon_1,\ldots ,\varepsilon_n)\in \left\{ 0,1\right\}^n\) define the cylinder \[
C(\varepsilon _1,\ldots ,\varepsilon _n) := \left\{ x \in X \; : \; x_1=\varepsilon _1,\ldots ,x_n=\varepsilon _n\right\} \subset X.
\] Let \(\mathcal S\) be the set of all cylinders (over all \(n\) and all tuples \(\varepsilon\)), \(\mu \left(C(\varepsilon_1,\ldots ,\varepsilon_n)\right) := 2^{-n}\) (all \(2^n\) outcomes of the first \(n\) tosses are equally likely). How can we use this data to construct a probability measure on \(X\) corresponding to an infinite sequence of independent fair coin tosses?
A content (finitely additive set function). \(\mathcal S\) is a semiring (check this), and \(\mu\) is additive on it: the cylinder \(C(\varepsilon_1,\ldots ,\varepsilon_n)\) splits exactly into two cylinders of length \(n+1\) — \(C(\varepsilon_1,\ldots ,\varepsilon_n,0)\) and \(C(\varepsilon_1,\ldots ,\varepsilon_n,1)\) — and \(2^{-n} = 2^{-(n+1)}+2^{-(n+1)}\), so additivity on partitions of this form is obvious (an arbitrary finite partition of a cylinder into cylinders reduces to this).
Premeasure (the subtlest point). We need to check countable additivity: if a cylinder \(C\) splits into a disjoint countable union of cylinders \(C = \bigsqcup_{k=1}^\infty C_k\), then \(\mu (C) = \sum_k \mu (C_k)\). This is no longer elementary: intuitively, we need to rule out the situation where “mass escapes to infinity” through cylinders that are arbitrarily long but together still cover \(C\). The standard (and elegant) way to prove this is to note that \(X=\left\{ 0,1\right\}^{\mathbb {N}}\) is compact with respect to the natural product topology (Tychonoff’s theorem), and that cylinders in it are simultaneously open and closed; then countable additivity reduces to the fact that a decreasing sequence of nonempty closed sets in a compact space cannot have empty intersection unless it is already empty from some point on. We will not carefully work through this here — it is the first step toward the Kolmogorov extension theorem, the most important tool of the theory of random processes, and in full is the subject of a separate course.
The extension. Taking on faith that \(\mu\) is a premeasure, we apply Carathéodory’s theorem (Theorem 2): \(\mu\) (which is \(\sigma\)-finite, even finite: \(\mu (X)=1\)) extends uniquely to a probability measure \(\mathbb {P}\) on \(\sigma (\mathcal S)\). This \(\sigma (\mathcal S)\) is precisely the Borel sigma-algebra of the space \(X\) (with respect to the same product topology), and \(\mathbb {P}\) is exactly the distribution of an infinite sequence of independent fair coins that was already used implicitly in the first part of the course.
The monotone class theorem, the \(\pi\)-\(\lambda\) theorem, the good sets method
Definition 3 A family \(\mathcal{S} \subset 2^{\Omega }\) is a \(\pi\)-system over \(\Omega\) if it is nonempty and closed under pairwise intersections.
Definition 4 A sequence of sets \(A_1, A_2, \dots\) is called increasing if \(A_n \subset A_{n+1}, \; \forall n \in \mathbb {N}\), and decreasing if \(A_n \supset A_{n+1}, \; \forall n \in \mathbb {N}\).
Definition 5 A family \(\mathcal{S}\) is a \(\lambda\)-system (or Dynkin system) if
\(\Omega \in \mathcal{S}\);
If \(A, B \in \mathcal{S}, A \subset B\), then \(B \setminus A \in \mathcal{S}\);
It is closed under increasing countable unions, i.e., if \(A_1, A_2, \dots \in \mathcal{S}\) and \(A_1 \subset A_2 \subset A_3 \subset \dots\), then
\[
\bigcup _{n=1}^\infty A_n \in \mathcal{S}
\] The smallest \(\lambda\)-system containing an arbitrary system \(\mathcal{D}\) is denoted \(\lambda (\mathcal{D})\)
Definition 6 A family of subsets \(\mathcal{S}\) of a set \(\Omega\) is called a monotone class if it is closed under increasing countable unions and decreasing countable intersections, i.e.,
If \(A_1, A_2, \dots \in \mathcal{S}, A_1 \subset A_2 \subset \dots\), then \(\bigcup_{i=1}^n A_i \in \mathcal{S}\)
If \(B_1, B_2, \dots \in \mathcal{S}, B_1 \supset B_2 \supset \dots\), then \(\bigcap_{i=1}^n B_i \in \mathcal{S}\) The smallest monotone class containing an arbitrary system \(\mathcal{D}\) is denoted \(M(\mathcal{D})\)
Theorem 3 Let \(\Omega\) be an arbitrary set, \(\mathcal{S} \subset 2^\Omega\) a family of subsets. The following conditions are equivalent:
\(\mathcal{S}\) is a \(\sigma\)-algebra;
\(\mathcal{S}\) is simultaneously a \(\pi\)-system and a \(\lambda\)-system;
\(\mathcal{S}\) is simultaneously an algebra and a monotone class.
Theorem 4 (Monotone class theorem) Let \(\mathcal{A}\) be an algebra contained in some monotone class \(\mathcal{E}\), i.e., \(\mathcal{A} \subset \mathcal{E}\). Then \(\sigma (\mathcal{A}) \subset \mathcal{E}\).
In particular, for an algebra \(\mathcal{A}\) we have \(\sigma (\mathcal{A}) \subset M(\mathcal{A})\). Any sigma-algebra is a monotone class, so there are more monotone classes, meaning that taking the intersection over all monotone classes gives us a smaller set:
\[
\sigma (\mathcal{A}) = \bigcap _{\substack {\mathcal{F} \supset \mathcal{A} \\ \mathcal{F} \text{ -- $\sigma $-algebra}}} \mathcal{F} \quad \supset \quad \bigcap _{\substack {\mathcal{E} \supset \mathcal{A} \\ \mathcal{E} \text{ -- mon.} \\ \text{ class}}} \mathcal{E} = M(\mathcal{A})
\]
That is, \(\sigma (\mathcal{A}) \supset M(\mathcal{A})\). We finally obtain \(\sigma (\mathcal{A}) = M(\mathcal{A})\) (this statement is sometimes also called the monotone class theorem).
Theorem 5 (On \(\pi\)-\(\lambda\) systems) Let \(\mathcal{I}\) be a \(\pi\)-system contained in some \(\lambda\)-system \(\mathcal{D}\), i.e., \(\mathcal{I} \subset \mathcal{D}\). Then \(\sigma (\mathcal{I}) \subset \mathcal{D}\). In particular, \(\sigma (\mathcal{I}) \subset \lambda (\mathcal{I})\).
Prove the \(\pi\)-\(\lambda\) theorem.
Denote by \(\lambda (\mathcal{I})\) the smallest \(\lambda\)-system containing \(\mathcal{I}\). It suffices to prove that \(\lambda (\mathcal{I})\) is a \(\sigma\)-algebra, since then \(\sigma (\mathcal{I}) \subset \lambda (\mathcal{I}) \subset \mathcal{D}\).
First let us prove that \(\lambda (\mathcal{I})\) is an algebra. The axioms of an algebra:
\(\Omega \in \lambda (\mathcal{I})\)? By the definition of a \(\lambda\)-system, it contains \(\Omega\).
Is \(\lambda (\mathcal{I})\) closed under complements? \(A \in \lambda (\mathcal{I}) \Rightarrow A^c = \Omega \setminus A \in \lambda (\mathcal{I})\).
Is \(\lambda (\mathcal{I})\) closed under finite intersections? This is the laborious point. For \(A \in \lambda (\mathcal{I})\) define:
\[
\mathcal{D}_A = \{ B \in \lambda (\mathcal{I}) : A \cap B \in \lambda (\mathcal{I}) \} .
\]
Let us check that \(\mathcal{D}_A\) is a \(\lambda\)-system:
\(\Omega \in \mathcal{D}_A\), since \(A \cap \Omega = A \in \lambda (\mathcal{I})\).
Let \(B_1, B_2 \in \mathcal{D}_A\), \(B_1 \subset B_2\). Then
\[
A \cap (B_2 \setminus B_1) = (A \cap B_2) \setminus (A \cap B_1) \in \lambda (\mathcal{I}),
\]
since \(A \cap B_2, A \cap B_1 \in \lambda (\mathcal{I})\) and \(A \cap B_1 \subset A \cap B_2\).
Let \(B_n \in \mathcal{D}_A\), \(B_n \uparrow B\). Then \(A \cap B_n \uparrow A \cap B\), hence \(A \cap B \in \lambda (\mathcal{I})\), that is, \(B \in \mathcal{D}_A\).
If \(A \in \mathcal{I}\), then for any \(B \in \mathcal{I}\) we have \(A \cap B \in \mathcal{I} \subset \lambda (\mathcal{I})\), hence \(\mathcal{I} \subset \mathcal{D}_A\). But \(\mathcal{D}_A\) is a \(\lambda\)-system, so \(\lambda (\mathcal{I}) \subset \mathcal{D}_A\).
Now let \(A \in \lambda (\mathcal{I})\). For any \(B \in \mathcal{I}\), by the previous step \(B \in \mathcal{D}_A\), that is, \(A \cap B \in \lambda (\mathcal{I})\). This means that \(\mathcal{I} \subset \mathcal{D}_A\), and again \(\lambda (\mathcal{I}) \subset \mathcal{D}_A\).
Thus, for any \(A, B \in \lambda (\mathcal{I})\) we have \(B \in \mathcal{D}_A\), that is, \(A \cap B \in \lambda (\mathcal{I})\).
So, \(\lambda (\mathcal{I})\) is an algebra. For a \(\sigma\)-algebra it suffices to check closure under countable unions. Let \(A_n \in \lambda (\mathcal{I})\). Set
\[
B_n = A_1 \cup \dots \cup A_n.
\]
Then \(B_n \uparrow \bigcup_{n=1}^\infty A_n\). But \(B_n \in \lambda (\mathcal{I})\), since
\[
B_n = A_1 \cup (A_2 \setminus A_1) \cup (A_3 \setminus (A_1 \cup A_2)) \cup \dots
\]
and \(\lambda (\mathcal{I})\) is closed under differences and finite unions (the latter follows from closure under intersections and De Morgan’s formula).
In particular, for a \(\pi\)-system of subsets \(\mathcal{I}\) we have \(\sigma (\mathcal{I}) \subset \lambda (\mathcal{I})\). Since any \(\sigma\)-algebra is a \(\lambda\)-system, we have \(\sigma (\mathcal{I}) \supset \lambda (\mathcal{I})\). We finally obtain \(\sigma (\mathcal{I}) = \lambda (\mathcal{I})\) (this statement is sometimes also called the \(\pi\)-\(\lambda\) theorem).
The good sets method
The \(\pi\)-\(\lambda\) theorem (or the monotone class theorem) is very convenient when one needs to prove that some condition \(P\) holds for all elements of some sigma-algebra \(\mathcal{F}\). Denote by \(\mathcal{S}_P \subset 2^\Omega\) those subsets of \(\Omega\) (not necessarily lying in \(\mathcal{F}\)) for which condition \(P\) holds. These are the so-called good sets. We need to prove that \(\mathcal{F} \subset \mathcal{S}_P\). The proof proceeds in 2 stages. First, we need to find a \(\pi\)-system (or algebra) \(\mathcal{I}\) such that \(\sigma (\mathcal{I}) = \mathcal{F}\), and for which condition \(P\) holds, i.e., \(\mathcal{I} \subset \mathcal{S}_P\). Next, we need to prove that \(\mathcal{S}_P\) is a \(\lambda\)-system (or, respectively, a monotone class). As a result, by the theorem, we obtain \(\mathcal{F} = \sigma (\mathcal{I}) = \lambda (\mathcal{I}) \subset \mathcal{S}_P\) (respectively, \(\mathcal{F} = \sigma (\mathcal{I}) = M(\mathcal{I}) \subset \mathcal{S}_P\)).
Theorem 6 (For probability measures to coincide, it suffices that they coincide on a generating \(\pi\)-system) Let \(\mathbb {P}_{1}, \mathbb {P}_{2}\) be 2 probability measures on \((\Omega , \mathcal{F})\), where \(\mathcal{F}\) is a sigma-algebra on \(\Omega\). Suppose their values coincide on some \(\pi\)-system \(\mathcal{I}\), i.e., \(\mathbb {P}_{1}|_{\mathcal{I}} \equiv \mathbb {P}_{2}|_{\mathcal{I}}\), and moreover this \(\pi\)-system generates the sigma-algebra: \(\sigma (\mathcal{I}) = \mathcal{F}\). Prove that \(\mathbb {P}_{1} \equiv \mathbb {P}_{2}\), i.e., that they coincide on all of \(\mathcal{F}\).
In other words, for 2 probability measures to coincide (on some sigma-algebra), it suffices that they coincide on some pi-system that generates the sigma-algebra.
Prove theorem Theorem 6.
Let \(\mathcal{S}\) be our good sets, i.e., all those sets on which \(\mathbb {P}_{1}, \mathbb {P}_{2}\) coincide:
\[
\mathcal{S} = \left\{ B \in \mathcal{F} \; : \; \mathbb {P}_{1}\left(B\right) = \mathbb {P}_{2}\left(B\right)\right\}
\]
We know that \(\mathcal{I} \subset \mathcal{S}\). We want to show that \(\mathcal{F} \subset \mathcal{S}\). Suppose we have managed to prove that \(\mathcal{S}\) is a \(\lambda\)-system. Then, by the \(\pi\)-\(\lambda\) theorem, \(\mathcal{F} = \sigma (\mathcal{I}) = \lambda (\mathcal{I}) \subset \mathcal{S}\), and the proof is complete.
It remains to prove that \(\mathcal{S}\) is a \(\lambda\)-system.
\(\Omega \in \mathcal{S}\)? We are working with probability measures, so \(1 = \mathbb {P}_{1}\left(\Omega \right) = \mathbb {P}_{2}\left(\Omega \right)\). Hence \(\Omega \in \mathcal{S}\).
If \(A, B \in \mathcal{S}, A \subset B\), is \(B \setminus A \in \mathcal{S}\)? We have \(B = A \sqcup (B \setminus A)\). By additivity of measure we get
\[
\mathbb {P}_{1}\left(A\right) + \mathbb {P}_{1}\left(B \setminus A\right) = \mathbb {P}_{1}\left(B\right) = \mathbb {P}_{2}\left(B\right) = \mathbb {P}_{2}\left(A\right) + \mathbb {P}_{2}\left(B \setminus A\right)
\]
Since the measures are finite, \(\mathbb {P}_{1}\left(A\right), \mathbb {P}_{2}\left(A\right) < \infty\), so we can cancel them and are left with
\[
\mathbb {P}_{1}\left(B \setminus A\right) = \mathbb {P}_{2}\left(B \setminus A\right)
\]
Is there closure under increasing countable unions, i.e., if \(A_1, A_2, \dots \in \mathcal{S}\) and \(A_1 \subset A_2 \subset A_3 \subset \dots\), is \(\bigcup_{n=1}^\infty A_n \in \mathcal{S}\)? By continuity from below for measures we get
\[
\mathbb {P}_{1}\left(\bigcup _{n=1}^\infty A_n\right) = \lim _{n \to \infty }\mathbb {P}_{1}\left(A_n\right) = \lim _{n \to \infty }\mathbb {P}_{2}\left(A_n\right) = \mathbb {P}_{2}\left(\bigcup _{n=1}^\infty A_n\right)
\]
Hence \(\bigcup_{n=1}^\infty A_n \in \mathcal{S}\).
Proved.
A more general case: we weaken the requirement on the measures, allowing them to be non-probability measures as well. But we are forced to strengthen the requirements on the \(\pi\)-system.
Theorem 7 (For \(\sigma\)-finite measures to coincide, it suffices that they coincide on a generating \(\pi\)-system that also contains sets covering the whole space, on which the measures are finite) Let \(\mu_{1}, \mu_{2}\) be 2 \(\sigma\)-finite measures on a measurable space \((\Omega , \mathcal{F})\). Suppose their values coincide on some \(\pi\)-system \(\mathcal{I}\), i.e., \(\mu_{1}|_{\mathcal{I}} \equiv \mu_{2}|_{\mathcal{I}}\), and moreover
this \(\pi\)-system generates the sigma-algebra: \(\sigma (\mathcal{I}) = \mathcal{F}\);
there exist \(\Omega_1, \Omega_2, \ldots \in \mathcal{I}\) such that \(\bigcup_{n=1}^{\infty } \Omega_n = \Omega\) and \(\mu_{1}(\Omega_n) = \mu_2(\Omega_n) < \infty\) for every \(n \in \mathbb {N}\).
Prove that then \(\mu_{1} \equiv \mu_{2}\), i.e., that they coincide on all of \(\mathcal{F}\).
What happens if we drop condition from theorem Theorem 7? Namely, let \((\Omega , \mathcal{F}) = (\mathbb {Z}, 2^{\mathbb {Z}})\), and let \[
\mathcal{I} = \left\{ [-n,n] \cap \mathbb {Z} \; : \; n \in \mathbb {N}\right\}
\]
Prove that \(\mathcal{I}\) is a \(\pi\)-system.
Come up with different probability measures on \((\mathbb {Z}, 2^{\mathbb {Z}})\) that coincide on \(\mathcal{I}\).
What sigma-algebra does \(\mathcal{I}\) generate?
What happens if we drop condition from theorem Theorem 7? Namely, let \((\Omega , \mathcal{F}) = (\mathbb {Z}, 2^{\mathbb {Z}})\), and let \[
\mathcal{I} = \left\{ (-\infty ,n] \cap \mathbb {Z} \; : \; n \in \mathbb {Z}\right\}
\]
Prove that \(\mathcal{I}\) is a \(\pi\)-system.
Prove that \(\mathcal{I}\) generates \(2^{\mathbb {Z}}\): \(\sigma (\mathcal{I}) = 2^{\mathbb {Z}}\).
Come up with different \(\sigma\)-finite measures on \((\mathbb {Z}, 2^{\mathbb {Z}})\) that coincide on \(\mathcal{I}\).
Completeness of the extension
Definition 7 Let \(\nu\) be a measure on a measurable space \((X, \mathcal{F})\). A set \(B\) is called \(\nu\)-negligible if \(\exists A \in \mathcal{F}: \; B \subseteq A, \nu (A) = 0\). Note that it is not required that \(B \in \mathcal{F}\). Denote by \[
\mathcal{N}_\nu := \bigcup _{A \in \mathcal{F}: \nu (A) = 0} 2^A
\] the family of all \(\nu\)-negligible sets. The measure \(\nu\) is called complete on \((X, \mathcal{F})\) if \(\mathcal{N}_\nu \subset \mathcal{F}\), i.e., all negligible sets are measurable (and their measure is, respectively, equal to zero). Note that although the notion of completeness is applied to the measure, all 3 components \(X, \mathcal{F}, \nu\) matter in the definition of completeness.
Let \(a \in \mathbb {R}\). Let \(\mathcal{F} = \sigma (\left\{ b\right\} : b \in \mathbb {R} \setminus \left\{ a\right\} )\). Describe \(\mathcal{F}\). Prove that the Dirac measure \(\delta_a\) (unit mass at the point \(a\)) is a complete measure on \((\mathbb {R}, \mathcal{F})\).
Lemma 2 (On the completeness of the extension) Suppose we are in the same setting as in Thm. Theorem 2 (and suppose \(\mu\) is \(\sigma\)-finite on \(\mathcal{S}\)). The measure \(\mu = \mu^*|_{\sigma (\mathcal{S})}\) on \((X, \sigma (\mathcal{S}))\) may fail to be complete, but the measure \(\mu^*|_{\mathcal{M}(\mu^*)}\) is always complete on the space \((X, \mathcal{M}(\mu^*))\). Moreover, \((X, \mathcal{M}(\mu^*), \mu^*_{\mathcal{M}(\mu^*)})\) is the completion of \((X, \sigma (\mathcal{S}), \mu^*_{\sigma (\mathcal{S})})\), in the sense that the sigma-algebra \(\mathcal{M}(\mu^*)\) is the minimal sigma-algebra containing \(\sigma (\mathcal{S})\) on which one can define a complete extension of the measure \(\mu^*_{\sigma (\mathcal{S})}\). Moreover, the following holds: \[
\begin{align} \mathcal{M}(\mu ^*) & = \sigma \left(\sigma (\mathcal{S}) \cup \mathcal{N}_{\mu ^*_{\sigma (\mathcal{S})}}\right) = \left\{ A \cup B: A \in \sigma (\mathcal{S}), B \in \mathcal{N}_{\mu ^*_{\sigma (\mathcal{S})}}\right\} \\ \mu ^*|_{\mathcal{M}(\mu ^*)}(A \cup B) & = \mu ^*|_{\sigma (\mathcal{S})}(A), \qquad A \in \sigma (\mathcal{S}), B \in \mathcal{N}_{\mu ^*_{\sigma (\mathcal{S})}} \end{align}
\]
Let \(X = [0,1]\) (or any other uncountable set), \[
\mathcal{S} = \left\{ A \subset [0,1]: \text{ either } A \text{, or } A^\complement \text{ is at most countable}\right\}
\] Note that \(\mathcal{S}\) is a \(\sigma\)-algebra. Define the counting measure on \(\mathcal{S}\): \[
\mu (A) = \begin{cases} \left|A\right|, & \left|A\right| < \left|\mathbb {N}\right| \\ +\infty , & \left|A\right| \geq \left|\mathbb {N}\right| \end{cases}
\]
Prove that
\(([0,1], \mathcal{S}, \mu )\) is a measure space, and it is not \(\sigma\)-finite;
\(\mu\) is complete on \(([0,1], \mathcal{S})\);
Let \(\mu^*\) be the outer measure generated by the measure \(\mu\) (via countable covers). Prove that \(\mathcal{M}(\mu^*) = 2^{[0,1]}\).
This problem shows that in the completion lemma it is important that \(\mu\) be sigma-finite from the outset. Otherwise, as happens here, \((X, \mathcal{M}(\mu^*), \mu^*|_{\mathcal{M}(\mu^*)})\) will not be the completion of \((X, \sigma (\mathcal{S}), \mu^*|_{\sigma (\mathcal{S})})\): in this case the space \((X, \sigma (\mathcal{S}) = \mathcal{S}, \mu^*|_{\sigma (\mathcal{S})} = \mu )\) is already complete, while \(\mathcal{S} \neq \mathcal{M}(\mu^*)\).
References
Klenke, Achim. 2020. Probability Theory: A Comprehensive Course. 3rd ed. Springer Nature.