Characteristic Functions

WarningDraft

This text is still being edited: it may contain typos, inaccuracies, and locally unpolished spots. If you find something, please let the author know (contact details on the course page).

The characteristic function is the Fourier transform of a distribution, and its appearance in the course is dictated by sheer pragmatism: convolution turns into a product. The distribution of a sum of a hundred independent terms via convolution is a hundred-fold integral; via characteristic functions, it is the hundredth power of a single function. Add to this the uniqueness theorem (the characteristic function is a complete passport of the distribution) and the continuity theorem (pointwise convergence of characteristic functions is equivalent to weak convergence) — and you get the main computational tool of limit theorems: we will prove the CLT in topic 9 in five lines. This is the same trick an engineer uses to analyze a signal via its spectrum: an operation that is awkward in the time domain (filter convolution) becomes multiplication in the frequency domain.

1 Definition and properties

Definition 1 The characteristic function (CF) of a random variable \(\xi\) is \[ \varphi _\xi (t) = \mathbb {E}\left[e^{it\xi }\right] = \mathbb {E}\left[\cos (t\xi )\right] + i\, \mathbb {E}\left[\sin (t\xi )\right], \qquad t \in \mathbb {R}. \]

It is always defined (\(\left|e^{it\xi }\right| = 1\) — integrability comes for free, unlike for moment generating functions). Geometric interpretation: \(e^{it\xi }\) is a point on the unit circle, and \(\varphi_\xi (t)\) is the center of mass of the distribution wound onto the circle at speed \(t\); the more spread out the distribution, the closer the center of mass is to zero.

Proposition 1 (Properties of the CF)  

1. \(\varphi (0) = 1\), \(\left|\varphi (t)\right| \leq 1\), \(\varphi (-t) = \overline{\varphi (t)}\); \(\varphi\) is uniformly continuous on \(\mathbb {R}\).

2. \(\varphi_{a\xi + b}(t) = e^{itb}\varphi_\xi (at)\).

3. If \(\xi \; \perp \! \! \! \perp \; \eta\), then \(\varphi_{\xi + \eta }(t) = \varphi_\xi (t) \, \varphi_\eta (t)\).

4. \(\varphi\) is real \(\iff\) the distribution of \(\xi\) is symmetric (\(\xi \overset {d}{=} -\xi\)).

5. If \(\mathbb {E}\left[\left|\xi \right|^k\right] < \infty\), then \(\varphi \in C^k(\mathbb {R})\) and \(\varphi^{(k)}(0) = i^k \mathbb {E}\left[\xi^k\right]\); in particular,

   \[ \varphi (t) = 1 + it\, \mathbb {E}\left[\xi \right] - \frac{t^2}{2}\mathbb {E}\left[\xi ^2\right] + o(t^2), \qquad t \to 0. \]

Item 3 is the heart of the topic (proof: \(\mathbb {E}\left[e^{it(\xi +\eta )}\right] = \mathbb {E}\left[e^{it\xi }\right]\mathbb {E}\left[e^{it\eta }\right]\) by independence — topic 1). Item 5 is “moments sit in the derivatives at zero”: smoothness of the CF at zero encodes lightness of the tails.

Theorem 1 (Uniqueness and inversion) The characteristic function uniquely determines the distribution. If in addition \(\varphi \in L^1(\mathbb {R})\), then the distribution has a bounded continuous density \[ p(x) = \frac{1}{2\pi }\int _{\mathbb {R}} e^{-itx}\varphi (t)\, dt. \]

Theorem 2 (Bochner-Khinchin; statement) A function \(\varphi\) with \(\varphi (0)=1\) is a characteristic function if and only if it is continuous and positive semi-definite: \(\sum_{j,k} \varphi (t_j - t_k) z_j \bar z_k \geq 0\) for any \(t_j \in \mathbb {R}\), \(z_j \in \mathbb {C}\).

Standard set of CFs (some are problems below, verify the rest yourself):

\(\operatorname {Bern}(p)\)	\(1 - p + pe^{it}\)	\(\mathscr {N}\left(a, \sigma ^2\right)\)	\(e^{ita - \sigma ^2t^2/2}\)
\(\operatorname {Bin}(n,p)\)	\((1-p+pe^{it})^n\)	\(\operatorname {Exp}(\lambda )\)	\(\frac{\lambda }{\lambda - it}\)
\(\operatorname {Pois}(\lambda )\)	\(e^{\lambda (e^{it}-1)}\)	\(\Gamma (\alpha , \lambda )\)	\(\left(\frac{\lambda }{\lambda - it}\right)^\alpha\)
\(\operatorname {U}[-1,1]\)	\(\frac{\sin t}{t}\)	\(\operatorname {Cauchy}(0,1)\)	\(e^{-\left\|t\right\|}\)

NoteInteractive illustration placeholder

The interactive illustration has not been created yet. See the source: /home/nedosh/Documents/online_math_notes/math-notes_FPMI_2/courses/nedosh_courses/courses/04-probability/02_PT/contents/specials/080/01_char_functions_fig.tex.

ProblemProblem 1

Compute the characteristic functions: (a) \(\operatorname {Pois}(\lambda )\); (b) \(\operatorname {U}[-1,1]\); (c) \(\operatorname {Exp}(\lambda )\); (d) the Laplace distribution with density \(\frac{1}{2}e^{-\left|x\right|}\). Deduce from (a) that a sum of independent Poisson variables is Poisson.

ProblemProblem 2

Prove that the characteristic function of the Cauchy distribution equals \(e^{-\left|t\right|}\). Deduce from this: if \(\xi_1, \ldots , \xi_n\) are independent standard Cauchy variables, then \(\frac{\xi_1 + \ldots + \xi_n}{n}\) is again a standard Cauchy variable. What does this say about the law of large numbers?

ProblemProblem 3

Recover the distribution from its characteristic function: (a) \(\varphi (t) = \cos t\); (b) \(\varphi (t) = \cos^2 t\); (c) \(\varphi (t) = \frac{1}{2 - e^{it}}\).

ProblemProblem 4

Does there exist a random variable with characteristic function \(\varphi (t) = e^{-t^4}\)?

ProblemProblem 5

Let \(\varphi\) be the characteristic function of a variable \(\xi\). Prove that \(\left|\varphi \right|^2\) is also a characteristic function. Of which variable?