A sequence of jointly distributed random variables \(X_{1}, X_{2}, \ldots, X_{n}\) with finite means and variances is said to obey the (classical) central limit theorem if the sequence \(Z_{1}, Z_{2}, \ldots, Z_{n}\) , defined by

\[Z_{n}=\frac{S_{n}-E\left[S_{n}\right]}{\sigma\left[S_{n}\right]}, \quad S_{n}=X_{1}+X_{2}+\cdots+X_{n}, \tag{4.1}\] 

converges in distribution to a random variable that is normally distributed with mean 0 and variance 1. In terms of characteristic functions, the sequence \(\left\{X_{n}\right\}\) obeys the central limit theorem if for every real number \(u\) 

\[\lim _{n \rightarrow \infty} \phi_{Z_{n}}(u)=e^{-1/2 u^{2}}. \tag{4.2}\] 

The random variables \(Z_{1}, Z_{2}, \ldots, Z_{n}\) are called the sequence of normalized consecutive sums of the sequence \(X_{1}, X_{2}, \ldots, X_{n}\) .

That the central limit theorem is true under fairly unrestrictive conditions on the random variables \(X_{1}, X_{2}, \ldots\) was already surmised by Laplace and Gauss in the early 1800’s. However, the first satisfactory conditions, backed by a rigorous proof, for the validity of the central limit theorem were given by Lyapunov in 1901. In the 1920’s and 1930’s the method of characteristic functions was used to extend the theorem in several directions and to obtain fairly unrestrictive necessary and sufficient conditions for its validity in the case in which the random variables \(X_{1}, X_{2}, \ldots\) are independent. More recent years have seen extensive work on extending the central limit theorem to the case of dependent random variables.

The reader is referred to the treatises of B. V. Gnedenko and A. N. Kolmogorov, Limit Distributions for Sums of Independent Random Variables , Addison-Wesley, Cambridge, Mass., 1954, and M. Loève, Probability Theory , Van Nostrand, New York, 1955, for a definitive treatment of the central limit theorem and its extensions.

From the point of view of the applications of probability theory, there are two main versions of the central limit theorem that one should have at his command. One should know conditions for the validity of the central limit theorem in the cases in which (i) the random variables \(X_{1}, X_{2}, \ldots\) are independent and identically distributed and (ii) the random variables \(X_{1}, X_{2}, \ldots\) are independent but not identically distributed.

Equation (4.4) is called Lyapunov’s condition for the validity of the central limit theorem for independent random variables \(\left\{X_{n}\right\}\) .

We turn now to the proofs of theorems 4A and 4B. Consider first independent random variables \(X_{1}, X_{2}, \ldots, X_{n}\) , identically distributed as the random variable \(X\) , with mean 0 and variance \(\sigma^{2}\) . Let \(Z_{n}\) be their normalized sum, given by (4.1). The characteristic function of \(Z_{n}\) may be written

\[\phi_{Z_{n}}(u)=\left[\phi_{X}\left(\frac{u}{\sigma\left[S_{n}\right]}\right)\right]^{n}.\] 

Now \(\sigma\left[S_{n}\right]=\sqrt{n} \sigma[X]\) tends to \(\infty\) as \(n\) tends to \(\infty\) . Therefore, for each fixed \(u, \log \phi_{X}\left(u / \sigma\left[S_{n}\right]\right)\) exists (by lemma 3A) when \(n \geq 3 u^{2}\) . For \(n\) as large as this, using the expansion given by (3.8),

\[\begin{array}{r} \log \phi_{Z_{n}}(u)=n\left\{-\frac{1}{2} \frac{u^{2}}{n}-\frac{u^{2}}{n \sigma^{2}} \int_{0}^{1} d t(1-t) E\left[X^{2}\left(e^{i t u X / \sqrt{n\sigma} }-1\right)\right]\right. \tag{4.6} \\ \left. +3 \theta \frac{u^{4}}{n^{2} \sigma^{4}} \sigma^{4}\right\} \end{array}\] 

Theorem 4A will be proved if we prove that

\[\lim _{n \rightarrow \infty} \log \phi_{Z_{n}}(u)=-\frac{1}{2} u^{2}. \tag{4.7}\] 

It is clear that to prove (4.7) will hold we need prove only that the integral in (4.6) tends to 0 as \(n\) tends to infinity. Define \(g(x, t, u)=x^{2}\left(e^{i t u x / \sqrt{n\sigma} }-1\right)\) . Then, for any \(M>0\) 

\[E[g(X, t, u)]=\int_{|x|

Now, \(|g(x, t, u)| \leq x^{2}|u t x| / \sigma \sqrt{n} \leq x^{2}|M u t| / \sigma \sqrt{n}\) for \(|x|and \(|g(x, t, u)|\) \(\leq 2 x^{2}\) for \(|x| \geq M\) , in view of the inequalities \(\left|e^{i w}-1\right| \leq|w|,\left|e^{i w}-1\right| \leq 2\) . From these facts we may conclude that for any \(M>0\) and real numbers \(u\) and \(t\) 

\[|E[g(X, t, u)]| \leq \sigma \frac{M|u t|}{\sqrt{n}}+2 \int_{|x| \geq M} x^{2} d F_{X}(x). \tag{4.9}\] 

Then

\[\left|\int_{0}^{1} d t(1-t) E[g(X, t, u)]\right| \leq \sigma \frac{M|u|}{\sqrt{n}}+2 \int_{|x| \geq M} x^{2} d F_{X}(x), \tag{4.10}\] 

which tends to 0, as we let first \(n\) tend to \(\infty\) and then \(M\) tend to \(\infty\) . The proof of the central limit theorem for identically distributed independent random variables with finite variances is complete.

We next prove the central limit theorem under Lyapunov’s condition. For \(k=1,2, \ldots\) , let \(X_{k}\) be a random variable with mean 0, finite variance \(\sigma_{k}^{2}\) , and \((2+\delta)\) th central moment \(\mu(2+\delta ; k)\) . We have the following expansion of the logarithm of its characteristic function, for \(u\) such that \(3 u^{2} \sigma_{k}^{2} \leq 1\) :

\[\log \phi_{X_{k}}(u)=-\frac{1}{2} u^{2} \sigma_{k}^{2}+2 \theta|u|^{2+\delta} \mu(2+\delta ; k)+3 \theta u^{4} \sigma_{k}^{4}.\tag{4.11}\] 

To prove (4.11), merely use in (3.8) the inequality \(\left|e^{i w}-1\right| \leq 2|w|^{\delta}\) , valid for any real number \(w\) and \(0 \leq \delta \leq 1\) .

Now, (4.4) and theoretical exercise 4.3 imply that

\[\left(\max _{1 \leq k \leq n} \frac{\sigma_{k}^{2}}{\sigma^{2}\left[S_{n}\right]}\right)^{(2+\delta) / 2} \leq\left(\max _{1 \leq k \leq n} \frac{\mu(2+\delta ; k)}{\sigma^{2+\delta}\left[S_{n}\right]}\right) \rightarrow 0.\tag{4.12}\] 

Then, for any fixed \(u\) it holds for \(n\) sufficiently large that \(3 u^{2} \sigma_{k}^{2} / \sigma^{2}\left[S_{n}\right] \leq 1\) for all \(k=1,2, \ldots, n\) . Therefore, \(\log \phi_{Z_{n}}(u)\) exists and is given by

\begin{align} \log \phi_{Z_{n}}(u)= & \sum_{k=1}^{n} \log \phi_{X_{k}}\left(\frac{u}{\sigma\left[S_{n}\right]}\right)=-\frac{1}{2} u^{2} \sum_{k=1}^{n} \frac{\sigma_{k}^{2}}{\sigma^{2}\left[S_{n}\right]} \tag{4.13} \\ & +2 \theta|u|^{2+\delta} \sum_{k=1}^{n} \frac{\mu(2+\delta ; k)}{\sigma^{2+\delta}\left[S_{n}\right]}+3 \theta u^{4} \frac{1}{\sigma^{4}\left[S_{n}\right]} \sum_{k=1}^{n} \sigma_{k}^{4}. \end{align} 

The first sum in (4.13) is equal to 1, whereas the second sum tends to 0 by Lyapunov’s condition, as does the third sum, since

\[\left(\frac{\sigma_{k}}{\sigma\left[S_{n}\right]}\right)^{4} \leq\left(\frac{\sigma_{k}}{\sigma\left[S_{n}\right]}\right)^{2+\delta} \leq \frac{\mu(2+\delta ; k)}{\sigma^{2+\delta}\left[S_{n}\right]}.\] 

The proof of the central limit theorem under Lyapunov’s condition is complete.

Theoretical exercises