By the use of characteristic functions, we may give a solution to the problem of addition of independent random variables. Let \(X_{1}, X_{2}, \ldots, X_{n}\) be \(n\) independent random variables, with respective characteristic functions \(\phi_{X_{1}}(\cdot), \ldots, \phi_{X_{n}}(\cdot)\) . Let \(S_{n}=X_{1}+X_{2}+\cdots+X_{n}\) be their sum. To know the probability law of \(S_{n}\) , it suffices to know its characteristic function \(\phi_{S_{n}}(\cdot)\) . However, it is immediate, from the properties of independent random variables, that for every real number \(u\)
\[\phi_{S_{n}}(u)=\phi_{X_{1}}(u) \cdots \phi_{X_{n}}(u) \tag{4.1}\]
or, equivalently, \(E\left[e^{i u\left(X_{1}+\cdots+X_{n}\right)}\right]=E\left[e^{i u X_{1}}\right] \cdots E\left[e^{i u X_{n}}\right]\) . Thus, in terms of characteristic functions, the problem of addition of independent random variables is given by (4.1) a simple and concise solution, which may also be stated in words: the probability law of a sum of independent random variables has as its characteristic function the product of the characteristic functions of the individual random variables.
In this section we consider certain cases in which (4.1) leads to an exact evaluation of the probability law of \(S_{n}\) . In Chapter 10 we show how (4.1) may be used to give a general approximate evaluation of the probability law of \(S_{n}\) .
There are various ways, given the characteristic function \(\phi_{S_{n}}(\cdot)\) of the sum \(S_{n}\) , in which one can deduce from it the probability law of \(S_{n}\) .
It may happen that \(\phi_{S_{n}}(\cdot)\) will coincide with the characteristic function of a known probability law. For example, for each \(k=1,2, \ldots, n\) suppose that \(X_{k}\) is normally distributed with mean \(m_{k}\) and variance \(\sigma_{k}^{2}\) . Then, \(\phi_{X_{k}}(u)=\exp \left(i u m_{k}-\frac{1}{2} u^{2} \sigma_{k}^{2}\right)\) , and, by (4.1),
\[\phi_{S_{n}}(u)=\exp \left[i u\left(m_{1}+\cdots+m_{n}\right)-\frac{1}{2} u^{2}\left(\sigma_{1}^{2}+\cdots+\sigma_{n}^{2}\right)\right].\]
We recognize \(\phi_{S_{n}}(\cdot)\) as the characteristic function of the normal distribution with mean \(m_{1}+\cdots+m_{n}\) and variance \(\sigma_{1}^{2}+\cdots+\sigma_{n}^{2}\) . Therefore, the sum \(S_{n}\) is normally distributed with mean \(m_{1}+\cdots+m_{n}\) and variance \(\sigma_{1}^{2}+\cdots+\sigma_{n}^{2}\) . By using arguments of this type, we have the following theorem.
Theorem 4A. Let \(S_{n}=X_{1}+\cdots+X_{n}\) be the sum of independent random variables.
If, for \(k=1, \ldots, n\) , \(X_{k}\) is \(N\left(m_{k}, \sigma_{k}^{2}\right)\) , then \(S_{n}\) is \(N\left(m_{1}+\cdots+m_{n}, \sigma_{1}^{2}+\cdots+\sigma_{n}^{2}\right)\)
If for \(k=1, \ldots, n\) , \(X_{k}\) is binomial distributed with parameters \(N_{k}\) and \(p\) , then \(S_{n}\) is binomial distributed with parameters \(N_{1}+\cdots+N_{n}\) and \(p\) .
If, for \(k=1, \ldots, n\) , \(X_{k}\) is Poisson distributed with parameter \(\lambda_{k}\) , then \(S_{n}\) is Poisson distributed with parameter \(\lambda_{1}+\cdots+\lambda_{n}\) .
If, for \(k=1, \ldots, n\) , \(X_{k}\) is \(\chi^{2}\) distributed with \(N_{k}\) degrees of freedom, then \(S_{n}\) is \(\chi^{2}\) distributed with \(N_{1}+\cdots+N_{n}\) degrees of freedom.
If, for \(k=1, \ldots, n\) , \(X_{k}\) is Cauchy distributed with parameters \(a_{k}\) and \(b_{k}\) , then \(S_{n}\) is Cauchy distributed with parameters \(a_{1}+\cdots+a_{n}\) and \(b_{1}+\cdots+b_{n}\) .
One may be able to invert the characteristic function of \(S_{n}\) to obtain its distribution function or probability density function. In particular, if \(\phi_{S_{n}}(\cdot)\) is absolutely integrable, then \(S_{n}\) has a probability density function for any real number \(x\) given by
\[f_{S_{n}}(x)=\frac{1}{2 \pi} \int_{-\infty}^{\infty} e^{-i u x} \phi_{S_{n}}(u) du. \tag{4.2}\]
In order to evaluate the infinite integral in (4.2), one will generally have to use the theory of complex integration and the calculus of residues.
Even if one is unable to invert the characteristic function to obtain the probability law of \(S_{n}\) in closed form, the characteristic function can still be used to obtain the moments and cumulants of \(S_{n}\) . Indeed, cumulants assume their real importance from the study of the sums of independent random variables because they are additive over the summands. More precisely, if \(X_{1}, X_{2}, \ldots, X_{n}\) are independent random variables whose \(r\) th cumulants exist, then the \(r\) th cumulant of the sum exists and is equal to the sum of the \(r\) th cumulants of the individual random variables . In symbols,
\[K_{r}\left[X_{1}+\cdots+X_{n}\right]=K_{r}\left[X_{1}\right]+\cdots+K_{r}\left[X_{n}\right]. \tag{4.3}\]
Equation (4.3) follows immediately from the fact that the \(r\) th cumulant is (up to a constant) the \(r\) th derivative at 0 of the logarithm of the characteristic function and the log-characteristic function is additive over independent summands, since the characteristic function is multiplicative.
The moments and central moments of a random variable may be expressed in terms of its cumulants. In particular, the first cumulant and the mean, the second cumulant and the variance, and the third cumulant and the third central moment, respectively, are equal. Consequently, the means, variances, and third central moments are additive over independent summands; more precisely, \begin{align} E\left[X_{1}+\cdots+X_{n}\right] & =E\left[X_{1}\right]+\cdots+E\left[X_{n}\right] \\[5mm] \operatorname{Var}\left[X_{1}+\cdots+X_{n}\right] & =\operatorname{Var}\left[X_{1}\right]+\cdots+\operatorname{Var}\left[X_{n}\right] \\[5mm] \mu_{3}\left[X_{1}+\cdots+X_{n}\right] & =\mu_{3}\left[X_{1}\right]+\cdots+\mu_{3}\left[X_{n}\right], \tag{4.4} \end{align} where, for any random variable \(X\) , we define \(\mu_{3}[X]=E\left[(X-E[X])^{3}\right]\) ; (4.4) may, of course, also be proved directly.
Exercises
4.1. Prove theorem 4A.
4.2. Find the probability laws corresponding to the following characteristic functions: (i) \(e^{-u^{2}}\) , (ii) \(e^{-|u|}\) , (iii) \(e^{\left(e^{i u}-1\right)}\) , (iv) \((1-2 i u)^{-2}\) .
4.3. Let \(X_{1}, X_{2}, \ldots, X_{n}\) be a sequence of independent random variables, each uniformly distributed over the interval 0 to 1 . Let \(S_{n}=X_{1}+X_{2}+\cdots+\) \(X_{n}\) . Show that for any real number \(y\) , such that \(0
\[f_{S_{n+1}}(y)=\int_{y-1}^{y} f_{S_{n}}(x) dx;\]
hence prove by mathematical induction that \begin{align} f_{S_{n}}(x) & = \begin{cases} \frac{1}{(n-1)!} \sum_{j=0}^{\lfloor x \rfloor} \binom{n}{j} (-1)^{j} (x - j)^{n-1}, & \text{if } 0 \leq x \leq n \\ 0, & \text{if } x < 0 \text{ or } x > n. \end{cases} \end{align}
4.4. Let \(X_{1}, X_{2}, \ldots, X_{n}\) be a sequence of independent random variables, each normally distributed with mean 0 and variance 1. Let \(S_{n}=X_{1}^{2}+X_{2}^{2}+\) \(\ldots+X_{n}^{2}\) . Show that for any real number \(y\) and integer \(n=1,2, \ldots\)
\[f_{S_{n+2}}(y)=\int_{0}^{y} f_{S_{2}}(y-x) f_{S_{n}}(x) dx.\]
Prove that \(f_{S_{2}}(y)=\frac{1}{2} e^{-1 / 2 y}\) for \(y>0\) ; hence deduce that \(S_{n}\) has a \(\chi^{2}\) distribution with \(n\) degrees of freedom.
4.5. Let \(X_{1}, X_{2}, \ldots, X_{n}\) be independent random variables, each normally distributed with mean \(m\) and variance 1. Let \(S=\sum_{j=1}^{n} X_{j}^{2}\) .
(i) Find the cumulants of \(S\) .
(ii) Let \(T=a Y_{\nu}\) for suitable constants \(a\) and \(v\) , in which \(Y_{v}\) is a random variable obeying a \(\chi^{2}\) distribution with \(v\) degrees of freedom. Determine \(a\) and \(v\) so that \(S\) and \(T\) have the same means and variances. Hint: Show that each \(X_{j}^{2}\) has the characteristic function
\[\phi_{X_{j}^{2}}(u)=\frac{1}{(1-2 i u)^{1 / 2}} \exp \left[-\frac{1}{2} m^{2}\left(1-\frac{1}{1-2 i u}\right)\right]\]
Answer
(i) \(k\) th cumulant of \(S\) is \(n 2^{k-1}(k-1) !\left(1+k m^{2}\right)\) ;
(ii) \(y=\frac{\left(1+m^{2}\right)^{2}}{1+2 m^{2}}\) , \(a=\frac{1+2 m^{2}}{1+m^{2}}\) .