In section 9 we treated in some detail the problem of obtaining the individual probability law of a function of random variables. It is natural to consider next the problem of obtaining the joint probability law of several random variables which arise as functions. In principle, this problem is no different from those previously considered. However, the details are more complicated. Consequently, in this section, we content ourselves with stating an often-used formula for the joint probability density function of \(n\) random variables \(Y_{1}, Y_{2}, \ldots, Y_{n}\) , which arise as functions of \(n\) jointly continuous random variables \(X_{1}, X_{2}, \ldots, X_{n}\) : \begin{align} Y_{1}=g_{1}\left(X_{1}, X_{2}, \ldots, X_{n}\right), & Y_{2}=g_{2}\left(X_{1}, X_{2}, \ldots, X_{n}\right), \cdots \\ & Y_{n}=g_{n}\left(X_{1}, X_{2}, \ldots, X_{n}\right). \end{align} We consider only the case in which the functions \(g_{1}\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) , \(g_{2}\left(x_{1}, x_{2}, \ldots, x_{n}\right), g_{n}\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) have continuous first partial derivatives at all points \(\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) and are such that the Jacobian
\[J(x_1, x_2, \dots, x_n) = \begin{vmatrix} \dfrac{\partial g_1}{\partial x_1} & \dfrac{\partial g_1}{\partial x_2} & \dots & \dfrac{\partial g_1}{\partial x_n} \\ \dfrac{\partial g_2}{\partial x_1} & \dfrac{\partial g_2}{\partial x_2} & \dots & \dfrac{\partial g_2}{\partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \dfrac{\partial g_n}{\partial x_1} & \dfrac{\partial g_n}{\partial x_2} & \dots & \dfrac{\partial g_n}{\partial x_n} \end{vmatrix} \neq 0\] at all points \(\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) . Let \(C\) be the set of points \(\left(y_{1}, y_{2}, \ldots, y_{n}\right)\) such that the \(n\) equations \begin{align} y_{1}&=g_{1}\left(x_{1}, x_{2}, \ldots, x_{n}\right),\\ y_{2}&=g_{2}\left(x_{1}, x_{2}, \ldots, x_{n}\right),\\ &\vdots\\ y_{n}&=g_{n}\left(x_{1}, x_{2}, \ldots, x_{n}\right) \end{aligned}\tag{10.3}\] possess at least one solution \(\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) . The set of equations in (10.3) then possesses exactly one solution, which we denote by \begin{align} x_{1}&=g^{-1}\left(y_{1}, y_{2}, \ldots, y_{n}\right),\\ x_{2}&=g^{-1}\left(y_{1}, y_{2}, \ldots, y_{n}\right),\\ &\vdots\\ x_{n}&=g^{-1}\left(y_{1}, y_{2}, \ldots, y_{n}\right). \end{aligned} \tag{10.4}\]
If \(X_{1}, X_{2}, \ldots, X_{n}\) are jointly continuous random variables, whose joint probability density function is continuous at all but a finite number of points in the \(\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) space, then the random variables \(Y_{1}, Y_{2}, \ldots, Y_{n}\) defined by (10.1) are jointly continuous with a joint probability density function given by \begin{align} & f_{Y_{1}, Y_{2}, \ldots, Y_{n}}\left(y_{1}, y_{2}, \ldots, y_{n}\right) \\[2mm] &= f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right) \left|J\left(x_{1}, x_{2}, \ldots, x_{n}\right)\right|^{-1},\tag{10.5} \end{align} if \(\left(y_{1}, y_{2}, \ldots, y_{n}\right)\) belongs to \(C\) , and \(x_{1}, x_{2}, \ldots, x_{n}\) are given by (10.4); for \(\left(y_{1}, y_{2}, \ldots, y_{n}\right)\) not belonging to \(C\) \begin{align} f_{Y_{1}, Y_{2}, \ldots, Y_{n}}\left(y_{1}, y_{2}, \ldots, y_{n}\right)=0, \tag{10.5$^\prime$} \end{align}
It should be noted that (10.5) extends (8.18) . We leave it to the reader to formulate a similar extension of (8.22) .
We omit the proof that the random variables \(Y_{1}, Y_{2}, \ldots, Y_{n}\) are jointly continuous and possess a joint probability density. We sketch a proof of the formula given by (10.5) for the joint probability density function. One may show that for any real numbers \(u_{1}, u_{2}, \ldots, u_{n}\) \begin{align} & f_{Y_{1}, Y_{2}, \ldots, Y_{n}}\left(u_{1}, u_{2}, \ldots, u_{n}\right) \\[2mm] &=\lim _{u_{1} \rightarrow 0, h_{2} \rightarrow 0, \ldots, h_{n} \rightarrow 0} \frac{1}{h_{1} h_{2} \cdots h_{n}} P\left[u_{1} \leq Y_{1} \leq u_{1}+h_{1},\right. \\[2mm] &\quad\quad\quad \left.u_{2} \leq Y_{2} \leq u_{2}+h_{2}, \ldots, u_{n} \leq Y_{n} \leq u_{n}+h_{n}\right]. \tag{10.6} \end{align} The probability on the right-hand side of (10.6) is equal to
\begin{align} P[u_{1} \leq g_{1}\left(X_{1}, X_{2}, \ldots, X_{n}\right) & \leq u_{1}+h_{1}, \ldots, \\[2mm] & u_{n} \leq g_{n}\left(X_{1}, X_{2}, \ldots, X_{n}\right) \leq u_{n}+h_{n} ] \\ = \int \int \dots \int & f_{X_1, X_2, \dots ,X_n}(x_1, x_2, \dots, x_n) dx_1 dx_2 \cdots dx_n, \tag{10.7} \end{align}
in which \[\begin{array}{r} D_{n}=\left\{\left(x_{1}, x_{2}, \ldots, x_{n}\right): \quad u_{1} \leq g_{1}\left(x_{1}, x_{2}, \ldots, x_{n}\right) \leq u_{1}+h_{1}, \ldots,\right. \\[2mm] \left.u_{n} \leq g_{n}\left(x_{1}, x_{2}, \ldots, x_{n}\right) \leq u_{n}+h_{n}\right\}. \end{array}\]
Now, if \(\left(u_{1}, u_{2}, \ldots, u_{n}\right)\) does not belong to \(C\) , then for sufficiently small values of \(h_{1}, h_{2}, \ldots, h_{n}\) there are no points \(\left(x_{1}, x_{2}, \ldots, x_{n}\right)\) in \(D_{n}\) and the probability in (10.7) is 0. From the fact that the quantities in (10.6), whose limit is being taken, are 0 for sufficiently small values of \(h_{1}, h_{2}, \ldots, h_{n}\) , it follows that \(f_{Y_{1}, Y_{2}, \ldots, Y_{n}}\left(u_{1}, u_{2}, \ldots, u_{n}\right)=0\) for \(\left(u_{1}, u_{2}, \ldots, u_{n}\right)\) not in \(C\) . Thus (10.5 \(^\prime\) ) is proved. To prove (10.5), we use the celebrated formula for change of variables in multiple integrals (see R. Courant, Differential and Integral Calculus , Interscience, New York, 1937, Vol II, p. 253, or T. Apostol, Mathematical Analysis , Addison-Wesley, Reading, Massachusetts, 1957, p. 271) to transform the integral on the right-hand side of (10.7) to the integral
\begin{align} & \int_{u_{1}}^{u_{1}+h_{1}} d y_{1} \int_{u_{2}}^{u_{2}+h_{2}} d y_{2} \cdots \int_{u_{n}}^{u_{n}+h_{n}} d y_{n} \tag{10.8}\\ & \quad\quad\quad\quad f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right)\left|J\left(x_{1}, x_{2}, \ldots, x_{n}\right)\right|^{-1}. \end{align}
Replacing the probability on the right-hand side of (10.6) by the integral in (10.8) and then taking the limits indicated in (10.6), we finally obtain (10.5).
Example 10A . Let \(X_{1}\) and \(X_{2}\) be jointly continuous random variables. Let \(U_{1}=X_{1}+X_{2}, U_{2}=X_{1}-X_{2}\) . For any real numbers \(u_{1}\) and \(u_{2}\) show that
\[f_{U_{1}, U_{2}}\left(u_{1}, u_{2}\right)=\frac{1}{2} f_{X_{1}, X_{2}}\left(\frac{u_{1}+u_{2}}{2}, \frac{u_{1}-u_{2}}{2}\right). \tag{10.9}\]
Solution
Let \(g_{1}\left(x_{1}, x_{2}\right)=x_{1}+x_{2}\) and \(g_{2}\left(x_{1}, x_{2}\right)=x_{1}-x_{2}\) . The equations \(u_{1}=x_{1}+x_{2}\) and \(u_{2}=x_{1}-x_{2}\) clearly have as their solution \(x_{1}=\left(u_{1}+u_{2}\right) / 2\) and \(x_{2}=\left(u_{1}-u_{2}\right) / 2\) . The Jacobian \(J\) is given by
\[J=\begin{vmatrix} \dfrac{\partial g_{1}}{\partial x_{1}} & \dfrac{\partial g_{1}}{\partial x_{2}} \\ \dfrac{\partial g_{2}}{\partial x_{1}} & \dfrac{\partial g_{2}}{\partial x_{2}} \end{vmatrix}=\begin{vmatrix} 1 & 1 \\ 1 & -1 \end{vmatrix}=-2.\]
In view of these facts, (10.9) is an immediate consequence of (10.5).
In exactly the same way one may establish the following result:
Example 10B . Let \(X_{1}\) and \(X_{2}\) be jointly continuous random variables. Let \[R=\sqrt{X_{1}^{2}+X_{2}^{2}}, \quad \theta=\tan ^{-1}\left(X_{2} / X_{1}\right). \tag{10.10}\] Then for any real numbers \(r\) and \(\alpha\) , such that \(r \geq 0\) and \(0 \leq \alpha \leq 2 \pi\) , \[f_{R, \theta}(r, \alpha)=r f_{X_{1}, X_{2}}(r \cos \alpha, r \sin \alpha). \tag{10.11}\] It should be noted that we immediately obtain from (10.11) the formula for \(f_{R}(r)\) given by ( 9.13 \()\) , since
\[f_{R}(r)=\int_{0}^{2 \pi} d \alpha\, f_{R, \theta}(r, \alpha). \tag{10.12}\]
Example 10C . Rotation of axes . Let \(X_{1}\) and \(X_{2}\) be jointly distributed random variables. Let \[Y_{1}=X_{1} \cos \alpha+X_{2} \sin \alpha, \quad Y_{2}=-X_{1} \sin \alpha+X_{2} \cos \alpha \tag{10.13}\] for some angle \(\alpha\) in the interval \(0 \leq \alpha \leq 2 \pi\) . Then \[f_{Y_{1}, Y_{2}}\left(y_{1}, y_{2}\right)=f_{X_{1}, X_{2}}\left(y_{1} \cos \alpha-y_{2} \sin \alpha, y_{1} \sin \alpha+y_{2} \cos \alpha\right). \tag{10.14}\] To illustrate the use of (10.14), consider two jointly normally distributed random variables with a joint probability density function given by (9.31), with \(m_{1}=m n_{2}=0\) . Then \begin{align} f_{Y_{1}, Y_{2}}\left(y_{1}, y_{2}\right)= & \frac{1}{2 \pi \sigma_{1} \sigma_{2} \sqrt{1-\rho^{2}}} \tag{10.15}\\[2mm] & \times \exp \left[\frac{-1}{2\left(1-\rho^{2}\right)}\left(A y_{1}^{2}-2 B y_{1} y_{2}+C y_{2}^{2}\right)\right], \end{align} where \begin{align} & A=\frac{\cos ^{2} \alpha}{\sigma_{1}^{2}}-2 \rho \frac{\cos \alpha \sin \alpha}{\sigma_{1} \sigma_{2}}+\frac{\sin ^{2} \alpha}{\sigma_{2}^{2}} \\ & B=\frac{\cos \alpha \sin \alpha}{\sigma_{1}^{2}}-\rho \frac{\sin ^{2} \alpha-\cos ^{2} \alpha}{\sigma_{1} \sigma_{2}}-\frac{\cos \alpha \sin \alpha}{\sigma_{2}^{2}} \tag{10.16}\\ & C=\frac{\sin ^{2} \alpha}{\sigma_{1}^{2}}+2 \rho \frac{\cos \alpha \sin \alpha}{\sigma_{1} \sigma_{2}}+\frac{\cos ^{2} \alpha}{\sigma_{2}^{2}}. \end{align}
From (10.15) one sees that two random variables \(Y_{1}\) and \(Y_{2}\) , obtained by a rotation of axes from jointly normally distributed random variables, \(X_{1}\) and \(X_{2}\) , are jointly normally distributed. Further, if the angle of rotation \(\alpha\) is chosen so that
\[\tan 2 \alpha=\frac{2 \rho \sigma_{1} \sigma_{2}}{\sigma_{1}^{2}-\sigma_{2}^{2}}, \tag{10.17}\]
then \(B=0\) , and \(Y_{1}\) and \(Y_{2}\) are independent normally distributed. Thus by a suitable rotation of axes, two jointly normally distributed random variables may be transformed into two independent normally distributed random variables .
Theoretical Exercises
10.1 . Let \(X_{1}\) and \(X_{2}\) be independent random variables, each exponentially distributed with parameter \(\lambda\) . Show that the random variables \(X_{1}+X_{2}\) and \(X_{1} / X_{2}\) are independent.
10.2 . Let \(X_{1}\) and \(X_{2}\) be independent random variables, each normally distributed with parameters \(m=0\) and \(\sigma>0\) . Show that \(X_{1}^{2}+X_{2}^{2}\) and \(X_{1} / X_{2}\) are independent.
10.3 . Let \(X_{1}\) and \(X_{2}\) be independent random variables, \(\chi^{2}\) distributed with \(n_{1}\) and \(n_{2}\) degrees of freedom, respectively. Show that \(X_{1}+X_{2}\) and \(X_{1} / X_{2}\) are independent.
10.4 . Let \(X_{1}, X_{2}\) , and \(X_{3}\) be independent identically normally distributed random variables. Let \(\bar{X}=\left(X_{1}+X_{2}+X_{3}\right) / 3\) and \(S=\left(X_{1}-\bar{X}\right)^{2}+\left(X_{2}-\bar{X}\right)^{2}+\) \(\left(X_{3}-\bar{X}\right)^{2}\) . Show that \(\bar{X}\) and \(S\) are independent.
10.5 . Generation of a random sample of a normally distributed random variable . Let \(U_{1}, U_{2}\) be independent random variables, each uniformly distributed on the interval 0 to 1. Show that the random variables \begin{align} & X_{1}=\left(-2 \log _{e} U_{1}\right)^{1 / 2} \cos 2 \pi U_{2} \\ & X_{2}=\left(-2 \log _{e} U_{1}\right)^{1 / 2} \sin 2 \pi U_{2} \end{align} are independent random variables, each normally distributed with mean 0 and variance 1. (For a discussion of this result, see G. E. P. Box and Mervin E. Muller, “A note on the generation of random normal deviates,” Annals of Mathematics Statistics , Vol. 29 (1958), pp. 610-611.)
Exercises
10.1 . Let \(X_{1}\) and \(X_{2}\) be independent random variables, each exponentially distributed with parameter \(\lambda=\frac{1}{2}\) . Find the joint probability density function of \(Y_{1}\) and \(Y_{2}\) , in which (i) \(Y_{1}=X_{1}+X_{2}, Y_{2}=X_{1}-X_{2}\) , (ii) \(Y_{1}=\) maximum \(\left(X_{1}, X_{2}\right), Y_{2}=\operatorname{minimum}\left(X_{1}, X_{2}\right)\) .
Answer
(i) \(\frac{1}{8} e^{-1 / 2 y_{1}}\) if \(y_{1}>0,\left|y_{2}\right| \leq y_{1} ; 0\) otherwise; (ii) \(\frac{1}{2} e^{-1 / 2\left(y_{1}+y_{2}\right)}\) if \(0 \leq y_{2}
10.2 . Let \(X_{1}\) and \(X_{2}\) have joint probability density function given by \begin{align} f_{X_{1}, X_{2}}\left(x_{1}, x_{2}\right) & = \begin{cases} \dfrac{1}{\pi}, & \text{if } x_{1}^{2}+x_{2}^{2} \leq 1, \\ 0, & \text{otherwise.} \end{cases} \end{align} Find the joint probability density function of \((R, \theta)\) , in which \(R=\) \(\sqrt{X_{1}^{2}+X_{2}^{2}}\) and \(\theta=\tan ^{-1} X_{2} / X_{1}\) . Show that, and explain why, \(R^{2}\) is uniformly distributed but \(R\) is not.
10.3 . Let \(X\) and \(Y\) be independent random variables, each uniformly distributed over the interval 0 to 1. Find the individual and joint probability density functions of the random variables \(R\) and \(\theta\) , in which \(R=\sqrt{X^{2}+Y^{2}}\) and \(\theta=\tan ^{-1} Y \mid X\) .
Answer
\(f_{R, 0}(r, \alpha)=r\) if \(0
10.4 . Two voltages \(X(t)\) and \(Y(t)\) are independently and normally distributed with parameters \(m=0\) and \(\sigma=1\) . These are combined to give two new voltages, \(U(t)=X(t)+Y(t)\) and \(V(t)=X(t)-Y(t)\) . Find the joint probability density function of \(U(t)\) and \(V(t)\) . Are \(U(t)\) and \(V(t)\) independent? Find \(P[U(t)>0, V(t)<0]\) .