In section 3 we have been concerned with problems of the following type. Suppose one has a box containing 100 light bulbs, of which five are defective. What is the probability that a bulb selected from the box will be defective? A natural extension of this problem is the following. Suppose a light bulb (chosen from a box containing 100 light bulbs, of which five are defective) is found to be defective; what is the probability that a second light bulb drawn from the box (now containing 99 bulbs, of which four are defective) will be defective? A mathematical model for the statement and solution of this problem is provided by the notion of conditional probability.
Given two events, \(A\) and \(B\) , by the conditional probability of the event \(B\) , given the event \(A\) , denoted by \(P[B \mid A]\) , we mean intuitively the probability that \(B\) will occur, under the assumption that \(A\) has occurred. In other words, \(P[B \mid A]\) represents our re-evalaution of the probability of \(B\) in the light of the information that \(A\) has occurred .
To motivate the formal definition of \(P[B \mid A]\) , which we shall give, let us consider the meaning of \(P[B \mid A]\) from the point of view of the frequency interpretation of probability (since it is our desire to give to \(P[B \mid A]\) a mathematical meaning that corresponds to its meaning as a relative frequency). Suppose one observes a large number \(N\) of occurrences of a random phenomenon in which the events \(A\) and \(B\) are defined. Let \(N_{A}\) denote the number of occurrences of the event \(A\) in the \(N\) occurrences of the random phenomenon. Similarly, let \(N_{B}\) denote the number of occurrences of \(B\) . Next, let \(N_{A B}\) denote the number of occurrences of the random phenomenon in which both the events \(A\) and \(B\) occur.
Example 4A . Thirty observed samples of size 2 . Consider the following results of thirty repetitions of the experiment of drawing, without replacement, a sample of size 2 from an urn containing six balls, numbered 1 to 6:
| \((1,6)\) , | \((4,5)\) , | \((1,4)\) , | \((5,3)\) , | \((3,2)\) , | \((4,3)\) |
| \((3,1)\) , | \((5,1)\) , | \((2,1)\) , | \((2,3)\) , | \((4,5)\) , | \((5,6)\) |
| \((5,4)\) , | \((3,1)\) , | \((6,3)\) , | \((5,6)\) , | \((2,5)\) , | \((6,4)\) |
| \((1,3)\) , | \((6,2)\) , | \((4,1)\) , | \((1,5)\) , | \((4,6)\) , | \((6,3)\) |
| \((2,3)\) , | \((5,2)\) , | \((3,6)\) , | \((6,4)\) , | \((6,4)\) , | \((1,2)\) |
If the balls numbered 1 to 4 are colored white, and the balls numbered 5 and 6 are colored red, then the outcome of the thirty trials can be recorded as follows:
| \((W, R)\) , | \((W, R)\) , | \((W, W)\) , | \((R, W)\) , | \((W, W)\) , | \((W, W)\) |
| \((W, W)\) , | \((R, W)\) , | \((W, W)\) , | \((W, W)\) , | \((W, R)\) , | \((R, R)\) |
| \((R, W)\) , | \((W, W)\) , | \((R, W)\) , | \((R, R)\) , | \((W, R)\) , | \((R, W)\) |
| \((W, W)\) , | \((R, W)\) , | \((W, W)\) , | \((W, R)\) , | \((W, R)\) , | \((R, W)\) |
| \((W, W)\) , | \((R, W)\) , | \((W, R)\) , | \((R, W)\) , | \((R, W)\) , | \((W, W)\) |
Let \(N_{A}\) denote the number of experiments in which a white ball appeared on the first trial. Let \(N_{B}\) denote the number of experiments in which a white ball appeared on the second trial, and let \(N_{A B}\) denote the number of experiments in which white balls appeared at both trials. By direct enumeration, one obtains that \(N_{A}=18, N_{B}=21\) , and \(N_{A B}=11\) .
In terms of the frequency definition, the unconditional probabilities of the events \(A, B\) , and \(A B\) are given by \[P[A]=\frac{N_{A}}{N}, \quad P[B]=\frac{N_{B}}{N}, \quad P[A B]=\frac{N_{A B}}{N}. \tag{4.1}\]
On the other hand, the conditional probability \(P[B \mid A]\) of the event \(B\) , given the event \(A\) , represents the fraction of experiments in which \(A\) occurred that \(B\) also occurred; in symbols, \[P[B \mid A]=\frac{N_{A B}}{N_{A}}. \tag{4.2}\] It should be noted that (4.2) makes sense only if \(N_{A}\) is not zero. If \(N_{A}\) is zero, we must regard \(P[B \mid A]\) as being undefined.
Equation (4.2) represents the meaning of the notion of conditional probability from the frequency point of view. Now, (4.2) may be written in a manner that will indicate a formal definition of \(P[B \mid A]\) , which will embody the properties of conditional probability as it is intuitively conceived. We rewrite (4.2) (in the case that \(N_{A}\) is not zero): \[P[B \mid A]=\frac{\left(N_{A B} / N\right)}{\left(N_{A} / N\right)}=\frac{P[A B]}{P[A]}. \tag{4.3}\]
In analogy with (4.3) we now give the following formal definition of \(P[B \mid A]\) :
Formal Definition of Conditional Probability . Let \(A\) and \(B\) be two events on a sample description space \(S\) , on the subsets of which is defined a probability function \(P[\cdot]\) . The conditional probability of the event \(B\) , given the event \(A\) , denoted by \(P[B \mid A]\) , is defined by \[P[B \mid A]=\frac{P[A B]}{P[A]} \quad \text { if } P[A]>0, \tag{4.4}\] and if \(P[A]=0\) , then \(P[B \mid A]\) is undefined.
Example 4B . Computing a conditional probability . Consider the problem of drawing, without replacement, a sample of size 2 from an urn containing four white and two red balls. Let \(A\) denote the event that the first ball drawn is white, and \(B\) , the event that the second ball drawn is white. Let us compute \(P[B \mid A]\) . By (2.6) , it follows that \(P[A B]=\) \((4 \cdot 3) /(6 \cdot 5)=\frac{12}{30}\) , whereas \(P[A]=\frac{4}{6}=\frac{20}{30}\) . Therefore, \(P[B \mid A]=\frac{12}{2} \frac{2}{0}=\) 0.6, which accords with our intuitive ideas, since the second ball is drawn from an urn containing five balls, of which three are white. Compare these theoretically computed probabilities with the observed relative frequencies in example 4A. We have \(N_{A B} / N=\frac{11}{30}, N_{A} / N=\frac{18}{30}, N_{A B} / N_{A}=\) \(\frac{11}{18}=0.611\) .
We next give a formula that may help to clarify the difference between the unconditional and the conditional probability of an event \(B\) . We have, for any events \(B\) and \(A\) such that \(0
Equation (4.5) is proved as follows. From the definition of conditional probability given by (4.4) one has the basic formula (4.6) \[P[A B]=P[A] P[B \mid A]. \tag{4.6}\]
Similarly, one has \(P\left[A^{c} B\right]=P\left[A^{c}\right] P\left[B \mid A^{c}\right]\) . Now, the events \(A B\) and \(A^{c} B\) are mutually exclusive, and their union is \(B\) . Consequently, \(P[B]=\) \(P[A B]+P\left[A^{c} B\right]\) . The desired conclusion may now be inferred.
Example 4C . A numerical verification of (4.5) . Consider again the problem in example 4B. One has \(P[A]=\frac{2}{3}\) . Therefore, \(P\left[A^{c}\right]=\frac{1}{3}\) . Next, one has \(P[B \mid A]=\frac{3}{5}\) . However, from this it does not follow that \(P\left[B \mid A^{c}\right]=\) \(\frac{2}{5}\) . Rather, by use of definition (4.4) , \(P\left[B \mid A^{c}\right]=\frac{4}{5}\) ; one may also obtain this result by intuitive reasoning (which is made rigorous in section 4 of Chapter 3), for if a white ball were not drawn on the first draw, there would be four white balls among the five balls in the urn from which the second draw would be made. Then, by (4.5) , \(P[B]=\left(\frac{3}{5}\right)\left(\frac{2}{3}\right)+\left(\frac{4}{5}\right)\left(\frac{1}{3}\right)=\) \(\frac{10}{15}=\frac{2}{3}\) .
Example 4D yields conclusions which students, on first acquaintance, often think startling and contrary to intuition.
Example 4D . Consider a family with two children. Assume that each child is as likely to be a boy as it is to be a girl. What is the conditional probability that both children are boys, given that (i) the older child is a boy, (ii) at least one of the children is a boy?
Solution
Let \(A\) be the event that the older child is a boy, and let \(B\) be the event that the younger child is a boy. Then \(A \cup B\) is the event that at least one of the children is a boy, and \(A B\) is the event that both children are boys. The probability that both children are boys, given that the older is a boy, is equal to \[P[A B \mid A]=\frac{P[A B]}{P[A]}=\frac{1 / 4}{1 / 2}=\frac{1}{2} . \tag{4.7}\]
The probability that both children are boys, given that at least one of them is a boy, is equal to since \((A B)(A \cup B)=A B\) \[P[A B \mid A \cup B]=\frac{P[A B]}{P[A \cup B]}=\frac{1 / 4}{3 / 4}=\frac{1}{3}. \tag{4.8}\]
Example 4E . The outcome of a draw, given the outcome of a sample . Let a sample of size 4 be drawn with replacement (without replacement), from an urn containing twelve balls, of which eight are white. Find the conditional probability that the ball drawn on the third draw was white, given that the sample contains three white balls.
Solution
Let \(A\) be the event that the sample contains exactly three white balls, and let \(B\) be the event that the ball drawn on the third draw was white. The problem at hand is to find \(P[B \mid A]\) . In the case of sampling with replacement \[P[A]=\frac{\left(\begin{array}{l} 4 \tag{4.9} \\ 3 \end{array}\right) 8^{3} 4}{(12)^{4}}, \quad P[A B]=\frac{\left(\begin{array}{l} 3 \\ 2 \end{array}\right) 8^{3} 4}{(12)^{4}}, \quad P[B \mid A]=\frac{\left(\begin{array}{l} 3 \\ 2 \end{array}\right)}{\left(\begin{array}{l} 4 \\ 3 \end{array}\right)}=\frac{3}{4}.\]
In the case of sampling without replacement \[P[A]=\frac{\left(\begin{array}{l} 4 \tag{4.10} \\ 3 \end{array}\right)(8)_{3} 4}{(12)_{4}}, \quad P[A B]=\frac{\left(\begin{array}{l} 3 \\ 2 \end{array}\right)(8)_{3} 4}{(12)_{4}}, \quad P[B \mid A]=\frac{\left(\begin{array}{l} 3 \\ 2 \end{array}\right)}{\left(\begin{array}{l} 4 \\ 3 \end{array}\right)}=\frac{3}{4}.\] More generally, it may be proved (see theoretical exercise 4.4) that if a sample of size \(n\) contains \(k\) white balls then the probability is \(k / n\) that on any specified draw a white ball was drawn. Note that this result is the same, no matter what the composition of the urn and irrespective of whether the sample was drawn with or without replacement. In a sense, one may express the results just stated by the statement that on any given draw all balls in the sample are equally likely to occur. Many students attempt to solve the problem given here by reasoning that on the third draw any one of the four balls in the sample could have occurred and of these three are white, so that the (conditional) probability of a white ball on the third draw is \(\frac{3}{4}\) , in agreement with the foregoing equations. However, this line of reasoning consists in making assumptions in addition to those made in our derivation of these equations. It is desirable to prove that these new assumptions are a consequence of the model postulated in deriving (4.9) and (4.10) .
Theoretical Exercises
4.1 . Prove the following statements, for any events \(A, B\) , and \(C\) , such that \(P[C]>0\) . These relations illustrate the fact that all general theorems on probabilities are also valid for conditional probabilities with respect to any particular event \(C\) . \begin{align} &\text{(i)} \quad\quad\quad\quad P[S \mid C]=1 \quad \text{where}\;S\;\text{is the certain event.} \\ &\text{(ii)} \quad\quad\quad\quad P[A \mid C]=1 \quad \text{if}\; C \; \text{is a subevent of} A. \\ &\text{(iii)} \quad\quad\quad\quad P[A \mid C]=0 \quad \text{if}\; P[A]=0. \\ &\text{(iv)} \quad\quad\quad\quad P[A \cup B \mid C]=P[A \mid C]+P[B \mid C]-P[A B \mid C]. \\ &\text{(v)} \quad\quad\quad\quad P\left[A^{c} \mid C\right]=1-P[A \mid C]. \end{align}
4.2 . Let \(B\) be an event of positive probability. Show that for any event \(A\) ,
\begin{align} & \text{(i)} \quad\quad\quad\quad A \subset B \text { implies } \quad\quad P[A \mid B]=P[A] / P[B]. \\ & \text{(ii)} \quad\quad\quad\quad B \subset A \text { implies } \quad\quad P[A \mid B]=1. \end{align}
4.3 . Let \(A\) and \(B\) be two events, each with positive probability. Show that statement (i) is true, whereas statements (ii) and (iii) are, in general, false: \begin{align} &\text{(i)} \quad P[A \mid B]+P\left[A^{c} \mid B\right] =1 \\ &\text{(ii)} \quad P[A \mid B]+P\left[A \mid B^{c}\right] =1 \\ &\text{(iii)} \quad P[A \mid B]+P\left[A^{c} \mid B^{c}\right] =1. \end{align}
4.4 . An urn contains \(M\) balls, of which \(M_{W}\) are white (where \(M_{W} \leq M\) ). Let a sample of size \(n\) be drawn from the urn either with replacement or without replacement. For \(j=1,2, \ldots, n\) let \(B_{j}\) be the event that the ball drawn on the \(j\) th draw is white. For \(k=1,2, \ldots, n\) let \(A_{k}\) be the event that the sample (of size \(n\) ) contains exactly \(k\) white balls. Show that \(P\left[B_{j} \mid A_{k}\right]=k / n\) . Express this fact in words.
4.5 . An urn contains \(M\) balls, of which \(M_{W}\) are white. \(n\) balls are drawn and laid aside (not replaced in the urn), their color unnoted. Another ball is drawn (it is assumed that \(n\) is less than \(M\) ). What is the probability that it will be white? Hint: Compare example 2B .
Exercises
4.1 . A man tosses 2 fair coins. What is the conditional probability that he has tossed 2 heads, given that he has tossed at least 1 head?
Answer
\(\frac{1}{3}\) .
4.2 . An urn contains 12 balls, of which 4 are white. Five balls are drawn and laid aside (not replaced in the urn), their color unnoted.
(i) Another ball is drawn. What is the probability that it will be white?
(ii) A sample of size 2 is drawn. What is the probability that it will contain exactly one white ball?
(iii) What is the conditional probability that it will contain exactly 2 white balls, given that it contains at least 1 white ball.
4.3 . In the milk section of a self-service market there are 150 quarts, 100 of which are fresh, and 50 of which are a day old.
(i) If 2 quarts are selected, what is the probability that both will be fresh?
(ii) Suppose that the 2 quarts are selected after 50 quarts have been removed from the section. What is the probability that both will be fresh?
(iii) What is the conditional probability that both will be fresh, given that at least 1 of them is fresh?
Answer
(i), (ii) \(\frac{66}{149}\) ; (iii) \(\frac{99}{149}\)
4.4 . The student body of a certain college is composed of \(60 \%\) men and \(40 \%\) women. The following proportions of the students smoke cigarettes: \(40 \%\) of the men and \(60 \%\) of the women. What is the probability that a student who is a cigarette smoker is a man? A woman?
4.5 . Consider two events \(A\) and \(B\) such that \(P[A]=\frac{1}{4}, P[B \mid A]=\frac{1}{2}, P[A \mid B]=\) \(\frac{1}{4}\) . For each of the following 4 statements, state whether it is true or false: (i) The events \(A\) and \(B\) are mutually exclusive, (ii) \(A\) is a subevent of \(B\) , (iii) \(P\left[A^{c} \mid B^{c}\right]=\frac{3}{4}\) ; (iv) \(P[A \mid B]+P\left[A \mid B^{c}\right]=1\) .
Answer
(i) False, since \(P[A B]=\frac{1}{8}\) ; (ii) false; (iii) true; (iv) false.
4.6 . Consider an urn containing 12 balls, of which 8 are white. Let a sample of size 4 be drawn with replacement (without replacement). What is the conditional probability that the first ball drawn will be white, given that the sample contained exactly (i) 2 white balls, (ii) 3 white balls?
4.7 . Consider an urn containing 6 balls, of which 4 are white. Let a sample of size 3 be drawn with replacement (without replacement). Let \(A\) denote the event that the sample contains exactly 2 white balls, and let \(B\) denote the event that the ball drawn on the third draw is white. Verify numerically that (4.5) holds in this case.
4.8 . Consider an urn containing 12 balls, of which 8 are white. Let a sample of size 4 be drawn with replacement (without replacement). What is the conditional probability that the second and third balls drawn will be white, given that the sample contains exactly three white balls?
4.9 . Consider 3 urns; urn I contains 2 white and 4 red balls, urn II contains 8 white and 4 red balls, urn III contains 1 white and 3 red balls. One ball is selected from each urn. What is the probability that the ball selected from urn II will be white, given that the sample drawn contains exactly 2 white balls?
Answer
\(\frac{10}{11}\) .
4.10 . Consider an urn in which 4 balls have been placed by the following scheme. A fair coin is tossed; if the coin falls heads, a white ball is placed in the urn, and if the coin falls tails, a red ball is placed in the urn.
(i) What is the probability that the urn will contain exactly 3 white balls?
(ii) What is the probability that the urn will contain exactly 3 white balls, given that the first ball placed in the urn was white?
4.11 . A man tosses 2 fair dice. What is the (conditional) probability that the sum of the 2 dice will be 7, given that (i) the sum is odd, (ii) the sum is greater than 6, (iii) the outcome of the first die was odd, (iv) the outcome of the second die was even, \((v)\) the outcome of at least 1 of the dice was odd, (vi) the 2 dice had the same outcomes, (vii) the 2 dice had different outcomes, (viii) the sum of the 2 dice was 13?
Answer
(i) \(\frac{1}{3}\) ; (ii) \(\frac{2}{7}\) ; (iii), (iv) \(\frac{1}{8}\) ; (v) \(\frac{2}{9}\) ; (vi) 0; (vii) \(\frac{1}{5}\) ; (viii) undefined.
4.12 . A man draws a sample of 3 cards one at a time (without replacement) from a pile of 8 cards, consisting of the 4 aces and the 4 kings in a bridge deck. What is the (conditional) probability that the sample will contain at least 2 aces, given that it contains (i) the ace of spades, (ii) at least one ace? Explain why the answers to (i) and (ii) need not be equal.
4.13 . Consider 4 cards, on each of which is marked off a side 1 and side 2. On card 1, both side 1 and side 2 are colored red. On card 2, both side 1 and side 2 are colored black. On card 3, side 1 is colored red and side 2 is colored black. On card 4, side 1 is colored black and side 2 is colored red. A card is chosen at random. What is the (conditional) probability that if one side of the card selected is red the other side of the card will be black? What is the (conditional) probability that if side 1 of the card selected is examined and found to be red side 2 of the card will be black? Hint : Compare example 4D .
Answer
\(\frac{2}{3}, \frac{1}{2}\) .
4.14 . A die is loaded in such a way that the probability of a given number turning up is proportional to that number (for instance, a 4 is twice as probable as a 2).
(i) What is the probability of rolling a 5, given that an odd number turns up.
(ii) What is the probability of rolling an even number, given that a number less than 5 turns up.