It has already been seen that the geometric and negative binomial probability laws arise in response to the following question: through how many trials need one wait in order to achieve the \(r\) th success in a sequence of independent repeated Bernoulli trials in which the probability of success at each trial is \(p\) ? In the same way, exponential and gamma probability laws arise in response to the question: how long a time need one wait if one is observing a sequence of events occurring in time in accordance with a Poisson probability law at the rate of \(\mu\) events per unit time in order to observe the \(r\) th occurrence of the event?
Example 4A. How long will a toll collector at a toll station at which automobiles arrive at the mean rate \(\mu=1.5\) automobiles per minute have to wait before he collects the \(r\) th toll for any integer \(r=1,2, \ldots\) ?
We now show that the waiting time to the \(r\) th event in a series of events happening in accordance with a Poisson probability law at the rate of \(\mu\) events per unit of time (or space) obeys a gamma probability law with parameter \(r\) and \(\mu\) ; consequently, it has probability density function
\begin{align} f(t) & = \begin{cases} \dfrac{\mu}{(r-1)!} (\mu t)^{r-1} e^{-\mu t}, & \text{if } t \geq 0 \tag{4.1} \\ 0, & \text{if } t < 0. \end{cases} \end{align}
In particular, the waiting time to the first event obeys the exponential probability law with parameter \(\mu\) (or equivalently, the gamma probability law with parameters \(r=1\) and \(\mu\) ) with probability density function
\begin{align} f(t) & = \begin{cases} \mu e^{-\mu t}, & \text{if } t \geq 0 \tag{4.2} \\ 0, & \text{if } t < 0. \end{cases} \end{align}
To prove (4.1), first find the distribution function of the time of occurrence of the \(r\) th event. For \(t \geq 0\) , let \(F_{r}(t)\) denote the probability that the time of occurrence of the \(r\) th event will be less than or equal to \(t\) . Then \(1-F_{r}(t)\) represents the probability that the time of occurrence of the \(r\) th event will be greater than \(t\) . Equivalently, \(1-F_{r}(t)\) is the probability that the number of events occurring in the time from 0 to \(t\) is less than \(r\) ; consequently,
\[1-F_{r}(t)=\sum_{k=0}^{r-1} \frac{1}{k !}(\mu t)^{k} e^{-\mu t} \tag{4.3}\]
By differentiating (4.3) with respect to \(t\) , one obtains (4.1).
Example 4B. Consider a baby who cries at random times at a mean rate of six distinct times per hour. If his parents respond only to every second time, what is the probability that ten or more minutes will elapse between two responses of the parents to the baby?
Solution
From the assumptions given (which may not be entirely realistic) the length \(T\) in hours of the time interval between two responses obeys a gamma probability law with parameters \(r=2\) and \(\mu=6\) , Consequently,
\[P\left[T \geq \frac{1}{6}\right]=\int_{1 / 6}^{\infty} 6(6 t) e^{-6 t} d t=2 e^{-1}, \tag{4.4}\]
in which the integral has been evaluated by using (4.3). If the parents responded only to every third cry of the baby, then
\[P\left[T \geq \frac{1}{6}\right]=\int_{1 / 6}^{\infty} \frac{6}{2 !}(6 t)^{2} e^{-6 t} d t=\frac{5}{2} e^{-1}.\]
More generally, if the parents responded only to every \(r\) th cry of the baby, then
\begin{align} P\left[T \geq \frac{1}{6}\right] & =\int_{1 / 6}^{\infty} \frac{6}{(r-1) !}(6 t)^{r-1} e^{-6 t} d t \tag{4.5} \\ & =e^{-1}\left\{1+\frac{1}{1 !}+\frac{1}{2 !}+\cdots+\frac{1}{(r-1) !}\right\}. \end{align}
The exponential and gamma probability laws are of great importance in applied probability theory, since recent studies have indicated that in addition to describing the lengths of waiting times they also describe such numerical valued random phenomena as the life of an electron tube, the time intervals between successive breakdowns of an electronic system, the time intervals between accidents, such as explosions in mines, and so on.
The exponential probability law may be characterized in a manner that illuminates its applicability as a law of waiting times or as a law of time to failure. Let \(T\) be the observed waiting time (or time to failure). By definition, \(T\) obeys an exponential probability law with parameter \(\lambda\) if and only if for every \(a>0\)
\[P[T \geq a]=1-F(a)=\int_{a}^{\infty} \lambda e^{-\lambda t} d t=e^{-\lambda a}. \tag{4.6}\]
It then follows that for any positive numbers \(a\) and \(b\)
\[P[T>a+b \mid T>b]=e^{-\lambda a}=P[T>a]. \tag{4.7}\]
In words, (4.7) says that, given an item of equipment that has served \(b\) or more time units, its conditional probability of serving \(a+b\) or more time units is the same as its original probability, when first put into service of serving \(a\) or more time units. Another way of expressing (4.7) is to say that if the time to failure of a piece of equipment obeys an exponential probability law then the equipment is not subject to wear or to fatigue.
The converse is also true, as we now show. If the time to failure of an item of equipment obeys (4.7), then it obeys an exponential probability law. More precisely, let \(F(x)\) be the distribution function of the time to failure and assume that \(F(x)=0\) for \(x<0, F(x)<1\) for \(x>0\) , and
\[\frac{1-F(x+y)}{1-F(y)}=1-F(x) \quad \text { for } x, y>0. \tag{4.8}\]
Then necessarily, for some constant \(\lambda>0\) ,
\[1-F(x)=e^{-\lambda\;x} \quad \text { for } x>0. \tag{4.9}\]
If we define \(g(x)=\log _{e}[1-F(x)]\) , then the foregoing assertion follows from a more general theorem.
Theorem. If a function \(g(x)\) satisfies the functional equation
\[g(x+y)=g(x)+g(y), \quad x, y>0 \tag{4.10}\]
and is bounded in the interval 0 to 1,
\[|g(x)| \leq M, \quad 0
for some constant \(M\) , then the function \(g(x)\) is given by
\[g(x)=g(1) x, \quad x>0. \tag{4.12}\]
Proof
Suppose that (4.12) were not true. Then the function \(G(x)=\) \(g(x)-g(1) x\) would not vanish identically in \(x\) . Let \(x_{0}>0\) be a point such that \(G\left(x_{0}\right) \neq 0\) . Now it is clear that \(G(x)\) satisfies the functional equation in (4.10). Therefore, \(G\left(2 x_{0}\right)=G\left(x_{0}\right)+G\left(x_{0}\right)\) , and, for any integer \(n, G\left(n x_{0}\right)=n G\left(x_{0}\right)\) . Consequently, \(\lim _{n \rightarrow \infty}\left|G\left(n x_{0}\right)\right|=\infty\) . We now show that this cannot be true, since the function \(G(x)\) satisfies the inequality \(|G(x)| \leq 2 M\) for all \(x\) , in which \(M\) is the constant given in (4.11). To prove this, note that \(G(1)=0\) . Since \(G(x)\) satisfies the functional equation in (4.10) it follows that, for any integer \(n, G(n)=0\) and \(G(n+x)=G(x)\) for \(0
For references to the history of the foregoing theorem, and a generalization, the reader may consult G. S. Young, “The Linear Functional Equation”, American Mathematical Monthly , Vol. 65 (1958), pp. 37–38.
Exercises
4.1. Consider a radar set of a type whose failure law is exponential. If radar sets of this type have a failure rate \(\lambda=1 \mathrm{set} / 1000\) hours, find a length \(T\) of time such that the probability is 0.99 that a set will operate satisfactorily for a time greater than \(T\) .
Answer
\(T=10\) hours.
4.2. The lifetime in hours of a radio tube of a certain type obeys an exponential law with parameter (i) \(\lambda=1000\) , (ii) \(\lambda=1 / 1000\) . A company producing these tubes wishes to guarantee them a certain lifetime. For how many hours should the tube be guaranteed to function, to achieve a probability of 0.95 that it will function at least the number of hours guaranteed?
4.3. Describe the probability law of the following random phenomenon: the number \(N\) of times a fair die is tossed until an even number appears (i) for the first time, (ii) for the second time, (iii) for the third time.
Answer
\(\quad N-r\) obeys a negative binomial probability law with parameters \(p=\frac{1}{2}\) and (i) \(r=1\) , (ii) \(r=2\) , (iii) \(r=3\) .
4.4. A fair coin is tossed until heads appears for the first time. What is the probability that 3 tails will appear in the series of tosses?
4.5. The customers of a certain newsboy arrive in accordance with a Poisson probability law at a rate of 1 customer per minute. What is the probability that 5 or more minutes have elapsed since (i) his last customer arrived, (ii) his next to last customer arrived?
Answer
(i) 0.0067; (ii) 0.0404.
4.6. Suppose that a certain digital computer, which operates 24 hours a day, suffers breakdowns at the rate of 0.25 per hour. We observe that the computer has performed satisfactorily for 2 hours. What is the probability that the machine will not fail within the next 2 hours?
4.7. Assume that the probability of failure of a ball bearing at any revolution is constant and equal to \(p\) . What is the probability that the ball bearing will fail on or before the \(n\) th revolution? If \(p=10^{-4}\) , how many revolutions will be reached before \(10 \%\) of such ball bearings fail? More precisely, find \(K\) so that \(P[X>K] \leq 0.1\) , where \(X\) is the number of revolutions to failure.
Answer
\(1-(1-p)^{n}; n=\log (0.9) / \log (0.9999)=1054\) .
4.8. A lepidopterist wishes to estimate the frequency with which an unusual form of a certain species of butterfly occurs in a particular district. He catches individual specimens of the species until he has obtained exactly 5 butterflies of the form desired. Suppose that the total number of butterflies caught is equal to 25. Find the probability that 25 butterflies would have to be caught in order to obtain 5 of a desired form, if the relative frequency \(p\) of occurrence of butterflies of the desired form is given by (i) \(p=\frac{1}{5}\) , (ii) \(p=\frac{1}{6}\) .
4.9. Consider a shop at which customers arrive at random at a rate of 30 per hour. What fraction of the time intervals between successive arrivals will be (i) longer than 2 minutes, (ii) shorter than 4 minutes, (iii) between 1 and 3 minutes.
Answer
(i) 0.368; (ii) 0.865; (iii) 0.383.