More About Expectation

In this section we define the expectation of a function with respect to (i) a probability law specified by its distribution function, and (ii) a numerical $n$ -tuple valued random phenomenon.

Stieltjes Integral . In section 2 we defined the expectation of a continuous function $g(x)$ with respect to a probability law, which is specified by a probability mass function or by a probability density function. We now consider the case of a general probability law, which is specified by its distribution function $F(\cdot )$ .

In order to define the expectation with respect to a probability law specified by a distribution function $F(\cdot )$ , we require a generalization of the notion of integral, which goes under the name of the Stieltjes integral . Given a continuous function $g(x)$ , a distribution function $F(\cdot )$ , and a half open interval $(a,b]$ on the real line (that is, $(a,b]$ consists of all the points strictly greater than $a$ and less than or equal to $b$ ), we define the Stieltjes integral of $g(\cdot )$ , with respect to $F(\cdot )$ over $(a,b]$ , written ${\int }_{a+}^{b}g(x)dF(x)$ , as follows. We start with a partition of the interval $(a,b]$ into $n$ subintervals $(x_{i-1},x_i]$ , in which $x_0,x_1,\dots ,x_n$ are $(n+1)$ points chosen so that . We then choose a set points x_{1}^{\prime}, x_{2}^{\prime}, \ldots x_{n}^{\prime} , one in each subinterval, so that x_{i-1} for $i=1,2,\dots ,n$ , We define

\int_{a+}^{b} g(x) d F(x)=\underset{n \rightarrow \infty}{\operatorname{limit}} \sum_{i=1}^{n} g\left(x_{i}^{\prime}\right)\left[F\left(x_{i}\right)-F\left(x_{i-1}\right)\right] \tag{6.1} 

in which the limit is taken over all partitions of the interval $(a,b]$ , as the maximum length of subinterval in the partition tends to 0.

It may be shown that if $F(\cdot )$ is specified by a probability density function $f(\cdot )$ , then

\int_{a^{+}}^{b} g(x) d F(x)=\int_{a}^{b} g(x) f(x) d x, \tag{6.2} 

whereas if $F(\cdot )$ is specified by a probability mass function $p(\cdot )$ then

\int_{a+}^{b} g(x) d F(x)=\sum_{\substack{\text { over alf } x \text { such that } \\ a0}} g(x) p(x). \tag{6.3} 

The Stieltjes integral of the continuous function $g(\cdot )$ , with respect to the distribution function $F(\cdot )$ over the whole real line, is defined by

\int_{-\infty}^{\infty} g(x) d F(x)=\lim _{\substack{a \rightarrow-\infty \\ b \rightarrow \infty}} \int_{a+}^{b} g(x) dF(x). \tag{6.4} 

The discussion in section 2 in regard to the existence and finiteness of integrals over the real line applies also to Stieltjes integrals. We say that ${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(x)dF(x)$ exists if and only if ${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}|g(x)|dF(x)$ is finite. Thus only absolutely convergent Stieltjes integrals are to be invested with sense.

We now define the expectation of a continuous function $g(\cdot )$ , with respect to a probability law specified by a distribution function $F(\cdot )$ , as the Stieltjes integral of $g(\cdot )$ , with respect to $F(\cdot )$ over the infinite real line; in symbols,

E[g(x)]=\int_{-\infty}^{\infty} g(x) dF(x). \tag{6.5} 

Stieltjes integrals are only of theoretical interest. They provide a compact way of defining, and working with, the properties of expectation. In practice, one evaluates a Stieltjes integral by breaking it up into a sum of an ordinary integral and an ordinary summation by means of the following theorem: if there exists a probability density function $f(\cdot )$ , a probability mass function $p(\cdot )$ , and constants $c_1$ and $c_2$ , whose sum is 1, such that for every $x$ 

F(x)=c_{1} \int_{-\infty}^{x} f\left(x^{\prime}\right) d x^{\prime}+c_{2} \sum_{\substack{\text { over all } x^{\prime} \leq x \text { such } \\ \text { that } p\left(x^{\prime}\right)>0}} p\left(x^{\prime}\right), \tag{6.6} 

then for any continuous function $g(\cdot )$ 

\int_{-\infty}^{\infty} g(x) d F(x)=c_{1} \int_{-\infty}^{\infty} g(x) f(x) d x+c_{2} \sum_{\substack{\text { over all } x \text { such } \\ \text { that } p(x)>0}} g(x) p(x). \tag{6.7} 

In giving the proofs of various propositions about probability laws we most often confine ourselves to the case in which the probability law is specified by a probability density function, for here we may employ only ordinary integrals. However, the properties of Stieltjes integrals are very much the same as those of ordinary Riemann integrals; consequently, the proofs we give are immediately translatable into proofs of the general case that require the use of Stièltjes integrals.

Expectations with Respect to Numerical $n$ -Tuple Valued Random Phenomena . The foregoing ideas extend immediately to a numerical $n$ -tuple valued random phenomenon. Given the distribution function $F(x_1,x_2,\dots ,x_n)$ of such a random phenomenon and any continuous function $g(x_1,\dots ,x_n)$ of $n$ real variables, we define the expectation of the function with respect to the random phenomenon by

E\left[g\left(x_{1}, x_{2}, \ldots, x_{n}\right)\right] =\underset{R_n}{\iint \cdots \int} g\left(x_{1}, x_{2}, \ldots, x_{n}\right) d F\left(x_{1}, x_{2}, \ldots, x_{n}\right) \tag{6.8} 

in which the integral is a Stieltjes integral over the space $R_n$ of all $n$ -tuples $(x_1,x_2,\dots ,x_n)$ of real numbers. We shall not write out here the definition of this integral.

We note that (6.2) and (6.3) generalize. If the distribution function $F(x_1,x_2,\dots ,x_n)$ is specified by a probability density function $f(x_1,x_2,\dots ,x_n)$ so that \left(7.7^{\prime}\right) of Chapter 4 holds, then

E[g(x_{1}, x_{2}, \ldots, x_{n})] = \underbrace{\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty}}_{\text{$n$ integrals}} g(x_{1}, x_{2}, \ldots, x_{n}) f(x_{1}, x_{2}, \ldots, x_{n}) \, dx_{1} \, dx_{2} \cdots \, dx_{n} \tag{6.9} 

If the distribution function $F(x_1,x_2,\dots ,x_n)$ is specified by a probability mass function $p(x_1,x_2,\dots ,x_n)$ , so that \left(7.8^{\prime}\right) of Chapter 4 holds, then

\begin{align} & E\left[g\left(x_{1}, x_{2}, \ldots, x_{n}\right)\right] \tag{6.10} \\ & =\sum_{\substack{\text { over all }\left(x_{1}, x_{2}, \ldots, x_{n}\right) \\ \text { such that } p\left(x_{1}, x_{2}, \ldots, x_{n}\right)>0}} g\left(x_{1}, x_{2}, \ldots, x_{n}\right) p\left(x_{1}, x_{2}, \ldots, x_{n}\right) . \end{align}

Exercises