The most important relation among the vectors of an inner product space is orthogonality. By definition, the vectors \(x\) and \(y\) are called orthogonal if \((x, y)=0\) . We observe that this relation is symmetric; since \((x, y)=\overline{(y, x)}\) , it follows that \((x, y)\) and \((y, x)\) vanish together. If we recall the motivation for the introduction of \((x, y)\) , the terminology explains itself; two vectors are orthogonal (or perpendicular) if the angle between them is \(90^{\circ}\) , that is, if the cosine of the angle between them is \(0\) . Two subspaces are called orthogonal if every vector in each is orthogonal to every vector in the other.

A set \(x\) of vectors is orthonormal if whenever both \(x\) and \(y\) are in \(\mathcal{X}\) it follows that \((x, y)=0\) or \((x, y)=1\) according as \(x \neq y\) or \(x=y\) . (If \(\mathcal{X}\) is finite, say \(\mathcal{X}=\{x_{1}, \ldots, x_{n}\}\) , we have \((x_{i}, x_{j})=\delta_{i j}\) .) We call an orthonormal set complete if it is not contained in any larger orthonormal set.

To make our last definition in this connection, we observe first that an orthonormal set is linearly independent. Indeed, if \(\{x_{1}, \ldots, x_{k}\}\) is any finite subset of an orthonormal set \(\mathcal{X}\) , then \(\sum_{i} \alpha_{i} x_{i}=0\) implies that \begin{align} 0 &= \Big(\sum_{i} \alpha_{i} x_{i}, x_{j}\Big)\\ &= \sum_{i} \alpha_{i}(x_{i}, x_{j})\\ &= \sum_{i} \alpha_{i} \delta_{i j}\\ &= \alpha_{j}; \end{align} in other words, a linear combination of the \(x\) ’s can vanish only if all the coefficients vanish. From this we conclude that in a finite-dimensional inner product space the number of vectors in an orthonormal set is always finite, and, in fact, not greater than the linear dimension of the space. We define, in this case, the orthogonal dimension of the space, as the largest number of vectors an orthonormal set can contain.

Warning: for all we know at this stage, the concepts of orthogonality and orthonormal sets are vacuous. Trivial examples can be used to show that things are not so bad as all that; the vector \(0\) , for instance, is always orthogonal to every vector, and, if the space contains a non-zero vector \(x\) , then the set consisting of \(\frac{x}{\|x\|}\) alone is an orthonormal set. We grant that these examples are not very inspiring. For the present, however, we remain content with them; soon we shall see that there are always "enough" orthogonal vectors to operate with in comfort.

Observe also that we have no right to assume that the number of elements in a complete orthonormal set is equal to the orthogonal dimension. The point is this: if we had an orthonormal set with that many elements, it would clearly be complete; it is conceivable, just the same, that some other set contains fewer elements, but is still complete because its nasty structure precludes the possibility of extending it. These difficulties are purely verbal and will evaporate the moment we start proving things; they occur only because from among the several possibilities for the definition of completeness we had to choose a definite one, and we must prove its equivalence with the others.

We need some notation. If \(\mathcal{E}\) is any set of vectors in an inner product space \(\mathcal{V}\) , we denote by \(\mathcal{E}^{\perp}\) the set of all vectors in \(\mathcal{V}\) that are orthogonal to every vector in \(\mathcal{E}\) . It is clear that \(\mathcal{E}^{\perp}\) is a subspace of \(\mathcal{V}\) (whether or not \(\mathcal{E}\) is one), and that \(\mathcal{E}\) is contained in \(\mathcal{E}^{\perp \perp}=(\mathcal{E}^{\perp})^{\perp}\) . It follows that the subspace spanned by \(\mathcal{E}\) is contained in \(\mathcal{E}^{\perp \perp}\) . In case \(\mathcal{E}\) is a subspace, we shall call \(\mathcal{E}^{\perp}\) the orthogonal complement of \(\mathcal{E}\) . We use the sign in order to be reminded of orthogonality (or perpendicularity). In informal discussions, \(\mathcal{E}^{\perp}\) might be pronounced as "E perp."

EXERCISES

Exercise 1. Given four complex numbers \(\alpha\) , \(\beta\) , \(\gamma\) , and \(\delta\) , try to define an inner product in \(\mathbb{C}^{2}\) by writing \[(x, y)=\alpha \xi_{1} \bar{\eta}_{1}+\beta \xi_{2} \bar{\eta}_{1}+\gamma \xi_{1} \bar{\eta}_{2}+\delta \xi_{2} \bar{\eta}_{2}\] whenever \(x=(\xi_{1}, \xi_{2})\) and \(y=(\eta_{1}, \eta_{2})\) . Under what conditions on \(\alpha\) , \(\beta\) , \(\gamma\) , and \(\delta\) does this equation define an inner product?

Exercise 2. Prove that if \(x\) and \(y\) are vectors in a unitary space, then \[4(x, y)=\|x+y\|^{2}-\|x-y\|^{2}+i\|x+i y\|^{2}-i\|x-i y\|^{2}.\] 

Exercise 3. If inner product in \(\mathcal{P}_{n+1}\) is defined by \[(x, y)=\int_{0}^{1} x(t) \overline{y(t)} \,d t,\] and if \(x_{j}(t)=t^j\) , \(j=0, \ldots, n-1\) , find a polynomial of degree \(n\) orthogonal to \(x_{0}, x_{1}, \ldots, x_{n-1}\) .

Exercise 4. 

  1. Two vectors \(x\) and \(y\) in a real inner product space are orthogonal if and only if \(\|x+y\|^{2}=\|x\|^{2}+\|y\|^{2}\) .
  2. Show that (a) becomes false if "real" is changed to "complex."
  3. Two vectors \(x\) and \(y\) in a complex inner product space are orthogonal if and only if \(\|\alpha x+\beta y\|^{2}=\|\alpha x\|^{2}+\|\beta y\|^{2}\) for all pairs of scalars \(\alpha\) and \(\beta\) .
  4. If \(x\) and \(y\) are vectors in a real inner product space, and if \(\|x\|=\|y\|\) , then \(x-y\) and \(x+y\) are orthogonal. (Picture?) Discuss the corresponding statement for complex spaces.
  5. If \(x\) and \(y\) are vectors in an inner product space, then \[\|x+y\|^{2}+\|x-y\|^{2}=2\|x\|^{2}+2\|y\|^{2}.\] Picture?