Let us now tie up linear transformations with the theory of tensor products. Let \(\mathcal{U}\) and \(\mathcal{V}\) be finite-dimensional vector spaces (over the same field), and let \(A\) and \(B\) be any two linear transformations on \(\mathcal{U}\) and \(\mathcal{V}\) respectively. We define a linear transformation \(\overline{C}\) on the space \(\mathcal{W}\) of all bilinear forms on \(\mathcal{U} \oplus \mathcal{W}\) by writing \[(\overline{C} w)(x, y)=w(A x, B y).\] The tensor product \(C=A \otimes B\) of the transformations \(A\) and \(B\) is, by definition, the dual of the transformation \(\overline{C}\) , so that \[(C z)(w)=z(\overline{C}w)\] whenever \(z\) is in \(\mathcal{U} \otimes \mathcal{V}\) and \(w\) is in \(\mathcal{W}\) . If we apply \(C\) to an element \(z_{0}\) of the form \(z_{0}=x_{0} \otimes y_{0}\) (recall that this means that \(z_{0}(w)=w(x_{0}, y_{0})\) for all \(w\) in \(\mathcal{W}\) ), we obtain \begin{align} (C z_{0})(w) &= z_{0}(\overline{C}w)\\ &= (x_{0} \otimes y_{0})(\overline{C} w) \\ &= (\overline{C} w)(x_{0}, y_{0})\\ &= w(A x_{0}, B y_{0})\\ &= (A x_{0} \otimes B y_{0})(w). \end{align} We infer that \[C z_{0}=A x_{0} \otimes B y_{0}. \tag{1}\] Since there are quite a few elements in \(\mathcal{U} \otimes \mathcal{V}\) of the form \(x \otimes y\) , enough at any rate to form a basis (see Section: Product bases ), this relation characterizes \(C\) .

The formal rules for operating with tensor products go as follows. \begin{align} A \otimes 0 &= 0 \otimes B=0, \tag{2}\\ 1 \otimes 1 &= 1, \tag{3}\\ (A_{1}+A_{2}) \otimes B &= (A_{1} \otimes B)+(A_{2} \otimes B), \tag{4}\\ A \otimes(B_{1}+B_{2}) &= (A \otimes B_{1})+(A \otimes B_{2}), \tag{5}\\ \alpha A \otimes \beta B &= \alpha \beta(A \otimes B), \tag{6}\\ (A \otimes B)^{-1} &= A^{-1} \otimes B^{-1}, \tag{7}\\ (A_{1} A_{2}) \otimes (B_{1} B_{2}) &= (A_{1} \otimes B_{1})(A_{2} \otimes B_{2}). \tag{8} \end{align} The proofs of all these relations, except perhaps the last two, are straightforward.

Formula (7), as all formulas involving inverses, has to be read with caution. It is intended to mean that if both \(A\) and \(B\) are invertible, then so is \(A \otimes B\) , and the equation holds, and, conversely, that if \(A \otimes B\) is invertible, then so also are \(A\) and \(B\) . We shall prove (7) and (8) in reverse order.

Formula (8) follows from the characterization (1) of tensor products and the following computation: \begin{align} (A_{1} A_{2} \otimes B_{1} B_{2})(x \otimes y) &= A_{1} A_{2} x \otimes B_{1} B_{2} y\\ &= (A_{1} \otimes B_{1})(A_{2} x \otimes B_{2} y)\\ &= (A_{1} \otimes B_{1})(A_{2} \otimes B_{2})(x \otimes y). \end{align} As an immediate consequence of (8) we obtain \[A \otimes B=(A \otimes 1)(1 \otimes B)=(1 \otimes B)(A \otimes 1). \tag{9}\] 

To prove (7), suppose that \(A\) and \(B\) are invertible, and form \(A \otimes B\) and \(A^{-1} \otimes B^{-1}\) . Since, by (8), the product of these two transformations, in either order, is \(1\) , it follows that \(A \otimes B\) is invertible and that (7) holds. Conversely, suppose that \(A \otimes B\) is invertible. Remembering that we defined tensor products for finite-dimensional spaces only, we may invoke Section: Inverses , Theorem 2; it is sufficient to prove that \(A x=0\) implies that \(x=0\) and \(B y=0\) implies that \(y=0\) . We use (1): \[A x \otimes B y=(A \otimes B)(x \otimes y).\] If either factor on the left is zero, then \((A \otimes B)(x \otimes y)=0\) , whence \(x \otimes y=0\) , so that either \(x=0\) or \(y=0\) . Since (by (2)) \(B=0\) is impossible, we may find a vector \(y\) so that \(B y \neq 0\) . Applying the above argument to this \(y\) , with any \(x\) for which \(A x=0\) , we conclude that \(x=0\) . The same argument with the roles of \(A\) and \(B\) interchanged proves that \(B\) is invertible.

An interesting (and complicated) side of the theory of tensor products of transformations is the theory of Kronecker products of matrices. Let \(\mathcal{X}=\{x_{1}, \ldots, x_{n}\}\) and \(\mathcal{Y}=\{y_{1}, \ldots, y_{m}\}\) be bases in \(\mathcal{U}\) and \(\mathcal{V}\) , and let \([A]=[A; \mathcal{X}]=(\alpha_{i j})\) and \([B]=[B; \mathcal{Y}]=(\beta_{p q})\) be the matrices of \(A\) and \(B\) . What is the matrix of \(A \otimes B\) in the coordinate system \(\{x_{i} \otimes y_{p}\}\) ?

To answer the question, we must recall the discussion in Section: Matrices concerning the arrangement of a basis in a linear order. Since, unfortunately, it is impossible to write down a matrix without being committed to an order of the rows and the columns, we shall be frank about it, and arrange the \(n\) times \(m\) vectors \(x_{i} \otimes y_{p}\) in the so-called lexicographical order, as follows: \[x_1 \otimes y_1, \ldots, x_1 \otimes y_m, \quad x_2 \otimes y_1, \ldots, x_2 \otimes y_m, \quad \ldots \quad, x_n \otimes y_1, \ldots, x_n \otimes y_m.\] We proceed also to carry out the following computation: \begin{align} (A \otimes B)(x_{j} \otimes y_{q}) &= A x_{j} \otimes B y_{q}\\ &= \Big(\sum_{i} \alpha_{i j} x_{i}\Big) \otimes \Big(\sum_{p} \beta_{p q} y_{p}\Big) \\ &= \sum_{i} \sum_{p} \alpha_{i j} \beta_{p q}(x_{i} \otimes y_{p}). \end{align} This process indicates exactly how far we can get without ordering the basis elements; if, for example, we agree to index the elements of a matrix not with a pair of integers but with a pair of pairs, say \((i, p)\) and \((j, q)\) , then we know now that the element in the \((i, p)\) row and the \((j, q)\) column is \(\alpha_{i j} \beta_{p q}\) . If we use the lexicographical ordering, the matrix of \(A \otimes B\) has the form \[\begin{bmatrix} \alpha_{11}\beta_{11} & \cdots & \alpha_{11}\beta_{1m} & \cdots\cdots & \alpha_{1n}\beta_{11} & \cdots & \alpha_{1n}\beta_{1m}\\ \vdots & \vdots & \vdots & \cdots\cdots & \vdots & \vdots & \vdots\\ \alpha_{11}\beta_{m1} & \cdots & \alpha_{11}\beta_{mm} & \cdots\cdots & \alpha_{1n}\beta_{m1} & \cdots & \alpha_{1n}\beta_{mm}\\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots\\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots\\ \alpha_{n1}\beta_{11} & \cdots & \alpha_{n1}\beta_{1m} & \cdots\cdots & \alpha_{nn}\beta_{11} & \cdots & \alpha_{nn}\beta_{1m}\\ \vdots & \vdots & \vdots & \cdots\cdots & \vdots & \vdots & \vdots\\ \alpha_{n1}\beta_{m1} & \cdots & \alpha_{n1}\beta_{mm} & \cdots\cdots & \alpha_{nn}\beta_{m1} & \cdots & \alpha_{nn}\beta_{mm} \end{bmatrix}.\] In a condensed notation whose meaning is clear we may write this matrix as \[\begin{bmatrix} a_{11}[B] & a_{12}[B] & \cdots & a_{1 n}[B] \\ a_{21}[B] & a_{22}[B] & \cdots & a_{2 n}[B] \\ \vdots & \vdots & \ddots & \vdots \\ a_{n 1}[B] & a_{n 2}[B] & \cdots & a_{n n}[B] \end{bmatrix}.\] 

This matrix is known as the Kronecker product of \([A]\) and \([B]\) , in that order. The rule for forming it is easy to describe in words: replace each element \(\alpha_{i j}\) of the \(n\) -by- \(n\) matrix \([A]\) by the \(m\) -by- \(m\) matrix \(\alpha_{i j}[B]\) . If in this rule we interchange the roles of \(A\) and \(B\) (and consequently interchange \(n\) and \(m\) ) we obtain the definition of the Kronecker product of \([B]\) and \([A]\) .

EXERCISES