2.3: Preliminary- The Gram Product

Given the array of \(n_{A}\) vectors \(A= [a_{1}], \dots, [a_{n_{A}}]\) and the array of \(n_{B}\) vectors \(B= [b_{1}], \dots, [b_{n_{B}}]\) from a given inner product space, let \(\prec A , B \succ\) denote the \(n_{A} \times n_{B}\) matrix whose (i, j)-th element is \(<a_{i}, b_{j}>\). We shall refer to this object as the Gram product (but note that this terminology is not standard!).

If the vector space under consideration is \(\mathbf{R}^{m}\) or \(\mathbf{C}^{m}\), then both A and B are matrices with m rows, but our definition of \(\prec A , B \succ\) can actually handle more general A, B. In fact, the vector space can be infinite dimensional, as long as we are only examining finite collections of vectors from this space. For instance, we could use the same notation to treat finite collections of vectors chosen from the infinite-dimensional vector space \(\mathcal{L}^{2}\) of square integrable functions, i.e. functions a(t) for which \(\int_{-\infty}^{\infty} a^{2}(t) d t<\infty\). The inner product in \(\mathcal{L}^{2}\) is \(<a(t), b(t)>=\int_{-\infty}^{\infty} a^{*}(t) b(t) d t\). (The space \(\mathcal{L}^{2}\) is an example of an infinite dimensional Hilbert space, and most of what we know for finite dimensional spaces - which are also Hilbert spaces! - has natural generalizations to infinite dimensional Hilbert spaces. Many of these generalizations involve introducing notions of topology and measure, so we shall not venture too far there. It is worth also mentioning here another important infinite dimensional Hilbert space that is central to the probabilistic treatment of least squares estimation: the space of zero-mean random variables, with the expected value E(ab) serving as the inner product \(<a, b>\).)

For the usual Euclidean inner product in an m-dimensional space, where \(<a_{i}, b_{j}>={a_{i}}^{\prime} b_{j}\), we simply have \(\prec A , B \succ = A^{\prime}B\). For the inner product defined by \(<a_{i}, b_{j}>=a_{i}^{\prime} S b_{j}\) for a positive definite, Hermitian matrix S, we have \(\prec A , B \succ = A^{\prime}SB\).

• Verify that the symmetry and linearity of the inner product imply the same for the Gram product, so \(\prec A F, B G+C H \succ=F^{\prime} \prec A, B \succ G+F^{\prime} \prec A, C \succ H\), for any constant matrices F , G, H (a constant matrix is a matrix of scalars), with A, B, C denoting arrays whose columns are vectors.