Skip to main content
Engineering LibreTexts

2.2: Computing the Estimate

  • Page ID
    24235
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    The solution, \(\hat{x}\), of Equation 2.1 is characterized by:

    \[(y-A \hat{x}) \perp \mathcal{R}\tag{A}\]

    All elements in a basis of \(\mathcal{R}(A)\) must be orthogonal to \((y-A \hat{x})\). Equivalently this is true for the set of columns of \(A, [{a_{1}, \dots a_{n}}]\). Thus

    \[\begin{aligned}
    (y-A \hat{x}) \perp \mathcal{R}(A) & \Leftrightarrow a_{i}^{\prime}(y-A \hat{x})=0 \quad \text { for } i=1, \ldots, n \\
    & \Leftrightarrow A^{\prime}(y-A \hat{x})=0 \\
    & \Leftrightarrow A^{\prime} A \hat{x}=A^{\prime} y
    \end{aligned}\]

    This system of m equations in the m unknowns of interest is referred to as the normal equations. We can solve for the unique \(\hat{x}\) iff \(A^{\prime}A\) is invertible. Conditions for this will be derived shortly. In the sequel, we will present the generalization of the above ideas for infinite dimensional vector spaces.


    This page titled 2.2: Computing the Estimate is shared under a CC BY-NC-SA 4.0 license and was authored, remixed, and/or curated by Mohammed Dahleh, Munther A. Dahleh, and George Verghese (MIT OpenCourseWare) via source content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request.