1.5: Linear Systems of Equations

Last updated
Save as PDF

Page ID: 24231

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

Suppose that we have the following system of real or complex linear equations:

\[A^{m \times n} x^{n \times 1}=y^{m \times 1}\]

When does this system have a solution x for given A and y?

\[\exists \text { a solution } x \Longleftrightarrow y \in \mathcal{R}(A) \Longleftrightarrow \mathcal{R}([A \mid y])=\mathcal{R}(A)\]

We now analyze some possible cases:

If n = m, then \(\operatorname{det}(A) \neq 0 \Rightarrow x=A^{-1} y\), and x is the unique solution.
If m > n, then there are more equations than unknowns, i.e. the system is "overconstrained". If A and/or y reflect actual experimental data, then it is quite likely that the n-component vector y does not lie in \(\mathcal{R}(A)\), since this subspace is only n-dimensional (if A has full column rank) or less, but lives in an m-dimensional space. The system will then be inconsistent. This is the sort of situation encountered in estimation or identification problems, where x is a parameter vector of low dimension compared to the dimension of the measurements that are available. We then look for a choice of x that comes closest to achieving consistency, according to some error criterion. We shall say quite a bit more about this shortly.
If m < n, then there are fewer equations than unknowns, and the system is "underconstrained". If the system has a particular solution \(x_{p}\) (and when rank(A)= m, there is guaranteed to be a solution for any y) then there exist an infinite number of solutions. More specifically, x is a solution iff (if and only if)

\[\begin{aligned}
&x=x_{p}+x_{h}, \quad A x_{p}=y, \quad A x_{h}=0\ &\text { i.e. } x_{h} \in \mathcal{N}(A)
\end{aligned}\]

Since the nullspace \(\mathcal{N}(A)\) has dimension at least n - m, there are at least this many degrees of freedom in the solution. This is the sort of situation that occurs in many control problems, where the control objectives do not uniquely constrain or determine the control. We then typically search among the available solutions for ones that are optimal according to some criterion.