14.1: Matrix Methods for Linear Equations

Systems of linear equations are fundamental in various fields of engineering, science, and mathematics. While simple systems can be solved using substitution or elimination, more complex systems with many variables become cumbersome. This is where matrix methods provide a powerful and systematic approach to represent and solve such systems efficiently.

System Representation in Matrix Form

A system of m linear equations with n variables can be compactly written in matrix form as:

\[Ax=B\]

Where:

• A is the coefficient matrix (an m×n matrix containing the coefficients of the variables).
• x is the variable vector (an n×1 column vector containing the unknown variables).
• B is the constant vector (an m×1 column vector containing the constants on the right-hand side of the equations).

For example, consider the system of m linear equations with n unknowns:

\[\begin{align}
\begin{split}
a_{11}x_1 & + & a_{12}x_2 & + \dots + a_{1n}x_n & = b_1 \\
a_{21}x_1 & + & a_{22}x_2 & + \dots + a_{2n}x_n & = b_2 \\
\vdots & & \vdots & & \vdots \\
a_{m1}x_1 & + & a_{m2}x_2 & + \dots + a_{mn}x_n & = b_m
\end{split}
\end{align}\]

This system can be written in matrix form as:

\[ \begin{bmatrix}
a_{11} & a_{12} & \cdots & a_{1n}\\
a_{21} & a_{22} & \cdots & a_{2n} \\
\vdots & \vdots && \vdots \\
a_{m1} & a_{m2} & \cdots & a_{mn}
\end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix}\]

There are several possible solutions for a set of m linear equations with n unknowns, depending on the relationship between m and n, and the nature of the equations themselves. Here's a breakdown of the possibilities:

General Concepts

Before diving into specific cases, here are some fundamental ideas:

• Consistent System: A system of linear equations is consistent if it has at least one solution.
• Inconsistent System: A system of linear equations is inconsistent if it has no solutions.
• Unique Solution: A system has a unique solution if there is exactly one set of values for the unknowns that satisfies all equations.
• Infinitely Many Solutions: A system has infinitely many solutions if there are an infinite number of sets of values for the unknowns that satisfy all equations. This typically occurs when there are "redundant" equations or "free variables."

Case Analysis

Consider the possibilities based on the relationship between m (number of equations) and n (number of unknowns).

m=n (Square System)

When the number of equations equals the number of unknowns, it is a well-determined "square" system.

• Unique Solution: This is the most common and often desired outcome. A unique solution exists if and only if the coefficient matrix is invertible (non-singular). In terms of determinants, this means the determinant of the coefficient matrix is non-zero (\(\det(A) \neq 0\)). You can solve such systems using methods like Gaussian elimination, Cramer's Rule, or matrix inversion.
• No Solution (Inconsistent): If the determinant of the coefficient matrix is zero (\(\det(A) = 0\)), the system might be inconsistent. This happens when the equations are contradictory (e.g., \(x + y = 5\) and \(2x + 2y = 3\)). In Gaussian elimination, you would end up with a row like [0 0 ... 0 ∣ c] where \(c \neq 0\).
• Infinitely Many Solutions: If the determinant of the coefficient matrix is zero (\(\det(A) = 0\)), the system might have infinitely many solutions. This happens when the equations are dependent (e.g., \(2x + 3y = 5\) and \(4x + 6y = 10\)). One equation is essentially a multiple of another. In Gaussian elimination, you would end up with one or more rows of all zeros ([0 0 ... 0 ∣ 0]), indicating free variables

m<n (Underdetermined System)

When the number of equations is less than the number of unknowns, the system is "underdetermined." This implies there are more variables than constraints, which usually leads to multiple solutions.

• Infinitely Many Solutions: This is the most common scenario for underdetermined systems. If the system is consistent, there will be infinitely many solutions. You will typically have "free variables" that can take on any value, and the other variables will be expressed in terms of these free variables. Gaussian elimination will often result in rows of zeros, indicating these free variables.
• No Solution (Inconsistent): It is still possible for an underdetermined system to be inconsistent. This occurs if some of the equations are contradictory, even with fewer constraints than variables. For example, \(x + y + z = 5\) and \(x + y + z = 3\).
• Unique Solution: A unique solution is not possible for a consistent underdetermined system. If a solution exists, there will always be infinitely many because you have more unknowns than independent equations to uniquely determine them.

• m>n (Overdetermined System)

When the number of equations is greater than the number of unknowns, the system is "overdetermined." This implies more constraints than variables, making it more likely for inconsistencies.

• No Solution (Inconsistent): This is the most frequent outcome for overdetermined systems. It's often difficult for a single set of unknown values to satisfy all the equations simultaneously if there are too many independent constraints. Gaussian elimination will often lead to a row like [0 0 ... 0 ∣ c] where \(c \neq 0\).
• Unique Solution: A unique solution is possible, but it's a specific case. This happens when n of the equations are linearly independent and consistent, and the remaining m−n equations are redundant (i.e., they are linear combinations of the first n equations and are consistent with them). You can think of it as solving a subset of n equations and then checking if this solution satisfies the remaining m−n equations.
• Infinitely Many Solutions: This is also a possibility, though less common than no solution. This occurs when the n unknowns are not uniquely determined by any subset of the equations, and the remaining equations are consistent and redundant. For instance, if you have many identical or dependent equations.

Methods for Solving and Determining Solutions

To determine which solution type applies and to find the solutions, you typically use:

1. Gaussian Elimination (Row Echelon Form): This is the most robust and widely used method. By transforming the augmented matrix into row echelon form or reduced row echelon form, you can easily identify:

• Inconsistency: A row of the form [0 0 ... 0 ∣ c] where \(c \neq 0\).
• Unique Solution: A pivot position in every column corresponding to an unknown.
• Infinitely Many Solutions: Columns without pivot positions (indicating free variables) and no inconsistent rows.

1. Determinants (for m=n): As mentioned, the determinant of the coefficient matrix can quickly tell you if a unique solution exists (non-zero determinant). If it's zero, further analysis (like Gaussian elimination) is needed to distinguish between no solution and infinitely many solutions.
2. Matrix Inversion (for m=n and unique solution): If the coefficient matrix A is invertible, then \(Ax = b\) has a unique solution \(x = A^{−1}b\).
3. Rank of a Matrix: The rank of the coefficient matrix (A) and the rank of the augmented matrix ([A∣b]) are crucial for formal determination:

• Consistent System: rank(A)=rank([A∣b])
• Inconsistent System: rank(A)<rank([A∣b])
• Unique Solution: rank(A)=rank([A∣b])=n (number of unknowns)
• Infinitely Many Solutions: rank(A)=rank([A∣b])<n

In summary, the number of equations and unknowns, along with the specific coefficients and constants, dictate whether a system of linear equations has a unique solution, no solution, or infinitely many solutions. Gaussian elimination and the concept of matrix rank are the most powerful tools for analyzing these possibilities.

A linear system may behave in any one of three possible ways:

1. The system has a unique solution if n = m and A is nonsingular.
2. The system has infinitely many solutions when n > m.
3. The system has no solution if n < m.

A system of linear equations has a non-unique solution when it has either infinitely many solutions or no solution. in contrast, is a single, specific solution that satisfies all equations in the system.

Here's a more detailed breakdown:

1. Infinite Solutions:

• In a linear system, if there are more variables than independent equations, the system will typically have infinitely many solutions.
• Geometrically, this means the equations' lines or planes intersect along a line or a plane, rather than at a single point.
• Examples include systems where one equation is a multiple of another, or where equations represent the same line or plane.

2. No Solution:

• If a system of equations is inconsistent, meaning the equations contradict each other, then there is no solution.
• Geometrically, this means the lines or planes represented by the equations do not intersect at any point.
• For example, if two equations have the same variable terms but different constant terms, they represent parallel lines and have no solution.

3. Determining Non-uniqueness:

• Rank of Matrices: In linear algebra, the ranks of the coefficient matrix (A) and the augmented matrix (A|b) are used to determine the nature of solutions.
  • If the rank of A is less than the rank of (A|b), there are no solutions.
  • If the rank of A is equal to the rank of (A|b), but less than the number of variables, there are infinitely many solutions.
• Determinant of the Coefficient Matrix:

  For a square matrix (same number of equations and variables), if the determinant of the coefficient matrix is zero, the system either has no solution or infinitely many solutions.

4. Example:

• Consider the system:

\[\begin{align}
\begin{split}
2x + 3y &= 5 \\
4x + 6y &= 10
\end{split}
\end{align}\]
This system has infinitely many solutions because the second equation is simply twice the first equation, meaning they represent the same line.

5. Singular Matrices:

• A matrix is considered singular if it is not invertible (has no inverse).
• A singular matrix is associated with linear systems that have either no solution or infinitely many solutions.