07 Linear Algebra, Solver Equations & Factorizations

Overview: Numerical Matrix Computations

MATLAB stands for Matrix Laboratory. It is built on highly optimized LAPACK and BLAS linear algebra libraries. This note covers solving systems of equations, matrix generation, and matrix factorizations.

---

1. Solving Linear Systems: Backslash (\) vs. Inverse (inv)

To solve the system of linear equations $Ax=b$:

x = A \ b; % Preferred (Backslash left-division)

The Backslash Solver (\, mldivide)

• Syntax: x = A\b
• Mechanics: The compiler analyzes the matrix properties of $A$:
  • If $A$ is symmetric/positive-definite, it solves via Cholesky factorization.
  • If $A$ is square, it solves via LU decomposition with partial pivoting.
  • If $A$ is rectangular/overdetermined, it computes a QR factorization to find the least-squares solution.
• Why it is superior to inv(A)*b: Computing the explicit inverse matrix using inv(A) is computationally expensive ($3\times$ slower) and numerically unstable. It amplifies rounding errors, especially for ill-conditioned matrices. The backslash operator solves the system directly without calculating the full inverse, preserving precision.

---

2. Standard Matrix Generation Functions

Function Signatures:

• zeros
  • Syntax: A = zeros(m, n)
  • Description: Returns an $m\times n$ matrix of zeros.
• ones
  • Syntax: A = ones(m, n)
  • Description: Returns an $m\times n$ matrix of ones.
• eye
  • Syntax: A = eye(m, n)
  • Description: Returns an $m\times n$ identity matrix with ones on the main diagonal.
• linspace
  • Syntax: v = linspace(x1, x2, n)
  • Description: Generates a row vector of n linearly spaced points between x1 and x2 (inclusive).
• diag
  • Syntax: D = diag(V) (creates diagonal matrix from vector V) or v = diag(A) (extracts diagonal elements of matrix A).

---

3. Matrix Factorizations

MATLAB provides wrapper functions for complex matrix factorizations:

A. LU Decomposition:

• Syntax: [L, U, P] = lu(A)
• Outputs:
  • P: Permutation matrix (records row swapping/pivoting).
  • L: Lower triangular matrix.
  • U: Upper triangular matrix.
• Relation: $PA=LU$. Used to solve equations and calculate determinants.

B. Eigen-Decomposition:

• Syntax: [V, D] = eig(A) or d = eig(A) (returns eigenvalues only as a vector).
• Outputs:
  • V: Matrix whose columns are the right-hand eigenvectors of $A$.
  • D: Diagonal matrix containing eigenvalues.
• Relation: $AV=VD$.

C. Singular Value Decomposition (SVD):

• Syntax: [U, S, V] = svd(A)
• Outputs:
  • U: Unitary matrix.
  • S: Diagonal matrix of singular values in descending order.
  • V: Unitary matrix.
• Relation: $A=US{V}^T$.

---

💻 Conceptual Code Demonstration

% 1. Create a system Ax = b
A = [2 1 1; 1 3 2; 1 0 4];
b = [4; 5; 6];
 
% 2. Solve system (performs LU pivoting automatically)
x = A \ b; 
 
% Verify solution
residual = A*x - b; % Should be close to 0
 
% 3. Extract Eigensystem
[V, D] = eig(A);
 
% 4. Matrix pseudo-inverse (for non-square / singular matrices)
% Syntax: X = pinv(A)
A_singular = [1 2; 2 4]; % Singular matrix (det = 0)
% inv(A_singular) % ERROR: Matrix is singular to working precision!
X_pseudo = pinv(A_singular); % Calculates Moore-Penrose pseudo-inverse safely