Math 2109: Vector, Matrix and Transform

Matrix Algebra: Comprehensive Theory & Concepts

Document Overview

This note compiles all the core theoretical concepts, definitions, matrix types, rules, shortcuts, and theorems covered in the Matrix Algebra course (Classes 1 to 8), including both class notes and past exam papers (PYQs).

• Restructured Format: To maintain a clean, rapid-reference structure, all step-by-step proofs and derivations have been removed, leaving only the final mathematical formulations, definitions, and theorems. All numerical examples are also excluded.

---

1st Cycle | 13.07.2026 | Class-01

Core Matrix Definitions

Formal Definition: Matrix

A matrix $A$ of order (or size) $m\times n$ (read as ”$m$ by $n$”) is a rectangular array of $m\cdot n$ numbers (real or complex) arranged in $m$ horizontal rows and $n$ vertical columns, enclosed in square brackets $\left[\begin{array}{c}\dots \end{array}\right]$ or parentheses $(\dots )$.

Mathematically, it is written as:

a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} $$ * **Element Indexing:** $a_{ij}$ denotes the entry located at the **$i$-th row** and **$j$-th column** ($1 \le i \le m$, $1 \le j \le n$). * **Order / Dimension:** $m \times n$, where $m$ is row count and $n$ is column count.

Principal Diagonal & Trace

For a square matrix $A$ of order $n\times n$:

• Main / Principal Diagonal: Elements $a_{11},a_{22},\dots ,a_{nn}$ where row index equals column index ($i=j$).
• Trace of Matrix $\text{tr}(A)$: The sum of all elements on the principal diagonal: $\text{tr}(A)={\sum }_{i=1}^n{a}_{ii}=a_{11}+a_{22}+\cdots +a_{nn}$

---

Basic Matrix Types

Types of Matrices

1. Row Matrix: A matrix having only one row ($m=1$) and any number of columns ($1\times n$).
2. Column Matrix: A matrix having only one column ($n=1$) and any number of rows ($m\times 1$).
3. Rectangular Matrix: A matrix where the number of rows is not equal to the number of columns ($m\ne n$).
4. Square Matrix: A matrix where the number of rows equals the number of columns ($m=n$).
5. Diagonal Matrix: A square matrix in which all non-diagonal elements are zero, i.e., $a_{ij}=0$ for all $i\ne j$.
6. Scalar Matrix: A diagonal matrix in which all principal diagonal elements are equal to a constant scalar $k$, i.e., $a_{ij}=k$ when $i=j$, and $a_{ij}=0$ when $i\ne j$.
7. Identity (Unit) Matrix: A scalar matrix whose principal diagonal elements are all equal to $1$. Denoted by $I$ or $I_n$.
8. Null (Zero) Matrix: A matrix of any dimension whose entries are all zero. Denoted by $O$.
9. Symmetric Matrix: A square matrix that is equal to its transpose, i.e., $A=A^T$. In element form: $a_{ij}=a_{ji}$ for all $i,j$.
10. Skew-Symmetric (Anti-Symmetric) Matrix: A square matrix whose transpose equals its negative, i.e., $A=-A^T$. In element form: $a_{ij}=-a_{ji}$ for all $i,j$.
11. Upper Triangular Matrix: A square matrix where all entries below the principal diagonal are zero, i.e., $a_{ij}=0$ for all $i>j$.
12. Lower Triangular Matrix: A square matrix where all entries above the principal diagonal are zero, i.e., $a_{ij}=0$ for all $i<j$.
13. Submatrix of a Matrix: A matrix obtained by deleting some rows and/or columns of a given matrix.

Property of Identity Matrix

For any matrix $A_{m\times n}$: $A\cdot I_n=A$ and $I_m\cdot A=A$.

Determinant Shortcut for Triangular Matrices

The determinant of any upper or lower triangular matrix equals the product of its diagonal elements: $\det (A)=a_{11}\cdot a_{22}\cdots a_{nn}$

Key Property: Diagonal Elements of Skew-Symmetric Matrices

All principal diagonal elements of any skew-symmetric matrix must always be zero ($a_{ii}=0$).

---

14.07.26 | Class-02

Key Theorems on Real Matrices

Theorem: Unique Deconstruction of Real Square Matrices

Every square matrix $A$ over real numbers can be uniquely expressed as the sum of a symmetric matrix $B$ and a skew-symmetric matrix $C$: $A=B+C$ where:

• $B=\frac{1}{2}(A+A^T)$ is symmetric ($B^T=B$).
• $C=\frac{1}{2}(A-A^T)$ is skew-symmetric ($C^T=-C$).

---

Complex Matrices & Conjugate Transpose

When matrix entries belong to the complex field $\mathbb{C}$ (i.e., $a_{ij}=x+iy$), we define:

1. Complex Conjugate Matrix ($\bar{A}$): Replacing every entry $a_{ij}$ with its complex conjugate ${\bar{a}}_{ij}=x-iy$.
2. Conjugate Transpose Matrix ($A^{\ast }$): Transpose of the conjugate matrix, i.e., $A^{\ast }=(\bar{A}{)}^T=\overline{(A^T)}$.

Hermitian Matrix

A square matrix $A=[a_{ij}]$ over complex numbers is Hermitian if it equals its conjugate transpose: $A^{\ast }=A⟺a_{ij}={\bar{a}}_{ji}\text{for all }i,j$

Key Property: Diagonal Elements of Hermitian Matrices

All principal diagonal elements of a Hermitian matrix must be purely real numbers.

Skew-Hermitian Matrix

A square complex matrix $A$ is Skew-Hermitian if its conjugate transpose equals its negative: $A^{\ast }=-A⟺a_{ij}=-{\bar{a}}_{ji}\text{for all }i,j$

Key Property: Diagonal Elements of Skew-Hermitian Matrices

All principal diagonal elements of a Skew-Hermitian matrix must be either zero or purely imaginary.

Theorem: Unique Deconstruction of Complex Square Matrices

Any complex square matrix $A$ can be uniquely expressed as the sum of a Hermitian matrix $B$ and a Skew-Hermitian matrix $C$: $A=B+C$ where:

• $B=\frac{1}{2}(A+A^{\ast })$ is Hermitian ($B^{\ast }=B$).
• $C=\frac{1}{2}(A-A^{\ast })$ is Skew-Hermitian ($C^{\ast }=-C$).

---

Special Types of Square Matrices

Matrix Type	Defining Condition	Key Properties / Determinant / Inverses
Singular	$\det (A)=0$	Has no matrix inverse ($A^{-1}$ does not exist).
Non-Singular	$\det (A)\ne 0$	Inverse $A^{-1}$ exists.
Orthogonal	$A{A}^T=A^{T}A=I$	$A^{-1}=A^T$, $\det (A)=\pm 1$.
Idempotent	$A^2=A$	$A^k=A$ for any integer $k\ge 1$. Eigenvalues are $0$ or $1$.
Nilpotent	$A^n=O,A^{n-1}\ne O$	$n$ is index/order of nilpotency (e.g. $\left[\begin{array}{cc}0& 1\\ 0& 0\end{array}\right]$ is order 2). $\det (A)=0$.
Periodic	$A^{m+1}=A$	$m$ is the period (Idempotent has period $m=1$).
Involutory	$A^2=I$	$A^{-1}=A$, $\det (A)=\pm 1$.
Unitary	$A{A}^{\ast }=A^{\ast }A=I$	Complex analogue of Orthogonal matrix ($A^{-1}=A^{\ast }$). $\Vert \det (A)\Vert =1$.

Concrete Example of a Unitary Matrix

$U=\frac{1}{\sqrt{2}}\left[\begin{array}{cc}1& i\\ i& 1\end{array}\right]\text{where }{U}^{\ast }=\frac{1}{\sqrt{2}}\left[\begin{array}{cc}1& -i\\ -i& 1\end{array}\right]⟹U{U}^{\ast }=I$

---

Key Algebraic Properties

Theorem: Uniqueness of Matrix Inverse

If a square matrix $A$ has an inverse, then the inverse is strictly unique.

Theorem: Anti-Commutative Square Sum

If square matrices $A$ and $B$ of the same order anti-commute ($AB=-BA$), then: $(A+B{)}^2=A^2+B^2$

---

Partitioning & Block Matrices

Partitioning Rule for Matrix Multiplication

Two partitioned matrices $A$ and $B$ are conformable for block multiplication if the column partitioning of $A$ matches the row partitioning of $B$.

---

20.07.26 | Class-03

Elementary Transformations

Three Elementary Row Operations (EROs)

An Elementary Transformation refers to performing any of three fundamental operations on rows (or columns) of a matrix without changing its solution space or rank:

1. Row Interchange ($R_i\leftrightarrow R_j$): Swap two rows $i$ and $j$.
2. Row Scaling ($R_i\to k{R}_i$): Multiply all entries in row $i$ by a non-zero constant scalar $k\ne 0$.
3. Row Addition ($R_i\to R_i+k{R}_j$): Add to row $i$ a scalar multiple $k$ of row $j$.

Matrix Equivalence ( $\sim$)

If a matrix $B$ is obtained from $A$ by performing a finite sequence of elementary row operations, then $A$ and $B$ are equivalent matrices, denoted as: $A\sim B$ Equivalent matrices share identical rank, determinant properties (nullity vs non-nullity), and system solutions.

Elementary Matrix Definition & Properties

An Elementary Matrix $E$ is a matrix obtained by performing a single elementary row operation on an Identity Matrix $I$.

• Concrete Example: Row swap matrix $E_{12}$ (swaps row 1 and row 2 of a $2\times 2$ identity matrix): $E_{12}=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]$
• Key Properties:
  • Pre-multiplying a matrix $A$ by $E$ is equivalent to applying ERO to $A$: $E\cdot A$.
  • Every elementary matrix is invertible, and its inverse is also an elementary matrix.

Strict Rules for Writing Row Operations

To maintain rigorous notation in exams, follow these exact syntax rules:

1. Target Row Order: In $R_i^{\prime }\to R_i\pm k{R}_j$, the row being modified ($R_i$) MUST be written first.
   • $✓R_2^{\prime }\to R_2-3R_1$ (Correct)
   • $\times R_2^{\prime }\to 3R_1-R_2$ (Incorrect notation format)
2. No Simultaneous Inter-dependency: Never alter two rows based on each other in the exact same step!
   • $\times \{\begin{array}{l}{R}_1^{\prime }\to R_1-R_2\\ R_2^{\prime }\to R_2-R_1\end{array}$ (Will destroy row information and zero out rows invalidly).

---

Rank of a Matrix

Dual Definition of Matrix Rank

Definition 1 (Linear Independence): The rank of a matrix $A$, denoted by $\text{rank}(A)$ or $\rho (A)$, is the maximum number of linearly independent rows (or columns) in $A$.

Definition 2 (Minor-Based Definition): A matrix $A$ is said to have rank $r$ if and only if:

1. There exists at least one non-zero minor of order $r$ in $A$.
2. Every minor of order higher than $r$ (i.e., order $r+1$ or greater) vanishes completely (equals zero).

Essential Properties of Matrix Rank

1. For any matrix $A_{m\times n}$: $\text{rank}(A)\le \min (m,n)$.
2. $\text{rank}(A)=\text{rank}(A^T)$.
3. Elementary row transformations do not alter the rank of a matrix.
4. $\text{rank}(AB)\le \min (\text{rank}(A),\text{rank}(B))$.
5. A square matrix $A_{n\times n}$ is non-singular $⟺\text{rank}(A)=n$.

---

Echelon Forms

Conditions for Row-Echelon Form (REF)

A matrix is in Row-Echelon Form (REF) if it satisfies three structural conditions:

1. Zero Rows at Bottom: All rows consisting entirely of zeros are grouped at the bottom of the matrix.
2. Staircase Leading Entries: The leading entry (the first non-zero number from the left) of a non-zero row is strictly to the right of the leading entry of the row above it.
3. Zeros Below Pivots: All entries in a column directly below a leading entry are zero.

Reduced Row-Echelon Form (RREF)

A matrix is in Reduced Row-Echelon Form (RREF) if it satisfies all REF conditions PLUS:

1. Every leading entry (pivot) is strictly $1$.
2. Every pivot $1$ is the only non-zero entry in its entire column (entries both above and below the pivot are zero).

How to Find Rank Using Echelon Form

Once a matrix $A$ is transformed into Row-Echelon Form via elementary row operations: $\text{rank}(A)=\text{Number of Non-Zero Rows in its Row-Echelon Form}$

Uniqueness of Echelon Forms

• Row-Echelon Form (REF): Is not unique; different sequences of EROs can lead to different echelon forms.
• Reduced Row-Echelon Form (RREF): Is strictly unique; every matrix has a single unique RREF.

---

Key Matrix Polynomial Theorems

Cayley-Hamilton Theorem

Every square matrix satisfies its own characteristic equation. If $p(\lambda )=\det (\lambda I-A)={\lambda }^n+c_{n-1}{\lambda }^{n-1}+\cdots +c_0=0$ is the characteristic equation of matrix $A$, then: $p(A)=A^n+c_{n-1}{A}^{n-1}+\cdots +c_{0}I=O$

---

27.07.26 | Class-04

Normal Form of a Matrix

Definition: Normal Form (Canonical Form)

A square or rectangular matrix $A$ of order $m\times n$ can be reduced by a sequence of elementary row and column operations (EROs and ECOs) to one of the following four standard block structures, known as the Normal Form of the matrix:

1. $I_r$
2. $\left[\begin{array}{cc}{I}_r& O\end{array}\right]$
3. $\left[\begin{array}{c}{I}_r\\ O\end{array}\right]$
4. $\left[\begin{array}{cc}{I}_r& O\\ O& O\end{array}\right]$

where $I_r$ is the Identity Matrix of order $r$, and $O$ represents zero submatrices.

Rank and Normal Form: The size $r$ of the identity block $I_r$ corresponds exactly to the Rank of the matrix: $\text{rank}(A)=r$.

---

28.07.26 | Class-05

Linear Dependence of Rows

Linear Dependence and Independence

• A zero row (a row consisting entirely of zeros) is always linearly dependent.
• In a matrix reduced to Row-Echelon Form, every non-zero row is linearly independent.
• The number of independent rows in any matrix always equals the number of independent columns.

---

03.08.26 | Class-06

Systems of Linear Equations & Elimination Methods

Notation and Representation

A system of $m$ linear equations in $n$ variables can be written as: $Ax=b$ where:

• $A$ is the coefficient matrix of order $m\times n$.
• $x$ is the solution vector of size $n\times 1$.
• $b$ is the RHS constant vector of size $m\times 1$.

Classification of Systems of Equations

• Consistent System: A system that has at least one solution (either a unique solution or infinitely many solutions).
• Inconsistent System: A system that has no solution.

Theorem: Rank Criterion for Consistency

Let $[A|b]$ denote the augmented matrix of a system of equations.

1. The system is consistent if and only if: $\text{rank}(A)=\text{rank}([A|b])$
2. The system is inconsistent if and only if: $\text{rank}(A)\ne \text{rank}([A|b])$

---

Elimination Algorithms

Gaussian Elimination vs. Gauss-Jordan Elimination

1. Gaussian Elimination: Transforms the augmented matrix $[A|b]$ to Row-Echelon Form (REF), followed by back-substitution to find the values of the variables.
2. Gauss-Jordan Elimination: Transforms the augmented matrix $[A|b]$ all the way to Reduced Row-Echelon Form (RREF). The values of the variables are read off directly from the augmented column without back-substitution.

Matrix Inversion Algorithm

The inverse of a square matrix $A$ can be calculated by appending the identity matrix $I$ and performing EROs on the partitioned system until the identity matrix is obtained on the LHS: $[A|I]\sim [I|A^{-1}]$

---

04.08.26 | Class-07

Parameter Analysis in Systems of Equations

Underdetermined Systems & Free Variables

When a consistent system of equations has fewer independent equations (rank $r$) than variables ($n$), the remaining variables are designated as free variables.

Dimension Analysis Formula: $\text{Number of Free Variables}=\text{Number of Variables }(n)-\text{Rank }(r)$

---

06.08.26 | Class-08

Eigenvalues and Eigenvectors

Fundamental Definitions

Let $A$ be an $n\times n$ square matrix. A scalar $\lambda$ is called an Eigenvalue of $A$ if there exists a non-zero vector $x\ne 0$ (the corresponding Eigenvector) such that: $Ax=\lambda x$

Physical Interpretation: Multiplying vector $x$ by matrix $A$ scales the vector by factor $\lambda$ without changing its spatial direction.

Homogeneous Formulation: $\lambda x-Ax=0⟹(\lambda I-A)x=0$ Since $x\ne 0$ for a valid eigenvector, this homogeneous system must have non-trivial solutions.

Homogeneous Systems and Non-Trivial Solutions

A homogeneous system of linear equations $Ax=0$ is always consistent because the trivial solution ($x=0$) always exists.

• Trivial Solution: $x_1=x_2=\cdots =x_n=0$.
• Non-Trivial Solution: At least one variable is non-zero. A non-trivial solution exists if and only if $\text{rank}(A)<n$, which for square matrices is equivalent to: $\det (A)=0$

Therefore, the homogeneous eigenvalue system $(\lambda I-A)x=0$ has a non-trivial solution if and only if: $\det (\lambda I-A)=0\text{— characteristic equation}$ Expanding the determinant $\det (\lambda I-A)$ yields the characteristic polynomial.

---

Properties of Eigenvalues

Properties of Eigenvalues

For any $n\times n$ matrix $A$ with eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$:

1. Sum of Eigenvalues (Trace): ${\sum }_{i=1}^n{\lambda }_i=\text{tr}(A)=a_{11}+a_{22}+\cdots +a_{nn}$
2. Product of Eigenvalues (Determinant): ${\prod }_{i=1}^n{\lambda }_i=\det (A)$
3. Eigenvalues of the Inverse Matrix ($A^{-1}$): If $A$ is invertible ($\det (A)\ne 0$), then the eigenvalues of $A^{-1}$ are the reciprocals of the eigenvalues of $A$: $\frac{1}{{\lambda }_1},\frac{1}{{\lambda }_2},\dots ,\frac{1}{{\lambda }_n}$
4. Eigenvalues of a Scaled Matrix ($kA$): The eigenvalues of $kA$ (where $k$ is a scalar) are scaled proportionally: $k{\lambda }_1,k{\lambda }_2,\dots ,k{\lambda }_n$
5. Eigenvalues of Matrix Powers ($A^m$): For any integer exponent $m\ge 1$, the eigenvalues of $A^m$ are: ${\lambda }_1^m,{\lambda }_2^m,\dots ,{\lambda }_n^m$

---

Advanced Polynomial & Diagonalization Concepts

Minimal Polynomial Definition

The minimal polynomial $m(x)$ of an $n\times n$ matrix $A$ is the unique monic polynomial of lowest degree such that: $m(A)=O$

Key Properties:

• The minimal polynomial $m(x)$ divides any polynomial $p(x)$ for which $p(A)=O$. In particular, it divides the characteristic polynomial of $A$.
• The roots of the minimal polynomial $m(x)$ are exactly the distinct eigenvalues of $A$.

Diagonalization Concepts

A square matrix $A$ of order $n$ is diagonalizable if it is similar to a diagonal matrix $D$, i.e., there exists an invertible matrix $P$ such that $P^{-1}AP=D$.

• Modal Matrix ($P$): The matrix whose columns are the linearly independent eigenvectors of $A$.
• Spectral Matrix ($D$): The diagonal matrix whose diagonal elements are the eigenvalues of $A$ corresponding to the eigenvectors in $P$.
• Spectral Radius ($\rho (A)$): The maximum of the absolute values of the eigenvalues of $A$: $\rho (A)=\max \left\{|{\lambda }_1|,|{\lambda }_2|,\dots ,|{\lambda }_k|\right\}$

---

Geometric Interpretation of Eigenvalues

The linear transformation $Ax=\lambda x$ scales the eigenvector $x$ by a factor of $\lambda$. There are four geometric cases depending on the value of $\lambda$:

Case 1: Contraction ($0<\lambda <1$)

The vector scales down in length but maintains its direction.

       λx     x
o----->----->

$\lambda x<x$

Case 2: Dilation ($\lambda >1$)

The vector stretches in length while maintaining its direction.

       x      λx
o----->----->

$\lambda x>x$

Case 3: Negative Contraction ($-1<\lambda <0$)

The vector scales down in length and reverses its direction.

   λx     o     x
<-----o----->

$|\lambda x|<x$

Case 4: Negative Dilation ($\lambda <-1$)

The vector stretches in length and reverses its direction.

     λx          o     x
<----------o----->

$|\lambda x|>x$