Canonical_Form

Canonical Form of a Matrix

linear-algebra

---

What is the Canonical Form?

The canonical form of a matrix is the simplest, most standardized version a matrix can be reduced to — while preserving its essential properties (like eigenvalues).

Think of it this way: two very different-looking matrices can secretly represent the same transformation. The canonical form strips away the cosmetic differences and reveals the underlying structure.

Formally: A matrix is reduced to canonical form via a similarity transformation, meaning we find an invertible matrix $P$ such that $J=P^{-1}AP$.

Why does this matter in Data Science?

Canonical forms (especially Jordan form) appear in:

• Financial forecasting (modeling compound growth over time)
• Population models (tracking how populations evolve)
• Epidemic spread models (like COVID-19 compartmental models)
• User behavior modeling (Markov chains, session analysis)

---

Eigenvalues and Eigenvectors

Definition

If $A$ is an $n\times n$ matrix, a nonzero vector $x\in {\mathbb{R}}^n$ is called an eigenvector of $A$ if multiplying it by $A$ just scales it — it doesn’t change direction:

$Ax=\lambda x$

• $\lambda$ (a scalar) is the eigenvalue
• $x$ (a nonzero vector) is the eigenvector corresponding to $\lambda$

Intuition: An eigenvector is a special direction that the matrix only stretches or compresses (by factor $\lambda$), never rotates.

---

The Characteristic Equation

To find eigenvalues, we solve the characteristic equation:

$\det (A-\lambda I)=0$

Where:

• $I$ is the identity matrix of the same size as $A$
• $\lambda$ is the unknown we solve for

This gives a polynomial in $\lambda$, whose roots are the eigenvalues.

---

Key Terms: Algebraic and Geometric Multiplicity

Before getting into Jordan form, you need to know two important terms:

Term	Meaning
Algebraic Multiplicity (AM)	How many times an eigenvalue appears as a root of the characteristic equation
Geometric Multiplicity (GM)	Number of linearly independent eigenvectors for that eigenvalue = dimension of the null space of $(A-\lambda I)$

Important rule: $1\le \text{GM}\le \text{AM}$ always.

• If GM = AM for all eigenvalues → matrix is diagonalizable (nicest case)
• If GM < AM for any eigenvalue → matrix is defective → needs Jordan blocks

---

Jordan Canonical Form

What is a Jordan Block?

A Jordan block of size $k$ for eigenvalue $\lambda$ looks like this:

$J_k(\lambda )=\left[\begin{array}{ccccc}\lambda & 1& 0& \cdots & 0\\ 0& \lambda & 1& \cdots & 0\\ ⋮& & \ddots & \ddots & ⋮\\ 0& \cdots & 0& \lambda & 1\\ 0& \cdots & 0& 0& \lambda \end{array}\right]$

Eigenvalue $\lambda$ on the diagonal, 1s on the superdiagonal, zeros everywhere else.

The full Jordan matrix $J$ is assembled by placing Jordan blocks along the diagonal.

---

Six Cases for a $3\times 3$ Matrix

These cover every possible Jordan structure for a $3\times 3$ matrix:

Case 1: Three distinct eigenvalues (AM: 1, 1, 1 — GM: 1, 1, 1)

Fully diagonalizable. $J=\left[\begin{array}{ccc}{\lambda }_1& 0& 0\\ 0& {\lambda }_2& 0\\ 0& 0& {\lambda }_3\end{array}\right]$

Case 2: One repeated eigenvalue (AM = 2, GM = 2) + one distinct

Still diagonalizable — two independent eigenvectors for the repeated one. $J=\left[\begin{array}{ccc}\lambda & 0& 0\\ 0& \lambda & 0\\ 0& 0& \mu \end{array}\right]$

Case 3: One repeated eigenvalue (AM = 2, GM = 1) + one distinct — defective

Only one eigenvector for the repeated eigenvalue → needs a Jordan block of size 2. $J=\left[\begin{array}{ccc}\lambda & 1& 0\\ 0& \lambda & 0\\ 0& 0& \mu \end{array}\right]$

Case 4: One eigenvalue only (AM = 3, GM = 3)

Fully diagonalizable — three independent eigenvectors. $J=\left[\begin{array}{ccc}\lambda & 0& 0\\ 0& \lambda & 0\\ 0& 0& \lambda \end{array}\right]$

Case 5: One eigenvalue only (AM = 3, GM = 2)

Two eigenvectors → one Jordan block of size 2 + one block of size 1. $J=\left[\begin{array}{ccc}\lambda & 1& 0\\ 0& \lambda & 0\\ 0& 0& \lambda \end{array}\right]$

Case 6: One eigenvalue only (AM = 3, GM = 1) — most defective

Only one eigenvector → one single Jordan block of size 3. $J=\left[\begin{array}{ccc}\lambda & 1& 0\\ 0& \lambda & 1\\ 0& 0& \lambda \end{array}\right]$

---

Step-by-Step Algorithm: Finding the Jordan Form

Step 1 — Find Eigenvalues

Solve the characteristic equation: $\det (A-\lambda I)=0$

Note each eigenvalue’s algebraic multiplicity (AM) from how many times it appears as a root.

Step 2 — Find Eigenvectors and Geometric Multiplicity

For each eigenvalue $\lambda$, solve the homogeneous system: $(A-\lambda I)x=0$

The GM = number of free variables = number of independent eigenvectors you get.

Step 3 — Determine Jordan Block Structure

For each eigenvalue, use AM and GM to decide the blocks:

• Number of Jordan blocks = GM
• Total size of all blocks = AM
• If GM = AM → all blocks are $1\times 1$ → diagonalizable
• If GM < AM → at least one block has size > 1

Step 4 — Find Generalized Eigenvectors (if AM > GM)

When a block has size > 1, you need generalized eigenvectors to complete the basis.

Build a Jordan chain by solving sequentially:

$\begin{array}{rl}(A-\lambda I)v_1& =0\leftarrow \text{ordinary eigenvector}\\ (A-\lambda I)v_2& =v_1\leftarrow \text{1st generalized eigenvector}\\ (A-\lambda I)v_3& =v_2\leftarrow \text{2nd generalized eigenvector}\end{array}$

Each equation in the chain uses the previous vector as the right-hand side.

Step 5 — Construct $P$ (the Change-of-Basis Matrix)

Arrange all eigenvectors and generalized eigenvectors column by column, in Jordan chain order:

$P=\left[\begin{array}{ccc}|& |& |\\ v_1& v_2& v_3\\ |& |& |\end{array}\right]$

Step 6 — Compute $J$

$J=P^{-1}AP$

The result is the Jordan canonical form.

---

Worked Example (Example 3.1)

Given: $A=\left[\begin{array}{ccc}0& 0& -2\\ 1& 2& 1\\ 1& 0& 3\end{array}\right]$

---

Part (i): Characteristic Equation

$A-\lambda I=\left[\begin{array}{ccc}-\lambda & 0& -2\\ 1& 2-\lambda & 1\\ 1& 0& 3-\lambda \end{array}\right]$

Expanding the determinant along the first row (only two nonzero entries: column 1 and column 3):

$\det (A-\lambda I)=(-\lambda )\left|\begin{array}{cc}2-\lambda & 1\\ 0& 3-\lambda \end{array}\right|+(-2)\left|\begin{array}{cc}1& 2-\lambda \\ 1& 0\end{array}\right|$

$=(-\lambda )(2-\lambda )(3-\lambda )+(-2)(0-(2-\lambda ))$

$=-\lambda (2-\lambda )(3-\lambda )+2(2-\lambda )$

$=(2-\lambda )[-\lambda (3-\lambda )+2]$

$=(2-\lambda )({\lambda }^2-3\lambda +2)$

$=(2-\lambda )(\lambda -1)(\lambda -2)$

So the characteristic equation is:

$\boxed{(2-\lambda )(\lambda -1)(\lambda -2)=0}$

---

Part (ii): Eigenvalues

From the characteristic equation:

$(2-\lambda )(\lambda -1)(\lambda -2)=0$

Note that $(2-\lambda )=-(\lambda -2)$, so this is $-(\lambda -2{)}^2(\lambda -1)=0$.

Eigenvalues: $\lambda =2$ (with AM = 2) and $\lambda =1$ (with AM = 1)

---

Part (iii): Jordan Canonical Form

For $\lambda =2$ (AM = 2):

$A-2I=\left[\begin{array}{ccc}-2& 0& -2\\ 1& 0& 1\\ 1& 0& 1\end{array}\right]$

Row reduce: all three rows reduce to the single equation $x_1+x_3=0$, i.e. $x_1=-x_3$.

Let $x_3=s$ and $x_2=t$ (both free):

$x=s\left(\begin{array}{c}1\\ 0\\ -1\end{array}\right)+t\left(\begin{array}{c}0\\ 1\\ 0\end{array}\right)$

Two free variables → two independent eigenvectors → GM = 2

Since AM = GM = 2, no Jordan chain needed for this eigenvalue.

---

For $\lambda =1$ (AM = 1):

$A-I=\left[\begin{array}{ccc}-1& 0& -2\\ 1& 1& 1\\ 1& 0& 2\end{array}\right]$

From row 1: $x_1=-2x_3$. Substituting into row 2: $-2x_3+x_2+x_3=0\Rightarrow x_2=x_3$.

Let $x_3=1$: $x=\left(\begin{array}{c}-2\\ 1\\ 1\end{array}\right)$

One eigenvector → GM = 1 = AM = 1. ✓

---

Conclusion:

Since GM = AM for both eigenvalues, the matrix is diagonalizable and the Jordan form is simply diagonal:

$\boxed{J=\left[\begin{array}{ccc}2& 0& 0\\ 0& 2& 0\\ 0& 0& 1\end{array}\right]}$

This is a Case 2 scenario: one repeated eigenvalue with full geometric multiplicity, plus one distinct eigenvalue.

---

Exercise 3.1

Given: $A=\left[\begin{array}{ccc}3& 1& 1\\ -1& 3& 0\\ 1& -1& 2\end{array}\right]$

(i) Write down the characteristic equation.

Answer: $(3-\lambda {)}^2(2-\lambda )=0$

(ii) Find the eigenvalues.

Answer: $\lambda =3$ (AM = 2), $\lambda =2$ (AM = 1)

(iii) Find the Jordan canonical form.

Answer: $J=\left[\begin{array}{ccc}3& 1& 0\\ 0& 3& 0\\ 0& 0& 2\end{array}\right]$

This is Case 3: for $\lambda =3$, AM = 2 but GM = 1 (defective), so a $2\times 2$ Jordan block appears. For $\lambda =2$, a $1\times 1$ block.

---

Quick Reference Summary

Situation	AM vs GM	Jordan Form Type
All eigenvalues distinct	AM = GM = 1 each	Diagonal
Repeated λ, enough eigenvectors	AM = GM	Diagonal
Repeated λ, too few eigenvectors	GM < AM	Jordan blocks (superdiagonal 1s)

The rule of thumb: Count eigenvectors. If you have enough (GM = AM for all), you get a diagonal matrix. If you’re short, Jordan blocks fill the gap.