Eigenvalues & Eigenvectors

The directions that refuse to turn

A matrix $A$ is a machine that eats a vector and spits out another. For a typical input $v$, the output $Av$ points somewhere new. But ask: are there directions $v$ that $A$ leaves pointing the same way, only longer or shorter? A direction $v$ where the output is just a scaled copy of the input:

$A v = \lambda v, \qquad v \ne 0.$

When this holds, $v$ is an eigenvector of $A$ and the scalar $\lambda$ is its eigenvalue ("eigen" is German for "own" — these are the matrix's own directions). The condition $v \ne 0$ is non-negotiable: $A \cdot 0 = \lambda \cdot 0$ holds trivially for every $\lambda$, so the zero vector would tell us nothing. Eigenvectors are nonzero by decree.

Geometrically, eigenvectors are the axes the transformation is built around. Drag a direction and watch where $A$ sends it — the eigendirections are the ones where the output arrow stays collinear with the input, glowing when you hit them:

These special directions are the map's skeleton — its actual identity, buried under all that matrix arithmetic. Everything else $A$ does is just a blend of stretching along them.

Finding the eigenvalues: $\det(A - \lambda I) = 0$

How do we hunt down the $\lambda$? Rearrange the defining equation. $Av = \lambda v$ becomes $Av - \lambda v = 0$, and writing $\lambda v = \lambda I v$ (so we can factor a matrix out):

$(A - \lambda I)\,v = 0.$

Now think about what this demands. We need a nonzero $v$ killed by the matrix $A - \lambda I$. A matrix has a nonzero vector in its kernel exactly when it is not invertible — when it squashes some direction to zero. And from Determinants, a matrix is non-invertible precisely when its determinant is zero. So:

$Av = \lambda v \text{ for some } v \ne 0 \iff (A - \lambda I) \text{ is singular} \iff \det(A - \lambda I) = 0.$

That's the whole derivation, and I'll be damned if it isn't beautiful: the determinant is a squash-detector, and an eigenvalue is exactly a $\lambda$ that makes $A - \lambda I$ squash. The expression

$p(\lambda) = \det(A - \lambda I)$

is the characteristic polynomial. For a $2\times 2$ matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$,

$p(\lambda) = \det\begin{pmatrix} a - \lambda & b \\ c & d - \lambda \end{pmatrix} = (a-\lambda)(d-\lambda) - bc = \lambda^2 - (a+d)\lambda + (ad - bc).$

Notice the coefficients are old friends: $a + d = \operatorname{trace}(A)$ and $ad - bc = \det(A)$. So $p(\lambda) = \lambda^2 - (\operatorname{trace})\lambda + (\det)$. Set it to zero, solve the quadratic, and out fall the eigenvalues.

Finding the eigenvectors: the null space of $A - \lambda I$

Once you have an eigenvalue $\lambda$, plug it back in and find the nonzero $v$ with $(A - \lambda I)v = 0$. That's just finding the kernel (null space) of the matrix $A - \lambda I$ — a homogeneous system you already know how to solve from Gaussian Elimination.

The full set of solutions, $\ker(A - \lambda I)$, is called the eigenspace of $\lambda$. It's a subspace (kernels always are — proved back in Linear Maps), so it's closed under scaling and addition: if $v$ is an eigenvector, so is $5v$ and so is $-v$. Eigenvectors come in entire lines (or planes), never as lone points. By convention we report one clean representative — pick any nonzero specimen from the eigenspace and you're done.

When there are no real eigenvectors: rotation

Here's the twist that exposes what eigenvalues really mean, and it's the kind of thing that should shake you. Take the $90^\circ$ rotation

$R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}.$

Its characteristic polynomial is $\lambda^2 - (0)\lambda + (1) = \lambda^2 + 1$. Set it to zero: $\lambda^2 = -1$. No real solutions. And of course not — geometrically, a $90^\circ$ rotation turns every direction off its line. There is no direction it merely stretches, because it rotates everything. The algebra (no real roots) and the geometry (nothing stays put) are saying the identical thing. This is why "no real eigenvalues" isn't a failure — it's a rotation confessing its nature. (Over the complex numbers $\lambda = \pm i$ appear, which is exactly how rotation hides inside the imaginary unit, but real eigenvectors genuinely don't exist here.)

Distinct eigenvalues force independence

One structural fact we'll lean on hard in the next two lessons. Eigenvectors belonging to distinct eigenvalues are linearly independent. Let me prove it for two.

Suppose $Av_1 = \lambda_1 v_1$ and $Av_2 = \lambda_2 v_2$ with $\lambda_1 \ne \lambda_2$, both eigenvectors nonzero. Assume a dependence:

c_1 v_1 + c_2 v_2 = 0. \tag{1}

Apply $A$ to both sides: $c_1 \lambda_1 v_1 + c_2 \lambda_2 v_2 = 0$. Now also multiply (1) by $\lambda_1$: $c_1 \lambda_1 v_1 + c_2 \lambda_1 v_2 = 0$. Subtract:

$c_2(\lambda_2 - \lambda_1) v_2 = 0.$

Since $\lambda_2 - \lambda_1 \ne 0$ and $v_2 \ne 0$, we're forced to $c_2 = 0$. Plug back into (1): $c_1 v_1 = 0$, and $v_1 \ne 0$ forces $c_1 = 0$ too. The only dependence is the trivial one — they're independent. $\blacksquare$ (The general statement: eigenvectors from $k$ distinct eigenvalues are always independent; same trick, induction.)

This is the seed of the next lesson. Distinct eigenvalues hand you a basis of eigenvectors for free, and a basis of eigenvectors is the magic grid in which the matrix goes diagonal. File it away — it fucking detonates immediately. Go do the gauntlet, you beautiful disaster.