What is the best way to intuitively explain what eigenvectors and eigenvalues are, AND their importance?

Question

How can we break down the complexity of eigenvalues/vectors to something that is more intuitive for students. I feel like the proofy way isn't a good intuitive representation of the mechanism that eigenvalues/vectors represent. What are the best reasons as to why a student might need to understand eigenvalues, and the tangible real world applications for eigenvalues, and eigenvectors?

Teaching this to all ages, high school, through college.

Can assume students have a foundation in calculus (differentiation ~ multivariable)

Answer 1

Here's an example I use for myself. I don't teach this topic in regular class but I have used this example in private conversations with advanced students.

Think of an object (perhaps a globe) that is stretched on one or more directions then rotated in various ways and perhaps reflected. We can show that at least one line through the object is either still pointing the same direction or pointing the opposite direction. The vector for this direction is an eigenvector. The amount of stretching in that direction is the eigenvalue for that eigenvector. If the direction is opposite the original direction, the Eigenvalue is negative.

This works since one-directional stretching, rotating, and reflecting are linear functions, and 3-dimensional space requires at least one real eigenvalue.

Answer 2

Although the following quantum-ish-physics-y "explanation" begs-the-question in several ways, it is genuine, and may convey something to students: given a linear operator (a.k.a. "matrix"), an eigenvector is "a pure state" (of what, we don't quite ask), meaning that the operator acts on it in an especially simple fashion. In good situations, a general "state" is a "superposition" (a.k.a. "linear combination") of pure states.

Although one might dismiss such half-explanations as too vague, the "hook" of "quantum ..." can instill belief that the thing is serious and "big-time".

Answer 3

I think a good motivation is the idea of dynamical systems and stability a la Markov chains. If we have a system which can be modelled by taking a vector of data $v(0)$ and then some matrix $A$ we have $v(t) = Av(t-1)$. Observe that such a system is in some sense stable, and will undergo consistent exponential growth if it is a eigenvector. Even more, one can actually data and see that often the maximal or minimal eigenvalue will dominate. This can be motivated by supply/demand/competition in a marketplace, discrete models of populations with predator prey populations, or many other things.

Answer 4

To be perfectly honest, the applications of eigenvalues tends to be much more complex than the actual theory of finding eigenvalues itself.

The first time I found a practical application of them was in Differential Equations, particularly solving the problems of

$$y' = Ax$$

whereas $A$ is a constant matrix $y$ is a vector function of $x$ which itself is a vector.

To some high school/very bright middle school students that have seen calculus this may be worth exploring but obviously this example isn't the best application for a wide-spread audience.

Another perhaps more primitive understanding is "eigenvalues" and "eigenvectors" are curious objects that satisfy the equation

$$Ax = \lambda x$$

which in itself is an interesting problem (for reasons mentioned above particularly @Rory Daulton and her stretching/rotating analogy)

And it just so happens these things can also be used to do a hell of a lot more for example factoring matrices, etc...

See here: http://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix#Practical_implications.5B1.5D

Answer 5

Familiar, everyday applications of eigenpairs abound:

for young musicians: the natural frequencies of a musical instrument string are eigenvalues (actually they're the squares, but, for teaching purposes...). The vibration modes are the eigenvectors. The lowest eigenvalue for the string is the fundamental frequency, the higher values are harmonics. When plucked, the string will vibrate at a superposition of the fundamental and several harmonics - if you want the fundamental to be dominant, pluck the string at the mid-point.

for destructive teenagers: the compressive force at which something like a plastic ruler or a thin sword-blade buckles is an eigenvalue, the shape into which it buckles is an eigenvector (usually only the lowest value is seen).

for undergraduate engineers: principal stresses are the eigenvalues of the stress state, principal directions are the eigenvectors.

Answer 6

As a video called "The 4 Ways to Tell if a Matrix is Diagonalizable [Passing Linear Algebra]" by the YouTube channel "STEM Support" mentions, Eigenvalues and Eigenvectors are useful to create Diagonalizable Matrices.

Diagonalizable Matrices make very resource-intensive (for computers) long matrix multiplications much, much simpler.

Let's say I want to change my camera angle in a video game. I need to apply a rotation matrix to my current camera angle (camera angle being interpreted as beams of light closer to the player having larger angles than from distant objects).

Great, I applied the rotation matrix. What's the issue? Well, for around 1,000,000 pixels (estimate for average 1,000x1,000 pixel laptop), I only generated one new image from multiplying that matrix by my current camera angle, a single frame in a game that should be operating around 60 Frames Per Second (fps).

Let's say that my rotation matrix isn't the standard rotation matrix (involves sine and cosine in a 2x2 or 3x3 matrix). Let's say that I have my game only using a "camera-angle-modifying" matrix that doesn't require computing resource-intensive sines and cosines for every possible combination of player camera angle movements.

Next Camera Angle = [Transformation Matrix][Current Camera Angle Matrix]

5 Camera Angles in the Future = [Transformation Matrix]⁵[Current Camera Angle Matrix]

Assuming my fixed-angle view-modifying transformation matrix is $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ and my current camera angle is $\begin{bmatrix}18 & 23\\ 39 & 42\end{bmatrix}$ , changing my camera angle only three times requires computing $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}^3$ $\begin{bmatrix} 18 & 23 \\ 39 & 42 \\ \end{bmatrix}$ , which is computationally the same as $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}18 & 23\\ 39 & 42\end{bmatrix}$.

$\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ = $\begin{bmatrix}1(1)+2(3) & 1(2)+2(4)\\ 3(1)+4(3) & 3(2)+4(4)\end{bmatrix}$ = $\begin{bmatrix}7 & 10\\ 15 & 22\end{bmatrix}$

$\begin{bmatrix}7 & 10\\ 15 & 22\end{bmatrix}$ $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ = $\begin{bmatrix}7(1)+10(3) & 7(2)+10(4)\\ 15(1)+22(3) & 15(2)+22(4)\end{bmatrix}$ = $\begin{bmatrix}37 & 54\\ 81 & 118\end{bmatrix}$

$\begin{bmatrix}37 & 54\\ 81 & 118\end{bmatrix}$ $\begin{bmatrix}18 & 23\\ 39 & 42\end{bmatrix}$ = $\begin{bmatrix}37(18)+54(39) & 37(23)+54(42)\\ 81(18)+118(39) & 81(23)+118(42)\end{bmatrix}$ = FinalCamAngle = $\begin{bmatrix}2772 & 3119\\ ... & ...\end{bmatrix}$

• $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}1 & 2\\ 3 & 4\end{bmatrix}$ $\begin{bmatrix}18 & 23\\ 39 & 42\end{bmatrix}$ = FinalCamAngle = $\begin{bmatrix}37 & 54\\ 81 & 118\end{bmatrix}$ $\begin{bmatrix}18 & 23\\ 39 & 42\end{bmatrix}$

Multiplying each of those matrices one at a time is an ugly and royal PiTA without any noticeable simplifying patterns, so let's try using a Diagonalized Transformation Matrix instead of any Transformation Matrix to apply camera angle rotations:

$\begin{bmatrix}2 & 0\\ 0 & 7\end{bmatrix}$ $\begin{bmatrix}2 & 0\\ 0 & 7\end{bmatrix}$ = $\begin{bmatrix}2(2)+0(0) & 2(0)+0(7)\\ 0(2)+7(0) & 0(0)+7(7)\end{bmatrix}$ = $\begin{bmatrix}2^2 & 0\\ 0 & 7^2\end{bmatrix}$

$\begin{bmatrix}2^2 & 0\\ 0 & 7^2\end{bmatrix}$ $\begin{bmatrix}2 & 0\\ 0 & 7\end{bmatrix}$ = $\begin{bmatrix}2^2(2)+0(0) & 2(0)+0(7)\\ 0(2)+7(0) & 0(0)+7^2(7)\end{bmatrix}$ = $\begin{bmatrix}2^3 & 0\\ 0 & 7^3\end{bmatrix}$

$\begin{bmatrix}2 & 0\\ 0 & 7\end{bmatrix}$ $\begin{bmatrix}2 & 0\\ 0 & 7\end{bmatrix}$ $\begin{bmatrix}2 & 0\\ 0 & 7\end{bmatrix}$ = $\begin{bmatrix}2^3 & 0\\ 0 & 7^3\end{bmatrix}$

• $\begin{bmatrix}2^3 & 0\\ 0 & 7^3\end{bmatrix}$ $\begin{bmatrix}18 & 23\\ 39 & 42\end{bmatrix}$ = FinalCamAngle = $\begin{bmatrix}2^3(18) & 2^3(23)\\ 7^3(39) & 7^3(42)\end{bmatrix}$

Which matrix is a prettier (and much easier to compute) transformation matrix?

$\begin{bmatrix}37 & 54\\ 81 & 118\end{bmatrix}$ or $\begin{bmatrix}2^3 & 0\\ 0 & 7^3\end{bmatrix}$

I'm pretty sure you said the Diagonalized Matrix.

That's why eigenvalues and eigenvectors are important. They allow the creation of diagonalizable matrices, which drastically simplify computational and time complexities.

Answer 7

Eigenvectors are vectors that map onto themselves. (Eigen= "oneself" in German.)

To accomplish this, they are defined by vectors of eigenvalues that "solve" the so-called "characteristic equation" (which defines them) for the value zero.

This equation is derived from: $A v = \lambda v$, where A is the "transformation matrix, v is the eigenvector, and $\lambda$ is a vector to be solved for. If you subtract $\lambda v$ from both sides, you get zero on the right hand side.

By factoring out the v, and through matrix operations on the left hand side, you get a polynomial equation that can be solved for $\lambda$.

Answer 8

As illustrated by other answers, Markov processes and linear ODE systems are the foremost applications. I like to mess a little with students explaining that the "only" thing that Google machines do all day is compute one large eigenvector over and over agin.

But to get a feeling for why are eigenvectors useful, think simply of base changes: Let's say there is no issue with multiplicity, so our transformation $A:\mathbb{R}^n \longrightarrow \mathbb{R}^n$ has $n$ distinct eigenvectors. Then the basis of eigenvectors is the most natural one since it is invariant (up to scale). How does an object transform under $A$? It stretches by eigenvalue #$1$ in the first direction, by eigenvalue #$2$ in the second direction, and so on. It is the convenience of this description that makes eigenstuff useful...

Answer 9

For students of all ages, I would go the “dynamical system road” with pictures (never tried it though, mostly by lack of opportunity).

Introduce a problem that is in fact a simple recursive vector-valued sequence (such as the evolution of a population with youngsters and olds, in which at each step a given proportion of youngster grows old while the other die, and every old gets a given number of young children and then die), and then show pictures of the possible evolutions given different starting points (you may want to choose another model to allow for negative values).

Typically, if you choose an hyperbolic $2\times 2$ matrix (i.e. two positive eigenvalues, one $<1$ and the other $>1$), you will have interesting pictures. Some very particular trajectories are going to zero or diverge linearly, with most trajectories starting close to the first type then close to the second type.

After showing pictures for different starting values, and then for different systems, you can easily introduce eigenvectors at the observed special directions, eigenvalues as rate of dilation/contraction, etc. It also gives a nice illustration of why decomposing a vector on a non-canonical basis is meaningful. Student will have a mental picture, which they usually lack even after many years of mathematical studies.

Answer 10

Many other answers mentioned ODE, but the discrete version seems simpler to introduce at an early stage. One could use system of sequences to show a magic trick and then explain it with eigenvectors, it might give both intuition and motivation. Let me be more specific.

Example problem

Suppose we have two sequences $(u_n)$ and $(v_n)$ such that for all $n\ge0$: $$\begin{cases} u_{n+1}=3 u_n+v_n \\ v_{n+1} = 2u_n+2 v_n\end{cases}$$ ( if needed insert any modelling motivation, such as evolution of a population of insects with young ones and adults, and adjust the parameters to make it somewhat realistic.)

How does the sequences $(u_n)$ and $(v_n)$ evolve? Can we give exact expressions for them?

Side note: before going on, at this point I would discuss what can easily be said without computing anything: if $u_0$ and $v_0$ are positive then the sequences are increasing and tend to $\infty$ for example.

Magic trick

Let us define for all $n\in\mathbb{N}$: $a_n=2 u_n+v_n$ and $b_n=u_n-v_n$. Then observe that $$a_{n+1} = 2 u_{n+1} + v_{n+1} = 8 u_n+4 v_n = 4 a_n$$ and similarly $b_{n+1}=b_n$. So, $a_n=4^n (2u_0+v_0)$ and $b_n=u_0-v_0$. The two recursions have been decoupled! Then by solving a system we get $$\begin{cases} u_n = 4^n(\frac23 u_0+\frac13 v_0) + \frac13(u_0-v_0)\\ v_n = 4^n(\frac23 u_0+\frac13 v_0) -\frac23(u_0-v_0) \end{cases}$$

We can now answer far more questions on the system easily (for which values of $u_0,v_0$ does $u_n$ go to infinity? Does $u_n$ grow much faster than $v_n$? etc. - one can ask this questions beforehand for more impact) But to students it should look like a magic trick: how could one think of introducing precisely $(a_n)$ and $(b_n)$ as above?

To the matter

Then one should be in a good shape to explain eigenvectors and eigenvalues, by rewriting the system as a vectorial sequence with a recursive property of the form $U_{n+1} = A U_n$ where $A$ is a matrix. The case of diagonal matrices is easy, the whole point of eigenvector being (at this point) to diagonalize a matrix. Alternatively, one can dissect the trick, and look at how one should choose the coefficients in $(a_n)$ and $(b_n)$ to make it work. Then eigenvectors and eigenvalues show up (but one has to introduce things in matrix form at some point, of course).

Answer 11

Say you have 3 cities A, B, C and for each of them you know the percentage of people each year remain in the city or move to one of the two other cities. Given the population distribution at time 0 how will people distribute in the long run?

Say you have 3 car rental X,Y,Z and for each of them you know the percentage of rented car that are returned in the same place or to one of the two others. Given the car distribution at time 0 how will car distribute in the long run?

What if in the first model I add the percentage of dying and newborns? An in the second one I add lost cars and new cars?

These are simple real life examples (where of course the 3 can be easily changed into N) where eiganvalues and eigenvectors come into play into a Markov chain form, as commented before. The question you consider is easily motivated (I'm running the car rental and I want to know if in 2-3 years I will have to pay someone to bring back cars to their original disposition or not) and the answer easily understood.

Answer 12

See these two posts: eigshow: week 1, eigshow: week 2 written by C. Moler about illustrating eigenvalue in MATLAB software.

Answer 13

Disclaimer: My answer only attempts to clarify what they are, not their importance.

Eigenvectors are the vectors that do not change aside from being scaled (do not "rotate") when some linear transformation (matrix) is applied to them. The transformed vector will be on the same line as the original vector. So, such a vector is the eigenvector of that matrix. The factor by which they are scaled is known as the eigenvalue. If the direction of the vector is reversed due to the linear transformation, the eigenvalue is negative. Similarly, if the vector remains the exact same, the eigenvalue is 1.

This answer is an attempt to give a slightly more specific explanation, building up on @RoryDaulton 's answer.

(Also, I recommend watching the Essence of Linear Algebra series by 3Blue1Brown.)

Answer 14

I would take a look at the treatment in Kreyszig Advanced Engineering Mathematics. Or the treatment in Speigel Applied Differential Equations. In both cases, they bring the topic in, in the context of pretty stripped down and engineer-oriented overviews of linear algebra. So presumably they are not "proofy".

The Kreysig treatment is after the calc 3 topic. (Also technically after ODES, but not dependent on it.) The Speigel treatment is more diffyq oriented since it is a diffyQ text.

In both cases they emphasize resonant frequencies as applications. Which seems common sense to me. I wouldn't bring in Markov process since this is higher math for most college sophomores/junior STEMs. Even if I don't have a background in mechanics or spectral analysis, the idea of a resonant frequency (famous bridge shaking down) is pretty darned intuitive.

I would also think the analogy to quantum energy levels in chemistry is helpful. It's a course that most people have had. You don't even need to know the whole Schroedinger development. But it's well taught that the electrons can only have certain energies, only certain frequencies of emitted light, etc. (People know this without following the math for it, basically trusting until p-chem or quantum physics that it will be justified.) So, the eigenvalues are like the allowable energies.

P.s. I confess to never having a LA course. Or really knowing what Igon values are. See the famous Pinker takedown of Malcolm Gladwell. https://www.nytimes.com/2009/11/15/books/review/Pinker-t.html So, I could be getting it wrong. I'm probably right in the target zone of someone who ought to be able to grok and intuitive explanation. Not a proof, mind you. But a what sort of animal is this, explanation.

P.s.s I think it makes more sense to introduce this topic after or within first LA or ODE course. (I saw it first in "engine math", which was special topics advanced calc smorgasbord for engineers and physicists post ODEs, so ~fall of junior year in standard US college.) This is well beyond most high schoolers. Essentially bringing it in where it is, rather than trying to push it earlier into the curriculum. We can't do everything and have to have an order. So I think leaving this stuff for later is fine. (I realize this is a comment on "if you should", rather than "how" that you are asking.)