9
$\begingroup$

I'm teaching a linear algebra class and I'm considering presenting eigenvectors and eigenvalues without using determinants, as in Axler's book Linear Algebra Done Right. (See also Axler's paper "Down with determinants!".)

However, I need students to be able to compute eigenvalues and eigenvectors by hand on homework and quizzes. I am considering having them do it like this: To find the eigenvalues of a matrix $A$, we need to find the scalars $\lambda$ for which $A - \lambda I$ has a nonzero null vector. We can find those values of $\lambda$ just by doing row reduction on $A - \lambda I$.

Does anyone foresee any problems with this approach? Is there a better way to handle this?

Edit: The course I'm teaching is a 5 week review of linear algebra "boot camp" for Master's of Science in Data Science students at the University of San Francisco. The textbook is Introduction to Linear Algebra by Strang. The course is not very abstract -- we work mainly in $\mathbb R^n$ and do a lot of computations by hand -- but the students tend to be strong. The course could be described as an accelerated, 5 week review of a sophomore level undergraduate introduction to linear algebra course, covering topics such as least squares, eigenvalues and eigenvectors, and the SVD (and PCA). The students come from various technical backgrounds, but most have taken such a sophomore level course (at least) while they were in college.

I am also interested in the idea of taking this approach to computing eigenvalues by hand (avoiding determinants) in a sophomore level undergrad introduction to linear algebra course.

$\endgroup$
5
  • 4
    $\begingroup$ You need to explain the background of students, learning outcomes of the course, and how the course fits into the curriculum of your institution. My first reaction is that if you have decided to not use determinants to calculate eigenvalues for pedagogical/philosophical reasons, then why not do this right and use QR factorization? This is, after all, the most important algorithm for eigenvalue computations. $\endgroup$
    – user52817
    Jul 25 at 15:54
  • 1
    $\begingroup$ Why avoid determinants ? I guess I love wedge products too much. $\endgroup$ Jul 26 at 3:23
  • 2
    $\begingroup$ @JamesS.Cook Axler discusses this in his article "Down with determinants!" It's true that determinants are an important topic for parts of higher math. But, some might argue that introducing determinants properly at this point in a linear algebra class is a diversion from the main thread of ideas, and moreover it is arguably not the most illuminating or direct way to understand eigenvalues or eigenvectors. $\endgroup$
    – littleO
    Jul 26 at 3:51
  • $\begingroup$ @littleO I respectfully disagree. Determinants are a very simple way to calculate eigenvalues. There not really abstract or difficult and understanding them is basic to a proper appreciation of vector math in calculus III. It's not higher math. $\endgroup$ Jul 26 at 5:57
  • 2
    $\begingroup$ @littleO: Thanks for the clarifications. That's very helpful. $\endgroup$
    – user507
    Jul 26 at 12:53
8
$\begingroup$

I sympathize with Axler's "Down with Determinants!" because when I was originally taught determinants the definition was offered without any motivation. Later, in an upper division course, I learned that a determinant could be interpreted as the product of the eigenvalues, and it was only at that point that I felt that I understood what the determinant was. This is the definition that Axler proposes using.

There is nothing wrong with this approach in principle, but it only makes sense when you're doing linear algebra over the complex numbers. That means that it's incompatible with your chosen text, since Strang does determinants fairly early and gets into complex numbers only at the very end of the book. I don't think it's a good idea to depart so radically from the sequence of topics in your chosen book. You describe two courses, a standard sophomore review class and a 5-week rapid review for students who have already had that class. For the standard intro class, I would predict massive failure. Most of your students will be engineering majors, and although you're at a school with highly selective admissions, I think you'd find many students really struggling to figure out what was going on if you departed this much from the presentation in the text. For the review class, you might be able to get away with more, because they've already seen the subject, but the rapid pace of a 5-week class also leaves less room for error.

Axler's main complaint, echoed by my experience, was the lack of proper motivation for the determinant. I looked at the motivation for this topic in the fourth edition of Strang, and IMO it is indeed pretty bad. He introduces the determinant casually in problem set 2.3, #8, with essentially no motivation (or none that a real-world student would be likely to pick up on). Then he never mentions determinants again until ch. 5, where he abruptly starts talking about them in a tone that suggests that students already know all about them.

My suggestion would be either to use a better book, or just to use your lectures to patch up these defects in Strang. As an alternative book, Hefferon is really nice, it's free, and the motivation for the determinant seems much more acceptable to me than what Strang does.

If I was constrained to teach from Strang or wanted to use it despite the bad presentation of the determinant, then I think my "patch" presentation would go something like the following.

We've seen that a matrix can be interpreted as a kind of transformation, a function that takes a point in R^n to some other point in R^n. As with many things in life, these operations can be undoable or not undoable. If you make a typo in your English paper, you use the undo function in your word processor. But if you launch the nuclear missiles, there is no undo.

Unfortunately, it's not always obvious from looking at a matrix whether it's invertible or not. This is mainly because a matrix can be expressed in more than one basis. For example, diag(1,0) is obviously not undoable -- geometrically, it's a projection onto the x axis. But if we rotate our basis vectors by 45 degrees, we get this matrix ..., and in this form it's not at all obvious that the matrix isn't invertible.

A nice way to see what's going on here is to take a unit square in the Cartesian plane and see what happens to it when we apply the matrix. Diag(1,0) takes this square and squashes it flat, making its area become zero. Clearly if you eliminate the area of a geometrical figure, you're losing some information, so that's not an invertible operation. Every linear transformation in R^2 has some number associated with it, which is the factor by which it changes areas. That number is called the determinant. Here are some other examples, interpreted geometrically. (Show a rotation and diag(1,2) as examples.) You can see that for a diagonal 2x2 matrix with positive elements, the determinant is the product of the diagonal elements.

Our definitions will all actually come out simpler if we reinterpret "area" to be a signed quantity. You've already seen this kind of redefinition in high school geometry and trig, where at some point you learn about negative angles. With this redefinition, the determinant of a diagonal 2x2 matrix is always the product of its diagonal elements. When the determinant is negative, we're mirror-imaging things.

Two sweet things about this definition: (1) It's independent of what basis we choose. (2) It's actually not hard to generalize to R^n. In R^1 we would just change "area" to "length", in R^3 we'd use volume, and so on.

$\endgroup$
3
  • 4
    $\begingroup$ +1. In a class where most students are engineering majors: they will be assumed to know about determinants in the future. In fact, even math majors will be assumed to know about determinants in the future. $\endgroup$ Jul 26 at 16:22
  • $\begingroup$ @GeraldEdgar: Yeah, even Axler doesn't propose eliminating it from the curriculum. In a service course, you really can't skip major topics. $\endgroup$
    – user507
    Jul 26 at 17:15
  • 4
    $\begingroup$ We should also note that Axler bills his textbook (which I like!) as being for a second course in linear algebra. In the first course, the student likely learned about determinants, so there is no need to treat determinants as forbidden knowledge. $\endgroup$
    – user52817
    Jul 26 at 18:14
6
$\begingroup$

I am going to just play here. Take

$$ A = \begin{bmatrix} 1 & 2 \\ 3 & -4 \end{bmatrix} $$

The "Linear Algebra Done Right" idea for establishing existence of eigenvectors is that an eigenvector is invariant after multiplication by $A$ (up to scaling). So it makes sense to iterate $A$ in our hunt for an eigenvector.

The list of vectors $\vec{e}_1, A(\vec{e_1}), A^2(\vec{e_1})$ must be linearly dependent since it is a collection of 3 vectors in $\mathbb{R}^2$.

$$ \textrm{RREF}\left( \begin{bmatrix} 1 & 1 & 7\\ 0 & 3 & -9\end{bmatrix} \right) = \begin{bmatrix} 1 & 0 & 10\\ 0 & 1 & -3\end{bmatrix} $$

Thus we have

$$A^2(\vec{e}_1) = 10\vec{e_1}+-3A(\vec{e}_1)$$

Bringing all the terms to one side we have

$$ (A^2+3A-10I)\vec{e}_1 = \vec{0} $$

This did give us the characteristic polynomial without determinants! It will not always (such as in the case your first guess actually was an eigenvector). I am not sure about the relationship in general, aside the the fact that the polynomial you get must be a factor of the characteristic polynomial.

Factoring we have

$$ (A+5I)(A-2I)\vec{e}_1 = \vec{0} $$

So either $\vec{e_1}$ is an eigenvector with eigenvalue $2$ (which it is clearly not) or $(A-2I)\vec{e}_1$ is an eigenvector with eigenvalue $-5$.

So we have determined that $(A-2I)(\vec{e_1}) = \begin{bmatrix} -1 \\ 3\end{bmatrix}$ must be an eigenvector with eigenvalue $5$, which we can easily check.

This does give us one eigenvector/eigenvalue pair (as it necessarily must), but I think it is no coincidence that we got the eigenvector with greatest magnitude. Iterating this way would tend to select for this eigenvector. I am not sure if there is a useful way to extract the other eigenvector for $A$ this way.

Sorry if this was not much help!

EDIT: Playing again. What if we factor in the other order?

$$ (A-2I)(A+5I)\vec{e}_1 = \vec{0} $$

Then we are saying either $\vec{e_1}$ is an eigenvector with eigenvalue $-5$ (which it clearly is not), or $(A+5I)(\vec{e}_1)$ is an eigenvector with eigenvalue $2$.

So we have in fact determined that $\begin{bmatrix} 6 \\ 3\end{bmatrix}$ is an eigenvector with eigenvalue 2!

This gives us our basis of eigenvectors.

This actually seems pretty satisfying. I would like to play more with this (especially in the case of higher dimensional eigenspaces, higher dimensions, etc). Maybe this method is "workable" enough for most purposes of an intro linear algebra class. I also really like that it combines finding the eigenvalue/eigenvector pair into one piece of reasoning, instead of first finding the eigenvalues and then finding the corresponding eigenvector.

$\endgroup$
5
  • 3
    $\begingroup$ The iteration idea is also nice because you can point out that in most cases (in some sense with probability 1), iterating will converge exponentially to the dominant eigenvector. I think it's helpful for students to see that there's more than one way to do things, so they develop beyond the level where they think "eigenvector" is just a coded command to perform this algorithm with a lambda and a determinant in it. $\endgroup$
    – user507
    Jul 26 at 17:19
  • 1
    $\begingroup$ Nice answer. I remember when I took linear algebra years (decades) ago, I learned about the Jordan canonical form. This just stayed in my head as a curiosity/dead end. Years later, in a talk about Google PageRank, I learned how the Jordan form makes it obvious that iteration of a vector converges to the dominant eigenvalue, and also delivers the largest eigenvalue. I thought incorrectly, for years, that the Jordan form was kind of useless. The course no doubt used Cayley-Hamilton and determinants to stage the presentation of Jordan form. $\endgroup$
    – user52817
    Jul 26 at 22:25
  • 1
    $\begingroup$ @user52817 I was also somewhat baffled by the point of Jordan canonical form until I saw the connection with dynamical systems and differential equations. $\endgroup$ Jul 27 at 12:38
  • 1
    $\begingroup$ Great answer! If you wish to see more about the relationship in general, look up Krylov subspaces and breakdown in the Arnoldi iteration. $\endgroup$ Aug 20 at 21:20
  • $\begingroup$ @FedericoPoloni Excellent! Thanks for giving me those keywords. I knew this stuff must have been studied a long time ago (it is too natural and too basic to have been novel). $\endgroup$ Aug 21 at 1:02
4
$\begingroup$

Students tend to remember what they do much better than what they're told. If this is a student's first introduction to linear algebra, what's more important for them to remember about eigenvalues?

I'm personally fond of setting up the equation $A\pmatrix{x\\y} = \lambda \pmatrix{x\\y}$ and solving the system of equations itself (using row reduction techniques but not writing it as a matrix). This got me really far in some physics classes where we needed to solve for eigenvalues and the corresponding eigenvectors. This approach helps reinforce the definition of eigenvector/value and the relationship between row reduction and solving systems of equations, but you risk students not remembering the existence of the $\det(A-\lambda I)=0$ equation if they don't use it.

Building on user52817's answer, we can solve $$\pmatrix{3&1\\-4&1}\pmatrix{x\\y} = \lambda \pmatrix{x\\y}$$ By setting up the equations $$\begin{eqnarray}3x+y=\lambda x\\ -4x+y = \lambda y\end{eqnarray}$$ Bring everything to one side: $$\begin{eqnarray}(3-\lambda)x+y=0\\ -4x+(1-\lambda)y = 0\end{eqnarray}$$ Multiply the top by $-4$ and the bottom by $(3-\lambda)$: $$\begin{eqnarray}-4(3-\lambda)x-4y=0\\ -4(3-\lambda)x+(3-\lambda)(1-\lambda)y = 0\end{eqnarray}$$ Subtract top from bottom: $$((3-\lambda)(1-\lambda)-4)y=0$$ So $y=0$ (an important case, leading to the zero eigensystem) or $\lambda^2-4\lambda+7=0$ You can stop here if you just want to solve for $\lambda$ or go one step further and plug in to solve for the relationship between $x$ and $y$. This system is underconstrained, reinforcing the idea that you really have an eigenspace, and any multiple of an eigenvector is an eigenvector.

This seems to be a modification of what you're doing. I encourage you to give it a try! The only risk is that your students might not remember the existence of the $\det(A-\lambda I)=0$ trick.

Having students solve $\det(A-\lambda I)=0$ reinforces the skills for calculating determinants, and your students are likely to remember the existence of this formula. But you risk students remembering this equation next semester as just a computation trick, and not hanging on to the theory behind why it works (since this isn't used in applying the formula). The formula does produce a nice aha moment when you do remember why it works, though.

$\endgroup$
1
  • 3
    $\begingroup$ I like this approach since it reinforces the definition. I also like the opportunity to point out that this system of equations is nonlinear, since it involves a product of the variables. It is, however, linear in each variable separately. The determinant is also a multilinear gadget, and this points the way to a connection. $\endgroup$ Jul 27 at 12:32
3
$\begingroup$

Presumably, on a homework or quiz, the matrices involved with be 2x2. Let's look at the bookkeeping labor involved for this example:

$$A=\pmatrix{\ \ 3&1\\ -4&1}$$

We need to row reduce $$\pmatrix{3-\lambda&1\\ -4& 1-\lambda}$$

The main step is $R_2\to \frac{4}{3-\lambda}R_1+R_2$ (Side note, remember to investigate $\lambda=3$)

This results in $$\pmatrix{3-\lambda& 1\\ 0&\frac{4+(3-\lambda)(1-\lambda)}{3-\lambda}}=\pmatrix{3-\lambda& 1\\ 0&\frac{\lambda^2-4\lambda+7}{3-\lambda}}$$

So the condition we arrive at is that $\lambda^2-4\lambda+7=0$.

Think about the bookkeeping overhead required, only to arrive at the same quadratic equation which could have been found much quicker with $\det(A-\lambda I)=0$.

On a quiz, is it realistic to ask students to go through this method of row reduction which has more room for clerical errors, just to find the eigenvalues of a modest 2x2 matrix? Why not point out that in end, we are going to get a quadratic which can be more easily arrived at by using a determinant?

If the choice to avoid all mention of determinants is made, then I would either not ask students to find eigenvalues without software, or I would make the pedagogical choice to teach QR, which is among the most important algorithms in numerical linear algebra.

$\endgroup$
4
  • 2
    $\begingroup$ I think it's premature to offer this type of answer when the OP hasn't yet told us anything about the type of course or the student population. And recommending that they teach the QR algorithm doesn't make sense. This is not a numerical analysis class. Numerical analysis is a different subject. It includes things that a linear algebra class doesn't include, and it doesn't include things that a linear algebra class does include. $\endgroup$
    – user507
    Jul 25 at 18:35
  • 4
    $\begingroup$ I agree about "more room for clerical errors", and I'll add that those clerical errors (as well as more substantial errors) can make grading a nightmare. $\endgroup$ Jul 25 at 18:59
  • 1
    $\begingroup$ @BenCrowell: I am not so sure that it does not make sense to teach QR in a linear algebra course. Here is a course outline for somebody who uses Axler's textbook "Linear Algebra Done Right." Look at the March 18 and March 23 topics. math.temple.edu/~szyld/3051.html $\endgroup$
    – user52817
    Jul 25 at 19:18
  • $\begingroup$ @user52817 The QR decomposition and the QR (Francis) iteration for eigenvalues are two different things, and it seems that you are mixing them up. Szyld teaches the QR decomposition, while if you want to compute algorithm you need the QR/Francis iteration, which is way more complicated and technical. $\endgroup$ Aug 20 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.