8
$\begingroup$

I am looking for a good linear/matrix algebra textbook, suitable for self-study, that covers topics relevant to statistics and machine learning. I have access to Gentle's "Matrix Algebra", but have found it to be too dry and more of a reference book for a practitioner who's already studied the subject before.

Some of the books I'm considering are:

Seber, "A Matrix Handbook for Statisticians"
Searle, "Matrix Algebra Useful for Statistics"
Harville, "Matrix Algebra from a Statistician's Perspective"
Gruber, "Matrix Algebra for Linear Models" (just came out)

and the more generally oriented:

Strang, "An Introduction to Linear Algebra"
Strang, "Linear Algebra and Its Applications"
Axler, "Linear Algebra Done Right"

The trouble is, all these texts have excellent reviews on Amazon, but so did Gentle's text and it doesn't really suit my purposes.

Improve this question
$\endgroup$
6
  • 6
    $\begingroup$ If it's for self-study, and not teaching, then perhaps math.stackexchange.com is a more natural place to ask. $\endgroup$
    – Dag Oskar Madsen
    Oct 19, 2014 at 19:47
  • 5
    $\begingroup$ You might also consider stats.stackexchange.com or datascience.stackexchange.com. $\endgroup$
    – J W
    Oct 20, 2014 at 5:18
  • 1
    $\begingroup$ In applications to machine learning one often encounters calculus of matrix valued functions and functions of matrices. Your best bet may be to seek a book which integrates multivariable calculus and linear algebra; you will want to familiarize yourself with Lagrange multipliers and Taylor series of vector valued functions anyway. Also, there are some topics in linear algebra - like PCA and QR decompositions - which are best learned from a statistics / ML textbook once you have the proper foundations. $\endgroup$
    – Paul Siegel
    Oct 20, 2014 at 11:50
  • $\begingroup$ This is a really old post, but one thing I should say is that Harville and Searle will not have what you're looking for. They do cover the linear algebra needed for stats, but they do not discuss its context within stats. $\endgroup$
    – Clarinetist
    Oct 29, 2017 at 6:16
  • $\begingroup$ @DagOskarMadsen: While I agree (hence my suggestion to consider stats.stackexchange.com or datascience.stackexchange.com as well), answers could be useful for teachers of linear algebra with an eye to machine learning and statistics applications. $\endgroup$
    – J W
    Jan 9, 2019 at 21:25

3 Answers 3

Reset to default
2
$\begingroup$

I used: Axler, "Linear Algebra Done Right" to teach myself the material in university.

I recommend this book because it focuses on understanding the concepts rather than memorizing the proofs. It also has great exercises.

Good Luck!

Improve this answer
$\endgroup$
4
  • $\begingroup$ A linear algebra text that doesn't mention Cramer's rule? $\endgroup$
    – Gerald Edgar
    Jan 26, 2015 at 15:51
  • $\begingroup$ @GeraldEdgar The classic book Linear Algebra by Hoffman and Kunze does not cover Cramer's rule. It might not be common that a book doesn't include it, but it's not uncommon. $\endgroup$
    – Chris C
    Jan 27, 2015 at 2:27
  • $\begingroup$ @ChrisC Case in point: I never even heard of Cramer's rule during my entire undergraduate and postgraduate maths degree. $\endgroup$
    – DavidButlerUofA
    Feb 25, 2015 at 22:46
  • $\begingroup$ I've heard of it (as in used) Cramer's rule aplenty in high school. What I heard of it in CSE college and later... just don't use it; computationally it is very inefficient. $\endgroup$
    – the gods from engineering
    Feb 27, 2015 at 10:41
2
$\begingroup$

Stephen Boyd and Lieven Vandenberghe (also authors of a well-known book on convex optimization) have recently written an introductory linear algebra book, Introduction to Applied Linear Algebra: Vectors, Matrices and Least Squares that may suit your needs. From the preface:

This book is meant to provide an introduction to vectors, matrices, and least squares methods, basic topics in applied linear algebra. Our goal is to give the beginning student, with little or no prior exposure to linear algebra, a good grounding in the basic ideas, as well as an appreciation for how they are used in many applications, including data fitting, machine learning and artificial intelligence, tomography, navigation, image processing, finance, and automatic control systems.

Also from the preface:

The book covers less mathematics than a typical text on applied linear algebra. We use only one theoretical concept from linear algebra, linear independence, and only one computational tool, the QR factorization; our approach to most applications relies on only one method, least squares (or some extension). In this sense we aim for intellectual economy: With just a few basic mathematical ideas, concepts, and methods, we cover many applications. The mathematics we do present, however, is complete, in that we carefully justify every mathematical statement. In contrast to most introductory linear algebra texts, however, we describe many applications, including some that are typically considered advanced topics, like document classification, control, state estimation, and portfolio optimization.

The book is indeed replete with applications, many of which come from machine learning (or data mining) and from statistics in the guise of data fitting. It's very much an introduction though, as you'll need another source for learning topics such as vector spaces, subspaces, LU decomposition, eigenvalues, eigenvectors, the singular value decomposition, and various other topics. It's very much at the other end of the spectrum from Axler's book, but they do have something in common - little or no emphasis on determinants. In fact, the word "determinant" does not even appear in Boyd & Vandenberghe.

On balance, I think it's worth a look as a relatively gentle introduction with a wealth of applications.

Improve this answer
$\endgroup$
2
$\begingroup$

Gilbert Strang has a new book (published January 2, 2019!) that focuses on linear algebra's connection to machine learning.

G. Strang. Linear Algebra and Learning from Data SIAM, 2019.

I obviously haven't had time to review this book yet, but the TOC looks like it might suit your needs.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Good find! One important difference between Strang's new book and Boyd & Vandenberghe's new book is that the Strang appears to be positioned after a first linear algebra course whereas B&V starts at a basic level. $\endgroup$
    – J W
    Jan 10, 2019 at 6:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.