# Concrete vectors spaces without an obvious basis or many "obvious" bases?

I am teaching a class on linear algebra to sophomore and junior science majors, and am having some trouble illustrating the difference between $$\mathbb{R}^n$$ and an n-dimensional vector space. The main example the textbook uses is the set of polynomials with degree $$n-1$$ or less, but the students see this as essentially the same as $$\mathbb{R}^n$$ because the canonical polynomial basis $$1, t, t^2,$$ etc is so immediate. A couple were convinced that $$1, (t-1), (t-1)^2$$ etc . would work too, but most thought that this was just as unnatural as using anything but the standard basis in $$\mathbb{R}^n$$. I told them that it will be important to be able to change bases even in $$\mathbb{R}^n$$ later, but I would like to do more for them now.

The set of colors is almost a vector space, and they were convinced more-or-less that for this to make sense, you'd have to agree ahead of time if you were going to use RYB or CMY and that this amounts to choosing a basis. (Apologies if there is a slight abuse here, the argument I gave them was mostly by analogy.)

Can you think of other examples where the basis of the (finite dimensional, please!) vector space is not apparent, or many bases appear without one being obviously simplest/best?

• What about a plane through the origin as a subspace of $\mathbb{R}^3$? Finding a basis there is a pretty common exercise, but the answer may not be obvious to them (depending on how the plane is defined). Mar 8, 2022 at 20:50
• As an aside, I'm afraid your RGB/CMY analogy is incorrect. Not only is CMY(K) not even "almost" linear (scalar multiplication of ink would yield the same color, for instance), it isn't even the same range of colors as RGB. Yeah, there are colors of light that cannot be expressed in ink, and the conversion between the two formats is Lovecraftian in its complexity. I'd stick with the conversion between cartesian and polar/cylindrical/spherical coordinates as the model for non-trivial changes of base. Mar 8, 2022 at 21:25
• I wonder if there's a good crystallography example of wanting a base that has axes that are not orthogonal. Mar 8, 2022 at 22:51
• YPbPr is gotten by applying a matrix to R'G'B': en.wikipedia.org/wiki/YCbCr#R'G'B'_to_Y'PbPr Mar 9, 2022 at 3:43
• @Stef: Ehm, it’s “the art of giving the same name to different things”. Mar 9, 2022 at 11:53

Some physical examples from physics:

Consider two spaceships that meet each other in deep space with arbitrary orientations (pitch, roll, and yaw). Even if they take the origin to be the midpoint between them, each spaceship will having their own coordinate system that is rotated with respect to the other. Each can claim to be using the standard $$\mathbb{R}^3$$ basis for their own ship, but they will both describe positions in the other ship as if they were using a different basis. There's no natural basis that both ships will choose.

Another physical example involves the following mass and springs system (link to source): This system has two degrees of freedom and can be represented by a 2D vector space. An intuitive one would be to specify the displacement of both blocks from their equilibrium positions, yielding $$(x_1(t), x_2(t))$$. However, solving the motion of the blocks in this coordinate system is very complicated with their motion being coupled. Using $$x_1 + x_2$$ and $$x_1 - x_2$$ as a basis decouples the equations and makes them much easier to solve (the eigenvalue analysis is what finds this basis).

Transforming the coordinate system using the eigenvector basis reveals that this system has two characteristic ("eigen") motions:

1. The boxes always move in the same direction back and forth in sync with each other.
2. The boxes always move in opposite each other, first towards each other, then away from each other.

All other possible motions are linear combinations of these eigenvectors. Thus, the space of motions can be represented by a 2D vector space. There are two natural bases: the physical position basis of $$(x_1(t), x_2(t))$$, and the eigenmode basis. Transforming to the eigenmode basis is 90% of the solution.

Before seeing the problem worked out, I don't think anyone would naturally see a vector space in the motion of these masses. Here's a video that has demonstrations with two pendulums linked by springs (similar math) and a system of three masses in a similar configuration.

If your students are not ready for calculus or differential equations, you can skip the derivation and present the eigenmodes and the linear combinations of motions--especially if you can demonstrate it physically or with animations.

A simpler physics example: consider a box sliding down an incline. The natural basis of all motion problems is to put the x-axis parallel to the flat ground and the y-axis parallel to gravity. A basis that makes the math easier to to align the x-axis with the incline. After doing some trigonometry to get the force of gravity along the incline, the one-dimensional motion only requires one coordinate to vary.

• The coupled springs system is the very example that convinced me 15 years ago that there really is something to all this linear algebra and basis changing. Thank you for posting! Mar 9, 2022 at 11:11
• No need to consider deep space. A GPS navigator can change views between the canonical bidimensional basis and the relative tangent-normal basis. Mar 9, 2022 at 17:27
• @Miguel I used deep space in order to eliminate any natural coordinate systems. Two spaceships are on equal footing. A spaceship and a planet are not, at least in learners' minds. Mar 9, 2022 at 22:10

Two more examples:

1. The set of infinite Fibonacci-type sequences (those of the form $$a_n=a_{n-1} + a_{n-2}$$) (with point-wise addition and scaling) forms a 2-dimensional (real) vector space. E.g., elements like:

$$a_n = \left(2,8,10,18,28,46,...\right)$$

$$b_n = \left(-3,1,-2,-1,-3,-4,...\right)$$

where one basis for the space is the set:

$$e_1 = \left(1,\frac{1-\sqrt{5}}{2},\left(\frac{1-\sqrt{5}}{2}\right)^2,...\right)$$ and $$e_2 = \left(1,\frac{1+\sqrt{5}}{2},\left(\frac{1+\sqrt{5}}{2}\right)^2,...\right)$$.

1. The set of $$3\times 3$$ magic squares (with point-wise addition and scaling) forms a 3-dimensional (real) vector space. E.g., elements like:

$$A=\begin{bmatrix} 2&7&6 \\ 9&5&1 \\ 4&3&8 \end{bmatrix}$$

where one basis for the space is the set:

$$e_1=\begin{bmatrix}1&1&1 \\ 1&1&1 \\ 1&1&1\end{bmatrix}$$, $$e_2=\begin{bmatrix}1&-1&0 \\ -1&0&1 \\ 0&1&-1\end{bmatrix}$$, $$e_3=\begin{bmatrix}0&-1&1 \\ 1&0&-1 \\ -1&1&0\end{bmatrix}$$

• I'm afraid these may not be "concrete"-enough for your purposes, but the first one is easy to verify as a vector space, and they both have non-obvious (to me) bases. Mar 8, 2022 at 21:34
• For example 1. sequences (1,0,...) and (0,1,...) seem like an "obvious" base to come up with. Mar 9, 2022 at 10:10
• @MichaelBächtold Good point, there are many choices. However, it's not easy to give a closed form of these two sequences (i.e. Fibonacci sequence and its shift). This is what $e_1,e_2$ is for, actually. Mar 9, 2022 at 10:45
• I like this approach. Start with a known vector space (the space of all real sequences, with pointwise addition and scaling). Find a subspace of that vector space (the space of Fibonacci sequences). Although there might be a "natural" basis for the subspace, this basis cannot be extended to the space of all sequences, so although the Fibonacci sequences form a space isomorphic to R², it makes sense to see it as a different space than R².
– Stef
Mar 9, 2022 at 11:09
• Great examples. I also think @Stef has captured the general principle here. Another important example which fits into this general scheme is the set of solutions of a linear homogeneous differential equation. Mar 9, 2022 at 11:42

That is a linear algebra course? So presumably before you get to this point of abstract vector space, you already did solution of systems of linear equations? For example, solution of matrix equation $$Mx=0$$ (say $$M$$ is a given $$p \times q$$ matrix, and $$x$$ is an unknown $$q \times 1$$ column). In general, the set of solutions is a vector space, but has no canonical choice of basis.

Then you can discuss the similar case of a linear (ordinary) homogeneous differential equation with variable coefficients. The set of solutions is a vector space, but in general has no canonical choice of basis.

• That set is called the null space of M, and I think there is a canonical basis (based on free variables after row reduction). Mar 9, 2022 at 17:39
• However row reduction in a different order may produce a different basis. Mar 10, 2022 at 1:11
• I would note that this problem often does have a canonical basis: the one you wrote the elements of M in!
– Kai
Mar 10, 2022 at 3:22

I think you're on the right track with the polynomials. They're not wrong that $$(a,b,c)\mapsto (x\mapsto ax^2+bx+c)$$ is an obvious linear isomorphism from $$\mathbb R^3$$ to what I will call $$\text{Poly}_2(\mathbb R)$$, but it is not the only useful one. So is $$\Phi(a,b,c):\in \{f\in\text{Poly}_2(\mathbb R)\mid f(0)=a,f(1)=b,f(2)=c\}$$ Master this identification and the basis of $$\text{Poly}_2(\mathbb R)$$ that is the image of the canonical basis of $$\mathbb R^3$$ under this identification, and you have rediscovered Lagrange interpolation. Yet another example is $$\Psi(a,b,c):\in \{f\in\text{Poly}_2(\mathbb R)\mid f(0)=a,f'(0)=b,f''(0)=c\}$$ Admittedly, the basis elements at the heart of this is just scalar multiples of what your students are thinking about, but thinking about it with this framing gives us Taylor polynomials.

• (Hyperbolic (pun intended) comment) Anyone who has studied differential equations knows that sinh and cosh are a better basis for the space of solutions to f''=f than e^x and e^{-x} Mar 8, 2022 at 20:55
• @AlexanderWoo Thank you for that. I also wanted to mention Fourier transforms as another reframing of real functions with an obvious appeal to linear algebra. But we're dipping our toes into infinite dimensional spaces in a way that might not serve the OP's needs. Mar 8, 2022 at 21:03
• The second example is also important for ODEs, plus you can get parameterized families out of it. e.g. at each $t\in \mathbb{R}$, the 2-D vector space of solutions to $y''+y=0$ has a basis $y_1(t)=1, y_1'(t)=0$ and $y_2(t)=0, y_2'(t)=1$. (They are all rotations of each other.)
Mar 9, 2022 at 0:26

Here's another example, and a comment. First the comment:

I think this is a great opportunity to talk to students about a trend in abstract mathematics where we progressively transition more and more things from "intuitively available" to "defined as part of the mathematical structure". A basis is a choice you make, without it the vector space is just a blank slate and you have no coordinates. You only get coordinates once you've chosen the basis. The "obvious" basis for students is always defined in terms of some already available external frame (in class this is often the edges of the board!). But even if they all accepted that changing bases is important... it's the same deal all over again with the inner product - now you're going to tell me there's more than one way for vectors to be orthogonal!? And there's more than one way to measure the length of vectors? And more than one way to measure length and volume (measure theory)? Or multiply things together (abstract algebra)?

So it's a good opportunity to try to impress upon them that challenging the intuitive "this is the obvious way to do X" (for X = describe coordinates in this case) and putting in place some mathematical methods to describe that is useful. They might not get it completely, but it will give them a hint of what's to come. And for when they ask why: because it allows solving so many, oh so very many problems (as the many answers here show).

At any rate with that aside, this example is similar to Mark H's spaceships:

If you consider the sphere as a 2 dimensional manifold and fix a point p on the sphere, you can talk about the tangent space at that point. It's a two dimensional space, how do you choose a basis? There's nothing on the sphere to give any one basis a preference (cue discussion about the basis being a choice). You can guide them through a thought experiment where two teams get transported onto the same point of, say, the moon, and they choose a basis (draw it on the board without seeing what the other team drew, but don't let them use the edges of the board or the ceiling/walls as reference). Now they have two different maps around that spot. They find an alien ship at a different spot, how can they communicate with each other? They need to convert one team's coordinates to the other's. Try to make them see that any basis that seems "natural" is really only defined in terms of some other existing basis in their ambient environment.

• You got me thinking. :-) If you give someone a (big) paper disk (or a very irregular shape?), and considering that a manifestation of the plane (a vector space), you ask for an "obvious base" there, what would they say? I think it's obvious that there's no preferred base / coordinate system. Mar 14, 2022 at 13:59

# Introduction

The OP mentioned science majors, and since there are loads of examples from science, we might as well start (and probably finish) there. There is a mathematical viewpoint about the utility of bases and abstract vector spaces, but I'll leave that aside and stick to practical concerns of scientific computing. Real examples are complicated and not just by their detail. Pedagogically they are complicated by drawing on bits of linear algebra from throughout a standard first course and sometimes from extra bits you pick up as needed in your work after the standard course. They are further complicated by being based on principles of science, each of which might be unfamiliar to some of one's students. I think one can get around these complicating factors and motivate students. If the problem, including the science and the basic role of linear algebra, is easy enough to understand, then the problem might tempt their curiosity. The main point of these examples will be that when basis vectors are meaningful, their need and usefulness seems unquestionable. In the last group of examples, they are less meaningful but their utility is apparent; and it is less easy to see the vectors, which are functions, as $$n$$-tuples in a natural way. $$\def\y{{\bf y}}\def\x{{\bf x}}\def\A{{\Bbb A}}\def\R{{\Bbb R}}$$

The problem with $$\R^n$$ that the OP describes has several sources.

• The vectors and their coordinates are indistinguishable.
• Students have been dealing with $$\R^2$$ for years.
• The standard basis is orthonormal, which is a most convenient sort of basis.

Any other basis looks pretty inconvenient to a beginner. So $$\R^n$$ feels "canonical." Furthermore:

• A main purpose of a basis is to make computation possible by mapping a vector space to $$\R^n$$.
• To a practically minded person, the real work, the computation, is done in $$\R^n$$, and that is where the real importance lies.
• So why not do this in the easiest, most obvious way possible? For instance, use the coefficients of polynomials.
• It would help the student to accept a basis if the basis vectors had a meaning or represented elementary concepts.
• The standard basis in $$\R^n$$ for $$n=2,3$$ has such a meaning: They tell you how to plot a point in a most simple way. And that meaning is ingrained in the beginner's mindset.

One extra thing I would point out to the students: You won't be doing much number-crunching yourself. You'll be setting up computations to solve problems. It will be easier to set the problem up in a convenient basis and let the computer figure out how to get back to $$\R^n$$.

In the scientific examples below, I've selected examples in which a basis vector corresponds to a fundamental element or structure in the problem being modeled. In function approximation, in which way too "many bases appear," this is less the case. But these are used in important applications. For instance, in radial basis function interpolation (already a mouthful), "surface reconstruction" may sound abstruse, and students might need help imagining what is so important about modeling a surface. Well, monitoring the growth of a cancer tumor or visualizing the surfaces in a heart seem important applications to me.

# Linear Approximations

Science, to speak broadly, uses mathematics to model phenomena, often under simplifying assumptions that make the model approximate. The mathematics itself often needs to be computed approximately. Linear approximations are the simplest and are remarkably robust and accurate for solving real problems.

## Spectra (chemistry, physics, group theory)

In a comment I made soon after the question was asked (now deleted), I mentioned that the normal/fundamental modes form a basis for the small vibrations of a molecule or other oscillatory system (small = linearized approximation). In the linearized approximation, the oscillations are given by (complicated) functions of time representing the displacements of coordinates from the rest position. These oscillations form a linear system, a vector space. The remarkable thing is that any vibrational motion is a linear combination of the basis of normal modes, each of which may have its own frequency. This applies to spring-mass systems (@Mark H) as well.

Normal-mode decompositions are used in spectroscopy in chemistry, say, to identify compounds in a mixture. This use of spectroscopy will have been seen by all the chemistry majors at my college by the end of their sophomore year, although they will have had little explanation of the mathematics. They'll see it again in upper-level chemistry along with group theory, where they might learn this gem that was a highlight of my undergraduate education: The irreducible representations of the vibrational symmetries of methane (or other tetrahedral molecule) cannot be computed without the complex numbers. That is to say, the real phenomenon cannot be fully understood with just real numbers. As a liberal arts major, I could rejoice that what was real was complex.

Something I learned later was the significance of the two complex-conjugate irreducible representations that appear in the analysis of methane. The vibrational modes associated to these symmetries are the only ones that generate an oscillating dipole that is necessary for infrared absorption. There are three modes associated to each of the two symmetries, both labeled $$t_2$$ below but with different frequencies. Further, they oscillate at infrared frequencies (wavenumbers $$3156.8\ \text{cm}^{-1},\ 1367.4\ \text{cm}^{-1}$$. It is why methane is a powerful greenhouse gas. ## Approximating matrices

When you have data, such as the oscillations of the coordinates of the atoms in a molecule as in the preceding example -- the $$\R^n$$ representation -- the standard coordinates don't necessarily in themselves represent an important concept in the problem, unlike the normal-mode coordinates, which indicate how much of each fundamental element, a normal mode, makes up a given vibration. In the vibration problem, there are as many normal modes as degrees of freedom, and each normal-mode vector is equally important. In some problems, some vectors are more important than others, and identifying them can reveal the structure of phenomena. We'll look at one way to do that.

It turns out that any matrix $$A$$ can be written $$A = U S V^*\,,$$ where

• $$V^*$$ is the transpose of $$V$$ (or conjugate transpose if $$A$$ is complex),
• $$A$$ is an $$m\times n$$ matrix,
• $$U$$ and $$V$$ are $$m \times m$$ and $$n \times n$$ orthogonal matrices (or, resp., unitary ones if $$A$$ is complex -- that is, in both cases, $$V^*=V^{-1}$$),
• and $$S$$ is a $$m \times n$$ diagonal matrix with nonnegative real diagonal entries $$s_{1} \ge s_{2} \ge \cdots \ge s_{n} \ge 0 \,.$$

If $$r$$ is the rank of the matrix $$A$$ and $$r < n$$, then we have $$s_{1} \ge s_{2} \ge \cdots \ge s_{r} > 0 \ \text{and}\ s_{r+1}=s_{r+2}=\cdots=s_{n}=0\,.$$ If the real numbers $$s_{1}, s_{2}, s_{3},\dots$$ decrease rapidly, we can replace the small ones by zero and change the matrix $$A$$ by only a small amount. This factorization of $$A$$ is called the singular value decomposition or SVD and the nonzero $$s_j$$ are called the "singular values" of $$A$$. The SVD factorization is not always covered in a first linear algebra course. It could be stated as a fact and used with the examples below to motivate future learning. It is worth considering simply because it's a really cool structure theorem that is in widespread use to solve real problems. Have the students google "principal component analysis" or "PCA." The SVD is readily available in any decent system or library for linear algebra, and students may have to use it without ever covering it in class: Python, R (svd()), Sage, Maple, Matlab, Mathematica, Maxima GNU GSL, etc.

We'll give a brief explanation of how it identifies "principal" components of a linear model. First, let's break down the factorization of $$A$$. The matrix $$A$$ represents a linear mapping from $$\R^n$$ to $$\R^m$$. The columns of $$U$$ and $$V$$ are orthonormal bases (the best kind) for the output $$\R^m$$ and the input $$\R^n$$, respectively. First some notation: for a matrix $$M$$, let $$M_{ij}$$ denote the $$ij$$ entry (two indices) and $$M_j$$ denote $$j$$-th column matrix (one index). Then to compute $$Ax$$ for a vector $$x$$, we may write $$Ax = \Big(\sum_{j,k} U_{ij} s_j V_{kj}x_k\Big)_{i=1}^m = \sum_j (s_j\; (V_j^* \cdot x)) U_{j} \,. \tag{1}$$ Thus we first compute $$V^*x$$, which are the coordinates of $$x$$ with respect to the basis given by the columns of $$V$$. Then $$SV^*x$$ scales these coordinates by the $$s_j$$. These scaled coordinates are the coordinates of $$Ax$$ with respect to the basis given by the columns of $$U$$, that is $$Ax = U(SV^*x)$$. It's pretty amazing that every matrix action can be viewed in terms of choosing two orthonormal bases and scaling coordinates. We can also see what happens when we replace $$s_j$$ by $$0$$ for a single value of $$j$$. It is in effect the same as deleting the $$j$$th term from (1), which introduces an error bounded by $$\|(s_j\; (V_j^* \cdot x)) U_{j}\| \le s_j\,\| x\| \,,$$ since $$\|U_j\|=\|V_j\|=1$$ and $$s_j \ge 0$$. If we replace all $$s_j$$ by $$0$$ for $$j >J$$, then the error relative to $$\|x\|$$ will be bounded by $$\sum_{j >J} s_j \,. \tag{2}$$

### Image processing (data compression)

I'm counting it as "science" because we'll use the image of a scientist. Mainly, I introduce the notion this way because it's visual, and it's hard to believe one's eyes. In this application, we're approximating a matrix itself. Since the matrix entries are extracted from image data and vice versa, students can see the approximation improve. The initial matrix entries are the grayscale values (between $$0$$ and $$1$$) of the grid of pixels. The singular values this matrix decrease from $$s_1=338$$ to $$s_4=30$$, to $$s_{50}=3.3$$, to $$s_{100}=1.4$$, and finally to $$s_{841}=0.039$$ -- a decrease of five orders of magnitude. Below are the images from the matrix $$A^*=\sum_{j=1} ^J s_j\, U_{j} (V_j)^T$$ for the singular values up to $$J=4,\,50,\,100,\,841$$ (all). The relative error (RE) was calculated using matrix 2-norms, $$\text{RE} = \|A^*-A\|/\|A\|$$, but one should expect a weak correlation between the error of approximating a linear map and how accurate the picture looks. The Frobenius norm gives relative errors of 21%, 5.6%, 3.6%, and 0% respectively, which perhaps seem closer to how the images appear. All norms give estimates much lower than the upper bound $$\sum_{j>J} s_j$$ deduced above. Here are a couple of codes for constructing the image approximation (I used this image from Wikipedia, scaled by a factor of 1/3 and converted to grayscale):

(* Mathematica *)
img = Import["<path/url to image file>"]; (* convert to grayscale if necessary *)

{U, S, V} = SingularValueDecomposition[ImageData@img]; (* assuming "GrayScale" image *)
J = 4;  (* change as desired *)
Image[U[[All, ;; J]] . S[[;; J, ;; J]] . Transpose@V[[All, ;; J]]]

% MATLAB
img = imread('<path to image file>');  % convert grayscale if necessary

[U,S,V] = svd(im2double(img)); % convert grayscale img to double
J = 50; % change as desired
newimg = U(:,1:J) * S(1:J,1:J) * V(:,1:J)';
imshow(newimg)


### DNA analysis (biology)

Quantitative Understanding in Biology: Principal Component Analysis (lecture notes by Ju et al.) is a wonderful introduction to principal component analysis (PCA), which may be read by any student in linear algebra. It refers the connection to the singular value decomposition to a technical paper, but it does a good job explaining why identifying principal components and using them as basis vectors helps one tame an unwieldy dataset. The authors describe how some common DNA datasets have 10,000+ measurements per sample and others around 600,000-900,000. They point out, "These datasets are typically never fully visualized because they contain many more datapoints than you have pixels on your monitor." They introduce the notion of PCA with a simple numerical example, consisting of a dataset of two samples resulting from the measurement of 5 genes ($$g_1$$ through $$g_5$$), in which there is hidden correlation. They show that the 5-dimensional representation may be reduced to 2 dimensions, using the principal components. Further they explain the weakness of the standard coordinates: "knowing the expression level of $$g_1$$ will not give us any information about other genes." And they explain the significance of the principal components to understanding the biology:

However, in reality the expression levels of multiple genes tend to be correlated to each other (for example, pathway activation that bumps up the expression levels of $$g_1$$ and $$g_2$$ together, or a feedback interaction where a high level of $$g_3$$ suppresses the expression level of $$g_4$$ and $$g_5$$), and we don't have to focus on the expression levels individually. This would mean that by knowing the expression level of $$g_1$$ we can get some sense of the expression level of $$g_2$$, and from the level of $$g_3$$ we can guess the levels of $$g_4$$ and $$g_5$$.

Thus the principal basis vectors reflect biological function. That's pretty cool.

### Protein Conformational Motions of HIV-1 Protease (biology, medicine, biomedical)

In Singular Value Decomposition of Protein Conformational Motions: Application to HIV-1 Protease, Teodoro et al. analyze the conformational changes of an enzyme via the SVD. "An illustrative example of these changes is the opening and closing of the binding site of HIV-1 protease (HIV Pr)...." HIV Pr plays an important role in the life cycle of the HIV virus, and the opening and closing of the binding site allows the enzyme to do its job. Disrupting its function is the focus of protease inhibitors. The analysis of the motions is similar to the molecular spectra discussed above, except proteins exhibit both small vibrations and larger motions, and the interest lies in the larger motions. The dimension of the space of motions is $$597$$, and the SVD was used to focus on a few principal basis vectors.

The left singular vectors [i.e., the columns of $$U$$] obtained from the decomposition correspond to modes of collective motion whose displacements are directly proportional to the value of the corresponding singular values. The calculated right singular vectors [i.e., the columns of $$V$$] correspond to the projection of the original trajectory in the new basis.

In this way, they were able to show that "the protein jumps rapidly between stable intermediate conformations until it reaches the open conformation." Again, the problem cannot easily be investigated in terms of $$597$$-tuples, and choosing the right basis, in which the vectors reflect structures within the problem, allows one to deduce how those structures work.

## Function approximation

Function approximation is not a science topic per se, but it is used broadly in numerical solutions to scientific problems. Let's say you have a bunch of function values $$y^{(j)}$$ at a number of points $$\x^{(j)}=(x_1^{(j)},x_2^{(j)},\dots,x_n^{(j)})$$ for $$j=1,2,\dots,m$$, and you'd like to construct a continuous function $$f(\x)=f(x_1,x_2,\dots)$$ based on this data. How to do it?

Viewed as a linear problem, we would choose a basis of functions $$\{\phi_1,\dots,\phi_n\}$$ and solve for coefficients $$c_j$$ such that $$f(\x) = \sum c_j \phi_j(\x)$$ meet some criteria. The interpolation problem is to find a function that passes through each point so that $$f(\x^{(j)})=y^{(j)}$$. The fitting problem is to find a function that minimizes the error between the values $$f(\x^{(j)})$$ of the function and the values $$y^{(j)}$$ at each point; the error typically used is least squares, which problem can be solved with linear algebra. When the values $$y^{(j)}$$ are the values of a given function, then the problem is one of approximating the given function. If the basis functions have the property that $$\phi_k(\x^{(k)})=1$$ and $$\phi_k(\x^{(j)})=0$$ for $$j\ne k$$, then the basis is particularly nice for the interpolation problem: $$c_j=y^{(j)}$$ and there's no need to solve a system of equations for $$c_j$$. Such a basis is called a cardinal basis.

### Polynomial interpolation

The power basis $$1,\ x,\ x^2,\dots, x^n$$ is often numerically poorly conditioned for the interpolation problem. The Chebyshev polynomials $$T_j$$ form a better basis, and the Lagrange basis, which is a cardinal basis, is better still in its barycentric form (see Berrut & Trefethen, eq. (4.2)). There are other families of orthogonal polynomials besides Chebyshev that have their uses.

### Chebyshev approximation

A truncated power series (Taylor polynomial) is often not a good approximation away from its center of expansion. However, for a function that is analytic over a finite interval $$[a,b]$$, its Chebyshev series converges rapidly on $$[a,b]$$ with an error bound that is easy to state and apply practically. Let $$f(x)$$ have a Chebyshev series $$f(x) = \sum_{j=0}^\infty c_j T_j\left( \frac{2 x - (a+b)}{b-ab} \right) \,.$$ If the series is truncated at the term $$j=J$$, then since $$|T_j| \le 1$$ on the interval, the truncation error is bounded by $$\sum_{j>J} |c_j| \,, \tag{3}$$ which is similar to the SVD truncation error bound in (2). If the coefficients $$c_j$$ are rapidly decreasing, then $$c_J$$ is usually a good bound on the error. If we interpolate $$f(x)$$ at the Chebyshev points $$x_k = \cos(\pi k /J)$$, $$k=0,1,\dots,J$$ using the Chebyshev basis, we approximate the truncated Chebyshev series $$p(x) = \sum_{j=0}^J a_j T_j\left( \frac{2 x - (a+b)}{b-ab} \right) \approx \sum_{j=0}^J c_j T_j\left( \frac{2 x - (a+b)}{b-ab} \right) \approx f(x)\,.$$ The Chebyshev basis is an orthogonal basis with respect to the inner product $$\langle f, g \rangle = \int_a^b f(x) g(x) / \sqrt{(x-a)(b-x)}\, dx$$, and truncating the series is equivalent to orthogonal projection onto a finite dimensional subspace (= polynomials of degree at most $$J$$). As such, the truncated series is an optimal solution to approximating the function $$f(x)$$ with a polynomial of degree at most $$J$$ with respect to the norm induced by the inner product. Further it is easy to evaluate, integrate, differentiate and find the roots of Chebyshev interpolations. It is one of the best ways to solve transcendental equations (see Boyd (2014)). Chebyshev approximation is the basis of the Chebfun project.

### Fourier approximation

Virtually the same thing can be said for Fourier approximation, which in fact might be viewed as the basis for Chebyshev approximation. The Fourier basis, mentioned by @Kuba, for the interval $$[-\pi,\pi]$$ is $$\phi_j(x) = e^{i j x}$$, $$j\in{\Bbb Z}$$. Since $$|e^{i j x}|=1$$ and the coefficients of Fourier series for analytic functions are rapidly decreasing, we get similar approximation properties, including orthonormality of the basis (with respect to the inner product $$(2\pi)^{-1}\int_{-\pi}^\pi f(x) g(x) \, dx$$). One reason this and the Chebyshev approximation are important is that the Fast Fourier Transform (FFT) makes these approximations computationally efficient.

### Sinc interpolation (engineering, signal processing, image processing)

The sinc function $$(\sin t)/t$$ or the normalized sinc function $$\sin(\pi t)/(\pi t)$$ (for $$t\ne0$$) can be used to form a cardinal basis for interpolating sampled signals or other regularly spaced time series. It should be understood that the sinc functions are taken to be continuous and have the value $$1$$ at $$t=0$$. Suppose we have a regular sample rate of $$1/h$$ and thus the sampled times are $$t_j=jh$$. The sinc basis functions are $$\phi_j(jh) = 1\,,\ \text{otherwise}\ \phi_j(t) = {\sin(\pi (t-jh)) \over \pi (t-jh)} \,.$$ The interpolation formula for the sampled data $$y^{(j)}=f(jh)$$ of a signal $$f(t)$$ is $$f(t) \approx \sum_j y^{(j)} \phi_j(t) \,.$$

### Radial basis function interpolation (neural networks, engineering, signal processing)

Radial basis function (RBF) interpolation can be used to interpolate "unstructured" data of any dimension (the sampling nodes need not be on a regular grid). For instance, the sampling nodes could be the locations in a network of weather stations or other geographic phenomena. Applications include the approximation of functions on arbitrary surfaces; surface reconstruction from medical imaging or geological stratigraphic data; machine learning; and so forth. An RBF approximation is built on a radial basis $$\phi_j(x)=\phi(\| \x-\x_j\|)$$ and has the form $$f(\x) \approx \sum_{j=1}^J c_j \phi(\| \x-\x_j\|) \tag{4}$$ for some function $$\phi(r)$$ of the distance from a sampling node, typically a Gaussian $$e^{-\alpha^2 r^2}$$ for some "shape parameter" $$\alpha$$. Here $$\x$$ may be in $$\R$$ or $$\R^n$$. Solving for the coefficients $$c_j$$ is a linear problem and might be done either as a least-squares fitting problem or as an interpolation problem. As an example, Maz'ya & Schmidt (2007) and Boyd & Wang (2009) independently derived a cardinal basis for Gaussian RBF approximation on an infinite grid with uniform spacing $$h = x_j - x_{j-1}$$ ($$-\infty) and shape parameter $$\alpha$$: $$\phi_j(x)=\phi(x-x_j),\quad\text{where}\quad \phi(x)= {\alpha ^2\over\pi}\, {\sin \left({\pi x}/{h}\right) \over {\sinh}\left({\alpha^2 x}/{h}\right)} \,.$$ Since for this grid, the sum would be infinite, one usually truncates it and sums over a finite interval. (It is related to sinc interpolation but the $$x$$s and $$h$$s in $$\sin \left({\pi x}/{h}\right)/(\pi x/h)$$ and $${\sinh}\left({\alpha^2 x}/{h}\right)/(\alpha^2 x/h)$$ cancel out.)

• Thank you very much, @Chris. Your appreciation is very much appreciated. :) Mar 28, 2022 at 0:09

To simplify some examples already given here, consider $$\{(x,y) : x+y=0\}$$ or $$\{(x,y,z) : x+y+z = 0\}$$.

Consider physical values (things like 5 m/s, 17 keV, 42.34) with:

• the multiplication of values as vector addition,
• the exponentiation of values as scalar multiplication.

You get a mix between ℤ-module and vector space, which you might treat like a vector space for applications.

The typical basis for this vector space are the SI base units (and 10), but this is clearly a human convention and you can as well choose the CGS system, natural units, imperial units, etc. Mind that in particular natural units differ by more than scaling of the basis vectors, i.e., basis vectors point in different directions. One practical application of this vector space is transforming values to a different unit system.

• Somewhat simpler (to my mind) is taking the set of physical values of the same physical dimension, for instance length. The usual addition is vector addition and the usual scalar multiplication is scalar multiplication. This is a one dimensional vector space without a canonical basis (unless maybe Planck length, but I doubt students would come up with that). One might explain by historical examples how humans have tried to come up with a standard basis (unit) but have still not managed. Mar 9, 2022 at 9:39
• The difference is that the real numbers have a canonical base (1) while this space does not. Mar 9, 2022 at 9:50
• To me this is like the primordial example, since a) it's one dimensional and b) humans have used this vector space for at least 2000 years and have struggled to agree on a basis for as long as they've used it. Mar 9, 2022 at 10:23
• Sort of repeating my comment in another answer: if you draw a (long, "infinite") line, it is a vector space if you give it an arbitrary origin. But it has no "1". So it's obvious that the choice of base is arbitrary. (After you choose a base you get the "1".) Mar 14, 2022 at 14:04
• @LSpice as far as I can tell from Wikipedia, the SI standard calls that physical dimension "length" and I assume also negative values belong to it. But I agree that for students, used to everyday usage, lengths should be positive. Maybe "displacements" would be a better word? Alternatively, the first physical dimension that comes to mind that naturally has a sign would be electrical charge. But that seems to be naturally quantized, so more of a $\mathbb{Z}$ module. Mar 25, 2022 at 7:55

If $$P_n$$ is the space of polynomials of degree $$n$$ or less, there are natural reasons why the canonical basis is less useful than others. You can show them that while $$\mathbb R^n$$ has a natural Euclidean structure (that is, an inner product that induces the natural distance), using it in $$P_n$$ gives you a structure that completely ignores the fact that polynomials are functions. From this point of view, the Euclidean inner product is $$\langle p,q\rangle_E=\sum_{k=0}^n\frac{p^{(k)}(0)\,q^{(k)}(0)}{(k!)^2},$$ while there are more natural inner products like $$\langle p,q\rangle_N=\sum_{k=0}^n p (k)q(k)$$ and, with more long-term consequences, $$\tag1 \langle p,q\rangle_2=\int_0^1p(t)q(t)\,dt.$$ For these products it is easy to see that the canonical basis is not orthonormal. The natural orthonormal basis for $$(1)$$ are the Legendre polynomials, which come from doing Gram-Schmidt to the monomials. The article mentions several applications in physics.

For an example with a different flavour, in Quantum Information many things happen in, or use, $$M_2(\mathbb C)$$. While there is an obvious canonical basis (the matrix units $$E_{k j}$$ with $$1$$ in the $$k,j$$ entry and zeroes elsewhere), the preferred basis is the Pauli basis $$\begin{bmatrix}1&0\\0&1\end{bmatrix},\quad \begin{bmatrix}0&1\\1&0\end{bmatrix},\quad \begin{bmatrix}0&-i\\i&0\end{bmatrix},\quad \begin{bmatrix}1&0\\0&-1\end{bmatrix}.$$ Most common applications in physics are mentioned here.

Somewhat in the spirit of @AlexanderWoo's answer, one could consider the collection(s) of linear polynomials on $$\mathbb R$$ vanishing at $$x_o$$, or quadratic polynomials vanishing at $$x_0$$, ... or at both $$x_1$$ and $$x_2$$ ... and/or cubic polynomials vanishing at ...

Vanishing conditions on (finite or infinite-dimensional) spaces of functions are genuine...

And "orthogonality": cubic polynomials on $$\mathbb R$$ whose integrals over $$[0,1]$$ are $$0$$. Or, whose integrals against $$x^2$$ over $$[0,1]$$ are $$0$$...

You've said that your students found the monomial basis to be "immediate" and you even refer to it as "canonical". There are other useful bases for the space of polynomials and I'd even argue that there's nothing "canonical" about monomials. The Bernstein polynomials are much more advantageous for both curve fitting and for computer-aided geometric design, see for example Bezier curves. In particular, the curve

$$\gamma(t) = \sum_{k = 0}^nB_{k, n}(t)P_k$$

defined as an expansion in degree-$$n$$ Bernstein polynomials $$B_{k, n}$$ lies in the convex hull of the control points $$P_k$$. The reason that's true is because the Bernstein polynomials form a partition of unity, which is not true of the monomial basis. For a concrete example, you can open up any vector graphics drawing program and show them how easy it is to make shapes with Bezier curves. Trying to do that with monomials would be nigh-impossible.

Likewise, for numerical solution of PDE, the Legendre polynomials can be a much better choice because of orthogonality, but that might be getting too far afield.

Here's one inspired by @Wrzlprmft:

Weight* is a vector space, without a universally standard basis. (I regularly have to "change bases" from ounces to grams in the kitchen.)

Reasons I like this example:

1. it connects something familiar to a new mathematical concept
2. it highlights the difference between a vector space and a vector space with coordinates or a fixed basis. We know we can add and scale lengths, but something crucial prevents us from immediately identifying lengths with real numbers
3. it requires no extra math

$$^*$$i probably mean mass, but I am not going to get into those weeds with my linear algebra students

• Ah i see now that this is the same idea as Michael Bächtold :-) Mar 9, 2022 at 18:50
• Indeed, but it’s good to have it as an answer since Michael Bächtold only put it in comments! Mar 10, 2022 at 11:46
• Anyone want to hazard a guess about the down vote? Mar 10, 2022 at 18:38
• @DavidSteinberg We never know with this site. Unfortunately, down votes are free to dole out. Since you're asking for a reason, maybe providing a specific example would help flesh out the answer? Again, we may never know. :/ Mar 10, 2022 at 21:29
• @NickC isn't the space of weights a specific example of a one dimensional vector space with many bases (kilogram, ounce, pound, dalton, slug...) but no obvious one? Mar 12, 2022 at 15:43

Here is a useful project in this regard:

We all know about b^2-4ac, the discriminant for a quadratic polynomial. It tells us, in terms of the coefficients of a quadratic polynomial, whether the polynomial has a double root.

We would like to do the same for a cubic polynomial. To simplify, let's make the polynomial monic, so we would like a polynomial in b, c, and d that tells us whether the cubic polynomial x^3+bx^2+cx+d has a double root.

Now, if we let r, s, and t be the roots of the polynomial, then the polynomial is (x-r)(x-s)(x-t), and b, c, and d are the elementary symmetric functions in terms of r, s, and t - to be precise, b=r+s+t, c=rs+st+rt, and d=rst.

At the same time, the polynomial as a double root if (r-s)(r-t)(s-t)=0. This isn't symmetric, so it's not a polynomial in b, c, and d, but its square is symmetric.

Now we can express (r-s)^2(r-t)^2(s-t)^2 easily in terms of the monomial basis of symmetric functions (in r, s, and t). What we want here is the change of basis to the elementary basis of symmetric functions, so that we write this expression as monomials in b=r+s+t, c=rs+st+rt, and d=rst.

Warning: I have had students who found this project too theoretical for their tastes; they didn't care enough about the cubic discriminant to think it worth spending that much time on.

• Is it really a basis, or rather a set of generators? In any case, the main problem here is not a change of it, but rather the fundamental theorem of symmetric polynomials, as $R[r,s,t]$ is larger than $R[b,c,d]$ (it's only because of $\Delta \in R_{\textrm{sym}}[r,s,t]$ and the theorem above, that we get $\Delta \in R[b,c,d]$). Mar 9, 2022 at 10:52
• @MichałMiśkiewicz: It's a basis. The monomial basis cardinality in degree d the number of integer partitions of d with at most 3 parts. The number of monomials in the elementaries in degree d (in b, c, d) is the number of integer partitions of d where each part has size at most 3. Same number, so it's a basis. Mar 9, 2022 at 15:52
• Oh, that's a nice argument (and in fact a proof of the fundamental theorem I mentioned). I'd like to see this explicitly in your answer, so that others can benefit too. Mar 10, 2022 at 9:53
• @MichałMiśkiewicz: It's not quite a proof of the fundamental theorem because I still need to show that the elementary basis either spans or is independent. However, for that I can use the usual argument for infinitely many variables. Mar 10, 2022 at 21:43

I think matrices are a good choice so I'll echo some of the other answers given.

$$C = \left\{ \left[\begin{array}{cc} a & -b \\ b & a \end{array} \right] \ \bigg{|} \ a,b \in \mathbb{R} \right\}$$ has natural basis $$I_2 = \left[\begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right] \qquad \& \qquad i = \left[\begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]$$ notice $$i^2 = -I_2$$. Of course, $$C$$ is just a model of the complex numbers, so as a vector space it is isomorphic to $$\mathbb{R}^2$$, but certainly $$C$$ is not a set of $$2$$-vectors directly.

Or, another example I love, the group algebra of the cyclic group of order three, $$\mathbb{R}_{C_3} = \left\{ \left[\begin{array}{ccc} a & c & b \\ b & a & c \\ c & b & a \end{array} \right] \ \bigg{|} \ a,b,c \in \mathbb{R} \right\}$$ here we find natural basis $$j, j^2, j^3=I_3$$ where $$j = \left[\begin{array}{ccc} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array} \right]$$ and you can verify $$aj^3+bj+cj^2 = \left[\begin{array}{ccc} a & c & b \\ b & a & c \\ c & b & a \end{array} \right]$$

In any event, I always had a contingent of students who refuse to admit that $$2 \times 2$$ matrices were not $$4 \times 1$$ vectors. By this I mean, they would circumvent the formal construction of coordinates and just glibly write equality of vectors and the coordinate vectors which represented said vectors. This is both good and bad. It's good these students see through the notation to realize an $$n$$-dimensional real vector space is just $$\mathbb{R}^n$$ in disguise. It's bad because their understanding of the coordinate construction is so fuzzy that coordinate change will forever be a mystery to such students.

The Fourier series elements form basis vectors in the linear vector space of functions of one variable (for simplicity). In fact, it's an excellent homework assignment to have the students prove this - they should know enough calculus at this point to be able to do it.

It seems like if we aim for a vector space without an obvious basis, then the kernel of a linear transformation is a easy example. If we aim for a vector space with multiple good bases, then the eigenbasis presents itself: the canonical basis is natural (maybe it represents position in a physical system), but the eigenbasis is convenient for computation (whether it be solving differential equations or taking powers of transformations). What is nice about the latter example is that when we have multiple, important bases, it becomes an important task to conveniently transform between the two.

Here is a mathematical example from combinatorics using polynomials, which fits neither of the above forms (as far as I am aware). Given $$n$$ people, the number of ways to fill a leaderboard with $$k$$ people is $$n(n-1)\cdots(n-k+1)$$. We define $$x^{\underline{k}} = x(x-1)\cdots(x-k+1)$$. Then the $$x^{\underline{k}}$$ form a basis for the space of polynomials which is not the monomial basis. When we write $$x^{\underline{k}}$$ in the monomial basis, we obtain the Stirling numbers of the first kind, which count the number of permutations with $$k$$ cycles. When we write monomials in the $$x^{\underline{k}}$$ basis, we obtain the Stirling numbers of the second kind, which count surjections. The point is, if you are going to be doing some counting, it may be more convenient to use the $$x^{\underline{k}}$$ basis.

A lot of the existing answers depend on specific subject-level outside knowledge, or a level of mathematical internalization that isn't necessarily the case for all introductory linear algebra students. So I'd keep it even simpler and more concrete.

Consider a city whose streets have a grid layout and some cross-cutting avenues (e.g. Manhattan at 6th Ave and W 8th St, but you can ideally find a local example that your students will be familiar with, so that you can define the origin as the lecture hall). Any place on the map (the reals in 2) can be described as going so-and-so many blocks parallel to any pair of perpendicular grid streets, or to one of the grid streets and one of the diagonals (both vectors in 2). Thus the streets constitute many different bases for the addresses in the city. But which basis is most convenient for giving/following directions depends very much on the location of the building you're trying to get to: you should probably choose the one that lets you follow the directions without walking through any walls! Just as in your mathematical careers, you'll choose a basis that's easiest to work with.

• I think this is perhaps more appropriate in the discussion of affine spaces, but obscures the point for linear spaces. In particular, in standard changes-of-bases, we do not change what the origin is, but rather what directions "count" as the first, second, ..., nth coordinate in describing location relative to the origin. Mar 10, 2022 at 20:44
• @OpalE Fair point, though I don't think it's one that will be meaningful for students who need this concrete of an example. In any event, I've clarified that the lecturer can and should define a fixed location as the origin. Mar 10, 2022 at 21:22

I guess it's up to me to advertise telcomm applications, so

Let's do wave!

The waves $$f(t)=A \cos(\omega t+\phi_0)$$ of a given frequency $$\omega$$ (and varying amplitude $$A$$ and phase $$\phi_0$$) form a $$2$$-dimensional vector space $$V_\omega$$ (over $$\Bbb{R}$$). Basically because of the trig formulas $$\cos(\omega t+\phi_0)=\cos\phi_0\cos (\omega t)-\sin\phi_0\sin(\omega t)$$ and $$\sin(\omega t)=\cos(\omega t-\pi/2)$$ that A) show that $$\sin(\omega t)\in V_\omega$$ and B) show that every wave is a linear combination $$\cos(\omega t)$$ and $$\sin(\omega t)$$. Conversely $$A\cos(\omega t)+B\sin(\omega t)=C\cos(\omega t+\phi_0),$$ where $$C=\sqrt{A^2+B^2}$$ and $$\phi_0=-\operatorname{atan2}(A,B)$$ is chosen such that $$A=C\cos\phi_0$$, $$B=C\sin\phi_0$$.

Ok. This may be a bad example in the sense that I already singled out $$\cos(\omega t)$$ and $$\sin(\omega t)$$ as a natural basis. Anyway, it may be not so immediately visualizable as the natural basis of $$\Bbb{R}^2$$.

Then you can further point out that 4G cell phone signals are in a higher dimensional space of the form $$\bigoplus_{i=0}^{2047}V_{\omega+ i\Delta\omega},$$ where $$\omega$$ and $$\Delta\omega$$ depend on the frequency band of the operator. Other powers of two (instead of $$2^{11}=2048$$) are also in use.