22
$\begingroup$

In teaching multi-variable calculus, it's insightful to discuss with students not only how certain concepts from single-variable calculus extend to multiple variables but also where these extensions fail. For example, while in single-variable calculus, a continuously differentiable function defined on an open interval that has a unique critical point, which is a local minimum, also has a global minimum at that point. However, this is not necessarily the case in multi-variable calculus, as there are known counterexamples. here is one for a two-variable function:

$z= f(x, y) = x^2 + (1 − x)^3y^2$ for this function, $(0, 0)$ is the single critical point that is a local minimum but it is not global enter image description here

enter image description here

I am compiling a list of such instances to enhance my curriculum. Could you share other specific results or theorems that hold in single-variable calculus but do not necessarily apply in the multi-variable setting? Examples where the multi-variable context introduces significant complexity or entirely new phenomena would be particularly valuable.

$\endgroup$
4
  • 4
    $\begingroup$ I can't think how to state this as a theorem that goes wrong but, in single variable calculus, when we talk about optimization on $[a,b]$, we spend almost all of our time on finding critical points in the interior $(a,b)$ and just a little time on checking the endpoints. In high dimensional optimization, as a heuristic, optima are often at corner points. In particular, it is common to optimize a linear function on a polytope (linear programming) or a convex function on a convex body (convex optimization), in which case the optimum is always on the boundary. $\endgroup$ Feb 22 at 18:56
  • 1
    $\begingroup$ This is an interesting question. It would also be interesting to ask the parallel question of which theorems do naturally generalize to $n$-variables. There are many interesing things to learn in that direction as well. Some of these are treated very poorly in many of the mainline calculus texts which give but a shadow of the true multivariate calculus. $\endgroup$ Feb 23 at 1:15
  • 1
    $\begingroup$ Mean value theorem gets significantly weaker in the multivariate setting. $\endgroup$ Feb 23 at 6:01
  • 1
    $\begingroup$ A variation on @DavidESpeyer's comment: In single-variable calculus, one of the "easiest" optimization problems is to max/min a quadratic function on an interval. Finding the critical points is painless, and there's only one of them to check along with two endpoints. Conversely, the problem of maximizing/minimizing a quadratic function on $[-1,1]^n$ is (NP-)hard, precisely due to the issues David mentioned: There's just far too many vertices to check. It looks like the easy single-variable calculus problems, but ends up being vastly different in terms of difficulty. $\endgroup$ Feb 26 at 20:37

6 Answers 6

36
$\begingroup$

Path-dependency of limits can lead to very counterintuitive results. For instance, consider the following limit:

\begin{align*}\lim\limits_{(x,y) \to (0,0)} \frac{x^2y}{x^4+y^2}\end{align*}

An intuitive-but-incorrect way to think about this limit is to imagine it only has one variable:

\begin{align*}\lim\limits_{x \to 0} \frac{x^2x}{x^4+x^2} = \lim\limits_{x \to 0} \frac{x}{x^2+1} = \frac{0}{1} = 0\end{align*}

The problem is that in the computation above, we evaluated the limit along the path $y=x,$ but this is just one of many possible paths.

Okay, then why don't we evaluate the limit along all lines through the origin? We can handle that in three cases: the general case $y=mx$ where $m$ is a nonzero constant, and then the cases $y=0$ and $x=0$ separately. The limit comes out to $0$ in all these cases:

\begin{align*}\lim\limits_{x \to 0} \frac{x^2(mx)}{x^4+(mx)^2} = \lim\limits_{x \to 0} \frac{mx}{x^2+m^2} = \frac{0}{m^2} = 0,\end{align*}

\begin{align*}\lim\limits_{x \to 0} \frac{x^2(0)}{x^4+0^2} = 0, \quad \quad \lim\limits_{y \to 0} \frac{(0^2)y}{0^4+y^2} = 0. \end{align*}

However, this is still not correct! Look what happens if you approach along the path $y=x^2.$ We get a different result:

\begin{align*}\lim\limits_{x \to 0} \frac{x^2(x^2)}{x^4+(x^2)^2} = \lim\limits_{x \to 0} \frac{x^4}{x^4+x^4} = \frac{x^2}{2x^4} = \dfrac{1}{2}\end{align*}

It turns out our limit $\lim\limits_{(x,y) \to (0,0)} \frac{x^2y}{x^4+y^2}$ evaluates to $0$ along all linear paths, but not along all paths in general (for instance, it came out to $\frac{1}{2}$ along a quadratic path). So generally speaking, our limit does not exist.

This is super weird and counterintuitive, but it's true, and you can get a better sense of why it happens if you look at the graph of $f(x,y) = \frac{x^2y}{x^4+y^2}{:}$

multivariable-surface-graph

$\endgroup$
6
  • 6
    $\begingroup$ This is the main issue. The existence of infinitely many paths as opposed to just the left and right limits, this is so much of what makes multivariate analysis challenging. $\endgroup$ Feb 23 at 1:10
  • $\begingroup$ Is it possible to carefully construct functions with an arbitrary number of limits at a point depending on the path taken, or is there some bound to the number of limits at a point? $\endgroup$
    – Chuu
    Feb 23 at 16:27
  • 3
    $\begingroup$ @Chuu Super easy way: $f(x,y) = \frac{y}{x}$. Then the limit along $y=mx$ is equal to $m$. $\endgroup$ Feb 23 at 18:55
  • 1
    $\begingroup$ Nice example. It makes my brain hurt with a solid depiction of just how many "paths" there are. $\endgroup$
    – Cort Ammon
    Feb 23 at 19:22
  • 3
    $\begingroup$ Wow, thank you, that example is so cursed. And it's not even that weird of a function. This has materially improved my understanding of limits in multivariable calculus (a class I took many years ago now.) $\endgroup$ Feb 25 at 4:24
22
$\begingroup$

Critical points of smooth functions in one-dimension tend to be local extrema. This is because $f''(c)=0$ is an unlikely event. When $f'(c)=0$, we expect a local min or max, depending on the sign of the second derivative.

In higher dimensions, the critical points correspond to the simultaneous vanishing of all the first-partial derivatives. The second derivative becomes a matrix of second partial derivatives called the Hessian. For a critical point to be a local maximum or minimum, the eigenvalues of this matrix all have to be the same sign.

Since there are $n$ eigenvalues, and each can be either positive or negative, we have $2^n$ types of critical point behavior. Only 2 of these are a max or a min. So we should expect roughly $\frac{1}{2^{n-1}}$ of all critical points to be local extrema, with the rest saddles. This is imprecise since we have not clarified what "picking a function at random" means, but it is a solid intuition.

$\endgroup$
7
  • 7
    $\begingroup$ This paper quantifies this intuition for polynomials with normally distributed coefficients: arxiv.org/abs/math/0702360. I will also say that this intuition is pretty important for understanding loss functions of neural networks! $\endgroup$ Feb 22 at 2:53
  • 8
    $\begingroup$ +1. The sentence "But for the critical point to be a local maximum or minimum, all the second partial derivatives have to have the same sign" seems to be a bit misleading, though. It's the eigenvalues of the Hessian that should have the same sign, not the second derivatives themselves. $\endgroup$ Feb 22 at 7:46
  • $\begingroup$ @JochenGlueck I agree. It is just a necessary condition. But I'm trying to keep with the vocabulary of single-variable calculus. $\endgroup$
    – user52817
    Feb 22 at 12:51
  • 2
    $\begingroup$ I guess the right formulation would be "all second directional derivatives should have the same sign", meaning all derivatives of the form $\frac{d^2}{dt^2} \big|_{t=0} f(c+tv)$, for all vectors $v$. This is indeed equivalent to the Hessian matrix being nonnegatively/nonpositively definite. $\endgroup$ Feb 22 at 19:39
  • 1
    $\begingroup$ @MichałMiśkiewicz True, but then you loose the intuition about how many "degrees of freedom" you have. If you assign the eigenvalues $\pm$ at random you get $2^n$ choices in $\mathbb{R}^n$, which let's you see that the chance of having a local max or min is roughly $\frac{1}{2^{n-1}}$. $\endgroup$ Feb 22 at 19:49
17
$\begingroup$

Existence of antiderivatives.

Every continuous function $f: \mathbb{R} \to \mathbb{R}$ has an antiderivative.

On the other hand, a continuous vector field $f: \mathbb{R}^d \to \mathbb{R}^d$ for $d \ge 2$ need not be the gradient of a function $u: \mathbb{R}^d \to \mathbb{R}$, in general. If the vector field $f$ is continuously differentiable, then the existence of a differentiable function $u: \mathbb{R}^d \to \mathbb{R}$ that satisfies $\nabla u = f$ is equivalent to the Jacobi matrix of $f$ being symmetric.

Another obstacle to the existence of such a $u$ that occurs in dimension $d \ge 2$ is the existence of holes in the domain of the function: let $\Omega \subseteq \mathbb{R}^d$ be non-empty and open and let $f: \Omega \to \mathbb{R}^d \to \mathbb{R}^d$ be a continuously differentiable vector field whose Jacobi matrix is symmetric. If $\Omega$ is, for instance, simply connected, then $f$ is a gradient, but there are more general domains $\Omega$ where this is not always true.

$\endgroup$
7
  • 3
    $\begingroup$ The existence & non-existence of antiderivatives plays a huge role in physics, where the force fields that are the gradients of scalar functions are so-called conservative forces for which a notion of conservation of energy can be defined. $\endgroup$ Feb 23 at 3:03
  • 1
    $\begingroup$ This is also the origin of de Rham cohomology, which is (in my opinion) the most natural entry point into topology. We can actually see a hint of the story in a freshman multivariable calculus class. $\endgroup$ Feb 23 at 18:56
  • $\begingroup$ @StevenGubkin: When you say "the most natural entry point into topology", I assume you're referring to algebraic topology? (Because the natural entry points to point set topology seem to be quite different.) $\endgroup$ Feb 24 at 16:42
  • 2
    $\begingroup$ I guess the main differences between our perspectives are: (a) Where I teach (in Germany) there is no "standard calculus track" - students are supposed to learn real analysis (including metric spaces) in their first year. (b) I believe we tend to think about different things when we say "a topological idea". I assume (please correct me if I'm wrong) that you mainly think about concepts related to algebraic topology. When I hear "topological idea", my first thought is "metric spaces and sequences don't suffice to understand convergence and continuity in a general setting". $\endgroup$ Feb 24 at 18:22
  • 2
    $\begingroup$ @JochenGlueck I was also first introduced to point set topology as a generalization of a metric space. Without a "killer example" it was hard for me to get excited about it. I agree that functional analysis provides great motivation! You might be right that my idea of topology is more "classifying manifolds" and less "interesting notions of convergence". $\endgroup$ Feb 25 at 22:26
5
$\begingroup$

The inverse function theorem on $\mathbb R$ is global: any differentiable function $f \colon \mathbb{R} \to \mathbb{R}$ satisfying $f'(x) \neq 0$ everywhere has to be one-to-one, and so it has a global inverse $f^{-1} \colon f(\mathbb{R}) \to \mathbb{R}$.

The inverse function theorem on $\mathbb{R}^n$ is only local: if $f \colon \mathbb{R}^n \to \mathbb{R}^n$ is $C^1$-regular and $Df(x)$ is invertible, then $f$ is invertible on some neighorhood of $x$.

One can appreciate the difference by looking at the example of $\exp \colon \mathbb{C} \to \mathbb{C}$, which is clearly not one-to-one (we have $\exp(0) = \exp(2\pi i)$) but its derivative is everywhere invertible.

$\endgroup$
4
$\begingroup$

One elementary property that fails already for functions of one variable with values in $\mathbb{R}^n$ is the chord-tangent property. For example, for the helix $$ h:t\mapsto (\sin t,\cos t, t),\quad t\in[0, 2\pi], $$ we have $h(2\pi)-h(0)=(0,0,2\pi)$, however at no point on $[0,2\pi]$ is $h'$ equal, or even parallel, to $(0,0,2\pi)$. Of course, we mostly care about the chord-tangent property because it gives the bound $|h(b)-h(a)|\leq |b-a|\max_{(a,b)}|h'|$, which still holds in higher dimensions.

$\endgroup$
1
  • 1
    $\begingroup$ A surprisingly useful extension of your bound is that for every $z$ there exists $\xi \in [a,b]$ such that $|\frac{h(b)-h(a)}{b-a} - z| \leq |h'(\xi) - z|$. $\endgroup$
    – mlk
    Feb 23 at 9:34
3
$\begingroup$

What about the generalized Stokes' Theorem ? In some sense this is a story about how Calculus III with it's three dimensional-centric formulae fails to generalize that story for arbitrary vector fields in higher dimensions. Why? Because something beyond vector fields are needed to understand the generalization of the FTC for higher dimensions.

In particular, we must introduce differential forms in $\mathbb{R}^n$. The degrees of freedom in a differential $p$-form in $\mathbb{R}^n$ is $\frac{n!}{p!(n-p)!}$. We need that many scalar functions to define a $p$-form. It just happens that $p=2$ and $p=3$ forms in $\mathbb{R}^3$ are both in one-to-one correspondance with vector fields. No such luck in higher dimensions in general, only the $1$-forms and $(n-1)$-forms are in direct correspondance with vector fields. Otherwise, the other objects described by $p=2,\dots , (n-2)$-forms are new objects which are not just another take on vector fields.

These new objects are fluxes. Yet, these play the role vector fields did in lower dimensions. For example, in electromagnetism in four spatial dimensions you'd find a Faraday tensor formed by more magnetic components than electric components.In short, the FTC does not generalize for vector fields, it generalized for differential forms. It is a happy accident of low dimensions that we didn't need differential forms to understand the FTC. $$ \int_M d\alpha = \int_{\partial M} \alpha $$

$\endgroup$
3
  • $\begingroup$ "FTC does not generalize for vector fields, it generalized for differential forms" is a large overstatement: the (generalized) Stokes' theorem is actually equivalent to the divergence theorem, up to technical differences (the latter involves a Riemannian metric, while the former assumes the manifold to be orientable). Note that in the theorem we're only considering an $(n-1)$-form on an $n$-manifold, which corresponds to a tangent vector field. $\endgroup$ Feb 28 at 21:56
  • 1
    $\begingroup$ @MichałMiśkiewicz but, I would think within the context of a particular $n$-dimensional manifold we might consider integrals of $k$-forms over submanifolds and as such there is a wealth of cases not present in low dimensions. ($2\leq k \leq n-2$ to keep it interesting I guess) $\endgroup$ Feb 29 at 5:25
  • $\begingroup$ Oh yes, this is true. $\endgroup$ Feb 29 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.