What is the right notation to use in multivariable chain rules?

Question

The following "chain rule" is in my multivariable calculus course:

If $f$ depends on $x$ and $y$, but $x$ and $y$ depend on $t$, then $\frac{df}{dt} = \frac{\partial f}{\partial x} \frac{d x}{d t} + \frac{\partial f}{\partial y} \frac{d x}{d t}$.

However, it doesn't apply to this question:

Suppose $f(x, y, t) = xyt$, with $x = t$ and $y = t$. Use the chain rule to find $\frac{d f}{d t}$.

Of course $f(t) = t^3$ so $\frac{d f}{d t}$ should be $3t^2$.

But if you apply the "chain rule" above, you end up with

$\frac{d f}{d t} = (yt)1 + (xt)1 = 2t^2$.

I need the result of my chain rule to look more like:

$\frac{d f}{d t} = (yt)1 + (xt)1 + (xy) = 3t^2$.

But I have no good notation for the missing "$xy$" here. Is it "$\frac{\partial f}{\partial t}$?" That's awful -- I can't write:

If $f$ depends on $x$ and $y$ and $t$, but $x$ and $y$ depend on $t$, then $\frac{df}{dt} = \frac{\partial f}{\partial x} \frac{d x}{d t} + \frac{\partial f}{\partial y} \frac{d x}{d t} + \frac{\partial f}{\partial t}$.

... can I? How do I fix this?

Answer 1

Do you ever introduce the Jacobian matrix, or the derivative of a function at a point as a linear map? This clarifies everything.

If not, and you must restrict yourself to "traditional multivariable calculus" notation, you could write the chain rule as $(f \circ \gamma)'(t) = \nabla f|_{\gamma(t)} \cdot \gamma'(t)$

In your example $f(x,y,t) = xyt$ so $\nabla f|_{(x,y,t)} = \langle yt, xt, xy \rangle $.

$\gamma(t) = (t,t,t)$, so $\gamma'(t) = \langle 1, 1, 1\rangle$.

We can now calculate $(f \circ \gamma)'(t) = \langle t^2, t^2, t^2 \rangle \cdot \langle 1, 1, 1 \rangle = 3t^2$.

Note: One must point out that the variable $t$ appearing in the argument of $f$, the argument of $\gamma$, and in the argument of $(f \circ \gamma)$ are all "different". Function definitions are universally quantified, and quantified variables have a scope which is limited to the sentence which they quantify. So even though they use the same letter, they are "unrelated".

I agree with Adam that the real difficulty here is paying attention to what the domain and codomain of your functions actually are, and also not confusing the scope of the variables in a function definition. The Leibniz notation confuses the scope of the variables inherently, and this does lead to endless confusion.

I have also seen $D_j f$ or $f_j$ used in place of $\frac{\partial f}{\partial x_j} $, and this clears up some confusion.

Answer 2

What's wrong with this?: $$ \frac{df}{dt} = \frac{\partial f}{\partial x} \frac{dx}{dt} + \frac{\partial f}{\partial y} \frac{dy}{dt} + \frac{\partial f}{\partial t} \frac{dt}{dt}$$ with $\frac{dt}{dt}=1$. It's what you would get if you had $f(x,y,z)$, except that $z=t$.

You do have to be a bit careful, though. You want to be clear that $f(x,t,t)=xt^2$ should either be thought of as $f(x,t)$ or as the composition of $f(x,y,z)=xyz$ and a curve in $\mathbb{R}^3$. (i.e. $\frac{\partial f}{\partial t}$ should be unambiguous at to which variable-slot is being differentiated.)

The general principle here is that when you write down something like $f(x,y,t)=\cdots$, you are defining a formula which works for any possible values of $x,y,t$ (where the formula is defined), regardless of any possible relations which might actually hold amongst these variables. This independence is what you need to define the partial derivatives. Otherwise, you cannot do the "hold all other variables constant" procedure in the definition. So the chain rule formula should not really care that $x,y=t$; it only knows that $f$ is a function of $x,y,t$. The formula shouldn't even really care that we are differentiating w.r.t. $t$. You just get $\nabla f \cdot \vec{v}$ where $\vec{v}$ is the vector of derivatives of the 1st, 2nd, and 3rd parameters whatever they may be called.

Answer 3

I guess you are aware of it, but since you don't say so explicitly: the chain rule on the top of your post does not apply to your example, since that $f$ is not a function of $x,y$ but of $x,y,t$. Also, as written, you are confounding the modern concept of "function" $f:\mathbb{R}^3\to\mathbb{R}$ with the original notion of "function of". This mix up leads to many of these "paradoxes".

If you work with the modern concept of function, the best solution is to use the notation $D_if$ suggested by Steven Gubkin (in fact, the $\partial/\partial x$ operation makes no sense when applied to $f:\mathbb{R}^3\to \mathbb{R}$, since such an $f$ does not depend on $x$). The correct way of writing the chain rule, with $f:\mathbb{R}^n\to \mathbb{R}$ and $(\gamma_1,\ldots,\gamma_n):\mathbb{R}^m\to\mathbb{R}^n$, is then $$ D_j(f\circ(\gamma_1,\ldots,\gamma_n))= \sum_{k=1}^n \left((D_kf)\circ (\gamma_1,\ldots,\gamma_n)\right)\cdot D_j\gamma_k $$ where $j=1,\ldots, m$. This is unfortunately quite distant from the thing shown in most calculus texts.

If, on the other hand, you work with the original idea of "functions of" (I've written elsewhere on MESE how to formalize that with modern functions on manifolds), then Adams answer is close to the truth. There is in principle nothing wrong with Leibniz notation $\partial z/\partial x$, except that it's not precise enough and you should also indicate which variables are held fixed, as in $(\partial z/\partial x)_y$. If you don't do this you eventually run into trouble, for instance if you try to write the chain rule with Leibniz notation in the context of reverse automatic differentiation (a special case of which is backpropagation in machine learning).

Answer 4

To clear up confusion from this, when I teach I make a massive emphasis of the fact that

a function takes in an ordered list of inputs

(and not something like 'variables'). Then I use notation like $D_1f$ to denote the partial derivative of $f$ with respect to its first input, $D_2f$ the second input etc. I go on about how each of these is a new function and not a procedure that you apply to the expression that computes $f$.

So in your example, there are 3 inputs to $f$. The chain rule as stated is clearly for a function that takes in two inputs. It therefore does not apply in a very literal sense, not because there's any actual subtlety.

---

My favourite nasty example is to define functions $f, g : \mathbf{R}^2 \to \mathbf{R}$ by something like $g(\alpha,r) = \alpha^2 + r$ and $f(r,\alpha) = r^2 + \alpha$ (I use something other than $x$ and $y$ deliberately to throw off any sense of the correct ordering). Observe that these are the same function in the sense that for any point $(x,y) \in \mathbf{R}^2$ we have $f(x,y) = g(x,y)$. So $$ g \equiv f, $$ right? But of course if you differentiate naively: $$ \frac{\partial f}{\partial r} = 2r \\ \frac{\partial g}{\partial r} = 1. $$

Answer 5

There is, as others have said, nothing wrong with $$\frac{df}{dt} = \frac{\partial f}{\partial x} \frac{dx}{dt} + \frac{\partial f}{\partial y} \frac{dy}{dt} + \frac{\partial f}{\partial t} \frac{dt}{dt}$$ Note that a partial derivative like $\frac{\partial f}{\partial t}$ is really to be understood as the derivative of $f$ with respect to its third variable, more so than it is the derivative of $f$ with respect to $t$. The fact that the first and second variable of $f$ are actually functions of the third is immaterial to the partial derivatives.

An alternative is to instead use $f(x, y, z)$, where $x, y$ and $z$ are all functions of $t$, and we just happen to have $z(t) = t$. In that case, I think you will agree without issue that the chain rule says $$ \frac{df}{dt} = \frac{\partial f}{\partial x} \frac{dx}{dt} + \frac{\partial f}{\partial y} \frac{dy}{dt} + \frac{\partial f}{\partial z} \frac{dz}{dt} $$ which after substitution turns out to be the same as the above.

Answer 6

One can fix this problem in another way - by expressing partial derivatives in a more natural form not involving quotients*.

Given $$f = xyt$$ we have $$df = yt\,dx + xt\,dy + xy\,dt$$ no matter how the variables $x$, $y$, and $t$ may or may not be related - and this simply reads:

The rate of change in $f$ is $yt$ times the rate of change in $x$ plus $xt$ times the rate of change in $y$ times $xy$ times the rate of change in $t$.

Any way you could imagine changing the variables $x$ and $y$ and $t$, this is true - it doesn't matter if maybe there's some secret variable $w$ that's out there but just ignored in the formula for $f$ and it doesn't matter if the given variables have some relationship among them or if some are actually constants. It's just true. It's also nice in the sense that the equation can be read out quite literally without assuming that everyone knows and agrees upon the meaning of "the rate of change of $a$ with respect to $b$."

If one notes that $x=y=t$ then it does follow that $dx=dy=dt$ so one can just substitute into the last equation to get $$df=yt\,dt+xt\,dt+xy\,dt=3t^2\,dt$$ as desired - although one could also get the result $df=3x^2\,dx$ or $df=3y^2\,dy$ or even $df = 3xy\,dt$ if desired - there are lots of ways to correctly write this, each corresponding to some way to describe the rate of change of $f$ in terms of some other rates.

Note that this is also quite flexible, since the original equation wasn't hiding anything - if I want to say $x=t^2$ then I can get $dx=2t\,dt$ and substitute those in for $x$ and $dx$. Maybe I'd then decide that I want $y^3=t$ and get $3y^2\,dy=dt$, which I could substitute in for $t$ and $dt$. I could even get fancier by asking that $x=ty$ and seeing $dx=y\,dt+t\,dy$ or even do this all with implicit equations. There is then no need to have a chain rule at all because the chain rule just becomes literal substitutions - corresponding to the idea that if I have some term of "the rate of change of $t$" and I happen to know that that rate of change is an expression of some others, I might as well use that expression.

There's a bit of sleight of hand here** - $f$ is no longer being spoken of as a function - but this is a handy point of view to understand (if only because many problems go away once you have two ways to look at them!) and one that is often useful, but seldom taught outside of differential geometry.

---

*I would argue that the notation $\frac{\partial f}{\partial x}$ is the source of the problem: in a single dimension, it makes sense that if you had $y=x^2$ you could write $dy = 2x\,dx$ and reasonably say that this should mean $\frac{dy}{dx} = 2x$ even if this doesn't formally make sense as a quotient.

This just doesn't work in more dimensions - in this context $\frac{\partial f}{\partial x}$ means "write $df$ as a sum of $dx$ and $dy$ and $dt$ and then tell me what the coefficient of $dx$ was" - which we can see to be $yt$ above, but there's a glaring problem in this notation: it requires that we talk about $dy$ and $dt$, but these variables are never mentioned in the partial derivative notation - it's easy to make the mistake then of writing $$df = \frac{\partial f}{\partial x}dx + \frac{\partial f}{\partial y}dy$$ which is essentially the trouble you are alluding to - it's not true because the change of $f$ doesn't depend only on $x$ and $y$, even though the notation fails to recognize this.

Perhaps equally bad is that when one has relationships such as $x=y=t$, the notation encourages us to forget that these things are related - despite one could quite happily notice that $\frac{\partial f}{\partial x}$ is not equal to $\frac{\partial (x^3)}{\partial x}$, despite the fact that the only difference between the two is a number of substitutions. This is a related issue to the previous one - the thing that changes between those expressions is the implicit basis used in this notation. The quantity "the change in $f$ with respect to $x$ alone" starts to seem somewhat arbitrary when one hits examples like this - and it comes nowhere near the intuitive sense of saying "the rate of change in $f$ is $3x^2$ times the rate of change in $x$" - which would, of course, be true, although not to the exclusion of other things being true as well.

I say this not to try to discredit a particular notation, but to point out what it means in full; this notation is sort of happiest when you have a function $\mathbb R^n\rightarrow\mathbb R$ and everyone agrees on how to give the domain coordinates - but then you get the tricky business that $f$ needs to be regarded as a function $\mathbb R^3\rightarrow\mathbb R$ and we are considering its values along a single line of that - which would be represented as some path $\mathbb R\rightarrow\mathbb R^3$ taking $t$ to $(t,t,t)$. As you observe, one needs to treat $x$, $y$, and $t$ with equal footing when working with $f(x,y,t)=xyt$, even though this feels counterintuitive when there are relations between them. Working things out this way gives the nice formulations with linear maps outlined in other answers - but this can be a bit tricky and is certainly not the only intuition we might apply to calculus. We just need to be careful what happens when we mix notation arising in different contexts.

**Of course, worth pointing out: there is a similar sleight of hand in the question! If you define $f(x,y,t)=xyt$ it does not make really sense to then say $x=y=t$ because those are not variables, but parameters - one could define the same function with different parameter names; if we were really talking about functions, then I could rephrase the question as "Let $f(a,b,c)=abc$. Suppose $x=y=t$, ..." where something is clearly off. The question is just asking for confusion to arise between the notation that treats "functions of several variables" and the notation that treats "various related quantities."