When analytic form of derivatives is preferred over numerical form?

Question

Is there a specific example when the analytic form of a derivative $\lim_{h\to 0}\frac{f(x+h)-f(x)}{h}$ is preferred to the numerical form $\frac{f(x+h)-f(x)}{h}$, $h \ll 1$? Are there cases when the numerical approach fails?

I'm trying to figure out why in the secondary school we focus on the analytic form and not on the numerical one. In my opinion, the numerical form is much easier to understand since it doesn't require a difficult mental transition from a secant to a tangent. In addition, the numerical approach provides good accuracy for calculating the derivative. Finally, calculators and computers use the numerical approach to plot derivatives so that they can be easily visualized.

On the other hand, we can say that the analytic form is important for definition of continuity, Taylor series etc. These topics, however, are higher level mathematics and can be left for the classes with advanced math or even college.

Answer 1

I think maybe if you try to write down precisely what you mean by $h<<1$ you will end up writing the definition for $\lim_{h \to 0}$.

From this point of view, there is really no difference between the two approaches; what you are calling the numerical approach is the same thing as the analytical approach.

---

For example, try to compute the derivative of $\frac{1}{(x-2.99)(x-3.01)}$ at $x = 3$.

The numerical approach will initially use $h = 0.1$, but you will notice that when you switch to $h = 0.01$, something qualitatively changes, so you go with $h = 0.001$ instead, and then $h = 0.0001$, and then you are convinced that things are going well.

What you are doing is finding the limit of a sequence, letting $h \to 0$ by using the sequence $h_n = 10^{-n}$.

Answer 2

I'm reading the difference between your two formulae as the fact that, in the first, you have a limit and, in the second, you compute the difference quotient for a single, small, value of $h$.

When we talk about the analytic derivative we usually are in the situation that $f$ has some definite, known, formula. That often allows you compute the limit algebraically. Students do encounter a few more cases like $e^x$ and the trig functions which require additional lemmas but in either case, you end up with collection of rules which transforms a formula to another formula. This is good because we get the value of the derivative everywhere at the cost of a fairly small amount of work.

Numerical differentiation is an approximate version which is useful in the case where we have some function, but we don't know a formula for it. For example, the function is known at a finite sample of points; like data from an experiment. Since you don't know a formula for the function or its value at most places, you cannot take a limit. You don't get as much out of the numerical derivative and it will not be perfectly accurate, but it is all you have available to you when you are working with discrete data.

See https://en.wikipedia.org/wiki/Numerical_differentiation for some error estimates and some multi-point numerical methods with greater accuracy.

Answer 3

To stress a point in Adam's answer: try to study the variations of $x\mapsto x^3-2x^2+x-1$ without the analytic form of derivative. You will do a large amount of computation and most student will likely make an error somewhere. With the analytic form of differentiation, this is very simple.

As far as numerical computation go, most mathematician should stop assuming they know how computers do get to compute things, because most of the time they don't. I believed that characteristic polynomial was useful to diagonalize matrices, while in fact it is the other way round: characteristic polynomial are used to compute numerically roots of polynomial.

In the case of derivative, to compute a derivative by the ratio $\frac{f(x+h)-f(x)}{h}$ to a given precision, you need a crazy precision on the computation of $f$, this is really slow. I guess many methods have been studied, but when your function is actually computed by a processor, then a beautiful method exists. It consist in taking the raw definition of the function, which ultimately reduces a (complicated) combination of the few operations that a processor can perform on floats. Then you replace all these basic operations by their equivalent in first-order calculus, where numbers are replaced by pairs $(x,dx)$. For example the product $(x,y) \mapsto x\times y$ is replaced by $((x,dx),(y,dy))\mapsto (x\times y, x\times dy+y\times dx)$ (here I use the notation $dx$ but this is a float). Once this substitution is done, the algorithm that got $x$ as entry and gave $f(x)$, now gives $(f(x),f'(x))$ whenever it is fed $(x,1)$. Voilà! The beauty of it is that it runs only a factor more slowly than the original algorithm, while using the slope approximation takes a completely different order of magnitude.