The derivative of $x \mapsto |x|$ at zero, or, say, the mean (not in a principal-value sense) of a Cauchy-distributed variable or the limit (in the finite real line) of $x_n = 2x_{n-1}; x_0 \neq 0$. Symmetry considerations show the result must be 0 if it were to exist, but to casually assume existence/convergence gets one burnt in places such as these.
This can also be used to show that while $f''(x) = \lim_{h \to 0; h > 0} \frac{f(x-h) - 2f(x) + f(x+h)}{h^2}$ and similar formulae can be used to calculate higher-order derivatives, they don't make good definitions -- applied to, e.g., the function that is -1 for negative arguments, +1 for positive arguments, and 0 at zero, you find the pathologies cancel and reach an absurd conclusion. (Or more simply $f'(x) = \lim_{h \to 0} \frac{f(x + h/2) - f(x - h/2)}{h}$ on the absolute value, as OP notes below.)
One can interpret the fundamental theorem of the calculus in various forms as another instance of this phenomenon. The statement typically requires that the integrand be integrable0, so one has, example,
Let $-\infty < a \leq b < +\infty$, and let $f,F : [a,b] \rightarrow \mathbb{R}$ be functions. If
(I)
$F$ is continuous (everywhere), if
(II)
$F'(x)$ exists and equals $f(x)$, except at a finite number (possibly none) of points $x \in [a,b]$, and if
(III)
$f$ is Riemann integrable,
then $\int_a^b f(x) \, dx = F(b) - F(a)$.
Viewing this as $\big( \text{(I) and (II)} \big) \Rightarrow \big( \text{(III)} \Rightarrow \text{formula} \big)$, the theorem rules out all but one number as a possible value of the integral, but kids tend to neglect or forget the subtle difference and claim, e.g., that $f : [0, 1/\pi] \rightarrow \mathbb{R}$ given by $f(0) := 0 , f(x) := \sin (1/x) - (1/x) \cos (1/x)$ has Riemann integral $\lim_{t \to 0; t > 0} \left( \left( x \sin (1/x) \right) \bigg|_t^{1/\pi} \right) = 0$. (No it doesn't! If the integral exists then it is zero as we have so lovingly described. Except it doesn't! (At least not properly, i.e., according to the definition they wrote down yesterday.) It's not even Lebesgue integrable, we just went over the $x \sin (1/x)$ function in an example on total variation. Aarg!) Similarly, if a real function has an (relative) interior extremum, then either it fails to be differentiable at that point, or the derivative there is zero. (I tend by nature to put failure and 'strings-attached' clauses and hypotheses first. It helps very little if any, but at least it seems not to impede. Oh well.)
0 even assuming (global) differentiability of the antiderivative, unless your formulation uses the gauge integral, in which case integrability very nicely becomes a conclusion as opposed to an assumption, but this integral is unpopular or passe or something (granted, it doesn't generalise as well as Lebesgue's owing to the structure it requires on the domain space, but (a) so long as one already is interested only in sticking to the real line, why not? and (b) I love it!! Are those alone not reasons enough??).