ryang has explained the issue, but I think it might be helpful to students to frame the issue in a more positive light--instead of "you can't use this substitution here", say "here's what you need to do to use this substitution." The problem is that $\tan x$ is not differentiable on $[0,\pi]$ because it's not even defined at $x=\frac{\pi}{2}$. So in order to use this substitution you have to find a way to rewrite the integral to avoid the troublesome point. In this example we can write
\begin{align}
\int_0^\pi\frac{1}{4+\sin^2x}dx&=\int_0^{\pi/2}\frac{1}{4+\sin^2x}dx+\int_{\pi/2}^\pi\frac{1}{4+\sin^2x}dx\\
&=\lim_{r\to\pi/2^-}\int_0^r\frac{1}{4+\sin^2x}dx+\lim_{r\to\pi/2^+}\int_r^\pi\frac{1}{4+\sin^2x}dx.
\end{align}
The purpose of introducing the limits is to keep $\frac{\pi}{2}$ out of the interval. Now your substitution works fine and you get
$$
\lim_{s\to\infty}\int_0^s\frac{1}{5t^2+4}dt+\lim_{s\to-\infty}\int_s^0\frac{1}{5t^2+4}dt.
$$
In this example since the integrand is an even function you can just double the first integral.
By explaining things this way, you don't have throw away a promising substitution and, by going through the process of choosing intervals carefully, you reinforce the idea of checking the needed conditions. My feeling is that by the time students are learning these kinds of substitutions they should already be strongly conditioned to recoil against any attempt to do a definite integral using the fundamental theorem of calculus over an interval where the integrand is not defined at every point. For an example of what can go badly wrong, consider how if you naively apply the fundamental theorem to
$$
\int_{-1}^1\frac{1}{x^2}dx
$$
you get obvious nonsense (a finite negative answer where the integrand is positive and, properly treated, the integral is infinite).