There are many proofs where the whole idea can be expressed by a picture and often naturally translated into a correct formal proof.
Often one has to argue with students that a picture is not a proof (but a good thing to get a proof started).
What is the most impressive (but at most undergraduate level) example, where the picture-proof seems okay, but the formal proof goes wrong, to convince a student in paying attention to do a formal proof?
This one can be presented to students at any level, really, although the way to explain "repeat to infinity" will certainly change for your audience. It can be used to teach them that weird things happen with limits and we can't just pass things through to the other side. It's also a good way to jumpstart a discussion of definitions: what's a proper way to define "convergence" here? Does it really converge to the circle?
The Curry Paradox is a classic. This animation resolves it:
This one can be shown to any student that understands the distance formula, and has a willingness to think about $n$-dimensional space for $n>3$.
(It is, more specifically, an indication that "inducting on pictures" is invalid, and less so of a "pay close attention to the picture". But I think it's striking enough to warrant inclusion here.)
Draw four circles with $r=1$ on the plane at the points $(\pm 1,\pm 1)$. Draw the box which perfectly encloses these circles, i.e. a square with side length 2. Now, draw a circle that is tangent to all four of the circles. Notice that this circle is inside the box.
You can repeat this for $n=3$. Draw spheres with $r=1$ at the eight points $(\pm 1,\pm 1,\pm 1)$. Draw the box that encloses these, i.e. a cube of side length 2. Now, draw a sphere that is tangent to all those spheres. Again, you can see it sits inside the box.
Repeat this in higher dimensions. When you get to $n\geq 10$, the "inner" sphere leaks outside the box!
Not exactly an answer to your question, but my favorite example where a picture proof can go wrong is the fake proof which shows that every triangle is isosceles.
The missing square puzzle is a nice one. Maybe it doesn's fit your question perfectly, since no one would use this for a proof. But you might take it as a proof for the theorem, that the area of a triangle is not well-defined.
See wikipedia. However, this parallels András Bátkais answer.
Maybe, one could argue that $f(x)=x^2-2$ has a rational zero using the number line. Just imagine its graph in $\mathbb{Q}^2$. You cannot "see" whether the completeness axiom of $\mathbb{R}$ is given or not. Historically, this problem was fixed by introducing the completeness axiom. Maybe many problems of "proof by picture going wrong" were solved by choosing the right axioms.
Another famous example is cutting the chocolate bar to increase its size, like in this example.
Since essentially the same answer has been given afterwords by others, let me add an other one in the same spirit which is a fake picture proof of the Pythagorean theorem.
I found a page with a few fallacious geometric proofs:
They both rely on inaccurate drawings of a geometric construction.
If this is already posted, my apologies, but, I didn't see it. I'm fond of this example as a warning against following intuition through limits:
The idea is that if you follow the purely horizontal or purely vertical paths in a manner which goes like a staircase from bottom to top then the total path length is always $3+4$. Imagine this picture continuing to make the stairs close to the hypotenuse smaller and smaller then as this process continues we obtain stair-steps arbitrarily close to hypotenuse. But, the path length of this sequence of paths is the constant $7$. The limit of a constant sequence is constant hence the length of limiting path of the sequence of paths must also be $7$. Furthermore, intuitively it is geometrically clear from the picture the path limit is identical to the hypotenuse. The length of the hypotenuse is given by $\sqrt{3^2+4^2} = 5$. Thus we have proved, $5=7$.
Of course this is not my idea, I'm not sure where this originates. Another nice picture for this is found at futility closet, pthagoras disproved. I like setting $a=3$, $b=4$ to make it easier for students to follow.
I believe the following "common false belief" suggested by Tao as an answer to this now famous Gowers' MO question can be also mentioned here.
The closure of the open ball of radius r in a metric space, is the closed ball of radius r in that metric space.
I guess if you check other answers of the Gowers' question, there would be some more of these.
I am very curious to learn the flaw in a Beth El "positive real numbers as pairs of reciprocals" diagram which seems to imply there are right triangles to which the Pythagorean Theorem may not always apply. More specifically, the diagram at HTTP//Mobius-ity.us/BENums.png uses an unorthodox but seemingly mathematically valid way of portraying the positive real number line. It does so by using a horizontal base part of the positive real number line that goes from 0 and comes to 1 and then it coninues on up by using a vertical elevation part of the positive real number line that vertically goes from 1 and comes towards infinity.
If, e.g., per the diagram, you use the value x=2, you get a 1/2, 1, 2 right triangle, one for which the length of its hypotenuse seems to be the taxicab metric 2 - 1/2, not the Pythagorean length. That seems to be so because each such right triangle's three vertices are by definition on a single one-dimensional line, even though that line looks two-dimensional IF and when you choose to disregard the important fact that the right side of each such triangle begins at 1 and not 0.
