The other answers seem mostly to be about giving this sort of question in a proof-based course, but, in the spirit of your question (which seemed to be about more introductory courses), I have used this methodology with (I think) great success in Business Calculus, Calculus for Engineers, and Differential Equations (for Engineers). Basically, whenever I am preparing an exam question on which I find myself thinking "they'll all make the following important mistake", I convert it into a question of the form:
Your friend did the following ….
(30%) What mistake did your friend make?
(70%) What is the correct solution?
I haven't done any formal testing to see what impact it has on grades or, more importantly, understanding, but at the very least it forces them to recognise the mistake (rather than just making it, and getting no chance to remedy it).
Another, perhaps more pointed, tactic, which was suggested to me by a colleague, is to give students the chance to correct their own mistakes (submitted a week after an exam as part of that week's homework)—but to require them to explain what they did wrong and why, rather than just to give a correct answer. I like this in principle, but found it extremely hard to judge fairly whether something was a 'real' explanation.