Certainly the student should be aware of the expectations beforehand. To this end, I think there are at least two approaches. One is to specify on individual questions whether counterexamples should be explained, and the other is to establish (preferably from the outset, i.e., setting "norms" at the beginning of a course) the expectation that reasoning/justification for all answers will be provided.
More generally, I think explaining counterexamples is important. Let me give just two reasons.
1) Consider the following problem:
Is every whole number divisible by 6 and 8 also divisible by 6 $\times$ 8 = 48? If so, prove your answer; if not, please provide a counterexample. Answer: 24.
But what does such a student understand? Perhaps she just remembered 24 is divisible by both and realized it is not divisible by 48. Or perhaps she realizes that the issue is that the two divisors in this proposed rule are not relatively prime, and could correct it by using, e.g., 3 and 16 instead. Or, even more seriously, perhaps she has the deeper understanding, but accidentally writes 32. In this last case, the understanding is the deepest, but it would be tough to justify giving the response any credit.
Asking that the counterexample be justified ameliorates this potential problem. Personally, I would expect more than just an explanation of the particular counterexample, e.g., more than:
No, and 24 is a counterexample: It is divisible by 6 because 24 = 4 $\times$ 6, and it is also divisible by 8 because 24 = 3 $\times$ 8. But it is not divisible by 48, because 24/48 is not a whole number.
I would prefer to see an answer that mentions being "relatively prime" or something equivalent, and perhaps even one that gives an example of two numbers that would yield a divisibility rule for 48.
2) More subtle and perhaps not related to the specific question you have in mind (if there is one): Note that (speaking slightly messily) counterexamples can settle for all questions, but not there exists questions. The previous example is the assertion that all whole numbers (etc). If this turns out to be false, then it can be settled with a counterexample. If it turns out to be true, then it will need a further proof or justification. On the other hand, a there exists statement that is true might be settled with a single example. If it turns out to be false, then it will need a further proof or justification.
This latter point about proving/disproving by (counter)example in the context of for all vs. there exists assertions may seem a bit pedantic, but it is a place in which many students who are just beginning to understand ideas around "proof" can struggle quite a bit.