Point Deductions - Exams and Quizzes

Question

With regard to an undergraduate statistics course, I am developing a standardized list of point deductions with the TAs (doctoral students) so that graders are consistent in what they are taking off intermediate points for. For example, most problems are 10 points total, and my proposed point deductions for intermediate math errors are (for example):

• -2 pts, erroneous +, - , *, /
• -2 pts, erroneous sign, e.g. 3.02 instead of -3.02
• -3 pts, failed to square, e.g. (x) instead of (x)^2
• -3 pts, failed to take square root, e.g. (x) instead of sqrt(x)

If, after grading, you discover on a particular exam that the final answers for five 10-point questions are incorrect because of only making a minor -2 point intermediate error, the student could conceivably obtain a score of 80% on an exam if they missed only -2 points per question (40/50).

However, in statistics, there is a contextual element to every question, not just solving for a numerical answer -- that is, in addition to the worked problem, students need to write a text-based response for the following:

• (2 pts) state whether the hypothesis test is significant or not
• (2 pts) state whether the null hypothesis is rejected or accepted
• (2 pts) state whether the p-value is less than 0.05 or not.

So if there was only one minor (-2 point) intermediate error made, causing an incorrect final numerical answer, the student will also incorrectly respond to the final text-based answers (above) as well.

Thus, would you also take off e.g. -2 points for an incorrect final numerical answer, as well as -6 points for missing the final text-based sub-items listed above?

In other words, would you only deduct -2 points for a complex (multi-step) algebra or calculus question if only a minor intermediate step was erroneous, or would you also deduct for having an incorrect final numerical answer as well?

Maybe I could propose to the TAs to augment the point deduction list with:

• -1 pt, incorrect final numerical answer
• -1 pt, state whether the hypothesis test is significant or not
• -1 pt, state whether the null hypothesis is rejected or accepted
• -1 pt, state whether the p-value is less than 0.05 or not.

Answer 1

When designing a grading system for a question on an exam, I would identify the key skills being tested on that question. This can involve looking at other questions on the exam: if there are other questions that test a skill, I like to weigh it less unless it's a vital skill. A good exam is actually created starting from a list of skills you want to test.

Then allot some points for each key skill being tested on the problem. For each skill, I would suggest either two possible scores (all or nothing) or maybe an additional "demonstrated a significant partial understanding of the skill" score.

I advise avoiding putting emphasis on skills which are not taught/reviewed in the course. So for instance, I won't deduct points for an arithmetic mistake since arithmetic is not a skill taught in the course. However there are similar skills that are taught: using the right formula, knowing how to use that formula (what plugs in where), and checking to make sure the answer makes intuitive sense.

What this means for multi-step problems is: if the student got step 1 wrong, they get points for step 2 if:

• Their answer would have been correct if the answer to step 1 had been what they said
• They demonstrated the skill in question

So, for instance, if they get the wrong p-value on a question, but interpret that incorrect p-value correctly to get the corresponding (but incorrect) interpretation, they get full points for demonstrating the "interpret p-values" skill. If they interpret their incorrect p-value incorrectly to get the interpretation of the correct answer, they don't get points for demonstrating the "interpret p-values skill".

Of course the best test of a grading system is to actually look at graded work and see if it gives reasonable scores. Try whatever you choose on some sample student work before locking it in.

Also, depending on the number of exams, questions, and TAs, consider having each TA grade a specific problem across all sections. This improves uniformity and distances the grader from the student.

Answer 2

I favor an additive grading scheme, where points are earned toward a possible maximum (say 10) instead of deducting points for the myriad possible mistakes one could make. Here, I would try to adopt a set of markers I am looking for and awarding points if they appear in the written work. This could help in standardizing your grading.

To avoid the situation you mentioned (where a student loses 6 points because they came to the wrong numerical conclusion and thereby has the wrong verbal interpretation), I might change your markers to see what to award points for, like:

• (+2 pt) correctly/appropriately calculate the p-value
• (+1 pt) state whether the p-value is less than 0.05
• (+1 pt) if answer above matches the calculated p-value
• (+1 pt) state whether the hypothesis test is significant or not
• (+1 pt) if statement above matches the calculated p-value
• (+1 pt) state whether the null hypothesis is rejected or accepted
• (+1 pt) if statement above matches the calculated p-value
• etc.

This way, a student can get the wrong p-value, but still answer the rest of the problem "correctly" and receive points.

Answer 3

I advise being less intricate and put less load on the graders. I personally would go with all, half or zero credit for every question.

*All is correct answer (and some reasonable explication, not an essay, but also not a bare number.

*Half shows some decent knowledge of the process, but founders partway. Or has a "dumb mistake" on the algebra/arithmetic.

*And zero is for a mess.

After all there is more than one question on a test. More than one test in the course. And more than one course that the students take. Life is statistical after all, so these things even out.

Don't invest too much time in intricate grading. The cost/benefit is not worth it.

Answer 4

Part of the problem is writing the exam questions in the first place. Others have noted that, when designing a grading rubric, you should identify what the key skills in the problem are. This seems backwards to me. First, identify the key skill that you want to test, and then write the exam questions which hit those skills.

Once the exam has been written, I would very much recommend that you make the rubric as simple as possible, given that you want consistent grading across a (possibly large) group of TAs. I typically grade on a 3 points scale:

• [3] The answer is nearly perfect.
• [2] The answer contains errors that are mechanical in nature (e.g. missing signs, incorrect computations, etc), but not conceptual. The mechanical errors are minor or are not central to the skill(s) being tested by the question.
• [1] There are serious mechanical errors and/or conceptual errors, but something correct or relevant has been written on the page, in a way which clearly demonstrates at least some conceptual understanding.
• [0] The answer is essentially ungradable (it is blank, or nonsensical, or whatever).

I will note that my [2] and [1] are essentially the 50% category in this answer. I think that it is worthwhile to distinguish between "dumb" arithmetic mistakes and more fundamental conceptual errors. That being said, my scheme is essentially the same idea—simple and quick to implement.

Experience has shown me that students really don't like this system. The students I work with are used to a grading scale in which an A is anything over 90%, a B is anything over 80%, and so on. Thus when they get 2 points out of 3 (67%), they feel like they are failing (since 67% is a D). However, my feeling is that [2] represents, roughly speaking, B or C level response, while a [1] represents a D or low C. This is something which has to be thought about when assigning letter grades—either add more "free" points into the course elsewhere, or grade on a different percentage scale, or accept that more students are going to fail.

If you are really stuck on a 90/80/70% scale, then (for example) remap [3] to 5 points, [2] to 4 points, [1] to 3 points, and [0] to 0 points.

If you want to weight different questions differently, continue to grade them on a 3 point scale, but weight them differently (easy-peasy). Students will be most happy if all of the questions are worth multiples of 3 points (because they don't really want to think about weighting).

In any event, the overall goal is to construct a grading scheme which is fast (you don't want your graders to have to spend a lot of time on things), and consistent (different graders should score a given response in the same way). Creating lots of deductions or opportunities for partial credit makes grading slower, hence I would tend to avoid it. Consistency is also easier to attain if there are fewer categories.