48
$\begingroup$

I am a young math educator. I've TAed four semesters of calculus for various instructors. Some instructors have required me to grade selected problems in homework sets. Another required me simply to note if it was turned in. Yet another did not require homework handed in at all. In all cases, the students were tested on weekly quizzes.

My impression was that the students learned just as well without having graded homework. The teaching burden was reasonably reduced without grading the homework and I always found the quizzes I wrote easier to grade. But I say this without knowing much. I would guess that more detailed comments are better than less but time is finite. So here is my question: Is it worth grading calculus homework?

Based on some clarifying questions in the comments, here's what I mean by "grading" and "worth it". Grading is collecting homework and specifically marking any number of problems for correctness or otherwise in greater detail. Not grading would be collecting homework but only marking for completion (for the whole assignment) or not collecting at all. Based on one of the answers I guess there is a middle ground of marking papers but not assigning "correctness" or a grade. By worth it, I am considering the efficiency of grading. I presume more effort or more skilled grading will produce better results. But if you spend too much time the gains will be marginally worse. To measure effectiveness, I guess exam scores, from the perspective of a TA, and actual understanding from a broader perspective are what we should consider. I'd allow alternative interpretations of these terms.

$\endgroup$
  • 6
    $\begingroup$ I'm not sure this question has an objective answer. Obviously it helps the students to grade the homework, but in some cases it may not be a very good use of time relative to other ways of helping the students. It also depends on the nature of the homework: if the homework is an end unto itself, then surely it should be graded; but if the homework is just practice exercises and there are answers in the back of the book, then grading becomes less important. $\endgroup$ – Jim Belk Mar 24 '14 at 3:21
  • 3
    $\begingroup$ You have a good point, @JimBelk. However, I'm wondering if there are significant studies out there that have looked at large lecture courses, and had only some section's homework graded (with other "control" sections), and compared final grades (or performance on the final). Although specialized to their assumptions about the nature of homework, I would be interested to see some actual statistics of that nature. $\endgroup$ – Brendan W. Sullivan Mar 24 '14 at 4:11
  • 2
    $\begingroup$ @brendansullivan07 That's a reasonable point. Actually, I suppose I there might also be some interesting "In my experience..." type answers that would "answer" the OP's question, in the sense that they would offer interesting insight relevant to the question asked. I withdraw my objection. $\endgroup$ – Jim Belk Mar 24 '14 at 5:54
  • $\begingroup$ I am looking for a strong case either way. At the very least I would want an analysis of possible benefits or problems. $\endgroup$ – abnry Mar 24 '14 at 13:29
  • 3
    $\begingroup$ When I say grading, I am referring to leaving no marks on the homework papers, other than perhaps noting it was collected. But I'm also allowing the considering of not collecting homework here too. As for worth, I guess I am consider the efficiency of grading. Do you get a good bang for your buck? I imagine you can grade and make very detailed comments, but after a while the time spent is not worth it. And calculus is boring to grade anyways. $\endgroup$ – abnry May 13 '14 at 2:59
53
+50
$\begingroup$

The evidence says no

What research I'm aware of is all about how giving any overall data about their own performance is actively harmful in promoting further learning. They learn considerably more from instruction about what to do differently in the absence of a grade or numerical score. To repeat; individual scores discourage learning! (Explanations for this correlation are about strong students relaxing and weak students giving up, all students concentrating on an emotional rather than academic response and ignoring any pointers.) The data says don't grade if you want more learning, instead instruct.

In my experience the earlier you do this correction/instruction-only feedback, the better. I've been delighted with the change in attitude its early use has resulted in, with students preferring to re-test than give up, and much higher pass rates (our course is assessed by a national externally marked test). Later in the course I've been able to mark tests with numbers, particularly near exams, but the foundational practice builds a culture of improvement post-testing rather than finality.

Note: To improve learning, provide instructions for how to improve work, without a mark

Warning: the research shows that any grading nullifies correctional/improvement feedback - don't bother doing both.

To clearly separate research findings from my own experience: research says the presence of any summative grade/percentage/score nullifies the effect of any formative advice/feedback/instruction, and that such feedback is far more effective at improving performance. My experience says that building an improvement culture early on by providing advice-only feedback with resits for underperformers can have a lasting impact on approach to learning, which allows them to ride out our numerical common test scores with their "not there yet, let's fix it" approach. We got the culture right first.

I like to tell them "You are not an arrow! If an arrow misses its target, it lies helplessly on the floor. You are a human - you have legs and a brain! If you missed the target, keep going. If you fell on your face, get up!"

Alternatives to grading

You absolutely need to check they've done the work and made a reasonable attempt at it. As some of our students pointed out in a candid after-the-exams conversation, "If we think there's a chance you won't check we did the work, we'll take that gamble, every time." This checking only takes seconds per student. The good news is Dylan William, an advocate of using feedback to improve learning says feedback should be more work for the student than for you, but what are some alternatives to marking and grading?

There are loads of ideas, but here are three.

Comment-only marking: a medal and a mission

Don't mark everything, point out a medal and a mission: something they did well, one thing to improve. If I came in your class and listed everything you did wrong as an educator you would be dejected and find your list of failings to rectify overwhelming. If instead I praise something and then tell you the most important thing to improve, you can handle that, and may well actually do it! Students are the same.

Next you should...

Quickly read through and assess where you think the student is at (eg strong & accurate, OK but error prone, clueless), and have a follow-up task or resource, but just set a few questions based on their ability: strong and accurate students get harder problems, clueless students get basic technique drill. If all students get the same number of follow up questions no-one feels penalised for what they did.

Peer assessment

Done well with thorough in-class training this can teach students to understand how they will be graded in the terminal exams, but more generally it spreads the mathematical authority around so that it's "is this valid mathematics?" as opposed to "did I write what the teacher wanted?"

Disadvantage: it's hard to train students to give good feedback that's not binary or numerical, and as I say, the number often drowns the improvement message in emotional response.

What is and isn't worth grading?

In my view, homework is for learning - I expect students to seek help (phone a friend, ask the internet, watch a video, read a book) and find out how to answer questions if they don't know. As a result their performance on homework is a very poor indicator of what they can do unaided, so formally grading it is misleading for you and them.

Tests under test conditions are a better indicator of what you can do in an exam, and giving good feedback about how to improve (i.e. not a number, letter and judgement, but something they can work on) is important and relevant, so I don't object to marking tests, I'm just very aware that summative numerical feedback or grading can encourage students to stop learning, and have found success through avoiding it in the first part of the course.

There's no reason to suppose I won't find holding out for longer even more effective in the future. (I won't hold out indefinitely, since one of my aims is that students understand how they will be assessed, and peer marking is the best way of teaching that. It's hard not to get numerical feedback out of a mark scheme!)

References/further reading

NCTM:Five “Key Strategies” for Effective Formative Assessment says:

The research on feedback shows that much of the feedback that students receive has, at best, no impact on learning and can actually be counterproductive. Kluger and DeNisi (1996) reviewed more than three thousand research reports on the effects of feedback in schools, colleges, and workplaces and found that only 131 studies were scientifically rigorous. In 50 of these studies, feedback actually made people’s performance worse than it would have been without feedback. The principal feature of these studies was that feedback was, in the psychological jargon, “ego-involving.” In other words, the feedback focused attention on the person rather than on the quality of the work——for example, by giving scores, grades, or other forms of report that encouraged comparison with others. The studies where feedback was most effective were those in which the feedback told participants not just what to do to improve but also how to go about it.

A nice introduction to comment only marking on a good blog here

'Inside the Black Box: Raising Standards Through Classroom Assessment' (Paul Black and Dylan Wiliam, Kings College, London, 1998).

People seem to reference Butler, R. (1988). 'Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance.' British Journal of Educational Psychology, 58,1-14. a lot but I haven't read it.

Search terms: Assessment for Learning (AfL), comment-only marking.

See here for an intro to AfL generally.

Where's the evidence?

(In response to comments challenging my statements about the existence or content of evidence.)

There are a number of educational researchers who advocate what is referred to as evidence based education. They have high standards for what constitutes a good study (you can read their selection criteria in their work) and review a large number of studies, and concentrate on effect size. Dylan William is a good example as he specialises to mathematics a great deal, and I recall his description of the comment-only evidence but have read too many papers, talks etc by him to locate which one(s) make the reference. John Hattie is another evidence-based education expert, and I'll quote him for those seeking reassurance beyond my experience in class and my recollection of the literature I've read.

In The Power of Feedback (Review of Educational Research March 2007, Vol. 77, No. 1, pp. 81-112 DOI: 10.3102/00346543029848) John Hattie introduces some shorthand:

Thus, there is a distinction between feedback about the task (FT), about the processing of the task (FP), about self-regulation (FR), and about the self as a person (FS). We argue that FS is the least effective, FR and FP are powerful in terms of deep processing and mastery of tasks, and FT is powerful when the task information subsequently is useful for improving strategy processing or enhancing self-regulation (which it too rarely does).

before going on to talk about where the evidence for comment-only marking, against grades, and the nullification of the effectiveness of comments by the presence of grades can be found:

The effectiveness of marks or written comments has also been investigated. There is considerable evidence that providing written comments (specific FT) is more effective than providing grades (Black & Wiliam, 1998; Crooks, 1988). In one of the early and influential studies, Page (1958) found that feedback in the form of short written comments rather than grades alone significantly improved the test performance of students in 74 classrooms (see also Cardelle & Como, 1981; Elawar & Como, 1985; McLaughlin, 1974). R. Butler (1987) demonstrated that grades can increase involvement, but they do not affect performance (relative to a no-FT condition). She also showed (R. Butler, 1988) that feedback through comments alone led to learning gains, whereas marks alone or comments accompanied by marks or giving praise did not. She claimed that such results called in question the whole classroom culture of marks, grades, gold stars, merit awards, competition rather than personal improvement. As will become a theme later in this article, feedback that mixes FS with FT is less effective than FT by itself.

The paper includes a considerable bibliography including of course the studies referenced in this extract, and if you'd like to delve further into the evidence than references you should download the paper and start investigating.

$\endgroup$
  • 19
    $\begingroup$ I think this may be the single most interesting answer I've read on this site so far. $\endgroup$ – Jim Belk May 13 '14 at 3:03
  • 5
    $\begingroup$ I really love this answer, but I think the comment about "tests under test conditions ..." Some students just don't do well during in class tests. Perhaps an exam that is half in class and half take home is more fair. Of course a lot of care has to be taken making the take home portion so it isn't google able and so the error prone and clueless can still demonstrate something. Oral exams are a great form of assessment that I think pair well with your HW philosophy, but unfortunately they are extremely time consuming. $\endgroup$ – WetlabStudent May 13 '14 at 5:50
  • 5
    $\begingroup$ This is an interesting answer, but I'm skeptical about some of the strong claims used, e.g., "research says the presence of any summative grade/percentage/score nullifies the effect of any formative advice/feedback/instruction." "Nullifies" is a strong word. I see some general informational references at the end of your post, but do you have any references to research that actually supports these very strong and categorical claims? To me this sounds more like an ideological credo, or possibly an overstatement of the actual research. $\endgroup$ – Ben Crowell May 13 '14 at 15:45
  • 3
    $\begingroup$ @BenCrowell I'm just a guy who read some stuff based on empiricism and found it worked. I'd appreciate it if you interacted with me on some other basis than that I must be wrong about the evidence in some way, or wrong about the nature of the conclusions, as it's starting to offend me. I'm really not making it all up, honestly. $\endgroup$ – AndrewC May 13 '14 at 22:25
  • 3
    $\begingroup$ Hi! I just was brought here via a flag. The discussion is interesting, but please do not go overboard. Perhaps if you do not find agreement now, let it at least sit for a while. Also there is always chat for more informal and longer exchanges. $\endgroup$ – quid May 13 '14 at 22:48
6
$\begingroup$

This is an "in my experience" answer. However, I should say that my experience is slightly odd in that I teach students from a "wide variety" or backgrounds (I have fewer than 10 students from any individual country in my foundation first year courses).

So, in my experience, grading is pointless but marking is useful (I would argue even "necessary"). This is because it is your first opportunity to give the students feedback on their work and how they present it, and it is also (probably) your only opportunity before their exam...

For example, I once had a student write three pages of mostly correct maths for an exam where some people used three answer booklets. They were a bright student, and they just passed the exam, but as answer books are anonymous you wonder if the anonymous person was cheating and they lost marks right, left and center for not showing their working. If I had given this student feedback then this would have been less of an issue. So, lesson learned, I will always set fresh-faced students work which I can mark and return to them.

$\endgroup$
5
$\begingroup$

I take homework to be "show you can do stuff with what has been taught" while exams are "show you understand what it means, and how it hangs together." In that sense, both should be graded as they asess different aspects of the student's mastery of the subject.

I agree that marking should be rather detailed (and furthermore complete solutions published soon after the deadline), and returned as soon as possible as feedback (that many don't pick up their work later is frustrating, "got the grade, forgot the subject" is all too common...).

Grading only part of the homework cuts down on the work, but requires several "equally hard" problems, and cooking them up is very complicated.

$\endgroup$
5
$\begingroup$

There seems to be a huge literature on this sort of thing, but most of it seems to be about younger students, whereas this question is about college students taking calculus. For a summary of the research on children, see this NCTM report. The research seems to be politicized and inconclusive, and there seem to be many mutually contradictory findings, which may have been influenced by styles and fads in education, e.g., many math and science teachers re-emphasized homework in the Sputnik era.

This paper by Sasser describes a study on college students in a remedial math class. Students were randomly assigned to two groups, who received identical instruction in the same room at the same time. One group was assigned graded homework and one was not. They were evaluated using the same pre- and post-test. The study showed a small but significant improvement in performance with graded homework. The paper also has some discussion of the previous literature on the topic.

$\endgroup$
  • 5
    $\begingroup$ The paper compares graded homework to NO homework, not ungraded homework. No-one is advocating not setting homework to calculus student. How would they practice or improve? $\endgroup$ – AndrewC May 13 '14 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.