"[Myth] that exams are objectively graded. Daniel Stark and Edward Elliot sent two English essays to 200 high school teachers for grading. They got back 142 grades. For one paper, the grades ranged from 50 to 99; for the other, the grades went from 64 to 99. But English is not an "objective" subject, you say. Well, they did the same thing for an essay answer in mathematics and got back grades ranging from 28 to 95. Though most of the grades they received in both cases fell into the middle ground, it was evident that a good part of any grade was the result of who marked the exam and not of who took it"

The study the author is referring to is: "Starch D, Elliott EC. Reliability of grading work in mathematics." http://www.jstor.org/stable/1076246?seq=1#page_scan_tab_contents

This is pretty consistent with my experience, both as a grader and a student. The cited experiment was done in 1913, however. I am curious about more recent, broader studies about testing the variance of grading - especially for math exams such as those we give in college algebra or calculus.

The same studies are referenced here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4041495/

The only recent reproduction I have found was for the Starch/Elliot experiment about english papers, not math exams: http://pareonline.net/getvn.asp?v=16&n=17

• I don't have an answer to the actual question, but it would be highly surprising if the variation in scores was small. Even if we all agree on what counts as right and wrong, there are so very many ways to weight a mistake. Did anyone ever expect universally comparable grades? It seems like a strawman position.
Dec 29, 2017 at 5:40
• Even if the grading were consistent among teachers, there is still tremendous variability in test questions. The important thing is that teachers grade their own test consistently and for that many people use a Rubric Dec 29, 2017 at 5:59
• Multiple choice is the format the minimizes variance (to zero). It also is the least time intensive form of grading. But it has many other drawbacks. Because there is wide variation in how different people grade the same work, it's worthwhile to try to avoid having multiple people grade different problems or, if that is necessary, to use a common grade scale that attempts to reduce variance between graders as much as possible. Dec 29, 2017 at 15:07
• @Adam Regardless of what we expect (as educators), we treat grades as if they were universally comparable; for example because employers and educators look at GPA when considering candidates. (And because of the way they are used, students learn to see grades as universally comparable.) Dec 29, 2017 at 18:32
• @AreaMan I don't think that employers or educators do treat the same grade from different institutions the same. Within the same institution, perhaps.
Dec 29, 2017 at 19:49