I'm curious if anyone knows of any books, studies, or other resources on the philosophy of creating and grading mathematics exams. After working as a graduate TA for 4 years and dealing with a wide variety of issues creating and grading precalculus and calculus exams, I feel like I've developed a lot of personal opinions about how things "should" be done. I've even had quite a few "discussions" (borderline arguments) about exam fairness and so forth, so now I'm wondering if there is any well-repsected consensus opinions out there. I'm being deliberately non-specific, so perhaps this question might work best as a community wiki?
Since you remark that your question is "deliberately non-specific," here is a (necessarily) incomplete response: First are two links to documents about assessment that might be of interest, and then two grading schemes that I have encountered in mathematics courses.
As far as the philosophy of creating examinations, early work on this was done by former AERA president Michael Scriven. In particular, he was the first to distinguish between formative assessment and summative assessment; this was done in "The Methodology of Evaluation" (1966) which can be found in full (as a pdf) here. The citation is:
Scriven, M. (1966). The methodology of evaluation (Vol. 110). Boulder, CO: Social Science Education Consortium.
For a more recent discussion of formative vs. summative assessment, see, for example, the following:
Taras, M. (2005). Assessment–summative and formative–some theoretical reflections. British Journal of Educational Studies, 53(4), 466-478. Jstor link.
Here is a relevant excerpt from the latter link:
Insofar as creating (mathematics) examinations is concerned, it will be important for you first to decide whether you intend the assessments strictly as summative instruments, or whether you wish them to serve a formative purpose as well.
Here are two that I have seen; I am sure other MESE users could provide many others.
Indicate to students roughly what will be covered; give three examinations; multiply the better two scores by 3, then take an average of the tests (i.e., if scores are $a \geq b \geq c$, then use $(3a+3b+c)/7$ as the student's average). Subsequently, use the course final as at most half of the student's grade; in particular, it can be worth more than half of the grade if the score is higher than the average going into the test.
Set the total number of points for a course at $N$ and allow for some fixed number of points that can be earned during the semester (through examinations, quizzes, etc). Make the final examination worth $N-($student's earned points$)$. The consequence here is that everyone enters the final with the possibility of getting full marks for the course, but also wanting to have accumulated points during the term.
There will be definitely some material on assessment in school. Since my research focus is on tertiary education, I present some resources from calculus and post-calculus:
- Abramovitz, B., Berezina, M., & Berman *, A. (2005). How not to formulate multiple choice problems. International Journal of Mathematical Education in Science and Technology, 36(4), 428–437. http://doi.org/10.1080/0020739042000322655
- Bergsten, C. (2002). Critical factors and prognostic validity in mathematics assessment. ICTM2, at Crete. Retrieved from https://www.researchgate.net/profile/Christer_Bergsten2/publication/228528166_Critical_factors_and_prognostic_validity_in_mathematics_assessment/links/0046352a4fa67a25e4000000.pdf
- Brown, G. A., Bull, J., & Pendlebury, M. (2013). Assessing Student Learning in Higher Education. Routledge.
- Challis, N., Houston, K., & Stirling, D. (2003). Supporting Good Practice in Assessment in Mathematics, Statistics and Operational Research. University of Birmingham. Retrieved from http://cebe.cf.ac.uk/sites/default/files/supportinggoodpractice.pdf
- Cretchley, P. (1999). An Argument for More Diversity in Early Undergraduate Mathematics Assessment. In Proceedings of the 2nd Australian Symposium on Modern Undergraduate Mathematics. Queensland, Australia. Available online at: http://www.sci.usq.edu.au/staff/spunde/delta99/Papers/cretchl.pdf. Citeseer. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.159.4387&rep=rep1&type=pdf
- Griffiths, H. B., & McLone, R. R. (1984). A critical analysis of university examinations in mathematics. Educational Studies in Mathematics, 15(4), 423–442. http://doi.org/10.1007/BF00311116
- Iannone, P., & Simpson, A. (2013). Students’ perceptions of assessment in undergraduate mathematics. Research in Mathematics Education, 15(1), 17–33. http://doi.org/10.1080/14794802.2012.756634
- Iannone, P., & Simpson, A. (2015). Mathematics lecturers’ views of examinations: tensions and possible resolutions. Teaching Mathematics and Its Applications, 34(2), 71–82. http://doi.org/10.1093/teamat/hru024
- Shorter, N. A., & Young, C. Y. (2011). Comparing assessment methods as predictors of student learning in an undergraduate mathematics course. International Journal of Mathematical Education in Science and Technology, 42(8), 1061–1067. http://doi.org/10.1080/0020739X.2010.550946
- Smith, G., & Wood, L. (2000). Assessment of learning in university mathematics. International Journal of Mathematical Education in Science and Technology, 31(1), 125–132. http://doi.org/10.1080/002073900287444
- Staff, N. R. C. (1993). Measuring What Counts: A Conceptual Guide for Mathematics Assessment. Washington: National Academies Press.
- Tallman, M. A., Carlson, M. P., Bressoud, D. M., & Pearson, M. (2016). A Characterization of Calculus I Final Exams in U.S. Colleges and Universities. International Journal of Research in Undergraduate Mathematics Education, 2(1), 105–133. http://doi.org/10.1007/s40753-015-0023-9
- Thomas, M. O. J., & Klymchuk, S. (2012). The school–tertiary interface in mathematics: teaching style and assessment practice. Mathematics Education Research Journal, 24(3), 283–300. http://doi.org/10.1007/s13394-012-0051-6
These papers maybe supplement the post of Benjamin Dickman as they are mostly specific to mathematics. I guess not all, but maybe some may help you.