Take the 2-minute tour ×
Mathematics Educators Stack Exchange is a question and answer site for those involved in the field of teaching mathematics. It's 100% free, no registration required.

Now, the title of the question definitely needs some background!

In some courses, everyone can get an A and it's no problem (especially later grad courses).

However, many times it is good to have a wide spread of grades. Most often, this is because they will use the material in future courses, and if they are not prepared, they should know it. In my current university, the engineering department requested us to make the grades lower because the engineers couldn't do calculus.

My problem is that I always overestimate the difficulty of my quizzes and exams and end up giving too good of grades at the beginning, and then face the task of either trying to lower their grades by being much tougher or not fulfilling departmental expectations.

What should I do when I realize mid-semester that students who haven't learned the material are still getting A's in a core class?

Like Jim said in the comments, this can be better put as,

How can I know how hard to make exams and quizzes?

share|improve this question
    
I think the only real way is to strive for some sort of consistency from year to year. Students do not change much (locally) in time, so if you give similar exams from year to year you can have a good spread of grades. –  Steven Gubkin Mar 26 at 0:44
4  
The problem you need to solve is that you overestimate the difficulty of quizzes and tests. Could you perhaps post a link to one of your exams? –  Jim Belk Mar 26 at 0:45
    
I'd be thrilled if all my students did well... –  vonbrand Mar 26 at 1:17
5  
If your exams and quizzes only give students As, how did you "realize mid-semester" that students "haven't learned the material"? On the flip side, if you have a way of finding out that the students have not learned the material, could you not try to use that somehow to help you design the exams? –  Willie Wong Mar 26 at 8:38
    
@vonbrand: It sounds to me like Brian is saying that his students are not actually doing as well as the grades suggest. –  Mark Meckes Mar 26 at 9:34

7 Answers 7

up vote 10 down vote accepted

The part of this question that raises red flags for me is the line:

However, many timesit [sic] is good to have a wide spread of grades.

Why? This seems backward reasoning to me wherein you know the distribution of grades that you want to give and are figuring out how to design the test to fit that distribution.

Your assessment should be criterion based. Look at your learning objectives for the course and design the assessment accordingly. For example, here is my scheme of work for one course. Note in the introduction how the skills are knit to the grades.

In designing the exam, the question that I have always before me is:

"What competence can I use this question as evidence for?"

I also imagine the following scenario. Suppose that a year later, another lecturer storms in to my office and demands to know why I gave a particular student a particular grade. What will I answer? That they scored 52 points? That they hit a particular number of standard deviations away from the mean? Or that in their exam then they demonstrated that they met the criteria for the given grade?

So when I have complete freedom to set the exam and the assessment criteria, then I design my exam by these principles. Each indecomposable unit on the test will be used as evidence for their grade, but each such unit comes with a maximum grade that it can be used for. So when assessing for an E, I ask "Has this student shown sufficient evidence that they deserve an E?" meaning that I count the number of answers rated "E" or above. If this is sufficient, they have secured an E. Next, I look at the "C" grade. Any answers rated only E are now dropped. Note that some answers could only get an E so these are automatically dropped. I do the same at the "A" grade. For "B" and "D", I take these to mean "almost the grade above". So a "B" is a "low A", not a "high C".

As an example, here's an exam designed on these principles. I don't say that it is perfect. Note the first question only asks about definitions and statements. So the answers to this question can only count for an E, but I give them lots of opportunities to show that they have achieved that level. The other questions basically allow for Cs and As (one part of each question can be given an A, I'll leave it as an exercise to work out which part).

In short, if you are worrying about the distribution of grades then you are worrying about the wrong thing. If you are worrying that the grades are not knit to the desired competencies of the course, then you are worrying about the right thing and the solution is obvious: test for the desired competencies, not for anything else. And never grade by just adding up points.

share|improve this answer
    
While this addresses some of the important points, one thing left out is the matter of length of the exam versus time given. This is often the hardest part to get "right", and it can be very hard to tell if the student ended up giving a poor answer due to time constraints or due to simply not being able to give a better answer. –  Tobias Kildetoft Mar 26 at 10:36
    
@TobiasKildetoft Isn't that addressed in the "time constraints" question? –  Loop Space Mar 26 at 11:01
    
Not sure what question you mean. I mainly wanted to point this out since you make it sound like getting the level of difficulty right for an exam is easy ("just test their competences"), but that while testing competences is certainly the right way to go, the distribution of grades (in a large enough class) is usually the correct way to see if you managed to get the length of the exam right. –  Tobias Kildetoft Mar 26 at 11:28
    
"What competence can I use this question as evidence for?" - I was thinking the opposite - I have a list of competences needing demonstrating, and must find questions that will do this for each listed competence. –  JoeTaxpayer Mar 26 at 11:35
    
@TobiasKildetoft I meant matheducators.stackexchange.com/q/805/112 . I certainly don't mean to give the impression that I think setting exams is easy. In fact, I'd say that it is harder to set an exam that tests competencies than one that is based on an accumulation system since it is always harder to count quality than quantity. But it is easier to know if your exam is a good one if it is based on criteria than accumulation. –  Loop Space Mar 26 at 11:36

You wrote:

My problem is that I ... end up giving too good of grades at the beginning, and then face the task of either trying to lower their grades by being much tougher or not fulfilling departmental expectations.

My advice is that, as a practical matter, things turn out much better when one is tougher in the beginning of a course, easing off if necessary only later on, rather than starting out easy and then hoping to get tough. Students are grateful when a hard course gets easier, but feel cheated when an easy course becomes difficult. Isn't this basic human nature?

You should set high standards initially---you may be surprised that many students meet them. And if they don't, then it is easy to relax a bit later when the situation is more clear. But it is extremely difficult to raise standards after the students realize that you probably don't really mean it; they will call your bluff.

For example, I often use a quiz-every-lecture format for assessment, and when I do, I make sure to announce it as quiz-every-lecture and start with a quiz at the second lecture, and really have one every lecture for the first few weeks, before I start relaxing to every-other-lecture or so (which was all along my real intention), once the culture has been established and the students are motivated to stay on top of their homework in preparation. If I were relaxed at first, then I think the students wouldn't be motivated to do the work.

share|improve this answer
1  
This is a very good answer. I'll up vote it tonight; I've used up all my votes for the day. –  Brian Rushton Mar 26 at 16:39

If all students gets good grades, this is a bad thing for the real good students (who will, at least for their grades, not be recognized). It is also bad for poor students since the get the feeling that the understood everything and don't have to repeat the material of the course again ("Hey, nothing is better than an A. Why should I repeat that?"). This seems to be true, especially if someone from the next courses is complaining.

That said, in order to answer

How can I know how hard to make exams and quizzes?

I would suggest to put a (very) hard question on the test; it should be something where you combine two topics of the course and where you can say: "If someone is able to this, he/she has a deep understanding and my respect." - In this case, they deserve the good grade. Even if no student was able to answer the hard question, you can tell the students that there are able to perform some basic things, but should repeat the course in order to have a deeper understanding which will be needed for the next courses.

I good rule of thumb (which I often use) is maybe the following: Construct questions in such a way that

  • If the students are able to perform standard questions which they have seen several times during the course and where the way you ask is completely clear (e.g., "take the derivative of the following (not sophisticated) function"). If there are a many questions where students have to calculate something, define the limit where a student passes the exam as that points minus a few points ("It's okay to have small mistakes in calculations").
  • Put in one very hard question. If someone fails to solve the question completely, he/she should not be able to get an A (But, if there is at least a small idea and everything was solved perfectly, an A should be possible).
  • The rest of the exam should contain questions with either a known way of questioning (but more sophisticated example) or questions with an unknown way of questioning which can be easily performed in a straight-forward way.

An exam more or less constructed in this will lead to a bigger scattering of grades (leading to have deserved good grades and to let students know how "good" they are compared to others which then maybe leads to repeating the course).

share|improve this answer
    
It also helps to start easy and make the questions more difficult as the exam goes. This not only prevents anxiety but it also prevents students accidentally spending large portions of the exam time on an earlier difficult question. –  Aleks Vlasev Mar 27 at 8:48

What should I do when I realize mid-semester that students who haven't learned the material are still getting A's in a core class?

This is the crux of the matter. I'd have no issue if a class got nothing but As. It would seem an outlier, but if the class were prescreened, i.e. a group of top performers who were all able to master the material, it's possible for them all to deserve an A. But your quote above implies otherwise. That those not quite understanding the material are still getting As. The questions on the exams need to be chosen in a way that to get them right, one must actually understand the work required.

share|improve this answer

In my current university, the engineering department requested us to make the grades lower because the engineers couldn't do calculus.

Now, I'm no expert in maths, making exams or anything, but why don't you just ask the engineering department what the students need to be able to do there exactly? Maybe even take some questions from there? If engineering is one possible followup class and they fail there miserably despite good grades, engineering should be the standard you should hold your students against.

share|improve this answer

When I design a test, I will typically designate questions (using the homework in the book as a guide) as basic skills, moderate, moderate-difficult, and separates A's from B's. If I am giving a 10 question test I will typically give 1 basic skill question, 4 moderate questions, 4 moderate difficult questions, and 1 separate A's from B's question.

If I see I made a test too difficult, then I will offer to let the students make corrections to earn back 1/4 of the points they lost. There is no partial credit on corrections, it's either right or wrong. If I find that I made a test too easy (non-normal distribution) then the next test I will either remove the basic skill question and replace it with a moderate difficult.

share|improve this answer

My exams usually are 4 to 6 questions (it depends on the amount of material covered, duration is designed for give or take one and a half hour, we schedule two hours to account for stragglers, people who are just slower, ... and the inevitable miscalculation).

Often the exam is a simple question (worth some 25 points) to check basic understanding of definitions and trivial application of the concepts. Often 3 to 5 short questions, graded mostly yes/no each. The rest of the questions range from moderately easy to moderately difficult, with a difficult question thrown in to "differentiate A from B" as was said here. To give leeway for minor mistakes, there are usually 120 points total (grades are 0 to 100 here).

Grades for each question (except the "did you even glance at the material?" one) are typically distributed with a peak on "very little" and another one at "good" spreading to "perfect." If it turns out like that, I'm satisfied. I'd like it more with just one peak at "good," but so is life.

Obviously much depends on selection of questions. Particularly critical is to make sure one doesn't repeat questions from previous years, students will have them (and solutions), whatever the policy of publishing them might be. It might be possible to alter a question enough to make it "not the same" but still "covers the same stuff, and isn't much harder/easier," but that has to be designed carefully. You can ask variants (or extensions) on work done in homework, gives an incentive to do it carefully. Works best if announced beforehand.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.