37
$\begingroup$

Background: I am fairly new at teaching, and in every subject I have taught, I have had difficulty estimating the length and difficulty of an exam. I need to write an exam for a university senior-level knot theory course. The exam must last one hour due to university constraints.

What is the most effective way to write an exam that must fit a given time constraint?

$\endgroup$
  • 7
    $\begingroup$ +1 I love this question. This is real shop talk. $\endgroup$ – benblumsmith Mar 14 '14 at 19:48
23
$\begingroup$

My background is in high school teaching, so my experience may not directly transfer, since the types of exams are different.

However, I have found a very useful rule of thumb to be this:

After writing the exam, I sit for it myself, i.e. I sit down to write down full answers in one sitting. Most students will take six to eight times as long as I did.

$\endgroup$
  • 2
    $\begingroup$ I really like the strategy. However, the problem is: If the one who created the exam wants to solve the exam for himself, he already knows how to answer the questions (If he has to calculate a lot of things, this is no real advantage due to the factor). I think, this really makes sense if someone else is trying to take the exam (at least if there are some questions where you have to think more than write or calculate). $\endgroup$ – Markus Klein Mar 14 '14 at 19:58
  • 6
    $\begingroup$ Definitely if the problems involve coming up with some ideas, and I already know what ideas I want the students to use, then of course I have an "unfair advantage." But I'm addressing the question from a purely practical standpoint. In my experience, across multiple types of exams, this rule of thumb held pretty well; the factor of 6 to 8 made up for the "unfair advantage." If you are in a situation where it's not realistic to get other people to test-drive your exams before you administer them (this was always my situation), then this rule of thumb still gives you something to go on. $\endgroup$ – benblumsmith Mar 14 '14 at 20:15
  • 5
    $\begingroup$ For a first year engineer's exam, one (Assistant Professor) would write the test, and then some of the TAs would solve it. Our multiplier to get the real time was 3 to 4. $\endgroup$ – Roland Mar 28 '14 at 19:58
  • $\begingroup$ I often use this method with a multiplier of 3 or 4. $\endgroup$ – Andreas Blass Mar 30 '14 at 2:40
  • 2
    $\begingroup$ Ah, the venerable rule of $2 \pi $. $\endgroup$ – Vandermonde Nov 2 '15 at 2:45
24
$\begingroup$

Here's a few useful strategies:

1) Once you've written the exam, time how long it takes you just to write down the answers. That gives you a baseline on how much time someone (you!) who already knows all the questions would take to answer the exam. This is particularly useful for exams that are writing-heavy (like proofs and "explain this" questions).

2) If you have TAs, ask them to take the exam. Give them some fraction of the time you intend to give the students. Depending on the topic/course, this could be 1/5, 1/3, 1/2. Your TAs (arguably) should get near perfect scores on your exams (unless you want a very difficult question). Make sure you tell them to actually time themselves and actually write the answers down. Also, make sure you actually review what they've written. Once, one of my TAs finished in the time he was given, and I hadn't reviewed his answers. It turned out (after the students took it) that he'd accidentally answered something incorrectly, and the exam was harder than I had intended. Ideally, you'd want to use undergraduate TAs for this, if at all possible, because they're "closer" in experience to the students in the course.

3) Try to mark your idea of a "difficulty level" for each question you write on the exam, as you add them. Generally, the best exams have a combination of "easy", "medium", and "difficult" questions. Certainly, if you don't balance this for your idea of what this means, the exam might end up skewed to one side. Then, if you have your TAs take it, ask them to rate the questions too. If their perspective is totally different from yours, that might indicate a problem.

$\endgroup$
  • $\begingroup$ Tacking another tip onto this: Time how long it takes to write and then explain the proofs to someone with little knowledge in the field. They do not need to understand completely, but simply writing it out then talking through it should give you some perspective on how long your students would take to work through a proof. $\endgroup$ – David G Mar 15 '14 at 6:53
  • 1
    $\begingroup$ @Skytso I really like this idea! You could probably even do it as a think-aloud to yourself too. So, tell the person who took the exam "pretend I don't know anything, and explain it to me". I'll definitely use this. :) $\endgroup$ – adamblan Mar 15 '14 at 15:27
11
$\begingroup$

I think there is no general solution, but here are some ideas which could help estimating the time:

  • Do you have access to old exams in the same subject or in something related? Look at them (inform yourself if the students were allowed to bring notes, books, etc.) and compare to what you want to do. Maybe you can also ask colleagues from a different univercity for their exams.
  • If you give homework during the term, ask the students how long they needed to solve the exercises. If you want to put a similar question on the exam, take only a certain amount of the time they said (The concept is not new to them; they should be prepared for the exam; plus students normally overestimate the time they needed [Since this seems to be an advanced class, this time should be quite fair].). You may also have a feeling how good the students are (or, at least, if there are some very good ones). A good student should be able to finish the exam in time.
  • Use multiple choice questions. If time is too short, many will take the option which makes sense to them in the first place. If there is enough time, they will maybe try to prove the statements in order to be sure. You can then adjust the time fact during the grading.
  • Use questions in the following style: "Please prove one of the following theorems"/"Perform a calculation with the following algorithm or write a C++ code for it". Since the student can choose, it is more possible that there is something he or she has actually learned. (Don't exaggerate on this. Students need more time to read and to decide alternative questions.)
  • Ask someone you trust (with background in the field) to perform the exam. Depending on the kind of questions, the ratio between a good student and a PhD student should be somewhere between 2 and 5 (the more "calculation" questions there are, the smaller the ratio; the more "proofs everyone in the field should know by heart" you ask, the bigger the ratio.). You may also stop the time how long your colleague needed for each question.
  • If you underestimate the time the students need, think about a plan B: What will you do (in terms of grading?) to balance the time issue? Is it possible to do so?
$\endgroup$
9
$\begingroup$

A useful rule of thumb I got when I started up here (which has turned out surprisingly accurate) is "Solve the exam yourself, multiply by four." In any case, when possible I schedule twice that (some time is lost until everybody is seated, exams are handed out, etc.; and there is always some straggler arriving half an hour late...). But we don't have regulations on exam lengths

$\endgroup$
  • 2
    $\begingroup$ I think that this is extremely useful in classes with calculations, but in a proof heavy class it would be significantly more difficult. $\endgroup$ – David G Mar 15 '14 at 6:51
5
$\begingroup$

Once I've written an exam, I take it to determine my rubric or how I'm going to grade each question. If a question has $n$ parts/steps to its solution, then that question will take about $n$ minutes. This is a rough estimate and some steps follow quicker than others, but it seems to average out and has worked well for me for the last handful of years. This works best in non-proof courses, but can also be used for very formulaic proofs. One good/bad aspect of this approach is that it gives students a hint as to how difficult/long the solution should be.

$\endgroup$
4
$\begingroup$

Srsly, the expansion factor should not be 4 or 10 but something like 40. If you are a quick, fluent, experienced person, and if we're talking about mathematics at any level beyond cook-book/filter calculus, and if we're talking about situations with kids who're still "not dry behind the ears" mathematically, a huger expansion factor is appropriate.

Indeed, as in other answers, one should be wary of self-deceit about the mere time to write out an appropriate response.

But, more insidiously, further, even for kids who do approximately know "the right answer", there will be a huge amount of dithering. And, there will be a huge amount of no-op self-doubt, etc.

In summary, as the level of the mathematics goes up, the expansion factor goes up, hugely. It is admittedly subtle in undergrad math, depending on circumstances, prior experience, etc. But, e.g., for PhD work, I allow an expansion factor of about 1,000. To explain why this is not a joke, and why this gives perspective on undergrad exams, I seriously comment that it is my (very considered) opinion that in starter-thesis situations, "the advisor" should "see" how to do any thesis problem suggested. And for the student this should involve much confusion, much reading, etc. So, if we ask about expansion factors, ... which, !!!, include self-education, etc., _of_course_ it'll take a novice 1,000 times longer than an expert. Hopefully one's advisor is sufficiently expert so that they can do things that much faster... as dis-heartening as this might be to a naive beginner.

When we dial-back this deconstruction to undergrad stuff, the lesson still has weight...

The dumbed-down summary's rhetorical version is "whatever factor you think is good, you should multiply by 5".

(But/and think how to avoid "time constraints"... which test things that are irrelevant...)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.