4
$\begingroup$

Do you think that 10 options for multiple choice questions is too much?

I am talking about Calculus Courses, if it makes difference.

Thanks

$\endgroup$
  • 3
    $\begingroup$ Responding by comment because I have no data, but I imagine this would be very tedious for the student. If they do all the work of determining the correct answer, they still have to check against 10 answers to see which one corresponds to their answer. $\endgroup$ – Steven Gubkin May 5 '18 at 22:27
2
$\begingroup$

I would say it completely depends on what is being asked. For example, something simple like "What is the derivative of $cos(x)$ with respect to $x$?" could have as many as 10 options, because no real work or computations are needed -- either you know this one or you don't. [Assuming, of course, that you don't include both $-sin(x)$ and $-\frac{\pi}{180}sin(x)$ as options.]

Because answers in calculus can often be simplified, it would be tedious for a student to have to verify if their answer was among 10 different options, just in a different form. Therefore, I am fine with a large number of possible options if they are fairly simple but obviously distinct.

$\endgroup$
6
$\begingroup$

Presumably, you are trying to reduce the chance that someone will get a correct answer by guessing. In that case, I think that 10 options are more than you need. Suppose that we have an $N$ question multiple choice exam, each of which has $M$ options. Then a randomly guessing student gets $Q$ questions correct, where $Q$ is sampled from the binomial distribution $B(N,1/M)$. If we define 60% as passing, what's the probability of passing for some values of $N$ and $M$?

$$ \begin{array}{c|c|c|c|} & N=5 & N=6 & N=7 \\ \hline M=3 & 0.2098 & 0.1001 & 0.0453 \\ \hline M=4 & 0.1035 & 0.0376 & 0.0129 \\ \hline M=5 & 0.0579 & 0.0170 & 0.0047 \\ \hline M=6 & 0.0355 & 0.0087 & 0.0020 \\ \hline M=7 & 0.0233 & 0.0049 & 0.0010 \\ \hline \end{array} $$ So there is maybe a case for moving from 4 to 5, if that's what your default was before.

Of course, the randomly guessing student model is essentially assuming that the student knows nothing, which is hopefully not true. They will be able to eliminate some. I don't think that I, or anyone else, would be very good at generating 9 incorrect, but plausible, options. It would end up something like
"What is $\int 2x dx$?"

1) $x^2+C$

2) $x^2$

3) $2x^2$

4) $2$

5) $2dx$

6) 42

7) aardvark

8) hat

9) ¯\ _ (ツ)_ /¯

10) don't pick this one, it is not the answer

So there are only 4 or 5 possibilities that are not obviously wrong.

$\endgroup$
3
$\begingroup$

There's a fair amount of research about this, mainly from psychology and medical education as well es test development research for high stakes tests.

In practical development, I usually resort to the "gold standard" and either collect sample answers analyzing the frequency and types of errors of students. Or I consult research articles on common misconceptions and typical mistakes to guide the formulation of distractors. In this sense:

"Done right: 3 is enough." (see e.g. Haladyna et al (2002) and Rodriguez (2005)).

I usually go with 4 Options on the most frequent mistakes (or anticipated from them). sometimes typical "slips of the pen"-mistakes make valid distractors as well (such as a change of "+" and "-" sign in answer).

Upshot: In the answer given by Adam, take the 4 or 5 options that are not obviously wrong and leave out the rest.

$\endgroup$
  • $\begingroup$ Thank you for the references! I think that this is a very nice answer. As to my own: Honestly, I think that even by answer 4, my options were stretching it a bit. The first 2 wrong answers correspond to conceivable mistakes, but any student answering #4 or 5 is profoundly lost. $\endgroup$ – Adam May 9 '18 at 13:59
2
$\begingroup$

I do think that 10 options for a multiple-choice question is excessive.

A few things to consider: This will be outside the range that standard automation tools can handle (TestGen application, Scantron sheets, premade testbanks, etc.)

When I started teaching, I thought that using multiple-choice questions would be a time saver for my classes. But what I found was that writing (and organizing, scrambling) the distractor options took more time than I was saving. Especially if the problems should change each semester, the only way that's feasible is with an automated tool (see above). That was just with 4 options each; the 10-question proposal would be infeasible × 2.5 in my experience.

I would already recommend that at the level of college credit-bearing courses, that the questions be short-answer anyway. Feel free to grade the problems on a simple 3- or 5-point rubric each. I think you'll find that to be actually less work than trying to write 10 distractor options per question. And you'll also get to see where the actual problems and strengths of the students lie, as opposed to being masked through the multiple-choice filter.

$\endgroup$
  • 2
    $\begingroup$ It would be more helpful as a question if there was some indication why the guy asked this (why contemplating 10 response). Some of these Q&A really feel out there. $\endgroup$ – guest May 8 '18 at 3:21
  • $\begingroup$ @guest: I've had a colleague suggest the same thing. I think this was motivated in response to criticism that college courses shouldn't be multiple choice, but their wishing to continue avoid grading work. $\endgroup$ – Daniel R. Collins May 9 '18 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.