Do you think that 10 options for multiple choice questions is too much?
I am talking about Calculus Courses, if it makes difference.
Thanks
$\begingroup$
$\endgroup$
3
-
7$\begingroup$ Responding by comment because I have no data, but I imagine this would be very tedious for the student. If they do all the work of determining the correct answer, they still have to check against 10 answers to see which one corresponds to their answer. $\endgroup$– Steven GubkinCommented May 5, 2018 at 22:27
-
$\begingroup$ If you are going to use a multiple-choice question with 10 answers, why is that different from a question with no answers suggested? $\endgroup$– DavidCommented Oct 14, 2019 at 11:19
-
$\begingroup$ May I congratulate this question for attracting multiple high-quality answers! $\endgroup$– kcrismanCommented Oct 16, 2019 at 16:45
Add a comment
|
7 Answers
$\begingroup$
$\endgroup$
I would say it completely depends on what is being asked. For example, something simple like "What is the derivative of $\cos(x)$ with respect to $x$?" could have as many as 10 options, because no real work or computations are needed -- either you know this one or you don't. [Assuming, of course, that you don't include both $-\sin(x)$ and $-\frac{\pi}{180}\sin(x)$ as options.]
Because answers in calculus can often be simplified, it would be tedious for a student to have to verify if their answer was among 10 different options, just in a different form. Therefore, I am fine with a large number of possible options if they are fairly simple but obviously distinct.
$\begingroup$
$\endgroup$
4
Presumably, you are trying to reduce the chance that someone will get a correct answer by guessing. In that case, I think that 10 options are more than you need. Suppose that we have an $N$ question multiple choice exam, each of which has $M$ options. Then a randomly guessing student gets $Q$ questions correct, where $Q$ is sampled from the binomial distribution $B(N,1/M)$. If we define 60% as passing, what's the probability of passing for some values of $N$ and $M$?
$$ \begin{array}{c|c|c|c|} & N=5 & N=6 & N=7 \\ \hline M=3 & 0.2098 & 0.1001 & 0.0453 \\ \hline M=4 & 0.1035 & 0.0376 & 0.0129 \\ \hline M=5 & 0.0579 & 0.0170 & 0.0047 \\ \hline M=6 & 0.0355 & 0.0087 & 0.0020 \\ \hline M=7 & 0.0233 & 0.0049 & 0.0010 \\ \hline \end{array} $$ So there is maybe a case for moving from 4 to 5, if that's what your default was before.
Of course, the randomly guessing student model is essentially assuming that the student knows nothing, which is hopefully not true. They will be able to eliminate some. I don't think that I, or anyone else, would be very good at generating 9 incorrect, but plausible, options. It would end up something like
"What is $\int 2x dx$?"
1) $x^2+C$
2) $x^2$
3) $2x^2$
4) $2$
5) $2dx$
6) 42
7) aardvark
8) hat
9) ¯\ _ (ツ)_ /¯
10) don't pick this one, it is not the answer
So there are only 4 or 5 possibilities that are not obviously wrong.
-
$\begingroup$ What about $x^2+C$, $x^2$, $2x^2$, $2x^2+C$, $2$,$2+C$, $1$, $1+C$? Your answers 2, 3 and 4 reveal that there are three errors you can imagine. Imposing these errors in all possible combinations gives 8 answers. If you don't do this, the savvy student chooses the reply at the center of the options. I don't actually endorse this as a good question, but I do often feel that I don't have enough answer options on the scantron because of this binary explosion. $\endgroup$ Commented Oct 15, 2019 at 12:11
-
$\begingroup$ The point is not that anyone will choose one of the bottom 4 options; the point is that the student who comes up with $2x^2+C$ should see $2+C$ and $2x^2$ sitting in the distractors, so it is less obvious that the $2$ is wrong. $\endgroup$ Commented Oct 15, 2019 at 12:13
-
-
$\begingroup$
$\endgroup$
1
There's a fair amount of research about this, mainly from psychology and medical education as well es test development research for high stakes tests.
In practical development, I usually resort to the "gold standard" and either collect sample answers analyzing the frequency and types of errors of students. Or I consult research articles on common misconceptions and typical mistakes to guide the formulation of distractors. In this sense:
"Done right: 3 is enough." (see e.g. Haladyna et al (2002) and Rodriguez (2005)).
I usually go with 4 Options on the most frequent mistakes (or anticipated from them). sometimes typical "slips of the pen"-mistakes make valid distractors as well (such as a change of "+" and "-" sign in answer).
Upshot: In the answer given by Adam, take the 4 or 5 options that are not obviously wrong and leave out the rest.
-
$\begingroup$ Thank you for the references! I think that this is a very nice answer. As to my own: Honestly, I think that even by answer 4, my options were stretching it a bit. The first 2 wrong answers correspond to conceivable mistakes, but any student answering #4 or 5 is profoundly lost. $\endgroup$– AdamCommented May 9, 2018 at 13:59
$\begingroup$
$\endgroup$
3
I do think that 10 options for a multiple-choice question is excessive.
A few things to consider: This will be outside the range that standard automation tools can handle (TestGen application, Scantron sheets, premade testbanks, etc.)
When I started teaching, I thought that using multiple-choice questions would be a time saver for my classes. But what I found was that writing (and organizing, scrambling) the distractor options took more time than I was saving. Especially if the problems should change each semester, the only way that's feasible is with an automated tool (see above). That was just with 4 options each; the 10-question proposal would be infeasible × 2.5 in my experience.
I would already recommend that at the level of college credit-bearing courses, that the questions be short-answer anyway. Feel free to grade the problems on a simple 3- or 5-point rubric each. I think you'll find that to be actually less work than trying to write 10 distractor options per question. And you'll also get to see where the actual problems and strengths of the students lie, as opposed to being masked through the multiple-choice filter.
answered May 6, 2018 at 23:02
-
2$\begingroup$ It would be more helpful as a question if there was some indication why the guy asked this (why contemplating 10 response). Some of these Q&A really feel out there. $\endgroup$– guestCommented May 8, 2018 at 3:21
-
$\begingroup$ @guest: I've had a colleague suggest the same thing. I think this was motivated in response to criticism that college courses shouldn't be multiple choice, but their wishing to continue avoid grading work. $\endgroup$ Commented May 9, 2018 at 13:05
-
1$\begingroup$ I totally agree, the time I saved in grading multiple choice was not larger than the time it took to create the questions. Furthermore, it's just healthier for students to have to write out their work and worry that we might care about the substructure of their thinking. Presumably university education is more about thinking than producing answers... $\endgroup$ Commented Oct 12, 2019 at 3:54
$\begingroup$
$\endgroup$
Like others, I feel this is too much. It turns the question into almost a mini research project (assuming there is some reasonable amount of checking needed). It's one thing to just scan through ten choices, but then if not immediately clear (in a false, false, true, false...series), than having to look at so many options and compare them is difficult--remember the human working memory (immediate cognition) probably can consider something like three to seven items at a time.
Also, if they are silly and don't really need checking, than there's little benefit of so much "chaff".
And then after running such a project-like enterprise, you still have a very digital result. And one with zero credit for partial work or for the second best of 10 options.
In addition, because of the added complexity, it means you can carry fewer total questions. So you're getting less statistical smoothing (of multiple tries).
In addition, I fail to see the benefit of so many options. Does one really have situations with ten likely options? And/or isn't it usually a bit of work even to get four or five (and test taking manuals explaining how often a couple of weaker choices or non-independent choices can be eliminated, even if the final choice of the smaller set not clear).
Finally students will not be familiar with it and you will be confounding anxiety about the material itself with anxiety about an unusual testing mechanism.
$\begingroup$
$\endgroup$
I thought that the New York State Regents math exams had an elegant strategy back in the 80's (and probably before, but this was when I was a student). They had a normal answer sheet with room for twelve multiple choice answers, but questions 13-24 had a short line where students would pencil in their answers. These questions were designed to have simple answers like $5$, $3\sqrt7$, or $2x-4$. (There were constructed responses later in the test that would require students to show their work.) Grading would presumably be only marginally more difficult than multiple choice, but you could be confident that students were constructing their own answers instead trying to game the multiple choice system.
You could definitely do something like that in a Calculus class. For instance:
If $f(x)=x\sin x$, what is $f'(\frac\pi3)?$
That's definitely a Part 1 type of question where you don't want to dive through their work, but multiple choice answers (no matter how many options) are still going to give them undesirable clues like "Oh, many of those answers have two terms, I didn't think about the Product Rule until I saw that". Let them do their work on a scrap piece of paper and fill in a short line, and you'll give them a more valid assessment without making your grading a headache.
answered Oct 17, 2019 at 11:31
$\begingroup$
$\endgroup$
Ten answers is a bit much. I agree with the other answers given, and there is also the test-taking anxiety factor. There are students out there who may know the material but simply find tests hard to take. For those students, ten possible answers can be incredibly overwhelming. Fewer possibilities will help these students push themselves to actually answer the questions given - showing their true level of understanding.
