# Future educators writing nonsense questions

I teach future elementary educators mathematics content courses.

We play a lot in class with tasks like "Write a variety of word problems which would require the student to multiply 2.3 by 1.4".

Often the questions which students produce are unintelligible, unanswerable, or target the wrong operation.

Unintelligible: "Bob has 2.3 pizza, and Jim has 1.4 pop. How much all together?"

Unanswerable: "I have 2.3 dollars, and want to buy 1.4 cups of ice cream. How much more money do I need to buy the ice cream?"

Target the wrong operation: "I have 2.3 cups of flour and 1.4 cups of water. If a recipe calls for 1 cup of each, how many recipes can I make?" (The operation here is $$\textrm{min}(2.3,1.4)$$)

No matter how much we practice this, when the exam comes around I still have a very large percentage of students writing these kinds of responses. This is extremely concerning to me, to the point that I don't feel comfortable with students who would produce such responses becoming educators.

Have others observed this phenomenon? Is there any research on it? Does anyone have success with fixing this problem?

• Have you tried starting this kind of lesson with an assignment where you provide a large list of problems for students to organize into those (or other) types of error-types? I would think this could set the stage for further discussion when they write and submit their own problems. – Nick C Sep 3 '20 at 13:17
• @guest Do you think someone who can flawlessly calculate 2.4*1.3 by hand, but writes a question like the unintelligible one above, is going to be well equipped to teach? Would they even know when to use multiplication if it came up in their real life (say buying 1.3 pounds of beans from the bulk bin at a price of \$2.40 per pound)? – Steven Gubkin Sep 3 '20 at 14:12
• @NickC I usually give such a problem to the class, collect anonymous responses, and then we discussions them and categorize them. I encourage a respectful classroom, where we do not ridicule errors, and this is usually an enjoyable and (seemingly) productive class period. – Steven Gubkin Sep 3 '20 at 14:15
• Will the students drop out if they fail to improve? This is, are you trying to help students pass by improving their skills, or are you on damage-control, trying to do what you can to improve their skills before they go to work? – Nat Sep 6 '20 at 7:54
• Hypothesis: the students are fully aware when they write these questions that the questions are unintelligible, unanswerable, and/or target the wrong operation, but the students are uncreative, and just can't generate anything better in the time available. Some of the approaches suggested in the answers will be able to test this hypothesis. – Daniel Hatton Sep 6 '20 at 11:05

You might try starting this kind of lesson with an assignment where you provide a list of different responses to the prompt "Write a variety of word problems which would require the student to multiply 2.3 by 1.4" and have students (perhaps in groups) arrange and rank them by clarity/mistakes/etc. Instead of having students start by writing their own word problems, by bringing your own list of good-and-bad examples, you're not setting students up to be ridiculed since you've created the set of responses.

I've had colleagues do this kind of thing with other tasks, such as learning how to show work in an intermediate algebra class. The first assignment is to take a stack of 8-10 teacher-created "solutions" to a word problem, read them with your group, and rank them by clarity and completeness. Then the class discussion revolves around lessons they can take from this when writing up their own solutions to word problems.

[Edit on 9/4]: I think another benefit of this plan is that if you follow up with a full-class discussion about why certain of the responses were objectionable, you'll likely build a class-specific vernacular that students may find light/playful to use when identifying errors in their/other's writing: Your "unintelligible" may be dubbed "what a mess" or "I can't even", etc. Since these descriptors aren't standard mathematical terms, I personally would readily accept new language if it means the class is learning the topic at hand and can communicate about it (though I would surely use my own terms from time to time in discussion).

• Iteration that I've used for similar exercises: Instead of instructor writing their own problems, recycle a stack of answers from the last semester you asked this in-class (with names removed, of course). I find there's many dimensions of "bad" responses that I would have no hope of accurately recreating on my own. – Daniel R. Collins Sep 3 '20 at 16:14
• This is probably the most helpful answer that could be written, but it still seems unrealistic to me. If someone is 20 years old and doesn't understand what multiplication is, then there is a vanishingly small chance that instruction in a college class will fix the problem. I have taught a teacher-prep class like this, and it was the worst experience I've had in 25 years of teaching. The students didn't want to be there and weren't willing to put in any effort outside of class. The underlying issue is the abysmally low academic level of the people who are in these programs. – Ben Crowell Sep 3 '20 at 20:09
• Writing instructors do this and call it norming. – Chris Cunningham Sep 3 '20 at 20:37
• @Discretelizard ""Bob has 2.3 pizza, and Jim has 1.4 pop. How much all together?" is a question that shows that someone does not understand what multiplication is. They may be able to mechanically multiply but do not understand what this means. The same way as being able to press a button and know the light will go on does not mean that someone understands why this happens (beside the vague "electricity" thing) – WoJ Sep 4 '20 at 16:26
• @WoJ No, the question "Bob has 2.3 pizza, and Jim has 1.4 pop. How much all together?" shows that someone has their language skills frozen at 6-year old level, and should have never be given a high school diploma, much less admitted to college. It shows pitiful state of American elementary and secondary education, which starts with non-working Whole Language / Balanced Literacy programs, promoted by organizations like NCTE. When kids cannot read and write, they cannot learn other subjects. Forget about math or physics. 2.3 what? And 1.4 what? Kilograms and liters? You cannot even add them up. – Rusty Core Sep 5 '20 at 0:00

Note: This is an answer from a non-US perspective, after reading some remarks from @Rusty Core I fear it won't be helpful.

How about you set certain standards and let sub-standard students fail your class? Just like in every other course the goal is to separate the wheat from the chaff.

I wouldn't want my children to be educated by teachers asking the kind of exam questions you provided as examples. If they don't improve until the re-exam they fail the class.

This is extremely concerning to me, to the point that I don't feel comfortable with students who would produce such responses becoming educators.

Why concerning? Why are you so invested in these students, it's not your fault, that they don't study. Just fail them.

• American elementary and secondary education is a political minefield that has little to do with education per se, and more with progressive — or conservative — ideas that the leaders of this, should I say, industry, want to push. Many great ideas have been tried in the last, let's say, 60 years, and failed. Many bad pedagogical, but nicely-sounding from progressives' perspective ideals have been applied since early 1920s and still hold their grip on the system. Want the best education for your kids? Homeschool them. – Rusty Core Sep 5 '20 at 0:10
• It is concerning to the OP, because his job at his institution is at stake. In the U.S. students can choose their prof, and if he fails bad students, no one will chose him. So he will be fired. Students are customers. Profs are providers of a service. Since the students pay seriously big money for their higher education, they expect result, not in the form of knowledge, but in the form of good grades and diploma. This is market economy in education to you. – Rusty Core Sep 5 '20 at 0:12
• I'm sorry, but this is hilarious. How can such a system ever ensure quality teaching? Who in his right mind would allow students to just switch professors without reasons going beyond "his class is hard"? Maybe my answer is not a good answer in such a broken system then. – user13921 Sep 5 '20 at 0:18
• Your updated comment makes sense. It's still hilarious though. Higher education is not something you buy in a store. Humboldt would scratch his eyes out reading this. It's frightening how free-market radical America is even when it comes to education. Here in Germany there are state-regulated examinations ("Staatsexamen") prospective students need to pass in order to be able to teach pupils, no matter their age. – user13921 Sep 5 '20 at 0:31
• I think more important than Rusty Core's claim is that in the U.S. educators are poorly paid and supported, there is no national standard or guidance, and for over 100 years education majors have been the lowest-qualified of all incoming college students, and especially weak in math. As noted in comments now in chat, we should have math specialists but don't. So the current state is a choice between poor teachers and none at all. web.archive.org/web/20150204031027/https://qz.com/334926/… – Daniel R. Collins Sep 6 '20 at 4:09

Partial answer regarding an approach to fix this problem.

First: Don't tell them (criticism), but lead them to find out themselves (insight). Now comes the fun part. Don't let them write just the questions.

1. Have them each write the question on one sheet, and write the correct solution, and a short (maybe single-sentence) explanation for their solution (choice of operation, solution method) on a different sheet.

2. Then have them swap just the question sheets and try to solve it :)

If the question is intelligible, answerable, using the correct operations, and the other person is sufficiently competent, he or she should not only be able to give the correct solution, but also to come up with a very similar explanatory statement.

In case the other person isn't able to do that, you'll have demonstrated to them that something isn't working yet... with no criticism involved.

• Of course, this requires the students to all have different prompts; if they all have "Write a variety of word problems which would require the student to multiply 2.3 by 1.4", then they'll have no difficulty "solving" each other's problems whether or not they're coherent. – ruakh Sep 5 '20 at 18:23
• I like this suggestion, and have in fact used it. I always emerge from these experiences hopeful that something has been learned, but this understanding does not seem to persist through time. – Steven Gubkin Sep 5 '20 at 18:46
• @ruakh, even if they know what the answer is supposed to be, they should struggle to come up with a sensible explanatory statement to nonsensical questions. I wonder if doing even just the first part (having the question authors also write solutions) would reduce the number of nonsensical questions, when they realize that they cannot answer their own question. – Joe Sep 6 '20 at 9:20
• @Steven Gubkin, it’s not surprising to me that the understanding doesn’t persist, for the reason given in the popular comment to the other answer by Ben Crowell. Your observations are making me believe we should have elementary MATH teachers, just like we have MATH teachers in higher grades, rather than having elementary education teachers try to teach all subjects. Of course, not to sound mean, but perhaps the students who gave the first two responses shouldn’t be teaching anything. – Joe Sep 6 '20 at 9:28

You might start out by giving them a test in the kind of arithmetic they are supposed to teach. This article in the Guardian from 2010 reported that many primary school teachers in the UK were unable to do the arithmetic required by the primary curriculum.

Fewer than four out of 10 of those who sat the test – designed for 11-year-olds – could calculate 2.1% of 400, and only a third answered correctly that 1.4 divided by 0.1 was 14. Overall, four out of ten scored 40% or below, only one got all the answers correct and the average mark was 12 out of 27 or 45%.

[...]

The test, carried out for the Channel 4 documentary series Dispatches, included addition and multiplication sums, ­simple algebra and questions involving fractions, conversions and averages. ­Teachers performed well on some of the easier questions. For example, 97% were able to work out 2 x 5 - 4 = 6 and 75% knew that three sevenths of 21 was 9. But only six teachers (4%) knew that the answer to 2 divided by 0 was infinity.

The test was set by Richard Dunne, a former teacher and the author of Maths Makes Sense.

Dunne said teachers were "thoroughly dedicated", but argued the test showed that more than half of them understood "so little maths that they cannot be conveying mathematics to their children in the classroom".

(Aside: the question about infinity is maybe a little unfair, given that its not on the curriculum. But bright kids will be asking about it.)

The problem seems to be that the kind of person who becomes a primary school teacher is often not strong on maths and science; the job tends to attract people who are stronger on the nurturing and caring side, who can bring an understanding attitude to a shoe full of wee. Many of them haven't done any maths since scraping through GCSE (i.e. age 16), and what understanding they had then has only decayed in the intervening years. So being a bit confused about what the technical term "multiplication" means is only to be expected.

• Actually, isn't the question about infinity given the wrong answer? Infinity isn't a number (at least, in any sense that pupils should be using), and the real answer is that 2 / 0 is undefined — or, in loose terms, can't be done. (Of course, in real numbers the result approaches infinity as the denominator approaches 0 from above; but then it approaches negative infinity if from below, so it's obvious that you can't infer one or the other for 2 / 0 itself.) And even if infinity's not on the curriculum, shouldn't pupils know that ‘you can't divide by zero’? – gidds Sep 5 '20 at 23:04
• There is actually an interesting conversation to be had about division by 0. 2/0 and 0/0 are undefined for different reasons. 2/0 is undefined because there is no number whose product with 0 is 2. 0/0 is undefined because every number has a product of 0 with 0! Discussing limits in an informal way is also intriguing. Small children and future educators alike can become very animated when discussing these kinds of pathological situations. – Steven Gubkin Sep 5 '20 at 23:45
• I think your last paragraph is essentially correct. Most people trying to become teachers were taught math from teachers who didn't know math, so there's no reason to expect them to be competent. I find the reporting about the test interesting too. 12/27 is closer to 44%, and 2/0 is undefined (there's no reason to assume we're considering anything other than the reals, and I see no reason to give the test maker the benefit of the doubt about even knowing about number systems where it is defined) – Thierry Sep 6 '20 at 1:00
• I agree with this - you have to teach them to solve the problems (and understand them) before they can write new ones. The question seems to be asking how to shortcut this, which seems fruitless. – usul Sep 6 '20 at 1:22
• @Thierry: Re: "12/27 is closer to 44% [than to 45%]": Presumably the average was not exactly 12, but rather, just some value in [11.5, 12.5). Any value in [12.015, 12.285) would give a percentage in [44.5%, 45.5%). – ruakh Sep 6 '20 at 9:32