# Question formats for online tests, to deter cheating

I'm teaching calculus 1 online this term and anticipate being plagued by the perennial problem of cheaters. I have seen suggestions for how to arrange the testing time to accommodate for traditional tests (such as this previous question and the questions it mentions) but I would rather improve my question-writing instead of impose onerous requirements on my students in terms of time. I would hope that by making these sorts of questions common in all assessment levels that this would also improve my students' learning, especially as they learn that I expect them to understand instead of merely regurgitate. I've also contemplated moving to projects, but have come to similar conclusions that it is difficult to evaluate students on procedural fluency with projects, and devising effective projects is probably going to take me more time than simply writing better questions.

So, with that said, what are some examples of types of questions which are suited for an online exam? A requirement here is that WolframAlpha & other solving websites often do in an unnatural way, or in a way which we do not ask the students to do it. Or equivalently, what ways of formulating questions are there that are difficult to put in to WolframAlpha?

The goal here is to make it less desirable to cheat (because it is difficult to do so) and to make it easier to catch those who cheat. I also do not think entirely avoiding mechanical differentiation problems is possible or desirable, as students need to develop the pattern-recognition to be able to learn integration in the next term, where it is possible we may end up going back face-to-face.

Some types of problems that I can think of include:

• In a question, requiring logarithmic differentiation to simplify problems which would involve large numbers of products and quotients.

• Differentiation of absolute value problems, if you ask for it to be done in the "piecewise" method instead of teaching them that the derivative of |x| is x/|x|

• Finding the value of a constant which makes a function continuous, or perhaps even its derivative continuous.

• "Abstract" Problems with chain rule, quotient rule, etc. that involve having a formula for one function and a simple f(x) for the other, represented as table or a graph.

• Find the place where a made-up student made an error, and complete the problem from that point on correctly. Or, explain what is wrong with a specific step and what should be done to correct it.

• Of course, applications problems. But these do not necessarily test their knowledge of the symbol manipulation and the deeper algebraic fluency which should come from a well-taught calculus class.

I have difficulty coming up with lower-DOK questions in this same vein, however!

• What does DOK stand for? – Kevin Arlin Sep 6 '20 at 6:01
• I think you're going to have a fundamental contradiction here, which is that it will be unacceptable to fail all the students who don't understand calculus. If you haven't already, you will find that many perhaps most of your students don't even realize there is such a thing as understanding mathematics and cannot be convinced otherwise in the space of one semester. You will be expected to distinguish between students who have at least put in the effort to regurgitate something and those who haven't. Almost by definition, any problem that can be done by regurgitation can be done by Alpha. – Alexander Woo Sep 6 '20 at 6:17
• DOK = Depth of Knowledge, it's a framework with more dimensions than Bloom's Taxonomy – Opal E Sep 6 '20 at 19:43
• – Joel Reyes Noche Feb 25 at 2:01
• Your fundamental issue here is that "low DOK" is pretty close to being a synonym for "easy to google". – user3482749 Feb 25 at 2:24

For problems that are beyond just calculations, if you can algorithmically create different problems for each student that are similar in difficulty, this may help with the near-guarantee that someone will be posting all of your problems to Chegg. Also, it may be done with any of the problem ideas you list above.

Here is an exam question I gave this term, together with a few different versions:

Version 1:

Version 2:

Version 3:

I think randomizing as much as possible is a good idea (up to the point where one student faces a much more difficult problem than another student).

WolframAlpha is not your problem. It really isn't. The students who are cheating by and large aren't good enough to figure out WolframAlpha's syntax and interpret its answers.

Your problem is that any of your students can buy a membership to Chegg.com and upload an image of a question. Chegg.com his hired people in less developed countries who will do the problem in half an hour and upload the result, and it will be accessible to anyone with a Chegg.com membership who is searching for that question. Then these students can just copy that answer.

• Ah -- but at least that has a "paper trail," so to speak. – Opal E Sep 4 '20 at 22:11
• It's funny, I was showing WolframAlpha to my precalculus kids and they told me that "photomath" is easier. No pesky typing. I told them that hiring a personal trainer to do sit-ups for you is about the same thing as using either of these to do the work. Check the answers, fine, but do the work to start... – James S. Cook Sep 6 '20 at 4:01
• Re: "do the problem in half an hour" -- I just tested uploading a novel problem of my own to Chegg (discrete math, multiple nested quantifiers, justifications required), and received a posted answer in 10 minutes. – Daniel R. Collins Dec 16 '20 at 18:45

One of the things I'm currently testing is the addition of "find the error" based problems, and possible variations. For example (related but not the same topic) last semester I taught indefinite integrals, so in a partial test I decided to ask my students the following problem:

The integral $$\int \dfrac{3-x}{(2+x)^2}~dx$$ can be solved in several ways. For example, if you try partial fractions, you can obtain $$-\dfrac{5}{x+2} - \ln |x+2| + C.$$ But if you perform integration by parts, the answer is $$\dfrac{x-3}{x+2} - \ln |x+2| + C.$$ I ask my students to check both results, and to explain why the same integral is giving two 'different' solutions.

Do I completely remove cheating? Absolutely no. But in this case, Wolfram solves the integral by partial fractions and coincidently a lot of my students felt overwhelmed by integration by parts.

• Interesting notion. I know that in writing finding errors is much harder than writing it out. In computer programming debugging is much harder than programming. – Sherwood Botsford Mar 1 at 1:55

Thoughts:

• Student can't go back. You have to solve each question to the extent you are able, then move on. If you are unable, you can mark it "return later" But when you return you get a new version of the problem. This effectively eliminates the Chegg problem, as there system has too much latency. This requires submitting your first partial answer, then your second answer later. After submitting both, you can view both, and choose which one is your real submission.

• Distinct versions of the problem mean that for a class of 100 students you will have hundreds of potential versions of the problem. Chegg works on "reselling" the answer. That is one person asks the question, one answer is generated, but it's used by a bunch of different students. This is very effective for textbook questions where there are tens of thousands of copies of popular textbooks, but is less effective with exams, and exams where each question is unique makes Chegg unprofitable.

• It also makes Chegg more difficult to use. Dr. Brown's Final Exam Question #3 isn't the same for all people, so for the Chegg user FINDING the same question even if you have only a limited field of questions becomes and issue.

• If the finding is done by image comparison, then having the program have several possible layouts for the question adds more noise to the Chegg system.

• Changing the order of questions will also throw a spanner in Chegg's gears. Now my #3 doesn't even look like your #3.

• Requiring problems to be worked out means, at best they cannot send the Chegg answer straight away, but have to copy the answer before scanning it. Otherwise each answer would be in a different handwriting.

Multiple choice questions are easier to randomize. A test giving webpage can randomize both the order of questions, the order of answers, and possibly change the number of possible answers. A clever one can leave "None of the above" as the last answer, and "All of the above" as the second last answer. Again problems can be generated so that each test is unique.

This sort of approach is an easy fallout from well done computer mediated instruction.

Last time I looked computer math instruction was slightly awful, with students moving thorugh a rigid lock step format. A good CMI math program would generate problems of appropriate difficulty for the student's ability at that time, and be able to understand the student's work. As the student improved, he'd be encouraged to use shortcuts, do multiple steps at once, but be flagged immediately when a shortcut was mis-applied or didn't work.

It also would be able to analyze mistakes to see if there were bugs in the student's software. (A study of the mistakes made by primary kids learning subtraction found that most errors were software errors. E.g. Doing the borrow to add 10 to the top digit, but forgetting to take it form the left digit. Or always subtracting the smaller digit from the larger regardless of positions.

• Regarding bullet #1, "you can mark it 'return later' But when you return you get a new version of the problem", can you identify a platform that supports that feature? – Daniel R. Collins Sep 6 '20 at 4:26
• This is almost precisely not what I want: This answer gives ways to make the test taking process more onerous instead of changing the questions I ask. I generally agree that randomizing, however, is a good approach. – Opal E Sep 6 '20 at 21:06
• @DanielR.Collins Sorry, can't. I figured that the test admin system would have to be written from scratch, as each problem would require a separate program generator. – Sherwood Botsford Sep 8 '20 at 1:21
• Thanks for clarifying, but then that's surely an infeasible suggestion for any instructor I know of. – Daniel R. Collins Sep 8 '20 at 17:28
• Instead of "Otherwise each answer would be in a different handwriting," did you mean to say "in the same handwriting"? – Joel Reyes Noche Feb 25 at 2:06

I hold the opinion that preventing or disincentivizing cheating is practically impossible.
If I want to quiz my students about partial fractions or trigonometric integrals or the sorts, I can’t prevent them from putting it into WolframAlpha or the like by just changing the phrasing a bit, because WolframAlpha can solve partial fraction integrals.
Instead, I think the best course of action is to make the students less frustrated and thus, less vulnerable to cheating.
Here are the things I see to reduce student frustration.
Hard $$\not \Leftrightarrow$$ frustrating
Actually, quite the contrary, a student will be less frustrated by 3 hard integrals then 20 easy ones.
Don’t fight
Students who already cheat are practically non-recoverable.(after all, it only gets easier after the first time)
A student won’t stop cheating because the he couldn’t backtrack in the test, but, if you do prevent backtracking, you are frustrating the students who don’t cheat.
I have elaborated on this here: What are the best practices for giving online tests?