# Question formats for online tests, to deter cheating

I'm teaching calculus 1 online this term and anticipate being plagued by the perennial problem of cheaters. I have seen suggestions for how to arrange the testing time to accommodate for traditional tests (such as this previous question and the questions it mentions) but I would rather improve my question-writing instead of impose onerous requirements on my students in terms of time. I would hope that by making these sorts of questions common in all assessment levels that this would also improve my students' learning, especially as they learn that I expect them to understand instead of merely regurgitate. I've also contemplated moving to projects, but have come to similar conclusions that it is difficult to evaluate students on procedural fluency with projects, and devising effective projects is probably going to take me more time than simply writing better questions.

So, with that said, what are some examples of types of questions which are suited for an online exam? A requirement here is that WolframAlpha & other solving websites often do in an unnatural way, or in a way which we do not ask the students to do it. Or equivalently, what ways of formulating questions are there that are difficult to put in to WolframAlpha?

The goal here is to make it less desirable to cheat (because it is difficult to do so) and to make it easier to catch those who cheat. I also do not think entirely avoiding mechanical differentiation problems is possible or desirable, as students need to develop the pattern-recognition to be able to learn integration in the next term, where it is possible we may end up going back face-to-face.

Some types of problems that I can think of include:

• In a question, requiring logarithmic differentiation to simplify problems which would involve large numbers of products and quotients.

• Differentiation of absolute value problems, if you ask for it to be done in the "piecewise" method instead of teaching them that the derivative of |x| is x/|x|

• Finding the value of a constant which makes a function continuous, or perhaps even its derivative continuous.

• "Abstract" Problems with chain rule, quotient rule, etc. that involve having a formula for one function and a simple f(x) for the other, represented as table or a graph.

• Find the place where a made-up student made an error, and complete the problem from that point on correctly. Or, explain what is wrong with a specific step and what should be done to correct it.

• Of course, applications problems. But these do not necessarily test their knowledge of the symbol manipulation and the deeper algebraic fluency which should come from a well-taught calculus class.

I have difficulty coming up with lower-DOK questions in this same vein, however!

• What does DOK stand for? Commented Sep 6, 2020 at 6:01
• I think you're going to have a fundamental contradiction here, which is that it will be unacceptable to fail all the students who don't understand calculus. If you haven't already, you will find that many perhaps most of your students don't even realize there is such a thing as understanding mathematics and cannot be convinced otherwise in the space of one semester. You will be expected to distinguish between students who have at least put in the effort to regurgitate something and those who haven't. Almost by definition, any problem that can be done by regurgitation can be done by Alpha. Commented Sep 6, 2020 at 6:17
• DOK = Depth of Knowledge, it's a framework with more dimensions than Bloom's Taxonomy Commented Sep 6, 2020 at 19:43
• – JRN
Commented Feb 25, 2021 at 2:01
• Your fundamental issue here is that "low DOK" is pretty close to being a synonym for "easy to google". Commented Feb 25, 2021 at 2:24

For problems that are beyond just calculations, if you can algorithmically create different problems for each student that are similar in difficulty, this may help with the near-guarantee that someone will be posting all of your problems to Chegg. Also, it may be done with any of the problem ideas you list above.

Here is an exam question I gave this term, together with a few different versions:

Version 1:

Version 2:

Version 3:

I think randomizing as much as possible is a good idea (up to the point where one student faces a much more difficult problem than another student).

• I ended up using this advice, to good results (it was very easy to find students who copied from each other!) Commented Dec 26, 2021 at 4:27

One of the things I'm currently testing is the addition of "find the error" based problems, and possible variations. For example (related but not the same topic) last semester I taught indefinite integrals, so in a partial test I decided to ask my students the following problem:

The integral $$\int \dfrac{3-x}{(2+x)^2}~dx$$ can be solved in several ways. For example, if you try partial fractions, you can obtain $$-\dfrac{5}{x+2} - \ln |x+2| + C.$$ But if you perform integration by parts, the answer is $$\dfrac{x-3}{x+2} - \ln |x+2| + C.$$ I ask my students to check both results, and to explain why the same integral is giving two 'different' solutions.

Do I completely remove cheating? Absolutely no. But in this case, Wolfram solves the integral by partial fractions and coincidently a lot of my students felt overwhelmed by integration by parts.

• Interesting notion. I know that in writing finding errors is much harder than writing it out. In computer programming debugging is much harder than programming. Commented Mar 1, 2021 at 1:55

WolframAlpha is not your problem. It really isn't. The students who are cheating by and large aren't good enough to figure out WolframAlpha's syntax and interpret its answers.

Your problem is that any of your students can buy a membership to Chegg.com and upload an image of a question. Chegg.com his hired people in less developed countries who will do the problem in half an hour and upload the result, and it will be accessible to anyone with a Chegg.com membership who is searching for that question. Then these students can just copy that answer.

• Ah -- but at least that has a "paper trail," so to speak. Commented Sep 4, 2020 at 22:11
• It's funny, I was showing WolframAlpha to my precalculus kids and they told me that "photomath" is easier. No pesky typing. I told them that hiring a personal trainer to do sit-ups for you is about the same thing as using either of these to do the work. Check the answers, fine, but do the work to start... Commented Sep 6, 2020 at 4:01
• Re: "do the problem in half an hour" -- I just tested uploading a novel problem of my own to Chegg (discrete math, multiple nested quantifiers, justifications required), and received a posted answer in 10 minutes. Commented Dec 16, 2020 at 18:45

Thoughts:

• Student can't go back. You have to solve each question to the extent you are able, then move on. If you are unable, you can mark it "return later" But when you return you get a new version of the problem. This effectively eliminates the Chegg problem, as there system has too much latency. This requires submitting your first partial answer, then your second answer later. After submitting both, you can view both, and choose which one is your real submission.

• Distinct versions of the problem mean that for a class of 100 students you will have hundreds of potential versions of the problem. Chegg works on "reselling" the answer. That is one person asks the question, one answer is generated, but it's used by a bunch of different students. This is very effective for textbook questions where there are tens of thousands of copies of popular textbooks, but is less effective with exams, and exams where each question is unique makes Chegg unprofitable.

• It also makes Chegg more difficult to use. Dr. Brown's Final Exam Question #3 isn't the same for all people, so for the Chegg user FINDING the same question even if you have only a limited field of questions becomes and issue.

• If the finding is done by image comparison, then having the program have several possible layouts for the question adds more noise to the Chegg system.

• Changing the order of questions will also throw a spanner in Chegg's gears. Now my #3 doesn't even look like your #3.

• Requiring problems to be worked out means, at best they cannot send the Chegg answer straight away, but have to copy the answer before scanning it. Otherwise each answer would be in a different handwriting.

Multiple choice questions are easier to randomize. A test giving webpage can randomize both the order of questions, the order of answers, and possibly change the number of possible answers. A clever one can leave "None of the above" as the last answer, and "All of the above" as the second last answer. Again problems can be generated so that each test is unique.

This sort of approach is an easy fallout from well done computer mediated instruction.

Last time I looked computer math instruction was slightly awful, with students moving thorugh a rigid lock step format. A good CMI math program would generate problems of appropriate difficulty for the student's ability at that time, and be able to understand the student's work. As the student improved, he'd be encouraged to use shortcuts, do multiple steps at once, but be flagged immediately when a shortcut was mis-applied or didn't work.

It also would be able to analyze mistakes to see if there were bugs in the student's software. (A study of the mistakes made by primary kids learning subtraction found that most errors were software errors. E.g. Doing the borrow to add 10 to the top digit, but forgetting to take it form the left digit. Or always subtracting the smaller digit from the larger regardless of positions.

• Regarding bullet #1, "you can mark it 'return later' But when you return you get a new version of the problem", can you identify a platform that supports that feature? Commented Sep 6, 2020 at 4:26
• This is almost precisely not what I want: This answer gives ways to make the test taking process more onerous instead of changing the questions I ask. I generally agree that randomizing, however, is a good approach. Commented Sep 6, 2020 at 21:06
• @DanielR.Collins Sorry, can't. I figured that the test admin system would have to be written from scratch, as each problem would require a separate program generator. Commented Sep 8, 2020 at 1:21
• Thanks for clarifying, but then that's surely an infeasible suggestion for any instructor I know of. Commented Sep 8, 2020 at 17:28
• Instead of "Otherwise each answer would be in a different handwriting," did you mean to say "in the same handwriting"?
– JRN
Commented Feb 25, 2021 at 2:06

I hold the opinion that preventing or disincentivizing cheating is practically impossible.
If I want to quiz my students about partial fractions or trigonometric integrals or the sorts, I can’t prevent them from putting it into WolframAlpha or the like by just changing the phrasing a bit, because WolframAlpha can solve partial fraction integrals.
Instead, I think the best course of action is to make the students less frustrated and thus, less vulnerable to cheating.
Here are the things I see to reduce student frustration.
Hard $$\not \Leftrightarrow$$ frustrating
Actually, quite the contrary, a student will be less frustrated by 3 hard integrals then 20 easy ones.
Don’t fight
Students who already cheat are practically non-recoverable.(after all, it only gets easier after the first time)
A student won’t stop cheating because the he couldn’t backtrack in the test, but, if you do prevent backtracking, you are frustrating the students who don’t cheat.
I have elaborated on this here: What are the best practices for giving online tests?

The following is not a foolproof solution, but one thing I've done for remote exams is have questions where you have students make up their own numbers, you can add some restrictions to make sure they don't make a problem too easy or find a problem done in the textbook.

For example, I've asked things like

1. here is $$3 \times 3$$ matrix with two or three entries blank. Place some non-zero numbers in those positions and compute the determinant." They have to show their work, and if they show the correct process, and I don't care about the answer and so that way I don't have to actually compute the determinant.

2. Find a 3 x 3 matrix with (1,1,1) in the nullspace. (When grading, I just have to sum the rows to make sure they add up to 0.)

3. Make up a system of 3 equations in 5 variables and show how to solve it.

Using this method, I have caught a few students using Chegg (their number choices were identical to the ones chosen by Chegg, in addition to rest of the work identical layout, etc). This doesn't circumvent someone from hiring a "tutor" (or going to a website with no searchable posts) to solve problems as then I do not have a paper trail.

I haven't taught calculus in a while, but maybe something like "let f(x) be a trig function, g(x) be an exponential function. Compute the derivative of some non-constant function built out of f and g and involving the term f*f (or you could be more specific and ask compute f(3x)g(9x), f(g(3x)), etc

Obviously things like WolframAlpha can still be used for this.

Edit March 2022: I’ve also in some classes giving up on exams by having students record a video of them explaining the material as if they were teaching it to someone who doesn’t know it… this isn’t directly relevant to your question about online exams, but I only came to this idea after getting tired of cheating on online exams despite trying various things, that this assignment is harder (but not impossible) to cheat on, and in some good ways, possibly better for learning than an exam, but I don’t have data to back this assertion.