# What are the best practices for giving online tests?

Many of us our coming off our first semester of required-online classes; and at some of our institutions we are preparing for what is most likely a required-online semester in the fall. (That is: The first time we know in advance of the all-online requirement, and with some experiential knowledge of how things progressed in the spring.)

Assume that we're going to be giving math tests online in the future. In my department there are a wide variety of opinions about how best to deploy those tests, and we've passed around competing published opinion papers on the matter (mostly from a time when courses were not all-online by necessity; i.e., prior publications assume students registered for online courses as a special case). Most of the conversation revolves around how to tamp down on cheating during those tests. Granted that, what are the best parameters under which to give tests? Some points that would be addressed in the best answer:

• Short vs. long availability period? E.g.: Some instructors want the testing to occur within the normal class hour, and require students to begin within a 15 minute window. Others look to give students maximum flexibility, say, a 6-hour or day-long window in which to take the one-hour test (so as to work around other courses, family/work obligations, etc.)
• Short vs. long timer? A short timer on the test may reduce opportunities for cheating, but likewise make a student more desperate and likely to resort to cheating; perhaps a longer timer would reduce that stress.
• Present questions all at once or one at a time?
• Randomize question order?
• Prohibit backtracking?

Now, my own situation involves teaching math and computing in the context of a community college STEM degree program; e.g., at the level of college algebra through calculus III, linear algebra, and discrete mathematics. Traditionally these tests are largely computational -- a small number of proof-based questions are asked at the uppermost levels, and it is a minority of our students who can answer any of them (e.g., making all questions proof-based is not an option at this level). Hopefully this question can cover an array of similar-level courses at other institutions.

The online system we use is Blackboard, but likewise I hope this question is appropriate for any online platform (e.g., my bulleted points above are basically the Blackboard test deployment options, excepting the last one).

The larger institution has made an executive decision to not license or employ any online proctoring system (e.g., video surveillance of students taking tests), so that is not an option in our case.

Note some related but distinct questions on ME.SE that have been recently asked:

So: What are the best-practice parameters with which to deploy an online test in the current environment (all classes by necessity being fully online)?

Note: Suggestions to refrain from giving tests are out of scope for the current question, and will not be selected as the answer. Please address any such possible practices in a separate question.

• Did I miss it? Did you rule out proctoring of tests ? There are options which still maintain social distancing if that is a concern. They are pretty cheap as well. I would add, these allow us to give tests similar to those we have been giving face-to-face all these years and as such surely there is more of a chance we'll be fair to past students in terms of maintaining quality of instruction etc. (if I missed it, my apologies) – James S. Cook Jun 22 '20 at 2:21
• @JamesS.Cook: If you're talking about online proctoring, the 5th paragraph of the question does point out that at least at my school, the institution has decided to not make that available. I suppose that could be part of proposed answer; but I'll say at the same time that multiple of my colleagues have already shown how that would be easily defeated anyway (having learned in turn from their students). – Daniel R. Collins Jun 22 '20 at 3:47
• It is there, thanks. I should have seen that. That said, I think that is a poor choice by the larger institution. There are ways to cheat, the same is true residentially. In any event, it's out of your hands, carry on... – James S. Cook Jun 22 '20 at 14:18
• Cheaters will cheat anyway. There's not much you can do about it. Mandatory webcam usage is adviseable – David Jul 13 '20 at 12:02
• Yet another method for online exams: when I was a student, one of my teachers had us do an exam with laptops & internet allowed (about C and X8086). His method to prevent cheating was very simple, the exam was way too long even for the best students (even now I would not finish it in time). I remember rushing to try and finish it, not having time to "help" my comrades. Doesn't prevent Chegg-like external help, but prevents answers sharing to some degree. – m.raynal Dec 11 '20 at 14:15

Here is my experience:

Throughout the term I gave several quizzes with

• long timer
• short availability
• questions all at once
• few modifications

The average score was around 80 %.

For the final exam, I gave a quiz with majority (around 2/3) of the questions very similar to those given throughout the term (or even the same questions, but reworded). However, the conditions were different:

• short timer
• questions one at a time
• randomized question order
• prohibit backtracking
• random questions (a random question from each category with 1 to 4 similar questions)

Besides this, I was tracking the progress of students to make sure that there are no students waiting for someone else to answer their question. This led to quizzes being mostly individual.

This time, the average score was 45 %. (And of course, there were many examples of students failing questions which they had correctly answered before.)

I should admit that cheating is a real plague here, starting from primary school. I am trying to fight it as much as I can. For instance, besides the quiz, there were problems to solve in the final exam, which I made as variable as I could (the exam was in Statistics, so I used random samples). The students do know how much I despise cheating and that they would get F if they are caught, so most of them will prefer to return a blank paper if there is a chance to get caught cheating. And still there were 4 F's in the final exam because of cheating (out of around 40 students).

So, speaking of quizzes, I am not sure that the second scheme is better, but I'd rather be using it. However, I understand that this may not be the best idea in other environments where cheating is not so ubiquitous as here.

• Those are valuable observations (incl. to my own case in a U.S. community college). What do the grades on your midterm and final exam typically look like (pre-online-testing)? – Daniel R. Collins Jul 2 '20 at 13:39
• @DanielR.Collins, last year the performance was about 60 % on average for midterm and 80 % on average for final exam (I use to give easier final exam sheets). – zhoraster Jul 2 '20 at 15:13
• I'm marking this as the selected answer because it addresses all the items in the standard deployment options, provides a broad context, experiments with two different approaches, and provides pretty clear evidence that cheating was happening in the more generous case. It's largely in line with more narrow observations in other answers. Other answers are still excellent and I hope readers will look at those as well. – Daniel R. Collins Sep 3 '20 at 17:16

On The Timer

I tried a long timer (about 3-5 days) for my exams and final exam last semester. The reason for this is that my favorite assignments from college -- the ones where I learned the most -- were challenging assignments that had a long availability period and allowed incremental work over the course of several days.

However, an unbelievably large number of my students used Chegg, Mathway, Symbolab, and Wolfram|Alpha to cheat. I gave F's for academic integrity violations to almost 25% of my students. That's just the ones where I had evidence that I felt was strong enough to win an appeal. Many more clever students may have used the resources without being so obvious about it.

Some students probably got a lot of value out of the long-timer assignment, the same way that I did when I was in school. If you go this route, be prepared to buy a subscription to Chegg at a minimum.

• @NickC I can't tell what you are asking, but buying a subscription to Chegg allowed me to find the "expert answers" that had been purchased by my students and use it as evidence against them in the academic integrity cases. These "expert answers" were usually relatively low-quality and were also usually exactly copied word-for-word by my students despite their low quality. You don't need a subscription to find that your questions are posted; it's only needed to see the "expert answers." – Chris Cunningham Jun 21 '20 at 23:36
• "You don't need a subscription to find that your questions are posted; it's only needed to see the "expert answers." That was my question. I know that you can just Google a unique portion of one of your questions with the word "Chegg" and see that students have posted screenshots, or whatever, but I was unsure if that brought up every instance of Chegg answering that question, or whether having a subscription would result in more hits for the search. Knowing as little as I do about Chegg, I didn't know if having an account increased your access to things-not-Googleable. – Nick C Jun 22 '20 at 0:43
• Don't forget about Course Hero and if you give days perhaps the Math Stack Exchange. (which would be really funny if you answered it) – James S. Cook Jun 22 '20 at 2:17
• Tip for future people: When you search your questions on Chegg, I recommend you highlight them with your mouse and copy-paste them directly into the Chegg search bar. Resist the urge to copy-edit the result -- the $\LaTeX$ will paste all weird and you will want to fix it to improve your chances at a successful search -- but the students you are trying to catch aren't clever enough to do that when they post the questions. So just copy-paste them directly when you search. – Chris Cunningham Jun 23 '20 at 1:19
• I wonder how many teachers who think that it is ok to use calculators to solve a system of linear equations or a quadratic equation because "these are just tools" and students need "to think conceptually" leaving boring calculations to the machines, are aggravated by sites like Chegg, which are, essentially, are just another kind of tools and machines. – Rusty Core Jul 23 '20 at 19:48

if you are trying to prevent cheating, trying to limit the student's ability to do it is very hard and mostly counterproductive, as it can introduce frustration , specificity in an online setting.
Instead, I recommend to minimize the student's motivation to cheat, after the first time it is just get's easier, if you design tests to not be frustrating you'll get a much better result instead of trying to fight the student.
Hard $$\not \Rightarrow$$ Frustrating, even the opposite, giving the student's three hard integrals is mostly better then 20 easy ones.
Short vs. long availability period
I’m on the Long availability period side, a student in the online setting might have difficulties to attend during specific times and thus it’d be better to let them have time.
Long timer vs. Short timer
I’m on the long timer side, from my experience, students cheat either out of habit or desperation.
in the long timer scenario, the student is less likely to be desperate and resort to cheating.
Backtracking
if you prohibit backtracking you are just annoying the student, there isn't something more frustrating then realizing you were wrong and can't fix it.
Randomize question order
It really doesn't matter from the student's perspective and they'l probably not even notice it.
from my experience, this won't deter a student from cheating.
General tips to minimize cheating
A majority of online resources cannot answer “why?” Questions. So, for example, having a question such as $$\int \frac{dx}{\sin(x)}$$ Could make a student just put the integral into WolframAlpha and get the solution.
On the other hand, if we add an intermediary question such as: “Why does substituting $$u=\sin(x)$$ not work?” WolframAlpha can’t help the student cheat anymore.
This question is especially good because most answers will use the $$\cot(x) + \csc(x)$$ trick, which would light a red flag as the answer expected will probably be the $$\cos(x)$$ substitution and partial fractions.

• I’m on the Long availability period side, a student in the online setting might have difficulties to attend during specific times and thus it’d be better to let them have time. I don't know about other schools, but at mine, for the classes that have been forced online due to covid, there are specific meeting times when the students have to be available. These are the times when you'd be doing your class sessions through video conferencing. – Ben Crowell Jun 23 '20 at 19:35
• Randomize question order doesn't matter from the students' perspective... Citation needed. – Matthew Leingang Jun 24 '20 at 22:01
• Randomize question order. I would say that on a short timer with a short availability period, and prohibiting backtracking, you might stop some people from sharing answers. If you have a long timer, or a long availability period, randomizing order isn't going to prevent people from copying answers. – Carson Graham Jun 25 '20 at 16:15
• It isn’t, but it wouldn’t change anything, so on the off chance it did. – razivo Jul 16 '20 at 15:46

Randomize questions

Other answers are good, but I think a very important aspect on online quizzes has not been covered: questions should be different for every student - as different as resources and fairness allow.

Questions can be randomized in several ways:

• Random parameters.
• Slightly different versions of the same question (e.g. one version with permutations with repeat and another version without repeat).
• Mathematically equivalent questions with apparently different settings (e.g. one question about husbands and wives and another about locks and keys).
• Actually different questions. If possible, a large pool of independent questions taken at random.

In online tests (and even more in online homework) my ideal goal is to make the students need to know statistics (or to learn them, in homework) even to able to cheat. This way, I need to minimize the "cheatable" part, ideally making it indistinguishable from learning the subject. For example, I like questions about computing a probability in a normal distribution because when parameters are different the only thing that can be copied is the procedure, which is what I want them to learn.

However, I must admit that preparing random quizzes is a lot of work.

Edit in response to comment:

The comment asks how to implement random quizzes for problems with answers that can't be expressed as a number with decimals.

Some ways that worked for me (in Moodle):

• Wiris quizzes, that allow for answers including formulas and can check fore mathematical equivalency. For example it allows the answer to be $$\frac{3 \cdot \pi}{2}$$ or even a function. Since Wiris is a commercial extension to Moodle, it is not installed in all institutions, but there may also be equivalent extension from other vendors.
• Multiple option questions: The answer can be as complex as you want, because in the end the student just needs to choose between a finite set of answers, and that can be easily graded by Moodle.
• Transforming the answer to a decimal number. If, instead of asking the derivative of a function, we ask for the value of the derivative at a given point, the answer is a real number and we just need to evaluate the answer against a given tolerance. However, I know that pure mathematicians usually dislike this approach.

To generate those kinds of questions I use three different tools:

• Moodle calculated questions. Quite easy but very little mathematical capabilities.
• R package exams. You can do in the question anything you can do in R. Great for statistics and numerical analysis, useful but not so great for calculus or algebra.
• Wiris, which has built-in random capabilities.

And of course, few automatic quizzes will outperform the teaching and assessing effect of a hand-graded essay or a long problem. However, automatic quizzes are great at easing logistics and introducing randomness.

We have also used random data in hand-graded exercises, but those are slow to grade and randomness needs to be limited. In the end all the students have the same problem with some different parameters and teachers can grade using a table of results.

• From prior investigations, it seems to me like this is mostly only possible for courses that have problems with decimal-number answers to problems -- like physics, biology, and I can also see statistics. Other math courses with symbolic answers I've failed to see how it can happen on a large scale (because the tools I have can't randomize & check symbolic answers). I really don't see any way to individualize and grade, e.g., college algebra, programming, or discrete math problems at large-scale. – Daniel R. Collins Jun 22 '20 at 13:39
• @DanielR.Collins - I hope the edit in the answer helps. – Pere Jun 22 '20 at 14:23
• @NickC: Thanks for the info & offer. However, I'd really need a system that scores the symbolic answers to begin considering it. On top of that, my own normal preference to score written justifications/proofs of the work -- although maybe for other instructors who are happy with multiple-choice responses then it would be more helpful (if it graded symbolic answers, not numerical approximations). – Daniel R. Collins Jun 22 '20 at 14:40
• Pere, thanks for the added suggestions. I'll +1 this because it sounds like some good options for instructors of numerically-based courses, even if it's not a solution that I can see using in my own case. – Daniel R. Collins Jun 22 '20 at 14:43
• @DanielR.Collins - In Moodle you can ask for justification/proof, too, with an essay type question or a Wiris open question, but those would need to be graded by hand. I've seen that used as a complement of a closed and automatically graded question. – Pere Jun 22 '20 at 15:04

The main issue with using proctoring software is that some students perceive this as being required to install spyware on their computer with all the features of tracking keystrokes etc., and when there is a recording of the exam session there are questions of where these recordings are stored and who has access to them.

At my institution we are moving toward live proctoring over Zoom without recording. However, proctoring through a front-facing camera is not very effective because students can easily communicate with others or access unauthorized materials on their other devices on their desktop if we cannot see their hands. For this reason, our preferred method for proctoring will be through the Zoom mobile app where the students sets up their cell phone as a side camera, clearly showing their workspace and with their computer screen tilted slightly toward the camera. Some schools are already doing this and have published detailed guidelines, including: Texas A&M University, Hong Kong University of Science & Technology, and KTH Royal Institute of Technology in Sweden. We expect that we may need to accommodate a small minority of students that cannot create a suitable environment for taking an exam at home in this way.

• This sounds like a good faith effort to maintain the legitimacy of test results. I prefer to require testing with a human proctor, but this is almost as good. – James S. Cook Jul 19 '20 at 0:27
• The problem is not going to be solved by using more and more invasive methods of surveillance. Your answer also seems to presuppose that every student has both a laptop/desktop and a smart phone, which is not the case. – Ben Crowell Mar 5 at 14:02

Do standard time exams in the normal course period. Keep as much the same and stable vise loosely goosey.

Email it as a pdf. Let the kids send handwritten work back, scanned.

Avoid the grad student impulse to give project style questions. Do normal problems similar to the drill work. Take it as a victory if they learn that.

Ask them to box their answers. If you want to make it easy on yourself, do short frequents questions and just grade the final result. Students can see the key after the exam to see where they went wrong. If still confused they can request a session to go over it. Not grade arguing but to learn.

Do a nonverbise integrity statement. Doesn't hurt.

• Or post on a website instead of email. But overall thrustvtemains. KISS. – guest Jun 21 '20 at 21:36
• Here's my concern about only doing normal problems similar to the drill work. If students only learn the drill work, then effectively they have gotten a second high school education, not a college education. A semester or two of that probably won't hurt much, but, not surprisingly, a student who hasn't gotten a college education (whether or not they got a college degree) will have trouble holding jobs that require one. – Alexander Woo Jun 22 '20 at 1:20
• Perfect is the enemy of better. – guest Jun 22 '20 at 19:13
• This doesn't answer the question. The question is about online courses during the covid epidemic. The answer could have been written a year ago. – Ben Crowell Jun 23 '20 at 19:37
• Yes it does.... – guest Jun 24 '20 at 15:46