# Formula sheets and books during tests and exams

Some teachers make memorizing formulas, definitions and others things obligatory, and forbid "aids" in any form during tests and exams. Other allow for writing down more complicated expressions, sometimes anything on paper (books, tables, solutions to previously solved problems) and in yet another setting students are expected to take questions home, study the problems in any way they want and then submit solutions a few days later.

Naturally, the memory-oriented problem sets are relatively easier (modulo time limit), encourage less understanding and more proficiency (in the sense that the student has to be efficient in his approach). As the mathematics is in big part thinking, I think that it is beneficial to students to let them focus on problem solving rather than recalling and calculating (i.e. designing a solution rather than modifying a known one). There is a huge difference between work in time-constrained environment (e.g. medical teams, lawyers during trials, etc.) where the cost of "external knowledge" is much higher and good memory is essential. However, math is, in general (things like high-frequency trading are only a small part math-related professions), slow.

On the other hand, memory-oriented teaching is far from being a relic of the past. Why is this so? As this is a broad topic, I will make it more specific:

What are the advantages of memory-oriented teaching?

What are the disadvantages of allowing aids during tests/exams?

It is easy to explain the most immediate disadvantage of allowing "aids" during exams: many students misjudge the situation, thinking that having books and/or papers means they can study less. In particular, they often misjudge information access time.

But many students benefit from some form or degree of open-access exams, because they can relax a little about memorization and data.

And, indeed, in real life, although there are obvious benefits to having things in one's head, it's not possible to have everything there, and it's simply not the way things work. So it is a bit exaggerated to conduct exams in such artificial environments.

Similarly, although allowing "aids" reduces stress (and misleads some students), the necessarily-repetitive aspects of memory-oriented teaching/learning have their own benefits, that seem not achieved by less "primordial" teaching/learning. The point is not only memorization per se, but the repetitive aspect, which is often misguidedly downplayed in contexts where it is pretended that we all have nearly-perfect memories.

In brief: "drill" is pretty inhuman(e), or can be, but has benefits not achievable otherwise. Simultaneously, having books and papers available during exams is much more like real life, and changes the emphasis of the exam from memory to function (ideally), but will mislead a substantial number of students who've grown up trying to "game" the system.

But perhaps the real point is the exam-design itself, not the circumstances.

My own choice has been to always do open-book-open-notes exams, but/and designed to strongly favor people who have the information in their heads. This design criterion can be taxing to fulfill if one allows essentially unlimited internet access, for example, and the course is a cliched one like Calculus I. One device which I have found useful, both because it promotes useful skills and because it complicates "gaming the system" is the requirement of coherent writing, full sentences, good grammar. Not just skeletal formulaic junk splashed on a page... working toward a formula to be circled at the end. Explanations are not (yet?) so easily found by Googling. Of course I tell people in advance that they will need to be able to do this, and the homework would give feedback.

• The "misjudgement of information access time" (also re-understanding the forgotten definitions, and many others) does pose some serious problems. One way to solve it to make a "fake" test at first, to make them worry and let them fix it somehow (or make it matter only a little). However, there always are some day-dreamers (esp. in math) and those who suffer from this are frequently the I'm-smart-and-hadn't-ever-had-to-learn types (which too often results in drop-outs). BTW Modern technology makes it really easy to cheat and so I'm in favor of paper-aids-only exams. – dtldarek Mar 13 '14 at 23:53
• I try to leave questions that require access to a wide variety of resources for homework, making exam questions focused on concepts, and giving any detailed data in the question itself or as a set of formulas on the back of the exam. – vonbrand Mar 14 '14 at 12:56
• I'd definitely do paper only for 100-200 level math (and below). Wolfram Alpha could probably wreck all but the most conceptual calc 1 exam. It starts to have trouble with calc 2, but it can decently do up to multivariable vector calc if you know how to work it correctly. Hell, with a pro account you can get it to tell you the steps it did to solve the problem. – LinearZoetrope Apr 6 '14 at 5:09

I disagree with one of the other answers when saying that "math is not about memory". Doing math is not only about memory, but remembering your definitions and theorems can be crucial to doing problems. The argument that a mathematician can just look of these things on books disregards the fact that when doing the problem, you need to collect all the aspects of the solution in your mind (in some way). If you want to solve a problem in group theory, you need a good understanding of what a groups is. That is, you need to have memorized the definition and have internalized it.

So I would say that this is the answer to the question: "What are the advantages of memory-oriented teaching?". The advantage (or an advantage) is that to "see" a solution to a problem one needs to be able to mentally play with the ingredients.

What are the disadvantages of allowing aids during tests/exams?

The disadvantages are, then, that students might not have internalized concepts.

In addition to this, I also believe that certain elementary principles and rules need to be memorized. You are not going to do well on a calculus exam if you don't remember basic algebra rules. If you want to succeed as a mathematician (assuming that par of the goal of undergraduate mathematics education is to prepare future mathematicians), you will have to memorize things. It would be very difficult to have a conversation about mathematics if you don't remember things.

To sum up, my answer is: I don't think that open-book exams are bad. I think they are great. But, there is much value in memorizing/remembering.

• Yeah, it would be inconvenient if students had to look up rules for differentiation each time. On the other hand, problems that are given in open-book exams are hard enough to ensure that you need to have the basic definitions internalized. – dtldarek Mar 13 '14 at 23:39
• I'm not sure if you addressing your question to me or not, but if you are, I upvoted your answer. – dtldarek Mar 14 '14 at 1:43
• @dtldarek: I am addressing it to the person who downvoted my answer. (So not to you:)) – Thomas Mar 14 '14 at 1:45
• Right, but there is a gulf between necessary and unnecessary memory. Definitions and basic principles are one thing, but if something can be trivially rederived holding it in your head is a bit needless. Though, of course, that line is a bit fuzzy and different for different levels of advancement. And obviously memory is A marker of mastery, if only due to the fact that if you have extensive work in a field you tend to remember it. – LinearZoetrope Apr 6 '14 at 5:17

I allow notes on tests, because math is less about memory than about understanding, and I don't want students to focus on the memory part. I don't allow notes on quizzes, because they are on just one problem type, and I want students to be ready to think it through.

You may find this blog post helpful: http://exzuberant.blogspot.com/2012/07/monkey-and-mathematician-learn-calculus.html

Getting more into particulars: One disadvantage of allowing notes for tests is that students will spend too much time hunting for the answer in their notes. To offset that, I allow only a 3x5 card.

One advantage of memory-oriented teaching is that it can help students retain. But if that retention is bought at the expense of deeper understanding, you've made a serious mistake. In pre-calc, I ask student to show the end behavior of a function with their arms. This is a memory technique. My hope is that it complements the understanding of why odd-degree functions have "one arm up, one arm down".

• Sure they did, thank you. Besides, it is a good practice, to use the @user pattern, so that the person you are directing the comment to will receive a notification. Sometimes this is automatic (e.g. with no indication, the post owner will receive a notification), but in many cases it isn't (e.g. I didn't know about your last comment until now). – dtldarek Mar 14 '14 at 9:43

Many of the disadvantages of allowing aids can be, in principle, resolved by

1. requiring that the only aids the students have are handwritten by themselves and
2. setting a length limit. (I've seen somewhere between one index card [for non Americans: a piece of paper around 10 x 15 cm squared] and 4 pages of A4 [for Americans: 4 pages of letter paper].)

Ideally this will cause the student to be structured in their preparation (choose the important things only!) and the copying by hand of the relevant formulae should in theory reinforce learning (at least memorisation).

In practice, however, some common downsides to this include

1. Students grow to rely on the crib sheets. For more creative problems you would end up with students who spend the entire exam trying vainly to apply random items from their crib sheet to the problem, instead of actually trying to solve the problem.
2. There's transcription error somewhere: either when the student copied the formulae from his notes or textbook to the crib sheet, or perhaps his handwriting is bad enough that he misreads it during the exam.
• Good point! Another downside are students who spent days or even weeks to write as small as possible without leaving anything out. A few weeks ago I have looked at the crib sheets and saw that it was possible to write down all definitions and theorems onto one A4 sheet. (However, those students spent the whole exam on reading to tiny things and it was clear that they have not sorted out things). This can also happen additionally to your points. – Markus Klein Mar 14 '14 at 17:19

I’m a high school maths teacher and I use a way to test where crib sheets are allowed; yet pupils are rewarded for not using it.

At the start of the test, the pupils use black ink and are not allowed to use the test aids (such as crib sheet and calculator). When they want to, they indicate to me that they want to use their aids, at which point they must now continue their exam in blue ink. They can only do this swap once, the black pen is removed from their desk. Solutions written in black carry more weight than solutions written just in blue (typically indicated on the exam sheet).

There are many extra advantages. The pupils like to know that at some point they can check their work. They can even correct their work if they find they have made a mistake (which gains credit). Because of this, students learn to present their work better, since they themselves have to come back to their work to fill in the gaps at a later time in the exam. Students who are algebraically strong get their due credit. Weaker students can let the calculator do the algebra, so they can still solve the questions but knowingly obtain fewer points. If a student knows a formula, they can get credit for it. Finally, it is easier for me to write the tests as I don’t need to write two parts: one without aids and one with.

I admit, it takes one test to get the students to appreciate how the system works and how they can benefit from it. The feedback I have had is mostly positive.

• @JoeTaxpayer The feedback I received from my classes who used this method these last two years was mainly positive. The weaker the student the more they liked it. Some of the stronger students thought they needed to work out every last detail in black to gain full marks. But for me, this is part of the mathematical maturity. I have no interest if they can do 1234x5678 by hand, but I do expect them to know sqrt(144)=12 and use that; similarly, I expect them to know sin(30 deg)=1/2 whereas sin(40 deg) to use the calc. – Geoff Jun 2 '14 at 18:03
• @JoeTaxpayer Finally, as I now mark their big final exam (4 hours with all aids available) it is interesting to see how many wrote the exam as if it were in "black ink". For me, this is a big positive, since previous years, it wasn't like this. – Geoff Jun 2 '14 at 18:04

I'd like to expand on points that Thomas made. Learning math is like learning a language, and a certain amount of memorization (note: not necessarily drilling!) is a necessary component of that. To use a language, you need to have immediate mental access to the basic vocabulary and grammar. (In math, that means not only the definitions, but the fundamental results of a given field.) This is not just an issue when solving exam problems, but for further learning. A student in a linear algebra class isn't going to get much understanding of the proof of the spectral theorem, for example, if they're still struggling to remember what a linear map is.

In many of the classes I've taught, I've allowed students to have a sheet of notes (e.g., a course on differential equations where some of the recipes they're asked to apply can be easily mixed up) but with the following two features:

1. I talk a lot about how, like others mention above, if they're going to rely on what they've written down during the exam, they probably won't have time to finish the exam. I sell it more as a way for them to organize their own understanding of the material.
2. I ask them to turn it in to me a few days before the exam and I return it to them in class, on the day of the exam. This helps them not only to start the studying process earlier, but also helps them to not rely on it in the last few days before the exam.

One disadvantage of allowing "memory aids" is that there will then be little or no credit given for the appropriate formula. This makes it more difficult for a student with only a modest level of understanding to get a C, or even to pass.

• I see this as a clear advantage, as the goal of the exam is precisely to measure whether the student developed a good understanding of the content. If you want weaker students to be able to pass, then the syllabus should be easier to understand or have less prerequesites. – Benoît Kloeckner Jul 17 '14 at 21:13

I' m only going to answer one of the questions:
What are the disadvantages of of allowing aids during tests/exams?

First a point of terminology: Here in Australia, a sheet of notes you are allowed to take into your exam is called a "cheat sheet", and in Asia it's known as a "help sheet". In the USA it seems to be most often called a "crib sheet" or "crib notes".

Anyway I read quite a bit about this a couple of years ago, and you can see a short literature review in my conference paper http://www.merga.net.au/documents/RP_BUTLER&CROUCH_MERGA34-AAMT.pdf . To summarise: some people say it improves grades, while others find no significant difference; some think it encourages a deep approach, while others think it encourages dependency; most agree it helps students feel less stressed, whereas my research indicates it actually increases stress for some students.

The strongest conclusion is that different students will react differently to the opportunity to make or use a cheat sheet. From the surveys in my own research, we learned that students have widely varied expectations for what a cheat sheet is capable of, and indeed what an exam is for, and this coloured their view of how useful it was for study and stress reduction. If you choose to allow them, then you need to give students guidance in how to use them to study for understanding, how to study generally, and manage their expectations of the exam. For example, do emphasise that a lot of the exam is not about memory and there will often be several questions that are not the same as anything they've seen before (if that's what your exam is like of course). I have a video of a seminar for students on this topic https://www.youtube.com/watch?v=-6GqeqabP8k . Please excuse the strange clicking in the sound, I'll redo it with better sound quality when I get time.

I would recommend if you allow cheat sheets, to make sure that students are aware that they don't have to make one. Indeed, I would actually also provide a formula sheet with the exam and let them know what is on the formula sheet. They can use the formula sheet as a basis for making their cheat sheet, or only include extra information on top of it, or not make a cheat sheet at all. Giving them this choie helps them to have more power over their own study process.

• To summarize, you say that the disadvantage is that students don't know how to make and use the cheat/help/crib sheets and that the teachers need to help students with that; is this correct? – dtldarek Jul 17 '14 at 21:49
• @dtldarek That's pretty much the main disadvantage I see. I would make the stronger statement that students don't know how to study for exams generally, or use this study to actually understand, and teachers need to help students with that regardless of whether they have cheat sheets or not. – DavidButlerUofA Jul 17 '14 at 23:11
• But that's true for almost anything, that is, whatever you use, it needs explaining, even how to read a book or listen to a lecture is not obvious. In other words, I would understand your answer as "I see no non-trivial disadvantages of using aids during exams". I would expect there are at least some (e.g. you don't get to practice your memory). – dtldarek Jul 18 '14 at 14:43
• It's more "Even if you see no clear and widespread disadvantages, students still need support to learn how to study", which I think is a worthy point. And I did mention two disadvantages of encouraging dependency and increasing stress. – DavidButlerUofA Jul 18 '14 at 20:27
• Also, an idea I just had: if you want to test memory and also test things without testing memory, perhaps you should have some tests with cheat cheets and some without. The ones with can focus on problem-solving/understanding. The ones without can focus more on memory. Just an idea. – DavidButlerUofA Jul 18 '14 at 20:32

Let the student decide if they want crib notes. It should be a test that shows the student can problem solve. Why make it a memory test that causes the student to fail? Create the test so that the student is successful.

For most courses, I feel like you learn the material better if it is no notes on the tests. You've really internalized the material. It sticks with you better years down the road (so even if you need to look at a book then, it comes back fast since you really learned it down pat before). And then in near term, it gives you more mastery so the tools can be used very conveniently and quickly in physics, chemistry, engineering, etc.

Furthermore, I don't think most math courses really require crib sheets (it's not such a mass of tedium). What math courses require is doing lots of practice problems. When you do lots of practice problems, you're doing much more than memorizing the procedure...getting practice in using it and what mistakes to watch out for. So remember the formula itself is least of the worries. If you don't remember the formulas than you have a weak level of knowledge indeed.

In many cases, there is a choice between memorizing and understanding. E.g., a calc student can memorize that $$\cos'=-\sin$$, or they can understand that it's a phase shift of 90 degrees and figure out the direction of the phase shift by visualization. Understanding is better, but it requires three things: (1) a teacher who has this type of understanding, (2) a teacher who emphasizes and aids in this kind of understanding, (3) a student who is intellectually capable of it, and (4) a student who is willing to do it. An advantage of memory-oriented teaching may be that it works in cases where $$\neg$$(1 and 2 and 3 and 4).

Especially in K-12 and community college education, there are many cases where 1 is false. When you get teachers who lack this level of understanding, they don't believe that it's useful or can substitute for a certain amount of memorization. For example, when I was a kid I did $$6\times8$$ by remembering $$3\times8=24$$ and then doubling that. But I have spoken to a lot of kids who were specifically told never to do this kind of thing, because it would be too slow. If they scribbled this kind of thing in the margin of their times-table test, they were penalized. This is what you get from teachers who don't themselves have the habit of understanding. They have never had the experience of doing this kind of memorization-avoidance strategy successfully, so they have never had the experience of seeing how it becomes faster and faster, until eventually it's very much like memorization.

• It's a tradeoff. There are benefits to adapt and overcome. But also sometimes it can hold you back to rely on a workaround. I see this in sports a lot. – guest Dec 31 '18 at 18:16