How can I learn to write better questions to test for conceptual understanding?

Question

I'm worried that I'm bad at realizing when a question I've written requires little or no conceptual understanding to answer. Like, when I'm writing a question for a homework assignment or exam, I'll be thinking of it conceptually because that's how I've learned mathematics. But I don't know how to ensure that the question demands that a student think conceptually. Here's a silly minimal example calculus question to illustrate what I mean:

Suppose that a truck's distance from you in meters at a time $t$ seconds after the big bang is given by the function $p(t) = \sin(t)+42 + \mathrm{e}^t$. What is the velocity of the truck $t=19$?

I wrote this question thinking that you need to understand the whole "velocity is the derivative of position" idea to answer. But really a student could answer this question by noticing that it has a function $p$ in it, take that function's derivative because that's the only thing you do to functions in an differential calculus class, and then plug in $t=19$ because why else would that $t=19$ be there.

How can I learn to write questions that require a student to think conceptually/deeply to answer?

Or if you want to ask this question from a pessimistic/adversarial perspective, how can I learn to write questions that'll thwart some learners' habits of cramming immediately before tests or exams so they can robotically reproduce what they've crammed, without ever really having understood it?

I'm interested in resources that address this, but I'm not sure such resources exist, so sage advice is very welcome. :)

Answer 1

Agreeing with comments and other posts: If you want more conceptual answers, give them less details in the set-up.

Using your velocity problem, here are a couple of examples of making it more conceptual:

• Suppose that a truck's distance from you in meters at a time $t$ seconds after the big bang is given by the function $p(t)$. What does $p'(19)$ tell you (and what are its units)?
• Suppose that a truck's distance from you in meters at a time $t$ seconds after the big bang is given by the function $p(t)$. What does it mean if $p'(a)<0$?

In the absence of details, the students cannot just 'plug and chug' naively and are forced to actually think about the pertinent concepts.

Answer 2

Asking students to explain why something happens can be useful for assessing understanding, although it is often harder to grade and works best with many demonstrations before the exam. (Students need to know what your expectations for a thorough explanation are.) I have found that asking students to critique a process will sometimes help me assess their conceptual understanding. I give them a sample problem and solution, and ask them to analyze the work. As an example, after introducing the indefinite integral but before discussing techniques like $u$-substitution I will give my students the following problem.

• A student is trying to find the following antiderivative $$\int xe^{x^2}\;dx$$ and comes up with $$\int xe^{x^2}\;dx = 2e^{x^2}+C.$$ Is the student's work correct? Convince me either way.

Answering this correctly requires students to (1) understand that differentiation of the antiderivative should result in the original integrand, (2) differentiate correctly, and (3) state a conclusion. Simply finding the derivative of $2e^{x^2}+C$ isn't enough.

I also second Aeryk's suggestions, and would add that asking about units can be a good way to go, such as

• If $r(t)$ represents the rate in grams per minute at which a bacterial colony is growing at time $t$, what are the units of $\int_2^7 r(t)\;dt$?

Answer 3

I agree with @BrendanW.Sullivan's comment. That is, when teaching an undergraduate course, like calculus, students need more than procedural knowledge. For a deeper understanding, and efforts to evaluate such, students should be asked on exams to answer a few "free form" questions, like the one Brendan suggested.

A good question to ask following any question you ask of students in class, in questions you assign for homework, and/or in questions you construct for an exam: "How/Why does your solution make sense to you?"

Conceptual understanding refers to an integrated and functional grasp of mathematical ideas. Students with conceptual understanding know more than isolated facts and methods. They understand why a mathematical idea is important and the kinds of contexts in which is it useful, and why it is useful.

Further, you want to convince your students to realize that success in your course (and math in general), requires more than just "getting the desired answer by mere procedural algorithms." To enhance this sort of learning (conceptual learning), begin modeling in class, to help your students learn the sorts of questions they should ask themselves to develop a deeper, conceptual understanding of material you are covering; ask them conceptual questions about recently covered material, and ask for input from the students, perhaps even asking students to work in pairs or groups of four, e.g., to answer such a question, review group-work, offering hints, or, if necessary, answer conceptual-oriented questions in class, so that students have some familiarity with that mode of learning, and can better, then, replicate it in an exam.

If you do the above regularly and consistently, your students will be more familiar questions to assess conceptual understanding in an exam.

See Teaching procedural and conceptual knowledge.

Also see Assessing conceptual understanding in Mathematics, where you can download the full pdf.

Answer 4

As stated in other comments, try not to "invite" your students to merely apply rules they do know. In the specific example you mention, I would prefer a multiple choice question of the form:

A truck's distance in metres from you as a function of time $t$ (in seconds) is given by a smooth function $p:[0,+\infty)\to\mathbb{R}$. Knowing that $p(19)=12$, $p'(19)=24$ and $p''(19)=3$, the velocity of the truck at $t=19s$ is:

1. 12 m/s,
2. 24 m/s,
3. 3 m/s?

Accordingly, explain your answer.

Under such a framework, one cannot avoid some conceptual processing - or exhibit, at least, the absence of it.

In general, the more abstract the context of a question is, the more possible it is that a student invokes conceptual processing.

Also, when you invoke some context - e.g. motion - do it for a reason and not just so as to make the question more "fancy". I've made this mistake several times myself.

Consider, for instance, the following questions, which both refer to the same issue:

• Find an antiderivative $F$ of $f(x)=4x+3$ such that $F(0)=3$.

• Let $f(t)=4t+3$ describe the way a particle's acceleration changes over time. Is it possible that the particle has a zero velocity at some time? From all the antiderivatives of $f$, which one describes a particle that starts from inertia and constantly accelerates with positive acceleration?

The second involves much more in-depth knowledge of the relation between a function and its antiderivative than the first one.

Answer 5

What one has to do to test for conceptual understanding is hard to state in terms of general principles (although Polya's books on Plausible Reasoning do a pretty good job of addressing the issue) and maybe is best addressed via examples.

Here is one example. Consider a cubic polynomial in one variable that is increasing as a function of its argument. Choose one that is not easily factored. One could ask to find its critical points and decide their nature. Or one could ask how many real roots the polynomial has. The first question is mechanical. One has to know the definition of critical point and some test to decide the type of a critical point. To answer the second question one has to realize that for the indicated polynomial to have more than one root, it must have a local minimum at which the value is negative. Then one has to do the same mechanical operations as in the first question to decide whether this is the case. The way the problem is posed forces much more serious thinking. Usually one has to ask both sorts of questions because the student who cannot solve the first has no hope of solving the second.

Another example. Students can be asked to evaluate an integral whose evaluation can be effected by making a change of variables. An alternative, which both isolates the change of variables issue and avoids recipes is the following. Give them the value of an integral such as $\int_{-\infty}^{\infty}e^{-x^{2}/2}\,dx$ and ask them to find the value of $\int_{-\infty}^{\infty}e^{-3x^{2}}\,dx$ or $\int_{0}^{\infty}e^{-x^{2}/2}\,dx$ or $\int_{0}^{\infty}e^{-3x^{2}}\,dx$ (the last is slightly harder than the first two, combining the issues present in both).

I have often used variations of both of these examples (and there are many more). They are effective for discriminating among those who understand more and those who merely have mastered formal manipulations. On the other hand, a test full of questions like these will not yield good results in most classrooms in most universities (it would be too hard).

These sorts of questions force deeper thinking without requiring that the student make it explicit, something which is out of reach for all but the most interested and enthusiastic students. Understanding, at whatever level, has to be evaluated operationally. At an introductory level it rarely works to ask "explain why ..." When many students can answer this it is usually the case that they are parroting phrases repeated often by the teacher, and then not much is being learned.

Answer 6

If your assumption on how the students may get the right answer, maybe rephrase the question in a way where doing that is the wrong answer. For instance, you could ask for acceleration rather than velocity.

Still, I don't think many students will get the right answer by following the logic you mentiond without having an understanding on what's behind!

Answer 7

One way is to use simple, atypical mathematical objects, fringe cases, non-examples and counterexamples, etc.

Answer 8

The "deeper" or at least off-beaten path solution should be much quicker than a standard but boring approach to create a system of equations or calculate a derivative or something like that. Give two-three problems and limit time so that they can solve them in time if they apply some sort of a trick. You'll get a lot of angry students :)

Here is an middle-school example, no derivatives, it can be solved with a system of linear equations:

Every day an engineer arrives to a station at 8 a.m. by train. Exactly at the same time a car, sent from a factory, drives up to the station, picks up the engineer and takes him to the factory. One day the engineer arrived at 7 a.m., decided not to wait for the car, and started walking towards the car. When the car met the engineer, it picked him up, turned back and arrived to the factory 20 minutes earlier than usual. For how long did the engineer walk? Consider the speeds of the engineer and the car constant.

Or you can solve it literally in 30 seconds if you think about what is happening, and use 3rd grade arithmetic.