Questions to test highest level of competency

Question

In mathematics we ask so many types of questions to check the student's knowledge of the subject. More oftenly we ask to define terms, state a formula or application of theorems. What would you suggest as the most appropriate way of testing students highest level of competency in mathematics?
Here are few suggestions,

1. Instead of just asking to define a term we can ask for comparison . As an example instead of asking what is mass we can ask to compare mass and weight or instead of asking what is point of inflection of a graph we can ask to comment on " point of inflection is a stationary point "
   
2. Instead of just letting them to apply relevant principle we can ask why that particular principle can be applied to the the question. As an example students can be asked to describe why the principle of conservation of energy can be applied to the given question or why particular substitution is valid to evaluate given definite integral.
   
3. Students can be asked to explain deviations of principles from usual way of applying or weak points of a principle such as proving second derivative need to be positive at a point to exit a minimum at that point is not sufficient .
   How would you like to grade these kind of approaches and what other options available for this purpose?

Answer 1

As asked, it is a bit too general to answer with anything but other generic words. So let's play with some simple particular topic like finding maxima of functions on intervals.

The lowest level: find the maximum of $f(x)=x^3/3-1.5x^2+2x-1$ on $[0,3]$ with explanation.

The next level: same problem for $g(x)=|f(x)|$ and $h(x)=\cos(f(x))$.

One up: Given that $f(0)=1.1$, which one is larger: $\max_x f(x)$ or $\max_x (2f(x)-f(x)^2)$ (on $[-1,1]$, say)?

One more up: Find the maximum of $1-x^2+x^3\cos(e^x-50x^4)$ on $[-1,1]$.

The highest one I would bother to check at the undergraduate level: Prove that $\max_{x>0} (1000x^2-4^x)\le 50000$.

Those are (IMHO) in the order of increasing difficulty and almost decreasing tediousness. I hope this example conveys the idea clearly enough so that you can invent a similar series in any other topic yourself.

Answer 2

What you might be looking for is cognitively demanding questions. Students/Pupils have typically comparatively easy time with algorithmic exercises (including the kinds of exercises, calculations or proofs, that they are used to doing) and can also learn to regurgitate conceptual knowledge like repeating definitions or remembering that a positive derivative means an increasing function.

Combining these two is more difficult.

One source for these things is Markus Hähkiöniemi's doctoral thesis: https://jyx.jyu.fi/bitstream/handle/123456789/22530/1/rep104.pdf (might have been one of the included articles, which you will need to dig up separately)

One way to look at the matter of representations for functions is attributed to Janvier. A function can be represented as

1. A formula
2. A list of inputs and outputs
3. A graph
4. A narrative description of a situation

The transitions between these are the key and you can vary these. Give the formula to a function and ask to describe a scenario where it could be used. Provide a graph and ask to recover the minimum and the maximum value and the points where those are reached. Give a list of (x, y) -coordinates and ask for a function that goes through them.

Some of these tasks are obviously easier than others, but what is important is that they require a more broad understanding of what a function is than the student/pupil might otherwise need. And one key idea in didactics of mathematics is that the conceptual understanding a student/pupil develops is the simplest one required to solve the problems they meet.

Janvier's idea is attributed to the following article, though it might be a bit implicit there:

BELL, Alan; JANVIER, Claude. The interpretation of graphs representing situations. For the learning of mathematics, 1981, 2.1: 34-42.

The idea of the students/pupils using the least complicated understanding of an idea that carries them forward I picked up from Matematikk på småskoletrinnet by Bjørnar Alseth, but it, too, is sure to be a more broadly known idea.

Cognitively demanding tasks

These are in the order from least to most demanding.

1. Memorizing
2. Procedures without a context (practicing a given algorithm)
3. Procedures within a context (you have to choose which algorithm to apply)
4. Problem solving (the problem is new in the sense that you do not have an algorithm to use on it)

I looked these up from Anita Valenta's article for Matematikksenteret: https://www.matematikksenteret.no/publikasjoner/kognitive-krav-i-matematikkoppgaver . The PDF there has further references, also in English.

Answer 3

I think you're better off concentrating on performance of actual problems versus memorizing definitions or theorems, when teaching math for general STEM. It is much more lasting to anchor something like exponent laws by doing problems with exponents (and being tested on them), versus just regurgitating them. In addition, much of math has a heavy algebra component and this is a weakness of kids. So, problems that require them to use several lines of algebra properly are a feature, not a bug.

I wouldn't completely stay away from "concept style" questions. But don't make them the main objective. So, for example, students learning second order constant coefficient equations should be tested with the ability to do a multi-step problem (abstracting the characteristic equation, solving the quadratic, maybe dealing with repeated roots, finding the particular solution if it is a heterogenous, and even perhaps finding the values of the C1 and C2, based on initial conditions.). If you want to ask a question about under/over/critical damped, fine. Makes sense. But that's a little garnish for the stew. The main thing is they need to be able to solve for y.

And yes, doing all the darned algebra involved in that, and properly. This will be needed/expected in their electrical engineering classes. It's OK and even optimal if you push the math problem first, rather than an electrical or control circuit. That should only be brought in later after the student has shown the ability to manipulate the math itself. Then (if there is time), they can later learn about how capacitors work different than inductors or what the heck a dashpot is. And if you don't get into it, they will get that in their majors courses. But it will go a lot smoother if they already have some familiarity with the ODE, while they are learning the physical components.

If you must push the concept stuff, there are ways to go about it that are a little more fun than recitation. T/F questions. Evaluating drawings. What has to be right for this relation to be true. Fill in the blank. Etc. But I would urge you not to jump too deep into "higher order concept" land and omit the needed work on solidifying their basic skills in the content. For one thing, these concept questions can be very logic/verbally demanding (the SAT question stuff) that ends up being a bit of a general intelligence test at keeping track of double negatives or divided by zero exceptions or the like. It's not that those are completely unimportant, but I just wouldn't emphasize them at the expense of normal calculations.

Of course in a class on logic or proof writing, the whole point is to learn to be careful about language and conditions, so fine. But this is not the case in most of the algebra/algebra 2/trig/analytic geometry/calculus/ODE/PDE coursework of an engineer. And even their (rather limited) needs in linear algebra should be more demonstrated hands on in terms of manipulating matrices versus proofs (and certainly versus tricky questions).

In particular, if you are teaching kids that are below elite (not CalTech, not Thomas Jefferson), it's probably especially important to be more traditional. They need the practice in multi step algebra. It's not trivial to them.