How to design a fair oral exam for >10 students?

Question

Oral exams take place at two different points in during their study:

1. As final examination to end their study (i.e., when the students have finished all necessary courses, they have an oral exam about 2-3 courses in a field, e.g., an oral exam about PDE and numerics of PDE).
2. As an alternative of exams for specific courses (i.e., there is are timeslots on 1-2 given days after the course has ended, and the students come there for a 20-30 minutes oral exam).

If you have maybe about 10 students in your course, you are not in charge to set a lot of rules what and how to ask at such an oral exam, but for more people it is hard to be fair since you either ask exactly the same questions (where the students coming at last have advantages) or the oral exams differ too much in the topics of the course or in difficulty.

In oral exams which end the study is it even more difficult since you examine the same content over years and decades.

What are good guidelines to have fair oral exams in both cases?

Answer 1

Cook up say 40 approximately equally hard problems covering the subject area, and have the students pick say 3 at random, could even go for without replacement (but that requires more problems). Might also go for say 15 problems in each area, and have them pick one of each. Or have a shading of problem complexities (2 easy ones, 1 medium hard). Make your imagination run wild.

Beauty is that your collection of problems will grow over time.

Yes, problems will get known (that's why you should go on adding new ones constantly, and perhaps retire ones "too many times asked" or some other criteria). But if someone is able to learn enough to solve a wide spattering of 40 problems or so, I'd contend they merit a passing grade anyway.

To make grading fair/uniform, set up a grading schema (might as well publish it): Solve the problem correctly, 6 points; use correct notation/notions, 1 points; answer 3 questions from the committee, 1 point each. Or something similar. Perhaps each of the points on a scale of 0 (nothing), 1 (partial), 2 (complete); I'd even add 3 (more than was asked for). Agree on a reasonably precise meaning of each grade among the grading committee(s) to ensure fairness (each crop and along time).

Answer 2

Usually, the course material is more than you could talk about in 20 - 30 minutes. This means that you have a large array of questions at hand, and you usually won't be able to ask every student the same set of questions.

One of the oral exams I took (iirc, it was linear algebra) had an interesting mechanism which removed some initial-quesion bias: Before the actual exam, you were presented two boxes of index cards. One for definitions, one for theorems, but the cards only contained the names. You (randomly) picked two of these and were to prepare these on a sheet of paper within 10 mins. After that time span, the actual exam started. These written-down versions were somewhat a conversation starter, and showed either some need to delve further, or skip to a different topic.

Generally speaking, it should be somewhat clear to the students which questions are to be expected, and which theorems are the important ones. If you don't want the students to be talking behind your back about the questions, just be up-front! Important theorems deserve special attention, and remarks like "When I was TA, many oral exams involved this theorem.", or "This kind of example could be asked for in an exam." are a sign of a good lecture, imho.

A final note: If you do oral exams regularly, students will start to write down what was asked and share this with other students. At one university, this went so far that for one professor, there wasn't only a binder with the mind protocols of the students, but actually a TeX'd "walkthrough" as well.

Answer 3

(Migrating my comments since they appear to serve as an answer.)

Why are the "students coming at last" in an advantageous position? Can they hear all the questions being asked of the students before them? If so: Does it suffice to administer half the examination student by student, and then administer the latter half in the reverse order? (E.g., if your students are A, B, C, then the order of testing is A-B-C-C-B-A.)

Your response is that this would be helpful in case one (a particular course) but not case two (where examinations are similar over a period of years or even decades). You also note that students mention questions and sometimes answers to one another during the examination period. To this I respond:

First, let students know that sharing questions/answers during the examination period is a violation of the school's (or your) academic policy, and will lead to failure if discovered. Second, I do not believe there is a shortage of problems (e.g., in PDEs), but if you can distill the essence of a subject down to a small enough collection of ideas such that memorizing a set-list of heuristics will allow a student to pass, then it seems to me like this would be a good thing. I would not consider it a cause for concern.