9
$\begingroup$

Our university and department, like most, are required to "assess" our program annually. In the past our department has done this by e.g. picking one of the learning outcomes for our program and a final exam question in one or two classes that are related to that outcome, and coming up with a numerical rubric to evaluate performance on those exam questions. But this doesn't really tell us much that we didn't already know, and no matter how carefully chosen, one exam question can be misleading and not representative of an entire program.

What successful ways have other mathematics departments found to assess their undergraduate programs and determine whether learning outcomes are being achieved?

Improve this question
$\endgroup$
6
  • 5
    $\begingroup$ You've successfully detected the fact that this is a nonsensical management fad, imposed by accrediting bodies. The constraints they impose are incompatible with using these things for any useful purpose. $\endgroup$
    – user507
    Sep 3, 2015 at 23:40
  • 3
    $\begingroup$ Sigh. My condolences. As @BenCrowell notes, such stuff is nonsense. But, perhaps not "nevertheless", it is popular with administrators and bosses generally, because any small openings can be used as "reasons" to not give you money/raises/space/security. The key point is two-sided: be sure that whatever you do mimics the admin-speak of the requirements, and that no one wastes too much time on it. The recipients of your self-assessment do not care about any of your goals, nor do they care about your students, either. You and your students are just pawns in adminturf stuff. Don't get suckered. $\endgroup$
    – paul garrett
    Sep 3, 2015 at 23:47
  • 1
    $\begingroup$ Assessment in this context is a "tale told by an idiot, full of sound and fury, signifying nothing." Just figure out how to give them what they need without wasting your time and that of your colleagues. Throw together a few grids and be sure to use appropriate wording and phrases (goals, outcomes ,formative, summative, rubric, closing the loop, etc.) and never say that you use grades for assessment. $\endgroup$
    – user52817
    Sep 4, 2015 at 15:18
  • 2
    $\begingroup$ Perhaps I got us off on the wrong foot by mentioning official requirements. What I would really like to know is, suppose for the sake of argument that we would really like to "assess" our program in the sense of getting useful information about the program as a whole that could help us redesign it to better serve the students, and that we are unencumbered by any particular bureaucratic requirement as to the form that this "assessment" should take. Surely this is not a pointless goal. How should we do it? $\endgroup$
    – Mike Shulman
    Sep 4, 2015 at 16:42
  • $\begingroup$ @MikeShulman: Start from the pedagogical literature. $\endgroup$
    – user507
    Sep 5, 2015 at 2:17

4 Answers 4

Reset to default
7
$\begingroup$

Two questions from Lynn Arthur Steen are perhaps not addressed frequently, but seem important to me:

(1) Are students learning how to use mathematics in other subjects? Do students recognize similar mathematical concepts and methods in different contexts?

(2) Do faculty take responsibility for the quality of students’ learning?

These are from "Asking the Right Questions," in Supporting Assessment in Undergraduate Mathematics. MAA Supporting link.

The MAA (Math Assoc Amer) also published Assessment Practices in Undergraduate Mathematics. MAA Practices link. The latter publication likely addresses your "What successful ways" question, but I am in no position to summarize this 292-page book.

Improve this answer
$\endgroup$
3
$\begingroup$

I was put in charge of assessment for our community-college math department as of last year. One resource that was immensely useful was the UMass "Program-Based Review and Assessment" handbook (link). There's an example of a math department assessment plan on p. 28.

That said, the process highlighted there is exactly as you describe: score selected questions embedded on final exams. On the one hand, we're fortunate in the math discipline that assessment is so amenable to our existing testing practices; on the other, it does look a bit silly and redundant. The silver lining for me is to know about specific skill mastery and not overall test or course averages.

The one thing that mystifies me is the practice of checking only one objective per year; that doesn't seem like the intent to me (although the UMass document, for example, does say "if you and other members of your department can agree on only one goal, don’t let this stall your progress. Focus on that one goal – more will come later." [p. 11]). Assessing just one goal per year seems like you'd be going a half-decade between pings on any objective to know whether they'd degraded or improved or whatnot, which seems crazy.

What I'm doing is to establish fixed questions in each of our terminal courses, and so assess every one of our objectives every year in a consistent manner. Hopefully this actually leads to less effort, because we won't have to spend mental energy creating new plans (re-inventing the wheel) every year; will also provide a "dashboard" of information on critical skills to compare year-to-year; and serves as a resolution to your very valid "one exam question can be misleading and not representative of an entire program" issue. Note also this is just the terminal courses (last ones to complete the degree), which I think meets the spirit of program assessment (what do students know when they complete our time with us), and also conveniently has a relatively smaller number of sections, and students, and instructors to wrangle for the task.

As extra steps I'm also receiving prerequisite grades and academic majors for all our students (to perform cross-correlations from the prerequisite course to identify particular strength or weakness), and sending out a survey to alumni this month for observations, how they did afterward, things they wish they were improved, and perceived strength on each of our program objectives. But that's definitely extra-credit for this task (I was expressly told by our Dean of assessment not to do that because it would be too much work), and can certainly be skipped if you just want to minimally meet accreditation demands. However, I did want to add a bit more depth/texture to our understanding of how our students are doing in later academic programs.

Improve this answer
$\endgroup$
1
$\begingroup$

Honestly, if you really want to see how your program is doing you need to rely on some external testing. Perhaps the major field assessment test (MFT), or, the subject math GRE. These are a start, but, I don't think they quite uncover the higher end of undergraduate learning. For example, I took courses on manifold theory, general relativity, fiber bundles, axiomatic classical mechanics and a few other things. None of that would have even registered a blip in those tests, yet, I can tell you to a certainty they were huge in my overall education. Often these assessments are about the bulk of the students, but, what really speaks to the quality of a program is not average student's progress. In my estimation, the bigger question, what are you offering the best students? What are you offering the students whose heart is more with the study of mathematics than the acquisition of a credential? I'm not sure I know how to "assess" that, but, I know it matters.

Improve this answer
$\endgroup$
0
$\begingroup$

You declare a number of learning outcomes already (for the program and presumably for the courses too). The learning outcomes for each course should be assessed by the grading (could e.g. track them by looking at the grades in individual questions of midterms/finals, check some other outcomes by results in term papers or such).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.