10
$\begingroup$

Assume that you have incoming college freshmen, so you might have access to their high school courses and grades, as well as their results on a placement test offered by your college. Also, assume that these are students who will need to progress along this sequence for their major (in chemistry or economics, e.g.) and will take the course in which they're placed during their first semester.

What are some effective methods to assess the abilities of incoming students and properly place them into these courses? Should you incorporate other data (e.g. high school courses and grades)? When designing a placement test, what are some considerations to be made to make it a good and effective test? What kinds of questions should be asked, and how should they be scored? What would you make you more apt to "bump up" a student beyond their placement, if they asked for it?

I am interested both in research about these questions (although I suspect it might be limited) as well as personal anecdotes/opinions/recommendations, especially if you were involved in designing and implementing a placement test.

$\endgroup$
  • $\begingroup$ I'm not familiar with the US, but around here in Chile high school grades (and contents really covered, as opposed to what is declared) are all over the map, I wouldn't place too much faith in them. Other than that, a single test won't be enough. Perhaps have a short time of "overall assessment" (not tests, but look how they fit in, how they do with homework, do they know how to study/organize their time, ...) would be ideal (if impractical). $\endgroup$ – vonbrand Mar 19 '14 at 3:04
  • $\begingroup$ From my own experience as a student, I started college after not having taken a math class in 10+ years and voluntary took College Algebra even though my classes, grades, and test grades should've placed me in Pre-calc due to time out of a classroom. As a tutor, I think that any student who has not had a math class in over 2-3 years will benefit from being placed in a class lower than the one for which they qualify. $\endgroup$ – David G Mar 19 '14 at 3:09
  • $\begingroup$ (One useful question to ask here, though I realize it's not the OP question, is also whether people are allowed to place above or below their "official" placement due to not enough courses being available or for other degree requirement reasons - or for "easy A" reasons. I know that is not part of your assumptions, but I think it's reasonable even under those assumptions because people change majors, etc.) $\endgroup$ – kcrisman Jan 23 at 14:56
6
$\begingroup$

In recent literature, this issue would be referenced as multiple measures placements. There's been quite a bit of work in this area in recent years, and a number of places cite positive results (in terms of shortened time to gain more credits, etc.). The displayed effect is usually stronger for English, and weaker for math. It certainly makes intuitive sense to me that using more data for decision-making ought to be more robust. However, I'm a bit suspicious of some of the findings because of ulterior motives by the institutes pushing them; usually these are joint with efforts to reduce time in school for students, reduce funding for developmental education, reduce costs for placement-testing, etc. For example, Columbia University's Teacher's College has been at the forefront of the push to end developmental classes in colleges.

Here's an overview article at Inside Higher Ed (2018). At SUNY 2016-2017 a disjunctive rule ("or") was used; students could qualify for certain classes with either a high Accuplacer score, or high school gpa (HSGPA), or state exam, or high school rank. Reported results are 14% of student were placed in a higher math course, and 41% in a higher English course. Those who did get placed higher than the placement exam would have received supplemental "corequisite" support programs throughout the semester. This resulted in 12% more students completing credit-bearing English classes in the first semester, and a 3% increase for math.

California in 2017 passed a law requiring community colleges to use multiple measures for placements. One paper supporting this was Ngo, et. al., "Course Placement In Developmental Mathematics: Do Multiple Measures Work?", USC Rosier, 2013. The practice at the time was to use a placement test as the basis, and then add or subtract some points based on survey questions and HSGPA. The overall results are broadly similar to the SUNY findings.

Note that the practice on the ground has changed rapidly between the time the OP asked this question (2014) and when I'm writing this answer (2020). For example, CUNY (where I work) has both announced the complete dissolution of developmental courses (in English, arithmetic, basic algebra, etc.), and also eliminated all system-wide placement testing. The current theory from the central university follows the SUNY practice above; any of several disjunctive criteria will qualify students for courses (SAT, or HSGPA, or state exam, or passing the class in high school, etc.). As noted, this should cut costs for placement testing and developmental education (namely, from classes that 80% of students would take to zero).

However, among the difficulties this poses is that very close to half of the students at my institution are foreign transfer students, and do not have standard high school transcripts, SAT scores, or state exams. The CUNY policy in these cases seems to promote accepting self-reported high school grades for this purpose.

Among the presentations that were distributed to us to support these decisions was one by John J. Hetts of the Educational Results Partnership (ERP), a non-profit working in this area and funded by corporate partners such as the Gates Foundation, Chevron, AT&T, etc. Hetts has a number of variations of a slide deck under the title "Let Icarus Fly"; one can be found here. This gives a glowing report in favor of multiple measures (or disjunctive measures as I'd call the SUNY/CUNY practice), quickened student time-to-credential, and reduced costs. However, some of the findings seem suspect. E.g., writing in favor of self-reported HSGPA, he writes that, "Under-reporting was 2-4X as common as over-reporting." (p. 45). But if we inspect the table on the prior page supporting this statement, we see that what under-reporting occurs is effectively negligible (e.g., in the topmost bracket, –0.04 point on average, for 1% of total GPA, N = 599), while the over-reporting is hugely significant (e.g., in the lowest bracket, +0.82 on average, or 80% of the whole GPA, N = 85). So: The students with the worst grades in high school show the highest propensity and payoff for misreporting grades to their advantage. If that wasn't an obvious outcome at the outset, one could reflect on principles such as the Dunning-Kruger effect, Campbell's Law, or the Freakonomics podcast episode on institutional-scale cheating in schools which pointed out what "school administrators were unable (or unwilling?) to foresee... when a rule change gives people incentive to behave badly, a small share of them inevitably will".

In summary: Using multiple measures for college placements (such as placement exam, high school GPA, etc.) has rapidly become widespread in recent years, and on its face should be a reasonable idea. However, the ulterior motives of the organizations pushing these findings leaves one somewhat skeptical of the very positive results that they assert (and some of the interpretations seem downright disingenuous). Moreover, the findings have been very rapidly taken up to counter-intuitively abolish all placement testing and courses like basic algebra at many large colleges. Keep in mind that even the most positive proponents show the most significant acceleration in English programs, and small or negligible accelerations in math programs.

$\endgroup$
  • 1
    $\begingroup$ Thanks for this good start at the US context. I think there may also be a nontrivial literature on different single placement strategies (Maple, SAT/ACT, home-grown placement) but I don't have it to hand right now. $\endgroup$ – kcrisman Jan 23 at 14:52
  • $\begingroup$ On a related note, do you have some links about some of the literature regarding relative (in)effectiveness of certain precursor courses (ineffective in the sense that people taking them do not mostly end up enrolling in the course that the precursor course is preceding)? I have read some of these regarding precalculus, but for "developmental" courses it's different because a lot of those students end up not persisting in the program, from what I have read. Anyway, if you have those references available, that would be very useful for the OP too. $\endgroup$ – kcrisman Jan 23 at 14:54
  • $\begingroup$ @kcrisman: I know what you're talking about, but I can't immediately find it, and I think it may be too off-topic for this particular Q&A. Columbia (as noted) likes to make this point that for students placed into math remediation, only 33% complete a credit program. (I think at each step maybe 10% who pass do not register for the follow-up course from memory)... $\endgroup$ – Daniel R. Collins Jan 23 at 17:28
  • $\begingroup$ However, a study by Long at Harvard (2005) found that students assigned to remedial math were 15% more likely to complete a degree than those (with similar entry qualities) who were not so assigned. $\endgroup$ – Daniel R. Collins Jan 23 at 17:28
4
$\begingroup$

What we found at our university was that most students who took the placement test were taking it to try to get into a higher mathematics course than their ACT score would allow. Taking these kids out of the mix, the next largest group would be the adults who are returning to school after some time off. What we have come to realize, is that you don't do the students any favors by putting them in a class for which they are not prepared. We have found that the ACT test does a fairly good job predicting mathematical ability. For those reasons, we rarely let a student into a higher mathematics course than what they actually tested into. If we do, we require the student (or parent of the student) to sign a paper that states "with your placement test /ACT test scores of ___ we disagree with your decision to take ______ and suggest that you reconsider your decision and take ________ instead, which is more in line with your placement test scores.

$\endgroup$
  • $\begingroup$ "most students who took the placement test were taking it to try to get into a higher mathematics course than their ACT score would allow" Does this mean that the placement test was not required for others, that they would just accept whatever placement they're given? How was that placement chosen? $\endgroup$ – Brendan W. Sullivan Mar 20 '14 at 3:17
  • $\begingroup$ Students just out of high school can either take the placement test, or we will use their ACT scores to put them into a math class. If they make a score on the placement test that puts them into a higher math than their ACT score would allow, then they can choose which math to go into. The placement test is basically a combination of the previous semester's the Algebra, Pre-Calculus, and Calculus I final exams. So basically the student must pass the final exam, which is comprehensive, in the subject. $\endgroup$ – Todd Thomas Mar 20 '14 at 12:15
3
$\begingroup$

In Spain such placement is impossible, because university curricula are fixed in the first few years (there are no choices to be made). On the other hand, entrance into a given degree program is based on a mix of high school grades and performance on (state level) exams administered at the end of the final year of high school (these cover high school calculus and linear algebra). In a sense the placement is made at the level of the degree program. Where I teach there are 7 different engineering degrees all of which have essentially the same first year courses (the calculus course is the same for all) but which have quite different entrance requirements. Looking at ten years of data (so about 6000 students in total) student performance in essentially the same class is strongly linked to the degree program, so implicitly strongly linked to entry level, that is to performance on the exams taken at the end of high school and grades in high school. A reasonable inference is that these exams are effective in discriminating readiness. The conclusion is that a well designed exam should facilitate placement, particularly if coupled with information about performance in relevant high school courses. The test to which I refer covers a sort of low level approximation of what students will encounter in the university. For example, with respect to calculus, it tests operational ability more than conceptual understanding, so a student who has done well on it at least knows the integrals and derivatives of polynomials and trigonometric functions and the like. Such a student is prepared for a university calculus class. A rule of thumb is that one wants to test something like the material one will teach in the first two or three weeks of a course. A student who can handle that material will probably do well in the course, and a student who cannot should probably take a predecessor course.

$\endgroup$
2
$\begingroup$

I suggest to research the placement tests and psychrometry of USMA, USNA, USAFA, etc. They have a very long experience in doing these tests and assessing how well the resultant students do in their courses. You could even reach out to someone in one of the departments and see what insights they give you.

In general, whenever you are looking for large data sets of assessment tests, performance history etc., it's worthwhile to look at the military. Both lower end like the ASVAB and up to more detailed assessments (pilot aptitude). They're not perfect of course but they do tend to have large populations, reasonably stable quality of entrants and stable course demands. It's much more statistically insightful than random one off studies you see in many ed journal articles, with low statistics or poor controls.

See here for some general practice in calculus: https://www.usna.edu/MathDept/academics/placement.php [but I think a detailed search/outreach would get you more.]

Note: They're basically sorting people into pre-calc (a small but not insignificant fraction of the class) versus standard engineering calculus. So you may need an easier test for sorting people into lower levels of pre-calc.

For validation of calculus, I would go by the standard 4-5 of AB gets you calc1 and 4-5 of BC gets you calc 1 and 2. For kids that didn't take AP but want to validate, just have some assessment that's similar (probably last year's calc final or the like).

I probably wouldn't waiver kids getting out of where the placement put them and would keep it simple by ignoring grades, SATs, etc. Of course if you want to create something more complicated you can. There's an argument that you get more correlation to end results by including more indicators. But I doubt you have really trained the formula well enough for that. And also psychologically it's just easier to tell the kids you're driving it off the test performance (put up or shut up!)

If you were going to waiver to let kids go higher, what you want to do is let kids that are really smart (but didn't learn things properly) get past the hurdle. More than the ardent workers but not as smart. Since shmartie is more likely (if he gets unlazy) to survive a course he is unprepared for. But really, I just wouldn't waiver. Too hard to justify. Keep It Simple, Stupid.

New contributor
guest is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
  • $\begingroup$ This is a good suggestion, but it would be great if you could flesh it out a little bit. Do you have more specific information about where an interested person could find the test/data used by the USAF? $\endgroup$ – Nate Bade Jan 21 at 1:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.