[Matt Groshong]

3. Accuracy and Credibility

Most of the following is paraphrased or quoted from the best study I have read on placement assessment: "Assessing Developmental Assessment in Community Colleges" (CCRC Working Paper No. 19, Assessment of Evidence Series) Paper by: Katherine L. Hughes & Judith Scott-Clayton - 02/2011.
URL: http://ccrc.tc.columbia.edu/Publication.asp?uid=856

A brief of the Hughes study is available at http://ccrc.tc.columbia.edu/Publication.asp?uid=857

Importance of context
Placement assessments exist as part of a larger context and are not an end in themselves. Here is the conundrum:

"An analogy can be made to a clinical trial in which individuals’ medical history is assessed in order to help estimate their ability to benefit from a certain treatment. If the individuals selected for the treatment do not benefit, it could be because the treatment is universally ineffective, because the initial assessment inadequately predicts who is likely to benefit, or because the assessment does not provide enough information to accurately target variations of the treatment to different people. Similarly, if developmental education does not improve outcomes, is it because the "treatment" is broken per se or because the wrong students are being assigned to it? Or is some different or additional treatment required?" (Hughes p.2)

In spite of this larger context, we must strive to improve our placement testing processes.

Accuracy in placement assessment
Accuracy is maximized when an assessment predicts the initial developmental English and math classes in which a student has the highest probability of success. The placement process minimizes placement of students into unneeded developmental classes (false negatives), minimizes placement of students into classes that are too difficult for them (false positives), maximizes placement of poorly-prepared students into developmental classes (true negatives), and maximizes placement of qualified students into college-level courses (true positives). For each instrument used to determine placement, there is solid evidence of validity, including significant correlations between the test scores and class performance or higher course success rates using the instrument.

Multiple measures:
It is important to emphasize that there are countless internal and external factors that influence a student's success or lack of success in a course: the college's ability to provide at-risk students the support they need, the "grit and tenacity" of the student, the amount of hours they are working, the variability of course grading between instructors, etc., etc., etc. For that reason many researchers are suggesting multiple measures for course placement: e.g., COMPASS scores plus high school or college GPA, grades in last high school math course, and surveys of affective characteristics such the LASSI (Learning and Study Stragegies Inventory) with scales for attitude, motivation, time management, anxiety and concentration among others.

Are our current assessments accurate and credible?
I strongly recommend you read the entire Hughes study for an in-depth analysis of this question, but if nothing else the chart on page 14 and the analysis that follows. This is not an issue that leads itself to simplistic answers: it is quite complex as you will see from Hughes.