Barriers to Accurate Assessment
Teachers evaluate student performance for many reasons: to determine the effectiveness of curriculum materials and methods; to compare achievement levels within and across schools, school systems, districts, and states; and to evaluate students' knowledge and skill. This last type of information is indispensable. Externally, it is used as a criterion for school and college admission. Internally, it helps teachers evaluate the effectiveness of our instruction, so that we can make adjustments necessary to keep students on track.
To be truly useful, assessments must evaluate the knowledge and skills relevant to students' goals—and they must do so accurately. This is more difficult to achieve than it seems, especially when the same test is administered to the entire class. Although using the same assessment tools and procedures for all learners might seem to be a fair and equal approach, in reality, this approach yields inaccurate results for many students. Any test that relies on a single medium inevitably, albeit unintentionally, evaluates talents that may not be relevant to instructional goals-talents that are bound up in the medium or methods being used. Thus, students' ability or inability to work with particular media and methods may confound evaluation of their knowledge and skills.
To understand confounding factors in measurement, consider a friendly neighborhood butcher, Al, and his scale. Ms. Smith, one of Al's favorite customers, stops by the butcher shop to purchase some lamb chops. Al neatly trims the fat, places her four chops on his precisely calibrated scale, records the weight, and rings up the price. Fifteen minutes later, another customer appears. Al rolls his eyes. It's Mr. Nyles, who always complains and never appreciates Al's fine meats. Al uses the same accurate scale to weigh the chicken Mr. Nyles has selected, but he "forgets" to remove the chicken from its plastic container and "accidentally" rests his thumb on the scale.
This example illustrates how extraneous factors—a plastic container and a thumb—can corrupt a measurement. Although no one intentionally builds inaccuracies into academic assessments (as Al did with his scale), these inaccuracies do occur. The precision and accuracy of an assessment tool is reliable only to the extent that extraneous factors are removed from the equation. In our view, the traditional model of academic assessment is flawed in four important ways:
- Student characteristics—individual learning differences—can confound results.
- Media characteristics can confound results.
- Withholding student supports can confound results.
- Poor integration with curriculum limits the value of assessment data.
Let's examine each of these factors, paying close attention to how the three brain networks and their interactions with different kinds of media can help us understand the barriers to and solutions for more accurate and valuable assessment.
