Time for a test on student testing
“Will you test my spaghetti sauce?” I ask anyone who happens to pass through the kitchen. Typically the person dips in a spoon, tastes, looks puzzled for a moment, and then says it’s got too much of this or too little of that. I then add whatever miscellaneous ingredients seem promising.
Testing students in schools and universities is almost nothing like that.
A spoonful of sauce taken from a pot gives you a pretty good idea of how a ladleful will taste on a plate of spaghetti. But a quiz, exam, or other test of a student’s knowledge – even a take-home essay assignment – provides only a tiny glimpse into what the student actually knows and understands.
As social studies of science and technology have shown, all tests presuppose that the test is similar to something else “out there” in the world. Technicians determine that a car at a crash-test facility is similar to a car on the freeway packed with kids, luggage, and the family dog. But identifying two things as similar requires setting aside the many ways they are also different: we may notice that two people have a similar skin color, which may obscure their many differences. To what extent is a student’s performance on a test similar to all the facts, concepts, skills, and long-term capacities and inspirations that educators hope their students will take away from the classroom?
Student tests mostly show how well students take tests. Beyond that different kinds of tests vary enormously in how well they stimulate and assess student learning. A quiz may compel my students to get the reading done, but that doesn’t mean it tells me very much about what they’ve learned, and it may induce them to focus on isolated facts rather than genuine understanding.
Educational researchers have repeatedly confirmed what many teachers say, and what I’ve seen first-hand as my kids have gone through school: over-reliance on standardized tests corrupts the curriculum and destroys intellectual curiosity. And when it comes to evaluating teachers, standardized tests easily produce misleading results. It’s important to assess student learning and teacher effectiveness, but we should do it in a smart way that actually fosters improvement.
Unfortunately, increased reliance on standardized tests has long been a nationwide trend. The New York Times reports that 30 states now link teacher evaluations to student test scores, and at least 13 states use student test scores to account for 50 percent of a teacher’s evaluation. And at universities, increasing class size pushes faculty to rely more on standardized tests.
A key issue in the massive Chicago teacher’s strike, now entering its second week, is the role of standardized tests in both teaching students and evaluating teachers. Among other things, Chicago Mayor Rahm Emanuel wants the percentage of teacher evaluations based on student tests to rise from 25 percent to 40 percent over the next five years.
Emanuel sends his own kids to a lavish private school, the University of Chicago Lab School, whose director has criticized efforts to measure learning outcomes with standardized tests.
Most importantly, the debate over standardized tests is not just about pedagogy, but the purpose of education itself. As Henry A. Giroux argues, excessive use of standardized tests turns teachers into deskilled technicians and fosters “curricular models that devalue critical thought and reduce imaginative inquiry to the teaching of marketable skills.” Education becomes vocational training and loses its capacity to reduce social inequality and enrich democracy.
Tests increasingly permeate our lives. Genetic tests, intelligence tests, personality tests, relationship tests – none of them gives a complete picture of what it claims to measure.
Maybe it’s time to start testing politicians on what they know about tests. It might not tell us how well they understand the issue, but it could be an enlightening experience for everyone. Or better yet: educators could start including a few questions on standardized tests on every standardized test. That could promote the sort of critical thinking that standardized tests so often undermine.
Whose University? 
Middle-class parents will often go to great lengths to make certain that their children have the very best teachers. Parents of low income often lack the efficacy to be advocates for their own children. No one wants their child to be stuck for a whole year with a bad teacher.
The critics of standardized testing often have nothing of their own to put in the place of these tests in order to hold teachers and schools accountable. It is much easier to criticize than to suggest workable alternatives.
I have read that Finnish education is the best performer of all the nations. Finland holds teachers in high esteem and pays them accordingly. In addition, unlike what is the case in the United States, only the best and the brightest are permitted to teach in Finnish schools. Exactly how we would go from the current situation in America to Finland is unclear. There is nowhere the political will and the financial support to do this.
I think that the current federal approach which holds entire schools accountable is to be preferred to a system which focuses on the value-added of individual teachers. Holding schools accountable gives principals and faculty an incentive to provide help to the low performing teachers or, when necessary, get rid of them. Nonetheless, the federal system also relies on standardized tests and it is not clear what other alternatives would be as effective to help ensure accountability..
There are certain courses where standardized tests may do a good job of evaluating the quality of education. Mathematics certainly comes to mind here. On the other hand, one could easily believe that standardized tests might provide very inadequate measures in English classes. I do not use standardized tests in my government class either.
One important result of the use of standardized testing is that decision-making in education become centralized. It is easy for some administrator who may know nothing of teaching or its challenges to make decisions on the basis of these tests, We might refer to this as the production model of teaching. A more holistic model of evaluation relying on context and expertise might be viewed as a custom or craft model of evaluation. However, if the custom model is used as a pretext so as not to improve the quality of education, it is understandable that critics would argue for the production model and that their arguments would win the day.
Thanks for the comment. I don’t think most critics of standardized testing are using that as a pretext to avoid addressing quality issues. Improving educational quality is exactly what they are trying to do. And they have plenty of alternative proposals. Some initiatives are discussed in today’s New York Times editorial: http://www.nytimes.com/2012/09/17/opinion/in-search-of-excellent-teaching.html
More on that in a future post.
“There are certain courses where standardized tests may do a good job of evaluating the quality of education. Mathematics certainly comes to mind here.”
Mathematics often gets trotted out as an example here, but it is actually a terrible one. Mathematics education has been badly hurt by this misconception. What standardized math tests teach you to do are to answer rote calculation assignments (I won’t call them “problems” because they rarely engage actual problem-solving abilities). Sometimes these can be quite complicated, and at the upper reaches they even dip into creative problem-solving abilities, but usually they just require relatively straightforward application of different sets of rules that must be memorized. Ask anyone who has had a rude awakening in transitioning from general ed math courses to upper-level courses for math majors, or from undergraduate to graduate work in math — the students often find that what made them “good at math” is useless here, and what has been left out is what matters.
You might say “so what,” as most of us are not going to become mathematicians, and instead just need to be able to do basic figures, but the research in situated cognition and math pedagogy shows that schoolhouse math skills don’t even transfer over well to applying math in real situations. Far from a paradigm example of testable skills, the tradition of mathematics education shows the opposite.
Very well written! I would like to say a few things about the standardized testing experience as I understand it, in particular throughout my educational career.
Beware, I have a brief anecdote: I attended a tiny rural high school (read: 300 people in a public school), and took quite a few AP courses there. AP courses, as I’m sure most everyone knows, are supposed to be as rigorous as college courses, that is why you get an inflated GPA, and why if you pass the exam at the end of the course, you receive multiple courses worth–the AP English exam is a particular bargain–of college credit. There were so few students taking the exam, and so many in the course, that the people who were taking it were sent, two weeks prior to the exam, to the library to study every day. Where, I can say with pride, I played some of the finest games of chess in my life, but never studied for the exam. I showed up on test day, broke open my packet and left with a 5 (out of 5).
While I might like this to reflect my natural brilliance, I think it demonstrates that there are certain subjects where standardized testing is effectively useless. Anyone who has a clue, can figure out what the Populist movement stood for if you give them 4 choices. They’ll forget it just as fast. Language arts, an academic sausage if there ever was one, has all the aesthetic, stylistic and content dimensions of literature ground out of it, and instead is a healthy blend of usage, punctuation and spelling. Every GRE study book under the sun has page upon page for the math-illiterate student. Most all of them talk about guessing numbers in between the given answers and using the calculator to check, using your scratch paper to determine scale and you know, everything except learning or understanding MATH. I understand the GRE is a special case, and they are testing reasoning and logic as well as basic skills, but I would guess the same is true of SAT books (another chapter in Ben’s academic apathy leading to success).
The problem with the big ‘uns: the SATs, STAR, GREs, Pre-SATs or the California’s High School Exit Exam is that they conceptualize what we, as a society, have determined as adequate knowledge for a person to have attained. Following Hannah Arendt, I believe that the worst thing about that sort of conceptualization is not that it is wrong, but that it could become true. If these tests indicate the depth and variety of knowledge we expect people to possess, the rest being irrelevant (or “fluff” as my high-school math teacher would say) for the solid 12 or so years you are being trained to pass them, then we shouldn’t be surprised when there are a lot of people who “know” all sorts of facts and figures without understanding anything. If we are serious about considering the sorts of things we test on as knowledge, and the failure to instill students with them a serious academic crime, we should also not be surprised if this “knowledge” becomes so much intellectual wedding-silver: Used once, maybe shown off a few times but with no larger value.
I write all this fully cognizant of the fact, that without some form of standardized testing, coming from a tiny rural high-school, my application would have been laughed out the door by every UC I applied for. But then again, every program, be it undergraduate or graduate, swears up and down that they consider “the whole applicant” as opposed to any set of standardized test scores. I also understand, that people do fail these tests and they carry with them a great deal of importance. But rather than relying on or emphasizing these tests because they determine who gets the money, or who is deficient or they are the only measure we have; couldn’t we first agree that what they test ought to be grounded in what a given subject is actually about? Rather than playing around with some mash-up of proto-knowledge, isn’t there another that doesn’t, as you mention, kill intellectual curiosity and doesn’t establish such a massive gulf between learning and being tested.