|
|
|
Putting "FairTest" to the Test
Date: Wed, 15 Dec 2004 Subject: How about putting "FairTest" to the Test? Below is a letter to the editor that I submitted to the Boston Globe earlier this week regarding an article they ran this past Friday December 10. The original article is here. -- Dave Ziffer Dear Boston Globe Editor: I was disturbed to read an editorial by Derrick Z. Jackson in your 12/10/2004 issue ("College Board is put to test"), in which he criticizes the College Board (maker of the SAT college entrance exam) while extolling the virtues of a relatively little-known organization named "FairTest." Mr. Jackson tells us that, "FairTest, the National Center for Fair and Open Testing, has long said that standardized tests are overused to assess student potential." He provides no further explanation of who FairTest might be, as if to imply that an organization that calls itself the "National Center for Something" must necessarily be precisely that. With his utterly one-sided support of FairTest's statement, presented entirely without supporting data showing that standardized tests are somehow inappropriate or overused, he seems to further imply that whatever FairTest says must be true. (Yes he presents data that testing costs us about $2 billion per year, but strangely he neglects to mention that this pales in comparison to the $350 billion we spend on public K-12 education as a whole.) Mr. Jackson's article goes on to tell about how the College Board has objected to FairTest's use of College Board SAT statistics, which when analyzed along the lines of race and income appear to demonstrate a significant performance gap among America's children, in which children of nonwhite and low-income families are the losers. But rather than blame the schools, parents and/or students who are obviously responsible for this disparity, Mr. Jackson apparently wants to shoot the messenger, namely the SAT and the College Board. Mr. Jackson assets that, "the College Board went after FairTest because the hopelessness of testing kids with no resources and the ridiculousness of assessing a human life through a test score has become undeniable." Huh? No resources? Does the $15,000 that the failed New York City public school system spends on every child constitute "no resources?" Does our threefold increase in public school spending (in real, inflation-adjusted dollars) since 1960, with a negative result to show for it, constitute "no resources?" And who said that the SAT constitutes the "assessing of a human life?" (I thought the SAT was just trying to predict the likelihood of college success.) The College Board was probably mistaken in objecting to FairTest's use of published SAT data, but I can understand its defensiveness, because the College Board has been under systematic attack from the public school education establishment for decades. The objective of the SAT is quite simple: to provide colleges with an practical comparative assessment of the likelihood as to whether any given student is likely to succeed in a college career. And the SAT accomplishes this mission quite nicely: it is widely regarded as the single best predictor of such success among many of the colleges that use it, better even than high school GPAs (i.e. grades) [1]. Which is why the education establishment is so desperate to torpedo the SAT, the College Board, and every other useful assessment tool. The SAT makes painfully obvious a few things that the ed establishment would rather not have you know. It would rather not have you know that it fails to serve low-income and nonwhite populations well, despite the frequently inordinate amount of money it spends on such populations (for example Washington DC and New York City spend as much per pupil as many very wealthy suburban districts, yet produce dismal results). The ed establishment wouldn't want you dwelling on the fact that we've tripled our spending on public education during the past 45 years [2] while proven longitudinal measures of academic success show either no significant improvement [3] or an actual decline in performance during those same years [4]. And there's the problem. It's the one thing the ed establishment can't afford to have widely known, namely that its eternal and adamant promise of better students in return for more taxpayer dollars is an entirely false one [5]. The ed establishment can't afford to have us looking at things like the SAT. It would much rather have us rely on information sources like FairTest and your columnist, who will go on eternally telling us that we will have better students if only we will dig deeper into our pockets.. So what exactly is FairTest's purpose and who exactly is FairTest? I don't know, and I'll bet you don't either. I can only guess. I do know a few things about FairTest, though. It seems to be very well funded with grants from foundations that favor the expansion of government into areas in which it has clearly failed miserably (urban education, for example) [6]. FairTest seems to have an enormous PR budget. Based on my experience in private industry, it must almost certainly spend several hundred thousand dollars per year to get the enormous amount of press coverage it receives. FairTest spends the vast majority of its time complaining about practical, standardized, statistically analyzable results (as it demonstrates so aptly through its criticism of the SAT [7]). And FairTest seems to be one of the primary driving forces behind its one and only proposed solution, i.e. the one supposedly "fair test" that FairTest offers - namely "authentic assessment." [8] What exactly is this miracle alternative to standardized testing? Like most of the concepts being pushed by the ed establishment, it is a general idea or philosophy rather than a specific, written, replicable instrument. It is a vague notion of evaluating students using methods that would defy the evaluator to make convenient comparisons among the assessed students. Authentic assessments might consist of teachers' written assessments, portfolios containing samples of student work, and possibly even students' self-assessments. But then again, they might not. The more one looks for a solid foundation beneath the concept, the more it becomes clear that there is very little, if anything, in the way of a substantive statement of exactly what constitutes an authentic assessment. As an example of this I refer you to one of FairTest's own descriptions [9], written FairTest's own executive director, one Mr. Monty Neill. Like many such documents this description of authentic assessment consists firstly and mostly of criticisms of anything not labeled "authentic," followed by a long list of statements of what an authentic assessment, if we could ever actually see one, might be like. The list of desirable criteria for "authenticity" includes attributes such as, "assessment is interwoven with curriculum and instruction," and "assessments must be able to indicate individual development as a thinker and doer," and "assessments must be 'theory referenced'," and "appropriately rich and well-developed." But alas, neither Mr. Neill, nor most of his many references apparently, is ready to actually show us an assessment that meets these criteria. Looking for solid examples of authentic assessment materials is like looking for clipart on the Web: almost all you ever see is sites that list other sites containing lists of other sites ... If this sounds unbelievable to you, then I invite you to spend a few hours doing a Google on "authentic assessment" and seeing how far you get in coming up with anything other than relatively vague ideas about how you might construct one. Authentic assessment is an intentionally vague and subjective concept - not a specific test or even a rigorous methodology - whereby teachers simply apply their own evaluations in whatever manner they see fit. Since one cannot conduct specific research upon non-specific subjects, there is to my knowledge no credible research supporting its use, nor could there ever be. Which makes it useless as a practical tool for large-scale evaluations of large populations. While the objectives of authentic assessment sound lofty (like so many other vague, subjective educational practices), I think the true objective is to make student, teacher, school, and district comparisons impossible. Because there is no escaping the fact that you can't compare two things that haven't been measured in a comparable fashion. Imagine for a moment that you are on the staff of a large state university preparing to admit ten thousand freshmen from among twenty thousand applicants. And in order to evaluate these twenty thousand students you are being asked to look through twenty thousand "authentic assessments" - twenty thousand informal written statements composed by thousands of teachers, each in his or her own style, along with twenty thousand portfolios of student work. And in the end you will have no way of comparing them, since they intentionally have different structure, style, and content. Preposterous. So how does the ed establishment go about making preposterous ideas politically correct and sensible ideas politically incorrect? By using condemnation. With its multi-hundred-billion dollar budget, the ed establishment has plenty of money to spend on both internal and external propaganda. So when it decides to target an organization like the College Board for elimination, it simply launches accusations of racism and elitism. Which works beautifully, of course. Since the public school system is the most racist and elitist organization on the face of the earth (just look at the results it produces), anyone reporting on its results will necessarily demonstrate evidence of those characteristics. And then all the ed establishment has to do is transfer its own guilt to the party doing the reporting. Shoot the messenger. And that, I believe, is why the College Board reacted so unreasonably to FairTest's legitimate use of its data. Because even though there is nothing morally or ethically wrong with the SAT data, the College Board knows that there is most definitely something wrong with FairTest's objective, which is to depict the College Board as an elitist, racist organization that brutalizes children and should consequently be eliminated. If I were the messenger and someone were actively shooting me, I'd get a little testy too. Anyone with a little imagination can see precisely where the public education establishment is taking all of us. Through its total control of our public schools it is eradicating all convenient summary information that could be used to compare one student's performance against another's (i.e. letter grades). Through its union contracts it has already long established a nationwide policy whereby teachers can be neither rewarded nor punished on the basis of performance, and it is working steadily to eradicate all objective measures whereby teachers, administrators, schools, and districts can be compared. And through its mouthpieces like FairTest, along with sympathizers like your columnist, it is working to eliminate all practical mechanisms that might shed any usable light on public school performance, especially those that demonstrate the incredible lack of correlation between public school funding and school performance across districts and states, and those like the SAT that show the lack of correlation between increased public school funding and school performance over time. When the ed establishment's mission is finally accomplished, parents will have no idea what their kids have learned or not learned; taxpayers will be completely incapable of locating any objective data on teachers, administrators, schools, and districts, or the effects of pumping ever more funds into those districts; and colleges will have absolutely no practical, standard measure by which to determine the relative merits of incoming students. In short, we will all be totally ignorant. What could be fairer than that? Sincerely, Dave Ziffer DaveZiffer@ProjectPro.com REFERENCES |
