Back when I was a younger man and a big Kurt Vonnegut fan, his novel Sirens of Titan was required reading. (Everything of his was required reading.) The book's main character wrote down a list of things he wanted to be sure to remember. I only recall one item: "If the questions don't make sense, neither will the answers." The statement hit me like a lightning bolt. It explained why, when I tried to answer some questions, I found myself tied up in knots. If a question was based on false assumptions and I answered it directly, I had to wrap my answer around the bad assumptions. Vonnegut's insight helped me realize that a direct response to a ridiculous question has to begin with, "Your question doesn't make sense."
I wrote Vonnegut's words down immediately. Decades later, they're among the 30 Stickies residing on my Mac desktop.
Now I'm adapting Vonnegut's statement to high stakes test results, which, since No Child Left Behind became the law of the land 16 years ago, have been the way we've answered the question, "Which schools are succeeding and which are failing?" The "answer" assumes the test results are a reliable way of determining school effectiveness. That's nonsense. Which leads me to the conclusion, if the high stakes tests don't make sense, neither will the results.
TUSD has decided to get rid of the AzMERIT tests at the high school level and replace them with a single ACT test given to 11th graders. I'm all for the change, for a few reasons. (1) It disrupts the A-F state grading system for schools. How do you compare the scores of high school students taking ACT, SAT and AzMERIT tests? It's apples and oranges, or maybe Granny Smith and Red Delicious apples. (2) It takes the "standard" out of "standardized testing," which weakens the whole high stakes testing movement. (3) As an unintended bonus, it takes a baby step toward democratizing the college admissions process.
Let's review what's wrong with AzMERIT and other high stakes standardized tests.
The Scores Are About Zip Codes
I've said this so many times, I'm tired of saying it. But it's true, and it can't be repeated too often. Standardized test scores have more to do with students' socioeconomic status than with the quality of the schools they attend or the teachers who instruct them.
Former Superintendent of Public Instruction John Huppenthal, who I usually disagree with, made the point simply and eloquently when he railed against the state's A-F school grading system which is based on AzMERIT scores.
"Here we have this letter grading system that comes in and is beating, to put it bluntly, beating the hell out of schools that are serving the most at-risk populations."Former TUSD and Flowing Wells superintendent John Pedicone criticized the state grading system for the same reason.
"If the assessment measures demographics rather than the students themselves, it's inappropriate."If you look at analyses of student scores on international tests, they agree on one point. No matter where you go, children from high income families generally score better on the tests than children from low income families.
The best thing you might say about high stakes test like the AzMERIT is, the results are valid, but they're being misused. Except that's not true. The results aren't valid. The stakes are too high. The reputations of schools and teachers rise and fall based on the test scores, so everyone has an incentive to improve their scores by any means necessary. So . . .
Whoever Teaches To the Test Best, Wins
While it's true, big picture, the tests measure students' socioeconomic status, they also measure how well teachers and schools teach to the test. The more time spent on test prep and the more adept teachers are at helping students game the test, the better the scores will be. The tests aren't supposed to measure test taking skills, but they do. And since teaching to the test is so important . . .
The Tests Distort the Curriculum
The tests narrows the curriculum to what will be on the test. Students spend too much time learning how to give correct answers to clear-cut questions and too little time on larger ideas, ambiguities and open-ended discussions, none of which help students learn to give the hard-and-fast answers which earn an extra point or two on the test. And why waste precious class time on subjects other than Language Arts, Math and Science? They won't be on the test.
It turns out, there are other ways to boost test scores, which means . . .
Cheaters Win
Cheating on high stakes test is rampant, all over Arizona, all over the country. Don't let anyone tell you different. An occasional scandal will bring a case of cheating out in the open, but most times, schools get away with it. Cheaters prosper. They raise their state grades and reputations above schools with similar students that play by the rules.
Cheating can be divided into two basic categories: cheat sheet cheating and test tampering.
Cheat sheet cheating is anything teachers do to increase student scores when the test is being administered. It can take the form of helpful information on the board or on posters hanging around the room which students can refer to while they're taking the test. It can be helpful little hints teachers drops to the class or to a student. Anything that leads to a few more right answers while students are taking their tests qualifies.
The real problem, though, is test tampering, which usually means Erase and Replace. After the tests are handed in, teachers or administrators erase some wrong answers and replace them with right answers. That's cheating in its purest form. According to independent research which has analyzed the number of wrong-to-right changes on test questions, it's not an uncommon occurrence. At some schools the statistical probability that students themselves made all the wrong-to-right changes is the same as the school being struck by lightning, yet their scores are left standing.
A Maybe-Raising-The-Stakes-On-The-ACT-Is-A-Good-Thing Note: How do I square this circle? First I say I dislike high stakes tests. Now I say increasing the stakes on the ACT test by substituting if for the AzMERIT may be a good thing. Here's how.
The ACT and SAT are standardized college entrance exams. Students' scores help determine where they go to college, or even if they decide to attend college. They're already high stakes tests. The problem is, the students who get test prep tutoring to boost their scores are generally from families with money. Everyone else walks into the testing room with scant preparation and takes their chances.
When TUSD's high schools make ACT their only high stakes test, they will devote the kind of attention to ACT they've lavished on AzMERIT. That means all students will get some serious test prep, including practice tests. They'll all have the opportunity to learn how to increase their scores like the wealthier kids do. I may not be fond of high stakes tests, including the ACT and SAT, but I'm for anything that levels the socioeconomic/educational playing field, even a little bit. If some students score higher on the ACT because TUSD elevates its importance, that could be a life changer for some folks. I say that's a good thing.
