Back when I was a younger man and a big Kurt Vonnegut fan, his novel Sirens of Titan was required reading. (Everything of his was required reading.) The book’s main character wrote down a list of things he wanted to be sure to remember. I only recall one item: “If the questions don’t make sense, neither will the answers.” The statement hit me like a lightning bolt. It explained why, when I tried to answer some questions, I found myself tied up in knots. If a question was based on false assumptions and I answered it directly, I had to wrap my answer around the bad assumptions. Vonnegut’s insight helped me realize that a direct response to a ridiculous question has to begin with, “Your question doesn’t make sense.”
I wrote Vonnegut’s words down immediately. Decades later, they’re among the 30 Stickies residing on my Mac desktop.
Now I’m adapting Vonnegut’s statement to high stakes test results, which, since No Child Left Behind became the law of the land 16 years ago, have been the way we’ve answered the question, “Which schools are succeeding and which are failing?” The “answer” assumes the test results are a reliable way of determining school effectiveness. That’s nonsense. Which leads me to the conclusion, if the high stakes tests don’t make sense, neither will the results.
TUSD has decided to get rid of the AzMERIT tests at the high school level and replace them with a single ACT test given to 11th graders. I’m all for the change, for a few reasons. (1) It disrupts the A-F state grading system for schools. How do you compare the scores of high school students taking ACT, SAT and AzMERIT tests? It’s apples and oranges, or maybe Granny Smith and Red Delicious apples. (2) It takes the “standard” out of “standardized testing,” which weakens the whole high stakes testing movement. (3) As an unintended bonus, it takes a baby step toward democratizing the college admissions process.
Let’s review what’s wrong with AzMERIT and other high stakes standardized tests.
The Scores Are About Zip Codes
I’ve said this so many times, I’m tired of saying it. But it’s true, and it can’t be repeated too often. Standardized test scores have more to do with students’ socioeconomic status than with the quality of the schools they attend or the teachers who instruct them.
Former Superintendent of Public Instruction John Huppenthal, who I usually disagree with, made the point simply and eloquently when he railed against the state’s A-F school grading system which is based on AzMERIT scores.
“Here we have this letter grading system that comes in and is beating, to put it bluntly, beating the hell out of schools that are serving the most at-risk populations.”
Former TUSD and Flowing Wells superintendent John Pedicone criticized the state grading system for the same reason.
“If the assessment measures demographics rather than the students themselves, it’s inappropriate.”
If you look at analyses of student scores on international tests, they agree on one point. No matter where you go, children from high income families generally score better on the tests than children from low income families.
The best thing you might say about high stakes test like the AzMERIT is, the results are valid, but they’re being misused. Except that’s not true. The results aren’t valid. The stakes are too high. The reputations of schools and teachers rise and fall based on the test scores, so everyone has an incentive to improve their scores by any means necessary. So . . .
Whoever Teaches To the Test Best, Wins
While it’s true, big picture, the tests measure students’ socioeconomic status, they also measure how well teachers and schools teach to the test. The more time spent on test prep and the more adept teachers are at helping students game the test, the better the scores will be. The tests aren’t supposed to measure test taking skills, but they do. And since teaching to the test is so important . . .
The Tests Distort the Curriculum
The tests narrows the curriculum to what will be on the test. Students spend too much time learning how to give correct answers to clear-cut questions and too little time on larger ideas, ambiguities and open-ended discussions, none of which help students learn to give the hard-and-fast answers which earn an extra point or two on the test. And why waste precious class time on subjects other than Language Arts, Math and Science? They won’t be on the test.
It turns out, there are other ways to boost test scores, which means . . .
Cheaters Win
Cheating on high stakes test is rampant, all over Arizona, all over the country. Don’t let anyone tell you different. An occasional scandal will bring a case of cheating out in the open, but most times, schools get away with it. Cheaters prosper. They raise their state grades and reputations above schools with similar students that play by the rules.
Cheating can be divided into two basic categories: cheat sheet cheating and test tampering.
Cheat sheet cheating is anything teachers do to increase student scores when the test is being administered. It can take the form of helpful information on the board or on posters hanging around the room which students can refer to while they’re taking the test. It can be helpful little hints teachers drops to the class or to a student. Anything that leads to a few more right answers while students are taking their tests qualifies.
The real problem, though, is test tampering, which usually means Erase and Replace. After the tests are handed in, teachers or administrators erase some wrong answers and replace them with right answers. That’s cheating in its purest form. According to independent research which has analyzed the number of wrong-to-right changes on test questions, it’s not an uncommon occurrence. At some schools the statistical probability that students themselves made all the wrong-to-right changes is the same as the school being struck by lightning, yet their scores are left standing.
A Maybe-Raising-The-Stakes-On-The-ACT-Is-A-Good-Thing Note: How do I square this circle? First I say I dislike high stakes tests. Now I say increasing the stakes on the ACT test by substituting if for the AzMERIT may be a good thing. Here’s how.
The ACT and SAT are standardized college entrance exams. Students’ scores help determine where they go to college, or even if they decide to attend college. They’re already high stakes tests. The problem is, the students who get test prep tutoring to boost their scores are generally from families with money. Everyone else walks into the testing room with scant preparation and takes their chances.
When TUSD’s high schools make ACT their only high stakes test, they will devote the kind of attention to ACT they’ve lavished on AzMERIT. That means all students will get some serious test prep, including practice tests. They’ll all have the opportunity to learn how to increase their scores like the wealthier kids do. I may not be fond of high stakes tests, including the ACT and SAT, but I’m for anything that levels the socioeconomic/educational playing field, even a little bit. If some students score higher on the ACT because TUSD elevates its importance, that could be a life changer for some folks. I say that’s a good thing.
This article appears in Jun 28 – Jul 4, 2018.
Tucson Unified School District ditching AzMerit test for high schoolers
By Hank Stephenson Arizona Daily Star Jul 2, 2018
No Accountability in TUSD: Public Education Child Abuse!!!
“….Jacob Mishook, director of assessment and accountability at Achieve, a national nonpartisan nonprofit focused on college and career readiness:……If the goal of the accountability system is to have an apples-to-apples comparison of schools, having a menu of assessments is a serious threat to that, he said.
Thats a problem for parents trying to decide where to enroll their students, he noted….”
TUSD Students, in general, have consistently scored very low/failed AzMerit; a reflection of a FAILED Educational Program!!
We are a representative Democracy. Arizona/TUSD Parents/Voters are responsible for this Failed System of Public Education by NOT Demanding Change and Accountability. It is Tantamount to Public Education Child Abuse!!!…SO BE IT!!!!!
Interesting. Wish we could transition to letter grades based on quality of services offered. This would take into account measures like % of classrooms staffed with fully qualified teachers and would add points for # of years of experience, high rankings in peer reviews, masters degrees in education, # of articles / books published (yes, there are some master teachers locally who are working at this level, and 20 years ago, some of them were in TUSDs magnet schools).
The interesting thing, David, is that you would find some correlation between quality of services offered and test scores. Low SES kids in the magnet schools who had benefited from regular instruction, year after year, from highly qualified faculty would score higher than low SES kids in schools where TUSD was staffing classrooms with long term subs rather than qualified teachers. The impact of qualified and experienced teachers on achievement is backed up by research, and the constant drumbeat of political propaganda that links income and test scores obscures the fact that part of the problem reflected by the score distribution you mention is that poor SERVICES are often offered in poor schools.
So why dont you write an article about the fact that TUSD used its new $15 million RedforEd funding supplement in ways that have little prospect of recruiting more qualified teachers for the scores of classrooms in the district in low income neighborhoods that are currently covered by long term subs with no teaching credentials?
Seems like if SOCIAL JUSTICE were really the goal and not SOMETHING ELSE, there might be some coverage of that topic in local Democrat-affiliated media venues.
Wow! It’s a good thing for you that you have no concern whatsoever for the dislikes you receive on your posts Former Dem, you know, the dislikes which you brought up in the conversation you had with Rick Spanier because you don’t care about them. Because if you were concerned with that stuff, you might actually be devastated with your post above. Somehow you raked in 69 of them in less than 10 hours, that has to be a record! If it’s less than 10 hours, that’s definitely not someone using 7 devices daily to pile them up. I’d wager that it’s just several people who are tired of your long-winded anti Democratic and/or anti TUSD rants. Care to bet?
Anybody ever done any longitudinal outcomes study ? In other words how do students do in college and the work force after they leave high school ? If this argument is correct, then there would be a sizable number of students who do well on the test but poorly in college or trade school. Do kids routinely ace the math section ( usually not the target of social justice warriors) but , in reality can’t add or subtract ? It is a serious charge that test takers are cheating/phony prepping their way to success, but don’t really know much. Pretty easy to prove.
There are other reasons to support the change. The AZMERIT took literally weeks to administer (year after year), and the lost instruction time, especially if you add benchmark tests, was phenomenal. Yes, as one Board member stated, our curriculum was “aligned” to the AZMERIT but what kind of horrible statement on the creativity or lack of it in our curriculum is THAT?! The ACT takes far less time to administer, one reason why at least at Tucson High (3500 students) the site council just about stood up and cheered when the idea was proposed. The bottom line is that standardized testing of all types does a huge disservice to teaching as a profession and to student education. There is absolutely no “proof” of its educational validity: but Pearson is raking in the cash. Even as one assessment among many (which would be a great improvement) all it measures is a students ability to take tests which, despite another board members’ repeated phrase in past years (“in real life you have to take tests all the time”) is not the most important life skill to work on. I did ask two students about the switch, and they too thought it would probably be better although both agreed that it was harder than AZ Merit and that the fact that it is TIMED adds stress. In this case one student takes tests well and the other bombs them, although both students are straight A honor students. Unfortunately their test taking ability made a clear difference in their ability to get into out of state colleges–a pretty sad commentary on Arizona’s educational level (but then what do you expect when your 50th in spending?) They also noted the disparity among those who could take a class on passing the test and those who couldn’t, a subject I much appreciated in the article.
And finally, why such a huge outcry against the idea of regulating the huge disparity in services in our schools? This is a huge problem, and all it takes is spending a half day at Dodge or Doolen and then another half day at Utterback. Get serious people–this is not about liking each other–its about judging what would actually improve our schools. And making sure that they all get their fair share of experienced teachers and principals should not cause such an outcry, whatever political party you align with. It is a structural improvement that is long long overdue in TUSD.
“…..The bottom line is that standardized testing of all types does a huge disservice to teaching as a profession and to student education. …” INCREDIBLE!!!
ACT (America College Testing): “… an entrance exam used by most colleges and universities to make admissions decisions. It is a multiple-choice, pencil-and-paper test administered by ACT, Inc. ..”
Wake UP!!!…NOT ALL HIGH SCHOOL GRADUATES WILL GO ON TO A COLLEGE/UNIVERSITY!!
Student Performance on AzMERIT, as with any Standardized Assessment Examination, will determine if Students are being taught properly and/or effectively learning the required body of information per Subject Area so as to move on to the the next Academic Level in achieving their CAREER GOALS: what ever they may be!!!.
This is the sole purpose of Standardized Assessment Examinations. AzMERIT is such an Assessment Examination and will indicate the effectiveness of the Teachers/Schools/Districts Academic Programs, so that, if necessary, remedial action can be taken.
Educational Accountability in a School and Classroom? HEAVEN FORBID!!!!!
@ try having condensed posts.
Devastated? Guess again. Delighted is more like it. Part of my point here is to give whoever happens to read the comment streams on this blog a sense of the kind of arguments that get aggressed against and disparaged when youre interacting with the Support Public Ed / Excuse TUSD crowd.
To anyone who understands education, the loading up of dislikes on posts like the one I made above, like the systematic sign stealing that happened in the last TUSD Board election cycle, just makes it plain what were dealing with here, which is not a shining example of a democratically controlled district where values like fair play, civility, and prioritizing student best interest will be allowed to prevail.
Widespread standardized testing makes a sham out of education. As noted above, significant money is made off of test-prep materials and courses and concentrated into a few hands, feeding into lobbyist dollars at the state and national level. This comment comes from someone who was quite successful at tutoring for an ETS test (makers of PSAT and GRE among many others) and also who wrote test items for a state-wide mathematics test in Arizona.
Over the last decade while the US has focused on “back-to-basics” and “accountability” and standardized testing and fallen or stayed near the median in international results (and far behind the top performing countries in Asia and the social democracies of Europe) , Finland has been near the top consistently including being #1 in the world in recent years.
What’s the secret test regimen they’re using in Finland? ZERO high stakes standardized tests K-12 in Finland (they do have an exit exam for 12th graders).
Instead Finland does what is obvious to most all master teachers I have known during my 25 years in classrooms: emphasize professionalism in education. If you want to evaluate your students the last thing you would do is ask ETS to make a “normalized” test for you. Professionalism in education means that you commission educators to design meaningful and relevant curriculum that you then give criterion reference tests to measure how well the students are learning. This is not rocket science. Feedback from this kind of relevant testing is the sort of thing that guides teachers in every normal classroom.
Finland values and empowers it’s teachers: Only the top 10 percent of college graduates are allowed to pursue their masters in education to go onto become teachers. Finland also puts great effort into equality in education, for example, placing their most qualified teachers into there second language classrooms for immigrants. Readers above have noted that greater equality boosts nation-wide educational performance.
This article from the Smithsonian gives good details about the professionalism of the Finnish model:
https://www.smithsonianmag.com/innovation/…
I would like to know if any of you (besides Betts Putnam-Hidalgo) go to the TUSD School Board meetings or watch it on line. I am fascinated that so many of you voice opinions without knowing with what the school districts in our area are dealing.
I attended or watched TUSD Board meetings for several years. During the time period when I paid closest attention (2013-2017), the worst problem in TUSD, a problem from which many other troubling problems originated, was its chronically dysfunctional Board. I don’t watch as closely as I used to, but in recent days it has been clear that this Board approved a budget that “dealt with” an unanticipated $15 million infusion of funds by choosing NOT to use it to solve the most conspicuous problem with “professionalism in education” in the district: too many classrooms staffed with long term substitutes rather than fully qualified teachers.
In this context, a lot of questions arise about the current change to testing policy. For example: If not having standardized tests theoretically allows teachers latitude to use the Finnish-style ‘professionalism in education’ referenced by Josh Leonard, does TUSD currently have the qualified teachers and curricular materials in place in all of its classrooms, district-wide, to realize excellence in instruction for each year in grades 9-12? If not, what will that transition look like, as they move from year-by-year accountability to doing whatever the “teachers” they employ (including long term subs) think is best? Will it be great in some settings and a train wreck in others? If so, are they relying on principals to identify and manage each train wreck in the absence of accountability benchmarks each year? And is that feasible?
The Star article on the topic reported that “TUSD Superintendent Gabriel Trujillo warned that his concern is the district will start focusing on test preparation at just one grade level, rather than ensuring all high school students are succeeding. ‘I remember the days of AIMS, and the huge problem I always had with AIMS was it was all about 10th grade. So ninth, 11th and 12th all of a sudden didnt matter.'”
I’m not a fan of standardized testing. But pragmatically, running schools without it requires that a lot of other things (most importantly, highly qualified faculty) be firmly in place. That is not the current situation in TUSD, and that leaves me wondering whether now is the right time to make this kind of a shift in policy.
It’s great that you are delighted with all the dislikes that you receive, because you seem to receive an awful lot of them. Although I would be more convinced about that if I wasn’t so sure that you’re adding likes to your own commments. Either way, your explanation as to why the dislikes delight you is quite a stretch. Instead of using the name Former Dem or any number of silly names to amplify your mostly nonsensical rants, you use the name Stretch. It also describes the space your comments take up because they’re stretched out 2 to 3 times longer than necessary. Way to go Stretch!
Keep it up. As November and the various SUPPORT PUBLIC ED initiatives Dems have cooked up will be on the ballot, the public needs you to keep reinforcing just how much certain factions within the Democratic Party in Southern Arizona celebrate ignorance and aggress against anyone who mentions relevant facts they dont want the voters to be aware of.
Makes sense. The ACT is at least relevant – even though it is flawed.
As long as cheap labor industries continue to bring in a million plus legal and illegal immigrants every year and dump their kids into the taxpayer funded education system, expect low scores and high voter apathy.
Just how many classrooms in TUSD are currently staffed with long term subs with no teaching credentials who are being paid an hourly rate rather than salary + benefits? Last time I saw the data it was over 100, but perhaps theyve made great progress in this area since I stopped paying close attention to the districts affairs.
David Safier?
Betts Putnam-Hidalgo?
(Does anyone dispute that the data is relevant in a context where the district is removing year-by-year accountability benchmarks in its secondary programs? If so, make the argument. Id honestly like to hear it. Perhaps theres something Im missing here.)