Tuesday, January 23, 2018

A Comparative Look at TUSD's AzMERIT Scores

Posted By on Tue, Jan 23, 2018 at 10:26 AM

click to enlarge A Comparative Look at TUSD's AzMERIT Scores
Courtesy of PhotoSpin
Back in September, I promised to create a comparison of Tucson Unified scores on the 2017 state AzMERIT test with similar schools in other districts, because I was unhappy with comparisons between Tucson's district and neighboring districts with wildly different demographics. I spelled out how I planned to approach the comparison before I looked at any of the data, and I've stayed true to my basic design.

I compared elementary schools with similar racial and economic characteristics in Tucson, Sunnyside, Flowing Wells, Douglas, Nogales and Yuma. Since very few of the other districts had schools with fewer than 60 percent of their students on free or reduced lunch, I only compared schools with F/R lunch percentages of 60 percent or higher. All the districts other than Flowing Wells have a high percentage of Hispanic students.

I compared the districts' passing percentages with one another using all the schools I looked at. I also divided the schools into four groups based on the number of low income students, using the percent of students on free or reduced lunch as the measure—60-69 percent, 70-79 percent, 80-89 percent and 90-99 percent—and compared the districts' passing percentages within each of the four groups.

Here are the overall findings:
• When looking at the passing percentages of all the schools, Tucson, Douglas and Sunnyside have identical passing percentages in Language Arts. Douglas and Sunnyside have slightly higher passing percentages than Tucson in Math — by 3 and 5 percentage points. Yuma, Nogales and Flowing Wells have significantly higher passing percentages than the other three: 8-14 percent higher in Language Arts, about 10 percent higher in math.

• When looking at the schools in the four categories based on income levels, Tucson's passing percentage is significantly lower than the others in the 60-69 percent F/R lunch category. The gap between Tucson and the other districts decreases as the number of low income students increases. In the 90-99 percent category, Tucson's passing percentage is about average.

• Tucson schools have significantly more variation in their passing percentages than other districts, with schools among the lowest and highest in all four categories.
I also looked at the passing percentages for Hispanic students in the schools. The comparisons were close enough to what I found when I looked all the students that a separate analysis of Hispanic passing percentages doesn't yield significantly different results.

For me, the most surprising finding is the wide variation in passing percentages of Tucson schools with similar student populations—or I should say with populations that look similar based on income levels and percent of Hispanic students, since those are the only criteria I used.

In the 60-69 percent F/R lunch category, for example, Tucson schools' passing percentages in Language Arts ranged from 27 percent to 56 percent. In the 90-99 percent F/R lunch category, they ranged from 14 percent to 45 percent.

Tucson schools with the highest passing percentages compared well with top schools in other districts. Tucson had 2 schools among the top 5 passing percentages in the 70-79 percent category, and 3 schools in the top 5 in the 90-99 percent category, including the highest passing rate of any district in that category.

That's the basic summary of the results I pulled from the data. For whatever reason, Tucson Unified doesn't score well in the overall comparison of students passing the 2017 AzMERIT test, though it has some schools in each of the four income levels with high passing percentages, and its overall percentages get closer to the other districts as the number of low income students increases.

Does this mean Tucson schools are doing a poorer job educating their students than schools in other districts with similar demographics? Using this comparison, that's a reasonable conclusion, but high stakes test scores are a crude indicator of student achievement, so I can't say if that's the case with any degree of confidence. The tests only assess a narrow band of student achievement, and there are many ways to increase students scores without increasing educational quality, some legitimate, others less legitimate. And, for all the fact that the schools look similar in their broad characteristics, they may not be as similar as they appear when the specific student bodies are looked at more closely.

A No-Score-Shaming, No-Score-Faming Note: I purposely did not name any specific schools in this analysis. I don't find it valuable to use high stakes test scores to shame or praise a school. As I said above, a test score is a crude indicator of the strengths and weaknesses of a school for a number of reasons, so I won't hang a test score label on individual schools.

Tags: , , , , , , , ,