Spurious Surveys

Statistical surveys mean squat without significance

This Christmas, I received a book from my parents that I'm really excited about: Statistics Done Wrong by Alex Reinhart. Through the introduction and first few chapters, the message is clear: We're really bad at using statistics.

The book takes aim at medical and scientific communities misinterpreting their data and the sensationalism that (much to my embarrassment) news media often attaches to insignificant results.

However, these subtle snafus aren't limited to doctors and scientists. No, in fact, the Arizona Criminal Justice Commission is prone to the same misinterpretations.

In early December, the ACJC released a biannual Arizona Youth Survey State Report with numbers on cannabis use by teens in grades eight through 12. ACJC Director Andrew Lefevre concluded that fewer kids are using prescription drugs or meth, turning to cannabis instead.

"More kids are abusing marijuana and the alternatives that come along with it," he told KTAR News, 92.3 FM. "We are seeing a dangerous growing trend in the number of students smoking marijuana and using marijuana concentrates."

Certainly, a cursory glance at the data would indicate such a conclusion. Cannabis use in 2018 is up 3.8 percent among high school seniors since 2016, 4.8 percent among juniors and 2.4 percent among eighth graders. And though they don't say it, these are statistically significant increases.

Lefevre's claim about concentrates is a bit dubious, as this is the first time teens have been surveyed about cannabis concentrates, so there's no way to indicate an increase.

Of course, any percent of teens using cannabis that isn't close to zero is worrying. Only 190 patients in Arizona are under the age of 18.

But the part of the study Lefevre's interpretation seems to ignore is that there was a statistically significant decrease in cannabis use among teens between 2014 and 2016. How does that fit into your analysis?

Sure, reported teen cannabis use is up in the past two years, but why did it fall the years before that? The problem here is that, like most cannabis data, there's not enough information to draw a conclusion.

If Arizona's medical cannabis program were the culprit, we'd expect to see a steady increase as the number of patients and proximity to dispensaries grow. But in three surveys, we've seen teen use go down, then up. And that's about all we can say.

Perhaps the more worrying part of the survey is that a quarter of teens report obtaining cannabis from a patient and 10 percent say they bought it from a dispensary themselves.

While respondents often have incentive to be dishonest in surveys, these numbers deserve a raised eyebrow even if they're ballpark estimates. How are 4,870 teens buying cannabis from dispensaries?

Moe Asnani, owner of Downtown Dispensary, said delivery services are to blame, whether they're taking fake IDs, or not checking IDs at all.

"Some of them will tell you if you have an out-of-state certification, they'll deliver to you," he said. "Others have little to no compliance at all. They're basically black market."

Delivery services could explain the increase in teen use in recent years as well, if their prevalence in Arizona has grown since 2016.

Asnani mentioned some legislation may be coming down the line in the next legislative session. But until then, as is often said about the Arizona cannabis market, it's still the wild west out here.