Common sense vs. statistical significance

In an article for the Brookings Institution, Thomas Kane questions the application of classical hypothesis-testing, which requires that a difference have no more than a 5 percent chance of being a fluke to be accepted as statistically significant, to education data. Under this model, the difference between a 20 percent likelihood and a 75 percent likelihood is insufficiently strong, though in practical, common-sense terms, significant. Improper use of the paradigm in education leads to costly mistakes, in Kane's view. Take teacher tenure: with classical hypothesis-testing, one starts from the premise that an incumbent teacher is average, and only denies tenure when that presumption is shown wrong beyond a reasonable doubt. A different approach would recognize that tenure decisions implicitly involve a choice between two teachers, the incumbent and an anonymous novice. This latter approach compares two teachers' predicted effectiveness, and chooses the one with the better-predicted outcome. Using data from three districts, Kane and a colleague calculated tenure results using both models, and found that under the first model, just one percent of teachers would be refused tenure; under the second, 25 percent. Education has many examples of decisions involving symmetric costs, Kane writes, and in such cases, decision-makers should choose the option with the better odds of success, whether or not it's "statistically significant." More

Source:  Public Education News Blast

Published by LEAP

Los Angeles Education Partnership (LAEP) is an education support organization that works as a collaborative partner in high-poverty communities.