How to Be Discerning About New Educational Products

In this article in American Educator, University of Virginia cognitive psychologist Daniel Willingham says, “The field of education is awash in conflicting goals, research ‘wars’, and profiteers.” How can we tell good classroom products and practices from flim-flam and outright junk? Most of us don’t have time to read and analyze all the data on textbooks, software packages, new teaching strategies, or school restructuring plans, so Willingham suggests a four-step shortcut:

• Strip it and flip it. The underlying content of a sales pitch is usually, “If I do X, then there is a Y percent chance that Z will happen.” Get to these basics by pruning all the emotional, fear- or hope-based language, claims that the persuader understands your struggles, appealing analogies, and ornamentation. Then flip the claim to see if it has an inverse – for example, “90 percent fat free” implies “10 percent fat.” Then ask yourself what is likely to happen if you don’t buy this product. Are you being pushed to buy it by fear of falling behind? Often, the distilled essence of a claim is straightforward and obvious – or quite vague. Willingham gives several examples of elaborate “brain-based” claims that boil down to very simple stuff. Here’s one: “There is massive brain plasticity during the early years of life. Brain plasticity is the process by which the physical structure of the brain changes, based on experience. New networks are formed, and unused networks are ‘pruned’ away – that is, are lost.” The basic message: “Little kids learn a lot.” 

• Trace it. Find out who the lead person is and look at his or her qualifications and motivations. 

• Analyze it. Look at why we are being asked to believe something. “If the claims about an education product fly in the face of what you know to be true, there is a problem,” says Willingham. Look at whether the research design included a control group.

• Decide. Adopt a program only when you have all the relevant information.

“Measured Approach of Magical Elixir? How to Tell Good Science from Bad” by Daniel Willingham in American Educator, Fall 2012 (Vol. 36, #3, p. 4-12, 40),

http://www.aft.org/pdfs/americaneducator/fall2012/Willingham.pdf 

From the Marshall Memo #456