Improving a specific skill is not hard. Or at least knowing what to do (practice) is not hard, even if actually doing it is not so easy. But improving at very general skills, the sort of skills that underlie many tasks we take on, has proven much more difficult. The grail among these general skills is working memory, as it's thought to be a crucial component of (if not nearly synonymous with) fluid intelligence. Brain training programs that promise wide-ranging cognitive improvements usually offer tasks that promise to exercise working memory, and so increase its capacity and or efficiency.

Claims of scientific support (e.g., here) have been controversial (see here), and part of the problem is that many of the studies, even ones claiming "gold standard" methodologies have not been conducted in the ideal way. This controversy usually arises after the fact; researchers claim that brain training works, and critics point out flaws in the study design.

A new study has examined more directly the possibility that brain training gains are due to placebo effects, and it indicates that's likely. 

Cyrus Foroughi and his colleagues at George Mason university set out to test the possibility that knowing you are in a study that purportedly improves intelligence will impact your performance on the relevant tests. The independent variable in their study was method of recruitment via an advertising flyer: either you knew you were signing up for brain training or you didn't. 

The flier at left might attract a different sort of participant than the one on the right. Or people may not differ except that some have been led to expect a different outcome of the experiment. 

All participants went through the same experimental procedure. They took two standard fluid intelligence tests. Then they participated in one hour of working memory training, the oft-used N-back task. The final outcome measures--the fluid intelligence tests--took place the following day. 

Even advocates of brain training would agree that a single hour of practice is not enough to produce any measurable effect. Yet subjects who thought brain training would make them smarter improved. Control subjects did not. 

It's well known that scores on IQ tests are sensitive to incentives--people do a little better if they are paid, for example. People in the placebo group  might try harder on the second IQ test because they  know how the experiment is "supposed" to come out and they unconsciously try to comply. This belief that training might either have been planted by the flier OR the flier might have been a screening device, luring people who believed brain training works, but not attracting people who didn't believe in brain training to the experiment. 

Most published experiments of brain training had not reported whether subjects were recruited in a way that made the purpose plain. Foroughi and his colleagues contacted the researchers behind 19 published studies that were included in a meta-analysis and found that in 17 of these subjects knew the purpose of the study. 

It should be noted that this new experiment does not show conclusively that brain training cannot work. It shows placebo effects appear to be very easy to obtain in this type of research. I dare say it's even a more dramatic problem than critics had appreciated, and more than ever, the onus is on advocates of brain training to show their methods work.