A red flag always goes up in my mind when I see an entire industry sprout up around something that doesn’t have a lot of research backing. That’s been the problem with these so-called “brain games” — you know, those video or online games that supposedly improve your memory or thinking.
Makers of these games like to point to studies of people who were older (usually seniors), had existing memory or cognitive problems or other issues who engaged in specific tasks and then were shown to have some improvement in memory or cognition. Few studies have been done on these games with everyday folks like you and I. And fewer still studies have been done on the specific games being marketed to consumers (often the studies use tasks that the games then try and replicate, but we don’t know if they’re replicating all the important bits or not). And fewer still studies have been done and published in peer-reviewed professional journals (not just on people’s websites).
So when the prestigious journal Nature publishes a study about real brain games used on a large sample size, this is going to be a piece of research that gets our attention.
The study recruited 8,600 participants ages 16 to 60 to play online brain games designed to improve people’s memory skills, reasoning and other cognitive abilities. They were compared to a group of 2,700 people who did not play these “brain games,” but instead spent a similar amount of time on the Internet and answering general knowledge questions.
The primary measure of change was an IQ test given to all participants before and after the experiment.
The researchers found that there were no significant differences between the two groups at the end of the experiment.
Now, granted, there are some obvious limitations to this experiment. The brain games were made up by the researchers — not existing games put to the test. We have no idea whether commercially available games would fare better or not.
So how did commercial producers of these “brain games” respond to the new study? Well, denial of course!
One maker of brain games said the BBC study did not apply to its products. Steve Aldrich, CEO of Posit Science, said the company’s games, some of which were funded in part by the U.S. National Institutes of Health, have been proven to boost brain power. […]
Posit Science has published research in journals including the Proceedings of the National Academy of Sciences showing their games improved memory in older people.
You can look at all of the training references yourself for Posit Science’s programs. Not a single one of them suggests the skills measured in these studies generalize to everyday life after the training is completed:
“There is precious little evidence to suggest the skills used in these games transfer to the real world,” said Art Kramer, a professor of psychology and neuroscience at the University of Illinois. He was not linked to the study and has no ties to any companies that make brain training games.
Nowhere on Posit Science’s marketing page for its “Brain Fitness Program” does it mention that every claim it’s making is based on a very specific population of older individuals. Or that the tasks it says it improves will help you with real-life tasks in memory or cognition. There’s nothing wrong with what science does exist that’s behind these games. But developers like Posit Science aren’t telling you that if you’re 21 and looking to improve your memory scores for help with studying for an exam, there’s little evidence this stuff is going to do anything for you.
So what’s a mind that wants to improve to do? Easy. Exercise remains the simplest thing you can do to improve your brain.
He said people should consider learning a new language or sport if they really wanted to improve their brain power. “To stimulate the intellect, you need a real challenge,” said [Philip Adey, an emeritus professor of psychology and neuroscience at King’s College in London], adding computer games were not an easy shortcut. “Getting smart is hard work.”
Other types of activities that challenge your brain with novelty include crossword puzzles and Sudoku.
Food for thought next time you consider buying one of those “brain training” games or videos. A crossword puzzle or a walk around your neighborhood may be just as effective (and far cheaper!).
Read the full article: Study: Brain games don’t make you smarter
30 comments
This sounds like BS to me. If you play a game and get better at it, then you are, by definition, learning. It’s a matter of whether you’re learning skills that are applicable to the real world. The skills you learn from Mario probably aren’t much help in any other context, but if you’re playing a math flashcard game, they certainly are.
“The brain games were made up by the researchers — not existing games put to the test.”
The real games worked too well, so they had to invent their own crappy games that didn’t, in order to prove their preconceived point?
“To stimulate the intellect, you need a real challengeâ€
Obviously never played a real video game in his life. Why do you think the difficulty increases with each level?
This article is really irresponsible. The author takes all brain games to task for not being proven, even though there ARE studies on many of them, then tells people to do sudoku and crosswords. Where are the published studies on sudoku and crosswords? I sure couldn’t find any… strikes me as hypocritical, unscientific, and misleading, if you ask me.
I think its important that we distinguish these so called “brain games” that have limited scientific validity from [computerized] cognitive remediation that does have scientific evidence to support its efficacy.
Computerized cognitive remediation to address cognitive deficits associated with TBI and schizophrenia has been around for a while now, and the research is promising and positive. Cognitive remediation is now also being investigated to help improve cognitive deficits associated with substance use as well as side effects to chemotherapy.
The programs used in the past for cognitive remediation were somewhat boring after a while and research is being conducted to look at more mainstream programs (ie video games) to assess their efficacy.
The difference, though, in the research cited in the article above and that I am referring to is that the positive findings reflect populations suffering from known cognitive difficulties, not your average person, such as those recruited in the study reference in the above article.
If someone has no major cognitive impairment, how much would we really see them improve anyway? A few clinically insignificant points on some scale?
In my opinion, the study Dr. Grohol’s article references is one of those “duh” studies.
The danger such research poses, though, is that people may misinterpret this study and its limitations and apply it to all forms of brain training and assume it has no benefit.
However, cognitive remediation in its true therapeutic form has established itself as a valid treatment for cognitive deficits associated with some disorders and disabilities.
Brain games don’t work. Fine. Most people suspected this.
“The danger such research poses, though, is that people may misinterpret this study and its limitations and apply it to all forms of brain training and assume it has no benefit.”
Exactly!
We know cognitive stimulation with medication can help more than medication alone.
We know cognitive stimulation alone is effective.
So, in people with cognitive decline, formal techniques of brain stimulation have been evidenced to help, in clinical practice. Which has face validity, being consistent with clinical outcomes seen.
Babies an’ bathwater – don’t throw out cognitive stimulation and non-pharmacological strategies!
The outcome of the BBC study was entirely predictable for a number of reasons:
1. The games were just that – games. As we understand it, they were not designed to develop cognitive skills in any systematic and integrated way. It is possible to train cognitive skills with a program that is specifically designed for that purpose.
2. 10 minutes a day, 3 times a week is not enough time to rebuild neural pathways and to improve skills. Most serious cognitive skills training programs recommend at least 30-60 minutes 3 to 5 times a week.
3. The six-week duration of the study was also insufficient time. In studies with BrainWare Safari (www.MyBrainWare.com), parents and teachers start to observe behavioral changes that are connected with cognitive functions after around 7 to 8 weeks of use. Substantial improvement in cognitive functioning, as measured by the Woodcock-Johnson Cognitive Battery, is observed after 10-12 weeks of use. “A Study of the Effectiveness of Cognitive Skill Therapy Delivered in a Video-Game Format†by Helms, D. and Sawtelle, S.M., Optometry & Vision Development, Volume 38, Number 1, 2007.
Scientists commenting on the study seemed to dismiss brain training that is fun and indicated that a more challenging program might have had an effect. These characteristics are not mutually exclusive. An effective brain-training program can be both fun and very challenging.
BrainWare Safari is both fun and challenging and has been shown to improve cognitive functioning by an average of over 4 years when used for 12 weeks. While the program is delivered in an engaging video-game format, it is serious cognitive training.
The most effective cognitive development programs incorporate the following principles:
1. Progressive challenge.
2. Comprehensive integration of cognitive skills related to learning.
3. Visual, auditory and other sensory integration.
4. Cognitive loading to drive skills to automaticity.
5. Frequency and intensity.
6. Engagement and motivation (fun).
I agree with the above commenters that the interpretation of this study is astonishingly over-reaching. The BBC researchers designed their own cognitive stimulation program, applied it at a very low intensity in healthy young people, and then saw no effect on cognitive function. To then claim that their data shows that “computerized mental workouts don’t boost mental skills” is akin to saying “sugar doesn’t help with a headache, and sugar and aspirin are both molecules, so aspirin must not help with headaches either.” This is an elementary logical fallacy.
The only conclusion from the BBC study is that very limited amounts of everyday cognitive stimulation does not improve cognitive function. This is an interesting conclusion, and the study should have reported it as such.
Henry Mahncke
http://www.positscience.com
I am a researcher at Posit Science, where I design and test cognitive training programs.
I am a scientist at Posit Science, where I work on the design and testing of these programs. At Posit Science, we are very careful to always connect the statements we make describing the benefits of our programs to published clinical trial data. Given that care, I would like to comment on Dr. Grohol’s description of the clinical evidence behind Posit Science’s programs:
1) Dr. Grohol states that “Not a single [study] suggests the skills measured in these studies generalize to everyday life after the training is completed.”
This is not correct. Multiple studies document the generalization from training to everyday life skills, including
http://www.ncbi.nlm.nih.gov/pubmed/12169801 – transfer to timed instrumental activities of daily living
http://www.ncbi.nlm.nih.gov/pubmed/14529195 – transfer to on-road driving skills
http://www.ncbi.nlm.nih.gov/pubmed/16960242 – transfer to health-related quality of life
http://www.ncbi.nlm.nih.gov/pubmed/19181719 – transfer to depressive symptoms
2) Dr. Grohol also states that “developers like Posit Science aren’t telling you that if you’re 21 and looking to improve your memory scores for help with studying for an exam, there’s little evidence this stuff is going to do anything for you.”
It’s not possible to enumerate every conceivable use a person might have for a cognitive training program and on a web page describe all of the ones which have not yet been studied in a clinical trial. On our website and materials, we do not claim benefits on exam performance for 21-year-olds, because we don’t have data on that topic. I would appreciate a reference from Dr. Grohol around where he believes Posit made any such claim.
I concur with Dr. Grohol that physical exercise is like an excellent way to improve both physical and mental health. However, I believe Dr. Grohol goes beyond the published literature in his endorsement of crossword puzzles and sudoku. While interesting correlational studies have associated cognitively stimulating activities with reduced dementia risk, there have to date been no randomized controlled trials that have established that playing crossword puzzles or sudoku actually improve cognitive function. Dr. Grohol should ensure that he meets the goals he states for others in connecting his statements to the published clinical literature.
Henry Mahncke
Posit Science
I’d like to re-emphasize Betsy’s comments, particularly regarding the time frame in which brain training takes place.
The studies done on cognitive remediation look at training that occurs over the course of many, many hours. Some research suggests a minimum of 20 hours, with sessions last at least 1 hour in length, not 10 minutes.
The study referenced in the article had folks playing games for around 3 hours – no where near the minimum suggested by other studies. Other research into cognitive remediation and brain training looks at around a 100 hours of training.
Additionally, an IQ test is completely insufficient as a primary measure of outcome. While an IQ test could be one measure utilized, there should have been multiple measures of cognitive functioning utilized, preferably a neuropsychological battery.
It should be observed that they didn’t actually use an IQ test in the traditional sense, rather some assessment items DEVELOPED BY THE AUTHORS OF THE STUDY. It does not appear that they used commonly accepted, standardized assessment instruments, rather their own assessment tool that they claim was validated by the organization they work for and who sponsored the study. Mmmmm….
The study conducted is a poor excuse for an empirical research project (participants selected from those who watch a particular show? Really?). Its unfortunate that Nature choose to publish such a methodologically flawed study.
You’re certainly entitled to your opinion Dr. Grohol; but you do owe it to the public that you have checked your facts and the data. While there are many Brain Games that are purely entertainment at best; as someone who has worked on & off with Cognitive Rehab. patients for years, I have to point you to the meta-analyses that, in fact, support the use of Posit Science especially. There appears to be quite significant evidence of its benefits for normal and brain-impaired patients. I wish I could say as much for the formal Cog. Rehab. software, virtually all of which (it is a limited pool) is profoundly more expensive, significantly less sophisticated and rarely interesting enough to maintain attention, regardless of who the user or patient may be.
Hi Henry – Thanks for posting a response. Let’s start with #2 first. That was one example of a customer scenario for demonstrative purposes; it was not meant as a recommendation for your marketing materials. Your marketing materials make no mention of the limitations of the current research — primarily, that it’s done with seniors. Instead they are marketed to allow a person to “Take charge of your brain’s future: The Brain Fitness Program can reverse these trends, improving the quality and quantity of the information your brain absorbs from your ears.” Quite a claim, and not one that’s directly supported by the research (because it is so broad as to be immeasurable).
As for the research citations, they do indeed suggest that in very specific situations, some skills may indeed transfer to real life situations. So I stand corrected.
All the studies cited (except #2) were done on elderly participants. Nothing wrong with that, except that if you’re going to market brain tests to everyone — not just seniors — you should note that research done on one group may not generalize to another. They all also involved direct researcher interaction with training often done one-on-one or in groups. This is a far cry from sitting at home with a computer program.
Let’s look at Study #1 for instance. The researchers employed 16 different measures of performance in this experiment, hoping that the “brain training” group would improve in most of them. Out of those 16 measures, only 3 were found to be significantly different between the two groups. Both groups improved on one of their primary measures — UFOV — but the training group improved significantly more. There was also a difference in the Controlled Oral Word Association. And the researchers claimed a statistical significance victory for the Timed IADL test. IADL — “instrumental activities of daily living” — refers to a laboratory test that examines the speed at which people can perform a set of defined tasks, such as looking up a telephone number in a phone book. The difference between the two groups in performing these tasks? The training group finished it 0.10 seconds faster (in what can be an 11 minute task). Wow.
What the researchers didn’t find is just as significant. No improvement in the Road Sign Test, which is also an attempt to replicate a real-world activity in the lab.
Most researchers who failed to find an effect for 13 out of 16 measures would probably be a little ashamed. But instead, these researchers claimed that their research showed that brain training helps improve time completing the IADL lab test. And it does. By 0.10 seconds.
Study #2 “cherry-picked” its participants and had other methodological flaws in how the researchers designed the study as to call into question its results.
Study #3 showed that memory and reasoning “brain training” have no effect in health-related quality of life — only speed of processing tasks matter. Again, a pretty big finding given how many brain training program emphasize the importance of memory and reasoning tasks in their programs. They also used a definition of “clinically significant change” that, although they justified, may be a bit questionable. Keep in mind that all groups in this study suffered from clinically significant decline (26 percent in the control group for instance); it’s just that there were fewer who suffered such decline (19.5 percent) in those who did speed-processing tasks. (If you did memory intervention brain training tasks, 28 percent of participants suffered from extensive decline — more than the control group!)
You get the picture… Each of these studies are indeed interesting and demonstrate that under very specific conditions, the speed of processing tasks may help. Maybe. If you’re elderly. And it draws me something back to what Henry said in an earlier comment —
“I agree with the above commenters that the interpretation of this study is astonishingly over-reaching.”
Indeed, that is my exact reaction with the claims made by virtually all of the “brain training” developers. That their claims are over-reaching and not specific enough to what the actual data show. Is a 0.10 second difference in a laboratory test truly translate into a real-world difference (or is it simply a statistical significance)?
Let developers be more specific in their marketing claims, and then my concerns about these kinds of training programs will go away.
Aside from cognitive remediation (which is an evidenced-based treatment for neuropsychological deficits), the only other activity demonstrated through good empirical research is exercise, as pointed out by Dr. Grohol.
As was mentioned by Henry, though, is that crossword puzzles, Sudoku, etc have limited research to back them up and most studies are correlational.
However, physical exercise has been demonstrated to have a positive impact on brain functioning, in addition to the physical benefit.
I submitted a post yesterday that for some reason never made it into this conversation so I want to point some things out about this so-called brain study:
1) The authors did not really use an “IQ” test as a primary measure. That was reported by media outlets since they do not know any better. The authors used a variety of tools to assess different cognitive abilities. They did not use one of the well-known or commonly accepted intellectual assessment instruments, such as the Wechsler or Stanford-Binet.
2) The “benchmarking tests” used were adapted from cognitive assessment tools DEVELOPED BY SOME OF THE AUTHORS OF THE STUDY and validated by the institution sponsoring the study. The adapted versions were never validated by peer-review. Additionally, the instruments used could hardly be considered adequate measures of cognitive functioning, and in fact, similar versions are used as actual “brain training programs” and not outcome measures. The outcome measures should have consisted of standardized and normed assessment instruments.
Nature did a great disservice to many disciplines and fields of research by publishing such a highly flawed project (I refuse to call it a study any longer). It was irresponsible of Nature to not review it first (I can only assume they didn’t review it since there are more holes in this study than Swiss cheese).
Dr Wayne – Good points all. But it makes you wonder — if Nature has no problem in publishing such flawed studies, what’s the point of peer-review?
I would also point out that very, very few of the “brain training” commercial programs — the actual version available to consumers — have been validated in the research. What they often point to are components of their programs that have been used in research. But the entire training program that most developers promote on their own sites have rarely undergone the peer-review process.
We don’t know whether the two are the same, and I suspect, they are not.
Thank you Dr. Grohol for your post, and thanks to everyone who contributed to this discussion of so-called “brain games.” As a woman with ADHD, I’ve been looking for non-pharmaceutical ways to address some of my challenges (memory and focus being two of them) and I too have become alarmed by the marketing of some of the new software, games, etc., that are being sold, including gadgets to create a baby Einstein before the kid is out of diapers. I question our goals (are happiness and balance in life more important than IQ and memory, for example), and I’m wary of a simple, quick fix in anything. I’m glad you pointed to the age-old standby of physical exercise as a tried and true method of not just brain, but overall health improvement. Thanks again for a lively and enlightening discussion.
Dr. Grohol, I agree with you. I “thought” playing online solitaire would help me with money. Well, I was in a hospital snack bar the other day and the cost of vending machine food differs. My mind was getting confused cuz I couldn’t add up from one item to another what I wanted. Adding numbers in my head was easier when I was in my forties. I’m considering carrying a small calculator with me—gads, it’s not fun anymore to grow older! The more you forget, the more apparatices(sp) you think you need to carry with you at all times. Ha!
The article title is “Brain Games Don’t Help Your Brain,” yet the article says there is not enough research suggesting that the games help your brain. If there is not enough research, then isn’t it true that we have no solid evidence either way…that brain games either do or do not help?
I think it merely raises the question…and like so many other aspects of life, if a person believes it helps them, it probably will to some extent (in my opinion). But concerning research…I’d like to see it!
where’s your reasearch BUb
Dr. Grohol said:
“…if Nature has no problem in publishing such flawed studies, what’s the point of peer-review?”
Nature is certainly not perfect and in the thousands of papers that do pass through its peer review editorial system each year, many are deeply flawed and can have huge holes poked in them. Many others are excellent and stand up well to scrutiny.
Another point I’d like to make, is that it is very rare for ONE single paper to thoroughly prove or disprove anything. Even solid, rigorous papers can often have limitations and ‘holes’ to be poked if someone is looking for them. That doesn’t mean their data and/or conclusions are invalid- just that further research is necessary to add to the evidence and clarify our understanding. So just because this paper does have some limitations and is not the final word on ‘brain games’, it is a valuable piece of evidence as the information trickles in on the value or lack of it in these games.
The point of peer review is that it’s certainly not perfect, but it’s the best system we have to keep scientific papers relevant, transparent, and rigorous. We should always be striving to make peer review stronger and keep the bar raised high to make sure published papers stand up well to scrutiny and transmit worthwhile findings to the scientific community and lay people.
Dr. Grohol’s article does a good job of calling some of the extraordinary claims made by these companies peddling their own brain games into question. That doesn’t mean the companies’ games have NO value, just not quite as much as they seem to claim. The profit motive often gets in the way of the ‘truth motive’ and people need to think critically before spending money on any of these products by looking at the evidence.
Of course you should take any advertising with a grain of salt, but it’s silly to declare the entire industry to have no value. True, not every program will work for every person, just like some students respond well to one teaching style, while it goes right over another student’s head. Each of us thinks in a unique way, so each of us has to find things that help us individually.
Anything that does not have the backing provided by proven research should indeed raise the proverbial red flag. This is why we sit with so many problems, and medical recalls and resultant lawsuits in todays time – no researched or conclusive findings.
Sorry I have a question for the doctors : on a site of so called brain enancing games they advertise that a new science called neuroplasticy offfers material proves that plaiyng their games new neurons or new synapsys would create … if this is true the all problem of the effectiveness in enancing brain power would be resolved …but my be I’m missing something and I thanks in advance who will be so kind of enlighten my brain on this point
Guess I’ll have to disagree with this guy. Even reading a book has been proven to keep Alzheimer’s at bay, so I’m having a tough time believing games dedicated to engaging our brains doesn’t provide some sort of benefit.
Games may not make one smarter but they might help the brain maintain “elasticity” as well as keep the brain active via eye/hand coordination, memorizing patterns; spellings, colors- timing- etc.
Games are good for the human brain because play time is good for all animals
Comments are closed.