Thank goodness the Wall Street Journal isn’t known for its outstanding health reporting.
In a story written by Rachel Emma Silverman, she reports on some preliminary research recently presented at a management conference. Like a lot of research that gives us “surprising” results, it was done on a single group of 96 undergraduate students at a single college campus.
And the task designed for the college laboratory setting by the researchers would be difficult to characterize as analogous to most people’s work environment or jobs — it was highlighting every single letter “e” or, in the second part, “a,” while reading.
The question the researchers asked — Can surfing the Internet help you to become a more productive employee?
The answer, according to the researchers, is an overwhelming, “Yes!” And it’s no wonder… Let’s look at how the researchers designed the study:
In the student experiment, 96 participants were randomly assigned to one of three groups — a control group, a rest-break group, and a browsing-the-Internet group. All subjects were first assigned to spend 20 minutes highlighting as many letter e’s as they could find in a text of 3,500 words. At the conclusion of this exercise, subjects spent 10 minutes in one of three ways: the control group was assigned a filler task that involved bundling sticks into groups of fives; the rest-break group was free to do anything they wanted except to use the Internet (their activities included visiting the washroom, making phone calls, and text-messaging friends); and the third group was allowed 10 minutes to browse pre-selected Web sites including those offering news, social networking, online gaming, entertainment, and hobby-related activities.
Finally, all participants were instructed to spend 10 minutes highlighting as many letter a’s as they could find in 2,000 words of text, this final assignment serving as a proxy for productivity.[…]
Chen and Lim report that participants in the Internet-browsing group were significantly more productive than those in the other two groups, highlighting a mean of 316 letter a’s, compared to 272 for the rest-break group and 227 for the control group; in other words, the Internet browsers were 16% more productive than the rest-break group and 39% more productive than the control group.
Wow, 16 percent more productive than the break group. Given that finding, the researchers could’ve also headlined the study saying, “Taking a break from work makes you more productive.” Because the “control group” wasn’t really a control group — they continued doing more work on their “break,” making it a certainty that since they actually worked through their break, they would likely do worse at a future task.
Whether 16 percent is a significant difference or not is hard to say, because as I mentioned, the laboratory task was in no way representative of a typical work environment or job. It was an arbitrary task designed by the researchers to be repetitive, boring, and without any purpose or meaning. (Nobody who took part in that study thought for a moment that the reason they were highlighting letters in some random text had any particular meaning.)
Compare the laboratory task to an actual work environment, where people often are working toward specific time-limited goals that help finish a project or forward their own careers.
One of the problems with this kind of research is that nobody slaps a researcher on the wrists for designing a poorly-conceived study. In fact, it gets presented at a professional conference, and then picked up by a financial newspaper’s website and reported on as a legitimate health and human behavior finding. Going through only minimal peer review (if any at all) when presented at a professional conference.
I suspect the researchers knew their first finding wasn’t all that robust, so they followed it up with a retrospective survey of close to 600 alumni of a business school, of which 191 replied. Not surprising, the researchers again found that tasks related to work — such as emailing — were related to negative emotional states, while non-work related tasks — such as web surfing — made people happier.
Hmm. Isn’t that like asking whether people felt happier while standing around and chatting at the water cooler back in the 1960s, as opposed to typing on their typewriter? Who would expect anyone to say they feel “sad” or “unhappy” while goofing off at work? What kind of “research” is this?!
Social Media’s Multiplying Effect of Bad Research
In addition, the Wall Street Journal’s reporting on the study failed to point out any of the study’s limitations. Yet it remains one of the most cited articles on Google and elsewhere for this particular study.
This behavior highlights one of the problems with social media and algorithms based upon popularity today — what is popular may not actually be what is right. A quick writeup of a study published on a large media website is going to be popular, despite the fact that it is simply regurgitating the news release, which itself is a regurgitation of the glossed-over findings forwarded by the researchers themselves. There’s no check or balance here.
And few people who will read the Wall Street Journal version of the story will ever know that the study being reported on is one step up from a steaming pile of dung. Because the Wall Street Journal doesn’t care — it’s not a health news organization. And after all, it’s just reporting on what it was told. Who says it needs to know the difference between good and bad research?
The upshot is that we don’t know if web surfing is really any better than just taking a normal break from your work to help you be more productive. A lot of research suggests that breaks throughout the workday are simply good in and of themselves, regardless of whether they employ surfing online or not. This research doesn’t really help shed light on this topic, and instead dumbs down the complexity of the issue.
Read the full article: Web Surfing Helps at Work, Study Says
Read the news release: Internet browsing at work? It’s a pause that refreshes workers and enhances their productivity, new research finds
1 comment
Thank you!I am increasingly annoyed with and alarmed by the things that are being passed on via electronic media, such as studies like this (so few people seem to understand the idea of proper scientific studies), as well as select portions of studies and reports, or dated information.Too few people bother to check on original sources, validity and credibility, and simply pass it on without question.We need to re-embrace the concept of healthy skepticism and independant thinking!
Comments are closed.