A cautionary tale about the limitations of big data
Posted 03.26.2013 at 10:00 am
10 Comments
Google's Flu Trends, mapped
Katie Peek
Last year, we wrote about Flu Trends, Google’s search engine–based influenza barometer. The takeaway: after calibrating its results against the numbers from the Centers for Disease Control--which are based on emergency-room visits--Google did a pretty good job of predicting a flu outbreak, and did so quickly, without having to wait for all those hospital reports to reach the CDC and be compiled into a weekly report.
But how did Google’s algorithm fare during this year’s fierce outbreak? Flu reached epidemic levels in January and--pertinent for the Flu Trends algorithm--was widely covered in the news. Would the widespread news coverage cause healthy people to enter influenza keywords into their search bars, thereby skewing Google’s results? We updated our graphic from last year to find out. Indeed, Google’s methods seem to wildly overstate the outbreak’s severity, outstripping the CDC’s figures by nearly a factor of two. It’s worth noting that we’re assuming sick people visited the hospital at the same rate as in previous years--in other words, that the CDC isn’t suddenly under-reporting influenza.
The result provides a cautionary tale for big data: if the data set doesn’t cleanly map the underlying terrain--that is, if people search for “flu symptoms” because they’ve heard flu is going around, not only because they have it--that data won’t yield reliable conclusions.
Neither Google nor the CDC could be reached for comment.
10 Comments
To comment, please Login.
Today on PopSci.com
Copyright © 2012 Popular Science
A Bonnier Corporation Company. All rights reserved. Reproduction in whole or in part without permission is prohibited.
I think you POPSCI may in fact be overlooking that Google may be more accurate indeed. You have no way to compare the data. Google wouldn't, in fact, be based on Hospital visits. It would be based on the total of people who probably had symptoms. People with Google Search don't necessarily have insurance and may not go to the hospital. Lol. You have no way to know do you POPSCI!
It remains to be seen if Google is in fact wrong.
"Do not try and bend the spoon. That's impossible. Instead... only try to realize the truth. There is no spoon."
@D13
Well, all these people apparently had access to hospitals in all previous years, because Google data and CDC data have aligned for nearly a decade. So you're suggesting that this year, unlike any other year before it, 50% fewer people went to the hospital for their flus? That's quite an anomaly.
Excellent point D13, it may be accessibility to the web via the growth of smart phones etc, shows a more accurate count of the flu, and in fact.... ALL prior years may have been under reported to a certain extent. I for one almost never go to the doctor for something like the flu. It's just not worth it to me, money or time wise. The flu will pass, and since it's a virus there really isn't that much they can do, they don't normally prescribe antivirals for the flu. The only thing I need to do is take care of my nasal passages, and I already know how to do that without a doctor. In fact there may be another factor - with more smart phone proliferation and web access then ever, and peoples increased reliance on it more people may be realizing that it doesn't make sense to go to the doctor for a flu. They can diagnose and look for help online. Your temp goes to high? take some ibuprofen. to stuffy? take so guafenesin(sp). Interested in lowering the symptoms and shortening the flu? make some osha root tea. It's very simple really, and cheaper, possibly more effective, and more convenient.
@suggestivesimon I would agree, I don't think 50% less people went, I think googles reporting got better. More people then ever on smart phones, more twitter and facebook and google + posts about having the sniffles, and better datamining then ever. I wouldn't be surprised if google showed high next year as well.
@nicholasjh1
Except that smartphone web browsing (and twitter and facebook) have been growing gradually for a few years now. They didn't explode in popularity in 2012. Which means that if this truly was the cause of the difference between CDC and Google data, we would've seen, starting in about 2009, a gradually growing difference between the two data sets... but such is not the case.
The good news is that the predicted genetic bomb has yet to surface
The fact is, it's all a lie.
Google conspired with the drug companies to publish false predictions to get more people to get treated with the petrochemical waste products they try to palm off as “vaccines”.
It may be that pro vaccination enemies of humanity didn't do as convincing a job, or did an unconvincing job. Certainly, worries about autism may have had an influence, and the pro vaccination tactic of saying, “I won't provide you any proof vaccines don't cause autism! You just believe what I tell you to believe! Because we have power, money, influence, position, popularity!”, may not have had the kind of convincing effect the pro vaccination forces hoped it would have with those they seem to see as “beasts of burden”.
Julian, show your proof.
A positive feedback loop is the likely culprit.
Google starts tracking flu searches -- uses data to predict outbreaks.
Media outlets (mainstream news, science news, and social media) report on Google's experiment with "big data" -- in particular the application to predicting flu outbreaks.
Google search data starts to show signs of a flu outbreak.
The signs of an outbreak get widely reported in the media.
People read the news reports -- search on Google for more information about the impending flu outbreak.
Google reports increasingly upward trends in searches on "flu".
Google data gets reported in media. Talk of an "epidemic" increases. This drives more and more people to search on "flu" ... and so on.