There is now some serious soul-searching in the mainstream media about their (previously) breath-taking coverage of the Big Data revolution. I am collecting some useful links here for those interested in learning more. Here's my Harvard Business Review article in which I discussed the Science paper disclosing that Google Flu Trends, that key exhibit of the Big Data lobby, has systematically over-estimated flu activity for 100 out of the last…
The article (link) in Science about the failure of Google Flu Trends is important for many reasons. One is the inexplicable silence in the Big Data community about this little big problem: it's not as if this is breaking news -- it was known as early as 2009 that Flu Trends completely missed the swine flu pandemic (link), underestimating it by 50%, and then in 2013, Nature reported that Flu…
Last week, Andrew Gelman (link) and I were kindred spirits: we both did a "numbersensing" exercise on two different data analyses. I was reading the MailChimp study on the effect of Google siphoning off "marketing" emails into a separate tab, and a noise buzzed my head when I saw that the aggregate click-to-open ratio was reported at an inconceivable 85%. (See Part 1 of my reaction here.) In the meantime,…
One of the dangers of "Big Data" is the temptation to get lost in the details. You become so absorbed in the peeling of the onion that you don't realize your tear glands have dried up. Hans Rosling linked to...
Business Insider (link) highlighted a map showing childhood food insecurity across the 50 states, with the data coming from a report by Brookings. This is a nice map. I like the tones of the chosen colors although the colors are...
USA Today, who brought us a dubious early signaling system for sudden cardiac arrest in a prior post, also gave us “Story Time!” on aspirin on the same day (link). The article started with the conclusion: “Taking an aspirin before bed may reduce the chance of having a heart attack or stroke in the morning”. What is the data used to support this cause—effect statement? *** The researchers looked at…
A recent article in USA Today is titled “Many with sudden cardiac arrest had early signs” (link). The signs include shortness of breath, faintness, chest pain, etc. Hold on to the headline because it’s the only thing believable in the entire article. The words “early signs” imply to readers that were the men to heed these warnings, they could have prevented the cardiac arrests. Think about the following two statements:…
I'm very excited to learn that the FDA has ordered 23andme to stop selling "personal genetic testing" to unsuspecting consumers. This is the company founded by the ex of Google's Sergei Brin (recent news here). So for a change, the FDA is doing something right for consumers, without waiting 10 or 15 years. Here is a report on this action in San Jose Mercury (link). These tests have never been…
I'm hoping to speak at a location near you in the near future but in the meantime, here are two prior occasions you may have missed. The first is the LISA conference. LISA is Leaders in Software and Art. You'd get a flavor of this fascinating group by viewing this 7-minute video (link). I gave a 5-minute lightning talk, which starts at 4:18 on the video. All the lightning talks…
Here we go again. Another useless study published in a peer-reviewed journal (Mayo Clinic Proceedings) with a relatively high impact factor and promoted as "Breaking News from the Editor" to the press who then attached a sensational headline and reported it as "science". This is what caught my eye: "Drinking more than 28 cups of coffee a week may be harmful for people younger than 55, according to a study."…
