I was trained as a mathematician and it was only last year, when I attended the Royal Statistical Society conference and met many statisticians that I understood how different the two groups are. In mathematics you often start with some axioms, things...
Medical researchers are somehow allowed to get away with statistical murder. It upsets me to read the article in Forbes titled "Pet Owners May Have Lower Risk For Heart Disease." (link) This article takes the form of many other similar articles that purport to find an association between some risk factor and a common disease. Note they always use the weasel word "may". If you see this word, and immediately…
On the Junk Charts blog (link), I discussed some charts from the NYT graphics team (@nytgraphics) for a feature on the NFL draft. In the second part of the behind-the-scenes blog post, they discussed how they visualized work by some economists. This is how the research was summarized: "across all players and positions, teams only picked a player better than the person who went next at that position 52 percent…
Mammograms continue to be an emotional and controversial topic. I blogged about it some time ago. (link) Felix Salmon, whose blog should be daily reading, praises an article by Peggy Orenstein called "Our Feel Good War on Breast Cancer", NYT Magazine (link). Salmon's blog provides a quick summary; Orenstein's article is very long. Orenstein's point of view has particular weight because she was diagnosed at an early age, and was…
When we visualize data, we want to expose the information contained within, or to use the terminology Nate Silver popularized, to expose the signal and leave behind the noise. When graphs are not done right, sometimes they manage to obscure...
Statistics is sometimes described as inverse probability. In a typical probability problem, one starts by positing that a certain quantity has some given probability distribution, say the number of people entering a bank branch follows a Poisson distribution, and then goes on to compute probabilities such as the chance that more than 100 people (max capacity) require service at the same time. In a typical statistical problem, one observes the…
There has been quite a bit of buzz recently about the Button et al. Nature Reviews Neuroscience paper on statistical power. Several similar reviews have been published in psychology and other disciplines and come to broadly the same conclusion - that m...
That's not really what the FDA said but based on the shameful actions unearthed by ProPublica and reported by Scientific American here, one would think one can get away with this scam. Three years ago, the FDA busted a Houston lab (based on a whistleblower report) that fabricated loads of research studies that were used by the FDA to approve about 100 drugs. Eighty-percent of those drugs were generic drugs…
Josh hated this "dataless visualization" from ABC. (link; warning: ads). Here are his comments: The report has planes leaving China, landing across the globe and instantly infecting us all with bird flu. It doesn't do a good job explaining how...
An interesting episode is developing in econometrics over the very high profile Reinhart-Rogoff paper that was heavily cited as a source to "prove" that high levels of national debt impede growth. It appears that that result was based on a combination of spreadsheet errors, and bad assumptions. 1. Andrew Gelman has a great discussion here. His main concern is ethics of data analysts. This is a very important point -…
