After the NBA Hackathon (see report here), I caught up with the winning team in the business analytics competition, DataBucket, composed of Barbara Zhan and Harold Li. Junkcharts: Congratulations for winning the business analytics competition at the NBA Hackathon. As a judge, I was very impressed by how much work you were able to do in 24 hours. Did you sleep or did you work all the way through? DataBucket:…
Kaiser Fung, author of Numbers Rule Your World and founder of Principal Analytics Prep, discusses why naive analysis of data hurts the debate over affirmative action.
Most of my friends with families fret over school quality when deciding where to buy their homes. It's well known that good school districts are also associated with expensive houses. A feedback cycle is at work here: home prices surge where there are good schools; only richer people can afford to buy such homes; wealth brings other advantages, and so the schools tend to have better students, which leads to…
One of my favorite statistics-related wisecracks is: the plural of anecdote is not data. In today's world, the saying should really say: the plural of anecdote is not BIG DATA. In class this week, we discussed a recent Letter to the Editor of top journal, New England Journal of Medicine, featuring a short analysis of weight data coming from a digital scale that, you guessed it, makes users consent to…
One of my favorite statistics-related wisecracks is: the plural of anecdote is not data. In today's world, the saying should really say: the plural of anecdote is not BIG DATA. In class this week, we discussed a recent Letter to the Editor of top journal, New England Journal of Medicine, featuring a short analysis of weight data coming from a digital scale that, you guessed it, makes users consent to…
Last week, my Columbia students discussed this nice article in the New York Times called "The Most Detailed Map of Gay Marriages in America". (link) The center of the article is this map: I asked the students to identify the problem that this dataviz is supposed to address. Someone responded that it tells us where gay married couples are found geographically. I asked her what is the answer to the…
In the business analytics universe, the discipline of "business intelligence" is often frowned upon. Business intelligence is primarily generating reports on business metrics, tracking them over time, and producing ad-hoc analyses explaining these trends. People often complain that such work is not challenging and not sexy. There is a stigma that BI work is data dumping. In reality, good BI work is rare and extremely valuable. Horrible BI work is…
The news is out that Uber got fined by the New York Attorney General's office for data breaches and privacy concerns. The headline writer for ZDNet nailed this one: "Uber fined peanuts in God View surveillance" (link). And the sub-lead has the kicker: "For a company with a valuation of over $50 billion, a $20,000 fine over user data protection is laughable." This settlement tells us one of the following…
One statistical concept that instructors frequently don't have time to cover in Stat 101 is the "interaction" effect. I will explain this concept using the fantastic interactive graphic by the visualization team at the German publication Zeit (please also read the corresponding post on Junk Charts here for some background.) When we ignore interactions, we end up with overly simplistic statistical summaries. For example, some study might find that drinking…
Today Destatis released a new projection of Germany’s population by 2060 accompanied by an all new animated population pyramid. It is …Continue reading →
