Blog Archives

Data Science Students Predict the Midterm Election Results

November 12, 2014
By
Data Science Students Predict the Midterm Election Results

As explained in an earlier post, one of the homework assignments of my CS109 class was to predict the results of the midterm election. We created a competition in which 49 students entered. The most interesting challenge was to provide intervals for the republican - democrat difference in each of the 35 senate races. Anybody

Read more »

538 election forecasts made simple

November 4, 2014
By

Nate Silver does a great job of explaining his forecast model to laypeople. However, as a statistician I've always wanted to know more details. After preparing a "predict the midterm elections" homework for my data science class I have a better idea of what is going on. Here is my best attempt at explaining the ideas of

Read more »

Why I support statisticians and their resistance to hype

October 28, 2014
By

Despite Statistics being the most mature data related discipline, statisticians have not fared well in terms of being selected for funding or leadership positions in the new initiatives brought about by the increasing interest in data. Just to give one example (Jeff and Terry Speed give many more) the White House Big Data Partners Workshop  had

Read more »

Thinking like a statistician: don’t judge a society by its internet comments

October 20, 2014
By
Thinking like a statistician: don’t judge a society by its internet comments

In a previous post I explained how thinking like a statistician can help you avoid  feeling sad after using Facebook. The basic point was that missing not at random (MNAR) data on your friends' profiles (showing only the best parts of their life) can result in the biased view that your life is boring and uninspiring in

Read more »

Bayes Rule in an animated gif

October 17, 2014
By
Bayes Rule in an animated gif

Say Pr(A)=5% is the prevalence of a disease (% of red dots on top fig). Each individual is given a test with accuracy Pr(B|A)=Pr(no B| no A) = 90% .  The O in the middle turns into an X when the test fails. The rate of Xs is 1-Pr(B|A). We want to know the probability

Read more »

I declare the Bayesian vs. Frequentist debate over for data scientists

October 13, 2014
By
I declare the Bayesian vs. Frequentist debate over for data scientists

In a recent New York Times article the "Frequentists versus Bayesians" debate was brought up once again. I agree with Roger: NYT wants to create a battle b/w Bayesians and Frequentists but it's all crap. Statisticians develop techniques. http://t.co/736gbqZGuq — Roger D. Peng (@rdpeng) September 30, 2014 Because the real story (or non-story) is way too

Read more »

Applied Statisticians: people want to learn what we do. Let’s teach them.

September 15, 2014
By

In this recent opinion piece, Hadley Wickham explains how data science goes beyond Statistics and that data science is not promoted in academia. He defines data science as follows: I think there are three main steps in a data science … Continue reading →

Read more »

Academic statisticians: there is no shame in developing statistical solutions that solve just one problem

July 25, 2014
By

I think that the main distinction between academic statisticians and those calling themselves data scientists is that the latter are very much willing to invest most of their time and energy into solving specific problems by analyzing specific data sets. … Continue reading →

Read more »

The Big in Big Data relates to importance not size

May 28, 2014
By
The Big in Big Data relates to importance not size

In the past couple of years several non-statisticians have asked me "what is Big Data exactly?" or "How big is Big Data?". My answer has been "I think Big Data is much more about "data" than "big". I explain below. … Continue reading →

Read more »

Confession: I sometimes enjoy reading the fake journal/conference spam

April 30, 2014
By

I've spent a considerable amount of time setting up filters to avoid getting spam from fake journals and conferences. Unfortunately, they are exceptionally good at thwarting my defenses. This does not annoy me as much as I pretend because, secretly, … Continue reading →

Read more »


Subscribe

Email:

  Subscribe