Election surprise, and Three ways of thinking about probability

November 11, 2016
By
Election surprise, and Three ways of thinking about probability

Background: Hillary Clinton was given a 65% or 80% or 90% chance of winning the electoral college. She lost. Naive view: The poll-based models and the prediction markets said Clinton would win, and she lost. The models are wrong! Slightly sophisticated view: The predictions were probabilistic. 1-in-3 events happen a third of the time. 1-in-10 […] The post Election surprise, and Three ways of thinking about probability appeared first on…

Read more »

One informat to rule them all: Read any date into SAS

November 11, 2016
By
One informat to rule them all: Read any date into SAS

If you obtain data from web sites, social media, or other unstandardized data sources, you might not know the form of dates in the data. For example, the US Independence Day might be represented as "04JUL1776", "07/04/1776", "Jul 4, 1776", or "July 4, 1776." Fortunately, the ANYDTDTE informat makes it […] The post One informat to rule them all: Read any date into SAS appeared first on The DO Loop.

Read more »

David Rothschild and Sharad Goel called it (probabilistically speaking)

November 11, 2016
By
David Rothschild and Sharad Goel called it (probabilistically speaking)

David Rothschild and Sharad Goel write: In a new paper with Andrew Gelman and Houshmand Shirani-Mehr, we examined 4,221 late-campaign polls — every public poll we could find — for 608 state-level presidential, Senate and governor’s races between 1998 and 2014. Comparing those polls’ results with actual electoral results, we find the historical margin of […] The post David Rothschild and Sharad Goel called it (probabilistically speaking) appeared first on…

Read more »

Can a census-tract-level regression analysis untangle correlation between lead and crime?

November 11, 2016
By
Can a census-tract-level regression analysis untangle correlation between lead and crime?

Daniel Hawkins pointed me to a post by Kevin Drum entitled, “Crime in St. Louis: It’s Lead, Baby, Lead,” and the associated research article by Brian Boutwell, Erik Nelson, Brett Emo, Michael Vaughn, Mario Schootman, Richard Rosenfeld, Roger Lewis, “The intersection of aggregate-level lead exposure and crime.” The short story is that the areas of […] The post Can a census-tract-level regression analysis untangle correlation between lead and crime? appeared…

Read more »

Open letter to my lab: I am not "moving to Canada"

November 11, 2016
By

Dear Lab Members, I know that the results of Tuesday’s election have many of you concerned about your future. You are not alone. I am concerned about my future as well. But I want you to know that I have no plans of going anywhere and I intend to de...

Read more »

How effective (or counterproductive) is universal child care? Part 2

November 10, 2016
By
How effective (or counterproductive) is universal child care?  Part 2

This is the second of a series of two posts. Yesterday we discussed the difficulties of learning from a small, noisy experiment, in the context of a longitudinal study conducted in Jamaica where researchers reported that an early-childhood intervention program caused a 42%, or 25%, gain in later earnings. I expressed skepticism. Today I want […] The post How effective (or counterproductive) is universal child care? Part 2 appeared first…

Read more »

How effective (or counterproductive) is universal child care? Part 2

November 10, 2016
By
How effective (or counterproductive) is universal child care?  Part 2

This is the second of a series of two posts. Yesterday we discussed the difficulties of learning from a small, noisy experiment, in the context of a longitudinal study conducted in Jamaica where researchers reported that an early-childhood intervention program caused a 42%, or 25%, gain in later earnings. I expressed skepticism. Today I want […] The post How effective (or counterproductive) is universal child care? Part 2 appeared first…

Read more »

Modéliser et prévoir, entre ‘avoir faux’ et ‘avoir pas de chance’

November 10, 2016
By
Modéliser et prévoir, entre ‘avoir faux’ et ‘avoir pas de chance’

Depuis plusieurs heures, il y a une phrase qui revient sans cesse, et qui m’agace au plus haut point, sur “le modèle est faux…“. C’est une phrase que j’ai entendu pas plus tard qu’au séminaire ce midi, lorsqu’un collègue a dit que très clairement les modèles étaient faux puisqu’ils n’avaient pas prédit le vainqueur des élections américaines. Par exemple, le jour des élections, Huffington Post annonçait que Donald Trump avait…

Read more »

Here are the cool graphics from the election

November 10, 2016
By
Here are the cool graphics from the election

There were some very nice graphics work published during the last few days of the U.S. presidential election. Let me tell you why I like the following four charts. FiveThirtyEight's snake chart This chart definitely hits the Trifecta. It is...

Read more »

variance of an exponential order statistics

November 9, 2016
By
variance of an exponential order statistics

This afternoon, one of my Monte Carlo students at ENSAE came to me with an exercise from Monte Carlo Statistical Methods that I did not remember having written. And I thus “charged” George Casella with authorship for that exercise! Exercise 3.3 starts with the usual question (a) about the (Binomial) precision of a tail probability […]

Read more »

Laplace noising versus simulated out of sample methods (cross frames)

November 9, 2016
By
Laplace noising versus simulated out of sample methods (cross frames)

Nina Zumel recently mentioned the use of Laplace noise in “count codes” by Misha Bilenko (see here and here) as a known method to break the overfit bias that comes from using the same data to design impact codes and fit a next level model. It is a fascinating method inspired by differential privacy methods, … Continue reading Laplace noising versus simulated out of sample methods (cross frames)

Read more »

Explanations for that shocking 2% shift

November 9, 2016
By
Explanations for that shocking 2% shift

The title of this post says it all. A 2% shift in public opinion is not so large and usually would not be considered shocking. In this case the race was close enough that 2% was consequential. Here’s the background: Four years ago, Mitt Romney received 48% of the two-party vote and lost the presidential […] The post Explanations for that shocking 2% shift appeared first on Statistical Modeling, Causal…

Read more »

Graphic Continuum Flash Cards

November 9, 2016
By
Graphic Continuum Flash Cards

Jon Schwabish and Severino Ribecca have turned their Graphic Continuum poster into a set of cards. They're a good way to expand your visual vocabulary and find new ideas for how to represent your data. Each card shows one visualization technique as a stylized image on one side and a short definition on the other. They […]

Read more »

A 2% swing: The poll-based forecast did fine (on average) in blue states; they blew it in the red states

November 9, 2016
By
A 2% swing:  The poll-based forecast did fine (on average) in blue states; they blew it in the red states

The big story in yesterday’s election is that Donald Trump did about 2% better than predicted from the polls. Trump got 50% of the two-party vote (actually, according to the most recent count, Hillary Clinton won the popular vote, just barely) but was predicted to get only 48%. First let’s compare the 2016 election to […] The post A 2% swing: The poll-based forecast did fine (on average) in blue…

Read more »

How effective (or counterproductive) is universal child care? Part 1

November 9, 2016
By

This is the first of a series of two posts. We’ve talked before about various empirically-based claims of the effectiveness of early childhood intervention. In a much-publicized 2013 paper based on a study of 130 four-year-old children in Jamaica, Paul Gertler et al. claimed that a particular program caused a 42% increase in the participants’ […] The post How effective (or counterproductive) is universal child care? Part 1 appeared first…

Read more »

Visualize a torus in SAS

November 9, 2016
By
Visualize a torus in SAS

This article uses graphical techniques to visualize one of my favorite geometric objects: the surface of a three-dimensional torus. Along the way, this article demonstrates techniques that are useful for visualizing more mundane 3-D point clouds that arise in statistical data analysis. Define points on a torus A torus is […] The post Visualize a torus in SAS appeared first on The DO Loop.

Read more »

Election forecasting updating error: We ignored correlations in some of our data, thus producing illusory precision in our inferences

November 9, 2016
By

The election outcome is a surprise in that it contradicts two pieces of information: Pre-election polls and early-voting tallies. We knew that each of these indicators could be flawed (polls because of differential nonresponse; early-voting tallies because of extrapolation errors), but when the two pieces of evidence came to the same conclusion, they gave us […] The post Election forecasting updating error: We ignored correlations in some of our data,…

Read more »

What if NC is a tie and FL is a close win for Clinton?

November 9, 2016
By

On the TV they said that they were guessing that Clinton would win Florida in a close race and that North Carolina was too close to call. Let’s run the numbers, Kremp: > update_prob2(clinton_normal=list("NC"=c(50,2), "FL"=c(52,2))) Pr(Clinton wins the electoral college) = 95% That’s good news for Clinton. What if both states are tied? > update_prob2(clinton_normal=list("NC"=c(50,2), […] The post What if NC is a tie and FL is a close win…

Read more »

Election updating software update

November 9, 2016
By

When going through the Pierre-Antoine Kremp’s election forecasting updater program, we saw that it ran into difficulties when we started to supply information from lots of states. It was a problem with the program’s rejection sampling algorithm. Kremp updated the program to allow an option where you could specify the winner in each state, and […] The post Election updating software update appeared first on Statistical Modeling, Causal Inference, and…

Read more »

Election day, finally!

November 9, 2016
By
Election day, finally!

Abstract: I defend my claim that voter suppression is 1000 times worse than voter fraud, and furthermore, that voter suppression introduces systematic bias, whereas fraud introduces noise.  And from a statistical point of view, bias is much worse ...

Read more »

Now that 7pm has come, what do we know?

November 9, 2016
By

(followup to this post) On TV they said that Trump won Kentucky and Indiana (no surprise), Clinton won Vermont (really no surprise), but South Carolina, Georgia, and Virginia were too close to call. I’ll run Pierre-Antoine Kremp’s program conditioning on this information, coding states that are “too close to call” as being somewhere between 45% […] The post Now that 7pm has come, what do we know? appeared first on…

Read more »

Not all forecasters got it wrong: Nate Silver does it again (again)

November 9, 2016
By
Not all forecasters got it wrong: Nate Silver does it again (again)

Four years ago we posted on Nate Silver’s, and other forecasters’, triumph over pundits. In contrast, after yesterday’s presidential election, results contradicted most polls and data-driven forecasters, several news articles came out wondering h...

Read more »

What might we know at 7pm?

November 8, 2016
By

To update our effort from 2008, let’s see what we might know when the first polls close. At 7pm, the polls will be closed in the following states: KY, GA, IN, NH, SC, VT, VA. Let’s list these in order of projected Trump/Clinton vote share: KY, IN, SC, GA, NH, VA, VT. I’ll use Kremp’s […] The post What might we know at 7pm? appeared first on Statistical Modeling, Causal…

Read more »


Subscribe

Email:

  Subscribe