(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Paul Campos pointed me to this post from a couple of days ago in which he wrote:
I think it’s fair to say that right now the consensus among elite observers across the ideological spectrum . . . is that the presidential race is over because Donald Trump has no chance of winning — or rather his chances of winning are so slim that they can be treated as functionally zero for all practical purposes. . . .
Which raises this question: why do the instruments (in the form of various predictive mechanisms) not agree? For example, Nate Silver’s methodology still gives Trump a nearly one in five chance of winning. The betting markets, in which people put their money where there the mouths are, aren’t as bullish, but they still give Trump a one in ten chance. . . .
So what’s going on? Are the various predictive mechanisms flawed? Or does Trump still have a very real chance?
I replied:
Nate puts himself out there so it’s only fair that he gets criticized, but really I think the problem of updating priors given data is super-tough, and it’s easy to make mistakes.
On one hand, try to use your best judgment and you find yourself subject to all sorts of cognitive biases and perverse incentives (such as Nate not wanting to underestimate Trump twice, or Nate underreacting to polls in part in reaction to earlier criticism that he was overreacting to polls). On the other hand, if you “tie yourself to the mast” by using a preset algorithm, you can find yourself staring while your estimate goes off the cliff. It’s a delicate balance, and Nate’s in a tough position, putting out these forecasts every day.
I had this sort of thing in mind when writing that Warriors post yesterday. From the perspective of the judgment and decision making literature, the challenge is integrating new information at the appropriate rate: not so fast that your predictions jump up and down like a yo-yo (the fate of naive poll-watchers) and not so slow that you’re glued to your prior information (as happened with the prediction markets leading up to Brexit).
Given how hard it is to do this sort of information updating even in controlled settings such as sports, it’s no surprise that pundits, data journalists, and political scientists can have difficulty integrating new information with elections, where the N is lower and the rules of the game keep changing.
Is there any work in cognitive psychology on the rate at which people incorporate new information? I’ve read about the base-rate fallacy but I don’t recall seeing any studies of this dynamic process. It seems like an important topic.
The post Updating fast and slow appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
