(via Reddit) Annmaria's Blog at thejuliagroup.com has a post up explaining how Nate Silver's election predictions worked, and why it was a really brave thing to do. I thought this post was really cool, because I didn't quite understand any of what was going on at Silver's blog -- all I knew was that he was forecasting an Obama victory, and that was comforting.
Well, I got a little more than that. I understand that when someone has a 90% chance of winning something, and that thing only happens once, it's still totally possible for the other person to win. In fact, roughly one tenth of unique 90%-probability events go the way of the 10% margin.
I didn't know, though, whether Silver was partisan, or whether his math was any good. The conversation about it was mostly over my head, and taking place in venues I don't closely follow. (Although I've been barely following anyone in the past couple weeks -- been kind of busy and flustered.)
It turns out, there's this thing called the Central Limit Theorem, which says, according to Annmaria's Blog,
the mean of an infinite number of reasonably large random samples will be the population mean.
No idea how you prove that, but apparently it's well-accepted statistical theory.
So, more realistically, the more reasonably large random samples, the more likely that their mean will be predictive of the actual result. Which I think means that about 90% of polls added up to an Obama victory in the electoral college.
Annmaria's Blog (sorry I keep referring to her by her blog's title, but I can't find her name) explains that Silver's forecasting was extraordinary -- she says heroic -- not just because he applied good statistics, which many statisticians could have done, but that he put that prediction out there, took the risk of a misunderstanding public and an (at the end) 8% chance of ruining his own career, to stand up for good applied math.
I agree -- it's a heroic thing to do. He elevated the quality of public discourse, and provided more vivid, undeniable evidence for the usefulness of data to reach conclusions. I think his contribution will have a genuinely substantial positive effect on political discourse in the next 4 years.