A cool explanation and praise of Nate Silver's election forecasting

(via Reddit) Annmaria's Blog at thejuliagroup.com has a post up explaining how Nate Silver's election predictions worked, and why it was a really brave thing to do.   I thought this post was really cool, because I didn't quite understand any of what was going on at Silver's blog -- all I knew was that he was forecasting an Obama victory, and that was comforting.

Well, I got a little more than that.  I understand that when someone has a 90% chance of winning something, and that thing only happens once, it's still totally possible for the other person to win.  In fact, roughly one tenth of unique 90%-probability events go the way of the 10% margin.

I didn't know, though, whether Silver was partisan, or whether his math was any good.  The conversation about it was mostly over my head, and taking place in venues I don't closely follow.  (Although I've been barely following anyone in the past couple weeks -- been kind of busy and flustered.)

It turns out, there's this thing called the Central Limit Theorem, which says, according to Annmaria's Blog, 

the mean of an infinite number of reasonably large random samples will be the population mean.

No idea how you prove that, but apparently it's well-accepted statistical theory.

So, more realistically, the more reasonably large random samples, the more likely that their mean will be predictive of the actual result.  Which I think means that about 90% of polls added up to an Obama victory in the electoral college.

Annmaria's Blog (sorry I keep referring to her by her blog's title, but I can't find her name) explains that Silver's forecasting was extraordinary -- she says heroic -- not just because he applied good statistics, which many statisticians could have done, but that he put that prediction out there, took the risk of a misunderstanding public and an (at the end) 8% chance of ruining his own career, to stand up for good applied math.

I agree -- it's a heroic thing to do.  He elevated the quality of public discourse, and provided more vivid, undeniable evidence for the usefulness of data to reach conclusions.  I think his contribution will have a genuinely substantial positive effect on political discourse in the next 4 years.

NY Times on Innovation

The New York Times posted a cool article a few days ago called "32 Innovations That Will Change Your Tomorrow," outlining what  the writers argue are important innovations that exist now, and will be very important in the future.

When we ignore how innovation actually works, we make it hard to see what’s happening right in front of us today. If you don’t know that the incandescent light was a failure before it was a success, it’s easy to write off some modern energy innovations — like solar panels — because they haven’t hit the big time fast enough.

My favorite parts:

Soon, coffee isn’t going to taste like coffee — at least not the dark, ashy roasts we drink today. Big producers want uniform taste, and a dark roast makes that easy: it evens out flavors and masks flaws. But now the best beans are increasingly being set aside and shipped in vacuum-sealed packs (instead of burlap bags).

I am so looking forward to the future where dark roast coffee isn't a thing anymore.  Dark roast is awful.  People who claim to like dark roast are, mostly, lying, and that crap can't go away soon enough.  (Crappy coffee is a major reason I switched to pills for my caffeine intake.  That, and it's way cheaper.)

The problem with laptops and tablets, says Mark Rolston of the design firm Frog, is that they’re confined by a screen. He wants to turn the entire room into a monitor, where you can have the news on your kitchen table while you place a video call on your fridge. And when you’re done, you can swipe everything away, like Tony Stark in “Iron Man.”

I want this.  That is all.

Wearing a small sensor on your head, at home, while you sleep, could be the key to diagnosing diseases early and assessing overall health. “This tech,” says Dr. Philip Low, the founder of a medical technology firm called NeuroVigil, “enables us to look for faint signals of, say, schizophrenia, Parkinson’s, depression or Alzheimer’s in the brain, even though there may be no obvious symptoms.” [...] Currently, Low is working on a newer version of the device, which will be the size of a quarter and will transmit brain scans directly to smartphones and tablet computers. “We’re using sleep,” Low says, “as the gateway to the brain.”

I suffer from depression, and I fully support any mechanism which allows me to monitor for its onset.  It's not easy to tell you're depressed when you're depressed, and it's even harder to get yourself to do something about it at the time.  I'd love to have a thing to put on my head that would just email my psychiatrist if my mood was dropping in a serious way.  On that topic:

This year, Eva Redei, a professor at Northwestern’s Feinberg School of Medicine, published a paper that identified molecules in the blood that correlated to major depression in a small group of teenagers. Ridge Diagnostics has also started to roll out a test analyzing 10 biomarkers linked to depression in adults. “Part of the reason there’s a stigma for mental illness, including depression, is that people think it’s only in their heads,” Redei says. “As long as there’s no measurable, objective sign, we’re going to stay in that mind-set of ‘Just snap out of it.’ ” Blood tests will take mental illness out of the squishy realm of feelings. And as Lonna Williams, C.E.O. of Ridge Diagnostics, says, they’ll help people understand “it’s not their fault.”

Stuff that seems not-so-great:

Researchers at Imperial College London are closing in on a formula for a new kind of booze — synthetic alcohol, it’s called — that would forever eliminate the next morning’s headache (not to mention other problems associated with drinking).

Maybe it'll be awesome, especially for things like punch, but I feel like taking out the stuff that causes hangovers and serious drunkenness is going to involve taking out a lot of the other important things about drinking, at least on a cultural level.  Personally, my favorite thing about drinking is the narrative aspect -- it feels nice to have a gin and tonic or a glass of whisky, I can't imagine it feeling the same with artificial buzz-water.