Somehow, Nate Silver’s political Web site escaped my notice until today. Silver is using the same techniques he and other used in building improved baseball statistics to analyze the performance of pollsters in 2008 elections, and to aggregate multiple polls into an accurate prediction of voter behavior.
The site provides a lot of interesting numbers, including the odds of various scenarios occurring, like “Obama wins all Kerry states” and “McCain loses OH/MI, wins election.” The site also provides return on investment rankings for the states, and the individual chance of the candidates winning each state.
The reason this post has the subject it does, though, is that it’s fun to watch sports analysis go mainstream. Sports analysis is a perfect training ground for statistical analysis because of the discrete raw statistics that can be used, and the fact that predictions can very easily be compared to actual results.
Most sports analysis comes down to a simple question, “Which things help teams win?” So if I’m a football analyst, I may argue that average time of possession better predicts winning than average margin of victory. I can then process the historical data for as many seasons of football as I like and test that argument. It doesn’t matter how beautiful my theory is, the data will quickly show whether I’m right or wrong.
It’s not surprising to me to see people who have cut their teeth in the world of sports analysis start applying their methods to other areas. The numbers may be different, but the discipline is the same. Silver is doing with polling numbers and election results what he did before with batting averages and baseball games.
If nothing else, it makes me feel like all of the time I’ve spent reading about quantitative analysis of sports hasn’t been a total waste.
If you’re into this sort of analysis, there’s also the Princeton Election Consortium, which posted a mild critique of Silver’s methodology. And for a more naive analysis that just looks at the latest poll result for each state, see electoral-vote.com.
John Myles White explains multi-armed bandit testing
If you’re interested in A/B testing on the Web, you should check out John Myles White’s talk at Tumblr on multi-armed bandit testing. You can learn a lot about standard A/B testing from the explanation he gives as a contrast to how multi-armed bandit tests work. I’ve read a lot of blog posts on multi-armed bandit tests, and this lecture is better than any of them in terms of explaining how this sort of testing actually works.