If you’re interested in A/B testing on the Web, you should check out John Myles White’s talk at Tumblr on multi-armed bandit testing. You can learn a lot about standard A/B testing from the explanation he gives as a contrast to how multi-armed bandit tests work. I’ve read a lot of blog posts on multi-armed bandit tests, and this lecture is better than any of them in terms of explaining how this sort of testing actually works.