Distributed database creator and LinkedIn engineer Jay Kreps pokes a hole in the widespread myth that systems that scale horizontally are inherently reliable. This is an important post, because intuitively it seems like a system that scales horizontally and makes provisions for fault tolerance should be reliable. Indeed, that’s the value proposition for many people. Rather than having to be smart enough to provision big servers intelligently and figure out how to make them fault tolerant, you can just throw commodity hardware at the problem and be ready to swap out systems when you encounter the occasional hardware failure.
If you enjoy that article, move on to Daniel Abadi’s post on Replication and the latency-consistency tradeoff. It’s not about system failure but about the performance characteristics of distributed systems. This is the sort of real-world issue that is glossed over when you talk to people about distributed systems a lot of the time.
The hype about the new distributed database systems is that they make life easy. The truth is that they’re incredibly complex, but they make it possible for small companies to achieve things that were out of reach for all but the largest companies until very recently. I’m just starting to wrestle with some of this stuff and you can expect more posts about this topic.
Leave a Reply