Sure, everybody is going to link to this article, but I’m not going to let that stop me. Google has revealed the server hardware it uses, as part of a larger effort to push for more energy efficient data centers. The details of how servers work are interesting, but what’s just as interesting to me is the degree to which the range of possibilities widens as things scale up. When you are rolling out servers by the thousand, it makes sense to design them yourself. I laughed at Google when I read about the study they did to select the right shade of blue, but their hardware efforts show what can be gained from taking a highly data driven approach. This article is a must-read.
Rethinking log messages
Paul Querna has written an interesting post arguing that developers should rethink how they handle logging — using a robust, machine-readable format (like JSON) rather than human-readable strings that are formatted so that each log entry is a single line long.
The big change is not in how you create log messages but rather in how you consume them. Right now, when a user notices an error, I tend to immediately log into the server and start looking for the log messages associated with the transaction in question using grep or my favorite pager.
Were I to log everything in a machine-readable format, it would make sense to have a more robust tool to parse the logs. Finding or building such a tool is doable, but it becomes yet another project. You need management to sign off on it, the systems administrators to agree to the infrastructure change, and someone to actually choose, test, and deploy the new tool for dealing with logs. Then you have to teach everyone who’s used to finding things in the old logs how to find things in the new logs.
That’s how things that seem like a great idea find a way to never become reality.