PC Week provides hammers more nails into the coffin for benchmarking in their shootout between 8 application servers. They provide so many qualifiers for the results of their testing that the numbers are meaningless. At least they state prominently that all of the products they tested are fast enough for anybody. Interspersed throughout the article are accounts of how the companies participating in the benchmark tweaked their software along the way to make it run better; most customers wouldn’t have the resources to do so. In fact, Microsoft wrote all of the state management code for their application specifically for the benchmarking. It forces one to wonder why anyone would take these benchmarks into consideration at all when deciding which application server to use. Reliability, capabilities of the development environment, and maintainability are all far more important than raw benchmarks, especially when you consider that all of the app servers tested support load balancing among multiple servers.