So the other day the New York Times wrote an article about a federal effort to create a monster database of data about all Americans that various government and law enforcement agencies can access without a warrant. The effort is being led by the creepy John Poindexter (former head of the NSA under Ronald Reagan and scumbag at large), and in addition to being incredibly invasive and offensive to everyone who believes in the most basic civil rights, is also completely misdirected.

The simple fact is that resources for analyzing information are limited, even for the federal government. This became completely obvious in the months after 9/11, when it was gradually revealed that we had more than enough information to track down the hijackers, but we didn’t have the resources to piece it all together. This new system is aimed at gathering huge additional amounts of information, and there’s some pie in the sky plan to sift through it with greater efficiency than we can currently manage. I just don’t think it can work. At some level, information has to percolate to decision makers in usable form, and generally that means from a real person who they trust to make decisions.

Getting back to baseball, I read an article a few months ago about how the Oakland A’s don’t scout high school players for the draft, they only scout college players. This is a pretty arbitrary way to decide who to scout, but it still works to the A’s advantage, because even though it eliminates a huge pool of players from consideration, it enables them to decide among the smaller pool of players with a much greater degree of confidence.

Spying on every single American and trying to pull needles out of the haystack (even in an automated fashion) is simply not going to be as productive as focusing on likely suspects. And trashing the civil liberties of every American in service of possibly slowing down terrorism is foolish in the extreme. There are countries out there that successfully monitor all of their citizens. Iraq is one of them — of course the intelligence services in Iraq are monstrous, and they spend nearly all of their time spying on Iraqis, and they have programs by which all citizens are encouraged to spy on their neighbors and family members. East Germany was another fine example of a country that had a working program for spying on all its citizens. Are these the countries we want to emulate?

We can do better than hiring criminals to spy on us.