Freedom to Tinker writers Andrew Appel and Ed Felten have put together a series of three posts you must read about AT&T data mining its call records on behalf of the government. How does the system work? The first post explains:

What is the “communities of interest” technology? It’s spelled out very clearly in a 2001 research paper from AT&T itself, entitled “Communities of Interest” (by C. Cortes, D. Pregibon, and C. Volinsky). They use high-tech data-mining algorithms to scan through the huge daily logs of every call made on the AT&T network; then they use sophisticated algorithms to analyze the connections between phone numbers: who is talking to whom? The paper literally uses the term “Guilt by Association” to describe what they’re looking for: what phone numbers are in contact with other numbers that are in contact with the bad guys?

When this research was done, back in the last century, the bad guys where people who wanted to rip off AT&T by making fraudulent credit-card calls. (Remember, back in the last century, intercontinental long-distance voice communication actually cost money!) But it’s easy to see how the FBI could use this to chase down anyone who talked to anyone who talked to a terrorist. Or even to a “terrorist.”

In the second post, Appel explains AT&T’s potential financial motive for eavesdropping:

Therefore, if it’s true that AT&T has such a secret room, then it may be simply that this is the only way AT&T knows how to make money off of shipping bits: it sells to the government all the information that passes through. Furthermore, economics tells us that in a commodity market, if one vendor is able to lower its price below cost, then other vendors must follow unless they also are able to make up the difference somehow. That is, there will be substantial economic pressure on all the other telecoms to accept the government’s money in exchange for access to everybody’s mail, Google searches, and phone calls.

In the third post, Ed Felten explains the economics of such a hypothetical arrangement. The post defies summarization, so just go read it.

There’s plenty more on the secret rooms in AT&T facilities that Appel discusses.