How will society adjust to ever-easier data collection?
1

How will society adjust to ever-easier data collection?

The New York Times ran two opinion pieces this weekend right next to each other that both stand at the intersection of the how the government and politics work and social change that results from technological change. In the first, Joe Nocera argues that the big question in the resignation of David Petraeus is whether we’re comfortable with the FBI snooping through our email on relatively flimsy grounds:

But the Petraeus scandal could well end up teaching some very different lessons. If the most admired military man in a generation can have his e-mail hacked by F.B.I. agents, then none of us are safe from the post-9/11 surveillance machine. And if an affair is all it takes to force such a man from office, then we truly have lost all sense of proportion.

The second was about what increased use of data in political campaigns means long-term. As I’ve mentioned, I’ve been working in the analytics world this year, so this topic is highly relevant to me. It’s also very complicated. On one hand, improving our ability to collect and analyze data enables us to better understand what people want and expect from our products, or, in the case of campaigns, our politicians. On the other hand, combining our more advanced understanding of human behavior with deeper data sets creates the opportunity for more effective manipulation in addition to more effective communication.

While the people creating big data tools may not be evil, the organizations that use them going forward may not agree to the same principles. The big question in both the Petraeus case and in the use of big data by campaigns is that regardless of our level of comfort with the government, campaigns, or companies knowing so much about us, we don’t really have control over the gathering of that information.

Links from May 28th
0

Links from May 28th

Tim O’Reilly on Aneesh Chopra
0

Tim O’Reilly on Aneesh Chopra

Since President Obama said he’d name a “federal CTO,” I’ve been watching with interest to see who he’d pick. A lot of people expected that he’d pick a big name industry figure, but he went another direction and chose Aneesh Chopra, the Secretary of Technology for the state of Virginia. Chopra’s name is not one I’d heard before, but Tim O’Reilly makes a compelling argument for why he’s a good choice.

Chopra’s experience is specifically in driving technological advancement in government. I agree with O’Reilly that picking someone who’s never worked in government would limit what can be achieved, because learning how things work in a government setting takes time. Chopra is not going to need to get up to speed.

What I’d like to see is Chopra writing a blog himself, or at least Tweeting. If the federal government is doing interesting things on the IT front (and I expect it will be), part of the value he can provide is teaching states and municipalities what they could be doing as well. I hope he takes the outreach part of the job seriously.

A Web 2.0 perspective on open government
4

A Web 2.0 perspective on open government

Ed Felten and some co-authors have written a paper discussing a more modern approach to open government. They suggest that the best approach is that rather than government agencies focusing on building Web sites that provide access to government data, they should focus on providing the data itself in an open format so that it can be utilized by anyone.

Here’s the meat of the proposal:

Rather than struggling, as it currently does, to design sites that meet each end-user need, it should focus on creating a simple, reliable and publicly accessible infrastructure that “exposes” the underlying data. Private actors, either nonprofit or commercial, are better suited to deliver government information to citizens and can constantly create and reshape the tools individuals use to find and leverage public data. The best way to ensure that the government allows private parties to compete on equal terms in the provision of government data is to require that federal websites themselves use the same open systems for accessing the underlying data as they make available to the public at large.

Sounds like a great idea to me. I love the idea of forcing government agencies to eat their own dog food in order to propel this kind of approach.