rc3.org

Strong opinions, weakly held

Month: December 2009 (page 2 of 7)

Trend of the decade: decentralized process

O’Reilly Radar’s best and worst tech of the decade inspired me to think about trends in software development. In the past decade, one prevailing trend has been the migration of process in software development from management to individual developers. Traditionally, process has been imposed on the development team, who usually ignore the process or actively resist it. Or in many cases there just isn’t any process at all.

Traditionally, “process” has been associated with project managers. Most processes had to be applied at the team level in order to work. Whether it was old school process like Waterfall or newer, agile processes like Scrum or Extreme Programming, everybody on the team has to participate for the processes to work. These processes are heavy, difficult to implement, and in many cases, completely ineffective.

The most powerful contribution of Extreme Programming, though, was that it was much more developer-friendly than the other approaches, and it really arrived as a bundle of specific practices. XP made it easy for developers to pick the practices that they liked best from the whole collection and start applying them, whether or not management bought into XP as an approach. So we’ve seen specific XP components like test-driven development, refactoring, continuous integration, and small releases really find democratic support among programmers.

A second major factor in encouraging developers to implement their own processes has been the growth of open source software. For one thing, it’s proven to be a laboratory for a lot of different approaches to building software, and for another, all of those approaches were implemented and used by developers themselves, and so they all work for at least some developers. The growth of open source software has also contributed a powerful set of programmer-friendly tools for implementing a development process, including version control, issue tracking, unit testing libraries, automated testing tools, continuous integration servers, project management applications, and more.

The third major change we’ve seen in the past decade or so has been the growth of blogging. People talk a lot about the effect of blogging on other fields, but its effect on software development has been profound. There were ever many magazines that were aimed at working software developers. Back in the day, there were magazines like Information Week, InfoWorld, and PC World that were for IT management — not for the people doing the real work. Developers had Usenet and mailing lists, but blogging led to an explosion in the amount of information that was written by programmers for programmers.

All of these factors contributed to developers stepping up and taking ownership of the process of developing software. These days, you see lone developers implementing a somewhat formal process for building software. Ten years ago, it was tough to find programmers who wrote unit test. Now there are a lot of programmers out there writing unit tests even when people aren’t looking. Programmers are taking ownership of the full life cycle of software development like never before as well. Take a look at some of the more recent writing on continuous deployment — it covers everything from requirements to maintenance to monitoring of running applications. More work done in public, public discussion of that work, and the introduction of new best practices have defined the trend of the decade — developers owning the processes under which they work. Hopefully it will continue in the next decade.

Huge problem, no solutions

As has been mentioned many times, what we’ve learned in the health care debate is that our government is currently hapless to really engage with the huge problems our country faces. First, let’s look at the magnitude of the health care crisis. Andrew Sullivan links to a chart from National Geographic that illustrates the problem in stark terms. Americans spend a whole lot more on health care than any other country, have ultimate outcomes in the mediocre to poor range, and don’t even go to the doctor very much. We’re spending an awful lot of money and not getting a lot for it. And the second order effect is that our future obligations to continue spending ever increasing amounts of money on health care are going to bankrupt the federal government and eat away at wage growth indefinitely, unless things change in a pretty radical fashion.

As has been mentioned any number of times, this is just one of our really big problems. We also have structural deficits in the federal budget that must be addressed at some point, we have climate change to deal with, and we have any number of other smaller, but still important problems that are also going to require attention.

What we have seen this year is a federal government that is unequipped to grapple with any of these problems in a serious way. Republicans refuse to attempt to solve any of these problems at all. Not only have the Republicans declined to work with the Democrats in moving any bills through the Senate or House this year, but in the six years where they were in charge (not counting 2007 and 2008 where Democrats ran Congress), they opted not to address any of these problems at all. They continue to deny that global warming is even an issue, and the only thing they did on the health care front was work with Democrats to pass an unfunded prescription drug benefit for seniors. And the only thing they did while they were running things was expand the budget deficit to a massive degree through tax cuts without offsetting spending cuts, new programs (like the Medicare drug benefit), and of course, multiple wars that they chose not to fund through revenue, either.

When President Bush was in charge, it was pretty obvious that the most acute problem facing the country was his administration. Now that he’s out of office, it’s clear that we can zero in on the de facto supermajority requirement in the Senate as the largest obstacle to progress. Liberals are angry with President Obama (for some good reasons and some bad ones), but it’s clear that the Senate is the biggest problem. President Obama could be twice as committed to the progressive agenda as he is, and Harry Reid would still need every member of his caucus plus both independents to vote for every bill to make progress. Ezra Klein has an op-ed today that explains just what a liability this is for effective government.

So the health care reform bill that the Senate passed will, I think, be good for the country, but it’s almost devoid of ambition. Beyond expanding coverage, it doesn’t take on any of the biggest problems we face going forward. And it seems clear it was the best bill we could possibly get from the Senate given the rules it operates under. Had Al Franken lost the recount battle in Minnesota, we would probably have nothing. It’s hard not to be pessimistic about the future of the country when one political party has as much power as any has had in decades, and it’s still unable to make real progress on its agenda.

Colbert discusses the White House Correspondents’ Dinner

James Warren attended an event where Stephen Colbert discussed his performance at the White House Correspondents’ Dinner in 2006. I’ve been fascinated by that performance since it happened, mainly due to Colbert’s audacity in wickedly satirizing the President and the press right in front of their faces. I’ve been dying to hear anything at all about what Colbert was thinking at the time, and Warren’s blog post delivers.

Here’s a snippet:

Tom Purcell, a Second City alum and the television show’s co-executive producer, explained the tactical rationale: Bush, he said, was the typical “Big Man” one might find in innumerable institutional settings. “We thought it was the Big Man. He hires somebody to make fun of him, and he chuckles. You see it at office Christmas parties. You say the boss is so cheap that….and he laughs and everybody laughs. That’s what we thought we were doing. They wouldn’t have brought us in if they didn’t know what jokes we did.”

If you missed it or want to watch it again, the full video of Colbert’s performance is available on Google Video.

Unusual Christmas traditions

First, here’s the Smithsonian’s food blog interviewing Vickie Kloeris, who handles the food for the International Space Station, about Christmas Dinner in space. A few year round rules: nothing that requires refrigeration, nothing carbonated, and no alcohol.

Second, Slate covers the Swedish national tradition of sitting down to watch a national broadcast of old Disney cartoons on Christmas Eve.

Merry Christmas!

Why not encrypt the Predator downlink?

Leave it to Bruce Schneier to come up with a reasonable explanation for why the video downlink from Predator drones is not encrypted. His theory is that it’s because it would be a key management nightmare, given the way the military manages encryption keys. Basically, the audience for the UAV downlinks changes frequently and has a wide variety of security clearances, and military encryption is for data is for top secret stuff.

Creating a true impression

For one thing, newspapers work very hard to report things that are true, but they are less concerned with whether the overall impression from their reporting is a true impression. Shark attacks, for instance, happen very rarely. But if you report excitedly on every shark attack that happens, people will think they happen quite a bit. You haven’t told anyone any lies, but your stories aren’t leaving your readers with a true impression of the world.

Ezra Klein in The media’s Sarah Palin problem — and ours.

Making it easier to monitor slow queries

Ronald Bradford has a good idea for developers who are debugging slow queries — include comments in your SQL so that the queries are easier to identify in MySQL’s process list or in the slow query log. I’m wondering how easy it would be to include a setting for various database libraries (Hibernate, ActiveRecord, any of several PHP frameworks, and so on) to include the context in which a query is called in a comment when the query is run. For example, it would be great, if inside every Hibernate query, the class and method from which the query was called were included.

I’m going to look into hacking this functionality into Kohana, the PHP framework I’ve spent a lot of time with in 2009.

dangerousmeta! celebrates 10 years

Garret Vreeland’s wonderful blog, dangerousmeta! is celebrating its 10 year anniversary. Garret usually keeps it brief, but his post on spending 10 years at the helm of a blog is worth reading in full. Congratulations, old man!

The safest choice in ethnic cuisine

Korean is perhaps the safest bet, for two reasons. First, non-Koreans are not usually interested in the food. They might enjoy Bul-Gogi but there will be plenty of other dishes for Korean patrons and these will not be “dumbed down.” The lack of mainstream interest limits the potential for sell-out behavior on the part of the restaurant. Second, many Korean dishes, most of all the pickled vegetables, “travel” relatively well and do fine in a culture — the USA — which is not obsessed with fresh ingredients.

Tyler Cowen: Which are the “safest” cuisines? He nominates Chinese as the most dangerous (in terms of the likelihood of getting a bad meal). I heard once that there are more Chinese restaurants in America than McDonalds and Wendy’s combined. With those kinds of numbers, the odds are never going to be very good.

You shouldn’t hate releasing code

You shouldn’t hate releasing software. Especially if your software is a Web site, not something that comes in an installer or disk image. And yet it’s really easy to get into a state where releases suck. I think that the thought of making releases suck less is the one of the main attractions of continuous deployment.

Here’s a partial list of reasons why releases can suck:

  1. Releases require downtime. When downtime is required, releases have to be scheduled far in advance, and the release schedule starts to affect other things. It becomes harder to work productively near releases (especially if your source code management is not up to par), and usually downtime releases must occur in the middle of the night or on weekends.
  2. The prerequisites for each release are not well documented. I’ve been in situations where the release time rolls around, everybody is online and ready, and the proper build wasn’t tagged. Or the production build doesn’t have the production log settings (a mistake I personally have made a number of times). Or the list of database changes that need to be made wasn’t prepared. Or the new build is deployed and the database changes weren’t applied and so you have to start over.
  3. Releases are too large. At some point, releases become large enough that nobody really knows everything that a release impacts. At the smallest level, a release includes a single bug fix. It’s easy to understand the possible risks involved with such a release. A release that includes hundreds or even dozens of changes can become unmanageable.
  4. Releases are insufficiently tested. Nothing’s worse than everyone waiting in a holding pattern while developers furiously bang away to fix a bug that slipped into the release and was only found when the code went out.
  5. People don’t have enough to do. In many cases, lots of people have to be available for a release, but they spend a lot of that time waiting for things to happen. Frustrating.
  6. Releases take too long. Sometimes the release process is just too cumbersome. Getting releases right is important but ultimately the actual release process is just a form of housekeeping. The more people that are involved and the longer it takes, the more resources are used that could be better applied to something else. Plus most of the work associated with a release is really boring.

My theory is that releases don’t have to suck. The releases I’m involved with exhibit few of these problems, but my goal for 2010 is to eliminate the remaining problems to excise the remaining suckiness.

If you liked this, you may also like my post When deployments go wrong from a couple of years ago.

Older posts Newer posts

© 2024 rc3.org

Theme by Anders NorenUp ↑