rc3.org

Strong opinions, weakly held

Tag: software development (page 8 of 16)

Extreme agility

Github developer Scott Chacon describes their development process:

At GitHub we don’t have a project tracker or todo list – we just all work on whatever is most interesting to us. No standup meetings, burndown charts or points to assign. No chickens or pigs. It’s sort of the open source software style of business – everyone itches thier own scratch. Inexplicably, it works really well and keeps everyone engaged, new features appearing quickly and bugs fixed rather fast. No managers, directors, PMs or departments – and it’s the most agile, focused and efficient team I’ve ever worked with. Maybe we should write a book about it.

The first question that occurs to me when reading this is, under what conditions would such an approach work? (The second is, do they have a quality assurance department, and if so, how do they plan their work?)

But let’s go back to the first. I can think of a few prerequisites:

  1. Your developers must be users of the product. In fact, I think this sort of approach could only work for companies that build tools for developers.
  2. Your developers must be able to iterate without relying too much on other members of the team.
  3. The business must not have customers who are promised certain features by a certain date. Customers of every software company I’ve ever worked for have requested features that no developer wants to work on, but they pay the bills, so we worked on them anyway.

There are probably a lot more conditions required to make this sort of arrangement work, but those are the ones that immediately leap out at me. The beautiful thing about this approach is that it insures that you get exactly the developers you’d like to have. The people who would not want to work under these conditions are not the ones you’d want anyway, and the developers you would want would leap at the chance to work in this fashion.

Thanks to Ryan Tomayko for the link.

Update: Be sure to read Ryan’s comment below, he adds a lot more details about how things work at GitHub.

Can we call hardware/software integration a trend?

Ted Leung points out some parallels between the iPad and Oracle’s new strategy integrating Sun’s hardware:

I spent most of yesterday watching the Oracle/Sun strategy webcast, and a major theme was the way that Oracle plans to tightly integrate Sun’s hardware, and to optimize the entire hardware and software stack. The Oracle Exadata database machine was repeatedly touted as an example of this kind of integration. If the benchmarks and early customer experiences are indicative, this integration has paid off handsomely, as it has also with the Sun Storage 7000.

And the bottom line:

Many of us in the “open” world decry vertical integration because it is almost inevitably closed, but the kind of engineering virtuosity that is on display does impress.

The argument against private methods

A few days ago I ran into Kent Spillern’s post, Private Methods are a Code Smell. Here’s the meat of it:

Private helper methods indicate classes are doing too many things. Moving private helper methods to different classes, including creating new classes if necessary, splits the original responsibilities across multiple classes leading to simpler, better designs.

This post hit really close to home for me because I’m a heavy user of private utility methods. Splitting code up into lots of methods is a form of self-documentation that I advocate. He talks about one misuse of private methods that I don’t indulge in:

Sometimes, private methods are created to split complex logic or processes into small, easily digested pieces. Often, such private methods have gnarly dependencies because they directly access or modify internal state.

I write private methods for the reasons described in the first sentence, but I very rarely do what’s described in the second sentence — write private methods that modify the member variables in a class. Recently, though, I’ve started moving away from private methods. I don’t mind classes with lots of methods, but when you’re writing tests, it’s easiest to deal with small bits of code with known inputs and outputs.

I’ve also gotten used to frameworks like Rails and Kohana that both support the concept of helpers, which are a sort of thowback to procedural programming. I like helpers because they’re easy to test, and because you can use them in any situation you like. The more object-oriented alternative to writing helper or “util” classes is to use superclasses. Classes that share functionality extend the same superclass that contains the methods with shared functionality. This is the approach that the Spring Framework takes.

Even though I’d thought about moving away from private and protected methods, I had never really thought of them as something to be deliberately avoided, but I’m halfway along the path to being convinced.

The continuous integration tool chain

Since last year, I’ve been somewhat obsessed with continuous deployment. One of my work projects for this year is to start moving toward a continuous deployment model. Making progress is going to require a lot of engineering work, and also a few cultural changes.

My first big goal is to start deploying bug fixes as soon as they have been tested. Eventually it would be nice to break features out of the deployment process, but I think that if you have identified a bug and checked in a fix, the fix shouldn’t wait around for the next scheduled release to go live.

In September, I wrote about what you need to do to prepare for continuous deployment. Now I’m going to write a little bit about the tools I’m planning on using.

For continuous integration, I’m using Hudson, it’s surprisingly simple to set up as long as you have a build script that will compile all your code and run your tests. The first step is to get it to check out our code, make a build, and run unit tests against the build. I’ve accomplished that on my local machine, the next step will be to set it up on a server and have it start running the build automatically when people check in changes.

To improve test coverage, I’m using the Eclipse plugin EclEmma. It builds your code and shows exactly which lines of code are not covered by your unit tests. I find that test coverage tools are indispensable, especially when you’re working with a code base that was not written in a TDD regime. (A fair amount of code in this code base was written before TDD was under discussion.)

For testing Web interfaces, I plan on using Selenium. Ben Rometsch at Carsonified explains how to run Selenium tests through Hudson.

We also have a Web services component of our application that I plan on testing end to end. For those purposes, I’m writing a tool in Ruby that accepts tests in YML format, submits a request to the Web service, validates the response against the expected data, and then goes and checks the database to make sure the proper data was stored there as well. At some point I’ll be hooking that into Hudson as well. At that point, the process will look something like this:

  1. Check out the code
  2. Make a build
  3. Run the unit tests
  4. Set up a test version of the database
  5. Deploy the application in a test environment
  6. Start the application
  7. Run the Web service tests

The Web interface is a separate project and will follow its own process.

Getting continuous integration up and running is but one of four steps toward getting a continuous deployment regime set up. The next big step I’m taking is nailing down the build process and then updating the build scripts to remove a few manual steps we’re performing right now.

Do you want to be in the software business?

Tim Bray’s post, Doing It Wrong is a great summary of the overall problem with large IT projects, contrasting the traditional practices of corporate IT with the more iterative, more realistic approaches used by Internet companies.

Having worked on dozens of projects over the years, this is a topic about which I have some thoughts.

Years ago it occurred to me that Amazon.com was a software company. That’s not news to anybody any more now that they’ve turned pieces of their infrastructure into cloud computing products that they lease out, but I realized it long before Amazon announced their Web services products, and even before they started outsourcing their online store and logistics to other retailers like Borders and Toys R Us. Early on, Amazon was thought of as a retailer, but their retail strategy was based on building the best software for running an online retailer. My guess is that Amazon.com knew from the beginning that they were in the software business, but a lot of companies that expend a large share of their resources building software don’t.

What I’ve also realized is that the best way to avoid failed software development projects is to avoid starting software development projects. Awhile back I worked with an organization that I won’t name. They were a very small shop that had a Web site, an online store, and an internal membership management application. All of them were custom and were maintained by an in house IT person who also performed every other IT task for this organization. The total head count at this organization was maybe five people, and yet they were maintaining complex applications built using ColdFusion and FileMaker. They had decided to rewrite these applications, combining them into a single application built using Ruby on Rails. The project was not successful.

While the scale was vastly different, the project was very similar to many massive IT projects. This organization’s entire business was run through this legacy software and the budget was very large by their standards. The rewrite was behind schedule, the requirements were not gathered well so the resulting application was not a good replacement for the (terrible) applications it replaced, and they were on the hook to keep Rails expertise in house or on retainer indefinitely because of this completely custom, highly specialized application they had built.

When I talked to them about fixing the new application, the question that popped into my mind was, “Do you guys even want to be in the software business?” Committing to this custom application meant that they would invest a sizable chunk of their resources into software development forever, and they lacked the expertise to manage such a project. The right solution for them really was to dump their custom application and use whatever existing software they could afford that would allow them to move forward, even if it was quite different from what they had before.

I think that’s true for a lot of organizations. The question people need to be asking is how little custom software can they get away with having. The ideal number is zero. If you’re working at one of those web design that rolls a new content management system for every customer, you’re doing your customers a disservice. Honestly, if you’re selling them your own proprietary CMS you’re probably doing them a disservice.

Software developers like to build things. And most developers are confident that they can provide something that perfectly solves whatever problem they’re confronted with as long as they can write it from scratch. Developers are horrible at estimating the long term costs of building applications yourself. And they have an incentive to be bad at it, because if they were good at it, nobody would let them loose to write custom software.

The first question everyone should ask when thinking about building custom software is, does building this give me a tangible strategic advantage over the competition? Specifically, does it provide enough of an advantage to make up for the amount you’ll spend on initial development and on maintenance and improvements going forward. Account for the fact that packaged software will likely be improving over time as well, so your custom solution may be great today, but the feature set in the packaged solution may blow away what you have two years down the road.

The second question is, how small can you start? The larger the project, the more likely it is to fail. And by small, I mean relative to the capabilities of your organization. If you have one in house developer who has other responsibilities as well, pretty much any project is large. You want to start with something you can release right away. Starting small is easier when you’re doing something totally new and difficult when you’re trying to replace something that already exists, but that’s really a warning against replacing things. Replacements for existing applications are the projects that are most likely to fail.

Trend of the decade: decentralized process

O’Reilly Radar’s best and worst tech of the decade inspired me to think about trends in software development. In the past decade, one prevailing trend has been the migration of process in software development from management to individual developers. Traditionally, process has been imposed on the development team, who usually ignore the process or actively resist it. Or in many cases there just isn’t any process at all.

Traditionally, “process” has been associated with project managers. Most processes had to be applied at the team level in order to work. Whether it was old school process like Waterfall or newer, agile processes like Scrum or Extreme Programming, everybody on the team has to participate for the processes to work. These processes are heavy, difficult to implement, and in many cases, completely ineffective.

The most powerful contribution of Extreme Programming, though, was that it was much more developer-friendly than the other approaches, and it really arrived as a bundle of specific practices. XP made it easy for developers to pick the practices that they liked best from the whole collection and start applying them, whether or not management bought into XP as an approach. So we’ve seen specific XP components like test-driven development, refactoring, continuous integration, and small releases really find democratic support among programmers.

A second major factor in encouraging developers to implement their own processes has been the growth of open source software. For one thing, it’s proven to be a laboratory for a lot of different approaches to building software, and for another, all of those approaches were implemented and used by developers themselves, and so they all work for at least some developers. The growth of open source software has also contributed a powerful set of programmer-friendly tools for implementing a development process, including version control, issue tracking, unit testing libraries, automated testing tools, continuous integration servers, project management applications, and more.

The third major change we’ve seen in the past decade or so has been the growth of blogging. People talk a lot about the effect of blogging on other fields, but its effect on software development has been profound. There were ever many magazines that were aimed at working software developers. Back in the day, there were magazines like Information Week, InfoWorld, and PC World that were for IT management — not for the people doing the real work. Developers had Usenet and mailing lists, but blogging led to an explosion in the amount of information that was written by programmers for programmers.

All of these factors contributed to developers stepping up and taking ownership of the process of developing software. These days, you see lone developers implementing a somewhat formal process for building software. Ten years ago, it was tough to find programmers who wrote unit test. Now there are a lot of programmers out there writing unit tests even when people aren’t looking. Programmers are taking ownership of the full life cycle of software development like never before as well. Take a look at some of the more recent writing on continuous deployment — it covers everything from requirements to maintenance to monitoring of running applications. More work done in public, public discussion of that work, and the introduction of new best practices have defined the trend of the decade — developers owning the processes under which they work. Hopefully it will continue in the next decade.

How programmers see each other

Ruby Inside’s Holiday Fun: How Programming Language Fanboys See Each Others’ Languages is legitimately funny and dead on accurate.

Bug of the year

I thought for sure the date bug that prevented the Motorola Droid’s camera from focusing properly in 24.5 day cycles (24.5 days of proper focus, followed by 24.5 days of focusing problems) would go down as the bug of the year, but then I learned about the face tracking software from HP that doesn’t work for black people.

Performance reviews for developers

It just occurred to me that I’ve never had (or given) a really well executed developer-centric performance review, and it had me thinking about what should be in such a review. There are a lot of things that should be discussed in any performance review, I’m not going to go over those. I want to talk about the kinds of questions a developer and their manager probably ought to discuss at their review.

To me, the most important part of a performance review is to help the person getting the review get better at their job. It’s a chance for people to get feedback that will help them figure out which skills they need to improve and determine where they may be lacking in terms of professionalism. The second most important part of a performance review is to help the manager get as much value out of the employee as possible. Does this person work better undisturbed most of the time or regularly collaborating with others? Is this person happiest grinding away at hard problems that discourage most people? Do they like or dislike refactoring old code that could use improvement?

With that in mind, here are some questions that managers might want to discuss with developers in their performance review:

  • What code that you wrote this year are you most proud of?
  • What code that you wrote this year would you rewrite if you had the time?
  • Did you work on anything this year that you felt was a waste of time?
  • How was the release schedule? Did we release too often or not often enough?
  • What skills got rusty this year that you want to get back to using?

The last thing I’d add is that I think annual performance reviews are really important. In many cases where programmers manage other programmers, they tend to be ignored or handled in a cursory fashion, but I think that’s a big mistake. So if you’re a manager, do your performance reviews and put some real work into them. Your employees will benefit, even if they don’t appreciate them.

ETL and MVC

Many Web developers think in MVC, the design pattern that describes Web development frameworks from Struts to Rails to CodeIgniter. What I’m finding lately, though, is that it’s not incredibly intuitive. For one thing, people know what a “view” is (the “v” in MVC), but trying to get people to understand the difference between a “model” and a “controller” can be difficult. Also, what we call controllers in most MVC implementations aren’t really controllers. The controller is a component of the MVC framework itself. Once you’ve built a couple of projects with MVC, that sort of thing doesn’t matter, as long as you get how the applications are built. But these problems present difficulties when training new developers.

The other day I was thinking about another acronym that’s relatively new to me — ETL. It stands for Extract, transform, load and is DBA jargon for the process of migrating data from one data source to another. But I think that for reports in particular, it describes how you should organize your code pretty well.

In a reporting application, extraction would refer to the process of figuring out which data the user wants using the URL and any filter parameters that are passed in, and running the query to retrieve that data. Transformation is the massaging of the data so that it can be presented in whatever format is needed. I strongly prefer having as little transformation code in my reports as possible, it’s usually much better to store the data in the format that will be needed for the reports. And then load, in this case, refers to the process of presenting the transformed data, whether it’s on a Web page or in a downloadable export file.

I think that if you can get developers to think about reports in terms of these three discrete steps, you’re more likely to wind up with code that’s more flexible and better organized that it would be otherwise. You can always yell at people and tell them not to put business logic in the view layer, but creating a mental model that explains why that’s a bad idea might help.

Older posts Newer posts

© 2025 rc3.org

Theme by Anders NorenUp ↑