I’ve been listening to the audiobook of Heart of Darkness this week, read by Kenneth Branagh. It’s fantastic. It also reminds me of some jobs I’ve had in the past.
There’s a great passage in which Marlow requires rivets to repair a ship, but finds that none are available. This, in spite of the fact that the camp he left further upriver is drowning in them. That felt familiar. There’s also a famous passage involving a French warship that’s blindly firing its cannons into the jungles of Africa in hopes of hitting a native camp situated within. I’ve had that job as well. Hopefully I can help you avoid getting yourself into those situations.
There are several really good lists of common traits seen in well-functioning engineering organizations. Most recently, there’s Pamela Fox’s list of What to look for in a software engineering culture. More famous, but somewhat dated at this point, is Joel Spolsky’s Joel Test. I want to talk about signs of teams that you should avoid.
This list is partially inspired by Ralph Peters’ Spotting the Losers: Seven Signs of Non-Competitive States. Of course, such a list is useless if you can’t apply it at the crucial point, when you’re interviewing. I’ve tried to include questions to ask and clues to look for that reveal dysfunction that is deeply baked into an engineering culture.
Preference for process over tools. As engineering teams grow, there are many approaches to coordinating people’s work. Most of them are some combination of process and tools. Git is a tool that enables multiple people to work on the same code base efficiently (most of the time). A team may also design a process around Git — avoiding the use of remote branches, only pushing code that’s ready to deploy to the master branch, or requiring people to use local branches for all of their development. Healthy teams generally try to address their scaling problems with tools, not additional process. Processes are hard to turn into habits, hard to teach to new team members, and often evolve too slowly to keep pace with changing circumstances. Ask your interviewers what their release cycle is like. Ask them how many standing meetings they attend. Look at the company’s job listings, are they hiring a scrum master?
Excessive deference to the leader or worse, founder. Does the group rely on one person to make all of the decisions? Are people afraid to change code the founder wrote? Has the company seen a lot of turnover among the engineering leader’s direct reports? Ask your interviewers how often the company’s coding conventions change. Ask them how much code in the code base has never been rewritten. Ask them what the process is for proposing a change to the technology stack. I have a friend who worked at a growing company where nobody was allowed to introduce coding conventions or libraries that the founding VP of Engineering didn’t understand, even though he hardly wrote any code any more.
Unwillingness to confront technical debt. Do you want to walk into a situation where the team struggles to make progress because they’re coding around all of the hacks they haven’t had time to address? Worse, does the team see you as the person who’s going to clean up all of the messes they’ve been leaving behind? You need to find out whether the team cares about building a sustainable code base. Ask the team how they manage their backlog of bugs. Ask them to tell you about something they’d love to automate if they had time. Is it something that any sensible person would have automated years ago? That’s a bad sign.
Not invented this week syndrome. We talk a lot about “not invented here” syndrome and how it affects the competitiveness of companies. I also worry about companies that lurch from one new technology to the next. Teams should make deliberate decisions about their stack, with an eye on the long term. More importantly, any such decisions should be made in a collaborative fashion, with both developer productivity and operability in mind. Finding out about this is easy. Everybody loves to talk about the latest thing they’re working with.
Disinterest in sustaining a Just Culture. What’s Just Culture? This post by my colleague John Allspaw on blameless post mortems describes it pretty well. Maybe you want to work at a company where people get fired on the spot for screwing up, or yelled at when things go wrong, but I don’t. How do you find out whether a company is like that? Ask about recent outages and gauge whether the person you ask is willing to talk about them openly. Do the people you talk to seem ashamed of their mistakes?
Monoculture. Diversity counts. Gender diversity is really important, but it’s not the only kind of diversity that matters. There’s ethnic diversity, there’s age diversity, and there’s simply the matter of people acting differently, or dressing differently. How homogenous is the group you’ve met? Do they all remind you of you? That’s almost certainly a serious danger sign. You may think it sounds like fun to work with a group of people who you’d happily have as roommates, but monocultures do a great job of masking other types of dysfunction.
Lack of a service-oriented mindset. The biggest professional mistakes I ever made were the result of failing to see that my job was ultimately to serve other people. I was obsessed with building what I thought was great software, and failed to see that what I should have been doing was paying attention to what other people needed from me in order to succeed in their jobs. You can almost never fail when you look for opportunities to be of service and avail yourself of them. Be on the lookout for companies where people get ahead by looking out for themselves. Don’t take those jobs.
There are a lot of ways that a team’s culture can be screwed up, but those are my top seven.
July 29, 2013 at 12:50 pm
I had a similar experience reading the same book, I guess bureaucracy and incompetence were not that different back in the day. I remember I found the Marlow worship within the company, and his relationship with the natives very familiar. Thanks for the reminder, it’s a great book I should definitely re-read sometime 🙂
July 29, 2013 at 1:42 pm
I posted this on /r/programming: http://www.reddit.com/r/programming/comments/1j9vro/seven_signs_of_dysfunctional_engineering_teams/
You may want to harden your WordPress.;-)
July 29, 2013 at 8:32 pm
Some of these are really good (I rarely see people identify Monoculture as a warning sign, which it definitely is), but “Preference for process over tools” seems backwards.
You say: “Healthy teams generally try to address their scaling problems with tools, not additional process. Processes are hard to turn into habits, hard to teach to new team members, and often evolve too slowly to keep pace with changing circumstances.”
This is the exact opposite of my experience. Tools that affect behavior are themselves a kind of process, and a clumsy one at that. Processes can be as simple as consisting of the one important thing you want done — “Do it this way” — whereas tools change an entire method of working just to get the one thing you want.
Circumstances change, and new people are hired, but if circumstances are changing on you so quickly that you can’t teach new people what you want done, you have much bigger problems that need addressing. Requiring a bunch of tools to enforce your desired processes is going to be even harder for your employees to figure out.
July 29, 2013 at 10:39 pm
I’m probably going to write a post just on process versus tools, because I think it’s an interesting topic.
July 31, 2013 at 9:30 pm
The challenge with talking about process is there’s a big difference between process done well vs. process done poorly, and in many situations the process is done poorly.
For example, if the process is brittle or does not adapt easily, that’s bad process design.
If the focus is on process enforcement rather than process improvement, that’s bad process design.
I really like the Toyota “Meals per hour” video that came out a few months ago (http://www.youtube.com/watch?v=EedMmMedj3M) because it shows the usefulness of process and process improvement done well.
August 1, 2013 at 3:57 pm
“Blamelessness” can be taken to a point where it is an obstacle to addressing quality deficits, even where they are clearly defined. People should be at least a little scared to break the build.
The “service-oriented mindset” can also lead to quality problems when it tips over into excessive deference to the customer’s notion of exactly what the problem is, or into a willingness to cut corners to deliver something fast. If you deliver with bugs, you haven’t delivered yet; you’ve only given the customer a not-quite-working prototype.
August 1, 2013 at 5:31 pm
Even in a culture of blamelessness, nobody wants to break the build, believe me. You’re still keeping dozens of people from getting their work done and potentially thousands of people from using a popular Web site.
August 3, 2013 at 7:50 am
Rafe, I’m interested to hear more of your thoughts on monoculture. Recently I worked on a project with multiple sites, and one of our toughest problems was the vast cultural differences between the sites. As a trivial example, one of the sites was militant about test automation and another site barely paid lip service to it.
So it seems like there is some happy medium between “multiple, incompatible cultures” and “monoculture”. I would be interested to hear your thoughts on where cultural homogeneity is helpful and where it is harmful.
August 5, 2013 at 7:30 pm
Here’s another voice that really questions “Process over tools”.
“Processes…evolve too slowly…”?? Whereas tools get embedded permanently (which combines easily with Not Invented Here) or teams can flit from tool to tool always looking for the next technological fix (Not Invented This Week). Though you mention asking about how many ‘standing meetings’ they have–so we’re talking about ‘too much process’? Definitely need some clarification around this point. Tools are also hard to turn into habits, if not impossible. “It took me a couple weeks to always remember to git.”?? It’s great to have the perfect tools for source control, bug database, wiki, build and test automation, but someone’ll need to use them. I guess one way to deconstruct this is to think about which is a worse company/team to work for: one with clumsy tools but robust process for using them, or one with great tools but processes that are clumsy? And there’s even a matrix when you compare the reality of the current tools n processes, against the current overall company/team focus on tools or process. E.g. you have new management which may want to focus on process because they think the ‘inherited’ tools are great especially if everyone would use them.
I can change the wiki for a lot of internal processes; I can’t make the source control behave differently or switch to a new one without a lot of changes to dependent automation.
August 12, 2013 at 10:42 am
All those who question or disagree with the observations in this article probably work in dysfunctional teams and just don’t know it, or will admit it yet 😉