I just read a job listing for a couple of Perl programmers, and in addition the job description and the basic requirements of the job, it included two homework assignments. One was to explain why you’d use a certain programming construct in Perl, and another to explain how you’d solve a big refactoring problem that the employer must need to solve.
There are two objectives for any interview: to determine whether the person is qualified for the position in question and to determine whether the person is a good fit for the organization. The second is the easiest for me. After interviewing someone, I almost always know whether I’d enjoy working with them and whether they have the enthusiasm requisite for the job. The first is more difficult. I’ve tried a number of approaches in that regard, and I’ve not been satisfied completely with any of them.
The one I rejected first is the “trivia question” interview. I’ve been on the receiving end of this one more often than I’ve inflicted it on others. Basically you ask obscure questions with very specific answers and see if the person knows them. This is not useful for determining whether someone can develop software. I’ve also tried asking essay-type questions, like, “Explain how branches are used in the version control system of your choice.” I never feel entirely comfortable testing people that way, though. I think it’s perfectly valid and useful, but I don’t enjoy it.
Lately I’ve tried to go with what I’d call the interrogative questioning style, reviewing the person’s work experience with them. I ask them to describe projects they’ve worked on, digging deeper and deeper into the details to make sure that they really did what they claimed and figuring out whether they understand the things they claim to. I think this approach is very solid and I would encourage others to adopt it.
Another common approach (that I’ve been subjected to but haven’t used) is to ask problem solving questions that the interviewee must solve in real time. Long ago I interviewed with amazon.com and the interviewer asked questions in that vein. I think it’s effective but it’s very stressful, particularly when the interview is a phone interview. You have to take time to think about the answers, and it’s not pleasant to just sit there and think when you’re on the phone with someone.
The homework assignment is a form of the “problem solving” approach and augments the interrogative approach (or any other approach) very well. It lets the person solve the problems on their own time with all of the tools available to them that they normally have. (Asking someone to solve a problem while sitting in front of strangers and without the use of Google, books, or even having their hands on a keyboard is artificial and stressful.) By giving the assignment, the depth of their answer not only tells you something about their problem solving skills, but also about their degree of interest in the position. If they show up with a poor answer, either they aren’t qualified or they don’t care enough to deserve the job.
I’m eager for the opportunity to try out this approach.
August 18, 2006 at 2:04 pm
We used the homework question once or twice at a previous company, and it worked very well. We tried to leave the problem open-ended enough to allow the candidate to architect an entire solution. We also told them that complete implementation was not necessary, as long as they could describe the parts that they didn’t get to, why they would do it that way, etc.
The biggest challenge was making the homework assignment big/challenging enough to get a good sense of the candidate but not make it too long. We set a target of four hours.
August 18, 2006 at 3:37 pm
“Homework” can be an effective screening technique, filtering out people who would otherwise take up a team’s time before washing out. It also signals to candidates that you’re serious, and gives them some idea of what style of work they’d be doing (which lets some candidates select themselves out). When I was last in a hiring role, we using homework to pre-screen. The candidates who did well were brought in for a bit of hands-on, collaborative problem solving (with an emphasis on collaborative–we were looking for people who could work well with others), and some pair programming on a real problem. We made some very good hires that way.
August 19, 2006 at 5:53 pm
I forgot to mention that when we used homework for candidates, we had the candidate present his solution to the team. Seeing him present was another good indicator of his suitability to the team.
August 21, 2006 at 11:18 am
we do something like that for editors — give them a dense technical paper with a range of error types, to see how careful they are in real time. (it also weeds out those who take one look at the material and freak out.) many people think they’re “good with language” but aren’t, essentially, anal enough, and I presume that there are plenty of “good programmers” with no creativity or follow-through.
spiff.