Get to the point

Dave Christiansen over at Information Technology Dark Side has a good graphic representing what he calls the FSOP Cycle (Flying by the Seat of Your Pants). The basic idea is that when smart people do good things, someone will try to reproduce their success by doing the same thing that worked the first time.

The problem with trying to do this is that every time someone tries to document a "successful process", they always leave off the first step: get smart people working on the project.

Dave outlines several reasons why capital-P Process will never solve some problems. Joel Spolsky described this same issue in his Hitting the High Notes article when he wrote, "Five Antonio Salieris won't produce Mozart's Requiem. Ever. Not if they work for 100 years."

So if Process can't solve your problem, what will? According to Dave, it's simple:

Put a smart PERSON in the driver's seat, and let them find the way from where you are to where you want to be. It's the only way to get there, because process will never get you there on its own.

I'll assume that when Dave says "smart" he really means "someone good at solving the current problem". I could have the world's smartest accountant and I wouldn't want him to remove my appendix. Okay, I don't want anyone to remove my appendix. But if I needed it done, I'd probably go find a doctor to do it. So what Dave is saying is that you'd rather have someone who's good at solving your current type of problem, than have Joe Random Guy trying to follow some checklist.

This isn't a complete answer, though. Some songs don't have any high notes.

In the middle of describing why you should choose the most powerful programming language, Paul Graham writes:

But plenty of projects are not demanding at all. Most programming probably consists of writing little glue programs, and for little glue programs you can use any language that you're already familiar with and that has good libraries for whatever you need to do.

Trevor Blackwell suggests that while that may be changing, it was still true at least until recently:

Before the rise of the Web, I think only a very small minority of software contained complex algorithms: sorting is about as complex as it got.

If it's true that most programming doesn't require the most powerful language, it seems fair to say most programming doesn't require the best programmers, either.

You might notice at this point (as I just did) that Dave wasn't talking about programming, or at least not only programming. The same principle seems to hold, though: Average people can do the most common things with the most common tools. Exceptional circumstances require exceptional tools and/or exceptional people. If only there were a way to predict when there will be exceptions …

The other problem

But let's say we're looking at genuinely exceptional people who have done great work. Should we ask them how they did it? After all, they're the experts.

Well, not really. They're only experts on what they did, not why it worked. They might have simply guessed right. Even if they didn't think they were guessing.

Need another example of false authority? Have you ever heard someone describe a car accident, and attribute their survival to not wearing a seatbelt?

First, they don't know that. They believe it. They obviously didn't do a controlled experiment where the only difference was the seatbelt. Second, even if they happen to be right in this case, statistics show that it’s much more common for the seatbelt to save you than harm you.

So if you want to design a repeatable process for creating software, you can't do it by asking people who are good at creating software.

The other other problem

One change I'd make in Dave's FSOP diagram is in the circle labeled "Process Becomes Painful". It's not that the Process changes. Really what's happening is that the project runs into some problem that isn't addressed by the current Process.

Every practice you can name was originally designed to solve a specific problem. On very large projects, there's the potential to encounter lots of problems, so extensive Process can simultaneously prevent many of those problems from appearing.

But attempting to prevent every conceivable problem actually causes the problem of too much time spent on the Process instead of the project.

That's where Agile comes in. It solves the problem of too much process. Do you currently suffer from too much process? Then you could incorporate some ideas from Agile.

But make a distinction between using ideas to deal with specific problems, and adopting a whole big-M "Methodology". Once you adopt a Methodology designed to solve the problem of too much Process, you face the danger of having too little Process.

Your expert programmers may be the best in the world at the problem you hired them for. But now you want them to do something different. And you have no way to recognize that they're not making progress, because you've eliminated all the governance that came with the heavyweight Methodology.

So what's the point, anyway?

Process is not something you can measure on a linear scale, having "more process" or "less process". Always adding practices -- even "best" practices -- to your current Process is simply trying to solve all possible problems, whether you're currently having them or not.

For each practice you have to consider what it was designed to solve, what was the point of it to begin with? Then don't treat these practices as a checklist that must be completed every time, but follow the principles behind them.

There's a line in the movie "Dogma" that I think describes this really well. Just substitute "process" for "idea" and "Methodology" for "belief":

You can change an idea. Changing a belief is trickier. Life should be malleable and progressive; working from idea to idea permits that. Beliefs anchor you to certain points and limit growth; new ideas can't generate. Life becomes stagnant.