Why Development Projects Go Wrong #4 – Estimates That Go Horribly Wrong

The software development industry has a chronic estimation problem. More specifically, we have a chronic underestimation problem.

It's common practice for IT managers, when they get an estimate from a software developer, to double it or even triple it before passing it up to their superiors. And even then, the industry is known for project overruns and unexpected delays. Sometimes these delays undermine the whole business case for the project in the first place.

Why are we so bad at estimating?

The answers to that question are numerous and complicated. For an in-dept exploration of the issues I recommend Software Estimation: Demystifying the Black Art, but for the key insights, keep reading.

The thing that's unique about software verses, say, cars or furniture or buildings, is that the cost of a piece of software is 0% raw materials, 0% manufacturing and 100% design. Writing code is a design process, not a manufacturing process. And while you may think that the design of a software system is done by a designer or business analyst before the developers get started, you'd be wrong. Writing code and testing it is actually much more a process of discovery than most people realise. There are many unknowns and it's only when the project is over and the system is working that the developers finally know how best to build that exact application. Of course, by then it's too late.

Writing software is, almost by definition, building something that nobody else has ever built before. Any code that's had to be written a dozen times in the past has probably been turned into a product or released as open source already. So there's no need to write it again.

The only time you really do need a developer is if you have a requirement that's totally unique and not already satisfied by something that someone else has already written. And it's that uniqueness that makes it very hard to predict how long it will take.

Also, software developers, being people who've chosen to pursue a career in creating something out of nothing, are also usually much too optimistic, and often forget to allow time for the unknown unknowns.

But there are a few basic approaches that help:

1. Involve lots of people in the estimation process - this increases the chances that someone involved in the process has done something like this before.
2. Break the solution down into a number of tasks that you have done before and then estimate each one.
3. Measure what you can from previous projects. For example, how long does testing usually take as a proportion of the project? Is it 40%? 60%? Apply that same rule to the next project.
4. Count whatever you can about the new project and multiply estimates by the number of times that element occurs. For example, if you know you need to create 15 screens in the new system, estimate the coding time for 1 screen and multiply it by 15.
5. Reduce the amount of new stuff you need to invent - reuse as many products and open source tools as possible. Then try to meet the remaining requirements using combinations of commonly used design elements that you have built before.

Needless to say, at D4 we use all of these techniques and unlike the rest of the industry, we're usually right when we estimate how long things will take.