Software estimation speed and accuracy

Question

Let's say you have a project that will involve two web applications (that will share DAL/DAO/BO assemblies and some OSS libraries):

• a semi complex management application that uses Windows Live ID for authentication and is also capable of communicating with various notifier services (email, sms, twitter etc.), targeted notifiers being about 10% of functionality
• a low to semi complex user application with less functionality but more robustness that also uses Windows Live ID for authentication

There are two of us with medium estimation capabilities and we won't be able to do it in two days even if we wanted/have to. At least it would be a far off estimate.

Questions

1. How long would/does it normally take you to make a reliable/valuable estimate?
2. What would you suggest to speed-up estimation without sacrificing accuracy?
3. How much slack (in terms of cost/time) would you add depending on estimation speed (when you would say: I could estimate it a bit more, because I think it's still quite off)

Answer 1

Since we use Agile methods (Scrum, specifically) it takes us about an hour longer than it takes the users to prioritize.

More time doesn't lead to more accuracy.

The hard part, then, is getting the users to prioritize. We hear this discussion all the time "if the whole thing isn't completed on time, it's all worthless." "Except for the XYZZY component, which does have some value." That argument can go on for hours until it's resolved that XYZZY should be first.

Generally, we try to create 4-week sprints. The first few are complicated because there's always something new. After the first two (or three) we seem to set a steady pace.

Each use case has a relatively simple, subjective valuation of how the effort it will take to finish it. Anything over one full sprint in duration has to be decomposed. Most times a few use cases are bundled into a single sprint.

The are formal ways of scoring each use case to better handle the cost and schedule issues. We don't use them because the extra effort doesn't help.

After the first two sprints,

1. There's new and different functionality,

2. The priorities have all changed,

3. The details of each use case have been dramatically revised.

What does "accuracy" mean when the thing you're trying to estimate changes at the end of each sprint?

---

One lesson learned. Parts of my company spend a long time fully defining exactly what will be delivered, and then measuring that they are delivering precisely what they want.

Customers notice this, and one said we "spend a lot of time delivering what the contract says, but it isn't what we needed."

The problem with firm up-front estimates is that they take on a life of their own. The more you "invest" in the estimating, the more the estimates seem to be a useful deliverable. They aren't useful because they're generally totally wrong. They're based on up-front assumptions that are totally wrong.

It's a bad policy to invest more time in estimating. The "accurate" answers aren't more accurate, but they are more treasured by every layer of management. As you and the customer learn, you invalidate numerous assumptions and you absolutely must re-estimate constantly.

Don't do it up front. If your contract requires you to do it up front, then make sure you have a change control provision and tell the customer that you absolutely will make changes as you go forward. As both you and the customer learn, you both must make changes.

Answer 2

Interesting question. I'm afraid the answer is "it really depends!" I know that's not terribly useful (although it IS true) so here are some factors:

1) Quality and completeness of requirements and their specification. This is, to me, most often the project-estimate killer. If you don't have quality requirements, you have no reasonable basis for an estimate. We use a "RUP-lite" style of product development here, so as the engineering manager I won't give anything but the coarsest-grain estimate until we've completed our "elaboration" phase and gotten sign-off from product management that the core 80% of the product features are in fact accurately covered.

2) The scope & nature of the product. Bigger/more expensive/more complicated = substantially longer to estimate. I've spent years working in telco-carrier land delivering solutions which have the normal robust carrier requirements ("5 9's" of uptime required by SLA mean you must really do a good job of solution design and failure recovery!). In that sort of environment with all the moving parts across functional areas of the business, the estimation of work is going to hinge on getting the whole picture...specifically, cross-functional dependencies and external dependencies can be a REAL killer here. That said, I've also built lots of shrink-wrapped and enterprise software, too. In those environments it's much easier as the scope is typically substantially smaller, so thus easier to estimate.

3) How "new" is this project? How "new" is the team to this product or technology set? The newer the product or team, the longer and more buffer you should allocate.

4) How specific do we need to be? If this is a "rough guess" then I'll lean on my engineering leads to provide a conservative estimate, then I'll pad that. If we need a "real" estimate (e.g., one which is used by my boss and which I'll be responsible for hitting), I'd need the input from a number of different managers and team members, who will need time to analyze the requirements and confer amongst themselves.

That can take as little as a couple days, or weeks..it all depends on the size. "Two or three days" is, frankly, not long enough for sizing anything but the most trivial of projects.

The best thing you can do to improve the quality of your estimates to to improve the quality of your requirements, and be ruthless in identifying hidden dependencies.

One final thing: FWIW, I've been doing this since '81 and I regard accurately estimating a project's duration/cost as the single most difficult/fraught with peril part of engineering management.