You don't have to guess to estimate

Hello– once again.

In theory this is a weekly newsletter – in practice I got kind of wrapped up in the Elixir project – which still isn't quite ready to be shown off, but we're getting there.

In the meantime – here's a quick note on estimates and how at least one other industry makes them.

A few months ago, Odd Lots did an episode about how retail companies select sites for storefronts. One bit in particular:

Chris (47:53):
In regards to traffic, I've probably spent more time in my life behind a windshield sitting at a corner, counting cars, than I would like to admit.

Joe (48:01):
Say more.

Chris (48:02):
It's pretty easy to find where the strong a.m. traffic pattern is versus p.m. You sit at an intersection and you watch traffic and you do that from like 8 to 10 a.m. or you do that at 4 to 5 p.m. and then the inverse logic is there. So if it's heavier at 4 or 5 p.m., you know that the other side of the road is heavier in the morning. That's the easiest way to find – at any coffee site — is to sit at an intersection and count cars if you're unfamiliar with that trade area.

There's a lot of numbers like that in the episode – this rule-of-thumb-based data collection. The folks who do this work have some heuristic models – how far away people will drive from to a particular kind of site, or what other kinds of stores their customers are interested in – and they go out and they take some samples and they do some math and they come back with an estimate for the volume they expect to see at a proposed site.

This, uh, isn't really how we do things in software. The method I've seen most often is, "Ask an engineer how long it will take." Maybe it's in time, maybe it's in "t-shirt size," maybe you put some effort into breaking the tasks down first and then add the estimates back up together, but it's still basically engineers making guesses and then someone adding them together.

I've always been vaguely bothered by this, but until now I haven't understood why. There are roughly three senses of "estimate." One is "a prediction of how much something will cost." One is "a guess."

But another definition is a rough calculation.

You have a model. You take some kind of proxy measurement. You make a calculation based on the model. Now that's an estimate.

This probably explains why my favorite estimate method here is story counting. Break stories down into manageable-feeling pieces, have the team work on them for a few weeks, and then average stories-per-week. Then write some more stories, and– tada! A real rule-of-thumb calculation – no guesses required.

(If you have a desperate need for estimates, incidentally, I recommend George Dinwiddie's book on the topic – Software Estimation Without Guessing. One thing I found particularly helpful about it is that it covers the use cases for estimates, and the different methods appropriate for different use cases.)

Speaking of napkin math – a friend of mine recently calculated that it had cost a client about $60,000 for him to develop the capability to deliver estimates for a particular project. This sounds about right to me as a ballpark figure – you should probably only try to use estimates if the estimate would be worth at least that much.

Anyway– trying to keep this short. Code to write and sushi to eat.

More soon.

- Nat