How to estimate development effort without a crystal ball

“How long will it take to lift and shift our website from our current CMS to the new one we haven’t picked yet?”

“Will two developers be enough to get this app built in six weeks?”

“Our cloud migration can be complete by November, right?”

In my twenty-three years (and counting!) of building websites for clients, nothing has caused me to lose more sleep than worrying about the estimates I’ve made. The worry doesn’t come at the start of the project, either. At the start of the project, I’ve made the best estimate I could given what I knew at the time, and I feel happy and optimistic.

The anxiety comes about a third of the way through the project, when the difference between “what I knew at the start” and “what I could have known” is clearer. And I worry that what I could have known is maybe what I should have known. 

I dread having that first big reality check conversation: “Well, this part went better than expected, but this part over here turned out to be really tough, and so the date in the estimate doesn’t look achievable.” When that happens, I feel embarrassed, guilty, and ashamed. I don’t want you to feel that way!

The solution is not “pad the estimate for safety” or “don’t ever try to estimate anything that can’t be accomplished in one sprint.” The solution is to change your perspective on what is possible with estimation. You can’t control the future, so you can’t predict the future. What you can do is come to terms with unpredictability. You can treat estimation as a process that gradually brings the possible into focus.  

And if I don’t promise more than I can bring into focus, I don’t lose sleep.

Introducing the Cone of Uncertainty

If you’ve done something ten times before, you already have a very good idea of how long it’ll take you to do it the eleventh time—no estimation needed. As software developers, we’re estimating when there is uncertainty:

• What exactly will the software need to do, and which features will be challenging?
• When will designers be done and their work ready for handoff?
• Who will be assigned to the project, and how experienced are they with the code and each other?
• Will stakeholders respond quickly with positive feedback, or will there be multiple rounds of review?

Not only are those things not fully known, some of them are unknowable. Even the best, most experienced software engineer in the world can’t predict what velocity a new team will hit with requirements that aren’t fully baked yet.

Imagine a graph where the horizontal axis is the project lifecycle from inception at the left to completion at the right. And the vertical axis is the degree of error; either a project being faster, easier, or cheaper than expected (low), or harder, slower, or more expensive than expected (high).

Illustration by Sarah Kula Marketing Designer

Software development luminary Steve McConnell calls this  the “Cone of Uncertainty.” It’s not just theoretical; it’s a demonstration of the past performance of skilled, trained professional estimators. In other words, at initial concept, it’s demonstrably not feasible for a dedicated professional to do any better than four hundred percent variability.

At this point in the project, it’s not realistic to predict the future. The best you can do is get between “it might take twenty-five cents or four dollars.” And if you make promises at this point, you’re going to wake up sweating later on.

You can’t predict the future, so stop trying

As tempting as it is, the solution isn’t to estimate harder. The unknowns in your project aren’t just unknown; they’re unknowable at this early stage. Will the Okta integration with your CMS go smoothly? Will the new developers be onboarded and writing features after two days or two weeks? Will the CMO be delighted and say “ship it,” or will you spend the next sprint “hardening” your features with surprise edge cases?

The solution isn’t just to provide a wide range and expect your stakeholders will be okay with it. “Listen, I hope your project will cost fifty thousand dollars, but… you should know, it could be close to half a million.”

The bad news is that it’s impossible to predict the future. But the good news is that you really don’t need to. Don’t forget about the control you have to manage that future. And that giving yourself what you need to manage to an outcome you want is the real purpose of estimation.

Set sustainable development goals

Even if some variables are unknowable, some things will always be under your control. And those things are probably some of the biggest levers of success.  

• Do you have flexibility about managing scope?
• Will you be able to select your team once features are more defined?
• Is there flexibility in the launch date?

If you (or the lead who will be executing the work) has the ability to manage the project, then the goal of estimation isn’t to make a guess that will either be right or wrong in the future. The goal is to give yourself and your team the things they need to manage to a successful conclusion.

In his magnificent book, Software Estimation: Demystifying the Black Art, Steve McConnell sums it up:

“The primary purpose of software estimation is not to predict a project’s outcome; it is to determine whether a project’s targets are realistic enough to allow the project to be controlled to meet them.”

– Steve McConnell, in Software Estimation: Demystifying the Black Art

Estimate, plan, control, repeat

I no longer approach estimation trying to understand whether a project will be done in twelve weeks of development effort. I’m trying to understand whether a project could be done in twelve weeks of development effort—and whether my team will have control in the project to realistically manage towards that goal.

Sometimes, the result is “that’s not a realistic goal,” which is a successful outcome of estimation. In fact, it’s the only useful outcome in this early-stage estimation, because it allows the team to ask questions to find realistic solutions. What about the goal could be changed to make it realistic? What risky items could be set aside?

Two weeks into the project, I have a much better understanding of the uncertainties. Some risks have evaporated! Other risks have manifested into problems. Now it’s time to re-estimate: 

“Knowing what I know now, can this project be completed in ten weeks of development effort?”
“What can I control, manage, or change to manage the project in that ten weeks?”

Two weeks after that—about the time that I would have started waking up in a sweat—it’s time to gather learnings, further refine the unknowns, and make more decisions that reduce uncertainty. “Okay, I’m quite sure that we can get the SSO working in eight weeks as long as the flow is finalized by Thursday. Can we do that?”

Start your software estimation project with what you know, then re-estimate

In my own experience, if the first third of the project is spent refining, re-estimating, and identifying the areas where you have control—then the last third of the project is full of opportunities to go above and beyond.

Instead of spending the last weeks of the project hacking requirements to make the original target, you’ve made decisions early, worked on the most important items, and now you’re looking for quick wins and “surprise and delight.” 

Plus, you’re sleeping blissfully all the way through the night.  

Here at Think Company, our tech leads work together with design leads and project managers to identify and set realistic goals for our clients, then manage them to successful completion. Do you have a project you’d like to discuss? Chat with our experts!