The Myth of Software Estimation

I cringe ever time I see or hear a mention of software estimation. After years of seeing my estimates and the estimates of my colleagues fail, I’ve come to the conclusion that beyond the roughest of figures software estimation is impossible. This is not due to negligence or incompetence. Nor is it due to an immaturity of the software field. It is simply a fact of software development that software estimation is a myth.

The Right Metaphor
To defeat the estimation myth, we must first discard the faulty metaphors assigned to software: construction, writing, engineering. In reality, developing software is more akin to a developing human relationship.

When I was in grad school I began to notice that from time to time a particular young woman would join my table at the cafeteria for lunch. I didn’t think much about it until one day I looked up at her and realized that perhaps the reason she was sitting at my table was because she liked me.

We geeks aren’t known for our social intuition so this was a big moment.  So I did some rough calculations on the speed and complexity of relationship building and making proper adjustments for personality factors I estimated we’d be married in 18 months . . . not!

small_wedding.jpgA marriage “go live” date was the last thing on my mind. In fact it was months and several relational millstones later before we even discussed a date. As it turned out, circumstances beyond our control delayed even that date. But, in the end the pieces fell together and we were happily married. Now we have a house, a mortgage and two small children.

If we substitute “software project” for “young woman” and “maintenance and support” for “mortgage and family responsibilities” we would have the perfect description of the software development project.

Relationships Are Unpredictable
Why can’t we predict relationships? It is certainly is not due to insufficient study. People have strived to understand how love works for thousands of years. Yet, at best matchmaking has always been an art of intuition and guesswork. There are simply too many variables and too many unknowns to predict how two humans will relate. The same is true of software. Each new project is built in unknown territory. Even a simple project has a myriad of variables and unknown factors. It is simply impossible to predict the result.

An Example
A few months ago a web service I’d written began to eat up memory like crazy. I’d coded it using a modern garbage controlled language so theoretically a memory leak was impossible. But, it was happening. I tracked down every obvious choice. I talked to my co-workers. No luck. Finally, after two weeks I finally discovered the problem–a small bug inside a loop was interacting with Microsoft Active Directory to create hundreds of AD tokens. Another bug caused these tokens to skip garbage collection. Schedule? Throw that to the wind.

Every software developer has stories like this one. You probably have one from just last month, or even last week! The interaction between human and computer is simply too complex to predict when the next brick wall will come sailing out of nowhere. I like to tell junior developers that the art of developing software is like running hurdles—except that instead of hurdles we run at brick walls and in the middle of the night. When we hit one we scramble up, dig beneath, or bash our way through then sprint headlong into the next wall.

Large Project Estimation
Individual estimation is impossible. However, if enough developers are involved the random occurrence of schedule breaking events averages out and becomes a measurable “white noise”. The “white noise” can be then factored into the estimate as overhead. Estimation models such as COCOMO are built on this predictability.

However, there are clear limitations to this type of model.

  • A limited number of critical path options may narrow a project schedule to a tiny set of dependencies. Once a schedule is dependent on a very small group (or individual) developer(s) the project schedule can (and will!) be blown about by random chance.
  • If a project depends on unproven technologies or technology unproven to the domain the number of random roadblock events increases unpredictably. A key assumption of large project estimation is that the average number of schedule delaying events per developer is known and can be factored in as an overhead. With new technologies this can not be known and can not be accurately factored into the estimation.
  • Range of Precision: Models are appealing because they provide a number—however, even in ideal conditions they are far from precise.
  • In short, I would be extremely cautions to schedule or budget based solely on the results of a software estimation model. However, these models are useful and should be carefully considered in the light of the limitations above. One of the best uses may in fact be as an antidote to unrealistic optimism–I.e. “No, this isn’t a 6 month project. It really WILL take *THREE FULL YEARS!*”

    Summary
    We (and our users) must accept that a software estimate is about as good as a prediction of marriage on the first date.

    Upcoming Article: How to keep your “customer’s” happy without estimates. (Coming in Early September)


    Further Reading:
    Large Limits to Software Estimation

    Advertisements

    37 Responses to “The Myth of Software Estimation”

    1. Jivlain Says:

      I think you mean “hurdles”, not “hurtles”.

      But still, good article.

    2. jeffspost Says:

      Yes, thanks!

    3. Keith B Says:

      Jeff,
      We have more choices that COCOMO. It is quite possible to have a team of developers estimate their own future work well enough, provided that they collaboratively estimate many small things within a short planning horizon, and that their estimates are given as a range.

      The “myth” of software estimation is not that it can be done at all, but that it can be done in a way that gives point-value predictions. Point-value predictions are what you need to fill in your MS Project plan. so that’s what a lot of project managers ask for. Unfortunately prediction is the work of charlatans, but estimation is the work of engineers.

    4. Micah Says:

      Great Post

    5. Harwell Says:

      Some good thoughts, and well expressed. I understand your frustration with the software estimation process, but the real world still requires some sort of estimate. I’ve found that the best way to get around some of the obstacles you mention is to 1. Keep the projects as small as possible by breaking big projects into small projects, 2. Do non-business-critical “proof of concept” projects first whenever you’re trying out a new technology or a new development process, 3. Avoid the temptation of adding more people to a late project (which makes it later), and 4. Negotiate requirements up front to get a real understanding of which things are absolutely critical and which things are just nice to have, then focus on the absolutely critical requirements and add the nice to haves only if you have time. See http://www.makingitclear.com/newsletters/subjectindex.html#projects for some of the articles I’ve written on the subject of projects and project management.

    6. Paul W Homer Says:

      When I started programming I felt as you did, that estimates were impossible. My logic was predicated on the fact that I had never done any of the work before, so it was not possible for me to ever know how long it will take. Seemed reasonable to me.

      Some years ago, my fate entirely depended on my ability to quickly estimate the work for complex projects, plus or minus a couple of weeks. Each time I failed, that was a financial hit to a very small company, and I depended on the company for my livelihood. It turns out, that if the stakes are high enough, you’ll actually find a way to get reasonable estimates. It is not as hard as one would image, it just takes a bit of analysis and deep thinking. If you consider it to be just another programming problem — this time in wetware — then it can be quite entertaining too.

    7. Hasith Says:

      I have a different view point. Trying to produce a to the point exact estimation is the principle mistake many does. There is no way you can derive that as the project cost itself has a probabilistic function.

      At any phase of a project certain amt of uncertainty is involved and that should be understood by the customer and the developers. It is a question of correctly identifying how certain you are about your estimation with available information at a given moment of time.

      Not understanding this fact is where we run in to problems. We all try to derive an accurate estimation that itself is a dream until you complete the project. Focus on producing a ‘good to go’ estimation and understand the uncertainties involved. Write down all assumption you took in deriving the estimations. Take actions depending on the uncertainties you see.

    8. Rob Crowther Says:

      It doesn’t spoil your metaphor much, but there is a scientific study which has a 95% success rate in predicting if your marriage will last:

      http://www.hachettebookgroupusa.com/books/68/0316010669/chapter_excerpt24301.html

      This is a much higher success rate than most software estimates IMX 🙂

    9. smaquois Says:

      software estimations would be fine, if they were actually accepted as “estimations” rather than concrete expressions of an end date.

      programmer: we estimate the project will take 120 days.

      proj mgr: ok.

      fast forward 100 days…

      proj mgr: what! you’re not going to be done in 20 days?!?!! but you said….

      related, but somewhat off topic…

      how many times have you been pitched a project and then been told that it has to be done by some specific future date? and then required to come up with a “plan” that fits everything into that timeframe, regardless of complexity or resources?

    10. James Hofmann Says:

      Many strong estimation methods have been developed that, much like statistics, can be used well in the *right context.* It’s a topic far too big to be dismissed by one blog post.

      You might want to try looking at the Stutzke book “Estimating Software-Intensive Systems” (http://sw-estimation.com/) – it’s the most thorough treatment of estimation around, if a bit heavy for an introduction.

    11. Fadzlan Says:

      –“software estimations would be fine, if they were actually accepted as ‘estimations’ rather than concrete expressions of an end date.”

      Heh2. Agreed. Most of the time the problem its the different perceptions that causes the problem.

      Then again, those who pour money on the project also needs to know how much the project will drain them. Of course there are risk of overbudgets (think underestimates….), but then again a lot of other businesses has the risk of overbudgets as well. Its the fact they have to embrace to be in the business.

      At least they can tell, in an ideal conditions how much the project will cost. That probably let them to mentally prepare what is the worst budget hit they can take before pulling the plug.

      And of course, we all know that most software projects are hardly ideals anyway. 🙂

    12. Laura Says:

      Thanks, nice post.

      I favor the maximum likelihood estimate model (p+4a+o)/6, where p=pessimistic scenario, o=optimistic scenario and a=likely scenario. Gives you a ballpark idea, has worked for us.

    13. Bruce P. Henry Says:

      Keith B (comment above) correctly notes that the primary problem is that project managers need a single number to plug into MS Project (or any number of similar tools). That coupled with Saquois’ observation (comment above) that these single-point guesses are treated as promises pretty much make a person want to give up on estimation altogether.

      If you always give estimates in ranges (e.g. 4-6 person weeks, 1-3 days, 6-9 staff months, etc) it is quite clear that it is an estimate, not a prediction. Giving a range opens up the discussion of what could happen (risks) that would push the effort towards the worst case end of the estimate or bring it in towards the best case end.

      Another nice thing about ranged estimates is that when you are asked to estimate something with poorly defined requirements you should give wide ranges to communicate the uncertainty. Well defined, believable requirements get estimates like 5-7 weeks whereas poorly defined, nebulous requirements get estimates like 4-20 weeks.

      There’s pretty much no way to get around doing estimates. Your business needs some idea of the investment required to complete a project in order to make trade-offs. Also folks in other departments (e.g. marketing, operations, manufacturing,…) need some idea of when they are likely to take delivery of the software. A team that can accurately (not precisely) estimate these things gives their company a real competitive advantage.

      So while it IS frustrating, there are ways to do it and retain your sanity. 🙂

    14. Gary Wollman Says:

      Laura’s post uses a weighted average estimation, to produce an end point giving 66% weight to the likely scenario and equal weight to the optimistic and pessimistic scenarios.

      Bruce wants to answer with a range and not be pinned down to a date. The comment about writing down all your assumptions is great.

      The purpose of estimating is to develop dates to use to measure the cost of development. One of the development costs is overhead, but a bigger cost and not mentioned is risk. Whenever we make an assumption we introduce risk. Then when we estimate using these assumptions we add risk to the estimation.

      Risk that we understand the requirements. Risk that you and the client assign the same meaning to each of the requirements and use cases. Risk that you understand how to use your estimation tool, etc. And lastly that the resources assigned to project are full time and committed to project until it is complete. This compounds the estimated risk.

      Project managers should use the level of risk to add lag time to task dependencies. This built-in lag time can absorb risks that become issues.

      Resources of time, tools and personnel come at a cost so a project plan can be assigned an overall monetary value which should include the cost of your expected risk. That is why if you beat the estimate and finish the project without utilizing the entire cost of the built-in risk, you should share the savings with the project team to provide an incentive to finish on or before schedule.
      .

    15. Tom Says:

      I’ll be waiting to read the next upcoming article 🙂

    16. Hector Says:

      I have a different point of view. The author is forgetting that exist something called Risk Management, that affect the estimation. You can estimate according the knowledge you have on your team skills, but then you must do a risk analysis that might affect your project.
      Some risks must be translated to the schedule, and some not.
      With the help of an estimation tools, like COCOMO or the one used by Laura, the knowledge of his team and a risk analysis, the estimation will be more close to the reality, and the budget too.
      Regards.

    17. Truao Tartamudo Says:

      Bruce P (following Keith B comments) may not be aware of a very simple and effective method for predicting delivery times statistically by using Monte Carlo simulations. The idea is pretty simple, break down your project in small tasks and use ranges for their delivery times (eg. 6-8 days) for best case *and* worst case scenarios, and combine both ranges into a single one. With estimated ranges for all tasks, run a Monte Carlo simulation and you’ll find what’s the most likely delivery time for your project, along with probabilities for delays.

      This method works fairly well if the estimates on best and worst case scenarios are fair, which is not that hard to achieve. Also, you can add random delays to each range during the simulation and see what happens to the delivery time. All in all, it’s a simple and efficient method to understand how your project will behave.

    18. Bruce P. Henry Says:

      @ Truao – I am aware of Monte Carlo simulations and they do solve some of the problems. The problem they don’t solve is that the person receiving the estimate taking the most likely delivery date as the promised delivery date. That can only be avoided by being very careful about how you negotiate the promised date. You are absolutely right about the estimates of best case and worst case needing to be good. Garbage-in-garbage-out still applies. 🙂

      There are some other methods (other than Monte Carlo that is) upcoming. One problem with Monte Carlos is that it can be very time consuming to play “what if” with a complicated project plan. Although Monte Carlo simulations are a trusted means to the end is you report back dates like, “March 15, plus or minus 3 weeks” or something like that.

      One of the major problems is that there is a fundamental lack of understanding of probability and likelihood among the consumers of our estimates. What we need is a tool that is transparent and easily understandable as giving a forecast (like the weather) rather than a prognostication (like Carnac the Magnificent). I think that when the business owner understands that the building of software is fundamentally a creative process and that there is significant uncertainty as to the size of a project at its initiation will we be allowed to express and manage that uncertainty throughout the life-cycle of the project.

      As Hector and Gary both point out, management of uncertainty (Risk) is critical to project success.

      (Shameless Plug Department: The company that I work for is developing a tool for managing software development projects – http://liquidplanner.com)

    19. Accurate Estimates « Outside of the Triangle Says:

      […] consider software estimation to be difficult, a black art or even, in Jeff Staddon’s case, impossible. I don’t agree and believe that if you follow the advice above you will find […]

    20. Matchmaking » The Myth of Software Estimation Says:

      […] NeoSemantics.com wrote an interesting post today onHere’s a quick excerptYet, at best matchmaking has always been an art of intuition and guesswork. There are simply too many variables and too many unknowns to predict how two humans will relate. The same is true of software. Each new project is built in … […]

    21. Phil Armour Says:

      The phrase “accurate estimate” is an oxymoron, like “British cuisine”.

      You are right to call estimation for what it is, and there are characteristics of software projects that make prediction difficult. In most cases, this is not the fault of the estimation practice though at most companies I find the practice to be at about the level of reading goat entrails, only messier.

      The problem is what people expect estimation to *do*. Specifically, they would like to to predict the future irrespective of any present uncertainty. In project estimation the formula is:
      garbage-in–>garbage-to-the-power-“n”-out,
      since general uncertainty and variability on input is often used as an exponent in a calculation. Now if I could figure out how to predict the future, I probably wouldn’t use it on something as mundane as project effort–I’d go after the lottery or the stock market.

      Speaking of which, there are many similarities between performance of financial investments and software projects, and perhaps this is the paradigm we should look to. Financial analysis does not predict the future in the equities market, but if it is done well it is a useful business tool and can help guide decisions or at least avoid certain stupid ones (eg., had the market applied even basic common sense to P/E ratios and trends, we wouldn’t have had the Internet bubble. Um, ditto housing bubble).

      The role of estimation is not to predict the future, it is to provide valuable management information that helps make business decisions.

      The value of an estimate is determined by (a) the lead time of the estimate (ie. how early the estimate is produced wrt the capital commitment and development activity and (b) the value of the estimate in terms of making a “good” decision that has a positive value result or avoiding a “bad” decision that has a negative result. The cost of an estimate is largely a function of the time and effort put into creating it. When the time and effort expended on an estimate creates management information that allows the most optimal decision to be made, the process is working. IN fact, some of the most valuable estimates I’ve done have clearly shown we *should not* do something which we then we didn’t do and so saved scads of money that might otherwise have been thrown away.

      Interestingly, this paradigm has a very close relationship to the calculation of the value of options in equity markets for much the same reasons. The Black-Scholes option pricing model holds out some hope for calculating, if not the value of a project, at least the value of an estimate.
      BTW, we don’t need an estimate to be accurate at all–we need the *decision* made that is based upon the estimation to have a risk-weighted positive return. Not the same thing.

    22. Larry Says:

      Jeff, Nice post and a lot of good commentary. Marriage is a metaphor for software projects that I hadn’t heard before. There’s a definite lack of consensus on scope in the early days of both. 😉 I also think the point about the point about the averaging of events is a good one.

    23. great deep thoughts Says:

      […] software development also unpredictable. Very deep and thought provoking with some great stories.https://jeffspost.wordpress.com/2007/08/26/the-myth-of-software-estimation/portland imc – 2005.07.01 – Deep Thoughts — by Great Republicans&quotThe American people should be […]

    24. Stodskneedogs Says:

      Just went through your pages. Are you playing with my sleek appraisal Good joke 🙂 What is Beethoven doing in his coffin right now? Decomposing.

    25. Miklos Hollender Says:

      Jeff, you just don’t get it.

      It’s not time, it’s money. Here, I have this nice used car for you. It’s really cool. But I won’t tell you the price before you sign the contract. You don’t know how much will you pay. Would you buy it?

      Estimation is not the right word. It’s quoting a price, that’s the right word.

      It does not matter that it’s not possible to do it accurately, we must do something.

    26. Jeff Staddon Says:

      The key issue is that software development is an R&D effort. One of the best parallels is medical research. Medical research costs a lot of $$. It’s been done for years and it’s a very advance field. But, they still can’t quote a price.

      It would be nice to know that for $10 billion invested I’ll get a medicine that will do exactly… But it doesn’t work that way. In medical research, what is purchased is specific processes and people. The results are variable.

      Building software has far more in common with this model than most of us are willing to admit.

    27. mythorfact Says:

      In my Experience, not theoretical, estimates are done by people who do not do the work, influenced by fear of appearing uninformed, mitigating the actual work to be done to sponsors so they will appear to be competent while those poor souls who have to live up to whatever some space cadet has imposed upon them are quietly having nervous breakdowns, the ones who actually care…woe to those who object or try some sanity for they will be forced out (after the job is done) and the insanity will continue as long as programmers/PMs/name your title are valued on their talk instead of the walk. Get a grip man, are you going to allow your wife to remodel your home without some idea of costs/risks/time vs your personal budget? Where the variables usually come in is each person’s own personal agenda how much attention they can garner in doing exactly what they want and it is usually avoidance of actually working together and valuing each other. It is a ME thing which is pervasive to keeping a job now whether one is a programmer or a politician.

    28. nonrydonEntex Says:

      Hi nto All
      My PC worked not correctly, too much mistakes and buggs. Please, help me to fix buggs on my computer.
      I used Windows7.
      Thx,
      nonrydonEntex

    29. jack Says:

      The most practical and accurate estimations depend on previous experience building a similar project using similar technologies. By similar, I mean almost identical.

      Unless you have a habit of always using the same programming language and building almost identical software every time, anyone who claims you can accurately estimate the time your project will take is wasting your time.

      That doesn’t mean software can’t be delivered on time – it often means re-scoping the project and always managing client expectations.

    30. reouuyte7 Says:

      Hack again?!

    31. Tweets that mention The Myth of Software Estimation « Jeff’s Post -- Topsy.com Says:

      […] This post was mentioned on Twitter by aidy lewis, Rupert Voleshafter. Rupert Voleshafter said: RT @aidy_lewis: Software Estimation? http://tinyurl.com/5tcafq3 […]

    32. PepmefSmamn Says:

      Guy .. Beautiful .. Superb .. I’ll bookmark your site and take the feeds additionallyI am happy to seek out so many useful info here in the submit, we want work out more strategies in this regard, thanks for sharing. . . . . .

    33. Buy Windows 7 Key Online Shop Says:

      Buy Windows 7 Key Online Shop…

      […]The Myth of Software Estimation « Jeff’s Post[…]…

    34. The Pirate Bay Proxy Says:

      They disassemble the software program package to its
      lowest form (assembly). Sakata studied marine biology before entering
      music, and has retained that interest by becoming a recognized expert about the Japanese water flea referred
      to as Mijinko. It can be better to possess a bootable disc that could diagnose drive and memory problems and
      supply the Windows recovery environment to restore your backup on the same or a new hard drive.

    35. pirate bay illegal Says:

      After sometime he becomes so associated with training and
      preparing for the competition that his wife, whose covetousness no longer has sufficient control, wants him to stop
      teaching and grow an accountant, to allow them to have more money but the
      teacher was so committed to his work he did his far better to
      develop his team. With the Job Search app, start that search
      anywhere. is the very first LNG ship construction shipyard, the success of five LNG ships built for the Hudong China
      has accumulated experience within this basis, Shanghai East China actively associated
      with the first six LNG vessels cum Shanghai LNG carrier construction project preparation, the current design
      optimization and production in the technical preparations are
      already completed.

    36. Знакомства Полтава Says:

      Hello there! This post could not be written any better!
      Reading through this post reminds me of my old room
      mate! He always kept chatting about this.
      I will forward this write-up to him. Pretty sure he will have a good read.
      Many thanks for sharing!

    37. froldplonry Says:

      Бесплатный сайт знакомств для секса.
      Знакомства онлайн с фото. Бесплатный порно видеочат.
      Частные фото и видео.

      http://www.bit.ly/1pm2xQj

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s


    %d bloggers like this: