It would be nice to know that for $10 billion invested I’ll get a medicine that will do exactly… But it doesn’t work that way. In medical research, what is purchased is specific processes and people. The results are variable.

Building software has far more in common with this model than most of us are willing to admit.

It’s not time, it’s money. Here, I have this nice used car for you. It’s really cool. But I won’t tell you the price before you sign the contract. You don’t know how much will you pay. Would you buy it?

Estimation is not the right word. It’s quoting a price, that’s the right word.

It does not matter that it’s not possible to do it accurately, we must do something.

You are right to call estimation for what it is, and there are characteristics of software projects that make prediction difficult. In most cases, this is not the fault of the estimation practice though at most companies I find the practice to be at about the level of reading goat entrails, only messier.

The problem is what people expect estimation to *do*. Specifically, they would like to to predict the future irrespective of any present uncertainty. In project estimation the formula is:
garbage-in–>garbage-to-the-power-”n”-out,
since general uncertainty and variability on input is often used as an exponent in a calculation. Now if I could figure out how to predict the future, I probably wouldn’t use it on something as mundane as project effort–I’d go after the lottery or the stock market.

Speaking of which, there are many similarities between performance of financial investments and software projects, and perhaps this is the paradigm we should look to. Financial analysis does not predict the future in the equities market, but if it is done well it is a useful business tool and can help guide decisions or at least avoid certain stupid ones (eg., had the market applied even basic common sense to P/E ratios and trends, we wouldn’t have had the Internet bubble. Um, ditto housing bubble).

The role of estimation is not to predict the future, it is to provide valuable management information that helps make business decisions.

The value of an estimate is determined by (a) the lead time of the estimate (ie. how early the estimate is produced wrt the capital commitment and development activity and (b) the value of the estimate in terms of making a “good” decision that has a positive value result or avoiding a “bad” decision that has a negative result. The cost of an estimate is largely a function of the time and effort put into creating it. When the time and effort expended on an estimate creates management information that allows the most optimal decision to be made, the process is working. IN fact, some of the most valuable estimates I’ve done have clearly shown we *should not* do something which we then we didn’t do and so saved scads of money that might otherwise have been thrown away.

Interestingly, this paradigm has a very close relationship to the calculation of the value of options in equity markets for much the same reasons. The Black-Scholes option pricing model holds out some hope for calculating, if not the value of a project, at least the value of an estimate.
BTW, we don’t need an estimate to be accurate at all–we need the *decision* made that is based upon the estimation to have a risk-weighted positive return. Not the same thing.

There are some other methods (other than Monte Carlo that is) upcoming. One problem with Monte Carlos is that it can be very time consuming to play “what if” with a complicated project plan. Although Monte Carlo simulations are a trusted means to the end is you report back dates like, “March 15, plus or minus 3 weeks” or something like that.

One of the major problems is that there is a fundamental lack of understanding of probability and likelihood among the consumers of our estimates. What we need is a tool that is transparent and easily understandable as giving a forecast (like the weather) rather than a prognostication (like Carnac the Magnificent). I think that when the business owner understands that the building of software is fundamentally a creative process and that there is significant uncertainty as to the size of a project at its initiation will we be allowed to express and manage that uncertainty throughout the life-cycle of the project.

As Hector and Gary both point out, management of uncertainty (Risk) is critical to project success.

(Shameless Plug Department: The company that I work for is developing a tool for managing software development projects – http://liquidplanner.com)