Smashing the Myth of Predictable Software Development Part 2: Living with Unpredictability


In a previous post I argue that software development is inherently unpredictable.  I believe the best metaphor for software is a growing human relationship from first meeting to marriage.  Relationships like software grow along a predictable path–but at a random, unpredictable rate.  Thus they perfectly illustrate the nature of a software “project” with too many unknowns to allow for a predictable schedule.

Effectively creating software requires a process that can flex with this unpredictability and still produce useful results.

Fundamentally there are 3 levers that control software development.

1)       Risk

2)       Scope

3)       Schedule


To have any hope of consistently delivering software developers must have control over risk.  This includes things like: leaving room in the budget for beefier hardware, developing and testing key architectural components before investing too heavily in the rest of the code, identifying scalability bottlenecks and testing them with code fragments that simulate full system activity, etc.   

Unfortunately, risk is consistently underestimated and often costs are cut in this area with catastrophic results.  In particular the concept of risk can be hard to get across to non-technical users.  Many people outside of IT misinterpret open and honest discussions of risk as a lack of confidence.

Personally, I’ve found the best way to approach risk with non-technical people is to (over) explain the reason for discussing risk in detail—namely that is how we avoid problems to produce “SUCCESS”!  We programmers (and other IT people) take it for granted that risk management is yellow brick road that leads to Oz, but non-technical users do not have knowledge hard wired in.  As the technical expert we programmers are often expected to double as teachers for those without the benefit of our technical training.  For those still in school—analyze your favorite teachers and take some notes of their methods.  Teaching skills can be a real asset to a programming career.


Control over scope is rare.  There is the infamous “scope creep” that runs havoc, but even more insidious is hidden scope.  I can’t count how many times I’ve had a request that seemed fairly straight-forward.  Then somewhere between two days and two weeks later I learn some funky little quirk in the business process that totally trips up the design.  The requirements remain unchanged while the project balloons to 4 times its original size.   

A few software houses have managed to control scope.  Intuit makers of popular financial software releases new products on a yearly cycle.  There are some hard-fast scope requirements—tax law for example—but most of their scope is flexible.  Features that aren’t stable in time for the 2008 version end up in the 2009 edition.

Starting with Windows 95 Microsoft seemed to be flirting with that model, but their markets (or maybe the company itself) seem to prefer well defined improvements in scope over precise release schedules so they never have been able to deliver consistently timed releases.  Since 2000 they seem to be steadily drifting away from dated releases.  (Ironically this excluding IT oriented products–SQL Server and Visual Studio—we developers like our naming schemes simple and mathematical.  Just the word “Vista” is enough to make me physically ill. J)


While scope is somewhat flexible it is typically not flexible enough to account for all the unexpected roadblocks.  The only other alterative is to vary the schedule enough to allow for the work to be completed.  Management (particularly non-IT management) finds this unsettling.  Many of the software horror stories circulating today have their origin in management that is unwilling to sacrifice either scope or schedule and end up sacrificing staff instead. 

In order to work around the scope/schedule problem some have resorted to saying—“We can meet the schedule, but must cut quality.”  Personally, I see this as a cop out.  Quality is an attribute of scope.  A buggy or hard to use solution is an unfinished solution.  Phrasing scope in terms of “quality” might be easier for non-technical people to understand.  However, it gives the idea that quality is optional.  The results are rarely pretty.

Personally, my approach to schedule management is to keep a close relationship with my user/customer community updating them regularly on the status and challenges of the current project.  Initially this approach often results in a burst of initial scope when people realize that someone in IT is finally listening!  However, it’s not long before a balance emerges.  I’ve found that (most) customers will voluntarily cut/modify scope in order to achieve their top priorities within a reasonable schedule.

However, a note of warning is due here.  A key part of a good relationship with a customer is to emphasize the infrastructural investments that must be made.  The customer must “own” not only their features, but the entire system that supports those features.  Otherwise they won’t value/appreciate the huge investment that must be made.  Software is a lot like an ice-berg.  That 20% that is visible—the UI—is supported by 80% that is under the water.


Far too often programmers are left without control of scope or schedule and are forced to absorb the randomness of software development by working extra hours.  If the software industry recognized the true nature of software we’d abandon our project-centric approach and turn to a more dynamic model.  (I see signs of this in the current wave of web 2.0 companies)  One of evolving systems (or sets of interacting systems) developed over the long-term with an eye towards process and integration.

Additional Reading:In this article Joel Spolsky comments that the construction phase of software can be estimated with decent accuracy, but the design and bug fixing phases of software are unpredictable.  An interesting idea with a number of practical applications.


5 Responses to “Smashing the Myth of Predictable Software Development Part 2: Living with Unpredictability”

  1. Larry Says:

    Jeff, a quick comment on Joel’s point about the predictability of the different phases. I think the design phase in nearly all projects is more uncertain than it needs to be because it gets little attention. There’s a great book – building products in half the time – that talks about how to reel in the ‘fuzzy front end’. Regarding the predictability of construction and unpredictability of debugging – I think that’s mostly an artifact of people pushing risk to the end of the project to meet interim dates. More debugging & testing during development could reverse that trend, Larry

  2. DmC Pre Order Says:

    certainly like your web site however you have
    to test the spelling on quite a few of your posts.

    Many of them are rife with spelling issues and I find it very
    troublesome to inform the reality nevertheless I will certainly come again again.

  3. gain weight Says:

    gain weight

    Smashing the Myth of Predictable Software Development Part 2: Living with Unpredictability | Jeff\’s Post

  4. 2014 mixtapeRude lyrics video Says:

    Greetings I am so delighted I found your blog page, I really found you by error, while I was looking oon Askjeeve
    for something else, Regardless I am here now and would just like to say thank
    you for a fantasticc ost and a all round entertaining blog (I also love the theme/design), I don’t have time to
    read through itt all aat the minute but I have saved iit and also included your RSS feeds, so when I have time I will
    be back to reazd more, Please do keep up the superb job.

  5. healthy lifestyles Says:

    Feeding your pet dry dog food is a great way to strengthen their jaw muscles,
    along with satisfying their natural chewing instinct. Take this completely with
    the top with a garnish associated with thin carrot
    chips ALONG WITH plump raisins. Fish is loaded with a lot of essential vitamins and minerals.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: