Archive for the ‘Uncategorized’ Category

Smashing the Myth of Predictable Software Development Part 2: Living with Unpredictability

September 25, 2007


In a previous post I argue that software development is inherently unpredictable.  I believe the best metaphor for software is a growing human relationship from first meeting to marriage.  Relationships like software grow along a predictable path–but at a random, unpredictable rate.  Thus they perfectly illustrate the nature of a software “project” with too many unknowns to allow for a predictable schedule.

Effectively creating software requires a process that can flex with this unpredictability and still produce useful results.

Fundamentally there are 3 levers that control software development.

1)       Risk

2)       Scope

3)       Schedule


To have any hope of consistently delivering software developers must have control over risk.  This includes things like: leaving room in the budget for beefier hardware, developing and testing key architectural components before investing too heavily in the rest of the code, identifying scalability bottlenecks and testing them with code fragments that simulate full system activity, etc.   

Unfortunately, risk is consistently underestimated and often costs are cut in this area with catastrophic results.  In particular the concept of risk can be hard to get across to non-technical users.  Many people outside of IT misinterpret open and honest discussions of risk as a lack of confidence.

Personally, I’ve found the best way to approach risk with non-technical people is to (over) explain the reason for discussing risk in detail—namely that is how we avoid problems to produce “SUCCESS”!  We programmers (and other IT people) take it for granted that risk management is yellow brick road that leads to Oz, but non-technical users do not have knowledge hard wired in.  As the technical expert we programmers are often expected to double as teachers for those without the benefit of our technical training.  For those still in school—analyze your favorite teachers and take some notes of their methods.  Teaching skills can be a real asset to a programming career.


Control over scope is rare.  There is the infamous “scope creep” that runs havoc, but even more insidious is hidden scope.  I can’t count how many times I’ve had a request that seemed fairly straight-forward.  Then somewhere between two days and two weeks later I learn some funky little quirk in the business process that totally trips up the design.  The requirements remain unchanged while the project balloons to 4 times its original size.   

A few software houses have managed to control scope.  Intuit makers of popular financial software releases new products on a yearly cycle.  There are some hard-fast scope requirements—tax law for example—but most of their scope is flexible.  Features that aren’t stable in time for the 2008 version end up in the 2009 edition.

Starting with Windows 95 Microsoft seemed to be flirting with that model, but their markets (or maybe the company itself) seem to prefer well defined improvements in scope over precise release schedules so they never have been able to deliver consistently timed releases.  Since 2000 they seem to be steadily drifting away from dated releases.  (Ironically this excluding IT oriented products–SQL Server and Visual Studio—we developers like our naming schemes simple and mathematical.  Just the word “Vista” is enough to make me physically ill. J)


While scope is somewhat flexible it is typically not flexible enough to account for all the unexpected roadblocks.  The only other alterative is to vary the schedule enough to allow for the work to be completed.  Management (particularly non-IT management) finds this unsettling.  Many of the software horror stories circulating today have their origin in management that is unwilling to sacrifice either scope or schedule and end up sacrificing staff instead. 

In order to work around the scope/schedule problem some have resorted to saying—“We can meet the schedule, but must cut quality.”  Personally, I see this as a cop out.  Quality is an attribute of scope.  A buggy or hard to use solution is an unfinished solution.  Phrasing scope in terms of “quality” might be easier for non-technical people to understand.  However, it gives the idea that quality is optional.  The results are rarely pretty.

Personally, my approach to schedule management is to keep a close relationship with my user/customer community updating them regularly on the status and challenges of the current project.  Initially this approach often results in a burst of initial scope when people realize that someone in IT is finally listening!  However, it’s not long before a balance emerges.  I’ve found that (most) customers will voluntarily cut/modify scope in order to achieve their top priorities within a reasonable schedule.

However, a note of warning is due here.  A key part of a good relationship with a customer is to emphasize the infrastructural investments that must be made.  The customer must “own” not only their features, but the entire system that supports those features.  Otherwise they won’t value/appreciate the huge investment that must be made.  Software is a lot like an ice-berg.  That 20% that is visible—the UI—is supported by 80% that is under the water.


Far too often programmers are left without control of scope or schedule and are forced to absorb the randomness of software development by working extra hours.  If the software industry recognized the true nature of software we’d abandon our project-centric approach and turn to a more dynamic model.  (I see signs of this in the current wave of web 2.0 companies)  One of evolving systems (or sets of interacting systems) developed over the long-term with an eye towards process and integration.

Additional Reading:In this article Joel Spolsky comments that the construction phase of software can be estimated with decent accuracy, but the design and bug fixing phases of software are unpredictable.  An interesting idea with a number of practical applications.


Change Without Risk?

September 9, 2007


“Change Without Risk” the slogan read.  This was on a poster advertising the latest Oracle DB release that came shrinkwrapped with the latest copy of Oracle Magazine.  “Change Without Risk”. . . what a laugh.  The very idea is ludicrous. Please, someone put a leash on those marketing guys!

Everything has risk.  Upgrading has risk.  Not upgrading can have even more risk.  Evaluating and managing risk is second nature to those of us in IT.  But, it’s easy to get into a rut and not think deeply about risk until some silly piece of marketing hype comes along.

There are four basic ways to handle risk:


The risks we keep.  Some of these are chosen—for example deciding to go with a backup tape vs. a hot offsite backup.  But most retained risks are not thought through and come as a surprise—particularly to non-technical people.  Unfortunately, a common “suit” strategy seems to be to “save” money by retaining more risk.  In most cases I think the “suits” simply don’t understand the extent and scope of the risks they are working with.


Find a way to reduce risk.  For example: backup the hard drive, provide a backup system, write maintainable code.  Ameliorating risk is usually the most effective way to manage risk.  However, it comes at a cost.  Usually the cost is relatively low, but one that non-technical decision makers may cut if they don’t understand the consequences.  I find when I tell real stores of real failures in similar situations people connect/understand risks a lot better and are more willing to take the necessary steps.


Chose a different solution that avoids risk altogether.  I often code the “hard” parts of my applications first so I can identify the areas of greatest risk when I still have time to re-design and avoid risky solutions.


This is mostly applicable in the insurance world, but I think there are some applications to programming.  For example, I can code a robust data access layer and share it across several servers so that if one server goes down the others can seamlessly pick up the load.


It’s impossible to live life without risk–but with proper precautions risk can be reduced to a point where it’s livable.  And to ya’ll over at Oracle:  Next time you make a poster—please advertise some real benefits–not some “Change Without Risk” hype.  I really would like a poster I can feel good about hanging up beside (or on top of) my Microsoft poster without embarrassment.  🙂


P.s. for those of you patiently waiting the follow up article to my previous post.  I’m working on it and should have it finished in the next few days.

The Myth of Software Estimation

August 26, 2007

I cringe ever time I see or hear a mention of software estimation. After years of seeing my estimates and the estimates of my colleagues fail, I’ve come to the conclusion that beyond the roughest of figures software estimation is impossible. This is not due to negligence or incompetence. Nor is it due to an immaturity of the software field. It is simply a fact of software development that software estimation is a myth.

The Right Metaphor
To defeat the estimation myth, we must first discard the faulty metaphors assigned to software: construction, writing, engineering. In reality, developing software is more akin to a developing human relationship.

When I was in grad school I began to notice that from time to time a particular young woman would join my table at the cafeteria for lunch. I didn’t think much about it until one day I looked up at her and realized that perhaps the reason she was sitting at my table was because she liked me.

We geeks aren’t known for our social intuition so this was a big moment.  So I did some rough calculations on the speed and complexity of relationship building and making proper adjustments for personality factors I estimated we’d be married in 18 months . . . not!

small_wedding.jpgA marriage “go live” date was the last thing on my mind. In fact it was months and several relational millstones later before we even discussed a date. As it turned out, circumstances beyond our control delayed even that date. But, in the end the pieces fell together and we were happily married. Now we have a house, a mortgage and two small children.

If we substitute “software project” for “young woman” and “maintenance and support” for “mortgage and family responsibilities” we would have the perfect description of the software development project.

Relationships Are Unpredictable
Why can’t we predict relationships? It is certainly is not due to insufficient study. People have strived to understand how love works for thousands of years. Yet, at best matchmaking has always been an art of intuition and guesswork. There are simply too many variables and too many unknowns to predict how two humans will relate. The same is true of software. Each new project is built in unknown territory. Even a simple project has a myriad of variables and unknown factors. It is simply impossible to predict the result.

An Example
A few months ago a web service I’d written began to eat up memory like crazy. I’d coded it using a modern garbage controlled language so theoretically a memory leak was impossible. But, it was happening. I tracked down every obvious choice. I talked to my co-workers. No luck. Finally, after two weeks I finally discovered the problem–a small bug inside a loop was interacting with Microsoft Active Directory to create hundreds of AD tokens. Another bug caused these tokens to skip garbage collection. Schedule? Throw that to the wind.

Every software developer has stories like this one. You probably have one from just last month, or even last week! The interaction between human and computer is simply too complex to predict when the next brick wall will come sailing out of nowhere. I like to tell junior developers that the art of developing software is like running hurdles—except that instead of hurdles we run at brick walls and in the middle of the night. When we hit one we scramble up, dig beneath, or bash our way through then sprint headlong into the next wall.

Large Project Estimation
Individual estimation is impossible. However, if enough developers are involved the random occurrence of schedule breaking events averages out and becomes a measurable “white noise”. The “white noise” can be then factored into the estimate as overhead. Estimation models such as COCOMO are built on this predictability.

However, there are clear limitations to this type of model.

  • A limited number of critical path options may narrow a project schedule to a tiny set of dependencies. Once a schedule is dependent on a very small group (or individual) developer(s) the project schedule can (and will!) be blown about by random chance.
  • If a project depends on unproven technologies or technology unproven to the domain the number of random roadblock events increases unpredictably. A key assumption of large project estimation is that the average number of schedule delaying events per developer is known and can be factored in as an overhead. With new technologies this can not be known and can not be accurately factored into the estimation.
  • Range of Precision: Models are appealing because they provide a number—however, even in ideal conditions they are far from precise.
  • In short, I would be extremely cautions to schedule or budget based solely on the results of a software estimation model. However, these models are useful and should be carefully considered in the light of the limitations above. One of the best uses may in fact be as an antidote to unrealistic optimism–I.e. “No, this isn’t a 6 month project. It really WILL take *THREE FULL YEARS!*”

    We (and our users) must accept that a software estimate is about as good as a prediction of marriage on the first date.

    Upcoming Article: How to keep your “customer’s” happy without estimates. (Coming in Early September)

    Further Reading:
    Large Limits to Software Estimation

    What Databases Should Do For Me

    August 12, 2007

    When I was a kid Christmas was the most exciting day of the year. I’d rip open the packages to see if I finally owned what I’d spent months hoping for.  In technology we don’t have a scheduled yearly Christmas, but occasionally we get exciting new technology.  And I can dream of the great things I’d like to see.  Here’s my wish list of what I’d like to see databases do:

    Automatic Transactions
    Many times when testing or troubleshooting I need to see the history of record changes. I wish the following SQL were valid:

      SELECT * FROM item_master
      WHERE item_master.item_id = 475

    Some of you are going to argue, “What’s wrong with creating a trigger and writing changes to a transactional table?”.  There isn’t anything wrong with that approach, but it’s a time waster: create the trigger, create the table, maintain the table when fields are added/dropped, create reports on the transactional table, blah, blah.  Millions of databases around the world face the same problem and apply the same generic solutions.  Generic problems should be solved in one place, not millions of places.

    Automatic Indexing
    In my opinion discussions about which database is fastest is usually a moot point. In the practical world the fastest database is the one with the best indexing. (And sufficient IO capacity) Most databases are severely under indexed. Above a certain cost threshold databases should keep statistics on queries and automatically create indexes for costly or common queries. (Probably a combination of the two factors: a very common query with low cost deserves and index as much as a uncommon query with high cost)  The RDBMS would also need to track the cost savings of the new indexes and drop them if they no longer provide sufficient savings.  There would need to be a cap on index creation if a table has too many indexes (i.e. if the indexes are interfering with updates and inserts)  Index creation would need to be load aware so it wouldn’t kick off during the busiest time of the day. . .  as you can see this would not be trivial.

    Automatic indexing would not replace manually defined (i.e. permanent) indexes, but it would be awesome.  Especially for purchased applictaions.  Purchased applications are chronically mal-indexed (one reason is that no one customer uses the application in exactly the same way) and no one at the customer sites knows what goes on under the hood to fix it.  Indexing problems in purchased applications are rarely resolved in a timely manner (or at all).

    Appeal to MySQL Developers
    I wish these features were part of Oracle or SQL Server, but other than natural product evolution I don’t think the giants have much innovation left in them. If we’re going to see innovation in databases it has to come from somewhere else.

    Jeff’s Book Recommendations:

    The Mythical Man-Month is the software engineering classic. This book should be mandatory reading for the professional programmer.