Depending upon which academic study you read, the failure rate of complex IT projects is reported as being between 50% and 80%! I thought I’d test this against my own experiences and took a look back over my career at the number of complex systems I have worked on and how many could be counted as being successful. Clearly the first thing you need to do here is to define “complex” and also “success” so I’m defining complex as being a system with:

• Multiple stakeholders involved.
• Multiple systems interfaces.
• Challenging or high risk non-functional requirements (including delivery schedule and budget).

and “success” as being:

• Delivered on time and within budget.
• Met the stakeholders requirements.
• Went into production and ran for at least 12 months.

By my count I have worked on 18 projects which meet the first set of criteria and of those I reckon 8 meet the second set of criteria so can be thought of as “successful”. So that’s a slightly under 50% success rate! Not brilliant but within the industry average (which is of course nothing to brag about).
As you might expect there is a wealth of information out there on the reasons why IT projects fail. Top amongst these are:

• Lack of agreed measures of success.
• Lack of clear senior management ownership.
• Lack of effective stakeholder management.
• Lack of project/risk management skills.
• Evaluation of proposals driven by price rather  business benefits.
• Projects not broken into manageable steps.

These typical failings were highlighted in a joint British Computer Society/Royal Academy of Engineering report from 2004 called The Challenges of Complex IT Projects. That was six years ago and I wonder what has changed since then? Anecdotaly I suspect not much. Certainly newspaper headlines about failed government IT projects of late (see, for example The Independent on 9th January 2010: Labour’s Computer Blunders Cost £26bn) would seem to indicate we are still not very good at delivering complex systems.

The interesting thing to observe about the above list of course is that none of these problems are technical in nature, not directly anyway. Instead they are to do with governance and process (or lack thereof) and what you might term “soft” aspects of systems delivery, how we manage and understand what people want. One of the right-brain activities which we IT folk sometimes fail to exercise is “empathy”. Our capacity for logical thought (i.e. left-brain activity) has gone a long way to creating the technological society we live in today. However in a world of ubiquitous information that is available at the touch of a button logic alone will no longer cut the mustard. In order to thrive we need to understand what makes our fellow humans tick and really get beneath their skin and to forge new relationships.

Happily this is not something that is easily outsourced, at least not yet! There is still something we can do as IT professionals therefore in engaging with stakeholders, understanding there wants and needs and trying to deliver systems that meet their requirements that can only be done with direct, personal contact.