Software Architecture Zen is Five Years Old

So Software Architecture Zen is five years old (actually on 28th July, I missed my own birthday).

I started this blog on the Blogger platform and moved to WordPress earlier this year. Whilst my WordPress following has not built up to the same level I had on Blogger I far prefer the tools and whole look and feel offered by WordPress so do not regret the move.

I’ve had just over 103,000 hits in total across both platforms in five years. Not quite up there with the Joel on Software blogs of the world but not too shabby either I think. Here are my top five posts of the last five years:

  1. Architecture vs. Design
  2. How to Create Effective Technical Presentations
  3. On Thinking Architecturally
  4. Two Diagrams All Software Architects Need
  5. The Moral Architect

I’m pleased the last one is up there as it’s a topic dear to my heart and one I plan to blog on more in the future. Here’s to the next five years!


Architecting Out Complexity

Complexity kills. Complexity sucks the life out of users, developers and IT. Complexity makes products difficult to plan, build, test and use. Complexity introduces security challenges. Complexity causes administrator frustration.

So said Ray Ozzie ex-Microsoft Chief Software Architect and creator of Lotus Notes.

Complexity, or more precisely, overly complicated systems (not to be confused with complex systems) is one of the key anti-patterns architects must continuously fight against. Complexity is caused not just by adding additional and unwanted functionality (although that certainly does not help) but also by muddled architectural thinking as well as poorly made architectural decisions. Here’s the real problem though, the initial architecture of almost any system, unless it borrows very heavily from other, similar, architectures will rarely be without complexity. There will almost always be refinements that can be made, over time, that remove complexity and make for a cleaner and more streamlined design. Sometimes you may even need to throw away the initial architecture and start again, using what you have learnt from the initial architecture to take out complexity. Frederick Brooks (author of The Mythical Man Month) famously said of software designs “plan to throw one away, you will anyway“.

The other problem with complexity in systems is that it tends to increase over time due to software entropy. As more changes are made, some not envisaged by the architect because change cases were not adequately thought through, a system naturally becomes more complicated and harder to maintain. It almost seems that the lifecycle of a system could be represented by the complexity curve in the diagram below.

Complexity does not just apply to systems, it also applies to whole styles of architecture. Cloud computing would still seem to be fairly early on in the complexity curve in this respect. Cloud computing is almost the ultimate in information hiding. Just put everything in the cloud and get what you want when you want it. If you’re inside the cloud looking out however you need to deal with a whole lot of pain to create that outward facing simplicity. If you’re a cloud architect therefore you need to understand and design for that complexity otherwise over time our clouds will become weighed down with out of date junk that no one wants. This is definitely a topic I’ll be returning to over the course of 2012.

Challenging What Architects Do

For most of this year I’ve been fortunate (sic) enough to travel around several countries (and indeed continents) delivering a series of architecture workshops to a client. These workshops aim to teach, through the use of instructor led case studies, best practice approaches to building solution architectures. The workshops take a holistic approach to developing architecture by ensuring both the application as well as infrastructure aspects are considered together and both the functional and non-functional requirements are taken into account.

An enjoyable side-effect of these workshops is I get to meet lots of different people who like to challenge the approach and test my understanding (I sometimes think I’m learning more than the workshop participants). Here’s one such challenge (paraphrased) I received recently: “The problem with architects and the work products they produce (for example models built using the Unified Modeling Language) is that the work products become an end in their own right and we can easily lose track of the fact that actually what we want to build is a system with running code on real hardware”. The clear implications behind this challenge could be described via three architecture anti-patterns:

  • Antipattern: Architecture Overkill. Symptoms: Architects create work that no one understands or cares about. Multiple artefacts with overlapping content get created that no one actually reads. They are too large, too complex and may even be delivered too late.
  • Antipattern: Ivory Tower Architecture. Symptoms: Architects are out of touch with real-world technical aspects. Instead they just sit in their ivory-towers making their pronouncements (which most of the time no one listens to).
  • Antipattern: Work Creation. Symptoms: The role of the architect becomes self-fulfilling. They create work products which only they understand and therefore need more architects to help create these work products.

Here’s my response:

  • Any architectural work product needs to have either an up-stream (e.g. to the business) or down-stream (e.g. to the implementers) justification. Each of these groups of stakeholders needs to be able to recognise how the architecture, as encompassed in the work products, addresses their particular concerns otherwise the work product really is only for its own sake. There are other stakeholders who will have interests or concerns in the architecture but I just focus on these to illustrate my point.
  • The role of the architect needs to be clearly defined and work produced around that role. The role may change as business needs etc evolve but the work created should be in tune with that role and ultimately fulfil the needs of the business. This does not, of course, mean that part of what architects  do is research new technology or try out new ideas. Indeed if they don’t do this the next point will be tricky to fulfil.
  • One of the difficulties with the role of the architect is that it can mean the architect easily becomes out of touch with the latest trends etc in programming. This can be caused by a number of things including: architects are paid too much to do programming, architects are too senior to do programming, architects are too busy to do programming etc. This particular problem is particularly endemic in many services organisations where architects may be charged out at premium rates and where the role of programming is seen as almost too “menial”. What to do? It’s vital that this is not the case. There must always be time allowed in any software architects work day to stay in touch with the stuff that gives him his reason for being.

On Thinking Architecturally

Tim Brown (CEO of IDEO) has written a great book on design thinking called Change by Design. Check out the link to see his mind-map of what the book is about.The basic premise of the book, why its about design thinking rather just design, is that design thinkers take a far more holistic approach to solving design problems. They use an interdisciplinary approach, think around the problem, including viewing the constraints as enablers rather than what should be fought against, and come up with ideas that would otherwise not have been thought of if ‘ordinary’ design had been applied. One of the case studies Tim uses in the book is the setting up of a live laboratory in the Mayo Clinic in Rochester, Minnesota to develop new approaches to patient care which involved designers, health care professionals, business strategists and patients to develop ideas for the “patient-provider experience”. A methodology called SPARC (See-Plan-Refine-Communicate) was adopted (which is I suspect based on the Deming Cycle) to show how design thinking could be applied not only to product design but also service design.

Returning to the mind-map that is on the inside front-cover of Change by Design its two starting points are ‘What’ (is it we are trying to do/solve) and ‘How’ (are we going to approach the design). This fits nicely with my own concept of what we term ‘architectural thinking‘ where we add an additional ‘node’ which is ‘Why’ (are we doing it this way). I prefer to illustrate this as a Venn diagram as shown below. The intersection of the three sets is what we consider when ‘thinking architecturally’.

  • What – The requirements we are trying to address. Architectural thinking focuses is on those requirements (functional, qualities and constraints) we think are architecturally significant in some way.
  • Why – Captures the key decisions we are making. Architectural thinking focuses on the architectural aspects which lead to major structuring, placement or procurement decisions. Could be explicit (as a fully documented decision with options looked at and rational for making the decision we did) or implicit in a diagram or model.
  • How – The design and implementation of the system. Architectural thinking focuses on those elements of the design that are significant to the architecture (maybe patterns applied, key principles adopted etc).

The key thing in all this is that the thinking applies to the significant elements not everything. The key skill of the architect is to understand which things are important and which can be left to someone else to fret over.

Architecture vs. Design

Yes it’s that old knotty problem again! Over at Hugh MacLeod is fond of using Venn diagrams to illustrate overlapping concerns so here’s one that I recently used for addressing the eternal architecture versus design debate that was the source of much discussion at a recent Architecture Thinking class I was giving.

As I remember it the discussion went something like this:

  • Student: So what’s the difference between architecture and design? It seems from what you’re saying its just a matter of scale?
  • Me: Whilst its true to say that architecture addresses the major components of the system, rather than the detail, it’s more than that. The architecture is the bridge between the “what” (that is the requirements) and the “how” (that is the design).
  • Student: Yeah but isn’t that we usually call “high-level design”.
  • Me: Not really. Grady Booch says: “All architecture is design but not all design is architecture”. (I cheated and looked up this quote afterwards and found that Booch goes on to say “architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.”). In my experience high-level design is just the the view that allows the complete system to be represented on one page.
  • Student: I still don’t see what the difference really is.
  • Me: Okay, here’s the real difference for me (at this point I draw the above Venn on a flip-chart). As well as defining the structure of the system, architecture must also embrace the “what” and the “how” of that structure.  The “what” in this context is the requirements (functional and non-functional) and so architecture involves reasoning about and resolving these sometimes conflicting requirements. It’s about addressing those architecturally significant requirements (the “what”) that will drive (and constrain) the “how” (the design).

Now, if I was drawing this again (and maybe I’ll do this next time) I would actually draw a third overlapping circle which I’d label the “why”. This is where we’d capture the rationale for why we make the (architectural) decisions we do.

Thanks Hugh, this is a neat way of explaining the way things are!

Architecture Anti-Patterns: Pattern #6 – Architecture Thinking Not Doing

Anti-Pattern Name: Architecture Thinking Not Doing
General Form:
An organisation or enterprise embarks on an initiative to retrain their designers and developers to be ‘architects’. They set up education classes, assign new roles to their people with the word ‘architect’ in their job title and may even pay them some more money to match their ‘elevated’ status. However after a period of time (could be 6 months, could be longer) the initiative dies, developers just keep developing, designers just keep designing and no true system architectures materialise.
Symptoms and Consequences:

  • People return from training but make no changes to their behaviour. They revert to whatever it was they were doing previously.
  • The person(s) responsible for the initiative move on and as it was “their baby” the initiative dies.
  • Management think that training is sufficient and that once students have been trained they will “know what to do differently”.
  • People want to put into practice what they have learnt but have no idea where to go for help or guidance.

Refactored Solution:

  • Architecture thinking has to be turned into architecture doing. Follow up training with projects which make immediate use of new skills.
  • Identify early on who the real leaders are and get them to mentor others.
  • Obtain a critical mass of core people who will continue to develop their skills and be advocates of the new approach even once the initial hype has died down.
  • Ensure the initiative to train architects has support from the highest (i.e. CxO) level of the organisation.
  • Get buy-in from the business as well as IT. Convince the business of the need for architecture by showing them time to market and quality will improve and their will be less maintenance costs.
  • If the organisation is immature but keen to learn ‘buy-in’ experience either by hiring experienced people or working with good technical consultancies who have a proven track record (or some mix of both).
  • Follow up training with good quality, written down guidance (including guidelines, examples and templates) which is readily available and easily found (preferably in the companies website).
  • As part of the training get a written-down commitment from each student that describes one thing they will do differently as a result of attending this training. Follow this up and see if they did it. If not find out why not.

See Pattern #5

When Is It Time To Quit?

Whilst teaching an IT architecture class in Moscow this week I was asked the above question by one of the students. The question was in relation to projects and when you should realise the project is not going to deliver and should therefore be cancelled rather than carrying on regardless. I hate using the phrase “it depends” in answer to a question like this but this was one case when that’s all I could think of! However, having had a bit of time to reflect on this (on a long journey back home) here are a few thoughts that I’ve had since then. This is also related to my previous blog entry.

In some ways working on a software delivery project is a bit like fighting a war (though admittedly you don’t usually risk losing your life). In a war you may not win every battle and may sometimes need to retreat and reconfigure however, the important thing is to focus on the strategic battles that will lead you to get to the overall objective (and win the war). In a software delivery project winning the war is to successfully deliver the system or application on time and within budget. The battles are what the architect and project manager must fight every day to address the technical and other issues which arise and must be overcome for fear of letting the project get out of control. So what are the battles, which if not won, are going to cause the project to fail and, more importantly for this topic, will lead to surrender if the cost gets too high? Here’s my top five in order of importance:

  1. No, or poor, sponsorship.If there is no clear (and strong) sponsorship from the right level in the enterprise then you may as well pack up and go home immediately. Even if the project does get to the delivery stage the chances are no one will really want or need the system. The job of the sponsor is to evangelise, convince and cajole a sometimes reluctant workforce of the need for the new system and the change that it might bring about. Weak sponsors will be unable or unwilling to do this.
  2. Not having a water tight business case.If the business case for why the new system is needed is not well thought through with a clear cost-benefit analysis then there is a danger that the project could be cancelled at any time. I once worked on a project that was cancelled days before we were due to exit system test because a review of the business case revealed a flaw in the return on investment (ROI) of the new system so the client decided to pull the plug on the project rather than deliver something that would not make the cost savings that were expected of it. Painful but probably the right decision!
  3. Not engaging all stakeholders. Not identifying and engaging (i.e. talking to, convincing, schmoosing, wining and dining or whatever it takes) all the stakeholders is a bit like death by a thousand cuts. Although you will start off well, entropy will gradually set in and and things will start to fall apart. Stakeholders who should have been informed early on will learn about the project through the grapevine and will not only not support it but may actively try to kill it. They may see it as a threat or may treat it as sour grapes, “no one bothered to tell me about it so why should I support this”?
  4. Underestimating the complexity of building interfaces to legacy systems. Very few projects have the luxury of a clean sheet of paper when starting out. There is nearly always some old, creaking legacy system that needs to be interfaced with, even short-term or temporarily. In my experience one should never underestimate the complexity of these legacy system interfaces. And I don’t just mean technology interfaces. Many times IT departments will see it as a threat to their existence that what starts out as an interface may ultimately lead to a complete replacement of the system that they run and manage so lovingly and will therefore do whatever they can to derail the project.
  5. Not having regular and viable releases of the system/application. Gone are the days when a project could go off for many years before delivering anything of value to the key stakeholders (or I hope they have). I cannot emphasise enough the importance of delivering something of value that users can get their hands on and starting using that will give them some business benefits. This not only shows them the value of the new application but also gives them a belief that IT can actually deliver stuff and will also get them to “buy-in” to the new system and may well have the useful side-effect that they become sales-people for the new system and will convince colleagues of its benefits.

I believe that if one or more of these issues exist on your project you should either take very serious steps to address them or consider looking for another job because there is a greater than 50% chance everything is going to get very messy and is going to end in tears or much, much worse.