architecture

Architecting Out of the Box

Peter Cripps architecture

I got this from a blog by Seth Godin. He was thinking about it in respect of marketers but it so applies to what we do as well.

Most of the time, we (IT architects or anyone else) think of our job as a set of tasks that take place in a box. We take inputs from upstream, add some value(hopefully) and create some outputs that we send downstream.

Architecting Out of the Box

It turns out, though (according to Seth), that if we go upstream and alter the stuff that comes to us, it’s a lot easier to do great work. And if we go downstream and teach people how to work with what we created, the final (work) product is better as well.

Seth gives as an example how a medical doctor can consider her work in the box of the examining room. But if she figures out how to get people to quit smoking before they come in, her results are better. If she figures out how to get people to take their pills after they leave, same thing.

As Seth says “the challenge lies in spending a lot of time and money on the upstream and downstream parts of the work, instead of always assuming that your [box] is just what happens inside your cubicle, or as a direct result of your actions”.

So here’s how I would translate this for the humble IT architect:

  • Visit your business sponsors by walking over to their desk (or calling them up on the phone if they happen to be on a different continent). Talk to them. Ask them what is troubling them today and see how you can help.
  • Take a long hard look at those requirements and make a point to go back to the person that wrote them if you don’t understand them. Don’t just assume stuff.
  • One persons architecture is another ones requirements. What you create will be used by designers and coders to build systems from. Make sure they really understand what you want. Go and visit them (possibly virtually) to see what makes them tick.
  • Here’s a really revolutionary idea. Write some code to a) convince youreself you can remember how to do it and b) show your downstream designers/implementers that you know what you are doing and have some empathy.

Most of what we do all day, intentionally or not, is aimed at keeping us in our boxes. Buck the trend and make an effort to get out of that box and make a difference.

Challenging What Architects Do

Peter Cripps architecture , ,

For most of this year I’ve been fortunate (sic) enough to travel around several countries (and indeed continents) delivering a series of architecture workshops to a client. These workshops aim to teach, through the use of instructor led case studies, best practice approaches to building solution architectures. The workshops take a holistic approach to developing architecture by ensuring both the application as well as infrastructure aspects are considered together and both the functional and non-functional requirements are taken into account.

An enjoyable side-effect of these workshops is I get to meet lots of different people who like to challenge the approach and test my understanding (I sometimes think I’m learning more than the workshop participants). Here’s one such challenge (paraphrased) I received recently: “The problem with architects and the work products they produce (for example models built using the Unified Modeling Language) is that the work products become an end in their own right and we can easily lose track of the fact that actually what we want to build is a system with running code on real hardware”. The clear implications behind this challenge could be described via three architecture anti-patterns:

  • Antipattern: Architecture Overkill. Symptoms: Architects create work that no one understands or cares about. Multiple artefacts with overlapping content get created that no one actually reads. They are too large, too complex and may even be delivered too late.
  • Antipattern: Ivory Tower Architecture. Symptoms: Architects are out of touch with real-world technical aspects. Instead they just sit in their ivory-towers making their pronouncements (which most of the time no one listens to).
  • Antipattern: Work Creation. Symptoms: The role of the architect becomes self-fulfilling. They create work products which only they understand and therefore need more architects to help create these work products.

Here’s my response:

  • Any architectural work product needs to have either an up-stream (e.g. to the business) or down-stream (e.g. to the implementers) justification. Each of these groups of stakeholders needs to be able to recognise how the architecture, as encompassed in the work products, addresses their particular concerns otherwise the work product really is only for its own sake. There are other stakeholders who will have interests or concerns in the architecture but I just focus on these to illustrate my point.
  • The role of the architect needs to be clearly defined and work produced around that role. The role may change as business needs etc evolve but the work created should be in tune with that role and ultimately fulfil the needs of the business. This does not, of course, mean that part of what architects  do is research new technology or try out new ideas. Indeed if they don’t do this the next point will be tricky to fulfil.
  • One of the difficulties with the role of the architect is that it can mean the architect easily becomes out of touch with the latest trends etc in programming. This can be caused by a number of things including: architects are paid too much to do programming, architects are too senior to do programming, architects are too busy to do programming etc. This particular problem is particularly endemic in many services organisations where architects may be charged out at premium rates and where the role of programming is seen as almost too “menial”. What to do? It’s vital that this is not the case. There must always be time allowed in any software architects work day to stay in touch with the stuff that gives him his reason for being.

Enterprise Architecture is Dead. Long Live Interprise Architecture

Peter Cripps architecture , , ,

ZapThink have recently been drip-feeding a series of ZapFlashes predicting the end of Enterprise Architecture (possibly allowing them to derive some further consulting/education business by creating a little fear, uncertainty and doubt). Here are some of their predictions and my thoughts. Comments welcome.

  • The secret to 21st century software innovation. Is, according to ZapThink, in harnessing the power of complex systems (i.e. rather than just enterprise-wide systems). A complex system is one that shows some form of emergent behaviour where the properties of the system as a whole exhibit behaviour that the individual components do not. Most people don’t get this confusing systems that are just complicated with ones that are complex. For me this is spot on. As the enterprise morphs into basically the whole of the internet traditional governance models breakdown. The boundaries of the enterprise can no longer be nicely defined and who is in the enterprise and who is not becomes harder to pin down. People who get this and who can successfully harness the “interprise” (my new word of the month) by using the positive effects of social networking etc will win out over the next few years. Of course this involves huge risks not least of which around how will enterprises keep control of their data and intellectual capital.
  • RIP enterprise software. Enterprise software as provided by the large package providers is seen as large and inflexible and not delivering on the benefits promised by SOA. Companies are finding themselves encumbered with expensive and hard to maintain software they can’t “bend” to do what they want. ZapThink’s take is that whilst enterprise software may have failed SOA has also failed to deliver on its promise of providing flexible business processes that can be quickly adopted to new needs. My take is that like everything else in our industry nothing is given chance to bed in before the next wave comes and sweeps everyone along with it. Many clients I see are still grappling with getting an effective process in place for developing traditional systems let alone service-based ones and definitely cannot deal with complex systems properly.
  • The beginning of the end for Enterprise Architecture frameworks. Architecture frameworks (especially enterprise ones like Zachman and DoDAF) are counterproductive to developing an effective EA strategy. These frameworks are inflexible, sometimes encouraging what ZapThink refer to as checklist architecture. Checklist architecture focuses on achieving goals laid down by the architecture framework rather than the changing needs of the business. True but having no framework at all leads to even greater chaos in my experience. A framework is a structure which is meant to contain relevant guidance and work products to deliver real-world solutions. Frameworks that become too theoretical are always doomed to fail as they should. I hope this forms the basis of something more relevant to the real world.

So how would I characterise “interprise architecture”?

  1. Interprise architecture recognises that it is not enterprise architecture that is dead but trying to constrain architecture by the bounds of the enterprise that is no longer achievable. The ‘architecture’ bit still applies, but not the ‘enterprise’ bit.
  2. Governance models need to recognise that this extended enterprise includes people and other systems that are not controlled by the IT department (people using social networking software who will periodically ‘overlap’ with the enterprises systems).
  3. The ‘architecture’ bit needs to take into account the fact that systems are complex (in the emergent sense of the word) and it’s not always possible to tie down the requirements in a nice orderly way. Indeed fully defining the requirements may be counter-productive and not allow emergent behaviour. Processes for developing such complex systems need to take this into account.

To be sure this is not a complete list. This idea is not yet fully-formed in my own mind and needs a bit more time to ’emerge’ properly. Watch this space as they say.

Six Non-IT Books for IT Architects

Peter Cripps architecture , , , , ,

There are now several myriad books out there on the topic of software architecture, including this one I have contributed to. There are other skills an architect needs to do their job which are not just to be found in IT books however. Here are six books which have helped me in my job together with a few reasons why I think they are useful:

  1. Change by Design by Tim Brown. Tim Brown is the CEO of IDEO a design company based in Palo Alto, California. Introduces the concepts of “design thinking” that can be applied to any problem and shows how empathy, play, storytelling and prototyping can all be bought together in coming up with new and innovative designs. Top tip: Deploy interdisciplinary teams of multi-talented people (i.e. true versatilists) to solve hard design problems. Even if you don’t get the book at least visit the link I’ve given to view the wonderful mid map.
  2. Presentation Zen by Garr Reynolds. Garr is the master of giving advice on how to create simple, clear and relevant presentations. Here he applies the zen principles of simplicity that will change how you think about creating presentations using PowerPoint or Keynote. Top tip: Picture superiority effect. Pictures are remembered better than words. Humans are hardwired for using images to communicate. Go visual wherever and whenever you can.
  3. A Whole New Mind by Dan Pink. Dan describe how, if we are to survive in the 21st century world of work, we must make more use of the left side, the creative side, of our brain rather than the traditional right (logical thinking) side. Top tip: Use stories to help illustrate your ideas. Stories represent a pathway to understanding that doesn’t run through the left side of the brain.
  4. Notes on the Synthesis of Form by Christopher Alexander. Alexander recognises that problems come with multiple, poorly understood requirements that interact with each other, creating conflicts and contradictions. Something we in IT have known for years. This book describes an approach for dealing with often multiple conflicting requirements to come up with the “best fit” solution. Top tip: It is difficult to specify a complete set of requirements that need to be met to achieve a “best fit”. A practical approach is to define ‘good fit’ as the absence of ‘misfits’, since these are usually what makes the problem obvious and can be ascertained through inspection of prior designs. Although designers may argue over the importance of a particular misfit, they are less likely to disagree on whether the misfit exists.
  5. Ignore Everybody by Hugh MacLeod. Hugh is an artist that makes a living creating art on the back of business cards, selling wine and running an extremely insightful web site on applying creativity to help you improve your job as well as your life. Top tip: Don’t try to stand out from the crowd, avoid crowds altogether.
  6. The Presentation Secrets of Steve Jobs by Carmine Gallo. Great book with lots of example on how Steve Jobs creates, prepares for and delivers his presentations to introduce new Apple gadgetry on the world. Top tip from book: Use plain English and photographs rather than techno mumbo-jumbo and slides densely packed with indecipherable text and bullet points.

When Systems Fail

Peter Cripps architecture , ,

This week I was a direct victim of a systems failure that set me thinking about how even mundane activities that we have been doing for several tens, if not hundreds, of years, like checking into a hotel  in this case, rely on systems that we take for granted and which, when they fail, throw everything into complete chaos.It’s a long and not particularly interesting story but in summary I checked into one of the large chain  hotels, which I use a lot, only to find when I opened my room door that the room was in a state of complete chaos and had clearly not been visited by housekeeping that day. On trying to change to another room I was told the system had been down since 4am (it was now 8pm) that morning and the staff could not tell what state rooms were in. Clearly not a great state of affairs and not great for client relations (there were a lot of grumpy people queueing in reception, some of whom I would guess would not be going back to that hotel). So what would an architect of such a system do to mitigate against such a system failure?

  1. I don’t profess to know too much about how hotel management systems work and whether they are provided centrally or locally however I would have thought one of the basic non-functional characteristics of such systems would have been a less than one hour recovery following a system failure (not 16 hours and counting). Learning point: Clarify your availability non-functional requirements (NFRs) and realise them in the relevant parts of the system. Maybe not all components need to be highly available (checking in a customer maybe more important then checking her out for example) but those that are need to be suitable ‘placed’ on high-availability platforms.
  2. There was a clear and apparent need for a disaster recovery plan that involved more than the staff apologising to customers. Learning point: Have a disaster recovery policy and test it regularly.
  3. A system is about more than just the technology; the people that use the system are a part of it as well. Learning point: The architecture of the system should include how the people that use that system interact with it during both normal and abnormal operating conditions.
  4. Often NFRs are not quantified in terms of their business value (or cost). When a problem occurs is the impact to the business (in terms of lost revenue, irate customers who won’t come back etc) really understood? Learning point: Risk associated with not meeting NFRs needs to be quantified so the right amount of engineering can be deployed to address problems that may occur when NFRs are not met.

Formal approaches to handling non-functional requirements in a systems architecture are a little thin on the ground. One approach suggested by the Software Engineering Institute is through the use of architectural tactics. An architectural tactic is a way of satisfying a desired system quality (such as performance) by considering the parameters involved (for example desired execution time), applying one or more standard approaches or patterns (such as scheduling theory or queuing theory) to address potential combinations of parameters and arriving at a reasoned (as opposed to random) architectural decision. Put another way an architectural tactic is a way of putting a bit of “science” behind the sometime arbitrary approach to making architectural decisions around satisfying non-functional requirements.

I think this is a field that is ripe for more work with some practical examples required. Maybe a future hotel management system that adopts such an approach to during its development will allow a smoother check-in process as well.

The Three C’s of IT Architects

Peter Cripps architecture, creativity

This is a completely unscientific observation but over the years of being an architect I have observed the following characteristics in people that claim to be members of this profession.  I refer to these as the three C’s of being an IT architect. Some people have only one of these but most have a mix of all three, with maybe one being dominant. The three C’s are: 

  • Creatives – These are the ideas people, that are keen to do new stuff. They are the people that build the solutions that address the business requirements. They have an intimate knowledge of technology (in their particular area of interest or concern) and want to use that technology. If you don’t have these people in your project/team/organisation then nothing will actually get done. The downside of having too much of this attribute is that eventually you have to stop creating and ship something so you need to know when to stop. 
  • Consumers – These are the people that use what the creatives create. Whilst they may not create anything new this type of architect is not without merit. They often combine what others have done in new and innovative ways. We sometimes refer to this activity as reuse, one of the Holy Grails of IT so it is not to be underestimated. If you don’t have these people in your project/team/organisation then chances are the ideas from the creatives will not get fully realised. The downside of having too much of this attribute is that there is a limit to what you can build out of reusing stuff and eventually someone has to come up with some new ideas. 
  • Connectors – These are the people that don’t create or reuse but know people that can do these things and join the two together. Again this is not to be an underestimated skill. After all if a seller cannot find a buyer what’s the point! If you don’t have these people in your project/team/organisation then the two previous types won’t find each other. The downside of having too much of this attribute is that you ain’t going to do anything if all you do is push ideas of others around without creating or reusing things.

From my observations I reckon a ratio of Creatives to Consumers to Connectors needs to be something like 4:2:1.

Incidentally, my guess is that these actually apply to other professions as well but I have even less scientific evidence about those.

Default Architecture

Peter Cripps architecture , ,

One of the attributes that many (if not all) complex systems have is the ability to change (customise) them in controlled ways. Indeed this, by definition, is why some systems are complex (that is exhibit  emergent behaviour) because sometimes the users of those systems select options or make choices that enable unexpected behaviour to occur. Giving users such choices clearly has a number of implications on the architecture of a system.

  1. Users have to make a choice, no choice is actually a choice itself as it means they are choosing the default.
  2. Making systems customisable assumes users have the will (and the time) to decide which options they want to change. Many times they don’t so the default becomes very important as it will dictate how the users actually use the system, possibly forever.
  3. The more options that are built into a system the more difficult it becomes to test each potential combination of those options. Indeed there comes a point at which it becomes impossible to test every combination (at least in a reasonable amount of time), hence the importance of beta software (let the users do the testing).

In his blog entry Triumph of the Default Kevin Kelly points out how “the influence of a default is so powerful that one single default can act as a very tiny nudge that can sway extremely large and complex networks”. The oft quoted example of how defaults can influence behaviour is that of organ donation. If you make the donation of organs upon death automatically an “opt out” choice (it happens unless you refuse beforehand) versus “opt in” (it does not happen unless you sign up). A opt out donor system greatly increases the number of organs donated.

For complex systems then, the default architecture of the system becomes very important. The choices the architect makes on behalf of the users of that system will not only dictate how the users will actually use the system but may also influence their behaviour (in both positive and negative ways). Defining the defaults is as important an architectural decision as to what technologies should be used and sufficient time should always be planned in to allow such decisions to be made in a sensible way. The system architecture that results can profoundly affect how that system will be used.

Evolutionary Systems Development Process

Peter Cripps architecture ,

As I’m now into my second year I reckon I can become more experimental in my blogs. This entry is therefore completely off the scale and will either not go any further or be the start of something much bigger. Comments suggesting which it is are welcome.If we want to be able to build real complex systems (see previous entry for what these are) what might we do differently to ensure we allow truly emergent properties to appear? Is there a development process that we could adopt that both ensured something was delivered in a timely fashion but that also allowed sufficient ‘flex’ such that emergent behaviour was not prevented from happening? In other words is it possible to allow systems to evolve in a managed way where those properties that we value are allowed to grow and thrive but those that are of no use can be prevented from appearing or stopped early on?

Here are some properties I think any such a development process (an evolutionary systems development process) should have if we are to build truly complex systems

  1. The process needs to be independent of any project. Normally we use processes to derive some kind of work breakdown structure which is used to plan the project that will deliver the system. In other words there is a well defined start and end point with clear steps stating who does what by when in between. However for a complex system there is by definition no stopping point; just a number of evolutions where each introduces new (and possible unpredictable) behaviour.
  2. The process should be driven not so much by what users want but by what users do. Most processes start with a list of requirements, possibly expressed as use cases, which describe scenarios for how the system will be used. The problem with this approach is that use cases only describe what a user can envisage doing with the system and will not capture what they actually end up doing with the system. In other words the system is deemed to be ‘complete’ once all the use cases have been realised. However what if the act of delivering the system itself results in new use cases emerging. Ones that were not planned for? How do they get realised?
  3. The process must be both flexible enough to allow users to experiment and envisage doing new things whilst at the same time being robust enough not to allow negative emergent behaviour that will prevent any system being delivered or lead to the system deteriorating over time.
  4. If new behaviour is to be adopted this must be of overall benefit to the majority and must still meet (a possibly changed) set of business objectives. The process must therefore allow for some kind of voting system where the majorities behaviour is allowed to dominate. The trick is not allow new and innovative behaviour to be crushed early on.
  5. The underlying rules that control how the systems responds to external stimuli must themselves be easily adaptable. The process must therefore treat as peers the rules that govern (allowable) behaviour together with the description of the behaviour itself. Rules set some kind of constraint on what is allowable and not allowable by the system.

Of course not all systems that we build will or should be complex ones. As noted previously safety-critical systems need to behave in entirely predictable ways and users should not be allowed to change the behaviour of these where lives could be put at risk. So what type of systems might benefit from an evolutionary approach to development?

  1. Social networking systems where the users are intimately bound up with the system itself.
  2. Commerce systems where the buying behaviour of users might change how and when certain products are sold and at what price.
  3. Financial systems where money markets may affect what users do and how they respond.
  4. Government systems where responses to new legislation or the behaviour of citizens needs fast responses.
  5. Other, you decide?

My next step is to think about what the elements (process phases, work products etc) of an evolutionary systems development process might be.

Complex Systems versus Complicated Systems

Peter Cripps architecture , , ,

Brian Kernighan,co-author of the C programming language, once said:

Controlling complexity is the essence of computer programming.

We live in a world where the systems that we use or come across in our day to day lives seem to be ever more complicated. The flight software system that controls a modern aircraft like the new Boeing 787 “Dreamliner” are, on the face of it at least, almost unimaginatively complicated, simply because of their sheer size. According to NASA the software that runs the 787 is almost seven million lines of Ada code, triple that of the 777. The F35 Joint Strike Fighter has 19 million lines of C and C++ code! Does all this mean that these systems are also complex however and if not what’s the difference? Inevitably there are a large number of definitions of exactly what a complex system is however they all seem to agree on a few common things:

  1. They are made up of a collection of components.
  2. The components interact with each other.
  3. These interactions can result in emergent behavior.

Emergent behavior refers to the property that a collection of simple components may exhibit when the interactions between them result in new, and sometimes unpredictable, behavior that none of the components exhibit individually. So whilst complicated systems may be complex (and exhibit emergent properties) it does not follow that complex systems have to be complicated. In fact relatively simple systems may exhibit emergent properties.

In his book Emergence Steven Johnson gives several examples of the emergent properties of all sorts of systems from colonies of ants through to software systems. Whilst some software systems clearly thrive on being complex systems where emergent behavior is a positive benefit (I’m thinking Facebook, World of Warcraft and SecondLife for example) we might be a bit more dubious about getting on an aircraft whose flight software system exhibits emergent behavior! My guess is that most of us would prefer it if that software showed entirely predictable behavior.

Here’s the thing however. Should it not be possible to build some of our business systems where emergent behavior could be allowed and for the systems themselves to be self adjusting to take advantage of that behavior? How might we do that and what software delivery lifecycles (SDLC) might we adopt to allow that to happen? The interesting thing about SDLCs of course is that almost by definition they build predictability into the systems that are developed. We want those systems to behave in the way the users want (i.e. we want them to “meet the requirements”), not in unpredictable ways. However the thing about systems like Facebook is that the users are an integral part of that system and they drive its behavior in new and interesting ways. The developers of Facebook are able to observe this and respond to new ways in which their system is being used, adapting it accordingly. Facebook has clearly been architected in a way that allows this to happen (Nice one Zuck). The problem with many of our business systems is that they are too rigid and inflexible and do not allow emergent behavior. Even the promise of Service Oriented Architecture (SOA) where we were supposed to be able to reconfigure our business processes almost at will by combining services together in new and interesting ways has not really delivered on that promise. I think that this is for two reasons:

  1. We have failed to adapt our (SDLC) processes to take into account that property and are instead building the same old rigid systems out of slightly new sets of moveable parts.
  2. We fail to recognise that the best complex systems have people in them as an integral part of their makeup and that it is often the combination of people and technology that drive new ways of using systems and therefore emergent properties.

Building and managing complex systems needs to recognise the fact that the same old processes (SOPs) may no longer work and what new processes we new develop need to better account for people being an integral part of the system as it is used and evolves. The potential for emergent behavior needs not only to be allowed for but to be positively encouraged in certain types of system.

 

On Thinking Architecturally

Peter Cripps architecture , ,

Tim Brown (CEO of IDEO) has written a great book on design thinking called Change by Design. Check out the link to see his mind-map of what the book is about.The basic premise of the book, why its about design thinking rather just design, is that design thinkers take a far more holistic approach to solving design problems. They use an interdisciplinary approach, think around the problem, including viewing the constraints as enablers rather than what should be fought against, and come up with ideas that would otherwise not have been thought of if ‘ordinary’ design had been applied. One of the case studies Tim uses in the book is the setting up of a live laboratory in the Mayo Clinic in Rochester, Minnesota to develop new approaches to patient care which involved designers, health care professionals, business strategists and patients to develop ideas for the “patient-provider experience”. A methodology called SPARC (See-Plan-Refine-Communicate) was adopted (which is I suspect based on the Deming Cycle) to show how design thinking could be applied not only to product design but also service design.

Returning to the mind-map that is on the inside front-cover of Change by Design its two starting points are ‘What’ (is it we are trying to do/solve) and ‘How’ (are we going to approach the design). This fits nicely with my own concept of what we term ‘architectural thinking‘ where we add an additional ‘node’ which is ‘Why’ (are we doing it this way). I prefer to illustrate this as a Venn diagram as shown below. The intersection of the three sets is what we consider when ‘thinking architecturally’.

 
  • What – The requirements we are trying to address. Architectural thinking focuses is on those requirements (functional, qualities and constraints) we think are architecturally significant in some way.
  • Why – Captures the key decisions we are making. Architectural thinking focuses on the architectural aspects which lead to major structuring, placement or procurement decisions. Could be explicit (as a fully documented decision with options looked at and rational for making the decision we did) or implicit in a diagram or model.
  • How – The design and implementation of the system. Architectural thinking focuses on those elements of the design that are significant to the architecture (maybe patterns applied, key principles adopted etc).

The key thing in all this is that the thinking applies to the significant elements not everything. The key skill of the architect is to understand which things are important and which can be left to someone else to fret over.