15 August 2014

A theory of agile architecture in a nutshell (part 1)

Today I deposited the final version of my thesis,  "Reconciling agility and architecture: a theory of agile architecture," with the university library. It is officially complete! 

This post presents the first part of a summary of my findings: my thesis in a nutshell. This part introduces my research and discusses what an agile architecture is. The second part, coming soon, discusses the theory of agile architecture itself.

If you want to read my thesis in its full academic glory, email me for a copy, or wait for it to become available at the Victoria University Library sometime next week.

The problem

The Agile Manifesto discourages too much planning ahead: if you spend too much time planning your architecture up-front, you're not agile because you're not delivering value or responding to change.

On the other hand, if you spend too little time planning your architecture up-front, your agility will probably also suffer because you'll end up spending all your time fixing architectural problems rather than delivering value.

So what is the right amount of up-front design effort, and what does it depend on?

My PhD research investigated how teams determine how much architecture to design up-front.

The research process

My research was empirical – the knowledge came from agile practitioners. These findings are all based on what agile teams actually do.

The research is qualitative –  the data is in the form of words rather than numbers. This means the findings are descriptive: the findings are an explanation of what is going on, rather than, for example, a recommendation that teams spend some percentage of their time planning.

And to really get into the methodology detail, the research strategy was Grounded Theory. Grounded Theory is a rigorous and methodical research methodology that abstracts the research participants' experiences into a theory –  hence "a theory of agile architecture." The theory is descriptive because it describes participants' experiences, rather than prescriptive –  it does not prescribe a methodology or a set of instructions or anything like that.

The findings

Defining agility

Before getting into the meat of the findings, we need to define agility: what does being agile mean? For this research, I defined agility as:
"...a software development team’s ability to create change, respond to change and learn from change so that it can better deliver value."
Of course, the most important part of this is being able to respond to change.

For the avoidance of doubt, this definition of agility is not necessarily the same thing as adhering to the philosophy of the Agile Manifesto, or following a particular agile methodology, or using a particular set of agile practices. You can be agile without following a particular set of agile practices, and just because you're using a particular methodology it doesn't make you agile. This research developed a theory of agile architecture – but first what is an agile architecture?

What is an agile architecture?

Simon Brown has an excellent definition of agile architecture: an architecture that provides agility by being flexible and adaptable. Hayim Makabee considers something similar with his adaptable design up-front. Philippe Kruchten noted that this definition of agile architecture is not necessarily the product of an agile process (that is, it could be created by a big up-front design); he compared this with a definition of agile architecture in which the architecture has emerged as part of the agile process, and noted this does not necessarily lead to a flexible architecture.

I define an agile architecture as comprising both of these definitions:
An agile architecture is first an architecture that is consistent with the definition of agility: the architecture can "respond to change" by either being easily modifiable or by being tolerant of change. Second, an agile architecture is the product of the team's agile process. 
Why does an agile architecture consist of both of these parts? Consider these five "tactics" that teams use to design an agile architecture:
  1. Keep designs simple – the agile principle of simplicity: only designing for what is immediately required; no over-engineering and gold plating.
  2. Prove the design with code iteratively – a tactic made possible through having design and development taking place simultaneously (i.e. not having the architecture design take place before development of features begins). Instead of testing a design through analysis, build it to see if it works.
  3. Use good design practices – of course, every software design should use good design practices. That's what it means! But the benefit is amplified in agile, because we are explicitly welcoming change, and having a well designed architecture makes it easier to change.
  4. Delay decision making – this tactic helps reduce rework because decisions are made when more information is known about the decision to be made, and hence more accurate decisions can be made. (Note this does not mean delay as long as possible! It simply means delay until the right time.) It also helps avoiding over-engineering by not forcing requirements to be defined prematurely.
  5. Plan for options – don't close off possible future requirements. That is, you know things are likely going to change, so don't design your architecture into a corner. That's not to say spend extra time designing for those possible future requirements – that's over-engineering which contravenes the first tactic.
Keeping the design simple (#1), following good design practices (#3) and planning for options (#5) all affect the architecture itself: the use of these tactics may be determined later by inspecting the architecture. These three tactics are not dependent on an agile process. On the other hand, proving the architecture with code and delaying decisions are features of the agile process – the architecting  – and not so much the architecture itself.

The first three tactics each increase the architecture's modifiability, while the last two make it more tolerant of change (the architecture is less affected by changing requirements).

The agile architecture tactics also impact the up-front design effort: keeping the design simple, proving the design with code and delaying decisions all reduce the up-front effort.

The impact of the tactics are summarised in the following table.

Impact on responsiveness to change
Affects architecture or architecting process?
Reduces up-front effort?
Keep designs simple
Increases modifiability
Prove the design with code iteratively
Increases modifiability
Architecting process
Use good design practices
Increases modifiability
Delay decision making
Increases tolerance of change
Architecting process
Plan for options
Increases tolerance of change

That's the first half of the nutshell. Coming shortly: part 2, which describes the theory of agile architecture.


  1. Your definitions of "agile architecture" are circular, and in any case, you say nothing about making the organization more agile. I think you need to work more on this.

    1. Hi Jason, thanks for your comment. The definition of "agile architecture" relies on the definition of agile/agility, and the definition of architecture/architecting.

  2. Could you expand on the tradeoff between keeping design simple and planning for options? I feel you hand-wave this a bit, because designing for modifiability almost by definition precludes simplicity (especially YAGNI or 'the simplest thing that can possibly work'). It's a tradeoff, but the crux is to provide architects/designers information on what the specifics are: e.g., writing Eclipse as a plugin architecture will enable X while costing Y performance/time to market.

    And in our experience (with the ATAM) testing the design is absolutely possible with analysis. Comprehensive system testing isn't the only way to establish design (and often is infeasible). It always surprises people how much they can get by gathering key stakeholders and walking through a few scenarios against the architecture.

    1. Hi Neil, my research participants wouldn't admit that planning for options increased the effort required and therefore broke the simplicity principle. (Although one said that their system with extra layers of abstraction took longer for new team members to learn.) Perhaps they kept planning for options to a minimum.

  3. Well firstly: Congratulations on finishing! - seems a long time ago we talked this over in London
    (For everyone else: I was one of the first people Michael interviewed, despite my attempts to dismiss architecture he kept on and found some wiser people to talk to.)

    Agile is something of a moving target so to come up with any definition (even circular) is quite a feat.

    I like the sound of the 5 tactics and I think they could be used as advice to many teams. I don't expect they are complete (there may be more) but 5 is plenty to be getting on with. (Umm, those 5 could be worked up as full patterns I think....)