Software: Which Design is Architectural?

“Not all design is architectural” is widely repeated. Non-circular answers to “which design is architectural?” are less common. Here's one:

A design or decision is architectural if it impacts the system as a Whole, not merely in part.

(from @visarch 2004, http://www.bredemeyer.com/pdf_files/WhitePapers/ArchitectureAsBusinessCompetency.PDF )

The Smallest Agile Methodology That Could Possibly Work

People

You need 3 hats. That might mean 3 people, or 3 groups of people, or 1 or 2 people wearing more than 1 hat:

  • 1 person who knows what they want to have.
  • 1 person who knows about how to provide that.
  • 1 person with enough money to pay for it to be provided.

People Checklist

1) Do you have someone to wear each of the 3 hats?

A: I don't have someone who knows what they want.

  • Consider a research project instead, in which you try to work out what you want.
  • Otherwise Stop Now.

A: I don't have someone who knows how to provide what I want

  • Find help. Otherwise Stop Now.

A: I don't know whether we have enough money.

  • Do the next 2 steps, Requirements and Priorities, and then do the Budget step. If you cannot afford to do those steps, then you do not have enough money and you should Stop Now.

Requirements

  • The person who know what they want must express what they want in enough detail that the person who is going to provide it can understand what they are supposed to be providing.

This sounds very simple. It is not. Many many conflicts in personal life, disastrous work projects, and famous cockups in politics follow from the fact that helping person A to express what they want well enough so that person B can deliver it for them is a skill that takes time to learn.

Solution: hire a Business Analyst to help person A to express what they want and to express it as requirements that persons A and B can both agree they understand and are happy with

Requirements Checklist

1) Does anyone involved have previous experience of how easy it is is to trip up on getting someone to express what they want correctly and in enough detail for someone to deliver it?

A: No

  • Find someone. Pay for their help.
  • Or, budget for doing everything 2 to 3 times in order to get it right. (Yes, 2 to 3 times is about right for your first ever project. It is not a worst-case.)

Priorities

  • The person who knows what they want must be able to express what their priorities are amongst the things they want. Either based on what they want fastest, or on what they value most.

This sounds very simple. It is. The only two things that can possibly go wrong are:

  • person A changes their mind.
  • you have not broken the work into small enough chunks. A small enough chunk is a chunk that the person doing the work is confident they can can complete in under a week or so.

Priorities Checklist

1) Can the person who knows what they want prioritise what they want and break it down into small enough chunks so that the person who knows how to deliver can confidently say, of at least the top 2 items, that they can do them in less than a week or so each?

A: No

  • Find someone to help with doing this.
  • Alternatively, budget for 2 to 3 times the cost of the first item for working out as you go along how to do it. If you have already budgeted 2 to 3 times cost because of no experience with requirements, that gets you to 4 to 9 times the cost for the first item. Yes, really.

Delivery

  • The person who knows how to deliver (build, buy, borrow, invent, bodge, whatever) what the person who knows what they want wants, does so.
  • [If there is more work involved than they can do in a week or so, they first chunk the work, in strict priority order, into batches of, say, a week's work.]
  • After they have completed as little of the first piece of work that is still showable to the person who knows what they want, they show it. The person who knows what they want confirm or corrects the understanding of what is wanted. The person who is delivering corrects course.
  • After they have completed the whole of the fist piece of work they show that.
  • [Or, in the larger chunked case, after the first batch they show the person who knows what they want, what they can have so far.]
  • The person who knows what they want then says, “yes I'll take that” or “that looks great, keep going” or “please change … <insert detail here>.”
  • If it is not finished, keep going. If it takes a long time, work to a regular rhythm.

Budget

  • After you have requirements and priorities, you may wish to plan a budget. A budget is often counted in money or in person-days of work.

You may think that offering a price for a job is what any experienced tradesman should be able to do. You may, or may not, be right. There are 3 main risks:

  • Nobody involved in the project has enough experience to estimate a fair budget, but no-one wants to admit to not having enough experience.
  • The people estimating the budget are pressured into underestimating the costs
  • The job, for reasons not yet foreeen, will incur unexpected additional costs.

Budget Checklist

1) Make sure you have reviewed each of the 2 to 3 times multipliers for inexperience in Requirements and Prioritisation.

2) Do you have someone on the project with enough experience to estimate a budget?

A: Yes

  • Is that person the person doing the work?
    A: Yes: Then accept their min/max estimates for costs and their choice of a prudent budget
    A: No: Then have them review their min/max estimates for costs with the person doing the work, let the person doing the work provide detail, and let them come to a consensus.

A: No

  • Find someone to help.
  • If you are able and willing to risk the cost of 1 or 2 weeks work, consider simply starting the work and reviewing progress after 1 or 2 weeks. Then, try again to estimate a budget.

Review

  • Whatever you do, review how you are working after 1 to 2 weeks. Ask yourselves questions to help understand your satisfaction and dissatisfaction, for instance:
    • What is going well?
    • What am I happy with or not happy with?
    • What can we do better?
    • What would I like to change?
      Then, make a plan to do at least one of the things you decided would make things better.

Note you are not asking, “is the thing we are delivering good?” You are asking “is the way we are working good?” The two things are closely linked of course, both in the positives and the negatives.

Review Checklist

1) Did you, in fact, do a review after 1 to 2 weeks?
2) Did you, in fact, choose at least one thing to improve?
3) If you are now 4 weeks into a project have you, in fact, improved the things you intended to improve.

Risk

  • A risk is something that might either stop your project or else make you wish you had not started it. Make a list of them.
  • All of the checklists above represent potential risks. If you did not have a whole-hearted pass for each item of each checklist, list that as a risk.
  • Each time you plan a piece of work, consider the associated risks and how you can make the risk smaller. If you don't really know how to, find help.
  • For risks that you cannot make any smaller, are you able to accept the price of that risk happening? Are you clear on who has what share of the price?

Risk Checklist

1) Did you make a risk list?

A: No

  • Stop Now.

2) Do you have someone on the project with prior experience in managing risk?

A: No

  • That is a risk. Get help.

3) Do the things you haved planned to minimise risk really minimise risk or are they just things you can write down even though they might have no effect in minimising the risk?

Considerations

  • It is a fact of craft work that the right answer is often, “Get someone with the experience to do it”
  • Minimalism is of little use to a beginner. Rather, they first need a step-by-step, then later they can understand how to try minimalism.
    This is why complaints that “<insert agile methodology name here> is terrible” are largely futile. All well-known agile methods gives a simple enough way to start off and then require you to review and change how you work.

Showing Software Architectures in 2+1 Views with UML

  • How can I show & explain an application or service architecture?
  • How can I draw “correct” (and what does that mean?) UML diagrams?

Let me show you how I document smallish software systems with, typically, no more than four UML diagrams. We'll cover the questions, what is worth diagramming? and what can I do with these diagrams anyway? and we'll cover enough UML to be useful.

I will use the example of developing a “lead generation” website, which should be simple enough to not get bogged down, but complex enough to answer some of the but how do I …? questions that more complex systems raise.

Why Diagrams?

Use diagrams as starting point to discuss, describe and explain your system. A diagram can give an instant overview of some aspect of the system. It can show important relationships on a single sheet of paper. It can raise important questions and show design decisions.

Why UML?

The UML is a visual modelling language. It has a vocabulary and grammar for diagramming software such that the diagram is a precise statement. So it can be used to show and explain software architecture.

The UML is the only diagramming standard left standing (with, perhaps, one exception that we'll see later). You may be tempted to compete in this uncrowded field of standards for software diagrams. I comment only that the UML contains several decades of work by several thoughtful people and if you can produce a usable standard that is simpler, quicker to learn, and not unhelpfully imprecise, then I shall be impressed.

You may also be tempted — as many, many of your colleagues have been — to just do without a standard. I hope these few examples will persuade you that learning a standard is less work than you fear and more useful than you expect.

Why 2+1 views?

The UML tells you how to diagram once you've decided what to diagram. “2+1” is a minimal decision about what to diagram. It suggests a couple of diagrams for the logical view of your software and one for the physical view (we'll explain what those words mean, too). That's the 2. But it starts with the 1, which is the context.

The System Context: Shown with a UML Use Case Diagram

Always start with context. Always start with simple.

The context diagram should make sense to your non-technical customer and can be so simple you wouldn't even give it a second glance. It shows:

  • Who and what will use the system
  • What the system will do
  • Who and what the system will rely on

In the UML, anyone or anything who uses the system, or is relied on by the system to do its thing, is an Actor. What the system will do is a UseCase. A UseCase is a instance of the system being used to do something. It is shown on a diagram just by its name, which is a single descriptive phrase, in an ellipse.

  • Anything inside the box is what the system does.
  • Anything outside the box is the context of the system.

How to Use a Context / Use Case Diagram?

Use the diagram to discuss scope, expectations and dependencies with the system's customers and with the development team who will build it. Its simplicity and brevity should also clarify what is not (or not yet) expected of the system. Like user stories, the brevity calls for conversation to clarify. Complex Use Cases call for detailed careful business analysis too, but that is done with words, not pictures.

For your developers, this diagram is the high level overview of what they're delivering — everything inside the box — and what the system will need for it to work — everything outside the box.

I really like having a “hand drawn” look when first showing diagrams, because it says “work in progress” and invites participation. A precisely drawn diagram risks the impression of being a final decision.

I try to draw diagrams to be read from top left to bottom right. So I put “active” Actors—the users—on the left, and “dependency” actors on the right. That's not a part of UML, but it's part of how I will talk through the diagram.

Here's the same diagram later on on the project. In phase two, we're giving visitors SMS feedback, and adding a whole new bit of functionality to read events from the customer service team and integrate then with web analytics.

Use Case diagram with several use cases, 2 user actors and 5 machine actors.


What if you have lots of use cases? Don't put more on one diagram than you can usefully discuss in a single session. Pick out the main use cases & those that show the main external dependencies (which probably means, the ones that pose the highest risk to your project). Optionally, have a second diagram for all use cases, but only if you have an audience for it.

UML Definitions

There are 3 things you need to know for this diagram:

Actor, UseCase, and Subject aka System Boundary

Here are their definitions, which I've abbreviated from the UML 2.5 spec, section 18 Use Cases.

UseCases are a means to capture the requirements of systems, i.e., what systems are supposed to do. The key concepts specified in this clause are Actors, UseCases, and subjects. Each UseCase’s subject represents a system under consideration to which the UseCase applies. Users and any other systems that may interact with a subject are represented as Actors.

A UseCase is a specification of behavior. An instance of a UseCase refers to an occurrence of the emergent behavior that conforms to the corresponding UseCase. Such instances are often described by Interactions.

An Actor models a type of role played by an entity that interacts with the subjects of its associated UseCases (e.g., by exchanging signals and data). Actors may represent roles played by human users, external hardware, or other systems.

NOTE. An Actor does not necessarily represent a specific physical entity but instead a particular role of some entity that is relevant to the specification of its associated UseCases. Thus, a single physical instance may play the role of several different Actors and, conversely, a given Actor may be played by multiple different instances.

NOTE. The term “role” is used informally here and does not imply any technical definition of that term found elsewhere in this specification.

A UseCase is shown as an ellipse, either containing the name of the UseCase or with the name of the UseCase placed below the ellipse. An optional stereotype keyword may be placed above the name.

A subject for a set of UseCases (sometimes called a system boundary) may be shown as a rectangle with its name in the top-left corner, with the UseCase ellipses visually located inside this rectangle.

An Actor is represented by a “stick man” icon with the name of the Actor in the vicinity (usually above or below) the icon. An Actor may also be shown as a Classifier rectangle with the keyword «actor»

Other icons that convey the kind of Actor may also be used to denote an Actor, such as using a separate icon for non- human Actors.


The spec: https://www.omg.org/spec/UML

I have used 3 more UML features to add explanations:

  • A Comment is “a textual annotation that can be attached to a set of Elements”. A Comment is shown as a rectangle with the upper right corner bent (this is also known as a “note symbol”). The rectangle contains the body of the Comment. The connection to each annotatedElement is shown by a separate dashed line. The dashed line connecting the note symbol to the annotatedElement(s) may be suppressed if it is clear from the context, or not important in this diagram.
  • A Dependency “implies that the semantics of the clients are not complete without the suppliers”. A Dependency is shown as a dashed arrow between two model Elements. The model Element at the tail of the arrow (the client) depends on the model Element at the arrowhead (the supplier) .
  • InformationFlows “describe circulation of information through a system in a general manner. They do not specify the nature of the information, mechanisms by which it is conveyed, sequences of exchange or any control conditions”. An InformationFlow is represented using the same notation as Dependency , with the keyword «flow» adorning its dashed line.

Conway’s Law & Distributed Working. Some Comments & Experience

The eye-opener in my personal experience of Conway's law was this:

A company with an IT department on the 1st floor, and a marketing department on the 2nd floor, where the web servers were managed by the marketing department (really), and the back end by the IT department.

I was a developer in the marketing department. I could discuss and change web tier code in minutes. To get a change made to the back end would take me days of negotiation, explanation and release co-ordination.

Guess where I put most of my code?

Inevitably the architecture of the system became Webtier vs Backend. And inevitably, I put code on the webserver which, had we been organised differently, I would have put in a different place.

This is Conway's law: That the communication structure – the low cost of working within my department vs the much higher cost of working across a department boundary – constrained my arrangement of code, and hence the structure of the system. The team "just downstairs" was just too far.  What was that gap made of? Even that small physical gap raised the cost of communication; but also the gaps & differences in priorities, release schedules, code ownership, and—perhaps most of all—personal acquaintance; I just didn't know the people, or know who to ask.

Conway's Law vs Distributed Working

Mark Seemann has recently argued that successful, globally distributed, OSS projects demonstrate that co-location isn't all it's claimed to be. Which set me thinking about communication in OSS projects.

In my example above, I had no ownership (for instance, no commit rights) to back end code and I didn't know, and hence didn't communicate with, the people who did. The tools of OSS—a shared visible repository, the ability to 'see' who is working on what, public visibility of discussion threads, being able to get in touch, to to raise pull requests—all serve to reduce the cost of communication.

In other words, the technology helps to re-create, at a distance, the benefits enjoyed by co-located workers.

When thinking of communication & co-location, I naturally think of talking. But @ploeh's comments have prodded me into thinking that code ownership is just as big a deal as talking. It's just something that we take for granted in a co-located team. I mean, if your co-located team didn't have access to each other's code, what would be the point of co-locating?

Another big deal with co-location is "tacit" knowledge, facilitated by, as Alistair Cockburn put it, osmotic communication. When two of my colleagues discuss something, I can overhear it and be aware of what's going on without having to be explicitly invited. What's more, I can quickly filter out what isn't relevant to me, or I can spontaneously join conversations & decisions that do concern me. Without even trying, everyone is involved when they need to be in a way that someone working in a separate room–even one that's right next door–can't achieve.

But a distributed project can achieve this too. By forcing most communication through shared public channels—mailing lists, chatrooms, pull request conversations—a distributed team can achieve better osmotic communication than a team which has two adjacent rooms in a building.

The cost, I guess, is that typing & reading is more expensive (in time) than talking & listening. Then again, the time-cost of talking can be quite high too (though not nearly as a high as the cost of failing to communicate).

I still suspect that twenty people in a room can work faster than twenty people across the globe. But the communication pathways of a distributed team can be less constrained than those same people in one building but separated even by a flimsy partition wall.

References