Tag Archives: software

Introducing: The Semantic Field. Or, The One Truly Correct Usage of Layered Architecture in the World

Bear with me if you already abandoned layered architecture long ago. You may be quite familiar with the thought that layered architectures often fail to apply the Dependency Inversion principle, and often thereby induce tight coupling of un-modular, un-testable layers.

I wish to do two things in this post. First, I propose that the notion “Semantic Field” better captures the one big idea that layered architecture nearly gets right. Second, I will discuss the One Truly Correct Usage of Layered Architecture In the World in order to show why it’s the wrong choice for nearly all other usages.

Semantic Field

“Semantic Field” or “Semantic Domain” is a term from linguistics. Words are in the same semantic field if they are related to the same area of reality. (The word domain is pretty much what you’d call it as a DDDer). Orange is in the same semantic domain (let’s call it the fruit domain) as Apple. But it’s also with Red in the semantic domain of Colour, whereas Apple isn’t. That’s how natural language rolls.

Kent Beck used the term conceptual symmetry to explain why he didn’t like this code snippet:

void process(){
input();
count++;
output()
}

and wants to change count++; into tally();. Somehow the count++ doesn’t seem to be on the same level as the method calls. Indeed it isn’t. It’s the same feeling you have when you see:

void applyToJoin(Customer customer){
    if(eligibilityRules.validate(customer)){
        membershipList.accept(customer);
        htmlBtnUpdate.setEnabled();
    }
}

that a method dealing in business rules and processes should not also know about html buttons. Semantic Field is the notion we want here. The clean code rule is “One Level of Abstraction per Function” and I propose to rename it as “One semantic field per method”. In fact, one semantic domain per class, namespace, module, or … layer.

This is what layers gets right: The idea that inside a given layer you understand a specific semantic domain, and don’t use vocabulary from the semantic fields of the layers above or below you.

Where layers goes wrong is, well, the layering. The belief that all top-level dependencies in a system can be expressed in one dimension, top to bottom. They just can’t. Squeezing your code into 1 dimension makes you do contortions that are utterly unhelpful. Strict layering adds to this a second failure mode: It makes you write pointless passthrough code, which ought to be deleted.

(Layering does get a second thing right: no cyclic dependencies. Code with mutual dependencies will try to morph into ball of mud architecture. I’m sure this is half the reason why layered architecture become wildly popular. It was a vast improvement on ball-of-mud).

The One Truly Correct Usage of Layered Architecture In The World

The other reason we were entranced by layered architecture for a decade was the ISO OSI 7 layer model for networking. It seemed so obviously, thoroughly, beautifully, correct.
OSI 7 layer architecture

Each layer is clearly (well, it was clear up to about layer 5, after that it got a bit hazy for some of us) and cleanly independent of the other layers. Each layer is a different semantic domain. The bottom layer deals with physical connectors and with what voltage represents a 1 or a 0 and how a byte sequence is encoded as an electrical waveform. The next layer deals with packets, complete with a destination and a source. The next layer deals  in routes: how to get to this destination from the source. The next layer deals in messages: how to turn them into packages and back again. And so on.
And, the layer-cake picture precisely models the dependencies between the layers. At least to layer 5, each layer relies on and adds value to the layer beneath it.

It was beautiful. It made sense. It was what I wanted my software to look like. It was a siren, luring us all to shipwreck our software on the rock of a beautiful but evil vision of how it should always be.

Why a Layered Architecture is Nearly Always Wrong For Any Other Software System

The bit that isn’t wrong

The part of the OSI model that is applicable to 99% of all known software is the separation into semantic fields. This is why we used to say that business logic shouldn’t be in the UI layer; html buttons live in a different semantic domain to customers and invoices. (Except: it was the wrong way to put it. The presentation layer does reference business logic because in an interactive system usability is achieved by having the UI reflect the business logic; for instance by hiding options that are not valid for the current user).

The bit that fails miserably

The part of the OSI model that is applicable to very very few systems is the layering. In the OSI architecture the strict layering works because the language of each layer can be defined in terms of layers beneath it. Session, Frame, Bit are in separate semantic domains, but the model allows Frame to be defined in terms of Bit, Session in terms of Frame, and so on.

This is almost never the case in layered business software. The vocabulary of a UI cannot be defined in terms of the vocabulary of commerce and business administration, and the vocabulary of a business cannot be defined in terms of data entities. They just are separate domains. The fact that that second one sometimes works a little bit (you can define a customer–incorrectly–as rows in data tables) is what seduces you into thinking it should work. But it doesn’t. You cannot define your business in terms of a data layer.

In particular then, a layered architecture with UI on top is always wrong; and business layer on top of data layer is always, but less obviously and more seductively, wrong. Hexagonal architecture (aka ports and adapters) is a much better model for most systems because it doesn’t confine dependencies to a single dimension (in addition to the already well-known fact that it gets your dependencies pointing the right way).

DDD: Treating the UI layer as a domain

Having recognised that user interface is a separate semantic domain, should we apply some DDD thinking and treat it as a bounded context with it’s own domain? The domain of an MVC web UI includes controllers, actions, routes, etc. But it must reference business logic all the time when deciding what to display, whether to accept user input, and ultimately to do anything with that input. To some, making the UI layer it’s own domain context, and giving it adapters to interface with the business domain seems like over-engineering, whilst others advocate almost exactly that.

I recommend that you should at least be aware that if you do not do this de-coupling (and in MVC web apps I personally almost never have) then your UI layer will have two semantic domains inside it. It’s a trade-off, but a sufficiently small one that I would usually come down in favour of which side has fewer total lines of code.

A Single Deployment Target is not a Monolith

Since the dawn of software architecture, we mostly knew that ‘Monolithic Architecture’ was more or less a synonym for ball of mud architecture. It mostly referred to code which was so entangled and coupled that it could not be separated out into components (or layers, or modules, or libraries, or equivalent). God classes, lack of information hiding, a change in one place means 20 other changes. That kind of thing. A failure of Modularity.

Recently the term Monolith has been taken to mean having a Single Deployment Target at runtime. This is a quite different meaning.

If you think that Monolith as described in para 1 above is the same as Monolith in para 2 above, then I suggest that you have confused, not separated, your concerns.

This is easy to understand if you are comfortable with architecture views. The para 1 definition is about the logical and (in 4+1 lingo) development views: the structure and relationships of classes, components, packages, and other kinds of modularity. The para 2 definition is only about the runtime deployment view (in 4+1: physical view).

The point is that you are at liberty in pretty much any operating system, runtime or language devised in the last 30 years, to structure your code and components as carefully and modularly as you like, whilst choosing your runtime deployment scenario independently of that modularity: it’s okay for 2 uncoupled components to run on the same machine. Honestly. *nix does it all the time. Ooh, so does Windows. And .Net and Java and Android and iOS and ….

The CTO at intilery.com showed me a couple of years ago how their server codebase can be deployed as either a single .war for a single webserver or split as separate .wars for separate machines by flicking a switch in the build config.

It’s not rocket science, it’s Separation of Concerns: the codebase is not the runtime.

DistributedMethodCallError: The belief that calling across a network is better than calling within a process

Distributed Method Call Error
The belief that methods and functions communicating across a network is somehow better than communicating within a single process on a single machine.

Process this error by politely throwing a verbal exception, inquiring as to what, exactly, is better. And then explain how the answers you’re getting back are the wrong answers.

Here are templates for the three main areas on why a distributed architecture does not make X better:

If X is one of: Response
Separation of Concerns, Coupling, Cohesion or similar But X is not primarily about deployment scenarios, so distributing your deployment does not improve X.
Reliability, Performance, Robustness or similar But as you’ll know from the Fallacies of Distributed Computing, if not from bitter experience, distributed computing makes things harder not better.
Deployability, continuous deployment or integration But deploying to multiple hosts is harder, not easier, than deploying to a single host.

Yes, there are problems for which distributed computing appear to be part of a solution. Redundancy as a reliability tactic appears to push you to distributed computing. So does horizontal scaling as a performance or capacity tactic. But these things are done extremely well at the infrastructure level by a black box: a load balancer. Notice that load balancing does not force a decision on whether each instance of your application-or-service is deployed entirely on a single box or is distributed.

So if you think that microservices or any other form of distributed deployment address issues such as dependency management, coupling, cohesion, continuous deployment, avoiding domino failure, then may I put it to you that you have confused, not separated, your concerns. In 4+1 terms, you may be confounding your physical and process models (i.e. the runtime models) with your logical & development (‘coding-time’) models. As Simon Brown pithily put it, “if you can’t build a structured monolith, what makes you think microservices are the answer?”.

PS

Yesterday I read a blog about ‘Monolithic’ architecture which said – with pictures and everything – that if your problem is how to carry on coding effectively (add new features, fix bugs, resolve technical debt etc) as the size and complexity of the code base increases, then the solution is a distributed deployment architecture with synchronous calls over http using xml or json.

I could weep. You manage your codebase by managing your codebase! Not by making your runtime deployment and threading models 50 times more complicated. This is what I mean by confounding the logical & development models with the process & deployment models.

PPS

If you’re not familiar with the 4+1 architecture views: The

  1. Logical view describes your classes’ behaviour and relationships; the
  2. Development view describes how software is organised and subdivided into e.g. modules, components, layers, and how they are turned into deployable artefacts; the
  3. Process view describes what threading and processes and distributed communications you use and the
  4. Physical view (though I’d rather call it the Deployment view) describes what machines run what code on the running system

The ‘+1’ is the use cases, or user stories.

When I first saw 4+1 I only really ‘got’ the logical view. As years passed, I realised that this reflected a lack of experience on my part. When you first do distributed or asynchronous computing, you begin to see why you’d want a process view and a physical (or deployment) view.

4+1 is quite long in the tooth and has evolved in use. There are other sets of viewpoints. Rozanski & Wood’s seven viewpoints show the benefit of a decade’s more experience. You may think 7 is a lot, but for a small or simple system some of them need only be a sentence or two.

Estimates and NoEstimates

We had a debate&discussion at XP-Man on NoEstimates for which I did some notes. Reading the NoEstimates stuff, I was most attracted to the sense of “Let’s not be satisfied with second rate” and of a thirst for continuous improvement.
I was left with the sense (possibly because I already believed it) that there are contexts in which NoEstimates works, and contexts in which it doesn’t. But I was very glad to be provoked to ask in each case, “What value if any is our estimate/planning effort adding?” and “Isn’t there a better way to deliver that value?”

What is an Estimate?

An estimate for a Project is (1) a list of things to work on; (2) a cost-range for those things; and (3) a list of risks, that is (3a) of Dependencies that 1&2 rely on, and (3b) of things that might cause significant change.

An estimate or plan for a sprint is (1) a list of things to work on, (2) a “cost” (eg story points) for those things and (3) a list of things we are uncertain about, or (4) need to get help with.

The value of a project estimate is to feed-in to (1) A go/no-go decision and (2) seeing things we want to see sooner rather than later (e.g. should we hire more people, do we need help from specific 3rd parties, is releasing in time for Christmas possible)

The value of a sprint estimate is, to see things we need to ask for in advance (ie external help or resources); to give everyone a sense of confidence about what we’re doing; to fail-faster, that is to see sooner what we can’t achieve.

When Agile isn’t Agile

Reading I Fear Our Mobile Group Being Forced To Follow Scrum crystallised in my mind what can go wrong when you treat Agile as a methodology. It describes a team successfully using kanban which is to potentially be required to use scrum — because that’s becoming the company standard.

Making a team follow an agile methodology is exactly *not* Agile.

Agile is “Individuals and interactions” being valued more highly than processes. Imposing Scrum looks like valuing the process more than the team.

Agile is “self-organising teams” and letting “the team [reflect] on how to become more effective, then tune and adjust accordingly.” Imposing conformity on a team that has already adjusted is a backwards step; you’re asking a team that has optimised somewhat for the individuals in the team to de-optimise again.

This doesn’t mean that you can’t teach an agile team anything. The manifest starts with “We are uncovering better ways of developing software by doing it and helping others do it.” A team that can’t be corrected, or won’t learn better ways, isn’t agile. For that matter, a team that won’t learn in any walk of life has started the downhill path to decline.

For what it’s worth, I’m sure that a competent lean team that tries Scrum for a while will learn from it, even if they end up optimising back to something more fluid.