Where Architecture Lives

Here’s a handy trick for testing an engineer’s seniority level - ask them to explain the architecture of a system they’re working on or have designed without mentioning any specific technologies.

Start them with, say, 10 or 20 points, and deduct a point for every time you hear a specific technology like database, or event bus. Deduct 2 points if you hear a specific piece of software like Kafka or Kubernetes. Deduct 5 points if you hear a specific programming language.

Ok, I’d probably fail this test too without any context! We are normally primed to speak in these technical terms with each other. But challenge yourself, anyway, see how you go!

My point is: none of those things are architecture. They’re implementation details.

Architecture lives one level above that - it lives in the concepts that make up the system, and the assumptions those concepts make about their environment.

The distinction sounds subtle, but in my experience that’s the difference between systems that remain coherent for years and systems that slowly dissolve into unnecessary complexity.

I also think “architecture” is the wrong word - but we’ll set that aside, for now.

Concepts Before Technology

Let’s “architect” something together - an online store. If we’re building a store, we’re going to need some orders. Let’s make that our first concept.

Before we write any code, choose a database, create an API, or provision a cloud account, we need to figure out what exactly is an order. Let’s have a go at a definition.

An Order represents a customer’s intent to purchase something.

Ok, not bad. Already we are seeing new things we’ll probably want to define more thoroughly eg. customer, intent, purchase and of course… something… but let’s just, for now, use the dictionary definitions since we don’t have time to actually map out our entire Shopify-killer.

What assumptions does our architecture make about orders? Asked another way, what is always true about orders? (You may use the words “assumptions”, “constraints”, “invariants” - these all can be used to describe what we’re effectively getting at here):

• An order belongs to one customer
• An order may contain one or more line items; but never zero
• An order has a status that changes over time

Notice we haven’t discussed SQL, microservices, queues, hybrid web / mobile technology choice - it’s just a list of things we can all assume about orders, within this architecture.

You might say this isn’t architecture, it’s a list of requirements! And I say (preparedly, and in response to my contrived reader in order to make myself feel very smart) that it’s not a list of requirements because these describe a solution - requirements describe a problem to be solved.

These assumptions are the solution architecture. And as long as those assumptions remain true, and consistent with each other, then the architecture remains coherent. The architecture survives because the concepts survive.

What matters isn’t the technology that implements this solution, what matters is whether the technology continues to satisfy the assumptions that the architecture was built upon. And this is where architecture actually lives.

Not in the implementation, but in the relationship between concepts and assumptions.

Even when you take something off-the-shelf like microservice architecture it is exactly the same thing - it’s a solution to the problem of building independently modifiable, testable, deployable and executable systems in very large engineering orgs. And it doesn’t specify technologies, at all - it lives in the realm of concepts and assumptions, just like all good architecture.

You could write it out much the same way:

• A microservice is an independently operable piece of software that is wholly owned by a single team
• A gateway acts an entry point into the system, exposing public operations

And so on. I’m not an expert in microservice architecture (thankfully) so take this point as illustrative: architecture lives in the realm of concepts and assumptions, and it describes a solution not a problem.

How Architecture Loses Coherence

Once you start looking at systems this way, architectural decay becomes much easier to spot.

Most systems don’t become difficult because the code is bad - they become difficult because the concepts start to drift away from their initial assumptions.

An order starts meaning one thing in one team and something slightly different in another. A user might now be a human or an agent. A campaign now could mean marketing or military action. The boundaries between concepts become blurry, and every new feature introduces another exception to compensate.

Eventually the implementation starts reflecting that confusion: more conditionals appear, more special cases emerge, the comment blocks get longer and more sweary.

The problem is often conceptual rather than technical - the architecture has lost coherence because nobody can clearly explain the concepts anymore.

That’s why one of the most valuable questions you can ask about a system is:

What assumptions does this concept make about its environment?

If nobody can answer that question consistently, the architecture is already in trouble.

Solid test driven development can take you a long way, but tests are much better at telling you what than why. Keeping a document alongside your code that explains the concepts and assumptions relevant to that code is much more powerful, and allows people to change these over time in a way that ensures long term coherence of the architecture.

Writing for the Future

In typically rapid software development environments, a lot of technical documents are written for people who already understand the system, because the goal is often speed of alignment.

The author knows the concepts. The reviewers are in the same team, so they already get it. And, as a result, the concepts and assumptions are spread out over years of documents and conversations and nobody can see it all at once unless they’ve been there since the beginning.

A really good exercise for AI coding tools is ask it to write you this document, if you don’t have it. Skip the file structures, and the class list - that’s to speed up the agents - have it help you write out the genuine architecture; the ideas that hold your work together.

Instead of asking it to write you an agents.md have it write you a humans.md, for the people who aren’t in the room yet.

You will probably very quickly find your system lacks total coherence - this is normal. But this exercise will help you quantify it and document it in a way that it can genuinely be actioned, rather than living in your head as that thing you’ve always been meaning to fix.

Don’t make others reverse-engineer your mental model, give it to them! The documentation is a great exercise on its own, but the document itself will help you all preserve conceptual coherence over time. It doesn’t guarantee you won’t end up with spaghetti, but spaghetti is delicious anyway!

Architecture Is Theory

One of the stranger lessons of senior engineering is that architecture becomes simultaneously more important and less visible. The longer you do this, the less impressed you become by complicated architectural diagrams and the more interested you become in whether the underlying concepts are sound.

Architecture becomes less like a blueprint and more like a theory, in the scientific sense. The problem you are asked to solve is like the real world phenomenon a scientist may be studying, and your architecture is like the theoretical model the scientist uses to explain their observations.

Just as a good scientific theory goes beyond simply explaining past observations and offers explanatory and predictive power that can be tested against new phenomena, a good architecture is able to extend beyond today’s problems and incorporate tomorrow’s with minimal disruption. And just like in science, this is more art than science - you shouldn’t beat yourself up because you weren’t able to predict the future.

In practice, the more resilient architectures tend to keep concepts smaller and assumptions fewer. This works better in the long run, since new additions to the architecture have less to “compete” with. This leads to fewer decisions made prematurely, and a much stronger shared understanding of trade-offs being made along the way.

Great minds have converged on this idea in many ways: Occam’s razor, YAGNI, KISS, MDL, MML, less is more, and so on.

What Have We Learned

Most systems don’t need newer or different technology - they need clearer concepts; they need clearer assumptions. This is where engineers should be having more conversations, and more debates.

The database, the queue, the event bus, the language, the platform - these are not architecture. These are all replaceable.

The thing that survives every rewrite, migration and platform transition is the conceptual model, the theory, of the solution underneath.

That’s where architecture lives - everything else is just details.