Architectural Metapatterns

Too much information often leads to confusion, or as the saying goes, if it isn't remembered, it never existed. In the realm of software and system architecture, countless patterns exist. It's impossible for anyone to know them all, and most people don't even try (if you claim to know them all, you might be referencing the archives of the Pattern Languages of Programs—have you really? Neither have I). Conferences alone produce hundreds of new patterns each year, alongside numerous publications and online resources. Older patterns are frequently rebranded, overlooked, or reinvented. A prime example is how the modern N-tier architecture is simply a rehash of the classic Layers concept.

This phenomenon undermines the foundational ideas that sparked the initial excitement around patterns:

1. Patterns as a common language. Nowadays, similar patterns often have different names, and some are so obscure they may never be recognized (refer to the PLoP archives).
2. Patterns as a means of knowledge transfer. When an old pattern is either reinvented or copied, much of its original knowledge is lost, resulting in a discontinuity of understanding.
3. Pattern language as an essential tool for architects. As patterns get reinvented, so do the languages associated with them. Currently, we see only domain-specific or architecture-limited systems of patterns (like DDD or microservices). The unified vision that early pattern advocates promised is nowhere to be found.

Have we been misled?

Metapatterns

Is there a way to bring some order to this overwhelming variety of patterns? There are simply too many, some very specialized, others obscure.

We can attempt this within a subset, which should be: - Significant enough to be relevant to most programmers. - Concise enough to be memorable or fit within a book. - Thorough enough to ensure we don’t overlook any critical elements.

Is such a set achievable? I believe it is.

Architectural Patterns

[POSA1] categorizes patterns into three main types: - Architectural patterns: These address the overall system structure and the roles of its components. - Design patterns: These focus on the relationships between objects. - Idioms: These provide abstractions relevant to specific programming languages.

Architectural patterns are inherently important ("Architecture is about the significant elements"). The first criterion (importance) is satisfied.

Every system possesses an internal structure. When developers refer to an "architectural style" [POSA1] or create structural diagrams, they often boil down to a combination of two or three recognized architectural patterns. By focusing our study on architectural patterns, we can draw from a wealth of literature that repeatedly describes similar designs. Furthermore, if a system deviates from current trends, it's often labeled as innovative, while its designers may be deemed outdated; thus, we can reasonably expect to be familiar with nearly all practical architectures. The third criterion (completeness) is also fulfilled.

To systematically organize this set of patterns, we will utilize the concept of:

Design Space

The design space [POSA1, POSA5] serves as a framework that allocates dimensions for every choice made in system architecture. It embodies all potential design avenues. The challenge, however, is that it is multidimensional and potentially infinite, with varying dimensions applicable to different systems.

One solution is to project the design space into a 2D or 3D framework that is more manageable for us. However, such projections inevitably lead to a loss of information. Interestingly, this can work in our favor—similar architectures, differing only in minor details, can appear identical once the distinguishing dimensions are removed. If we can identify two or three critical dimensions applicable to each architectural pattern we wish to examine, we can cover all known system designs.

Structure Defines Architecture

Systems typically have an internal structure. Those that lack one are derogatorily termed "Big Balls of Mud" due to their chaotic nature. Structure revolves around modules, their functions, and their interactions. Many architectural styles, such as Layers or Pipeline, derive their names from their structures, while others, like Event-Driven Architecture, highlight specific aspects that suggest structure fundamentally dictates a system's principal properties.

I am not the first to arrive at this conclusion. Metapatterns—clusters of similarly structured patterns—were identified soon after the initial design pattern collections emerged, but they never made a significant impact in software engineering. I believe this approach was prematurely applied to analyze [GoF] patterns, which represent an arbitrary and incomplete subset of design patterns, leading to overgeneralization. My goal is to chart the structures of the complete set of architectural patterns, grouping those with identical structures into metapatterns, and illustrate how a system's structure determines its properties. Quite an ambitious undertaking for a brief book, isn't it?

Our collection of architectural patterns is still incomplete and not particularly small. Moreover, the method of drawing structural diagrams varies widely, preventing direct comparisons unless we devise a universal:

System of Coordinates

Creating a generic coordinate system applicable to any pattern representation, from Iterator to Half-Sync/Half-Async, may prove challenging. However, we can certainly find something suitable for architectural patterns, as they all share the same scope—the system in its entirety. What dimensions would typically be utilized to plot a system's implementation?

1. Abstractness—This dimension spans high-level business logic to low-level details. A single highly abstract operation unfolds into numerous lower-level operations: Python scripts operate above a C runtime and assembly drivers, while orchestrators invoke API methods of services that themselves execute SQL queries against databases filled with low-level computations and disk operations.
2. Subdomain—Complex systems manage multiple subdomains. For instance, an operating system must handle various peripheral devices and protocols: a video card driver bears little resemblance to an HDD driver or the TCP/IP stack. Similarly, an enterprise comprises multiple departments, each utilizing software tailored to their unique needs.
3. Sharding—If multiple instances of a module are deployed, and this is integral to the architecture, we must represent these instances in the structural diagram.

We will position the abstractness axis vertically, with higher-level modules at the top, the subdomain axis horizontally, and sharding diagonally. Here is an example of such a diagram:

Map and Reduce

Now that we have established generic coordinates that seem to accommodate any architectural pattern, we can begin mapping our collection of architectural patterns into this coordinate system—effectively reducing the multidimensional design space into the few dimensions represented in our structural diagrams. After filtering out minor details, our collection of approximately one hundred published patterns should yield several clusters of geometrically equivalent diagrams—since only a limited number of simple systems can be illustrated before redundancy occurs. Each cluster will represent an architectural metapattern—a generalization of architectural patterns with similar structures and functions.

Returning to our classification criteria, we previously established the importance (point 1) of architectural patterns. The reasonable size of the resulting classification (point 2) is ensured by the existence of only a few simple 2D or 3D diagrams (metapatterns). The completeness of our analysis (point 3) arises from the geometric approach that makes any gaps (potential geometries with no known patterns) evident, combined with a substantial sample of architectural patterns we are categorizing.

Godspeed!

An Example of Metapatterns

Let’s consider the following structure:

This configuration features two (or more in real scenarios) high-level modules that communicate with or through a lower-level module. Which patterns align with this structure?

• Middleware—software facilitating communication among other components.
• Shared Database—a space allowing components to store and exchange data.
• Model-View-Controller—platform-agnostic business logic with tailored input and output methods.

The concept of categorizing patterns by their structure may have backfired—resulting in three distinct patterns sharing similar structural diagrams. The first two are related; both facilitate indirect communication, and their differences blur as a middleware can feature persistent message storage, while a table in a shared database may orchestrate services. The third pattern diverges significantly due to its business logic residing primarily in the lower layer, relegating the upper-level components to a minor role.

Nonetheless, each identified pattern belongs to a distinct cluster:

• Middleware is also referred to as (Message) Broker [POSA1, POSA4, EIP, MP] and is crucial to systems like Message Bus [EIP], Service Mesh [FSA], Event Mediator [FSA], Enterprise Service Bus [FSA], and Space-Based Architecture [SAP, FSA].
• Shared Database functions as a type of Shared Repository [POSA4] (including Shared Memory and Shared File System) and forms the basis for Blackboard [POSA1, POSA4], Space-Based Architecture [SAP, FSA], and Service-Based Architecture [FSA].
• Model-View-Controller [POSA1, POSA4] represents a specific type of Hexagonal Architecture (also known as Ports and Adapters, Onion Architecture, and Clean Architecture), which itself derives from Plugins [PEAA] (including Add-ons, Plug-In Architecture [FSA], and Microkernel Architecture [SAP, FSA]).

Analyzing a single structural geometry has unveiled a network of approximately twenty pattern names interconnected. At this rate, we hope to explore the entire fabric known as pattern language [GoF, POSA1, POSA2, POSA5].

Three lessons emerge from this exploration:

1. The distribution of business logic is a key component of structural diagrams.
2. Metapatterns are interconnected in numerous ways, forming a cohesive pattern language.
3. Each metapattern encompasses several well-established patterns.

What Does That Mean

Chemistry has the periodic table, and biology has the tree of life. This book aims to create something analogous for software and system architecture. You might argue, "This is nonsensical! Chemistry and biology are empirical sciences, while software architecture is not!" But is it?

>> The list of architectural metapatterns

References

• [EIP] Enterprise Integration Patterns. Gregor Hohpe and Bobby Woolf. Addison-Wesley (2003).
• [FSA] Fundamentals of Software Architecture: An Engineering Approach. Mark Richards and Neal Ford. O’Reilly Media, Inc. (2020).
• [GoF] Design Patterns: Elements of Reusable Object-Oriented Software. Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Addison-Wesley (1994).
• [MP] Microservices Patterns: With Examples in Java. Chris Richardson. Manning Publications (2018).
• [PEAA] Patterns of Enterprise Application Architecture. Martin Fowler. Addison-Wesley Professional (2002).
• [POSA1] Pattern-Oriented Software Architecture Volume 1: A System of Patterns. Frank Buschmann, Regine Meunier, Hans Rohnert, Peter Sommerlad and Michael Stal. John Wiley & Sons, Inc. (1996).
• [POSA2] Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects. Douglas C. Schmidt, Michael Stal, Hans Rohnert, Frank Buschmann. John Wiley & Sons, Inc. (2000).
• [POSA4] Pattern-Oriented Software Architecture Volume 4: A Pattern Language for Distributed Computing. Frank Buschmann, Kevlin Henney, Douglas C. Schmidt. John Wiley & Sons, Ltd. (2007).
• [POSA5] Pattern Oriented Software Architecture Volume 5: On Patterns and Pattern Languages. Frank Buschmann, Kevlin Henney, Douglas C. Schmidt. John Wiley & Sons, Ltd. (2007).
• [SAP] Software Architecture Patterns. Mark Richards. O’Reilly Media, Inc. (2015).

This is a chapter from my book of the same name. Any feedback is welcome!