Mastering Software Architecture
I haven’t posted much this year as I have spent the majority of my writing time working on my upcoming book, Mastering Software Architecture. I want to share an excerpt in the form of the preface to the book which describes its goals and approach. If this resonates, you can pre-order using the link above or sign-up here for updates and more.
Preface: Holism in Software Architecture
“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”
R. Buckminster Fuller
An undergraduate course on applied mathematics will focus on general principles and concepts, simplifying the complex by reducing the dimensionality of the problem space. We simplify for the sake of theory, overlooking the true complexity of reality. A mass might be attached to a “light, inextensible string.” There are only perfect pulleys, frictionless planes, and systems always operating in a vacuum.
It has often been said that “software architecture cannot exist in a vacuum” yet that vacuum remains one that is notoriously difficult to escape. This fact is compounded by the common approach of many works in this field, focusing only one or a small number of aspects of the field in the abstract. In reality, architecture forms part of a living, breathing ecosystem of humans, technologies, networks, machines, customers, dreams, and aspirations. Applying architecture requires making the abstract concrete and designing models that integrate all the fragments of architecture theory as well as the messiness of the reality that architecture must exist within.
Historically it has only been possible to connect these discrete pieces into a much larger understanding over the course of a lengthy career with a checkered record of successes and failures. A more comprehensive and holistic look at this field is long overdue, and this integrated view is the ambitious goal of Mastering Software Architecture.
Of all the various engineering disciplines that have emerged over the course of human history, software is arguably the youngest by a substantial margin. Civil, mechanical, and military engineering evolved over millennia. Chemical and electrical engineering span centuries. In contrast, software engineering has only been around for a handful of decades. We still have much to learn and discover.
It was only as early as 1975 when the first notions of structured software began to enter the industry lexicon. At that point, proto-architecture began to emerge as a unification of software engineering and systems engineering. In 1976, a handful of forward-looking individuals saw a future marked by increasingly complex software systems composed of numerous components built and maintained by multiple teams. These Pioneers in the software development space began to explore ideas around system components, modularity, and higher-level conceptual descriptions of software systems. A changing world also required changing software, so increasing effort went into novel approaches to optimally structure code for understandability, maintainability, and evolvability. By 1990 the first books with an explicit focus on what we now call software architecture appeared and the industry soon believed it had found its silver bullet. Yet we seem doomed to discover again and again that, as Fred Brooks asserted in 1986 and Roy Fielding reiterated in 2000, there are no silver bullets. Unfortunately, silver-bullet thinking still permeates our industry.
In software architecture, there are no best practices, there are no universal and objective ‘right answers.’ There are only trade-offs. The weight of this fact is so significant, that Neal Ford and Mark Richards codified this as the First Law of Software Architecture.
Designing systems today requires practitioners to evaluate many decisions, weigh many trade-offs, and arrive at a locally optimal design for a given project, system, subsystem, or component relative to the time of decision (although a system’s needs will change in the future). The decisions and trade-offs span many dimensions, from the technological to the human and from the environmental to the organizational. For the field of software architecture to continue to evolve, new models must be applied that take a more holistic perspective. The individual patterns, technologies, practices, and tools have value and continue to be necessary, but they have proven to be insufficient in isolation at making an architect effective. Consider the headwinds today’s architects face. Since Brad Cox first suggested there might be a “silver bullet” we have learned that many paths towards system design are available, and each path yields different outcomes. An outcome that is best for one project will be sub-optimal for another. Different systems require different sets of architectural characteristics and inherent system capabilities. These capabilities must originate from business requirements and needs, which are never communicated in the domain-specific language and idiom of the architect or programmer. Moreover, where these capabilities are not expressed, they must be inferred. If we fail at this foundational task, it is impossible to be effective as an architect.
Even if an architect can correctly infer these architectural requirements, if the architect’s metaphorical quiver only contains a relatively small number of patterns and potential implementations while lacking a more sophisticated and nuanced set of tools and mental models to derive architectures rather than shoehorning existing patterns into the problem, their efficacy will be severely constrained.
Assuming the architect can perfectly design a target architecture this, too, is not enough. Their vision and architecture must be communicated with high-fidelity, such that implementation teams may understand and execute effectively. If the most vital details of the design are lost in translation, even an optimal architecture for a system will be moot. Executing architecture within an organization provides yet another challenge. Virtually every decision an architect makes will be challenged. Many knowledgeable and experienced individuals are responsible for implementing any given project. These individuals may have different ideas around how the system should be built. The architecture may not be compatible with existing organizational biases, preferences, norms, and conventions; yet the macro system must be cohesive which requires adherence to architectural standards and conventions. To be effective, the architect must not only be skilled in the art of requirements analysis and system design, but also be an equally skilled communicator and change agent. If we are not able to build consensus that spans project stakeholders and teams, much of the design work will be for naught. Finally, the architect must be aware of messy realities that can be easily overlooked in theoretical discussions of architecture yet cannot be ignored in the practice of delivering software. These are external factors such as the nature, structure, and maturity of the organization; the skills, maturity, and practices of the teams; and the factors governing the environment within which the project exists.
Architecture is no longer as simple as a set of best practices to organize complex codebases, or modeling tools to describe a system at a high-level. It is not just about the newest patterns that have emerged over the past 20 years. There are many crucial aspects of software architecture beyond the ‘what’ and ‘how’ that require further exploration. In short, for our field to continue to evolve, we must embrace the idea of holism in our approach to architecture. The work that follows is an ambitious attempt to do exactly that.