A Rational Emergent Architecture: Fake it!

One aspect of the agile approach to software development that puzzles a lot of people is the idea of "emergent architecture." The idea that the architecture of a system can just "emerge" out of the development process seems strange enough to them; but if they are working in mission-critical sectors that include, say, safety-critical embedded systems, then the idea of emergent architecture seems like a showstopper.

The word "emergent" is anathema in the mission-critical domain, because emergent behavior can wreak havoc on all kinds of things – in particular, it makes it difficult to construct systems out of reusable components, because you keep having to do the validation effort over and over again at higher levels (there's a lot of work in that area going on right now).

Part of the problem, however, is that people are associating emergent architecture with emergent behavior. But "emergent architecture" is talking about the way the architecture is developed, not the way it behaves once it's developed.

It would be a good idea to re-read a paper written several years ago by Parnas and Clements called A Rational Design Process: How and Why to Fake It. In this paper, they argue that we never really follow the ideal, rational design process, for all the reasons we know about. But that doesn't mean that the finished product has to be irrational. On the contrary, it continues to be important to document it as though it had been the product of a rational design process. Consider this:

Mathematicians diligently polish their proofs, usually presenting a proof very different from the first one that they discovered. A first proof is often the result of a tortured discovery process. As mathematicians work on proofs, understanding grows and simplifications are found. Eventually, some mathematician finds a simpler proof that makes the truth of the theorem more apparent. The simpler proofs are published because the readers are interested in the truth of the theorem, not the process of discovering it. Analogous reasoning applies to software. Those who read the software documentation want to understand the programs, not to relive their discovery.

The principle of "emergent architecture" gives developers a chance to find the best possible architecture – like in the mathematical proofs described above, the well-understood and simplest architecture for the system at hand. During this process, they may well also make use of the same principles that others use to combat emergent behavior (e.g. "correctness by construction").

Once the architecture is there, people don't want to relive the process of discovery, they just want to know what's there; then it can be described in a rational way using the best practices for architecture description today, such as those of the IEEE 1471 Standard.

Architects of mission-critical systems need efficient, well-understood architectures as much as anybody else, and it would be a pity if they avoided the ideas about emergent architectures coming out of the agile community because of a misunderstanding.