Episode 2: the Controller Strikes Back

This post should have been about power law distribution of class / method sizes, organic growth of software and living organisms, Alexandrian level of scales, and a few more things.

Then the unthinkable happened. Somebody actually came up with a comment to Life without a controller, case 1, said my design was crappy and proposed an alternative based on a centralized, monolithic approach, claiming miraculous properties. I couldn’t just sit here and do nothing.

Besides, I wrote that post in a very busy week, leaving a few issues unexplored, and this is a good chance to get back to them.

I suggest that you go read the whole thing, as it’s quite interesting, and adds the necessary context to the following. Then please come back for the bloodshed. (Note: the original link is broken, as the file was removed; the "whole thing" link now points to a copy hosted on my website).

The short version
If you’re the TL;DR kind and don’t want to read Zibibbo’s post and my answer, here is the short version:
A caveman is talking to an architect.
Caveman: I don’t really like the architecture of your house.
Architect: why?

Can we really control the quality/cost trade-off?

An old rule in software project management (also known as the Project triangle is that you can have a system that:
- has low development cost (Cheap)
- has short development time (Fast)
- has good quality (Good)
except you can only choose two :-). That seem to suggest that if you want to go fast, and don't want to spend more, you can only compromise on quality. Actually, you can also compromise on feature set, but in many real-world cases, that happens only after cost, time and quality have already been compromised.

So, say that you want to compromise on quality. Ok, I know, most engineers don't want to compromise on quality; it's almost like compromising on ethics. Still, we actually do, more often that we'd like to, so let's face the monster. We can:

- Compromise on external quality, or quality as perceived by the user.
Ideally, this would be simply defined as MTBF (Mean Time Between Failures), although for many users, the perceived "quality" is more akin to the "user experience". In practice, it's quite hard to predict MTBF and say "ok, we won't do error checking here because that will save us X development days, and only decrease MTBF by Y". Most companies don't even measure MTBF. Those who do, use it just as a post-facto quality goal, that is, they fix bugs until they reach the target MTBF. We just don't know enough, in the software engineering community, to actually plan the MTBF of our product.

- Compromise on internal quality, or quality as perceived by developers.
Say goodbye to clarity of names, separation of concerns, little or no duplication of logic, extendibility in the right places, low coupling, etc. In some cases, poor internal quality may resurface as poor external quality (say, there is no error handling in place, so the code just crashes on unexpected input), but in many cases, the code can be rather crappy but appear to work fine.

- Compromise on process, or quality as perceived by the Software Engineering Institute Police :-).
What usually happens is that people start bypassing any activity that may seem to slow them down, like committing changes into a version control system :-) or being thorough about testing. Again, some issues may resurface as external quality problems (we don't test, so the user gets the thrill of discovering our bugs), but users won't notice if you cut corners in many process areas (well, they won't notice short-term, at least).

Of course, a "no compromises" mindset is dangerous anyway, as it may lead to an excess of ancillary activities not directly related to providing value (to the project and to users). So, ideally, we would choose to work at the best point in the quality/time and/or quality/cost continuum (the "best point" would depend on a number of factors: more on this later). Assuming, however, that there is indeed a continuum. Experience taught me the opposite: we can only move in discrete steps, and those are few and rather far apart.

Quality and Cost
Suppose I'm buying a large quantity of forks, wholesale. Choices are almost limitless, but overall the real choice is between:

- the plastic kind (throwaway)

it works, sort of, but you can't use it for all kinds of food, you can't use it while cooking, and so on.

- the "cheap steel" kind.

it's surely more resistant than plastic: you can actually use it. Still, the tines are irregular and feel uncomfortable in your mouth. Weight isn't properly balanced. And I wouldn't bet on that steel actually being stainless.

- the good stainless steel kind.

Properly shaped, properly balanced, good tasting :-), truly stainless, easy to clean up. You may or may not have decorations like that: price doesn't change so much (more on this below).

- the gold-plated kind.

it shows that you got money, or bad taste, or both :-). Is not really better than the good stainless fork, and must be handled more carefully. It's a real example of "worse is better".

Of course, there are other kind of forks, like those with resin or wood handles (which ain't really cheaper than the equivalent steel kind, and much worse on maintenance), or the sturdy plastic kind for babies (which leverage parental anxiety and charge a premium). So, the quality/cost landscape does not get much richer than that. Which brings us to prices. I did a little research (not really exhaustive – it would take forever :-), but overall it seems like, normalizing on the price of the plastic kind, we have this price scale:

Plastic: 1
Cheap steel: 10
Stainless steel: 20
Gold-plated: 80

The range is huge (1-80). Still, we have a big jump from throwaway to cheap (10x), a relatively small jump from cheap to good (2x), and another significant jump between good and gold-plated (4x). I think software is not really different. Except that a 2x jump is usually considered huge :-).

Note that I'm focusing on building cost, which is the significant factor on market price for a commodity like forks. Once you consider operating costs, the stainless steel is the obvious winner.

The Quality/Cost landscape for Software
Again, I'll consider only building costs here, and leave maintenance and operation costs aside for a moment.

If you build throwaway software, you can get really cheap. You don't care about error handling (which is a huge time sink). You copy&paste code all over. You proceed by trial and error, and sometimes patch code without understanding what is broken, until it seems to work. You keep no track of what you're doing. You use function names like foo and bar. Etc. After all, you just want to run that code once, and throw it away (well, that's the theory, anyway :-).
We may think that there is no place at all for that kind of code, but actually, end-user programming often falls into that case. I've also seen system administration scripts, lot of scientific code, and quite a bit of legacy business code that would easily fall into this category. They were written as throwaways, or by someone who only had experience writing throwaways, and somehow they didn't get thrown away.

The next step is very far apart: if you want your code to be any good, you have to spend much more. It's like the plastic/"cheap steel" jump. Just put decent error handling in place, and you'll spend way more. Do a reasonable amount of systematic testing (automated or not) and you're gonna spend more. Aim for your code to be reasonably readable, and you'll have to think more, go back and refactor as you learn more, etc. At this stage, internal quality is still rather low; code is not really stainless. External quality is acceptable. It doesn't feel good, but it gets [some] work done. A lot of "industrial" code is actually at this level. Again, maintenance may not be cheap, but I'm only considering building costs here; a 5x to 10x jump compared to throwaway code is quite reasonable.

The next step is when you invest more in -ilities, as well as in external quality through more systematic and deeper testing. You're building something that you and your users like. It's truly stainless. It feels good. However, it's hard to be "just a little stainless". It's hard to be "somehow modular" or "sort of thread-safe". Either you're on one side, or you're on the other. If you want to be on the right side, you're gonna spend more. Note that just like with fork, long-term cost is lower for good, stainless code, but as far as building cost is concerned, a 2X increase compared to "cheap steel" code is not surprising.

Gold-plating is obvious: grandiose design that serves no purpose (and that would appear as gratuitous complexity to a good designer), coding standards that add no value (like requiring a comment on each and every parameter of every function), etc. I've seen systems spending a lot of energy trying to be as scalable as Amazon, even though they only had to serve a few hundred concurrent clients. Goldplated systems spend more on building, and may also incur higher maintenance costs afterward (be careful not to scratch the gold :-). Of course, it takes experience to recognize goldplating. To someone that has never designed a system for testability, interfaces for mocking may seem like a waste. To an experienced designer, they tell a different story.

But software is... soft!
As usual, a metaphor can be stretched only so far. Software quality can be easily altered over time. Plastic can't be transformed into gold (if you can, give me a call :-), but we can release a bug-ridden version and fix issues later. We may incur intentional technical debt and pay it back in the next release. This could be the right strategy in the right market at the right time, but that's not always the case.

Indeed, although start-ups, customer discovery, innovative products, etc seem to get all the attention lately, if you happen to work in a mature market you may want to focus on early-on quality. Some customers are known to be forgiving about quality issues, in exchange for early access to innovative products. It's simply not the general case.

Modular Quality?
Theoretically, we could invest more in some "critical" components and less in others. After all, we don't need the same quality (external and internal) in every single portion. We can accept quirks here and there. Due to the fractal nature of software, this can be extended to the entire hierarchy of artifacts. It's surely possible. However, it takes a lot of maturity to develop on the edge of quality. Having to constantly reason about the quality/cost trade off can easily cost you more than you're saving: I'd rather spend that reasoning to create quality code.

Once again, however, we may have a plan to start with plastic and grow into stainless steel over time. We must be aware that it is easier said than done. I've seen it many times over my long career. Improving software quality is just another case of moving our software in the decision space. We decided to make software out of plastic early on, and now we want to move it to another place, say the cheap steel spot. Some work is required. Work is proportional to the distance (which is large – see above), but also to the opposing force. Whatever form that force takes, a significant factor is always the mass to be moved, and here lies the key. It's hard to move a monolithic mass of code into another place (in the decision space). It's easier if you can move it a module at time.

Within a small modular unit, you can easily improve local quality. Change a switch/case into polymorphism. Add an interface to decouple two concrete classes. It's easy and it's cheap, because mass is small. Say, however, that you got your modular structure wrong: an important concept has not been identified, and is now scattered all around. You have to tweak a lot of code. While doing so, you'll find that you're not just fighting the mass you have to move: you have to fight the gravitational attraction of the surrounding code. That may reveal more scattered concepts. It's a damn lot of work, possibly more than it's worth. Software is not necessarily soft.

So, can we really move software around? Yes, in the small. Yes, if you got the modular structure right, and you're only cleaning up within that modular structure. Once again, the fractal nature of software can be our best friend or our worst enemy. Get the right partitioning between applications / services, and you only have to work inside. Get the right partitioning between components, and you only have to work inside. And so on. It's fine to have a plastic component, inasmuch as the interface is right. Get the modular structure wrong, and sooner or later you'll need to make wide-range changes, or accept a "cheap steel" quality. If we got it wrong at a small scale, it's no big deal. At a large scale, it can be a massacre.

Conclusions
Once again, modularity is the key. Not just "any" modularity, but a modularity aligned with the underlying forcefield. If you get that right, you can build plastic components and evolve them to stainless steel quality over time. Been there, done that. Get your modular structure wrong, and you'll lack locality of action. In other words: if you plan on cutting corners inside a module, you have to invest at the interface level. That's simpler if you design top-down than if you're in the "emergent design" camp, because the module will be underengineered inside, and we can't expect a clean interface to emerge from underengineered code.

Design, Structure, and Decisions

Joe is a junior programmer in the IT department of SomeLargeCompany, Inc. His company is not in the business of software; however, they have a lot of custom programs that are constantly developed, adapted, tweaked to follow the ever-changing landscape of products, sales, regulations, etc.

Joe is largely self-taught. He doesn't know much about software engineering, but in the end, he gets things done.

Right now, Joe is writing some code dealing with money. He's using lots of floating points variables ("double" in his language) to store money.
Joe does not know about the inherent rounding error of decimal-to-binary conversion (and back). He doesn't know about a decimal floating point type. He doesn't know, and didn't look for, a Money class that could also store the currency. He's just coding his way out of his problem. He didn't choose to use a double after pondering on the alternatives, or based on previous experiences. It wasn't a choice at all. He just didn't know any better.

Joe is actually in the middle of a rather long function, doing some database lookup, some calculations, some reporting. He learned, the hard way, that is better to check for errors. He will handle any error in the simplest possible way: by opening an error box in the middle of the function, where the error can be diagnosed.
Joe doesn't know about design principles. He doesn't know that it's usually better to keep business logic decoupled from the user interface. He didn't choose to put some GUI code inside the business logic after thinking about alternatives. He just did it. He didn't know any better.

That function is so long because at some point Joe realized he had to handle several different cases. He did that the usual way: with a switch/case. He never really "got" objects and couldn't even think about using polymorphism there. He uses classes, somehow, but not at that fine granularity. So, again, he didn't choose the switch/case over something else. He just didn't know any better.

Structure vs. Design
Joe is a good guy, really; and his code kinda works, most of the times. But this story is not about Joe. It's about code, structure, design, and decisions: because in the end, Joe will write some serious amount of code. Code has an inner structure: in this case, structure will reveal heavy reliance on primitive types (double), will reveal that business logic is tangled with database access and with user interface concerns, and will also reveal that no extension mechanism is in place. Code never lies.

Now, the modern (dare I say post-agile?) way of thinking is that in the end, the code is the design. That is, if you want to know the real design, you have to look at the code. You'll find a lot of literature (not to mention blog posts), some from well-known authors, promoting or just holding on this idea. At some point, just about everybody gave in and accepted this as a truism. As you might guess, I didn't.

A process or a thing?
Software development is a young discipline, yet extremely dynamic. It's hardly surprising to find confusion and disagreement even on the fundamentals. That includes, of course, having several hundred different definitions of "software design". For some, it's a process. For some, it's the result of that process. We then have endless debates, like "is coding a form of design?", which again are often confusing "design" with "modeling" and wasting lot of ink about nothing.

It doesn't have to be that hard. Let's start with the high-level question: design as a process vs. design as a thing. It's a rather simple choice; we may even benefit from the digital version of some old, dusty etymology dictionary, where design is commonly defined as "mark out, devise, choose, designate, appoint". Design is an act, and therefore a process. Please don't read "process" as "a series of mechanical acts"; just consider a process as something you carry out over a period of time.
More exactly, design is decision process. We choose between alternatives, balancing forces as we go. We make decisions through a reflective conversation with our materials, taking place all the time, from early requirements to bug fixing.

Design, however, is not the code. It is not the diagram. It is not an artifact. If we really want to transform design into "a thing", then it is best defined as the set of choices that brought us to the artifact; assuming, of course, that we actually made any choice at all.

Side note: not every decision is a design decision. Design is concerned with form, that is, with the shape of our material. Including or excluding a feature from a product, for instance, is not a design decision; it is best characterized as a marketing decision. In practice, a product often results from the interplay of marketing and design decisions.

Design as a decision process
Back in 2006 I suggested that we shouldn't use terms like "accidental architecture" because architecture (which is just another form of design) is always intentional: it's a set of choices we make. I suggested using "structure" instead.

Artifacts always have structure. Every artifact we use in software development is made of things (depending on our paradigm) that are somehow related (again, depending on our paradigm). That's the structure. Look at a diagram, and you'll see structure. Look at the code, and you'll see structure. Look at the executable, and if you can make sense of it, you'll see structure.

There is a fundamental difference between structure and design. Structure can be the result of an intentional process (design) or the result of an accidental process (coding your way out of something). Design is always intentional; if it's not intentional, it's not design.

So, if you want to know the real structure (and yes, usually you want to), you have to look at the code. You may also look at diagrams (I would). But diagrams will show an abstract, idealized, and yes, possibly obsolete high-level structure. The "real" structure is the code structure. It would be naive, however, to confuse structure with design.

Consider Joe's code. We can see some structure in that code. That structure, however, is not design. There is no rationale behind that structure, except "I didn't know any better". Joe did not design his code. He just wrote it. He didn't make a single decision. He did the only thing he knew. That's not design. Sorry.

What is design, again?
Once we agree that design is a decision process, we can see different design approaches for what they are: different ways to come to a decision.

The upfront school claims that some decisions can be made before writing code, perhaps by drawing models. Taken to the extreme (which is always naive), it would require that we make all design decisions before writing code.

The emergent design school claims that system-level decisions should emerge from the self-organization of lower-level decision, mostly taken by writing code according to "good practices" like Don't Repeat Yourself. Taken to the extreme (which is always naive), it would require that we make no system-level choices at all, and wait for code to jell into a well-formed structure.

The pattern school claims that we can recognize forces and adopt pre-packaged decisions, embodied into a pattern.

Systematic techniques like my good old SysOOD claim that we can systematically recognize some problems and choose from a set of sound transformations.

The MDA school claims that we can organize decisions along two axes: decisions that we store in models, and decisions that we store in transformation tools. This would be a long story in itself.

Etc.

If you have a favorite design approach (say, TDD or Design by Contract) you may consider spending a little time pondering on which kind of decisions are better supported by that approach.

There is no choice
If there is no choice to be made, or if you can't see that there is a choice to be made, you are not doing design. This is so big that I'll have to say it again. If there is no choice to be made, you're not doing design. You might be drawing some fancy diagram, but it's not design anyway. I can draw a detailed diagram of the I/O virtualization layer for an embedded device in 10 minutes. I've done that many times. I'm not choosing anything, just replicating something. That's not design. It's "just" modeling.

On the other end of the spectrum, I've been writing quite a bit of code lately, based on various Facebook APIs (the graph api, the javascript sdk, fbml, whatever). Like many other APIs, things never really work as documented (or lacking documentation, as it would be reasonable to assume). Here and there, in my code, I had to do things not the way I wanted, but the only way that actually worked. In light of the above, I can honestly say that I did not design those portions. I often couldn't make a choice, although I wish I could (in this sense, it would be terrible to have someone look at that code and think that it represents "my design").

Of course, that's not my first web application ever. Over time, I've written many, and I've created a lot of small reusable components. I know them well, I trust them, I have the source code, and I'm familiar with it. When I see a problem that I can easily solve by reusing one, I tend to do it on the spot.
Now, this is a borderline case, as I'm not really making a choice. The component is just there, crying to be reused. I'm just going with the flow. It's perhaps 5% design (recognizing the component as a good candidate and choosing to use it) and 95% habits.
Reusing standard library classes is probably 1% design, 99% habits. We used to write our own containers and even string classes back in the early 90s. Today, I must have a very compelling case to do so (the 1% design is recognizing I don't have a very compelling case :-).

Trying to generalize a little, we may like to think that we're making decisions all the time, but this is far from true. In many cases, we don't, because:

- Like Joe, we don't know any better.

- Like above, we're working with an unstable, poorly documented, rather crappy third party library. It's basically trial and error, with a little clean-up in the end (so, let's say 2-5% design). The library is making the choices (95-98%).

- We've done this several times in the past, and we're just repeating ourselves again. It's a habit. Perhaps we should question our habit, but we don't. Perhaps we should see that we're writing the same code over and over and design a reusable component, but we don't.

- We (the company, the team, the community) have a standard way to do this.

- The library/framework/language we're using has already made that choice.

- We took a high-level decision before, and the rest follows naturally.

After pondering on this for a while, I came to the conclusion that in the past I've been mixing two aspects of the decision making process. The need to make a decision, and the freedom to make a decision. You can plot this as a quadrant, if you want.

Sometimes, there is just no need to make a choice. This can actually be a good thing. It could be a highly productive state where you just have to write down your logic. There might be short "design moments" where we make low-scale decisions (the fractal nature of software makes the decision process fractal as well), but overall we just go with the flow. As usual, the issue is context. Not making a decision because there is a natural solution is quite different from not making a decision because we can't even see the possibilities, or the need.

Sometimes, we need to make a choice. At that point, we need the freedom to do so. If the code is so brittle that you have to find your way by trial and error, if company standards are overly restrictive, if our language is too limited, if the architecture is too constraining, we lack freedom. That leads to sub-optimal local and global choices. Here is the challenge of architecture: make the right decisions, so that you offer a reasonable structure for growth, yet leave enough freedom for local choices, and a feedback loop from local to global.

Design, as a decision process, is better experienced in bursts. You make a set of choices, and then you let the natural consequences unravel themselves. You may unravel some consequences using models, if you like it. In some cases, it's very effective. Some consequences are best unraveled by writing code. Be smart, not religious.

Be aware when you have to make too many choices, too often. It's fine in the beginning, as the decision space is huge. But as you progress, the need for new choices should naturally decrease, in frequency and in scale. Think of coding as a conversation with your artifacts. Just like any conversation, it can turn into an argument and then into a fight. Your code just does not want to be shaped that way; it was never meant to be shaped that way; it is resisting your change. You change something here, just to find out that you really need to patch that other part, and so on. You have to think, think, think. Lot of effort, little progress. On a deeper level, you're fighting the forcefield. The design is not aligned with the forcefield. There is friction (in the decision space!) and friction energy is wasted energy.

Finally, if you draw a quadrant, you'll see an interesting spot where there is no need to make choices, but there is freedom to. I like that spot: it's where I know how to do it, because I've done it before, but it's not the only way to do it, and I might try something new this time. Some people don't take the opportunity, and always take the safe, known way. Depending on the project, I don't mind taking a risk if I see the potential for a sizeable reward.

What about good design?
Back in 1972, David Parnas wrote one of the most influential papers ever ("On the criteria to be used in decomposing systems into modules"), where two alternative modular structures were proposed for the same system. The first used the familiar functional decomposition. The second was based on the concept of Information Hiding. Later on, that paper has been heavily quoted in the OOP literature as an inspiration for encapsulation. Indeed, Parnas suggests hiding data structure details inside modules, but that was just an example. The real message was that modules should encapsulate decisions: every module [...] is characterized by its knowledge of a design decision which it hides from all others"; again: "we propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others".

Think about it, and it makes lot of sense. Design is a decision-making process. Decisions may turn out to be wrong, or may become obsolete as business and technology change. Some decisions are also meant to be temporary from the very beginning. In all cases, we would like to modularize those decisions, so that we can change them without global impact.

Unfortunately, this is not how most people design software. Most decisions are not modularized at all. Worse yet, most decisions are not made explicit, either in code, diagrams, sketchy notes, whatever. And worst of all, a lot of code is written without taking any decision at all, just like good ol' Joe does. "Seat of your pants" is not a design strategy.

Note: given current programming technology and tools, modularizing decisions is easier said than done. For instance, I decided to use AJAX in the web application I mentioned above. I would love to modularize that decision away. But it would be damn hard. Too hard to be worth it. So I let this decision influence the overall structure. There are many cases like this. Quite often, during a joint design session, I would say something like - guys, here we're making a pervasive, non-linear choice. This is like a magic door. Once we choose, we enter into a different world.

So, we're doing a good design job when we make modular decisions, that we can change later without global impact. I think this may shed a different light on delaying decisions (design decisions, not (e.g.) marketing decision).

The folklore is that if you wait, you can make a more informed decision. Of course, if you wait too much, you reach a state of paralysis where you can't progress anymore, and you may also miss some opportunity along the road.

From the decision modularization perspective, things are slightly more precise:

- If, given our current knowledge, the decision can't be modularized, it's better to wait, because we might find a way to make it modular, or get a better picture of our problem, and hopefully make the "right" nonmodular decision. Note that when a decision is not modular, undoing/changing it will incur a substantial cost: the mass to move in the decision space will be high, and that's our measure of work.

- If the decision can be modularized, but you think that given more knowledge you could modularize it better, it makes sense to wait. Otherwise, there is little or no value in waiting. Note that postponing a decision and postponing implementation are two different things (unless your only way to make a decision is by writing code). There is a strong connection with my concept of invariant decision, but I'll leave it up to you to dig around.

As an aside, I'm under the impression that most people expect to just "learn the right answer" by waiting. And by "right" they mean that there is no need to modularize the decision anymore (because it's right :-). This is quite simplistic, as there is no guarantee whatsoever that a delayed decision will be particularly stable.
So, again, the value of delaying should be in learning new ways to modularize away the decision. Learning more about the risk/opportunity trade-off of not modularizing away the decision is second best.

Conclusions
In a sense, software development is all about decisions. Yet, the concept is completely absent from programming and modeling languages, and not particularly explicit in most design methods as well.
In the Physics of Software I'm trying to give Decisions a central role. Decisions are gateways to different forcefields. Conversely, we shape the forcefield by making decisions. Decisions live in the Decision Space, and software evolution is basically a process of moving our software to another position in the Decision Space. Decisions are also a key ingredient to link software design and software economics. I have many interesting ideas here, mostly based on real option theories, but that's another story.

Next time: tangling.

Why you should learn AOP

A few days ago, I've spent some time reading a critic of AOP (The Paradoxical Success of Aspect-Oriented Programming by Friedrich Steimann). As often, I felt compelled to read some of the bibliographical references too, which took me a little more (week-end) time.

Overall, in the last few years I've devoted quite some time to learn, think, and even write a little about AOP. I'm well aware of the problems Steimann describes, and I share some skepticism about the viability of the AOP paradigm as we know it.

Too much literature, for instance, is focused on a small set of pervasive concerns like logging. I believe that as we move toward higher-level concerns, we must make a clear distinction between pervasive concerns and cross-cutting concerns. A concern can be cross-cutting without being pervasive, and in this sense, for instance, I don't really agree that AOP is not for singletons (see my old post Some notes on AOP).
Also, I wouldn't dismiss the distinction between spectators and assistants so easily, especially because many pervasive concerns can be modeled as spectators. Overall, the paradigm seems indeed a little immature when you look at the long-term maintenance effects of aspects as they're known today.

Still, I think the time I've spent pondering on AOP was truly well spent. Actually, I would suggest that you spend some time learning about AOP too, even if you're not planning to use AOP in the foreseeable future.

I don't really mean learning a specific language - unless you want/need to try out a few things. I mean learning the concepts, the AOP perspective, the AOP terminology, the effects and side-effects of an Aspect Oriented solution.

I'm suggesting that you learn all that despite the obvious (or perhaps not so obvious) deficiencies in the current approaches and languages, the excessive hype and the underdeveloped concepts. I'm suggesting that you learn all that because it will make you a better designer.

Why? Because it will expand your mind. It will add a new, alternative perspective through which you can look at your problems. New questions to ask. New concepts. New names. Sometimes, all we need is a name. A beacon in the brainstorm, and a steady hand.

As I've said many times now, as designers we're shaping software. We can choose many shapes, and ideally, we will find a shape that is in frictionless contact with the forcefield. Any given paradigm will suggest a set of privileged shapes, at macro and micro-level. Including the aspect-oriented paradigm in your thinking will expand the set of shapes you can apply and conceive.

Time for a short war story :-). In the past months I've been thinking a lot about some issues in a large CAD system. While shaping a solution, I'm constantly getting back to what I could call aspect-thinking. There are many cross-cutting concerns to be resolved. Not programming-level concerns (like the usual, boring logging stuff). Full-fledged application-domain concerns, that tend to cross-cut the principal decomposition.

Now, you see, even thinking "principal decomposition" and "cross-cutting" is making your first step into aspect-thinking. Then you can think about ways to bring those concerns inside the principal decomposition (if appropriate and/or possible and/or convenient) or think about the best way to keep them outside without code-level tangling. Tangling. Another interesting name, another interesting concept.

Sure, if you ain't using true AOP (for instance, we're using plain old C++), you'll have to give up some oblivousness (another name, another concept!), but it can be done, and it works fine (for a small scale example, see part 1 and part 2 of my "Can AOP inform OOP?")

So far, the candidate shape is causing some discomfort. That's reasonable. It's not a "traditional" solution. Which is fine, because so far, tradition didn't work so well :-). Somehow, I hope the team will get out of this experience with a new mindset. Nobody used to talk about "principal decomposition" or "cross-cutting concern" in the company. And you can't control what you can't name.

I hope they will gradually internalize the new concepts, as well as the tactics we can use inside traditional languages. That would be a major accomplishment. Much more important than the design we're creating, or the tons of code we'll be writing. Well, we'll see...

Form vs. Function: a Space Odyssey

I was teaching Object Oriented Design past week, and I mentioned the interplay between form and function (form follows function; function follows form). I'm rather cautious not to spend too much time on philosophy, although a little philosophy shouldn't hurt, and people who know me tend to expect a short philosophical digression every now and then.

Function follows form: that is to say, the shape of an object will suggest possible uses. Form follows function: the intended usage of an object will constrain, and therefore guide, its physical shape. This is true for software objects as well. It's just a different material, something we can't immediately see and shape with our hands.

Realizing that function follows form is a pivotal moment in the development of intelligence. You probably remember the opening of 2001: A Space Odyssey. The apes are touching the monolith and, shortly after, one of them is playing with a bone and bang! - it's not just a bone anymore: it's a tool. Function follows form. This chapter is known as "The Dawn of Man", and rightly so.

Watch a little more, and you'll see a doughnut-shaped space station. That's a very good example of form following function (exercise for the reader :-)

By the way, if you have never seen that apes stuff till now :-), here it is, at least until it gets removed from YouTube...

A ForceField Diagram

The Design Rationale Diagram I discussed in my previous post is hardly complete, and it could be vastly improved by asking slightly different questions, leading to different decision paths. Still, it's a reasonable first-cut attempt to model the decision process. It can be used to communicate the reasoning behind a specific decision, in a specific context.

That, however, is not the way I really think. Sure, I can rationalize things that way, but it's not the way I store, recall, organize information inside my head. It's not the way I see the decision space.
In the end, software design is about things going together and things staying apart, at all the granularity levels (see also my post on partitioning).
As I progress in my understanding of forces, I tend to form clusters. Clusters are born out of attraction and rejection inside the decision space. I've found that thinking this way helps me reach a better understanding of my design instinct, and to communicate my thoughts more clearly.

Now, although I've been thinking about this for long while (not full-time, lucky me :-), I can't say I have found the perfect representation. The decision space in inherently multi-dimensional, and I always end up needing more dimensions that I can fit either in 2D or 3D. Over time, I tried several notations, inventing things from scratch or borrowing from other domains. Most were dead ends. In the end, I've chosen (so far :-) a very simple representation, based on just 3 concepts (possibly 4 or 5).

- nodes
Nodes represent information, which is our material. Information has fractal nature, and I don't bother if I'm mixing up levels. Therefore, a node may represent a business goal, or the adoption of a tool or library, or a nonfunctional requirement, or a specific component, class, function. While most methods are based on a strict separation of concepts, I find that very limiting.

- an attraction relationship
Nodes can attract each other. For instance, a node labeled "reliable" may attract a node labeled "redundant" when reasoning about the large display problem. I just connect the two nodes using a thick line with little "hands" on the ends. I place attracted nodes close to each other.

- a rejection relationship
Nodes can reject each other. For instance, stateful most clearly reject stateless :-). Some technology might be at odd with another. A subsystem must not depend on another. And so on. Nodes that reject each other are placed at some distance.

It's all very simple and unsophisticated. Here is an example based on the large display problem, inspired by the discussion on design rationale:



and here are two diagrams I've used in real-world projects recently, scaled down to protect the innocent:





The relationship between a node, a cluster, and an Alexandrian center is better left for another time. Still, a node in one diagram may represent an entire cluster, or an entire diagram. Right now I'm tempted to use a slightly different symbol (which would be the fourth) to represent "expandable" nodes, although I'm really trying to keep symbols to a bare minimum. I'm also using colors, but so far in a very informal way.

As simple as it is, I've found this diagram to be very effective as a reasoning device, while too many diagrams end up being mere documentation devices. I can sit in front of my (large :-) screen, think and draw, and the drawing helps me focus. I can draw this on a whiteboard in a meeting, and everyone get up to speed very quickly on what I'm doing.

This, however, is just half the story. We can surely work with informal concepts and diagrams, and that's fine, but what I'm trying to do is to add precision to the diagram. Precision is often confused with details, like "a class diagram is more precise if you show all the parameters and types". I'm not looking for that kind of "precision". Actually, I don't want this diagram to be redundant with code at all; we already have many code-like diagrams, and they all get down the same roads (generate code from diagrams or generate diagrams from code). I want a reasoning device: when I want to code, I'm comfortable with code :-).

I mostly want to add precision about relationships. Why, for instance, is there an attraction between Slow Client and Stateful? Informally, because if we have a stateful system, the slow client can poll on its own terms, or alternatively, because the client may use a sophisticated subscription based on the previous state. Those options, by the way, could be represented on the forcefield diagram itself (adding more nodes, or a nested diagram); but that's still the "informal" reasoning. Can we make it any more formal, precise, grounded on sound principles?

This is where the ongoing work on concepts like gravity, frequency, and so on kicks in. Slow Client and Stateful are attracted because on a finer granularity (another, perhaps better, diagram) "Slow Client" means a publisher and a subscriber operating at different frequencies, and a stateful repository is a well-known strategy (a pattern!) to provide Isolation between systems operating at different frequencies (together with synchronization or transactions).

Now, I haven't introduced the concept of Isolation yet (though I mentioned something on my Facebook page :-), so this is sort of a spoiler :-)), but in the end I hope to come up with a simple reasoning system, where you can start with informal concepts and refine nodes and forces until you reach the "universal", fractal forces I'm discussing in the "Notes on Software Design" posts. That would give a solid ground to the entire diagram.

A final note on the forcefield diagram: at this stage, I'm just using Visio, or more exactly, I'm abusing some stencils in the Visio library. I wanted something relatively organic, mindmap-like. Maybe one day I'll move back to some 3D ideas (molecular structures come to mind), but I've yet to see how this scales to newer concepts, larger problems, and so on. If you want to play with it, I can send you the VSS file with the stencils.

Ok, I'll get back to Frequency (and Interference and Isolation and more :-) soon. Before that, however, I'd like to take a diversion on the Dependency Structure Matrix. See ya!

Bad Luck, or "fighting the forcefield"

In my previous post, I used the expression "fighting the forcefield". This might be a somewhat uncommon terminology, but I used it to describe a very familiar situation: actually, I see people fighting the forcefield all the time.

Look at any troubled project, and you'll see people who made some wrong decision early on, and then stood by it, digging and digging. Of course, any decision may turn out to be wrong. Software development is a knowledge acquisition process. We often take decisions without knowing all the details; if we didn't, we would never get anything done (see analysis paralysis for more). Experience should mitigate the number of wrong decisions, but there are going to be mistakes anyway; we should be able to recognize them quickly, backtrack, and take another way.

Experience should also bring us in closer contact with the forcefield. Experienced designers don't need to go through each and every excruciating detail before they can take a decision. As I said earlier, we can almost feel, or see the forcefield, and take decisions based on a relatively small number of prevailing forces (yes, I dare to consider myself an experienced designer :-).
This process is largely unconscious, and sometimes it's hard to rationalize all the internal reasoning; in many cases, people expect very argumentative explanations, while all we have to offer on the fly is aesthetics. Indeed, I'm often very informal when I design; I tend to use colorful expressions like "oh, that sucks", or "that brings bad luck" to indicate a flaw, and so on.

Recently, I've found myself saying that "bad luck" thing twice, while reviewing the design of two very different systems (a business system and a reactive system), for two different clients.
I noticed a pattern: in both cases, there was a single entity (a database table, a in-memory structure) storing data with very different timing/life requirements. In both cases, my clients were a little puzzled, as they thought those data belonged together (we can recognize gravity at play here).
Most naturally, they asked me why I would keep the data apart. Time to rationalize :-), once again.

Had they all been more familiar with my blog, I would have pointed to my recent post on multiplicity. After all, data with very different update frequency (like: the calibration data for a sensor, and the most recent sample) have a different fourth-dimensional multiplicity. Sure, at any given point in time, a sensor has one most recent sample and one set of calibration data; therefore, in a static view we'll have multiplicity 1 for both, suggesting we can keep the two of them together. But bring in the fourth dimension (time) and you'll see an entirely different picture: they have a completely different historical multiplicity.

Different update frequencies also hint at the fact that data is changing under different forces. By keeping together things that are influenced by more than one force, we expose them to both. More on this another time.

Hard-core programmers may want more than that. They may ask for more familiar reasons not to put data with different update frequencies in the same table or structure. Here are a few:

- In a multi-threaded software, in-memory structures requires locking. If your structure contains data that is seldom updated, that means it's being read more than written: if it's seldom read and seldom written, why keep it around at all?
Unfortunately, the high-frequency data is written quite often. Therefore, either we accept to slow down everything using a simple mutex, or we aim for higher performances through a more complex locking mechanism (reader/writer lock), which may or may not work, depending on the exact read/write pattern. Separate structures can adopt a simpler locking mechanism, as one is being mostly read, the other mostly written; even if you go with a R/W lock, here it's almost guaranteed to have good performance.

- Even on a database, high-frequency writes may stall low-frequency reads. You even risk a lock escalation from record to table. Then you either go with dirty reads (betting on your good luck) or you just move the data in another table, where it belongs.

- If you decide to cache database data to improve performances, you'll have to choose between a larger cache with the same structure of the database (with low frequency data too) or a smaller and more efficient cache with just the high-frequency data (therefore revealing once more that those data do not belong together).

- And so on: I encourage you to find more reasons!

In most cases, I tend to avoid this kind of problems instinctively: this is what I really call experience. Indeed, Donald Schön reminds us that good design is not for everyone, and that you have to develop your own sense of aesthetics (see "Reflective Conversation with Materials. An interview with Donald Schön by John Bennett", in Bringing Design To Software, Addison-Wesley, 1996). Aesthetics may not sound too technical, but consider it a shortcut for: you have to develop your own ability to perceive the forcefield, and instinctively know what is wrong (misaligned) and right (aligned).

Ok, next time I'll get back to the notion of multiplicity. Actually, although I've initially chosen "multiplicity" because of its familiarity, I'm beginning to think that the whole notion of fourth-dimensional multiplicity, which is indeed quite important, might be confusing for some. I'm therefore looking for a better term, which can clearly convey both the traditional ("static") and the extended (fourth-dimensional, historical, etc) meaning. Any good idea? Say it here, or drop me an email!

Notes on Software Design, Chapter 4: Gravity and Architecture

In my previous posts, I described gravity and inertia. At first, gravity may seem to have a negative connotation, like a force we constantly have to fight. In a sense, that's true; in a sense, it's also true for its physical counterpart: every day, we spend a lot of energy fighting earth gravity. However, without gravity, like as we know it would never exist. There is always a bright side :-).

In the software realm, gravity can be exploited by setting up a favorable force field. Remember that gravity is a rather dumb :-) force, merely attracting things. Therefore, if we come up with the right gravitational centers early on, they will keep attracting the right things. This is the role of architecture: to provide an initial, balanced set of centers.

Consider the little thorny problem I described back in October. Introducing Stage 1, I said: "the critical choice [...] was to choose where to put the display logic: in the existing process, in a new process connected via IPC, in a new process connected to a [RT] database".
We can now review that decision within the framework of gravitational centers.

Adding the display logic into the existing process is the path of least resistance: we have only one process, and gravity is pulling new code into that process. Where is the downside? A bloated process, sure, but also the practical impossibility of sharing the display logic with other processes.
Reuse requires separation. This, however, is just the tip of the iceberg: reuse is just an instance of a much more general force, which I'll cover in the forthcoming posts.

Moving the display logic inside a separate component is a necessary step toward [independent] reusability, and also toward the rarely understood concept of a scaled-down architecture.
A frequently quoted paper from David Parnas (one of the most gifted software designers of all times) is properly titled "Designing Software for Ease of Extension and Contraction" (IEEE Transactions on Software Engineering, Vol. 5 No. 2, March 1979). Somehow, people often forget the contraction part.
Indeed, I've often seen systems where the only chance to provide a scaled-down version to customers is to hide the portion of user interface that is exposing the "optional" functionality, often with questionable aesthetics, and always with more trouble than one could possibly want.

Note how, once we have a separate module for display, new display models are naturally attracted into that module, leaving the acquisition system alone. This is gravity working for us, not against us, because we have provided the right center. That's also the bright side of the thorny problem, exactly because (at that point, that is, stage 2) we [still] have the right centers.

Is the choice of using an RTDB to further decouple the data acquisition system and the display system any better than having just two layers?
I encourage you to think about it: it is not necessarily trivial to undestand what is going on at the forcefield level. Sure, the RTDB becomes a new gravitational center, but is a 3-pole system any better in this case? Why? I'll get back to this in my next post.

Architecture and Gravity
Within the right architecture, features are naturally attracted to the "best" gravitational center.
The "right" architecture, therefore, must provide the right gravitational centers, so that features are naturally attracted to the right place, where (if necessary) they will be kept apart from other features at a finer granularity level, through careful design and/or careful refactoring.
Therefore, the right architeture is not just helping us cope with gravity: it's helping us exploit gravity to our own advantage.

The wrong architecture, however, will often conjure with gravity to preserve itself.
As part of my consulting activity, I’ve seen several systems where the initial partitioning of responsibility wasn’t right. The development team didn’t have enough experience (with software design and/or with the problem domain) to find out the core concepts, the core issues, the core centers.
The system was partitioned along the wrong lines, and as mass increased, gravity kicked in. The system grew with the wrong form, which was not in frictionless contact with the context.
At some point, people considered refactoring, but it was too costly, because mass brings Inertia, and inertia affects any attempt to change direction. Inertia keeps a bad system in a bad state. In a properly partitioned system, instead, we have many options for change: small subsystems won’t put up much of a fight. That’s the dream behind the SOA concept.
I already said this, but is worth repeating: gravity is working at all granularity levels, from distributed computing down to the smallest function. That's why we have to keep both design and code constantly clean. Architecture alone is not enough. Good programmers are always essential for quality development.

What about patterns? Patterns can lower the amount of energy we have to spend to create the right architecture. Of course, they can do so because someone else spent some energy re-discovering good ideas, cleaning them up, going through shepherding and publishing, and because we spent some time learning about them. That said, patterns often provide an initial set of centers, balancing out some forces (not restricted to gravity).
Of course, we can't just throw patterns against a problem: the form must be in effortless contact with the real problem we're facing. I've seen too many good-intentioned (and not so experienced :-) software designers start with patterns. But we have to understand forces first, and adopt the right patterns later.

Enough with mass and gravity. Next time, we're gonna talk about another primordial force, pushing things apart.

See you soon, I hope!

On the concept of Form (1)

I postponed writing on Form (vs. Function) for a while now, mostly because I was struggling to impose a rational structure on my thoughts. Form is a complex subject, barely discussed in the software design literature, with a few exception I'll quote when relevant, so I'm sort of charting new waters here.

The point is, doing so would take too much [free] time, delaying my writings for several more weeks or even months. This is exactly the opposite of what I want to do with this blog (see my post blogging as destructuring), so I'll commit the ultimate sin :-) and talk about form in a rather unstructured way. I'll start a random sequence of posting on form, writing down what I think is relevant, but without any particular order. I won't even define the concept of form in this first instalment.

Throughout these posts, I'll borrow extensively from a very interesting (and warmly suggested to any software designer) book from Christopher Alexander. Alexander is better known for his work on patterns, which ultimately inspired software designers and created the pattern movement. However, his most releavant work on form is Notes on the Synthesis of Form. When quoting, I'll try to remember page numbers, in which case, I'll refer to the paperback edition, which is easier to find.

Alexander defines two processes through which form can be obtained: the unselfconscious and the selfconscious process. I'll get back to these two concepts, but in a few words, the unselfconscious process is the way some ancient cultures proceeded, without an explicit set of principles, but relying instead on rigid tradition mediated by immediate, small scale adaptation upon failure. It's more complex than that, but let's keep it simple right now.

Tradition provides viscosity. Without tradition, and without explicit design principles, the byproducts of the unselfconscious process will quickly degenerate. However, merely repeating a form won't keep up with a changing environment. Yet change happens, maybe shortly after the form has been built. Here is where we need immediate action, correcting any failure using the materials at hand. Of course, small scale changes are sustainable only when the rate of change (or rate of failure) is slow.

Drawing parallels to software is easy, although subjective. Think of quick, continuous, small scale adaptation. The immediate software counterpart is, very naturally, refactoring. As soon as a bad smell emerge, you fix it. Refactoring is usually small scale, using the "materials at hand" (which I could roughly translate into "changing only a small fraction of code"). Refactoring, by definition, does not change the function of code. Therefore, it can only change its form.

Now, although some people in the XP/agile camp might disagree, refactoring is a viable solution only when the desired rate of change is slow, and only when the gap to fill is small. In other words, only when the overall architecture (or plain structure) is not challenged: maybe it's dictated by the J2EE way of doing things, or by the Company One True Way of doing things, or by the Model View Controller police, and so on. Truth is, without an overall architecture resisting change, a neverending sequence of small-scale refactoring may even have a negative large-scale impact.

I've recently said that we can't reasonably turn an old-style, client-server application into a modern web application by applying a sequence of small-scale changes. It would be, if not unfeasible, hardly economic, and the final architecture might be far from optimal. The gap is too big. We're expecting to complete a big change in a comparatively short time, hence the rate of change is too big. The viscosity of the previous solution will fight that change and prevent it from happening. We need to apply change at an higher granularity level, as the dynamics in the small are not the dynamics in the large.

Curiously enough (or maybe not :-), I'll be talking about refactoring the next two days. As usual, I'll try to strike a balance, and get often back to good design princples. After all, as we'll see, when the rate of change grows, and/or when the solution space grows, the unselfconscious process must be replaced by the selfconscious process.

Got Multicore? Think Asymmetric!

Multicore CPU are now widely available, yet many applications are not tapping into their true potential. Sure, web applications, and more generally container-based applications have an inherent degree of coarse parallelism (basically at the request level), and they will scale fairly well on new CPU. However, most client-side applications don't fall in the same pattern. Also, some server-side applications (like batch processing) are not intrinsically parallel as well. Or maybe they are?

A few months ago, I was consulting on the design of the next generation of a (server-side) banking application. One of the modules was a batch processor, basically importing huge files into a database. For several reasons (file format, business policies), the file had to be read sequentially, processed sequentially, and imported into the database. The processing time was usually dominated by a single huge file, so the obvious technique to exploit a multicore (use several instances to import different files in parallel) would have not been effective.
Note that when we think of parallelism in this way, we're looking for symmetric parallelism, where each thread performs basically the same job (process a request, or import a file, or whatever). There is only so much you can do with symmetrical parallelism, especially on a client (more on this later). Sometimes (of course, not all the times), it's better to think asymmetrically, that is, model the processing as a pipeline.

Even for the batch application, we can see at least three stages in the pipeline:
- reading from the file
- doing any relevant processing
- storing into the database
You can have up to three different threads performing these tasks in parallel: while thread 1 is reading record 3, thread 2 will process record 2, and thread 3 will store [the processed] record 1. Of course, you need some buffering in between (more on this in a short while).
Actually, in our case, it was pretty obvious that the processing wasn't taking enough CPU to justify a separate thread: it could be merged with the read file operation. What was actually funny (almost exhilarating :-) was to discover that despite the immensely powerful database server, storing into the database was much slower than reading from the file (truth to be said, the file was stored in an immensely powerful file server as well). A smart guy in the bank quickly realized that it was our fault: we could have issued several parallel store operations, basically turning stage two of the pipeline into a symmetrical parallel engine. That worked like a charm, and the total time dropped by a factor of about 6 (more than I expected: we were also using the multi-processor, multi-core DB server better, not just the batch server multicore CPU).

Just a few weeks later (meaningful coincidence?), I stumbled across a nice paper: Understand packet-processing performance when employing multicore processors by Edwin Verplanke (Embedded Systems Design Europe, April 2007). Guess what, their design is quite similar to ours, an asymmetric pipeline with a symmetric stage.

Indeed, the pipeline model is extremely useful also when dealing with legacy code which has never been designed to be thread-safe. I know that many projects aimed at squeezing some degree of parallelism out of that kind of code fails, because the programmers quickly find themselves adding locks and semaphores everywhere, thus slowing down the beast so much that there is either no gain or even a loss.
This is often due to an attempt to exploit symmetrical parallelism, which on legacy, client-side code is a recipe for resource contention.Instead, thinking of pipelined, asymmetrical parallelism often brings some good results.
For instance, I've recently overheard a discussion on how to make a graphical application faster on multicore. One of the guy contended that since the rendering stage is not thread-safe, there is basically nothing they can do (except doing some irrelevant background stuff just to keep a core busy). Of course, that's because he was thinking of symmetrical parallelism. There are actually several logical stages in the pipeline before rendering takes place: we "just" have to model the pipeline explicitly, and allocate stages to different threads.

As I've anticipated, pipelines need some kind of buffering between stages. Those buffers must be thread safe. The banking code was written in C#, and so we simply used a monitor-protected queue, and that was it. However, in high-performance C/C++ applications we may want to go a step further, and look into lock-free data structures.

A nice example comes from Bjarne Stroustrup himself: Lock-free Dynamically Resizable Arrays. The paper has also a great bibliography, and I must say that the concept of descriptor (by Harris) is so simple and effective that I would call it a stroke of genius. I just wish a better name than "descriptor" was adopted :-).

For more predictable environments, like packet processing above, we should also keep in mind a simple, interesting pattern that I always teach in my "design patterns" course (actually in a version tailored for embedded / real-time programming, which does not [yet] appear on my website [enquiries welcome :-)]. You can find it in Pattern Languages of Program Design Vol. 2, under the name Resource Exchanger, and it can be easily made lock-free. I don't know of an online version of that paper, but there is a reference in the online Pattern Almanac.
If you plan to adopt the Resource Exchanger, make sure to properly tweak the published design to suit your needs (most often, you can scale it down quite a bit). Indeed, over the years I've seen quite a few hard-core C programmers slowing themselves down in endless memcpy calls where a resource exchanger would have done the job oh so nicely.

A final note: I want to highlight the fact that symmetric parallelism can still be quite effective in many cases, including some kind of batch processing or client-side applications. For instance, back in the Pentium II times, I've implemented a parallel sort algorithm for a multiprocessor (not multicore) machine. Of course, there were significant challenges, as the threads had to work on the same data structure, without locks, and (that was kinda hard) without having one processor invalidating the cache line of the other (which happens quite naturally in discrete multiprocessing if you do nothing about it). The algorithm was then retrofitted into an existing application. So, yes, of course it's often possible to go symmetrical, we just have to know when to use what, at which cost :-).