Ask not what an object is, but...

I can barely remember the days when objects were seen like a new, shiny, promising technology. Today, objects are often positioned between mainstream and retro, while the functional paradigm is enjoying an interesting renaissance. Still, in the last few months I stumbled on a couple of blog posts asking the quintessential question, reminiscent of those dark old days: “what is an object?”

The most recent (September 2012) is mostly a pointer to a stripped-down definition provided by Brian Marick: “It’s a clump of name->value mappings, some functions that take such clumps as their first arguments, and a dispatch function that decides which function the programmer meant to call”. Well, honestly, this is more about a specific implementation of objects, with a rather poor fit, for instance, with the C++ implementation. It makes sense when you’re describing a way to implement objects (which is what Marick did) but it’s not a particularly far-reaching definition.

The slightly older one (July 2012) is much more ambitious and comprehensive. Cook aims to provide a “modern” definition of objects, unrestricted by specific languages and implementations. It’s an interesting post indeed, and I suggest that you take some time reading it, but in the end, it’s still very much about the mechanics of objects ("An object is a first-class, dynamically dispatched behavior").

Although it may seem ok from a language design perspective, defining objects through their mechanics leaves a vacuum in our collective knowledge: how do we design a proper object-oriented system?

Don’t do it!

We used to be taught that, by spending enough time thinking about a problem, we would come up with a "perfect" model, one that embodies many interesting properties (often disguised as principles). One of those properties was stability, that is, most individual abstractions didn't need to change as requirements evolved. Said otherwise, change was local, or even better, change could be dealt with by adding new, small things (like new classes), not by patching old things.

That school didn't last; some would say it failed (as in "objects have failed"). At some point in time, another school prevailed, claiming that thinking too far into the future was bad, that it could lead to the wrong model anyway, and that you'd better come up with something simple that can solve today's problems, keeping the code quality high so that you can easily evolve it later, safely protected by a net of unit tests.

As is common, one school tended to mischaracterize the other (and vice-versa), usually by pushing things to the extreme through some cleverly designed argument, and then claiming generality. It's easy to do so while talking about software, as we lack sound theories and tangible forces.

Consider this picture instead:

Even if you don't know squat about potential energy, local minima and local maxima, is there any doubt the ball is going to fall easily?

No Controller, Episode 3: Inglorious Objekts

A week ago or so, Ralf Westphal published yet another critique of my post on living without a controller. He also proposed a different design method and therefore a different design. We also exchanged a couple of emails.

Now, I'm not really interested in "defending" my solution, because the spirit of the post was not to show the "perfect solution", but simply how objects could solve a realistic "control" problem without needing a centralized controller.

However, on one side Ralf is misrepresenting my work to the point where I have to say something, and on the other, it's an interesting chance to talk a bit more about software design.

So, if you haven't read my post on the controller, I would suggest you take some time and do so. There is also an episode 2, because that post has been criticized before, but you may want to postpone reading that and spend some time reading Ralf's post instead.

In the end, what I consider most interesting about Ralf's approach is the adoption of a rule-based approach, although he's omitting a lot of necessary details. So after as little fight as possible :-), I'll turn this into a chance to discuss rules and their role in OOD, because guess what, I'm using rules too when I see a good fit.

I'll switch to a more conversational structure, so in what follows "you" stands for "Ralf", and when I quote him, it's in green.

Life without Stupid Objects, Episode 1

So, this is not, strictly speaking, the next post to the previous post. Along the road, I realized I was using a certain style in the little code I wanted to show, and that it wasn't the style most people use, and that it would be distracting to explain that style while trying to communicate a much more important concept. 

So this post is about persistence and a way of writing repositories. Or it is about avoiding objects with no methods and mappers between stupid objects. Or it is about layered architectures and what constitutes a good layer, and why we shouldn't pretend we have a layered architecture when we don't. Or it is about applied physics of software, understanding our material (software) and what we can really do with it. Or why we should avoid guru-driven programming. You choose; either way, I hope you'll find it interesting.

Notes on Software Design, Chapter 16: Learning to see the Forcefield

When we interact with the physical world, we develop an intuitive understanding of some physical forces. It does not take a PhD in physics to guess what is going to happen (at a macroscopic level) when you apply a load at the end of a cantilever:

You can also devise a few changes (like adding a cord or a rod) to distribute forces in a different way, possibly ending up with a different structure (like a truss):

Software is not so straightforward. As I argued before, we completely lack a theory of forces (and materials). Intuitive understanding is limited by the lack of correlation between form and function (see Gabriel). Sure, many programmers can easily perceive some “technological” forces. They perceive the UI, business logic, and persistence to be somehow “kept apart” by different concerns, hence the popularity of layered architectures. Beyond that, however, there is a gaping void which is only partially filled by tradition, transmitted through principles and patterns.

Still, I believe the modern designer should develop the ability to see the force field, that is, understand the real forces pulling things together or far apart, moving responsibilities around, clustering them around new concepts (centers). Part of my work on the Physics of Software is to make those forces more visible and well-defined. Here is an example, inspired by a recurring problem. This post starts easy, but may end up with something unexpected.

Episode 2: the Controller Strikes Back

This post should have been about power law distribution of class / method sizes, organic growth of software and living organisms, Alexandrian level of scales, and a few more things.

Then the unthinkable happened. Somebody actually came up with a comment to Life without a controller, case 1, said my design was crappy and proposed an alternative based on a centralized, monolithic approach, claiming miraculous properties. I couldn’t just sit here and do nothing.

Besides, I wrote that post in a very busy week, leaving a few issues unexplored, and this is a good chance to get back to them.

I suggest that you go read the whole thing, as it’s quite interesting, and adds the necessary context to the following. Then please come back for the bloodshed. (Note: the original link is broken, as the file was removed; the "whole thing" link now points to a copy hosted on my website).

The short version
If you’re the TL;DR kind and don’t want to read Zibibbo’s post and my answer, here is the short version:
A caveman is talking to an architect.
Caveman: I don’t really like the architecture of your house.
Architect: why?

Life without a controller, case 1

In my most popular post ever, I argued (quoting Alan Kay and Peter Coad along the way) that classes ending in –er (like the infamous Controller, Manager, etc) are usually an indication that our software is not really OO (built from cooperating objects), but still based on the centralized style of procedural programming.

That post was very well received, with a few exceptions. One of the critics said something like “I’ll keep calling my controllers controllers, and my presenters presenters”. This is fine, of course. If you have a controller, why not calling it “controller”? Or, as Shakespeare said, a controller by any other name would smell as bad :-) . The entire idea was to get rid of controllers, not changing their name!

Now, getting rid of controllers is damn hard in 2012. Almost every UI framework is based on some variation of MVC. Most server-side web-app frameworks are based on MVC as well. Proposing to drop the controller would be just as useful as proposing that we drop html and javascript and start using something serious to write web apps :-).

Still, I can’t really sit here and do nothing while innocent objects get slaughtered, so I decided to write a few posts on living without a controller. I’ll start far away from UI, to avoid rocking the boat. I’ll move closer in some future post.

Turn that If into a When

I've been absent for a few months now. The thing is, I'm spending my little free time writing a non-trivial Android app. Of course, I'm also taking thousands of notes on software design, ready for a comeback :-). Anyway, staying away from my blog so long is kinda painful :-), so here I am with a short post, inspired by a very simple design choice I had to make.

When you start my little app for the first time, you get an EULA screen. If you accept the agreement, I'll initialize an internal database with some data and bring you the main screen. Ouch, did I say "if"?

Object-oriented programming was supposed to save us from switch/cases (replaced by polymorphism), and an if/else is a switch/case by another name. However, things like "if you accept the agreement" don't fit so well with polymorphism. You can make them fit, but it's not their natural shape. So we end up with just another (fat) controller.

On the other hand, perhaps I'm just saying it wrong. What if I change it into "When you accept the agreement..."? I don't know about you, but that instantly speaks "event" to me. Now, once we stop thinking about "conditions" and we think about "events", a few things happen:

- We can actually implement the concept on top of events if our language/library is event-based.

- We can fall back to the inner interface idiom, if we're in Java-land.

- We may even think aspect if we're bold enough.

Indeed, back in 2008, I wrote a post about reasoning in aspects and then implementing in objects (Can AOP inform OOP (toward SOA, too? :-) [part 1]). The idea was exactly that in some cases (cross-cutting business rules) an event-based implementation could be a poor man's alternative to an aspect-based interception (the subsequent post delved a bit more into some obliviousness issues and into the concept of cross-cutting business rules as candidate aspects).

Of course, we have to recognize the opportunity for an aspect (or event) based design. Sometimes, changing the words we use to describe [ourselves] the problem makes that easier. Next time you're facing an if, try changing it into a when. It won't always work, but it's worth trying :-).

That's it for now. Time to hit the road (snow actually :-). Places to go, clients to meet. Next time, I'm probably going to start a new series here, "life without a controller", because yeah, controllers ain't OO.

If you liked this post, you should follow me on twitter!

Cut the red wire!

We're all familiar with this scene: the good guys trying to save the day (or the world) by defusing a bomb. Quickly running out of time, they have to cut a wire. Failure is not an option. The red bar won't turn into a green bar.

What if you had to write code that works the very first time? What if you had no chance to test your code, let alone fixing bugs? You got only one chance to run your code and save the world. Would that change the way you write? The way you think? How? What can you learn from that?

Lest you think I'm kinda crazy: I'm not actually suggesting that you shouldn't test your code. I'm not actually suggesting that you stop doing your test-driven thing (if you do) or that you forget your test plan (if you have one), etc etc.
Consider this like a stretching exercise. We may learn something useful just by going outside our comfort zone. Give it a try.

Notes on Software Design, Chapter 14: the Enumeration Law

In my previous post on this series, I used the well-known Shape problem to explain polymorphism from the entanglement perspective. We've seen that moving from a C-like approach (with a ShapeType enum and a union of structures) to an OO approach (with a Shape interface implemented by concrete shape classes) creates new centers, altering the entanglement between previous centers.
This time, I'll explore the landscape of entanglement a little more, using simple, well-known problems. My aim is to provide an intuitive grasp of the entanglement concept before moving to a more formal definition. In the end, I'll come up with a simple Software Law (one of many). This chapter borrows extensively from the previous one, so if you didn't read chapter 13, it's probably better to do so before reading further.

Shapes, again
Forget about the Shape interface for a while. We now have two concrete classes: Triangle and Circle. They represent geometric shapes, this time without graphical responsibilities. A triangle is defined by 3 points, a Circle by center and radius. Given a Triangle and a Circle, we want to know the area of their intersection.
As usual, we have two problems to solve: the actual mathematical problem and the software design problem. If you're a mathematician, you may rightly think that solving the first (the function!) is important, and the latter is sort of secondary, if not irrelevant. As a software designer, however, I'll allow myself the luxury of ignoring the gory mathematical details, and tinker with form instead.

If you're working in a language (like C++) where functions exist [also] outside classes, a very reasonable form would be this:

double Intersection( const Triangle& t, const Circle& c )
  {
  // something here
  }

Note that I'm not trying to solve a more general problem. First, I said "forget the Shape interface"; second, I may not know how to solve the general problem of shapes intersection. I just want to deal with a circle and a triangle, so this signature is simple and effective.

If you're working in a class-only language, you have several choices.

1) Make that an instance method, possibly changing an existing class.

2) Make that a static method, possibly changing an existing class.

3) If your language has open classes, or extension methods, etc, you can add that method to an existing class without changing it (as an artifact).

So, let's sort this out first. I hope you can feel the ugliness of placing that function inside Triangle or Circle, either as an instance or static method (that feeling could be made more formal by appealing to coupling, to the open/closed principle, or to entanglement itself).
So, let's say that we introduce a new class, and go for a static method. At that point is kinda hard to find nice names for class and method (if you're thinking about ShapeHelper or TriangleHelper, you've been living far too long in ClassNameWasteLand, sorry :-). Having asked this question a few times in real life (while teaching basic OO thinking), I'll show you a relatively decent proposal:

class Intersection
  {
  public static double Area( Triangle t, Circle c )
    {
    // something here
    }
  }

It's quite readable on the calling site:

double a = Intersection.Area( t, c ) ;

Also, the class name is ok: a proper noun, not something ending with -er / -or that would make Peter Coad (and me :-) shiver. An arguably better alternative would be to pass parameters in a constructor and have a parameterless Area() function, but let's keep it like this for now.

Whatever you do, you end up with a method/function body that is deeply coupled with both Triangle and Circle. You need to get all their data to calculate the area of the intersection. Still, this doesn't feel wrong, does it? That's the method purpose. It's ok to manipulate center, radius, and vertexes. Weird, coupling is supposed to be bad, but it doesn't feel bad here.

Interlude
Let me change problem setting for a short while, before I come back to our beloved shapes. You're now working on yet another payroll system. You got (guess what :-) a Person class:

class Person
  {
  Place address;
  String firstName;
  String lastName;
  TelephoneNumber homePhone;
  // …
  }

(yeah, I know, some would have written "Address address" :-)

You also have a Validate() member function (I'll simplify the problem to returning a bool, not an error message or set of error messages).

bool Validate()
  {
  return 
    address.IsValid() && 
    ValidateName( firstName ) && 
    ValidateName( lastName ) && 
    homePhone.IsValid() ;
  }

Yikes! That code sucks! It sucks because it's asymmetric (due to the two String-typed members) but also because it's fragile. If I get in there and add "Email personalEmail ;" as a field, I'll have to update Validate as well (and more member functions too, and possibly a few callers). Hmmm. That's because I'm using my data members. Well, I'm using Triangle and Circle data member as well in the function above, which is supposed to be worse; here I'm using my own data members. Why was that right, while Validate "feels" wrong? Don't use hand-waving explanations :-).

Back to Shapes
A very reasonable request would be to calculate the intersection of two triangles or two circles as well, so why don't we add those functions too:

class Intersection
  {
  public static double Area( Triangle t, Circle c )
    {     // something here    }
  public static double Area( Triangle t1, Triangle t2 )
    {     // something here    }
  public static double Area( Circle c1, Circle c2 )
    {     // something here    }
  }

We need 3 distinct algorithms anyway, so anything more abstract than that may look suspicious. It makes sense to keep the three functions in the same class: the first function is already coupled with Triangle and Circle. Adding the other two doesn't make it any worse from the coupling perspective. Yet, this is wrong. It's a step toward the abyss. Add a Rectangle class, and you'll have to add 4 member functions to Intersection.

Interestingly, we came to this point without introducing a single switch/case, actually without even a single "if" in our code. Even more interestingly, our former Intersection.Area was structurally similar to Person.Validate, yet relatively good. Our latter Intersection is not structurally similar to Person, yet it's facing a similar problem of instability.

Let's stop and think :-)
A few possible reactions to the above:

- I'm too demanding. You can do that and nothing terrible will happen.
Probably true, a small scale software mess rarely kills. The same kind of problem, however, is often at the core of a large-scale mess.

- I'm throwing you a curve.
True, sort of. The intersection problem is a well-known double-dispatch issue that is not easily solved in most languages, but that's only part of the problem.

- It's just a matter of Open/Closed Principle.
Yeah, well, again, sort of. The O/C is more concerned with extension. In practice, you won't create a new Person class to add email. You'll change the existing one.

- I know a pattern / solution for that!
Me too :-). Sure, some acyclic variant of Visitor (or perhaps more sophisticated solutions) may help with shape intersection. A reflection+attribute-based approach to validation may help with Person. As usual, we should look at the moon, not at the finger. It's not the problem per se; it's not about finding an ad-hoc solution. It's more about understanding the common cause of a large set of problems.

- Ok, I want to know what is really going on here :-)
Great! Keep reading :-)

C/D-U-U Entanglement
Everything that felt wrong here, including the problem from Chapter 13, that is:

- the ShapeType-based Draw function with a switch/case inside

- Person.Validate

- the latter Intersection class, dealing with more than one case

was indeed suffering from the same problem, namely an entanglement chain: C/D-U-U. In plain English: Creation or Deletion of something, leading to an Update of something else, leading to an Update of something else. I think that understanding the full chain, and not simply the U-U part, makes it easier to recognize the problem. Interestingly, the former Intersection class, with just one responsibility, was immune from C/D-U-U entanglement.

The chain is easier to follow on the Draw function in Chapter 13, so I'll start from there. ShapeType makes it easy to understand (and talk about) the problem, because it's both Nominative and Extensional. Every concept is named: individual shape types and the set of those shape types (ShapeType itself). The set is also providing through its full extension, that is, by enumerating its values (that's actually why it's called an enumerated type). Also, every concrete shape is given a name (a struct where concrete shape data are stored).

So, for the old C-like solutions, entanglement works (unsurprisingly) like this:

- There is a C/D-C/D entanglement between members of ShapeType and a corresponding struct. Creation or deletion of a new ShapeType member requires a corresponding action on a struct.

- There is a C/D-U entanglement between members of ShapeType and ShapeType (this is obvious: any enumeration is C/D-U entangled with its constituents).

- There is U-U entanglement between ShapeType and Draw, or every other function that is enumerating over ShapeType (a switch/case being just a particular case of enumerating over names).

Consider now the first Intersection.Area. That function is enumerating over two sets: the set of Circle data, and the set of Triangle data. There is no switch/case involved: we're just using those data, one after another. That amounts to enumeration of names. The function is C/D-U-U entangled with any circle and triangle data. We don't consider that to be a problem because we assume (quite reasonably) that those sets won't change. If I change my mind and represent a Circle using 3 points (I could) I'll have to change Area. It's just that I do not consider that change any likely.

Consider now Person.Validate. Validate is enumerating over the set of Person data, using their names. It's not using a switch/case. It doesn't have to. Just using a name after another amounts to enumeration. Also, enumeration is not broken by a direct function call: moving portions of Validate into sub-functions wouldn't make any difference, which is why layering is ineffective here.
Here, Validate is U-U entangled with the (unnamed) set of Person data, or C/D-U-U entangled with any of its present or future data members. Unlike Circle or Triangle, we have all reasons to suspect the Person data to be unstable, that is, for C and/or D to occur. Therefore, Validate is unstable.

Finally, consider the latter Intersection class. While the individual Area functions are ok, the Intersection class is not. Once we bring in more than one intersection algorithm, we entangle the class with the set of shape types. Now, while in the trivial C-like design we had a visible, named entity representing the set of shapes (ShapeTypes), in the intersection problem I didn't provide such an artifact (on purpose). The fact that we're not naming it, however, does not make it any less real. It's more like we're dealing with an Intensional definition of the set itself (this would be a long story; I wrote an entire post about it, then decided to scrap it). Intersection is C/D-U-U entangled with individual shape types, and that makes it unstable.

A Software Law
One of the slides in my Physics of Software reads "software has a nature" (yeah, I need to update those slides with tons of new material). In natural sciences, we have the concept of Physical law, which despite the name doesn't have to be about physics :-). In physics, we do have quite a few interesting laws, not necessarily expressed through complex formulas. The Laws of thermodynamics , for instance, can be stated in plain English.

When it comes to software, you can find several "laws of software" around. However, most of them are about process (like Conway's law, Brooks' law, etc), and just a few (like Amdhal's law) are really about software. Some are just principles (like the Law of Demeter) and not true laws. Here, however, we have an opportunity to formulate a meaningful law (one of many yet to be discovered and/or documented):

- The Enumeration Law
every artifact (a function, a class, whatever) that is enumerating over the members of set by name is U-U entangled with that set, or said otherwise, C/D-U-U entangled with any member of the set.

This is not a design principle. It is not something you must aim for. It's a law. It's always true. If you don't like the consequences, you have to change the shape of your software so that the law no longer applies, or that entanglement is harmless because the set is stable.

This law is not earth-shattering. Well, the laws of thermodynamics don't look earth-shattering either, but that doesn't make them any less important :-). Honestly, I don't know about you, but I wish someone taught me software design this way. This is basically what is keeping me going :-).

Consequences
In Chapter 12, I warned against the natural temptation to classify entanglement in a good/bad scale, as it has been done for coupling and even for connascence. For instance, "connascence of name" is considered the best, or least dangerous, form. Yet we have just seen that entanglement with a set of names is at the root of many problems (or not, depending on the stability of that name set).

Forces are not good or evil. They're just there. We must be able to recognize them, understand how they're influencing our material, and deal with them accordingly. For instance, there are well-known approaches to mitigate dependency on names: indirection (through function pointers, polymorphism, etc) may help in some cases; double indirection (like in double-dispatch) is required in other cases; reflection would help in others (like Validate). All those techniques share a common effect, that we'll discuss in a future post.

Conclusions
Speaking of future: recently, I've been exploring different ways to represent entanglement, as I wasn't satisfied with my forcefield diagram. I'll probably introduce an "entanglement diagram" in my next post.

A final note: visit counters, sharing counters and general feedback tell me that this stuff is not really popular. My previous rant on design literature had way more visitors than any of my posts on the Physics of Software, and was far easier to write :-). If you're not one of the usual suspects (the handful of good guys who are already sharing this stuff around), consider spreading the word a little. You don't have to be a hero and tell a thousand people. Telling the guy in front of you is good enough :-).

Is Software Design Literature Dead?

Sometimes, my clients ask me what to read about software design. Most often than not, they don't want a list of books – many already have the classics covered. They would like to read papers, perhaps recent works, to further advance their understanding of software design, or perhaps something inspirational, to get new ideas and concepts. I have to confess I'm often at loss for suggestions.

The sense of discomfort gets even worse when they ask me what happened to the kind of publications they used to read in the late '90s. Stuff like the C++ Report, the Journal of Object Oriented Programming, Java Report, Object Expert, etc. Most don't even know about the Journal of Object Technology, but honestly, the JOT has taken a strong academic slant lately, and I'm not sure they would find it all that interesting. I usually suggest they keep current on design patterns: for instance, the complete EuroPLoP 2009 proceedings are available online, for free. However, mentioning patterns sometimes furthers just another question: what happened to the pattern movement? Keep that conversation going for a while, and you get the final question: so, is software design literature dead?

Is it?
Note that the question is not about software design - it's about software design literature. Interestingly, Martin Fowler wrote about the other side of the story (Is Design Dead?) back in 2004. He argued that design wasn't dead, but its nature had changed (I could argue that if you change the nature of something, then it's perhaps inappropriate to keep using the same name :-), but ok). So perhaps software design literature isn't dead either, and has just changed nature: after all, looking for "software design" on Google yields 3,240,000 results.
Trying to classify what I usually find under the "software design" chapter, I came up with this list:

- Literature on software design philosophy, hopefully with some practical applications. Early works on Information Hiding were as much about a philosophy of software design as about practical ways to structure our software. There were a lot of philosophical papers on OOP and AOP as well. Usually, you get this kind of literature when some new idea is being proposed (like my Physics of Software :-). It's natural to see less and less philosophy in a maturing discipline, so perhaps a dearth of philosophical literature is not a bad sign.

- Literature on notations, like UML or SysML. This is not really software design literature, unless the rationale for the notation is discussed.

- Academic literature on metrics and the like. This would be interesting if those metrics addressed real design questions, but in practice, everything is always observed through the rather narrow perspective of correlation with defects or cost or something appealing for manager$. In most cases, this literature is not really about software design, and is definitely not coming from people with a strong software design background (of course, they would disagree on that :-)

- Literature on software design principles and heuristics, or refactoring techniques. We still see some of this, but is mostly a rehashing of the same old stuff from the late '90s. The SOLID principles, the Law of Demeter, etc. In most cases, papers say nothing new, are based on toy problems, and are here just because many programmers won't read something that has been published more than a few months ago. If this is what's keeping software design literature alive, let's pull the plug.

- Literature on methods, like Design by Contract, TDD, Domain-Driven Design, etc. Here you find the occasional must-read work (usually a book from those who actually invented the approach), but just like philosophy, you see less and less of this literature in a maturing discipline. Then you find tons of advocacy on methods (or against methods), which would be more interesting if they involved real experiments (like the ones performed by Simula labs) and not just another toy projects in the capable hands of graduate students. Besides, experiments on design methods should be evaluated [mostly] by cost of change in the next few months/years, not exclusively by design principles. Advocacy may seem to keep literature alive, but it's just noise.

- Literature on platform-specific architectures. There is no dearth of that. From early EJB treatises to JBoss tutorials, you can find tons of papers on "enterprise architecture". Even Microsoft has some architectural literature on application blocks and stuff like that. Honestly, in most cases it looks more like Markitecture (marketing architecture as defined by Hohmann), promoting canned solutions and proposing that you adapt your problem to the architecture, which is sort of the opposite of real software design. The best works usually fall under the "design patterns" chapter (see below).

- Literature for architecture astronauts. This is a nice venue for both academics and vendors. You usually see heavily layered architectures using any possible standards and still proposing a few more, with all the right (and wrong) acronyms inside, and after a while you learn to turn quickly to the next page. It's not unusual to find papers appealing to money-saving managers who don't "get" software, proposing yet another combination of blueprint architectures and MDA tools so that you can "generate all your code" with a mouse click. Yeah, sure, whatever.

- Literature on design patterns. Most likely, this is what's keeping software design literature alive. A well-written pattern is about a real problem, the forces shaping the context, an effective solution, its consequences, etc. This is what software design is about, in practice. On the other hand, a couple of conferences every year can't keep design literature in perfect health.

- Literature on design in the context of agility (mostly about TDD). There is a lot of this on the net. Unfortunately, it's mostly about trivial problems, and even more unfortunately, there is rarely any discussion about the quality of the design itself (as if having tests was enough to declare the design "good"). The largest issue here is that it's basically impossible to talk about the design of anything non-trivial strictly from a code-based perspective. Note: I'm not saying that you can't design using only code as your material. I'm saying that when the problem scales slightly beyond the toy level, the amount of code that you would have to write and show to talk about design and design alternatives grows beyond the manageable. So, while code-centric practices are not killing design, they are nailing the coffin on design literature.

Geez, I am usually an optimist :-)), but this picture is bleak. I could almost paraphrase Richard Gabriel (of "Objects have failed" fame) and say that evidently, software design has failed to speak the truth, and therefore, software design narrative (literature) is dying. But that would not be me. I'm more like the opportunity guy. Perhaps we just need a different kind of literature on software design.

Something's missing
If you walk through the aisles of a large bookstore, and go to the "design & architecture" shelves, you'll all the literary kinds above, not about software, but about real-world stuff (from chairs to buildings). except on the design of physical objects too. Interestingly, you can also find a few morelike:

- Anthologies (presenting the work of several designers) and monographs on the work of famous designers. We are at loss here, because software design is not immediately visible, is not interesting for the general public, is often kept as a trade secret, etc.
I know only one attempt to come up with something similar for software: "Beautiful Architecture" by Diomidis Spinellis. It's a nice book, but it suffers from a lack of depth. I enjoyed reading it, but didn't come back with a new perspective on something, which is what I would be looking for in this kind of literature.
It would be interesting, for instance, to read more about the architecture of successful open-source projects, not from a fanboy perspective but through the eyes of experienced software designers. Any takers?

- Idea books. These may look like anthologies, but the focus is different. While an anthology is often associated with a discussion or critics of the designer's style, an idea book presents several different objects, often out of context, as a sort of creative stimulus. I don't know of anything similar for software design, though I've seen many similar book for "web design" (basically fancy web pages). In a sense, some literature on patterns comes close to being inspirational; at least, good domain-specific patterns sometimes do. But an idea book (or idea paper) would look different.

I guess anthologies and monographs are at odd with the software culture at large, with its focus on the whizz-bang technology of the day, little interest for the past, and often bordering on religious fervor about some products. But idea books (or most likely idea papers) could work, somehow.

Indeed, I would like to see more software design literature organized as follows:

- A real-world, or at least realistic problem is presented.

- Forces are discussed. Ideally, real-world forces.

- Possibly, a subset of the whole problem is selected. Real-world problems are too big to be dissected in a paper (or two, or three). You can't discuss the detailed design of a business system in a paper (lest you appeal only to architecture astronauts). You could, however, discuss a selected, challenging portion. Ideally, the problem (or the subset) or perhaps just the approach / solution, should be of some interest even outside the specific domain. Just because your problem arose in a deeply embedded environment doesn't mean I can't learn something useful for an enterprise web application (assuming I'm open minded, of course :-). Idea books / papers should trigger some lateral thinking on the reader, therefore unusual, provocative solutions would be great (unlike literature on patterns, where you are expected to report on well-known, sort of "traditional" solutions).

- Practical solutions are presented and scrutinized. I don't really care if you use code, UML, words, gestures, whatever. Still, I want some depth of scrutiny. I want to see more than one option discussed. I don't need working code. I'm probably working on a different language or platform, a different domain, with different constraints. I want fresh design ideas and new perspectives.

- In practice, a good design aims at keeping the cost of change low. This is why a real-world problem is preferable. Talking about the most likely changes and how they could be addressed in different scenarios beats babbling about tests and SOLID and the like. However, "most likely changes" is meaningless if the problem is not real.

Funny enough, there would be no natural place to publish something like that, except your own personal page. Sure, maybe JOT, maybe not. Maybe IEEE Software, maybe not. Maybe some conference, maybe not. But we don't have a software design journal with a large readers pool. This is part of the problem, of course, but also a consequence. Wrong feedback loop :-).

Interestingly, I proposed something similar years ago, when I was part of the editorial board of a software magazine. It never made a dent. I remember that a well-known author argued against the idea, on the basis that people were not interested in all this talking about ins-and-outs, design alternatives and stuff. They would rather have a single, comprehensive design presented that they could immediately use, preferably with some source code. Well, that's entirely possible; indeed, I don't really know how many software practitioners would be interested in this kind of literature. Sure, I can bet a few of you guys would be, but overall, perhaps just a small minority is looking for inspiration, and most are just looking for canned solutions.

Or maybe something else...
On the other hand, perhaps this style is just too stuck in the '90s to be appealing in 2011. It's unidirectional (author to readers), it's not "social", it's not fun. Maybe a design challenge would be more appealing. Or perhaps the media are obsolete, and we should move away from text and graphics and toward (e.g.) videos. I actually tried to watch some code kata videos, but the guys thought they were like this, but to an experienced designer they looked more like this. (and not even that funny). Maybe the next generation of software designers will come up with a better narrative style or media.

Do something!
I'm usually the "let's do something" kind of guy, and I've entertained the idea of starting a Software Design Gallery, or a Software Design Idea Book, or something, but honestly, it's a damn lot of work, and I'm a bit skeptical about the market size (even for a free book/paper/website/whatever). As usual, any feedback is welcome, and any action on your part even more :-)

Can we really control the quality/cost trade-off?

An old rule in software project management (also known as the Project triangle is that you can have a system that:
- has low development cost (Cheap)
- has short development time (Fast)
- has good quality (Good)
except you can only choose two :-). That seem to suggest that if you want to go fast, and don't want to spend more, you can only compromise on quality. Actually, you can also compromise on feature set, but in many real-world cases, that happens only after cost, time and quality have already been compromised.

So, say that you want to compromise on quality. Ok, I know, most engineers don't want to compromise on quality; it's almost like compromising on ethics. Still, we actually do, more often that we'd like to, so let's face the monster. We can:

- Compromise on external quality, or quality as perceived by the user.
Ideally, this would be simply defined as MTBF (Mean Time Between Failures), although for many users, the perceived "quality" is more akin to the "user experience". In practice, it's quite hard to predict MTBF and say "ok, we won't do error checking here because that will save us X development days, and only decrease MTBF by Y". Most companies don't even measure MTBF. Those who do, use it just as a post-facto quality goal, that is, they fix bugs until they reach the target MTBF. We just don't know enough, in the software engineering community, to actually plan the MTBF of our product.

- Compromise on internal quality, or quality as perceived by developers.
Say goodbye to clarity of names, separation of concerns, little or no duplication of logic, extendibility in the right places, low coupling, etc. In some cases, poor internal quality may resurface as poor external quality (say, there is no error handling in place, so the code just crashes on unexpected input), but in many cases, the code can be rather crappy but appear to work fine.

- Compromise on process, or quality as perceived by the Software Engineering Institute Police :-).
What usually happens is that people start bypassing any activity that may seem to slow them down, like committing changes into a version control system :-) or being thorough about testing. Again, some issues may resurface as external quality problems (we don't test, so the user gets the thrill of discovering our bugs), but users won't notice if you cut corners in many process areas (well, they won't notice short-term, at least).

Of course, a "no compromises" mindset is dangerous anyway, as it may lead to an excess of ancillary activities not directly related to providing value (to the project and to users). So, ideally, we would choose to work at the best point in the quality/time and/or quality/cost continuum (the "best point" would depend on a number of factors: more on this later). Assuming, however, that there is indeed a continuum. Experience taught me the opposite: we can only move in discrete steps, and those are few and rather far apart.

Quality and Cost
Suppose I'm buying a large quantity of forks, wholesale. Choices are almost limitless, but overall the real choice is between:

- the plastic kind (throwaway)

it works, sort of, but you can't use it for all kinds of food, you can't use it while cooking, and so on.

- the "cheap steel" kind.

it's surely more resistant than plastic: you can actually use it. Still, the tines are irregular and feel uncomfortable in your mouth. Weight isn't properly balanced. And I wouldn't bet on that steel actually being stainless.

- the good stainless steel kind.

Properly shaped, properly balanced, good tasting :-), truly stainless, easy to clean up. You may or may not have decorations like that: price doesn't change so much (more on this below).

- the gold-plated kind.

it shows that you got money, or bad taste, or both :-). Is not really better than the good stainless fork, and must be handled more carefully. It's a real example of "worse is better".

Of course, there are other kind of forks, like those with resin or wood handles (which ain't really cheaper than the equivalent steel kind, and much worse on maintenance), or the sturdy plastic kind for babies (which leverage parental anxiety and charge a premium). So, the quality/cost landscape does not get much richer than that. Which brings us to prices. I did a little research (not really exhaustive – it would take forever :-), but overall it seems like, normalizing on the price of the plastic kind, we have this price scale:

Plastic: 1
Cheap steel: 10
Stainless steel: 20
Gold-plated: 80

The range is huge (1-80). Still, we have a big jump from throwaway to cheap (10x), a relatively small jump from cheap to good (2x), and another significant jump between good and gold-plated (4x). I think software is not really different. Except that a 2x jump is usually considered huge :-).

Note that I'm focusing on building cost, which is the significant factor on market price for a commodity like forks. Once you consider operating costs, the stainless steel is the obvious winner.

The Quality/Cost landscape for Software
Again, I'll consider only building costs here, and leave maintenance and operation costs aside for a moment.

If you build throwaway software, you can get really cheap. You don't care about error handling (which is a huge time sink). You copy&paste code all over. You proceed by trial and error, and sometimes patch code without understanding what is broken, until it seems to work. You keep no track of what you're doing. You use function names like foo and bar. Etc. After all, you just want to run that code once, and throw it away (well, that's the theory, anyway :-).
We may think that there is no place at all for that kind of code, but actually, end-user programming often falls into that case. I've also seen system administration scripts, lot of scientific code, and quite a bit of legacy business code that would easily fall into this category. They were written as throwaways, or by someone who only had experience writing throwaways, and somehow they didn't get thrown away.

The next step is very far apart: if you want your code to be any good, you have to spend much more. It's like the plastic/"cheap steel" jump. Just put decent error handling in place, and you'll spend way more. Do a reasonable amount of systematic testing (automated or not) and you're gonna spend more. Aim for your code to be reasonably readable, and you'll have to think more, go back and refactor as you learn more, etc. At this stage, internal quality is still rather low; code is not really stainless. External quality is acceptable. It doesn't feel good, but it gets [some] work done. A lot of "industrial" code is actually at this level. Again, maintenance may not be cheap, but I'm only considering building costs here; a 5x to 10x jump compared to throwaway code is quite reasonable.

The next step is when you invest more in -ilities, as well as in external quality through more systematic and deeper testing. You're building something that you and your users like. It's truly stainless. It feels good. However, it's hard to be "just a little stainless". It's hard to be "somehow modular" or "sort of thread-safe". Either you're on one side, or you're on the other. If you want to be on the right side, you're gonna spend more. Note that just like with fork, long-term cost is lower for good, stainless code, but as far as building cost is concerned, a 2X increase compared to "cheap steel" code is not surprising.

Gold-plating is obvious: grandiose design that serves no purpose (and that would appear as gratuitous complexity to a good designer), coding standards that add no value (like requiring a comment on each and every parameter of every function), etc. I've seen systems spending a lot of energy trying to be as scalable as Amazon, even though they only had to serve a few hundred concurrent clients. Goldplated systems spend more on building, and may also incur higher maintenance costs afterward (be careful not to scratch the gold :-). Of course, it takes experience to recognize goldplating. To someone that has never designed a system for testability, interfaces for mocking may seem like a waste. To an experienced designer, they tell a different story.

But software is... soft!
As usual, a metaphor can be stretched only so far. Software quality can be easily altered over time. Plastic can't be transformed into gold (if you can, give me a call :-), but we can release a bug-ridden version and fix issues later. We may incur intentional technical debt and pay it back in the next release. This could be the right strategy in the right market at the right time, but that's not always the case.

Indeed, although start-ups, customer discovery, innovative products, etc seem to get all the attention lately, if you happen to work in a mature market you may want to focus on early-on quality. Some customers are known to be forgiving about quality issues, in exchange for early access to innovative products. It's simply not the general case.

Modular Quality?
Theoretically, we could invest more in some "critical" components and less in others. After all, we don't need the same quality (external and internal) in every single portion. We can accept quirks here and there. Due to the fractal nature of software, this can be extended to the entire hierarchy of artifacts. It's surely possible. However, it takes a lot of maturity to develop on the edge of quality. Having to constantly reason about the quality/cost trade off can easily cost you more than you're saving: I'd rather spend that reasoning to create quality code.

Once again, however, we may have a plan to start with plastic and grow into stainless steel over time. We must be aware that it is easier said than done. I've seen it many times over my long career. Improving software quality is just another case of moving our software in the decision space. We decided to make software out of plastic early on, and now we want to move it to another place, say the cheap steel spot. Some work is required. Work is proportional to the distance (which is large – see above), but also to the opposing force. Whatever form that force takes, a significant factor is always the mass to be moved, and here lies the key. It's hard to move a monolithic mass of code into another place (in the decision space). It's easier if you can move it a module at time.

Within a small modular unit, you can easily improve local quality. Change a switch/case into polymorphism. Add an interface to decouple two concrete classes. It's easy and it's cheap, because mass is small. Say, however, that you got your modular structure wrong: an important concept has not been identified, and is now scattered all around. You have to tweak a lot of code. While doing so, you'll find that you're not just fighting the mass you have to move: you have to fight the gravitational attraction of the surrounding code. That may reveal more scattered concepts. It's a damn lot of work, possibly more than it's worth. Software is not necessarily soft.

So, can we really move software around? Yes, in the small. Yes, if you got the modular structure right, and you're only cleaning up within that modular structure. Once again, the fractal nature of software can be our best friend or our worst enemy. Get the right partitioning between applications / services, and you only have to work inside. Get the right partitioning between components, and you only have to work inside. And so on. It's fine to have a plastic component, inasmuch as the interface is right. Get the modular structure wrong, and sooner or later you'll need to make wide-range changes, or accept a "cheap steel" quality. If we got it wrong at a small scale, it's no big deal. At a large scale, it can be a massacre.

Conclusions
Once again, modularity is the key. Not just "any" modularity, but a modularity aligned with the underlying forcefield. If you get that right, you can build plastic components and evolve them to stainless steel quality over time. Been there, done that. Get your modular structure wrong, and you'll lack locality of action. In other words: if you plan on cutting corners inside a module, you have to invest at the interface level. That's simpler if you design top-down than if you're in the "emergent design" camp, because the module will be underengineered inside, and we can't expect a clean interface to emerge from underengineered code.

Notes on Software Design, Chapter 13: On Change

We have a long history of manipulating physical materials. Cavemen built primitive tools out of rocks and sticks. I would speculate :-) that they didn't know much about physics, yet through observation, serendipity and good old trial and error they managed to survive pretty well, or we wouldn't be here today talking about software.

For centuries, we only had empirical observations and haphazard theories about the real nature of the physical world. The Greek Classical Element were Earth, Water, Air, Fire, and Aether, which is not exactly the way we look at materials today. Yet the Greeks managed to build large and incredibly beautiful temples.

Of course, today we know a quite a bit more. Take any kind of physical material, like a copper wire. In the real world, you can:

- apply thermal stress to the material, e.g. by warming it. Different materials react differently.

- apply mechanical stress to the material, e.g. by pulling it on both ends. Different materials react differently.

- apply electromagnetic forces, e.g. by moving the wire inside a magnetic field. Different materials react differently.

- etc.

To get just a little more into specifics, if we restrict ourselves to mechanical stress, and focus on the so-called static loading, we end up with just a few kind of stress: tension, compression, bending, torsion, and shear (see here and also here).

Although people have built large, beautiful structures without any formal knowledge of material science, it is hard to deny the huge progress made possible by a deeper understanding of what is actually going on with materials. Knowledge of forces and reactions allows us to choose the best material, the best shape, and even to create new, better materials.

The software world
Although we can build large, even beautiful systems, it is hard to deny that we are still in the dark ages. We have vague principles that usually can't be formally defined. We have different materials, like statically typed and dynamically typed languages, yet we don't have a sound theory of forces, so we end up with the usual evangelist trying to convert the world to its material of choice. It's kinda funny, because it's just as sensible as having a Brick Evangelist trying to convince you to use bricks instead of glass for your windows ("it's more robust!"), and a Glass Evangelist trying to convince you to build your entire house out of glass ("you'll get more light!").

Indeed, a key realization in my exploration on the nature of software was that we completely lack the notion of force. Sure, we use a bunch of terms like "flexibility", "heavy load", "fast", "fragile", etc., and they all seem to be somehow inspired by physics, but in the end it's just vernacular usage without any kind of theory behind.

As I realized that, I began looking for a small, general yet useful set of "software forces". Nothing too fancy. Tension, compression, bending, torsion, and shear are not fancy, but are incredibly useful for physical materials (no, I wasn't looking for similar concepts in software, but for something just as useful).

Initially, I focused my attention on artifacts, because design is very much about form, and form is made visible through artifacts. I got many ideas and explored several avenues, but most seemed more like ad hoc concepts or turned out to be dead ends. Eventually, I realized we already knew the fundamental forces, though we never used the term "force" to characterize them.

Interestingly, those forces are better known in the run-time world, and are usually restricted to data. Upon closer scrutiny, it turned out they also apply to procedural knowledge, and to the artifact world as well. Also, it turned out that those forces were intimately connected with the concept of entanglement, and could actually shed some light on entanglement itself. More than that, they could be used to find entangled information, in practice, no more philosophy, thank you :-).

So, let's take the easy road and see what we can do with run-time knowledge expressed through data, and build from there.

It's almost too simple
Think about a simple data-oriented application, like a book library. What can you do with a Book record? It's rather obvious: create, read, update, delete. That's it. CRUD. (I'm still thinking about the need for an identity-preserving Move outside the database realm, but let's focus on CRUD right now). CRUD for run-time data is trivial, and is a well-known concept.

CRUD in the artifact world is also relatively simple and intuitive:

Create: we create a new artifact, being it a component, a class, a function.

Read: this got me thinking for a while. In the end, my best shot is also the simplest: we (the contingent intepreter) read the function, as an act of understanding. More on the idea of contingent/essential interpreter in a rather whimsical :-) presentation by Richard Gabriel, that I already suggested a few years ago.

Update: we change an artifact; for instance, we change the source code of some function.

Delete: we remove an artifact; for instance, we remove a member function from a class.

Note that given the fractal nature of software, some operations are recursively identical (by deleting a component we delete all the sub-artifacts), while others change nature as we move into the fractal hierarchy (by creating a new class inside a component we're also updating the component). This is true in the run-time world of data as well.

Extending CRUD to run-time procedural knowledge is not necessarily trivial. Can we talk about CRUD for functions (as run-time, not compile-time, concepts)? We usually don't, but that's because the most common material and tools (programming languages) we're using are largely based on the idea that procedural knowledge is created once (through artifacts), then compiled and executed.

However, interpreted languages always had the possibility of creating, updating and deleting functions at run-time. Some compiled languages allow dynamic function creation (like C#, through Reflection.Emit) but not update/delete. The ability to update a function through reflection would be an important step toward AOP-like possibilities, but most languages have no support for that. This is fine, though: liquids can't be compressed, but that didn't prevent us from defining the concept of compression. I'm looking for concepts that apply in both worlds (artifacts and run-time) but not necessarily to all materials (languages, etc.). So, to summarize:

- Create, Update, Delete retain the usual meaning also for the run-time concept of functions. It means we're literally creating, updating and deleting a function at run-time. The same applies to the (run-time) concept of class (not object). We can Create, Update, Delete classes at run-time as well (given the appropriate language/material).

- Read, in the artifact world, is the human act of reading, or better, is the contingent interpreter (us) giving meaning to (interpreting) the function. At run-time, the equivalent notion is execution, with the essential interpreter (the machine) giving meaning to the function. Therefore, reading a function in the run-time world simply means executing the function. Executing a function may also update some data, but this is fine: the function is read, data are updated; there is no contradiction.

Where do we go from here?
I'm trying to keep this post short, so I'll necessarily leave a lot of stuff for next time. However, I really want to anticipate something. In my previous post I said:

Two clusters of information are entangled when performing a change on one immediately requires a change on the other.

I can now state that better as:

Two clusters of information are entangled when a C/R/U/D on one immediately requires a C/R/U/D on the other.

Yes, the two definitions are not equivalent, as Read is not a Change. The second definition, however, is better, for reasons we'll explore in the next few posts. One reason is that it provides guidance on the kind of "change" you need to consider. Just restrict yourself to CRUD, and you'll be fine. As we'll see next, we can usually group CD and RU, and in fact I've defined two concepts representing those groupings.

Entanglement through CD and/or RU is not necessarily a bad thing. It means that the entangled clusters are attracting each other, and, inasmuch as that specific force is concerned, rejecting others. In other words, they want to form a single cluster, possibly a composite cluster in the next hierarchical level (like two functions attracting themselves and creating a class). Of course, it might also be the case that the two clusters should be merged into a single cluster.

We'll see more about different form of CRUD-induced entanglement in the next few posts. We'll also see how many concepts (from references to database normalization) are actually just ways to deal with the many forms of entanglement, or to move entanglement from the artifact space to the run-time space. Right now, I'll use my remaining time to explain polymorphism through the entanglement perspective.

Polymorphism explained
Consider the usual idea of multiple shapes (circle, triangle, rectangle, etc.) that can be drawn on a screen. In the pre-object era (say, in ANSI C) we could have defined a ShapeType enum, a Shape struct with a ShapeType field and possibly a ShapeData field, where ShapeData would have probably been a union (with all fields for all different shapes). Draw would have been implemented as a switch/case over the ShapeType field.

We have U/U entanglement between ShapeType and ShapeData, and between ShapeType and Draw. U/U means that whenever we update ShapeType we need to update ShapeData as well. I add the Spiral type, so I need to add the relevant fields in the ShapeData union, and add logic to Draw. We also have U/U entanglement between ShapeData and Draw: if we change the way we store shape data, we have to change the Draw function as it depends on those data. We could live with that, but is not pretty.

What if we have another function that depends on ShapeType, like Area? Well, we need another switch-case, and Area will have the same kind of U/U entanglement with ShapeType and ShapeData as Draw. It is that entanglement that is bringing all those concepts together. ShapeType, ShapeData, Draw and Area want to form a cluster.

Note that without polymorphism (language-supported or simulated through function pointers) the code mass is organized into a potentially degenerative form: every function (Draw, Area, etc) is U/U-entangled with ShapeType, and is therefore attracting new procedural knowledge related to each and every ShapeType value. The forcefield will make those functions bigger and bigger, unless of course ShapeType is no longer extended.

Object-oriented polymorphism "works" because it changes the gravitational centers. Each concrete class (Circle, Triangle, etc) becomes a gravitational center for procedural and structural knowledge entangled with a single (ex)ShapeType value. In fact, ShapeType no longer needs to exist at all. There is still U/U-entanglement between the (ex)ShapeData portion related to Circle and the procedural knowledge in Circle.Draw and Circle.Surface (in a Java/C#ish syntax). But this is fine, because they have now been brought together into a new cluster (the Circle class) that then rejected the other artifacts (the rest of the ShapeData fields, the other functions). As I said in my previous post, entanglement between distant element is bad.

Note that we don't really change the mass of code. We just organize it differently, around different gravitational centers. If you draw functions as tall boxes in the switch-case form, you can see that classes are just "wide" boxes cutting through functions. The area of the boxes (the amount of code) does not change.

Object-oriented polymorphism in a statically-typed language also requires [interface] inheritance. This brings in a new form of U/U-entanglement, this time between the interface and each and every concrete class. Whenever we change the interface, we have to change all the concrete classes.

Interestingly, some of those functions might be entangled in the run-time world: they tend to be called together (R/R-entanglement in the run-time space). That would suggest yet another clustering (in the artifact space), like separating all the drawing operations from the geometrical operations in different interfaces.

Note that the polymorphic solution is not necessarily better: it's just a matter of probability. If the set of functions is unstable, plain polymorphism is not really a good answer. We may want to explore a more complex solution based on the Visitor pattern, or some other clustering of artifacts. Understanding entanglement for Visitor is indeed an interesting exercise.

Coming soon (I hope :-)
CD and RU-entanglement (with their real names :-), paving our way to the concept of isolation, or shielding, or whatever I'm gonna settle for. Oh guys, by the way: happy new year!

Notes on Software Design, Chapter 12: Entanglement

Mechanical devices require maintenance, some more than others. Maintenance is intended to keep them working, because mechanical parts tend to wear out. Maintenance is required so that they can still work "as new", that is, do the same thing they were built to do.

Software programs require maintenance, some more than others. Maintenance is intended to keep them useful, because software parts don't wear out, but they get obsolete. They solve yesterday's problems, not today's problems. Or perhaps they didn't even solve yesterday's problems right (bugs). Maintenance is required so that software can so something different from what it was built to do.

Looking from a slightly different perspective, software is encoded knowledge, and the value of knowledge is constantly decaying (see the concept of half-life of knowledge). Maintenance is required to restore value. Energy (human work) must be supplied to keep encoded knowledge current, that is, valuable.

Either way, software developers spend a large amount of their time changing stuff.

Change
Ideally, changes would be purely additional. We would write a new software entity (e.g. a class) inside a new artifact (a source file). We would transform that artifact into something executable (if needed; interpreted languages don't need this step). We would deploy just the new executable artifact. The system would automagically discover the new artifact and start using it.

Although it's possible to create systems like that (I've done it many times through plug-in architectures), programs are not written this way from the ground up. Plug-ins, assuming they exist at all, usually appears at a relatively coarse granularity. Below that level, maintenance is rarely additional. Besides, additional maintenance is only possible when the change request is aligned with the underlying architecture.

So, the truth is, most often we can't simply add new knowledge to the system. We also have to change and remove existing parts. A single change request then results in a wave of changes, propagating through the system. Depending on some factors (that I'll discuss later on) the wave could be dampened pretty soon, or even amplified. In that case, we'll have to change more and more parts as the wave propagates through the system. If we change a modularized decisions, the change wave is dampened at module boundary.

The nature of change
Consider a system with two hardware nodes. The nodes are based on completely different hardware architectures: for instance, node 1 is a regular PC, node 2 is a DSP+ASIC embedded device. They are connected through standard transport protocols. Above transport, they don't share a single line of code, because they don't calculate the same function.
Yet, if you change the code deployed in node 1, you have to change some (different) code in node 2 as well, or the system won't work. Yikes! Aren't those two systems loosely coupled according to the traditional coupling theory? Sure. But still, changes need to happen on both sides.

The example above may seem paradoxical, or academic, but is not. Most likely, you own one of those devices. It's a decoder, like those for DVB-T, sat TV, or inside DivX players. The decoder is not computing the same function as the encoder (in a sense, it's computing the inverse function). The decoder may not share any code at all with the encoder. Still, they must be constantly aligned, or you won't see squat.

This is the real nature of change in software: you change something, and a number of things must be changed at the same time to preserve correctness. In a sense, change is affecting very distant, even physically disconnected components, and must be simultaneous: change X, and Y, Z, K must all change, at once.

At this point, we may think that all the physical analogies are breaking down, because this is not how the physical world behaves. Except it does :-). You just have to look at the right scale, and move from the familiar Newtonian physics to the slightly less comfortable (for me, at least) quantum physics.

Quantum Entanglement
Note: if you really, really, really hate physics, you can skip this part. Still, if you never came across the concept of Quantum Entanglement, you may find it interesting. Well, when I first heard of it, I thought it was amazing (though I didn't see the connection with software back then).

You probably know about the Heisenberg Uncertainty Principle. Briefly, it says that when you go down to particles like photons, you can't measure (e.g.) both position and wavelength at arbitrarily high precision. It's not a problem of measurement techniques. It's the nature of things.

Turns out, however, that we can create entangled particles. For instance, we can create two particles A and B that share the same exact wavelenght. Therefore, we can try to circumvent the uncertainty principle by measuring particle A wavelength, and particle B position. Now, and this is absolutely mind-blowing :-), it does not work. As soon as you try to measure particle B position, particle A reacts, by collapsing its wave function, immediately, even at an arbitrary distance. You cannot observe A without changing B.

Quoting wikipedia: Quantum entanglement [..] is a property of certain states of a quantum system containing two or more distinct objects, in which the information describing the objects is inextricably linked such that performing a measurement on one immediately alters properties of the other, even when separated at arbitrary distances.

Replace measurement with change, and that's exactly like software :-).

Software Entanglement
The parallel between entangled particles and entangled information is relatively simple. What is even more interesting, it works both in the artifact world and in the run-time world, at every level in the corresponding hierarchy (the run-time/artifact distinction is becoming more and more central, and was sorely missing in most previous works on related concepts). On the other hand, the concept of entanglement has far-reaching consequences, and is also a trampoline to new concepts. This post, therefore, is more like a broad introduction than an in-depth scrutiny.

Borrowing from quantum physics, we can define software entanglement as follows:

Two clusters of information are entangled when performing a change on one immediately requires a change on the other.

A simple example in the artifact space is renaming a function. All (by-name) callers must be changed, immediately. Callers are entangled with the callee. A simple example in the run-time space is caching. Caching requires cache coherence mechanisms, exactly because of entanglement.

Note 1: I said "by name", because callers by reference are not affected. This is strongly related with the idea of dampening, and we'll explore it in a future post.

Note 2: for a while, I've been leaning on using "tangling" instead of "entanglement", as it is a more familiar word. So, in some previous posts, you'll find mentions to "tangling". In the end, I decided to go with "entanglement" because "tangling" has already being used (with different meaning) in AOP literature, and also because entanglement, although perhaps less familiar, is simply more precise.

Not your granpa coupling
Coupling is a time-honored concept, born in the 70s and survived to this day. Originally, coupling was mostly concerned with data. Content coupling took place when a module was tweaking another module's internal data; common coupling was about sharing a global variable; etc. Some forms of coupling were considered stronger than others.

Most of those concepts can be applied to OO software as well, although most metrics for OO coupling takes a more simplified approach and mostly consider dependencies as coupling. Still, some forms of dependency are considered stronger than others. Inheritance is considered stronger than composition; dependency on a concrete class is considered stronger than dependency on an interface; etc. Therefore, the mere presence of an interface between two classes is assumed to reduce coupling. If class A doesn't talk to class B, and doesn't even know about class B existence, they are considered uncoupled.

Of course, lack of coupling in the traditional sense does not imply lack of entanglement. This is why so many attempts to decouple systems through layers are so ineffective. As I said many times, all those layers in business systems can't usually dampen the wave of changes coming from the (seemingly) innocent need to add a field to a database table. In every layer, we usually have some information node that is tangled with that table. Data access; business logic; user interface; they all need to change, instantly, so that this new field will be put to use. That's entanglement, and it's not going away by layering.

So, isn't entanglement just coupling? No, not really. Many systems that would be defined as "loosely coupled" are indeed "heavily entangled" (the example above with the encoder/decoder is the poster child of a loosely coupled / heavily entangled system).

To give honor to whom honor is due, the closest thing I've encountered in my research is the concept of connascence, introduced by Meilir Page-Jones in a little known book from mid-90s ("What Every Programmer Should Know About Object-Oriented Design"). Curiously enough, I came to know connascence after I conceived entanglement, while looking for previous literature on the subject. Stille, there are some notable differences between connascence and entanglement, that I'll explore in the forthcoming posts (for instance, Page-Jones didn't consider the RT/artifact separation).

Implications
I'll explore the implications of entanglement in future posts, where I'll also try to delve deeper into the nature of change, as we can't fully understand entanglement until we fully understand change.

Right now, I'd like to highlight something obvious :-), that is, having distant yet entangled information is dangerous, and having too much tangled information is either maintenance hell or performance hell (artifact vs run-time).

Indeed, it's easy to classify entanglement as an attractive force. This is an oversimplification, however, so I'll leave the real meat for my next posts.

Beyond Good and Evil
There is a strong urge inside the human being to classify things as "good" and "bad". Within each category, we further classify things by degree of goodness or badness. Unsurprisingly, we bring this habit into computer science.

Coupling has long been sub-classified by "strength", with content coupling being stronger than stamp coupling, in turn stronger than data coupling. Even connascence has been classified by strength, with e.g. connascence of position being stronger than connascence of name. The general consensus is that we should aim for the weakest possible form of coupling / connascence.

I'm going to say something huge now, so be prepared :-). This is all wrong. When two information nodes are entangled, changes will propagate, period. The safest entanglement is the one with the minimum likelihood of occurrence. Of course, that must be weighted with the number of entangled nodes. It's very similar to risk exposure (believe or not, I couldn't find a decent link explaining the concept to the uninitiated :-).

There is more. We can dampen the effects of some forms of entanglement. That requires human work, that is, energy. It's usually upfront work (sorry guys :-), although that's not always the case. Therefore, even if I came up with some classification of good/bad entanglement, it would be rather pointless, because dampening is totally context-dependent.

I'll explore dampening in future posts, of course. But just to avoid excessive vagueness, think about polymorphism: it's about dampening the effect of a change (which change? Which other similar change is not dampened at all by polymorphism?). Think about the concept of reference. It's about dampening the effect of a change (in the run-time space). Etc.

Your system is not really aligned with the underlying forcefield unless you have strategies in place to dampen the effect of entanglement with high risk exposure. Conversely, a deep understanding of entanglement may highlight different solutions, where entangled information is kept together, as to minimize the cost of change. Examples will follow.

What's next?
Before we can talk about dampening, we really need to understand the nature of change better. As we do that, we'll learn more about entanglement as well, and how many programming and design concepts have been devised to deal with entanglement, while some concepts are still missing (but theoretically possible). It's a rather long trip, but the sky is clear :-).

Design, Structure, and Decisions

Joe is a junior programmer in the IT department of SomeLargeCompany, Inc. His company is not in the business of software; however, they have a lot of custom programs that are constantly developed, adapted, tweaked to follow the ever-changing landscape of products, sales, regulations, etc.

Joe is largely self-taught. He doesn't know much about software engineering, but in the end, he gets things done.

Right now, Joe is writing some code dealing with money. He's using lots of floating points variables ("double" in his language) to store money.
Joe does not know about the inherent rounding error of decimal-to-binary conversion (and back). He doesn't know about a decimal floating point type. He doesn't know, and didn't look for, a Money class that could also store the currency. He's just coding his way out of his problem. He didn't choose to use a double after pondering on the alternatives, or based on previous experiences. It wasn't a choice at all. He just didn't know any better.

Joe is actually in the middle of a rather long function, doing some database lookup, some calculations, some reporting. He learned, the hard way, that is better to check for errors. He will handle any error in the simplest possible way: by opening an error box in the middle of the function, where the error can be diagnosed.
Joe doesn't know about design principles. He doesn't know that it's usually better to keep business logic decoupled from the user interface. He didn't choose to put some GUI code inside the business logic after thinking about alternatives. He just did it. He didn't know any better.

That function is so long because at some point Joe realized he had to handle several different cases. He did that the usual way: with a switch/case. He never really "got" objects and couldn't even think about using polymorphism there. He uses classes, somehow, but not at that fine granularity. So, again, he didn't choose the switch/case over something else. He just didn't know any better.

Structure vs. Design
Joe is a good guy, really; and his code kinda works, most of the times. But this story is not about Joe. It's about code, structure, design, and decisions: because in the end, Joe will write some serious amount of code. Code has an inner structure: in this case, structure will reveal heavy reliance on primitive types (double), will reveal that business logic is tangled with database access and with user interface concerns, and will also reveal that no extension mechanism is in place. Code never lies.

Now, the modern (dare I say post-agile?) way of thinking is that in the end, the code is the design. That is, if you want to know the real design, you have to look at the code. You'll find a lot of literature (not to mention blog posts), some from well-known authors, promoting or just holding on this idea. At some point, just about everybody gave in and accepted this as a truism. As you might guess, I didn't.

A process or a thing?
Software development is a young discipline, yet extremely dynamic. It's hardly surprising to find confusion and disagreement even on the fundamentals. That includes, of course, having several hundred different definitions of "software design". For some, it's a process. For some, it's the result of that process. We then have endless debates, like "is coding a form of design?", which again are often confusing "design" with "modeling" and wasting lot of ink about nothing.

It doesn't have to be that hard. Let's start with the high-level question: design as a process vs. design as a thing. It's a rather simple choice; we may even benefit from the digital version of some old, dusty etymology dictionary, where design is commonly defined as "mark out, devise, choose, designate, appoint". Design is an act, and therefore a process. Please don't read "process" as "a series of mechanical acts"; just consider a process as something you carry out over a period of time.
More exactly, design is decision process. We choose between alternatives, balancing forces as we go. We make decisions through a reflective conversation with our materials, taking place all the time, from early requirements to bug fixing.

Design, however, is not the code. It is not the diagram. It is not an artifact. If we really want to transform design into "a thing", then it is best defined as the set of choices that brought us to the artifact; assuming, of course, that we actually made any choice at all.

Side note: not every decision is a design decision. Design is concerned with form, that is, with the shape of our material. Including or excluding a feature from a product, for instance, is not a design decision; it is best characterized as a marketing decision. In practice, a product often results from the interplay of marketing and design decisions.

Design as a decision process
Back in 2006 I suggested that we shouldn't use terms like "accidental architecture" because architecture (which is just another form of design) is always intentional: it's a set of choices we make. I suggested using "structure" instead.

Artifacts always have structure. Every artifact we use in software development is made of things (depending on our paradigm) that are somehow related (again, depending on our paradigm). That's the structure. Look at a diagram, and you'll see structure. Look at the code, and you'll see structure. Look at the executable, and if you can make sense of it, you'll see structure.

There is a fundamental difference between structure and design. Structure can be the result of an intentional process (design) or the result of an accidental process (coding your way out of something). Design is always intentional; if it's not intentional, it's not design.

So, if you want to know the real structure (and yes, usually you want to), you have to look at the code. You may also look at diagrams (I would). But diagrams will show an abstract, idealized, and yes, possibly obsolete high-level structure. The "real" structure is the code structure. It would be naive, however, to confuse structure with design.

Consider Joe's code. We can see some structure in that code. That structure, however, is not design. There is no rationale behind that structure, except "I didn't know any better". Joe did not design his code. He just wrote it. He didn't make a single decision. He did the only thing he knew. That's not design. Sorry.

What is design, again?
Once we agree that design is a decision process, we can see different design approaches for what they are: different ways to come to a decision.

The upfront school claims that some decisions can be made before writing code, perhaps by drawing models. Taken to the extreme (which is always naive), it would require that we make all design decisions before writing code.

The emergent design school claims that system-level decisions should emerge from the self-organization of lower-level decision, mostly taken by writing code according to "good practices" like Don't Repeat Yourself. Taken to the extreme (which is always naive), it would require that we make no system-level choices at all, and wait for code to jell into a well-formed structure.

The pattern school claims that we can recognize forces and adopt pre-packaged decisions, embodied into a pattern.

Systematic techniques like my good old SysOOD claim that we can systematically recognize some problems and choose from a set of sound transformations.

The MDA school claims that we can organize decisions along two axes: decisions that we store in models, and decisions that we store in transformation tools. This would be a long story in itself.

Etc.

If you have a favorite design approach (say, TDD or Design by Contract) you may consider spending a little time pondering on which kind of decisions are better supported by that approach.

There is no choice
If there is no choice to be made, or if you can't see that there is a choice to be made, you are not doing design. This is so big that I'll have to say it again. If there is no choice to be made, you're not doing design. You might be drawing some fancy diagram, but it's not design anyway. I can draw a detailed diagram of the I/O virtualization layer for an embedded device in 10 minutes. I've done that many times. I'm not choosing anything, just replicating something. That's not design. It's "just" modeling.

On the other end of the spectrum, I've been writing quite a bit of code lately, based on various Facebook APIs (the graph api, the javascript sdk, fbml, whatever). Like many other APIs, things never really work as documented (or lacking documentation, as it would be reasonable to assume). Here and there, in my code, I had to do things not the way I wanted, but the only way that actually worked. In light of the above, I can honestly say that I did not design those portions. I often couldn't make a choice, although I wish I could (in this sense, it would be terrible to have someone look at that code and think that it represents "my design").

Of course, that's not my first web application ever. Over time, I've written many, and I've created a lot of small reusable components. I know them well, I trust them, I have the source code, and I'm familiar with it. When I see a problem that I can easily solve by reusing one, I tend to do it on the spot.
Now, this is a borderline case, as I'm not really making a choice. The component is just there, crying to be reused. I'm just going with the flow. It's perhaps 5% design (recognizing the component as a good candidate and choosing to use it) and 95% habits.
Reusing standard library classes is probably 1% design, 99% habits. We used to write our own containers and even string classes back in the early 90s. Today, I must have a very compelling case to do so (the 1% design is recognizing I don't have a very compelling case :-).

Trying to generalize a little, we may like to think that we're making decisions all the time, but this is far from true. In many cases, we don't, because:

- Like Joe, we don't know any better.

- Like above, we're working with an unstable, poorly documented, rather crappy third party library. It's basically trial and error, with a little clean-up in the end (so, let's say 2-5% design). The library is making the choices (95-98%).

- We've done this several times in the past, and we're just repeating ourselves again. It's a habit. Perhaps we should question our habit, but we don't. Perhaps we should see that we're writing the same code over and over and design a reusable component, but we don't.

- We (the company, the team, the community) have a standard way to do this.

- The library/framework/language we're using has already made that choice.

- We took a high-level decision before, and the rest follows naturally.

After pondering on this for a while, I came to the conclusion that in the past I've been mixing two aspects of the decision making process. The need to make a decision, and the freedom to make a decision. You can plot this as a quadrant, if you want.

Sometimes, there is just no need to make a choice. This can actually be a good thing. It could be a highly productive state where you just have to write down your logic. There might be short "design moments" where we make low-scale decisions (the fractal nature of software makes the decision process fractal as well), but overall we just go with the flow. As usual, the issue is context. Not making a decision because there is a natural solution is quite different from not making a decision because we can't even see the possibilities, or the need.

Sometimes, we need to make a choice. At that point, we need the freedom to do so. If the code is so brittle that you have to find your way by trial and error, if company standards are overly restrictive, if our language is too limited, if the architecture is too constraining, we lack freedom. That leads to sub-optimal local and global choices. Here is the challenge of architecture: make the right decisions, so that you offer a reasonable structure for growth, yet leave enough freedom for local choices, and a feedback loop from local to global.

Design, as a decision process, is better experienced in bursts. You make a set of choices, and then you let the natural consequences unravel themselves. You may unravel some consequences using models, if you like it. In some cases, it's very effective. Some consequences are best unraveled by writing code. Be smart, not religious.

Be aware when you have to make too many choices, too often. It's fine in the beginning, as the decision space is huge. But as you progress, the need for new choices should naturally decrease, in frequency and in scale. Think of coding as a conversation with your artifacts. Just like any conversation, it can turn into an argument and then into a fight. Your code just does not want to be shaped that way; it was never meant to be shaped that way; it is resisting your change. You change something here, just to find out that you really need to patch that other part, and so on. You have to think, think, think. Lot of effort, little progress. On a deeper level, you're fighting the forcefield. The design is not aligned with the forcefield. There is friction (in the decision space!) and friction energy is wasted energy.

Finally, if you draw a quadrant, you'll see an interesting spot where there is no need to make choices, but there is freedom to. I like that spot: it's where I know how to do it, because I've done it before, but it's not the only way to do it, and I might try something new this time. Some people don't take the opportunity, and always take the safe, known way. Depending on the project, I don't mind taking a risk if I see the potential for a sizeable reward.

What about good design?
Back in 1972, David Parnas wrote one of the most influential papers ever ("On the criteria to be used in decomposing systems into modules"), where two alternative modular structures were proposed for the same system. The first used the familiar functional decomposition. The second was based on the concept of Information Hiding. Later on, that paper has been heavily quoted in the OOP literature as an inspiration for encapsulation. Indeed, Parnas suggests hiding data structure details inside modules, but that was just an example. The real message was that modules should encapsulate decisions: every module [...] is characterized by its knowledge of a design decision which it hides from all others"; again: "we propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others".

Think about it, and it makes lot of sense. Design is a decision-making process. Decisions may turn out to be wrong, or may become obsolete as business and technology change. Some decisions are also meant to be temporary from the very beginning. In all cases, we would like to modularize those decisions, so that we can change them without global impact.

Unfortunately, this is not how most people design software. Most decisions are not modularized at all. Worse yet, most decisions are not made explicit, either in code, diagrams, sketchy notes, whatever. And worst of all, a lot of code is written without taking any decision at all, just like good ol' Joe does. "Seat of your pants" is not a design strategy.

Note: given current programming technology and tools, modularizing decisions is easier said than done. For instance, I decided to use AJAX in the web application I mentioned above. I would love to modularize that decision away. But it would be damn hard. Too hard to be worth it. So I let this decision influence the overall structure. There are many cases like this. Quite often, during a joint design session, I would say something like - guys, here we're making a pervasive, non-linear choice. This is like a magic door. Once we choose, we enter into a different world.

So, we're doing a good design job when we make modular decisions, that we can change later without global impact. I think this may shed a different light on delaying decisions (design decisions, not (e.g.) marketing decision).

The folklore is that if you wait, you can make a more informed decision. Of course, if you wait too much, you reach a state of paralysis where you can't progress anymore, and you may also miss some opportunity along the road.

From the decision modularization perspective, things are slightly more precise:

- If, given our current knowledge, the decision can't be modularized, it's better to wait, because we might find a way to make it modular, or get a better picture of our problem, and hopefully make the "right" nonmodular decision. Note that when a decision is not modular, undoing/changing it will incur a substantial cost: the mass to move in the decision space will be high, and that's our measure of work.

- If the decision can be modularized, but you think that given more knowledge you could modularize it better, it makes sense to wait. Otherwise, there is little or no value in waiting. Note that postponing a decision and postponing implementation are two different things (unless your only way to make a decision is by writing code). There is a strong connection with my concept of invariant decision, but I'll leave it up to you to dig around.

As an aside, I'm under the impression that most people expect to just "learn the right answer" by waiting. And by "right" they mean that there is no need to modularize the decision anymore (because it's right :-). This is quite simplistic, as there is no guarantee whatsoever that a delayed decision will be particularly stable.
So, again, the value of delaying should be in learning new ways to modularize away the decision. Learning more about the risk/opportunity trade-off of not modularizing away the decision is second best.

Conclusions
In a sense, software development is all about decisions. Yet, the concept is completely absent from programming and modeling languages, and not particularly explicit in most design methods as well.
In the Physics of Software I'm trying to give Decisions a central role. Decisions are gateways to different forcefields. Conversely, we shape the forcefield by making decisions. Decisions live in the Decision Space, and software evolution is basically a process of moving our software to another position in the Decision Space. Decisions are also a key ingredient to link software design and software economics. I have many interesting ideas here, mostly based on real option theories, but that's another story.

Next time: tangling.