Don’t do it!

We used to be taught that, by spending enough time thinking about a problem, we would come up with a "perfect" model, one that embodies many interesting properties (often disguised as principles). One of those properties was stability, that is, most individual abstractions didn't need to change as requirements evolved. Said otherwise, change was local, or even better, change could be dealt with by adding new, small things (like new classes), not by patching old things.

That school didn't last; some would say it failed (as in "objects have failed"). At some point in time, another school prevailed, claiming that thinking too far into the future was bad, that it could lead to the wrong model anyway, and that you'd better come up with something simple that can solve today's problems, keeping the code quality high so that you can easily evolve it later, safely protected by a net of unit tests.

As is common, one school tended to mischaracterize the other (and vice-versa), usually by pushing things to the extreme through some cleverly designed argument, and then claiming generality. It's easy to do so while talking about software, as we lack sound theories and tangible forces.

Consider this picture instead:

Even if you don't know squat about potential energy, local minima and local maxima, is there any doubt the ball is going to fall easily?

Is there any doubt that this ball is in stable equilibrium instead?

The nice thing about mechanics is that we have both a sound theory of forces (so that you can formally prove that some configuration is stable, if needed) and an intuitive grasp of things (so that you don't need big theories when things are reasonable simple).

This is not so in software. When the guru claims something, the "proof" stands mostly in his argumentation ability (rhetoric). Theorems, like the CAP theorem, are few and far between. Lacking a theory of forces, people tend to propose oversimplified models, where (for instance) there is always a stable configuration for every problem (the old school) or there is never a stable configuration for any problem (some interpretations of the modern school).

Stability

Requirements change; actually, our own understanding of the problem changes during and after development. Technology shifts as we're writing our code. All those changes in the requirement space act as forces. Those forces materialize as changes in our artifacts (source code). Unless we have the right shape in place, some of those forces will ripple through our artifacts like waves. The problem, of course, is finding (or even defining) "the right shape", one where stability is maximized. The right shape is totally dependent on the problem (forces), so there is no “right shape” per se.

Unfortunately, we don't know much about forces and shapes. Sure, there is an echo of the "stability" property in some design principles. The "open/closed" principle is trying to keep a class stable by moving unstable parts into polymorphically-derived classes. The "dependency inversion" principle is trying to contain the wave of change by avoiding dependencies on concrete things (from the classic Design Principles and DesignPatterns: "One motivation behind the DIP is to prevent you from depending upon volatile modules. The DIP makes the assumption that anything concrete is volatile" [quite an assumption anyway]). Etc. But we're far from having a sound theory. Concrete things might be stable, and interfaces unstable, for instance.

Still, sometimes we can find small, "stable" abstractions. Sometimes, apparently, we can't. It would be ok if we could at least recognize unstable configurations, and perhaps move toward more stable shapes. This, however, would require an even more ambitious step: to actually classify the most common instabilities. Not unlike classifying mechanical stress into tension, compression, bending, torsion, and shear (see Notes on Software Design, Chapter 13: On Change for a digression on this stuff), this would bring the software design conversation to an entirely new level.

This post is not the right place to do so, although I've spent quite some time tinkering with the concept of stability in my exploration of the Physics of Software. But let's at least try to classify a few common instabilities:

1) Instability of internal structure or observable run-time behavior, behind a uniform interface

For instance, you have many shapes, each with its own optimized implementation, but they can all Draw() themselves.

2) Instability in multiplicity, with uniform processing

You have an unbounded collection of objects, but you treat them all the same way.

3) Instability in structure, with a uniform externalized behavior (usually reflective behavior)

For instance, you have widely different structures, but all you want to do is serialize them

4) Instability in structure, with a non-uniform (usually external) behavior

You have different structures or fields, and you do different things with each one

Etc. There aren’t many more cases anyway; for instance, encapsulation, closures and currying can deal with another kind of instability (guess which :-).

Progress in programming languages brought us some helpful concepts to deal with 1, 2, and 3 (in different paradigms), but nothing to deal with 4.

We may try to coerce 4 into 1, 2, or 3; for instance, by pretending that everything is just a list or a map, so that instability in structure becomes instability in multiplicity. That does not really work well. It may work when you try to convert 3 into 2 (because your language lacks reflection), but 4 is a different beast altogether.

Open sets

In my latest post on the physics of software, I used a visual metaphor to show how some configurations are necessarily unstable. To recap, any single concept that is U/ entangled with an open set of concepts is, by necessity, unstable.

In the end, I came up with the uncomfortable idea that this class is therefore unstable by construction:

class Person

{

  string firstName;

  string lastName;

  DateTime dateOfBirth;

  string phone;

  // …

}

because, quoting myself :-), "there is an unbounded set of attributes for a Person (mobile phone; email; living address; working address; place of birth; height; weight; etc), and we have a single class which is U/U entangled with that set".

What is worse, we're facing type-4 instability, as I will use the dateOfBirth to calculate your age and email to contact you. That's not uniform behavior.

I also said that the force field is suggesting a different shape, not necessarily something you want to use, but a different shape nonetheless. So this post is not really about instability, but about what the force field is telling us to do.

Don't do this at home

What I'm going to show is basically a logic consequence. It doesn't mean you have to do things that way, or that I'm proposing you do things that way. Actually, I totally discourage you from doing things that way. It doesn't play well with the way you've been taught to structure your code. It doesn't play well with your language and tools; with your database; with your UI library; etc. Of course, it might just be the opposite (UI libraries, tools, languages, etc. are not well aligned with the real nature of things). But the net result is the same. Don't do it. That said, here is what you could do if you were insane (yes, I’ve done that a few times :-).

Note: I've removed quite a lot of details from what follows, as this post was turning into an endless exploration of possibilities. That should be ok since you're not supposed to use this post as a starting point for anything practical anyway :-).

Step 1 (simple): Decompose to small Classes with a Role

Large abstractions are usually unstable (I have a nice theory about this, based on entanglement: to a screen near to you sooner or later). So let's do the simple thing first, and re-group fields into smaller abstractions aggregated by role:

class PhoneNumber

{

// some structure and responsibility here

}

class Address

{

// some structure and responsibility here

// possibly using fine-grained classes like

// Country, State, City, etc

}

class PersonalName

{

// some structure and responsibility here

// see http://en.wikipedia.org/wiki/Personal_name

}

class Sex

{

// ...

}

class DateOfBirth

{

// some structure and responsibility here

// (see also below)

}

At this point, we could at least represent our Person as:

class Person

{

  PersonalName who;

  Sex gender;

  DateOfBirth born;

  Address livingAddress;

  Address workingAddress;

  PhoneNumber homePhone;

  PhoneNumber mobilePhone;

  // etc

}

This does not solve the problem, of course. But small abstractions are often more stable, and given the proper overall design, partitioning might be enough to prevent a ripple of changes. For instance (in theory) any change to the structure of a phone number should be isolated into PhoneNumber and should not ripple into Person (if you can make your DB and UI play nice, that is).

This partitioning needs careful attention. For instance, DateOfBirth is now a class; so it’s not just a DateTime. The reason to do so is to have a place where we can move a responsibility formerly allocated to Person (calculate age). You can’t ask a DateTime to calculate Age; it wouldn’t be appropriate for such a general class. It’s a perfect responsibility for a DateOfBirth class, though.

Note that I probably need many more classes than you're willing to create. Address, for instance, is a very specific abstraction, describing a place with extreme detail. Sometimes we want a vaguer concept of place (like "place of birth"). A structural hierarchy of places can be in order here, and we may even find a simple way to deal with increasingly detailed information about a place.

One may also want to group the who, gender and born fields into a separate Demographics sub-center. It's fine to do so, of course.

The shape of step 1

Apparently, Step 1 is not making much of a difference. Yet there is something here anyway. The initial shape was basically a large, big, unstable center (the Person). Instability will occur, by necessity, inside that center (as there is no other place).

Now we have a number of sub-centers (the Address, the PersonalName, etc), and instability can be localized. Oh, yes, PersonalName can easily be unstable; move outside your familiar culture and you'll discover new needs and new fields.

Step 1 is also suggesting a potential fractal hierarchy of concepts (see Place) that I'm not going to investigate further, but is nonetheless important in the pursuit of a truly object-oriented form. In a sense, it's moving from a big circle (Person) to something like this (represented at an intermediate step in a fractal decomposition)

Step 1 is also suggesting that sub-centers, like PhoneNumber and Address, are full-fledged classes, with (at the very minimum) validation behavior and probably more. It's also suggesting that we should have a widget to edit a PhoneNumber, and a widget to edit an Address, and that a UI technology without widgets is a rather stupid idea not fitting well with this. It's also suggesting that the UI should be dynamically composed by reflectively exploring the Person structure, looking for widget-supported concepts.

Finally, Step 1 might suggest that we may want to have a PhoneNumberRepository and so on to deal with persistence in a more modular way, but then, let's wait for Step 2.

Step 2: the reversal

Abandon you hopes. Here is where the fun begins. Just like User shouldn't know about Credential (see that “Chapter 16” post), but Credential should know about User, Person shouldn't know about PhoneNumber. PhoneNumber should know about Person. (Yes, that might require a more abstract concept than a Person, say a base class, because companies have phone numbers too; I’ll ignore that for simplicity).

That's it, nice and easy. The way to avoid instability in structure is to disband the structure. There is no spoon. There is no Person as you tend to think of it. The role of Person, as we'll see, is to provide identity and (optionally) reflective behavior.

So Step 2 brings to this shape (as an object model)

Many PhoneNumbers and Addresses can be connected to a single Person, according to their role. If I come up with an Email class, I don't need to change the Person. The person just provides identity. [Yes, I’m stepping away from my usual “dependencies go up” standard because I want to show the gravitational pattern here].

This should really be the end of this post. That’s what the forcefield is telling us to do. Everything that follows is just one way to make it work, with its own consequences. There might be better ways. Actually, I hope so.

Note that the diagram above is a conceptual object model. In practice, you won’t see those associations in your classes, because they would be dealt with at the repository level. This might be a little confusing, so let's start from the bottom: the database.

The Database

A truly modular system would call for separate tables for each modular concept. Therefore, a table for PhoneNumber, a table for Address, etc. Most of those tables would have:

- the person ID

- the concept Role (like "home number"). In some cases (DateOfBirth) this is not needed.

- all the concept's fields (like country prefix and whatever)

(some of the satellite classes are unstable; guess you can figure what happens to their database counterpart)

There is also a Person table. The table provides only an ID, to act as a foreign key on the other tables. That's the persistence-level idea of providing identity.

Of course, that provides maximum modularity. Need a new role for an existing concept? Nothing to change at the persistence level. Need a new concept (email)? Just add a table.

Still, now it’s also inefficient to build the "entire" person (or a significant subset). Of course, it would be possible to come up with a database (or DBMS as they used to say) where you define a logical structure (tables) and then you specify a physical structure (like: pack all this stuff into a single row) and the database is smart enough to remove the logical joins when mapping to physical access. But our current toolset is not well aligned with this. In a sense, this is basically the opposite of views. 

Worst case: you do a lot of small queries. Alternative: a number of joins and some smart code (see below). In practice, what follows does not require separate tables; it’s just the shape most aligned with the force field. As I’ll talk about persistence code, you’ll see that the decision about database structure won’t percolate much.

The Repository

Coming from a traditional design background, it would make sense to have a Repository for every concept, like a PhoneNumberRepository (or perhaps a PhoneBook class :-). After all, this would preserve modularity as well (and avoid the hourglass shape we saw in chapter 16), again at a risk of poor performance.

Ignoring performance for a minute, single-concept repositories would usually be trivial. So, in practice, we could have classes like this (you may or may not like static methods; it’s a rather irrelevant details in this context; also, I'll assume that you can call a repository and get back a business object. See Life without Stupid Objects, Episode 1 for a way to do that while preserving strict layering.)

class AddressRepository

{

  public static Address GetPersonAddress( Id personIdentity, Role r );

  public static IEnumerable

GetPersonAddresses( Id personIdentity );

// …

}

The Address object will not need to contain a Person object; it’s enough to have its Id inside.

However, this choice will force us to do a select for each and every fine-grained concept. That’s usually unacceptable, so let’s play with this. I’ll assume a SQL database, because it’s what I know best and it’s the environment where I’ve actually implemented most of this weird stuff.

The Statement (part 1)

A Repository is usually charged with quite a few responsibilities. Say that I introduce a new class instead, that I can call Statement. The Statement, among other things, would offer the ability to:

- specify a center / start table, providing identity

- specify that we need some fields, from some other tables (joined with the center table)

- add conditions

- transform all that into SQL

- execute the SQL

- help Repositories turn that stuff into objects (I'll get back to this in a minute)

Using a statement, I could to things like this (at a rather low level):

Statement s = new Statement( “person”, “id” ) ; // the center table and identity field

s.AddFields( “role”, “street” ).FromTable( “address” ).RelatedBy( “id” ) ;

s.AddFields( “firstname” ).FromTable( “personalname” ).RelatedBy( “id” ) ;

Which of course would become a 3-table join with 4 fields selected. The real syntax needs to get a bit more complicated as the concepts get fractal, but let’s keep it simple.

I could also add conditions, like:

s.AddFields( “role”, “street” ).FromTable( “address” ).RelatedBy( “id” ).Where( equal, “role”, “home” ) ;

At this stage, the Statement is just a way to build SQL statements compositionally, and then execute them. The trick is, of course, that the composition can now be spread among Repositories, each dealing only with its own table.

Note: In .NET land, this stuff can leverage LINQ expression trees, and improve on my syntax as well using lambdas here and there. In other languages, it might be slightly more challenging, but still doable.

The new Repository

The role of a repository is now to:

- contribute in creating a statement

- process the result of a statement execution and build objects

So, the repository does not own the statement, does not execute the statement, does not entirely control the statement. It participates in building a statement, and in converting results back to objects.

Traditionally, a method like GetPersonAddresses does everything at the same time, so let’s split this thing in two:

class AddressRepository

{

  // mutable object syntax

  public static void PrepareGetAddresses( Statement s ); 

  private static List

ProcessGetAddress( Record r );

  // …

}

Of course, I can still provide the former GetPersonAddress method as a shortcut when composition is not needed. The nice part, however, is that now I can:

- create a statement for a center table

- call any number of satellite repositories to have their own tables, fields and conditions merged in, by calling the Prepare... methods

- execute the statement

Executing the statement, at first, will just get a bunch of records from the DB, but I want objects. Actually, I want each repository to create its own sub-object for every record that has been retrieved. That's quite simple: during prepare, the repository will instruct the Statement with a callback function / delegate / whatever (that's why ProcessGetAddress can be made private). The statement, after execution, will iterate over the retrieved records, and call all the callback functions in different repositories for each record. Taken together, what we get back is the object-land equivalent of a record. The problem, now, is where do we store that stuff.

(Oh, Record is just a convenient class to hide the technology-specific notion of a record).

The Statement (part 2)

Ideally, I'd like to prepare my statement (by calling various repositories; an example will follow shortly), then execute my statement, and get back a sequence of objects.

If you're in the dynamic typing side of the world, that's very simple. The statement can simply invoke the repository callback functions for each and every record, take the contribution from every repository, build a dynamic object, add it to a list, and that will be it.

If you're into static typing, that's a bit of a challenge. Consider a statement that retrieves a significant subset of fields formerly belonging to Person, like DateOfBirth, Address, Email, etc. The statement is built from the contribution of several different repositories, then executed, then the result is processed by different repositories, so for every record we get back a DateOfBirth, an Address, an Email, etc. As much as you might be tempted to bring this stuff together into a Person class, that’s exactly what we’re trying not to do, so don’t :-).

Note how this is suggesting that when we deal with unstable structures, at some point we may benefit from a dynamic object. Note the context. I’m not saying that every object has to be dynamic, or that we always need them. Just that in this case, in this specific role as a result of statement execution for unstable structures, a dynamic object is very useful.

An idea I’ve had no chance to apply so far in real projects is to adopt Tuples (like C# type-safe tuples, that is) as type-safe transport objects. Would be nice if it worked, although C# generics are quite limited; for instance, something like variadic templates in C++ would be useful to deal with fractal decomposition.

So, at this stage, I will assume that the Statement will return a dynamic object, and that object will be made of typed business objects like DateOfBirth. If your language lacks dynamic objects, an HashMap of some sort would be ok too.

We’re still missing a few things:

1) Where do I put the “coordination” logic, that is, the creation of the statement, the Prepare calls to repositories, the statement execution, the actual usage of the results.
2) How do I deal with cross-concept (business) logic.
3) What about the [unstable] UI?

Believe it or not, it’s easier to start with (2) because it will pave the way to (1) as well. I'll postpone the UI till the end, because (2) will show that it's unstable only in some use cases.

Cross-concept logic
Ok, so we can extract a Person, or any projection of a Person (say, only phone and age), as a dynamic bag of type-safe objects. What about cross-concept logic, that is, some processing involving more than one satellite class?

When I talk about this, people tend to come up with unrealistic examples because, in practice, cross-concept logic is not so common. If it were, those concepts wouldn't be real sub-centers. I don't need your phone number to know whether or not it's your birthday. Still, there are realistic examples, like:

Data extraction: I want to find all males over 35 living in a given town, say for spamming marketing purposes.

Default values: if I know where you live and I'm asking for your phone number, I may want to precompile the area prefix.

Business logic: even in this trivial Person domain, we can find something meaningful. For instance, when the user logs in, we want to check if it’s his birthday, and if so, we want to say "happy birthday, ". It borders on presentation and extraction logic, but it's not, and it’s cross-concept.

None of those is particularly challenging. Actually, the new shape helps revealing the nature of what we're doing much better than a monolithic Person class.

Data Extraction

The basic idea for cross-concept extraction is simple: we want to resolve a set of properties to a set of identities, then extract the individual concepts that we’re interested in, which are connected to those identities.

Well, that's what the Statement + Repositories can easily do. The only question left is where do we put the Statement composition, execution, and processing of the results. Depending on your view of layers, that question may already contain its own answer: if you only allow Application-level classes to access repositories, that logic should be in an Application (or Use-Case) layer class. A code sketch for this logic could be:

Statement spam = PersonRepository.PrepareIdentity( id );

PersonalNameRepository.PrepareGet( spam ) ;

EmailRepository.PrepareGet( spam ) ;

DateOfBirthRepository.SetCondition( above, 35, spam ) ;

AddressRepository.SetCondition( "city", "some city", spam ) ;

IEnumerable< dynamic > victims = spam.Exec();

// iterate over victims and do things with email and personal name

What does something like

PersonRepository.PrepareIdentity( id );

actually do? Quite simple:

return new Statement( “person”, “id” ).Where( equal, “id”, id ) ;

I guess you can figure out the rest. We're basically building a Statement from the cooperation of different repositories. PrepareGet is not the nicest name ever for a method, but it's very easy to understand (I hope), and that's my main concern at this time (as I'm leaving out lot of details). AddTo or IncludeIn could be nicer versions. Also, a more fluent approach like:

DateOfBirthRepository.SetCondition( above, 35 ).In( spam ) ;

would improve readability quite a bit.

I understand you may not like that code at first. It "looks" like the kind of code you want to hide behind repositories. But that's mostly about habits, not substance. The persistence stuff is indeed inside repositories (+ Statement). This is a type-4 unstable selection of things, and must be in the upper layers, not in the bottom layers.

Note: in this case, the set of fields was basically hard-coded. In other cases, it's dynamically determined by various conditions. The compositional nature of the Statement tends to play rather well with that.

Default Values

The default value thing can happen at different levels. In the specific case given above, it's obviously a user interaction concern, as there is no business rule about users having phones within the same prefix area where they live. It's just a convenience when there is a human on the other side of the screen, typing data.

So let’s decompose this further, in two distinct responsibilities:

- Find the most likely prefix given an Address

exercise: where do we put that logic? Is that about the entire Address or just a smaller concept like Area?

- Propose that as a default value.

We need a way to trigger a call to the logic above and use that as a default value; as this is a user interaction concern (for instance, it would make no sense to do that when we import people from a file), the right place for the trigger is the UI itself, or if you’re stuck in MVC, the controller. Triggering from the UI would make for a much more responsive application (think web).

AOP-like interception / injection would help here. Basically, we don’t want the PhoneNumber widget and the Address widget to know each other. We need a place that can see both and inject an aspect there (a user interaction aspect). Don’t have aspects in your UI layer? Told you :-), your UI technology isn't good for this stuff.

Business Logic

In the usual BigPerson approach, it's obvious that we can put this logic inside the Person, because guess what, all the data you need is there, just add methods :-). Alternatively, some would put this inside a Controller.

With this design, there is no natural place for cross-concept logic. This is a good thing. That void is telling you something (see also the quote at the end).

Without making a big fuss out of it:

- at the UI level, we should be able to simply put a Welcome widget on the home page.

- the Welcome widget should be populated using a Welcome API / service.

- the Welcome class, given the person identity, should gather the necessary objects and create the right message. Something like:

Statement welcome = PersonRepository.PrepareIdentity( id );

PersonalNameRepository.PrepareGet( welcome ) ;

DateOfBirthRepository.PrepareGet( welcome ) ;

dynamic res = welcome.ExecSingle();

PersonalName pn = res.PersonalName;

DateOfBirth dob = res.DateOfBirth;

if( dob.IsBirthday() ) // a stable responsibility :-)

{

// say happy birthday + name

}

else

{

// say hello + name

}

(ouch, an if; call the anti-if police :-).

Yes, the static + dynamic style sucks a bit. If you got time to spare and wanna try the Tuple thing, maybe you can make it largely type-safe in many practical cases.

But it's not object oriented!

No, what you mean is that it is not coarse-grained-domain-oriented, at least not the way you want to think about your domain, that is, based on big, fat, unstable classes with a lot of logic, like Person. Nobody said it should be like that (well, somebody did; it's just not universally right).

I actually have lot of objects flowing around. When you decompose to small enough pieces, you'll find stable abstractions, like DateOfBirth. They have clear responsibilities, like Age or IsBirthday. Actually, I am allowing, or even forcing, small abstractions like Area to emerge, instead of disappearing into the faceless blob we usually call Person. I also have a Welcome object. A Welcome widget too. I have many more (cooperating) objects than the BigPerson style tends to create.

What about the UI?

Well, the UI for some use cases is pretty stable (see the Welcome widget). In other cases, of course, it's not. The paramount case is when you want to edit or show "the Person" itself.

What we need to do, of course :-), is to dynamically discover the Person satellites, then dynamically build a UI using satellite-specific widgets (so, a PersonalName widget, a DateOfBirth widget, etc). That discovery should also tell us about role-based satellites like Address, so that we can provide a way to add / display multiple widgets of that kind (for every role)

Note that we cannot use regular reflection for this, as there is no Person class with all that stuff inside. Sure, the right class to ask would be Person. But that requires that Person implements its own reflective behavior, and that satellite concepts register with Person (so that Person can expose satellites, without knowing satellites; that's how we deal with instability here). This might easily move to a base class.

Things get more complicated when satellites have satellites, especially when you want to fine-tune appearance. In some cases, a trade-off between aesthetics and stability is necessary, and a custom widget might have to be created instead of relying on reflective composition of lower-level widgets. As usual, proper UI technologies might help or hinder. A technology lacking any kind of sophisticated layout management, for instance, is less than optimal (yet we're stuck with HTML and CSS :-).

The Value of Emptiness (read this part at your own risk :-)

When I first thought about this stuff, a very old story came back to my mind. There are many translations (see here for a few; pick the one you like most), but here is one I like:

Thirty spokes are joined together in a wheel,

but it is the center hole

that allows the wheel to function.

Person is the hole. By providing identity and nothing more, Person is leaving an empty space where things can't coalesce into a gravitational center. Yet, that hole acts as a virtual gravitational center for a constellation of finer-grained, more stable concepts. It is by being empty that Person allows the system to grow organically, instead of monolithically.

Don’t do it

A promise is a promise, so I had to write this stuff, but I’m glad it's over, so I can move on to some more interesting post; way more interesting, actually.

Please, don’t write me in anger. It’s ok not to like this. It’s ok to disagree. It’s probably not ok to pretend that your Person class is stable while is not, but it’s ok to look the other way just like everyone else is doing, keep patching it as requirements change, and claim you're being agile.

If, on the other hand, that blurb above seems tempting, be smart: don’t do it anyway.

There is no invitation to follow me on twitter, because you can’t possibly like this post. Actually, I’ve downvoted it myself :-).