Wednesday, May 16, 2012

Notes on Software Design, Chapter 16: Learning to see the Forcefield

When we interact with the physical world, we develop an intuitive understanding of some physical forces. It does not take a PhD in physics to guess what is going to happen (at a macroscopic level) when you apply a load at the end of a cantilever:

You can also devise a few changes (like adding a cord or a rod) to distribute forces in a different way, possibly ending up with a different structure (like a truss):

Software is not so straightforward. As I argued before, we completely lack a theory of forces (and materials). Intuitive understanding is limited by the lack of correlation between form and function (see Gabriel). Sure, many programmers can easily perceive some “technological” forces. They perceive the UI, business logic, and persistence to be somehow “kept apart” by different concerns, hence the popularity of layered architectures. Beyond that, however, there is a gaping void which is only partially filled by tradition, transmitted through principles and patterns.

Still, I believe the modern designer should develop the ability to see the force field, that is, understand the real forces pulling things together or far apart, moving responsibilities around, clustering them around new concepts (centers). Part of my work on the Physics of Software is to make those forces more visible and well-defined. Here is an example, inspired by a recurring problem. This post starts easy, but may end up with something unexpected.

I just want to log in
Let's start with a simple problem. We want to authenticate a user through the familiar id / password combination. It may be natural to start with something like this (in a Java/C# like syntax; it doesn’t really matter, of course):

class User
  public bool Authenticate( string uid, string pwd ) { … }
  // more stuff …

Unfortunately, that's not a brilliant OO design. You have to create a User before you can call Authenticate. But then, that user would be invalid until authentication (and after, if it fails). Objects are not supposed to be invalid: the constructor should give you back a valid object. This may trigger another common form:

class User
  public User( string uid, string pwd ) { … }
  // more stuff …

However, now you need to throw an exception when validation fails, which may or may not be nice, according to your view on exceptions.

The problem is simple in hindsight: we’re putting a responsibility in the wrong place. I think it’s easier to see that when we look at a larger picture and realize that authentication through uid / pwd is just one of many. In the average website, you may want to add facebook authentication or twitter authentication. In a banking environment, RSA token authentication. Etc.

Unstable sets
The decision of using a specific authentication technique is unstable: there is an open multiplicity of authentication techniques. Now, in the physics of software, if you map an unstable set into a single software element, that element will be unstable as well. In our case, that means User is going to be unstable. You have to crack it open and change it whenever you need a different authentication technique, with different parameters and different logic.

More formally: authentication techniques form an unstable set, which we could write down as {UidPwd, Facebook, Twitter, RSA, ... }. User is U/U entangled with that set (when you Update the set, User must be Updated as well). Instability of a set is propagated through U/U entanglement; this is just another aspect of the Enumeration Law (which in a more formal treatment could be expanded to better reflect cases like this). Moving to pictures, I tend to visualize this sort of scenario like:

Although it’s easier, once we understand each other, to depict it like this:

That’s not a nice shape. However, to the trained mind, the forcefield is actually suggesting a structural change:what is one must become many.

From Authentication to Credentials
The decision to support multiple authentication schemes is acting as a force on our (software) material. To align the shape of our material with that force (or, as Christopher Alexander said in his Notes on the Synthesis of Form, to put it in frictionless contact with the forcefield), we need to separate the authentication strategy from the user. We also need to avoid the creation of a new, single center with the same kind of U-U entanglement. Fortunately, within the OO paradigm, an unstable, unbounded set suggests the usual adoption of an inheritance hierarchy:

(Yeap, I'm moving to UML; it’s not a taboo :-)

Although some would have called the base class Authentication, I choose Credential, which suggests an object (not an action). This will also avoid the naming antipattern whereby you have class A with an A-ish method inside, like Authentication.Authenticate.

Every concrete Credential class will require a custom constructor, which is fine. After all, they need different parameters. They will also need different logic, which is fine; it’s polymorphism 101.

Switching to the more formal side once again, in the Physics of Software inheritance is a dampening mechanism (one of the many) which can be used to prevent propagation of an unstable decision. I tend to visualize this new shape like:

Or, once we got the idea, like this:

this shape is ok, as it suggests that instability is dealt with by increments, not changes. We now have two U/U entangled sets (conceptual authentication techniques and materialized Credential classes). The shape of our software is aligned with the forcefield.

Opposites, or: your domain is lying to you
It is now time to connect User and Credential. Now, if you think for a moment in the domain language, you have the obvious notion that a User has one or more Credentials, so you may end up with this:

I truly hope you feel some discomfort by looking at that shape :-). It seems like, in order to get a Credential object, we need to go through User. Doesn’t that expose us to the same validity problem we had before? Also, we’re not really expecting user to create the concrete Credential object, are we? Because that would bring us back to the same unpleasant shape above, with one center (the User) entangled with an unbounded set (the concrete Authentication classes).

Fortunately, it's all very simple. At some point (I’ll get back to this in the next paragraph) the flesh-and-bones user will choose an authentication method. The actual workflow will probably be different for different authentication methods, but in the end, you’ll have the right parameters to build a concrete Credential object and invoke the Authenticate method. What is the right service you'd like the Credential to offer? Of course, to either return a (valid) User, or some kind of error. Right: Credential has to know the User, not vice versa. Actually, the user has no need whatsoever to know about the Credential.

This will bring us to a favorable force field, with an unbounded set (the credentials) all entangled with a single center (the user):

Please, spend a minute to consider how simple and well aligned with the problem this is:

- You never have an invalid user or an exception (unless you want to; Authenticate can be exception-based)

- Concrete credential classes can be added at any time, without touching the user

- Actually, the user has no notion of authentication, which is a remarkable separation of concerns. Not the kind you’ve been told all the time (UI / logic / persistence). The hard kind: logic / logic / logic :-). Actually, it's more than separation of concerns. It's why I call obliviousness of concerns.

We just need to make sure we don’t waste all this at the service level: more on this in a moment.

A quick note on the “your domain is lying to you” thing, before the DDD crowd reaches for the gun. In my view, Design is about understanding forces and shaping a solution which is in frictionless contact with the forcefield. Domain modeling helps understanding some of those forces. But domain modeling is a beginning, not an end in itself. Just because the domain is suggesting an arrow, doesn't mean you need that arrow, or that you need it in that direction.

When I teach Object Oriented Analysis (not very frequently: analysis is dead) I recommend avoiding specifying navigability in associations. In analysis, associations are conceptual links, and conceptual links are bidirectional by definition. It used to be that navigability would be chosen during design, with a focus on the nonfunctional properties of our solution. When you compress analysis, design, and perhaps coding together, it's somewhat easier to get an arrow wrong, and then stick with it (not seeing the forest for the trees, I guess). Just be smart. Dig deeper. Use your brain. That's why I call it #braindrivendesign.

There is no Authentication Service, or: avoiding the hourglass forcefield
OK, if you didn’t get an anaphylactic shock till now, it is time to bring this thing to its logical end. Well, almost. I’ll save the real epiphany for the conclusions.

If you think about the UI for a moment, you probably need different pages, or widgets, or whatever, to carry out authentication with different credentials. After all, parameters are different; the workflow is different, etc. This is even more obvious if you also consider biometric credentials and the like. So, let’s say that you have a different widget or page for every concrete credential, possibly with some code reuse between some of them. So far, so good.

Say that you have a service layer. Many modern web apps have a service layer. It might be tempting to define an Authentication service as a class and expose all the different authentications as methods. That’s plain wrong, of course. The ultimate shape would look like this:

That hourglass-like shape is a sort of antipattern again, so very common exactly because we can’t easily see the forcefield. But it should be pretty obvious by now that it would be just replicating the initial problem elsewhere.  It’s trivial, and basically effortless, to overcome this issue. Just turn each authentication scheme into an independent service. That’s what the forcefield is desperately trying to tell you. Use an open grouping construct (like namespaces) if you want to keep those classes together.

Isn’t that just the Open/Closed Principle?
Yeah, sure. And a little Single Responsibility Principle too. And a sprinkle of Dependency Inversion Principle. And yeah well that hierarchy calls for some Liskov Substitution Principle as well.  Hmmm. I’m missing a bit of Interface Segregation, I guess.

Seriously guys, I’ll probably spend an entire post about this sooner or later, but although principles are all nice, they are also keeping software design in the Stone Age. It’s time to outgrow principles and move ahead.

The Consequences of Truth
If we accept the idea that this shape is undesirable:

Then we have no choice but to conclude that, moving from User to Person, this code is highly undesirable as well:

class Person
string firstName;
string lastName;
DateTime dateOfBirth;
string phone;
// …

There is an unbounded set of attributes for a Person (mobile phone; email; living address; working address; place of birth; height; weight; etc), and we have a single class which is U/U entangled with that set. It’s worse than that: even if we do the right thing, group some concepts together and form new classes like Place and Biometrics, Person will still be an unstable center (albeit slightly less so). Note how this instability is also closely correlated with the practical impossibility of building a reusable Person class, which after the initial hype discredited the entire concept of buying (instead of making) your business objects.

Of course, we all know this. Some (most) choose to ignore the issue. Some try to deal with instability using some coping technique: dynamic objects, hashmaps, etc. Been there, done that. It works, somehow, but it ain’t perfect, and won’t solve some issues anyway.

However, once we decide to look at the forcefield, there is an obvious answer in there. It’s a bit unsettling, and goes against everything we’ve being doing so far, which is why I’m sure most of you guys will reject it at once.

Click one of the like/share buttons below if you want to know the entire story (in my next post, that is). Seriously. Stop lurking and click the damn button :-).

If you liked this post, you should follow me on twitter!


Unknown said...

The next post is gonna be...wait for it...legendary :)

By the way, I always felt uncomfortable with libraries that makes you create an object in an invalid state just to call a couple (dozens?) methods later on and then cross your finger hoping you fill it in right! I think it was a sign that I have to listen to my material...

Actually, as a side note, you said "If we accept the idea that this shape is undesirable..." as regards Person object. I can spot why it is undesirable from a maintenance point of view, but (I ask you to not reveal the future, just an hint), but is your solution desirable even for throwaway systems? (Sometimes I built one time shot system that lived for some days, maybe a couple of weeks)

Carlo Pescio said...

The next post is gonna be...wait for it...legendary :)
Or it will turn out to be rather anticlimatic :-))

but is your solution desirable even for throwaway systems?
You already know that the dynamics of throwaway systems are different, and the overall design must be aligned with the real context. That said, in many cases doing the right thing is just about a different structure / distribution of responsibilities.

In the end, I think I'll have to write something like "Life without Mappers, Episode 1" before chapter 17, because the leap from what it's usually taught might be too large otherwise. Besides, those buttons aren't being clicked much :-)

James Waddington said...

Hi Carlo,

I just found your blog and am blown away by the ideas of the physics of software. Can't wait for your next post. Will it be soon?

Could you give a little bit more detail about how a Credential class might be implemented to "know the User" and how it can know whether the credentials are valid.

For example, what would you need to pass into the constructor of a UidPwd object in addition to the userId and password, if say the user's real password(hash) was stored in a database?

I'm guessing you'd want some kind of PasswordAuthority interface that would be implemented by a database-oriented class. And perhaps a UserBase for getting / creating a valid User object to be returned by the authenticate() method of the Credential.

But if you do that, the UidPwd's constructor is starting to become rather verbose. Would you introduce a factory to create a UidPwd object rather than constructing it directly? If so, what would you call that factory? If not, what would you do?

Or am I completely off-track?!


Unknown said...

Maybe I can give some hint to James.

Credentials c = CredentialsFactory.create( userName, Password);
User authUser = c.Authenticate();
catch( AuthenticationFailure failureReason )
The factory could create a Credentials object based on configuration (you want facebook, google+, openID, whatever). Extra parameter could be stored in configuration too.
Credentials is an interface, and the specific mechanism go into Credentials derived classes (FacebookCredentials, OpenIdCredentials, DBHashCredentials ugly name :))

Hope this helps :)

Carlo Pescio said...

Fulvio: unfortunately your CredentialsFactory looks very much like the hourglass neck I told you to avoid :-). In fact, it will not withstand the biometrics test: if you want to authenticate using a retinal scan or fingerprint, the parameters just won't fit. Also, the usual workflow for facebook authentication (for instance) won't reveal you the user's password. The RSA token is an additional parameter. Etc.

It's probably the presence of an inheritance hierarchy that is pushing people toward the hourglass shape. It does not work. Let it go. There is no way to change a configuration file and switch from uid+pwd to facebook to biometrics.

As I said, the page is different, the widget is different, the service is different, the concrete credential is different. The base class just provides [protected] common behavior.

A factory could still be used (I would not) to select different strategies within a single credential scheme (like: it's uid/pwd, but I could store the pwd hash or a symmetric cipher or whatever). I don't think it's worth the effort and, when I'll talk about living without mappers, the overall solution will take care of this for free, anyway.

Carlo Pescio said...

James: thanks :-), this series is rather unpopular :-) so I'm always happy when someone likes it. Unfortunately I'm always a bit swamped so my next post will take a few more weeks.

About your question. At the database level it's all pretty obvious. There would be a table, say UidPwdCredentials, with these fields:

- UserPrimaryKey (as a foreign key)
- UserId (plain text)
- PwdHash (possibly a blob)

So the barebone sequence, using a Repository which spits out business objects (you may not want that) would be:

- get a UidPwd credential object out of the repository. The underlying query is obvious (get all the fields given the UserId). Also, the user-provided password will be passed in.

- call Authenticate. This would hash the pwd and check against the hash.

- On success, we have to retrieve the user object with the given UserPrimaryKey. This can be accomplished by calling the User repository.

Now, as you understood, the real issue is where we want to put the different portions of that logic, that is, how much decoupling between business objects and database access we want, how much logic we want to put into the service layer and how much in the objects themselves, etc.

There are just too many choices here to be discussed in a comment, so I guess you'll have to wait for my next post. Generally speaking, (see also my answer to Fulvio above) we must be very careful not to introduce an hourglass shape at some point. A Factory, Authority, or just anything that has to handle all the variety of Credential types would be wrong (not aligned with the forcefield).

I also value simplicity, so introducing 6-8 classes just out of separation of concerns, when in practice some old-fashioned guy would have written all the logic (service,business,db) in less than 20 lines of code definitely looks like 90s-style overengineering. So having a factory/authority for each credential type (to avoid the hourglass) looks a bit bloated.

In a sense, however, while all this is still important, the central point is to reverse the dependency between user and credential, and avoid the hourglass. The rest, as important as it is, depends a lot on the particular style of database/bo/service partitioning you like. For instance, I'm not a fan of ORMs, but in a simple ActiveRecord-style ORM-based solution you would just have the User and the UidPwd class, and put the 20 lines of code in the latter. With consequences, of course, but overall ok within the ActiveRecord approach, and still with the right dependency and no hourglass...

I'll take a deeper look at the different styles / consequences on my post about living without mappers.

Unknown said...

Wow, now I see. This is enlightening.
In a sense, this is a step beyond management of creation dependency. I really look forward to your next post :)

xpmatteo said...

Hello Carlo,

Very interesting post! I like that it explains why partitioning things in layers is not enough to make code easy to change. I also like the teaser for the next part; the discomfort at the instability of the attributes of business entities is something that has been nagging me for years ;-)

I haven't read the previous posts in this series so please forgive me you've answered already. You're saying that stable things should not depend on unstable things. Is this the same as Uncle Bob's Dependency Inversion? I really dislike the DI name as it implies that our "normal" tendency is to get the direction of the dependency wrong :-) Do you have a better name for this? You have a name for the Hourglass, but you don't seem to have a name for the other two smaller patterns...

Also, you talk of polymorphism as a damping mechanism that improves the situation; do you have a way to visualize the damping

Carlo Pescio said...

You're saying that stable things should not depend on unstable things. Is this the same as Uncle Bob's Dependency Inversion?
Note that technically speaking, I'm "just" saying that instability tends to propagate to entangled entities, unless we find a way to dampen the wave of changes. I'm not turning that in some sort of principle. I'm interested in a theory of forces, and principles are more the "thou shall not" things you usually find in another kind of books. That said, *one way* to prevent the ripple is to invert the dependency. There are other ways, of course (like going at a meta level). So I tend to call "dependency inversion" the technique (where the name is fine) and not the principle. I actually speak more generally of dependency management when I talk about "principles".

You have a name for the Hourglass, but you don't seem to have a name for the other two smaller patterns...
You're right, I don't, probably because I'm thinking about that stuff at the visual level, not at the verbal level. Still, naming things is useful :-). In my next-next post I'll talk more about instability. Perhaps I'll have a name by then :-).

Also, you talk of polymorphism as a damping mechanism that improves the situation; do you have a way to visualize the damping
I've tried quite a few over the years. Something I use quite often (when I actually draw something) is a dashed circle around the unstable things, representing a shield of some sort against the internal instability. It's still unsatisfactory, and I don't think it's the real picture I tend to form in my mind, but it's damn hard to make those things surface to the conscious level, where they can be actually observed.

(you should really read part 0-15 anyway : )))

Tartley said...

Carlo, I've been enjoying your blog so much that I recently went back to the earliest posts, and have read through everything you've ever posted here. Some posts multiple times! :-)

Anyhow, you've remarked a couple of times that these posts are not your most popular, so I just wanted to assure you that I've been educated greatly by them, they have given me lots to think about, and I've often gone away to sketch some code or designs in response to the ideas I found most startling, and I plan to do a series of katas to implement things in a style which I think you would approve of, to absorb the lessons more.

I must confess, I think I came across your 'physics of software' ideas many years ago, but at the time I was young and foolish and I judged that there couldn't be much merit to an "arm-waving" comparison between software and physics, so I didn't read them. I'm now sorry that I made such a mistake, and wish I'd read your ideas years ago.

When I reach the most recent of your blog posts (soon I assume) I'll look up your papers too, see if I can glean anything from them.

So: thanks for everything thus far. Flicking ahead, I don't *think* you've yet found time/inclination to continue your thoughts about the unstable set of attributes in a Person class. Please consider this as an enthusiastic encouragement to do so.

Carlo Pescio said...

thanks a lot, really appreciated. The physics of software seems to suffer from the low popularity - high ranking syndrome: most people don't like it; those who do, however, like it a lot.

My post on instability is half-baked; however, I need to find the "right" day to give it its final shape. It's not something I can squeeze between two busy days, I really need to work on it with a fresh (ideally empty :-) mind.
It has been a busy summer and September looks equally busy, but I'll do my best.

You're also right about finding inclination. It's not something I'm particularly fond of. Not the kind of revelation people would like to read :-). It's just the logical consequence of mirroring the forces. In fact, my tentative title for the post is "Don't do it" :-), because it might be aligned with the force field, but terribly misaligned with existing tools, libraries, books, gurus, etc etc.

Still, a promise is a promise, so it's coming up in a few weeks.

If you decide to blog about your katas, don't forget to tell me! : )

Nice picture : )

Unknown said...
This comment has been removed by a blog administrator.
Edna Begum said...
This comment has been removed by a blog administrator.