A week ago or so, Ralf Westphal published yet another critique of my post on living without a controller. He also proposed a different design method and therefore a different design. We also exchanged a couple of emails.
Now, I'm not really interested in "defending" my solution, because the spirit of the post was not to show the "perfect solution", but simply how objects could solve a realistic "control" problem without needing a centralized controller.
However, on one side Ralf is misrepresenting my work to the point where I have to say something, and on the other, it's an interesting chance to talk a bit more about software design.
So, if you haven't read my post on the controller, I would suggest you take some time and do so. There is also an episode 2, because that post has been criticized before, but you may want to postpone reading that and spend some time reading Ralf's post instead.
In the end, what I consider most interesting about Ralf's approach is the adoption of a rule-based approach, although he's omitting a lot of necessary details. So after as little fight as possible :-), I'll turn this into a chance to discuss rules and their role in OOD, because guess what, I'm using rules too when I see a good fit.
I'll switch to a more conversational structure, so in what follows "you" stands for "Ralf", and when I quote him, it's in green.
My dear Ralf...
Overall, I've found your emails to be kind and reasoned, but your post to be filled with the usual rhetoric, whereby in order to look good, you have to make the other guy look bad. In the end, I didn't like it much when you said I'm cheating or that I'm proposing a spaghetti design. Especially when you email me after the fact and ask not to turn this into a zero-sum game. Usually you reap what you sow. But let's say I don't mind too much, and move forward.
Your post can be seen as organized around:
1) A critique of my design approach and of my presentation style (like using class diagrams instead of, say, activity diagrams and use cases)
2) A proposal for a design method, basically the well-known functional decomposition approach, with an interesting but underdeveloped twist toward a rule-based system.
3) A final design, represented by a "flow diagram" and a sort-of class diagram.
Interestingly, you didn't pay any attention to the most important thing, that is, the properties of the results. Because the result, Ralf, is not the class diagram. It's not even the flow diagram, of course. It's an allocation of responsibilities to different "things", those "things" being classes, methods, whatever, resulting in properties, like extendibility, reusability, and stuff like that.
It's very much like having these two nice houses:
and instead of discussing the properties of the houses (building and maintenance cost; serviceability; maximum load; heat transfer; etc) spending all your time in a rhetoric exercise on whether we should start with structural engineering calculations or by drawing the floor plan. Up to the point where you confuse the floor plan with the house. The product of my design process is not a class diagram, just like the product of an architect is not a floor plan. Is the set of properties embodied in that diagram, which will be reflected into the real thing (the code, the house).
You spend a lot of words babbling about process, but you never talk about the properties of what I get and the properties of what you get. All you say is that you don't have all my messy connections (in your diagram), which as we're going to see is incorrect at best. That makes everything rather moot, and with reference to what you said via email, it's hard to move the field forward by process rhetoric.
Anyway, I'll try to cover your critique as quickly as possible, I'll try not to criticize your design process much / at all, and I'll try instead to look at what you get in the end. Because I'm not really interested in discussing process, until we compare results. Results come first. Once someone can show consistently good (or bad) results, it's interesting to look at his process, to see if we can somehow extract some of those good or bad qualities. Babbling about process without looking at results is pointless. And I'll say that again: results are not diagrams; results are the properties of the things we build.
As I said, I didn't like this part much, because you grossly (and seemingly intentionally) misrepresented my work, while at the same time profiting from it. You make it look to the occasional reader as if I jumped straight from a pictorial description to a "complex" class diagram, without explanation, so that you cannot even understand how / if that thing works. But you perfectly know that's not what I did. I took the problem, piece by piece, discussed how I would allocate every portion of behavior to a class, presented small diagrams, and then put them together. I discussed likely changes and how the design was ready to deal with them.
For instance, you criticize me for beginning with a (simple) class diagram instead of bringing in the user. Well, your first diagram adds nothing to the pictorial description of the problem I have included, except the operator. Unfortunately, you choose to focus on the wrong user. The mine pump is not there to satisfy a data-hungry operator (a controller? :-)). It's obviously there to guarantee the safety of mine workers. Perhaps I should have gone the BDD way, adding stories like:
- As a miner, I want to know when there is too much methane, so I don't blow myself up.
- As a miner, I want to know when the pump is stuck, so I won't find myself drowning in mud.
- As a miner, I don't want the pump to turn on when the methane concentration is high, because I'd rather come back home alive.
I didn't, because it was already a long post and I thought it was kinda obvious. Maybe it was not, but anyway, the operator is not a central actor here (although, of course, in a real system there will be some operator console somewhere). Guess what, your design is fine anyway, because adding that [wrong] user into the diagram provided no additional value, except of course a healthy dose of process rhetoric. It's fine if you feel compelled to draw diagrams that add no value; I don't, sorry.
You're also assuming / implying / suggesting that I'm drawing class diagrams because I'm focusing on data. Actually you say that the role of a class diagram is to show the structure of data, and a comment on your blog even talks about "obsession with state" on the side of OO developers. This is very funny because those class diagrams show no data, exactly because I was focusing on allocating behavior.
I have relationships, but those relationships are not about data, but about collaboration. Perhaps you're looking at classes and class diagram from the wrong perspective.
You're also assuming that I never consider process in my reasoning, and that I jump into classes. Of course I consider process, but instead of modeling everything top-down a-la functional decomposition, I'm looking for candidate places for that behavior, as it emerge.
You already have one in mind from the very beginning - the "rules" collection, so you don't need to allocate behavior. You just have to fit every problem into a pre-cut design. Unfortunately, a plain sequence of rules won't work it in this case, and won't work it in any real-world case either. You need something more complex, like a forward-chaining rule engine, that you neglected to show, but I'll get to that in a short while.
It seems to me that, being so overly focused on a universal process and perhaps a universal blueprint, you think that I too am proposing some kind of universal process. I don't. I called that post "case 1" because it's just one of many. It's a realistic example though, and that's the reasoning I used in that case, because I saw a good fit with the problem. I have no problems using an activity diagram when it's called for. Or a state diagram when I'm dealing with a complex state machine. I think it's wrong to propose a fixed approach, irrespective of the problem. Of course, it's your right to differ and to propose your own universal process. You won't be the first to try; you won't be the last to fail.
I will gloss over the part where you say I cheated by choosing an automation problem. You may think that "it´s notoriously hard to find such objects in the requirements" outside real-world objects. Still, along the years, I've been working basically in every possible application domain, and I've a consistent record of finding good classes. There are many other examples in this blog as well. Finding good classes is a skill. A skill you decided not to learn. It's ok. It's not so ok when you call those who can do it cheaters, but I understand it's part of the usual old-school tactics.
Just a final comment on understanding the dynamics through a static view. You said: I deem it no virtue to be able to “reverse engineer” a class diagram to get an idea how things are working. Of course, you're entitled to your values. However, there is a long and consistent tradition in engineering to look at a static picture and infer the dynamics.
We expect mechanical engineers to look at fig. 2 and 3 here and understand the kinetic part.
We expect electronic engineers to look at this picture and understand it's a multivibrator.
In facts, I used to repair TVs as a kid. In my lucky days, I had a schematic for that TV set. Schematics are popular, because they're a rather efficient way to transfer information to skilled people. Of course, you have any right not to learn that skill. For instance, I'm not very good at reading musical notation. However, I would never step up and say that I do not consider that valuable for a musician or a composer.
So, yes, I would also expect a software engineer to look at this:
and understand that it's very likely to be a callback scenario. There would be more to say about stereotypical interactions and the use of colors, but this is not the right time, I guess.
Note how, in my second post about the controller, I proposed this engineering representation of the mine pump:
What you did and what you got
Once you cut the crap and get down to business (what you call the API level) you seem to drop all the big words on design being independent on classes etc and bring in the "dirty stuff" as you call it. I'll never understand why some people think / speak this way, but ok. By the way: part of reasons I'm trying to move software design toward properties, and more broadly toward a physics of software, is exactly to avoid this kind of ivory tower thinking / talking, which benefits no one. We (the software design community) should try to collectively raise the narrative bar and leave that rhetoric behind.
Anyway, at that point you're beginning to create what I've defined a "fake OO step 2" system, that is, you create classes for the simplest abstractions (pump and sensor) and you collapse all the other logic into a centralized place. Nothing new so far (we have to wait for rules to get something new).
There, however, you benefit from my speculation on likely changes, while at the same time trying to make my design look bad. The original problem statement was based on digital water sensors. A digital sensor will only say "it's full, up to my level, dunno above". The original specification of the hysteresis cycle was therefore based on having two digital sensors. That's why I started with a SumpProbe connected to 2 LevelSensor, and then discussed how moving to a continuous LevelProbe would require changes. Changes that I've hidden behind the SumpProbe class.
Now, in your design you have no visible place for a digital sensor, so I can only assume one of these two options (you didn't say; so much for your stuff being much more precise):
a) you have a composite sensor somewhere, exposing two digital as a sort of simulated analog.
b) you just return an integer instead of a boolean, and you deal with two sensors becoming one elsewhere (the WaterLevelSensor? maybe, but you don't even mention the issue).
If you go with (a), why did you move the logic for a composite digital sensor below the API level? The problem statement was all about digital sensors, and according to your "process", you're not trying to build new abstractions as you go. You get the barebone things at the API level and put the logic in that process thing. So doing (a) is sort of cheating, right?. Well, also (b) is sort of cheating, because you are moving part of the logic below the API level anyway, cloning parts of my design.
There is more. My SumpProbe deals with hysteresis, which is not a property of a physical sensor; it's a process thing, to avoid turning the pump on/off continuously. It's also a stateful thing, because hysteresis needs status. Where do you handle that? Again, you don't even mention this "detail", so we're left to wonder.
Say that, being a process thing, you do what you do for other process things and move it above the API. Then it' just another (neglected) oval in your process. Cool. How and where is the attached state preserved? You don't say. In the Rules class? What happens if the mine gets bigger and I need to add pumps, sensors, and glue them together? I can just instantiate more objects. What do you propose?
Say that it is inside the WaterLevelSensor. But then you're cheating again Ralf, moving part of the process below the API at your convenience, without any clear method, mimicking my design when it suits you.
In the end, you waste a lot of time talking about process, but neglect to say how things really work in your design. You omit details and claim simplicity, and you do this over and over.
You also spend a lot of words on why the hell did I use different classes for different gas sensors. As you're trivializing everything into a single class, you blissfully ignore, for instance, that sensors may have to linearize the input, compensate for drifts or ageing, detect specific failures, etc etc. Of course, in the real world we would have base classes and reusable algorithms for that, but no, it's not like we can simply put everything inside a GasSensor returning an integer and say it's done.
The worst part, however, is that all this critique was unnecessary to make your point; and honestly, it seems that you're trying to criticize something you don't fully understand.
Then, at some point, you propose storing all the sampled data into a large SensorData thing. It's not clear what you have in mind, as you choose not to represent that thing (sure, omitting "details" makes things look simpler, right? :-). Perhaps it is a dynamic bag. Perhaps it is a strongly typed structure. In this case, it's rather crappy, as it will break whenever you add more sensors (bye bye modularity; bye bye independent reuse). Also, who is responsible for storing each data in its slot? The individual sensor, or some kind of centralized samplER? I guess a samplER, as your sensor is just returning an int.
Anyway, that will get you exactly into the hourglass shape I was speaking of in my recent post on the forcefield. Of course, you don't seem to mind, so please, go ahead and ignore the underlying forces at your convenience. Actually, this piece alone, if you cared to explain it better, would make for a nice comparison of encapsulation vs. the typical "store all the data here where everyone can read and write everything" relics of structured programming. But you didn't provide details, so there isn't much to say.
Finally, we come to an under-specified class diagram where you say: "The classes are so little connected, I hardly need a class diagram at all."
This is so ridiculous it's almost insulting. It's not like you can just remove classes and dependencies and claim that it's simpler. It has to work. So in your case:
- The clock has to call the sampling, so it has to be directly or indirectly connected to a sampler (if any) and that in turn connected to sensors, but we don't see that. Sensors hang out nowhere.
- The sensors or sampler have to populate the SensorData thing, but we don't see that class at all and so we don't see any dependency.
- The SensorData thing has to go to your Rules class, but we don't see that.
- Somehow the PumpState has to move to the pump, but we don't see that.
- etc etc.
Connections in my diagram are there because data (or objects) do not flow from one place to another on the back of unicorns. Except in your diagrams, of course :-). Oh, maybe it's all in global variables. Or maybe there is a sort of workflow context that again you neglected to show to claim simplicity. Is that context type safe? A dynamic bag? Again, you never say that. Absolutely precise, sure :-).
Of course, you may (will) claim that there is some magic infrastructure taking care of all that (and more, see later about the rules). Guess what, I could have done the same. In episode 2, I discussed a wiring infrastructure. I also mentioned how one could simply use an IoC in some cases (also in the first post). Don't you see I could have just "assumed" that kind of infrastructure, just like you do? I could have drawn a diagram with basically no connections, assuming things would get wired at run-time through configuration. I choose not to. I choose to show how things have to be wired anyway. I choose to show something that didn't need any infrastructure.
It's so funny that you called me cheater, isn't it? Except that my spaghetti class diagram (spaghetti obviously meaning, for you, that I have very controlled dependencies toward more abstract classes to keep everything modular and reusable) is the diagram of a working system, while your diagrams is a depiction of something that does not work because is neglecting essential components and relationships, plus the entire damn infrastructure you never cared to mention. C'mon.
However, you do offer an alternative to the traditional, hard-coded controller, and that's the Rules class. If you didn't bury that in a pile of
So, in the end, the only relevant difference from the usual "simple classes below, monolithic controller above" (fake OO step 2) is that you decompose your controller in rules.
Of course, you're assuming some infrastructure here, because as presented, the Rules class is useless. Also, in your class diagram, there is no trace of the join between Assess_water_level and Assess_methane (there is also no output from that, but hey, you want to pretend you have no dependencies).
Obviously, you can't just have a list of rules. You need a graph and a forward-chaining engine or something like that (been there, done that). You need a machinery to make the join, and if you support independent threads, it will also have to wait until all the inputs are present before calculating the output or the guy in the mine will die.
And of course you're just assuming all that. I had to read some comments on your blog, follow a link to a "cheat sheet" of yours and click around for a while to discover that you're proposing your own homegrown rule engine. Phew. That took a while. Sorry, perhaps that thing is all rage in your neighbor, but I didn't know about it. It would have been nice of you to introduce that thing from the very beginning, perhaps removing some of the unnecessary process
After spending enough time understanding what you didn't care to explain, your design started to make a little more sense. You created a rule engine. So once you write classes for the actions, you "just" have to wire those things through configuration. My "spaghetti" diagram assumed no library, no support, no nothing. Your pseudo-diagram assumed a rule engine, a wiring library, and what else (and didn't even bother to show a placeholder for some of that). Then you claim simplicity, and call me cheater. How cool is that Ralf :-). Oh, and there is still the issue with the missing SensorData thing and so on, but I'm gonna cut you some slack.
The paradox here is that if you approached things another way, we would have found a lot of common ground. Because you know, I use rules and rule engines as well. For instance, I've just ported my GUI Rules library to Android. So now I can say things like (in a fluent-Java style):
It would be unfair to claim simplicity behind that thing, however. There is quite a lot of machinery. You don't see it when you combine rules. But you can't just pretend it's not there and claim that things are magically simplified by process. They are simplified by a library.
Now that I have brought this thing to what I hope is a more neutral territory, I'd like to explain how my view of rules differs from yours, and why.
Two-levels vs Fractal
I guess you're familiar with this kind of house:
It's bricks and concrete below, wood above. Of course, you have to stack them in the proper order :-). So there is a scalability issue here, because you can't just add another brick layer on top of a wood layer.
So Ralf, let me ask you something. As you know, if you read chapter 2 in the controller saga :-), I choose the mine pump because it's a well-known problem, people have already proposed meaningful changes, and I don't have to cheat, I don't have to make things up, I don't have to propose changes that I'm safely well-prepared to handle. There is literature on that.
A common, meaningful change that I have already discussed is the need to have two or more pumps, and rotate among them over time to reduce wearing. Where would you introduce this change in your design? You basically have two choices (if you want to preserve the overall design):
- you can introduce the change below the "api level" (bricks)
- you can introduce the change above, that is, in processes and rules (wood)
This is a "business rule", so to speak, so it would seem more appropriate for you to introduce the change using rules. You can add yet another oval after "switch pump", choosing which pump.
Then the next meaningful change is that if one of the pumps gets stuck, as observed through current sensing, you have to switch to another pump anyway (even if it's not its time yet).
If you deal with that at the process / rule level, you need an "assess current" oval, then you need to change the switch logic above, drawing another join to consider the current too, and switch to another pump when needed. You may think this is ok, because hey, it's the process.
And here is where you should have your epiphany, Ralf. When you say "Well, yes, that´s a kind of functional decomposition. And I don´t see what´s wrong with that" you're just ignoring 40 years of mankind experience with functional decomposition.
I can tell you what's wrong with that: as you scale it up, you're not building reusable abstractions along the way. You're just making the upper part more and more complex. And after you beef it up, you can't scale it down (see Parnas), without creating a clone of your large process and pruning things here and there (remember to remove unnecessary slots in your SensorData too :-).
You have only one, big gravitational center (your process), and that's it. Your rules are probably reusable (except you modeled them as method, not classes, so it's still copy&paste reuse), but the "hard knowledge" is in the wiring of the diagram, not in the rules (which, I hope you'll admit, tend / want to be very simple).
You see, just like you, I wasn't born with objects in my hands. I wrote my first code back in 1978. I know what is like to create large systems using structured programming and functional decomposition.
The real beauty of objects, once you really understand objects :-), is exactly in that thing you don't seem to appreciate: objects as virtual machines, things made of things made of things made of things, all of the same power.
When you stack simple objects under a ton of increasingly complex procedures (whether you bundle those procedures into a controller or in a rule engine) you'll never get the reusable abstractions I'm building along the way: the pump rack, the fail-safe pump rack, the sump probe, the safe engine, etc. Abstractions that can be reused in a completely different problem, where (for instance) methane is not an issue. Too bad in your process the methane comes first.
Your flow diagram is not reusable, and is bound to get bigger and bigger as you scale up requirements. You'll have to go the usual way, and adopt nested diagrams, whereby an oval on one level is an entire diagram on the next. Then you're going to discover you're passing a ton of data around (your SensorData on steroids), but wait, most of those data are only used in some places. And you're going to come closer to some form of encapsulation to solve that form of coupling (there is a long story on how the notion of coupling changed when moving from structured to OO programming, and there is something to be learnt from that. I'll talk about it in a future post). Keep going ahead, and sooner or later, you'll rediscover modules and then objects, because objects are a reasonable (although not definitive) response to some of the forces we routinely encounter in software development.
Oh, by the way, you could also add those responsibilities below the API line instead. Then I really would ask you, why only that? Why don't you go further and push down more and more responsibilities? What is, in your universal process, the guiding force in placing something below or above the API level? Why do we need that distinction at all? Can you see how your "standard blueprint" is trying to squeeze every problem into the same shape?
So why / how do I use rules ?
I just said that I use rules too, so this may seem like a contradiction. But it is not.
The thing is: you make process and rules the center of your system (and development process), and stack that on top of trivial objects, so you get a two-story shape that does not easily scale. I use objects as my primary decomposition technique. Inside some of those objects, I use rules. But it's "just" a convenient implementation detail. Let me elaborate with an example (one of many :-).
I tend to design my UI as a thing made of things, so I usually have a form made of widgets, made of widgets, made of widgets. There are recurring problems in UI implementation though, like disabling some controls when there is no input or when a checkbox is not ticked, etc. The popular thing is to use standard widgets + a large, form-wide controller or view model + binding (assuming you have a binding-friendly UI library, which is not the case in Android, for instance). This is a two-level approach, with relatively stupid, standard widgets below and a more complex monolithic controller on top.
I don't like monoliths, so I look for meaningful aggregates and bundle them together in a widget. So for instance I can bundle a label, a textbox, and another label or a combo box in a widget which can be [re]used to input something with a unit of measurement. Do that the right way, and you end up building an OO UI, with things made of things made of things.
However, even in that case, it's hard to reuse some common code, like disabling a button when some field is empty. There is, unfortunately, some repetitive machinery to implement, especially in lambda-unfriendly Java. You have to implement an interface, add to a collection of listeners [sic], etc etc. Things get worse when you see that (for instance) in Android you can't just disable a ViewGroup to disable the internal controls; you have to do it individually. In the end, this kind of logic is not something you can easily place in a single reusable widget. From an AOP perspective, it's a different aspect altogether, and plain OO can't deal well with that.
That same kind of repetitive logic, however, can be easily put inside rules, actions, etc. The infrastructure will then take care of implementing interfaces, adding listeners, combining results with joins, etc. That’s what my little library does. Actions and predicates, in this case, are also reusable classes (not custom methods inside a single Rules class). But here is the thing: this is not the dominating structure in my code, and certainly not in my thought processes.
It's a small set of rules, hidden inside a widget. Some widgets don't even need rules. Some do. As the widget structure is fractal, there is no explosion of rules, because the nature of the widget is to stay small (possibly by including other smart widgets), so I never end up with a humongous set of rules in a single place. Also, data is always local (the contained widgets), so I don't suffer from a large data structure flowing around. That's it. Objects outside, a few rules inside, when / where needed.
I have similar structures elsewhere. I tend to model communication protocols as workflows, and there is a workflow engine and a workflow context and all that stuff. But it's not a dominating structure. Inside (say) a Device, you'll find a workflow to handle the communication protocol. If the protocol is trivial, you may not even find a workflow. It's as simple as that. It's Device outside, workflow inside.
As you asked me, via email "What´s the right way? Is there a right way? Is there overlap? Is there a continuum?", I can honestly say that I don't pretend to know the "right" way, especially not some sort of universal "right" way, a good fit for every problem. But I have definitely combined objects with rules and with workflow engines, just not the way you do. You tend to create a two-layer stack. I tend to create a fractal structure, because it leaves a trail of meaningful, reusable, domain-specific abstractions I can reuse elsewhere, encourages information hiding, and can grow almost indefinitely while preserving decent maintainability. Inside some of those abstractions, I use rules and workflows.
As I said elsewhere in this blog, I value freedom, I value options, I value adaptability. I design things by looking at forces and responding to those forces, using a mixture of approaches and materials. Your post would have been more interesting if you presented it like another option, perhaps going into more details about your proposal instead of wasting way too much time trying to trash mine or promoting your own process as superior.
While you consider your design to be self-evident, I find your explanation lacking. You don't show how / which data is supposed to flow around, how you avoid the problem of excessive data coupling typical of functional decomposition, how you intend to deal with reusability, scaling up/down, how you react to changes, etc, that is, you never talk about the properties of your code, at a level of detail where we can actually believe you, not assuming more or less magic infrastructure.
That said, I guess / hope I can use your engine my own way, inside one of my classes. If I can't, then I just won't use it. I also guess / hope I don't need to become part of a method cult to use it, or to bow to your process and let it dominate the entire design. If I have to, I won't use it. However, I'm sure some people will happily surrender freedom in exchange for a standard blueprint. There is ample evidence of that (just think of J2EE :-). It's just not what I consider #braindrivendesign.
By the way, Ralf: can't we just be friends? :-)
If you liked this post, you should follow me on twitter!
The brick house is courtesy of David Sawyer (http://commons.wikimedia.org/wiki/File:Brick_House.jpg)
The wooden house is courtesy of Robzle (http://commons.wikimedia.org/wiki/File:Podbiel_wooden_houses.JPG)
The layered :-) house is courtesy of Joadl (http://commons.wikimedia.org/wiki/File:Noetsch_Saak_altes_Haus_Kulturdenkmal_2008_0814.jpg)