Monday, July 30, 2012

No Controller, Episode 3: Inglorious Objekts


A week ago or so, Ralf Westphal published yet another critique of my post on living without a controller. He also proposed a different design method and therefore a different design. We also exchanged a couple of emails.

Now, I'm not really interested in "defending" my solution, because the spirit of the post was not to show the "perfect solution", but simply how objects could solve a realistic "control" problem without needing a centralized controller.

However, on one side Ralf is misrepresenting my work to the point where I have to say something, and on the other, it's an interesting chance to talk a bit more about software design.

So, if you haven't read my post on the controller, I would suggest you take some time and do so. There is also an episode 2, because that post has been criticized before, but you may want to postpone reading that and spend some time reading Ralf's post instead.

In the end, what I consider most interesting about Ralf's approach is the adoption of a rule-based approach, although he's omitting a lot of necessary details. So after as little fight as possible :-), I'll turn this into a chance to discuss rules and their role in OOD, because guess what, I'm using rules too when I see a good fit.

I'll switch to a more conversational structure, so in what follows "you" stands for "Ralf", and when I quote him, it's in green.


My dear Ralf...
Overall, I've found your emails to be kind and reasoned, but your post to be filled with the usual rhetoric, whereby in order to look good, you have to make the other guy look bad. In the end, I didn't like it much when you said I'm cheating or that I'm proposing a spaghetti design. Especially when you email me after the fact and ask not to turn this into a zero-sum game. Usually you reap what you sow. But let's say I don't mind too much, and move forward.

Your post can be seen as organized around:

1) A critique of my design approach and of my presentation style (like using class diagrams instead of, say, activity diagrams and use cases)

2) A proposal for a design method, basically the well-known functional decomposition approach, with an interesting but underdeveloped twist toward a rule-based system.

3) A final design, represented by a "flow diagram" and a sort-of class diagram.

Interestingly, you didn't pay any attention to the most important thing, that is, the properties of the results. Because the result, Ralf, is not the class diagram. It's not even the flow diagram, of course. It's an allocation of responsibilities to different "things", those "things" being classes, methods, whatever, resulting in properties, like extendibility, reusability, and stuff like that.

It's very much like having these two nice houses:



and instead of discussing the properties of the houses (building and maintenance cost; serviceability; maximum load; heat transfer; etc) spending all your time in a rhetoric exercise on whether we should start with structural engineering calculations or by drawing the floor plan. Up to the point where you confuse the floor plan with the house. The product of my design process is not a class diagram, just like the product of an architect is not a floor plan. Is the set of properties embodied in that diagram, which will be reflected into the real thing (the code, the house).

You spend a lot of words babbling about process, but you never talk about the properties of what I get and the properties of what you get. All you say is that you don't have all my messy connections (in your diagram), which as we're going to see is incorrect at best. That makes everything rather moot, and with reference to what you said via email, it's hard to move the field forward by process rhetoric.

Anyway, I'll try to cover your critique as quickly as possible, I'll try not to criticize your design process much / at all, and I'll try instead to look at what you get in the end. Because I'm not really interested in discussing process, until we compare results. Results come first. Once someone can show consistently good (or bad) results, it's interesting to look at his process, to see if we can somehow extract some of those good or bad qualities. Babbling about process without looking at results is pointless. And I'll say that again: results are not diagrams; results are the properties of the things we build.

Process, process...
As I said, I didn't like this part much, because you grossly (and seemingly intentionally) misrepresented my work, while at the same time profiting from it. You make it look to the occasional reader as if I jumped straight from a pictorial description to a "complex" class diagram, without explanation, so that you cannot even understand how / if that thing works. But you perfectly know that's not what I did. I took the problem, piece by piece, discussed how I would allocate every portion of behavior to a class, presented small diagrams, and then put them together. I discussed likely changes and how the design was ready to deal with them.

For instance, you criticize me for beginning with a (simple) class diagram instead of bringing in the user. Well, your first diagram adds nothing to the pictorial description of the problem I have included, except the operator. Unfortunately, you choose to focus on the wrong user. The mine pump is not there to satisfy a data-hungry operator (a controller? :-)). It's obviously there to guarantee the safety of mine workers. Perhaps I should have gone the BDD way, adding stories like:

- As a miner, I want to know when there is too much methane, so I don't blow myself up.

- As a miner, I want to know when the pump is stuck, so I won't find myself drowning in mud.

- As a miner, I don't want the pump to turn on when the methane concentration is high, because I'd rather come back home alive.

- etc.

I didn't, because it was already a long post and I thought it was kinda obvious. Maybe it was not, but anyway, the operator is not a central actor here (although, of course, in a real system there will be some operator console somewhere). Guess what, your design is fine anyway, because adding that [wrong] user into the diagram provided no additional value, except of course a healthy dose of process rhetoric. It's fine if you feel compelled to draw diagrams that add no value; I don't, sorry.

You're also assuming / implying / suggesting that I'm drawing class diagrams because I'm focusing on data. Actually you say that the role of a class diagram is to show the structure of data, and a comment on your blog even talks about "obsession with state" on the side of OO developers. This is very funny because those class diagrams show no data, exactly because I was focusing on allocating behavior.
I have relationships, but those relationships are not about data, but about collaboration. Perhaps you're looking at classes and class diagram from the wrong perspective.

You're also assuming that I never consider process in my reasoning, and that I jump into classes. Of course I consider process, but instead of modeling everything top-down a-la functional decomposition, I'm looking for candidate places for that behavior, as it emerge.
You already have one in mind from the very beginning - the "rules" collection, so you don't need to allocate behavior. You just have to fit every problem into a pre-cut design. Unfortunately, a plain sequence of rules won't work it in this case, and won't work it in any real-world case either. You need something more complex, like a forward-chaining rule engine, that you neglected to show, but I'll get to that in a short while.

It seems to me that, being so overly focused on a universal process and perhaps a universal blueprint, you think that I too am proposing some kind of universal process. I don't. I called that post "case 1" because it's just one of many. It's a realistic example though, and that's the reasoning I used in that case, because I saw a good fit with the problem. I have no problems using an activity diagram when it's called for. Or a state diagram when I'm dealing with a complex state machine. I think it's wrong to propose a fixed approach, irrespective of the problem. Of course, it's your right to differ and to propose your own universal process. You won't be the first to try; you won't be the last to fail.

I will gloss over the part where you say I cheated by choosing an automation problem. You may think that "it´s notoriously hard to find such objects in the requirements" outside real-world objects. Still, along the years, I've been working basically in every possible application domain, and I've a consistent record of finding good classes. There are many other examples in this blog as well. Finding good classes is a skill. A skill you decided not to learn. It's ok. It's not so ok when you call those who can do it cheaters, but I understand it's part of the usual old-school tactics.  

Just a final comment on understanding the dynamics through a static view. You said: I deem it no virtue to be able to “reverse engineer” a class diagram to get an idea how things are working. Of course, you're entitled to your values. However, there is a long and consistent tradition in engineering to look at a static picture and infer the dynamics.

We expect mechanical engineers to look at fig. 2 and 3 here and understand the kinetic part.

We expect electronic engineers to look at this picture and understand it's a multivibrator.

In facts, I used to repair TVs as a kid. In my lucky days, I had a schematic for that TV set. Schematics are popular, because they're a rather efficient way to transfer information to skilled people. Of course, you have any right not to learn that skill. For instance, I'm not very good at reading musical notation. However, I would never step up and say that I do not consider that valuable for a musician or a composer.

So, yes, I would also expect a software engineer to look at this:

and understand that it's very likely to be a callback scenario. There would be more to say about stereotypical interactions and the use of colors, but this is not the right time, I guess.

Note how, in my second post about the controller, I proposed this engineering representation of the mine pump:

Once you get past the rhetoric about class diagrams being evil and sort-of-activity-diagrams being good, that picture has a 1-1 correspondence with my class diagram (polymorphism aside, which is here for non-functional reasons). Maybe you don't "get" that diagram either. Oh well. Let's focus on what you got instead.

What you did and what you got
Once you cut the crap and get down to business (what you call the API level) you seem to drop all the big words on design being independent on classes etc and bring in the "dirty stuff" as you call it. I'll never understand why some people think / speak this way, but ok. By the way: part of reasons I'm trying to move software design toward properties, and more broadly toward a physics of software, is exactly to avoid this kind of ivory tower thinking / talking, which benefits no one. We (the software design community) should try to collectively raise the narrative bar and leave that rhetoric behind.

Anyway, at that point you're beginning to create what I've defined a "fake OO step 2" system, that is, you create classes for the simplest abstractions (pump and sensor) and you collapse all the other logic into a centralized place. Nothing new so far (we have to wait for rules to get something new).

There, however, you benefit from my speculation on likely changes, while at the same time trying to make my design look bad. The original problem statement was based on digital water sensors. A digital sensor will only say "it's full, up to my level, dunno above". The original specification of the hysteresis cycle was therefore based on having two digital sensors. That's why I started with a SumpProbe connected to 2 LevelSensor, and then discussed how moving to a continuous LevelProbe would require changes. Changes that I've hidden behind the SumpProbe class.

Now, in your design you have no visible place for a digital sensor, so I can only assume one of these two options (you didn't say; so much for your stuff being much more precise):

a) you have a composite sensor somewhere, exposing two digital as a sort of simulated analog.

b) you just return an integer instead of a boolean, and you deal with two sensors becoming one elsewhere (the WaterLevelSensor? maybe, but you don't even mention the issue).

If you go with (a), why did you move the logic for a composite digital sensor below the API level? The problem statement was all about digital sensors, and according to your "process", you're not trying to build new abstractions as you go. You get the barebone things at the API level and put the logic in that process thing. So doing (a) is sort of cheating, right?. Well, also (b) is sort of cheating, because you are moving part of the logic below the API level anyway, cloning parts of my design. 

There is more. My SumpProbe deals with hysteresis, which is not a property of a physical sensor; it's a process thing, to avoid turning the pump on/off continuously. It's also a stateful thing, because hysteresis needs status. Where do you handle that? Again, you don't even mention this "detail", so we're left to wonder. 
Say that, being a process thing, you do what you do for other process things and move it above the API.  Then it' just another (neglected) oval in your process. Cool. How and where is the attached state preserved? You don't say. In the Rules class? What happens if the mine gets bigger and I need to add pumps, sensors, and glue them together? I can just instantiate more objects. What do you propose?
Say that it is inside the WaterLevelSensor. But then you're cheating again Ralf, moving part of the process below the API at your convenience, without any clear method, mimicking my design when it suits you. 


In the end, you waste a lot of time talking about process, but neglect to say how things really work in your design. You omit details and claim simplicity, and you do this over and over.


You also spend a lot of words on why the hell did I use different classes for different gas sensors. As you're trivializing everything into a single class, you blissfully ignore, for instance, that sensors may have to linearize the input, compensate for drifts or ageing, detect specific failures, etc etc. Of course, in the real world we would have base classes and reusable algorithms for that, but no, it's not like we can simply put everything inside a GasSensor returning an integer and say it's done. 


The worst part, however, is that all this critique was unnecessary to make your point; and honestly, it seems that you're trying to criticize something you don't fully understand.

Then, at some point, you propose storing all the sampled data into a large SensorData thing. It's not clear what you have in mind, as you choose not to represent that thing (sure, omitting "details" makes things look simpler, right? :-). Perhaps it is a dynamic bag. Perhaps it is a strongly typed structure. In this case, it's rather crappy, as it will break whenever you add more sensors (bye bye modularity; bye bye independent reuse). Also, who is responsible for storing each data in its slot? The individual sensor, or some kind of centralized samplER? I guess a samplER, as your sensor is just returning an int.
Anyway, that will get you exactly into the hourglass shape I was speaking of in my recent post on the forcefield. Of course, you don't seem to mind, so please, go ahead and ignore the underlying forces at your convenience. Actually, this piece alone, if you cared to explain it better, would make for a nice comparison of encapsulation vs. the typical "store all the data here where everyone can read and write everything" relics of structured programming. But you didn't provide details, so there isn't much to say.

Finally, we come to an under-specified class diagram where you say: "The classes are so little connected, I hardly need a class diagram at all."

This is so ridiculous it's almost insulting. It's not like you can just remove classes and dependencies and claim that it's simpler. It has to work. So in your case:

- The clock has to call the sampling, so it has to be directly or indirectly connected to a sampler (if any) and that in turn connected to sensors, but we don't see that. Sensors hang out nowhere.

- The sensors or sampler have to populate the SensorData thing, but we don't see that class at all and so we don't see any dependency.

- The SensorData thing has to go to your Rules class, but we don't see that.

- Somehow the PumpState has to move to the pump, but we don't see that.

- etc etc.

Connections in my diagram are there because data (or objects) do not flow from one place to another on the back of unicorns. Except in your diagrams, of course :-). Oh, maybe it's all in global variables. Or maybe there is a sort of workflow context that again you neglected to show to claim simplicity. Is that context type safe? A dynamic bag? Again, you never say that. Absolutely precise, sure :-).

Of course, you may (will) claim that there is some magic infrastructure taking care of all that (and more, see later about the rules). Guess what, I could have done the same. In episode 2, I discussed a wiring infrastructure. I also mentioned how one could simply use an IoC in some cases (also in the first post). Don't you see I could have just "assumed" that kind of infrastructure, just like you do? I could have drawn a diagram with basically no connections, assuming things would get wired at run-time through configuration. I choose not to. I choose to show how things have to be wired anyway. I choose to show something that didn't need any infrastructure.

It's so funny that you called me cheater, isn't it? Except that my spaghetti class diagram (spaghetti obviously meaning, for you, that I have very controlled dependencies toward more abstract classes to keep everything modular and reusable) is the diagram of a working system, while your diagrams is a depiction of something that does not work because is neglecting essential components and relationships, plus the entire damn infrastructure you never cared to mention. C'mon.

However, you do offer an alternative to the traditional, hard-coded controller, and that's the Rules class. If you didn't bury that in a pile of bullshit rhetoric, it could have been a more interesting, focused and pleasant conversation :-). So let's focus on that.

The Rules
So, in the end, the only relevant difference from the usual "simple classes below, monolithic controller above" (fake OO step 2) is that you decompose your controller in rules. 

Of course, you're assuming some infrastructure here, because as presented, the Rules class is useless. Also, in your class diagram, there is no trace of the join between Assess_water_level and Assess_methane (there is also no output from that, but hey, you want to pretend you have no dependencies).

Obviously, you can't just have a list of rules. You need a graph and a forward-chaining engine or something like that (been there, done that). You need a machinery to make the join, and if you support independent threads, it will also have to wait until all the inputs are present before calculating the output or the guy in the mine will die.

And of course you're just assuming all that. I had to read some comments on your blog, follow a link to a "cheat sheet" of yours and click around for a while to discover that you're proposing your own homegrown rule engine. Phew. That took a while. Sorry, perhaps that thing is all rage in your neighbor, but I didn't know about it. It would have been nice of you to introduce that thing from the very beginning, perhaps removing some of the unnecessary process crap stuff to make some room.

After spending enough time understanding what you didn't care to explain, your design started to make a little more sense. You created a rule engine. So once you write classes for the actions, you "just" have to wire those things through configuration. My "spaghetti" diagram assumed no library, no support, no nothing. Your pseudo-diagram assumed a rule engine, a wiring library, and what else (and didn't even bother to show a placeholder for some of that). Then you claim simplicity, and call me cheater. How cool is that Ralf :-). Oh, and there is still the issue with the missing SensorData thing and so on, but I'm gonna cut you some slack.

The paradox here is that if you approached things another way, we would have found a lot of common ground. Because you know, I use rules and rule engines as well. For instance, I've just ported my GUI Rules library to Android. So now I can say things like (in a fluent-Java style):

rules.add(
  enabled(okButton).
  when(notEmpty(uid).and(notEmpty(pwd)))
  );

It would be unfair to claim simplicity behind that thing, however. There is quite a lot of machinery. You don't see it when you combine rules. But you can't just pretend it's not there and claim that things are magically simplified by process. They are simplified by a library.

Now that I have brought this thing to what I hope is a more neutral territory, I'd like to explain how my view of rules differs from yours, and why.

Two-levels vs Fractal
I guess you're familiar with this kind of house:


It's bricks and concrete below, wood above. Of course, you have to stack them in the proper order :-). So there is a scalability issue here, because you can't just add another brick layer on top of a wood layer.

So Ralf, let me ask you something. As you know, if you read chapter 2 in the controller saga :-), I choose the mine pump because it's a well-known problem, people have already proposed meaningful changes, and I don't have to cheat, I don't have to make things up, I don't have to propose changes that I'm safely well-prepared to handle. There is literature on that.

A common, meaningful change that I have already discussed is the need to have two or more pumps, and rotate among them over time to reduce wearing. Where would you introduce this change in your design? You basically have two choices (if you want to preserve the overall design):

- you can introduce the change below the "api level" (bricks)

- you can introduce the change above, that is, in processes and rules (wood)

This is a "business rule", so to speak, so it would seem more appropriate for you to introduce the change using rules. You can add yet another oval after "switch pump", choosing which pump.

Then the next meaningful change is that if one of the pumps gets stuck, as observed through current sensing, you have to switch to another pump anyway (even if it's not its time yet).
If you deal with that at the process / rule level, you need an "assess current" oval, then you need to change the switch logic above, drawing another join to consider the current too, and switch to another pump when needed. You may think this is ok, because hey, it's the process.

And here is where you should have your epiphany, Ralf. When you say "Well, yes, that´s a kind of functional decomposition. And I don´t see what´s wrong with that" you're just ignoring 40 years of mankind experience with functional decomposition.

I can tell you what's wrong with that: as you scale it up, you're not building reusable abstractions along the way. You're just making the upper part more and more complex. And after you beef it up, you can't scale it down (see Parnas), without creating a clone of your large process and pruning things here and there (remember to remove unnecessary slots in your SensorData too :-).
You have only one, big gravitational center (your process), and that's it. Your rules are probably reusable (except you modeled them as method, not classes, so it's still copy&paste reuse), but the "hard knowledge" is in the wiring of the diagram, not in the rules (which, I hope you'll admit, tend / want to be very simple).

You see, just like you, I wasn't born with objects in my hands. I wrote my first code back in 1978. I know what is like to create large systems using structured programming and functional decomposition.
The real beauty of objects, once you really understand objects :-), is exactly in that thing you don't seem to appreciate: objects as virtual machines, things made of things made of things made of things, all of the same power.

When you stack simple objects under a ton of increasingly complex procedures (whether you bundle those procedures into a controller or in a rule engine) you'll never get the reusable abstractions I'm building along the way: the pump rack, the fail-safe pump rack, the sump probe, the safe engine, etc. Abstractions that can be reused in a completely different problem, where (for instance) methane is not an issue. Too bad in your process the methane comes first.
Your flow diagram is not reusable, and is bound to get bigger and bigger as you scale up requirements. You'll have to go the usual way, and adopt nested diagrams, whereby an oval on one level is an entire diagram on the next. Then you're going to discover you're passing a ton of data around (your SensorData on steroids), but wait, most of those data are only used in some places. And you're going to come closer to some form of encapsulation to solve that form of coupling (there is a long story on how the notion of coupling changed when moving from structured to OO programming, and there is something to be learnt from that. I'll talk about it in a future post). Keep going ahead, and sooner or later, you'll rediscover modules and then objects, because objects are a reasonable (although not definitive) response to some of the forces we routinely encounter in software development.

Oh, by the way, you could also add those responsibilities below the API line instead. Then I really would ask you, why only that? Why don't you go further and push down more and more responsibilities? What is, in your universal process, the guiding force in placing something below or above the API level? Why do we need that distinction at all? Can you see how your "standard blueprint" is trying to squeeze every problem into the same shape?

So why / how do I use rules ?
I just said that I use rules too, so this may seem like a contradiction. But it is not. 
The thing is: you make process and rules the center of your system (and development process), and stack that on top of trivial objects, so you get a two-story shape that does not easily scale. I use objects as my primary decomposition technique. Inside some of those objects, I use rules. But it's "just" a convenient implementation detail. Let me elaborate with an example (one of many :-).

I tend to design my UI as a thing made of things, so I usually have a form made of widgets, made of widgets, made of widgets. There are recurring problems in UI implementation though, like disabling some controls when there is no input or when a checkbox is not ticked, etc. The popular thing is to use standard widgets + a large, form-wide controller or view model + binding (assuming you have a binding-friendly UI library, which is not the case in Android, for instance). This is a two-level approach, with relatively stupid, standard widgets below and a more complex monolithic controller on top.

I don't like monoliths, so I look for meaningful aggregates and bundle them together in a widget. So for instance I can bundle a label, a textbox, and another label or a combo box in a widget which can be [re]used to input something with a unit of measurement. Do that the right way, and you end up building an OO UI, with things made of things made of things.

However, even in that case, it's hard to reuse some common code, like disabling a button when some field is empty. There is, unfortunately, some repetitive machinery to implement, especially in lambda-unfriendly Java. You have to implement an interface, add to a collection of listeners [sic], etc etc. Things get worse when you see that (for instance) in Android you can't just disable a ViewGroup to disable the internal controls; you have to do it individually. In the end, this kind of logic is not something you can easily place in a single reusable widget. From an AOP perspective, it's a different aspect altogether, and plain OO can't deal well with that.

That same kind of repetitive logic, however, can be easily put inside rules, actions, etc. The infrastructure will then take care of implementing interfaces, adding listeners, combining results with joins, etc. That’s what my little library does. Actions and predicates, in this case, are also reusable classes (not custom methods inside a single Rules class). But here is the thing: this is not the dominating structure in my code, and certainly not in my thought processes.

It's a small set of rules, hidden inside a widget. Some widgets don't even need rules. Some do. As the widget structure is fractal, there is no explosion of rules, because the nature of the widget is to stay small (possibly by including other smart widgets), so I never end up with a humongous set of rules in a single place. Also, data is always local (the contained widgets), so I don't suffer from a large data structure flowing around. That's it. Objects outside, a few rules inside, when / where needed.

I have similar structures elsewhere. I tend to model communication protocols as workflows, and there is a workflow engine and a workflow context and all that stuff. But it's not a dominating structure. Inside (say) a Device, you'll find a workflow to handle the communication protocol. If the protocol is trivial, you may not even find a workflow. It's as simple as that. It's Device outside, workflow inside.

As you asked me, via email "What´s the right way? Is there a right way? Is there overlap? Is there a continuum?", I can honestly say that I don't pretend to know the "right" way, especially not some sort of universal "right" way, a good fit for every problem. But I have definitely combined objects with rules and with workflow engines, just not the way you do. You tend to create a two-layer stack. I tend to create a fractal structure, because it leaves a trail of meaningful, reusable, domain-specific abstractions I can reuse elsewhere, encourages information hiding, and can grow almost indefinitely while preserving decent maintainability. Inside some of those abstractions, I use rules and workflows.

Conclusions
As I said elsewhere in this blog, I value freedom, I value options, I value adaptability. I design things by looking at forces and responding to those forces, using a mixture of approaches and materials. Your post would have been more interesting if you presented it like another option, perhaps going into more details about your proposal instead of wasting way too much time trying to trash mine or promoting your own process as superior.

While you consider your design to be self-evident, I find your explanation lacking. You don't show how / which data is supposed to flow around, how you avoid the problem of excessive data coupling typical of functional decomposition, how you intend to deal with reusability, scaling up/down, how you react to changes, etc, that is, you never talk about the properties of your code, at a level of detail where we can actually believe you, not assuming more or less magic infrastructure.

That said, I guess / hope I can use your engine my own way, inside one of my classes. If I can't, then I just won't use it. I also guess / hope I don't need to become part of a method cult to use it, or to bow to your process and let it dominate the entire design. If I have to, I won't use it. However, I'm sure some people will happily surrender freedom in exchange for a standard blueprint. There is ample evidence of that (just think of J2EE :-). It's just not what I consider #braindrivendesign.

By the way, Ralf: can't we just be friends? :-)


If you liked this post, you should follow me on twitter!

Acknowledgements
The brick house is courtesy of David Sawyer (http://commons.wikimedia.org/wiki/File:Brick_House.jpg)
The wooden house is courtesy of Robzle (http://commons.wikimedia.org/wiki/File:Podbiel_wooden_houses.JPG)
The layered :-) house is courtesy of Joadl (http://commons.wikimedia.org/wiki/File:Noetsch_Saak_altes_Haus_Kulturdenkmal_2008_0814.jpg)

23 comments:

Polka said...

You all do it wrong :P

For some reason people think objects have to be controlled from the outside, but knowing "what to do when" is rather a responsibility of the object itself than that of a collaborator, which 'controls' stupid things.

Therefore, the object's interface is most often imperative for control: DoActionX(), DoActionY(), when it should be reactive/event-like/declarative. The object should only know what happened in the domain: ThereWasX(), ThereWasY(), ThisChanged() and control itself by executing the right behavior.

Carlo Pescio said...

Interestingly, Polka, there are no stupid objects in my design. For instance, the GasSensor simply informs when a critical level is reached. The safe engine monitors that and reacts, by ANDing the condition from what comes from the SumpPump (for safety).
The SumpProbe does its own logic and informs others that it is time to drain. The SumpPump reacts to that.
So I see no real contradiction with what you say. Except that in practical cases, we always have a mixture of react and tell. The SumpPump reacts to the SumpProbe, by telling the SafeEngine. Reusability and separation of concerns play an important role there....

Ralf Westphal - One Man Think Tank said...

@Carlo: Thx for your extensive answer to my posting.

However it´s sad you found mine aggressive and switched into something I understand as "retaliation mode". I re-read my posting and found not nearly as many rebukes, devaluations, and outright attacks as you uttered here in your posting.

If you feel offended by a light sprinkle of "spaghetti code" or "I deem it no virtue" or maybe "just as structural diagram" let me apologize.

Can we be friends? Don´t know. I thought so after reading your original article and for what I´ve read from your physics approach to software development (although I disagree on some aspects of it).

This article, though, leaves me somewhat flabbergasted. You seem to be hurt. And you seem to feel the urge to defend your position by all means.

I understand that - somewhat. But I don´t want to get into the same mode.

You read misunderstanding from my article and even call it babbling. (Which hurts and surely is no widely accepted means to instill constructive dialog.) But be assured: I read misunderstanding from your response as well.

You left out details - and that´s ok. (For you at least. And I tried not to pick on that.)

And I left out details, too. That should be ok for you as well. It´s out of pure necessity. There´s limited time for both of us and the readers.

How to continue? I don´t know. I better do not answer right now. I would not be the calm answer the topic deserves. Maybe I even should not try to rip apart your text and comment on individual statements.

Instead I guess I should deliver what you (and I) haven´t: a working system. Well... maybe if I find time for that. (And if not, please don´t derive from that inability. Like you I´m doing this all the time for a living.)

So to sum up: It´s sad this went off in the wrong direction. (And please don´t blame that solely on me.)

Can we be friends? Maybe.

PS: Yes, there is one real difference in our standpoints. I do not value reusability very high. In my view that´s the most overrated promise object orientation has made. My reason is simple: I´ve seen too much money burned and customers put off by teams striving for reusability. It´s simply of limited practical value for every day projects.

I´m not against it, though. I love reusable widgets/controls/libraries of all sorts as much as you do. It´s great we´ve the technical means to package functionality in a reusable way. But that´s different from putting resuability high on the agenda of just about every team.

Carlo Pescio said...

Whoa, and I actually tried to be kind : )

I should change my tagline to PinkyAndTheBrainDrivenDesign to acknowledge the fact that I'm evil :-)

Thomas Zeman said...

I am following your two blogs for quite a while now.. Can anyone of you be so kind and point out the relation of the whole topic to something like the Windows Workflow Foundation? Is the WF supposed to work in or with objects?

Anonymous said...

Yes Carlo: could you please explain me a little more about the "workflow engine" you use to implement protocols? I can remember you already cited it in another post...

Searching on the web I only found definitions related to enterprise software... but I'm sure it's not what you're speaking about.
Is it similar to a FSM engine? What's the difference?

Thanks
Daniele

Carlo Pescio said...

Thomas: my opinion (but then, Ralf may well have a different view) is that WF was built with a specific set of features and therefore as set of trade/off, to support:
- relatively long-lived workflows, often with a human component
- relatively coarse-grained activities
- the ability to dehydrate state to a db
- etc

This is of course a different beast from a GUI rule engine, where things are very fine-grained and actions last a few microseconds, and also from the workflow I use in communication protocols, where again I have fine-grained actions spanning a few milliseconds.

To offer a parallel, WF is to a rule-engine or fine-grained workflow as I described like SQL server is to a .NET dictionary. Both can be thought like associative containers, but the set of trade-off is very different. So it's sort of natural to pop open a class and find a dictionary inside but it's less than natural to open a class and find "a database" inside, so to speak.

That parallel is deeper than it seems. It used to be that "database oriented" applications were totally permeated with the notion of persistence. In a sense, the architecture of the application was mandated by the DB. Actually, a group of developers I know is using a database-centric approach in process control as well (of course, they deal with slow processes and tend to coerce Oracle to keep everything in ram, but ok, it's their choice).

Over time, however, the preferred development style changed to a point where it's more common to have the DB "hidden" behind repositories, catalogs, etc than to have persistence spread everywhere, and applications are becoming more domain-centric than DB-centric.

The same can be said for things like WF. It's still very common to have that kind of large-scale component dictate the architecture of the application (so, to be used "with objects"). On my side, I like to keep it separate (but it may well be because that was a good fit with the kind of app where I've used a coarse-grained workflow engine). Very much like we have repositories for persistence, I have higher-level interfaces, typically event-based interfaces, and I'm communicating with the WF through those interfaces. So the WF is usually one of the [many] components of my architecture, and does not dictate the architecture.

Of course, as I keep saying, what is most important is to align the architecture with the set of underlying forces (the problem to be solved) and to make sure we're getting the expected properties out of our design.
This should be the real guideline. One may find that in particular scenarios (where, for instance, people used to adopt BizTalk), having WF as the spine of the project is the best choice. I rarely work in that kind of project (which is more like system integration than software development; I work on software development :-) so my perspective can be skewed.

Carlo Pescio said...

Daniele: to keep using a parallel, a small workflow engine like mine is to a FSM engine like an activity diagram is to a state diagram :-).
Very similar things, with some modeling differences. Depending on the protocol, one might be more appropriate than the other.
For instance, in the android app I basically go through a set of phases, like device identification, subprotocol negotiation, activities acquisition, lap acquisition, trackpoint acquisition, end. I never go back to a previous state, I accumulate knowledge along the way to be used in subsequent activities (hence a context), I dynamically choose a sub-workflow based on protocol negotiation. Every activity ends because I got all the data. It fits an activity diagram better than a state diagram, so a small wf engine better than a fsm engine. Then at the implementation level you have a few similarities, but also differences as you don't speak in term of events but natural conclusion...

Ralf Westphal - One Man Think Tank said...

@ThomasZeman: I agree with Carlos description of WF. Strange, isn´t it ;-)

But seriously: WF to me is a tool. Or even an implementation detail in some regards.

Some aspect [sic!] of a software system might be modeled best using a finite state machine. Some other aspect might best be modeled as a data flow. And still some other aspect might best be modeled as a work flow. And if that´s the case, well, then maybe WF is the implementation tool to use.

(At this point it´s not important that I think most parts of a software are best modeled using a data flow approach ;-) I don´t want to upset Carlo :-)

When is WF the tool to use for a work flow? In my view it´s a heavy weight. So unless the work flow is distributed and/or needs persistence, don´t use WF. But right here that´s a very roundabout recommendation.

What´s the difference between data-flow and work flow? It´s where the processing steps expect their data. In work flows (remember flow charts or structograms?) data is expected to reside in the environment. They work on global data of some sort. So work flows are some kind of procedural programming, I´d say.

Data flows on the other hand, well, let the data flow. This leads to less coupling or to more controlled/explicit coupling of processing steps to data.

This might seem as a superficial difference. But in my experience it is not. Once you get into "data flow thinking" the world looks different. Some problems become non-issues. Other problems appear. It´s a trade-off.

PS: By the way, I don´t view object orientation in opposition to flow orientation (data flows). So, Carlo, no need to aim your verbal cruise missiles at me again ;-)

Thomas Zeman said...

Thanks guys for shedding some light onto this. I (as a consumer of what you publish) appreciate that you spend so much time on sharing your thoughts.

I guess I will have to pay more attention on those aspects in my daily work. Especially the workflow vs. dataflow distinction. Right now it is not a 100% clear to me because even calling a plain method (in the same or a different class - doesn't matter) is already data flow because I am passing data but also work flow in some sort - pro actively (vs. passively steered by a framework like the WF). As far as I remember and understood Ralf's whole idea of flow design originated in abstracting and decoupling method calls. Even using interfaces slash contracts led to too strong bindings so his idea eventually reached the "connect small functional units" state.

Looking at for example processing data in a video pipeline or network stack I could pass data to the next step by talking to an interface IProcessingStep or more generic: some Action.

Carlo wrote "no please no handlERs, managERs, controllERs or processORs" :) (even though Processor ends with OR I suppose the same rule applies) but I have to admit I have a hard time finding a different name for a class when it just takes data, transforms and outputs it. (Network: Unpack envelope by envelope..)

But also: Looking at something like an "UserRepository" and methods like "AddUser", "RemoveUser" or "GetBirthdayForUser" or you name it, I can not see how data flow or abstracting method calls would be beneficial here. Smells like OOP.

So whats my conclusion here? I am probably doing both without reflecting on it so much so far.

Ralf Westphal - One Man Think Tank said...

@Thomas: Look at this:

var x = 0;
var y = 0;
p();
Console.WriteLine(y);
...
void p() {
y = x +1;
}

That´s a work flow :-)

But

var x = 0;
var y = f(x);
Console.WriteLine(y);
...
int f(int x) { return x + 1; }

is a data flow.

Data flows are nothing much special. We use them all the time.

My point is: try extending their use - and you´ll realize that software development can indeed become easier.

As with all tools/technologies that´s not a "one size fits all" solution of course. Some thinking is still required :-)

But if you followed my writing in my blogs and in the dotnetpro magazine you´ll have seen many examples where data flows complement object orientation. My intention is not to replace OO, but to put it in perspective.

If after 20+ years of heavy OO marketing average developers still have so many difficulties to arrive at sensible OO designs... well, then maybe the design approach is not as mass compatible as its proponents would like it to be.

How else can we understand the fervor with which Carlo is writing about OO here? He´s obviously seeing the same OO situation out there. But his approach to help is different. He´s trying to teach better OO thinking or even "true" OO thinking. Whereas I´m trying to show how in part the life of a programmer can be made easier by trying a different route.

At least, though, Carlo and I agree: no more controllers, managers, services, coordinators etc. They are a symptom of a root problem that needs to be solved.

Thomas Zeman said...

understood, Ralf.
I think for me, it would be helpful to approach the whole topic slightly from a more "scientific side" rather than examples where the focus is too much on the "best" solution.
Answers to questions like:
- How coarse grained are the 'units' data is flowing inbetween?
- How do I realize inheritance?
- How do I realize compostion?
- What about Aspect Oriented Programming?
- How to do Dependency Injection?
- Testing? Can I mock?
- Collections?
- Why is it and when is it helpful not to call "int Foo(int x)" but just Func or raising an event?

- I have concerns that you have 2 locations of programming logic. Code and flow description (file). For example for dI I guess you would have the same code but two different flow descriptions. Sounds somehow unhandy.

Maybe you can analyze the nature of the beast a bit more in the sense of these questions and write a blog post about it? That would be cool. (And sorry if you already did. I've probably missed that post then :/ )

Ralf Westphal - One Man Think Tank said...

@Thomas: I´d be most happy to answer all your questions. But I guess this is the wrong place for that. This is Carlo´s blog and he´s focussing on object orientation "the right way" - whereas you´re asking question about Flow-Design (although in conjunction with object oriented aspects).

Of course I don´t mind Carlo being part of the discussion, if he likes. But this blog, I guess, is the wrong place to dig deeper.

So let´s switch to either email conversation - or preferrably the google group on Event-Based Components/Flow-Design: https://groups.google.com/forum/?fromgroups#!forum/event-based-components

Please repeat your questions there and we´ll discuss in how far Flow-Design differs from "true" OO - or maybe not ;-)

Some answers are gonna be very easy and to your delight. Others might need a couple of more lines.

Yes, I´ve written about all this extensively. Nevertheless sometimes it´s good to answer questions again. It´s an opportunity to reflect on the subject again.

McZ said...

Objects are good for representing data and things. If you look at the problem, the sensors, the pump and the controller are the only things attached.

Things interact via verbs. If a verb is proactive, it's a method. If it's reactive, it's a callback or event. But verbs are things in OOP, too. Every time when a pattern comes in, we basically have a verb. GasAlarm is basically an observer, SafePump is a message-filter using a contract-less decorator.

The basic criticims by the FD-folks on this OOP-style of design is the problem, that you don't see how the software works. Who is calling GasAlarm.Watch()? When is it called? Is there more than one occasion? What is MinePlant doing, and how (seriously, if it is GasAlarm.Watch + SumpPump.Drain, then it is an outputless composite, not requiring any testing)? Why is GasAlarm dependent on Alarm? How can we add fully working functionality to the software every day?

And then I have to wonder, how you unit-test all possible inputs for GasAlarm and as such its behavior (you have no data-type for the sensor-results, I admit SensorData in Ralfs proposal)?

The word "trigger" is named a few times in the original post, but there is not something like a trigger (which is basically a reactive verb, "something happened") at all, a problem, which is common between both designs. If there is an actively notifying sensor (maybe through getting IP signals?), both results fail to work, but I concede, this is OK for discussions sake.

Here is the difference: changing the FD-based approach to make ISensor asynchronous is possible without touching the outer parts of the process. It's evolvable without refactoring.

To make the OOP-design work asynchronously will require change of almost any class short of MinePlant, Alarm, PumpEngine and DigitalOutput.

Btw, because you added it in a comment: SumpPump is not SoC-compliant. It observes SumpProbe, then it seems to know exactly, what to do, i.e. switching on/off a pump engine. And on top of that, it exactly knows, that it must be a SafeEngine. Thats three concerns, IMHO.

Anonymous said...

@McZ: I leave the answer to Carlo but I'd like to just point out a thing.

FD-folks keep on claiming you can't see how the OO solution is supposed to work by just looking at Carlo's diagram.

Really? Well, after his post, I implemented the code of the diagram (as an exercise about dependency injection). And, you know, it took me... 5 minutes to write the sw looking at the diagram! And I didn't need any additional information.

I think that if you don't understand that diagram, the problem is you cannot reasonate in term of abstraction.

This is just my two cents on the subject.

Daniele

Unknown said...

@McZ

I also try to give a comment. Daniele states well when he said that you have to learn how to read that diagram. If you research a bit about past works by Carlo, you'll find that he always tries to convey dynamic information into diagrams (using colors for classes and relationships) but, at the same time, keeps diagrams at a high-level because a diagram is drawn for reasoning, not to be a mere copy of the code.

Having said that, the diagram for the pump example is an architectural one. All your question are really detailed designed (Who is calling GasAlarm.Watch()? When is it called? Is there more than one occasion? What is MinePlant doing, and how (seriously, if it is GasAlarm.Watch + SumpPump.Drain, then it is an outputless composite, not requiring any testing)? Why is GasAlarm dependent on Alarm? How can we add fully working functionality to the software every day?

I can remember an excerpt by Carlo's "UML manuale di stile" (it's in Italian) when explaining a filesystem monitor architectural diagram, he honestly makes the reader note that the class Thread in that diagram is not a complete specification (Does the constructor start a thread? what happen if you call start() two times, etc). These are detailed design questions, but we are talking about architectures.

By the way, GasAlarm.Watch() should be called by MinePlant (it's the only class that has a relationship, and given that it's a builder as explained in the post, it's in charge to build and start everything. As Carlo have said, Watch() can live in its own thread, and I don't see any needs to refactor to make thing async, in C++ for example you can use std::async from MinePlant and run Watch() in a separate thread without any refactoring.

Anyhow, you seem to give critics without having read the post. Another point Carlo has always pointed out is that you need some documentation about what is not in the diagram. A great point, because if you drew a diagram putting any possible detail inside, than you'd be better writing the code and forget the diagram, at that point it is useless. And, in the post, threading issues, the role of the MinePlant, etc. are all well explained!

Just my two cents,

Carlo Pescio said...

Well Marcus,
seems like Daniele and Fulvio are offering me the chance to play the good cop here :-). As usual, I had to split my answer in a few parts.

The basic criticims by the FD-folks on this OOP-style of design is the problem, that you don't see how the software works.
--
I'll reuse something I wrote while answering a comment on coding conventions (full thing is here: http://t.co/6YxrRjpK) [who said reuse is overrated? :-))] " I hope you'll see that this is very much like coming to steel with a wood mindset, and insisting on cutting steel along the fibers, and expressing surprise and scorn because you can't find fibers".
If you (not you you; a generic you :-) come to OOP and pretend to see the functional decomposition in a class diagram you will leave with scorn. This is not a fault of OOP, just as much not having fibers is not a fault in steel. You look at a class diagram to see allocation of responsibilites and collaboration between classes. That's its role.


Who is calling GasAlarm.Watch()? When is it called? Is there more than one occasion? What is MinePlant doing, and how (seriously, if it is GasAlarm.Watch + SumpPump.Drain, then it is an outputless composite, not requiring any testing)? Why is GasAlarm dependent on Alarm? How can we add fully working functionality to the software every day?
--
leaving aside the final question (sure, a functional decomposition guarantees that we can "add fully working functionality to the software every day", c'mon :-), there seem to be a lot of process assumptions underlying this line of thought. Like the (wrong) idea that someone will concoct a class diagram alone, in the safety of his office, then hand out that diagram alone to someone else and say "do it". Of course, if you work like that, you have a problem. But it's not a problem with OOP, it's a much bigger problem :-)).
That said, I think I've been pretty specific in my post:
- I choose to use a class diagram because I was focusing on the allocation of responsibilities
- There is nothing wrong in complementing that with a sequence or activity diagram if you feel like it.
- You also have to learn to see the dynamic through the static to be really effective at OOD.

Some of your questions, of course, are there just for the sake of argumentation. "Why is GasAlarm dependent on Alarm"? Because Alarm will turn on some audible sound, while GasAlarm knows *when* to turn that sound on and off (Soc, remember? :-). If you didn't get that from the entire post, then perhaps you skimmed through it a bit too fast. Other things I've intentionally left underspecified (mostly threading / scheduling) because they were not the focus of the post (but see also below).

Carlo Pescio said...

[part 2]

And then I have to wonder, how you unit-test all possible inputs for GasAlarm and as such its behavior (you have no data-type for the sensor-results, I admit SensorData in Ralfs proposal)?
--
Here I have to agree with Fulvio, you seem not to have read the post at all, and not even looked closely at the diagram. GasAlarm is a very simple class. Its input is a *boolean*. Do I really have to explain how to unit test all possible input? :-)


GasAlarm is basically an observer, SafePump is a message-filter using a contract-less decorator.
--
I moved this portion of your comment here because I now have a little more context to answer. You see, this kind of reasoning (reducing a domain-specific solution to a domain-independent terminology) can help you sometimes, for instance when you want to familiarize with something by abstracting unfamiliar things to something else you already know. It may also impress some people ("did he just said "contract-less decorator?" :-). It's also a double-edged (or triple-edged :-) sword, because:
- you have to get your abstraction right. For instance, GasAlarm is *not* an observer, which is pretty obvious from the diagram as well, because is not being notified through a polymorphic call. A calling B doesn't make A an observer of B. B reacting to changes by calling back A through a polymorphic interface does.
- even if you get your abstractions right, it will prevent you from thinking in the right terms. It will never allow you to look at the class diagram and see the engineering diagram, because you're removing the domain from your reasoning. That's not going to help you see the obvious answer to your questions. The diagram is talking to you, but you speak a different language.


The word "trigger" is named a few times in the original post, but there is not something like a trigger (which is basically a reactive verb, "something happened") at all, a problem, which is common between both designs. If there is an actively notifying sensor (maybe through getting IP signals?), both results fail to work, but I concede, this is OK for discussions sake.
--
Once again, you guys seem to come to this post with a very peculiar mindset, that is, that I have proposed something like the only, unique, best solution. The post was appropriately called "case 1", because (a) it was just one case among many where you can go without a controller and (b) it was just one way among many to do it. For instance, an interesting alternative would have been to base everything on events. This is certainly possible, and would fit the OO paradigm well too (and would be more in line with your idea of trigger). I even say that: "You may take different choices, still leading to a controller-free shape. For instance, everything above is based on a polling loop (easier to deal with in safety critical stuff), but an event-based approach is ok too, and may lead you to a different structure.".
It would then be interesting to compare the properties of the two (not the process; not the mindset; not to claim what it's not there; just the properties). As I told Ralf, there is no need to trash something just to propose something else. Things can be appreciated for their own specific beauty. Unfortunately, most people tend to be stuck in the "if I'm good then you must be bad" mindset. That's a total waste.

Carlo Pescio said...

[part 3]

Here is the difference: changing the FD-based approach to make ISensor asynchronous is possible without touching the outer parts of the process. It's evolvable without refactoring.
To make the OOP-design work asynchronously will require change of almost any class short of MinePlant, Alarm, PumpEngine and DigitalOutput.

--
Now, I don't know how you got this impression, but it' *completely wrong*. Let's be clear. The FD-thing, as Ralf designed it, can be easily changed into what is technically called "half-synch, half-asynch" (look, I can talk patterns too :-))), possibly without a queuing layer [because the problem itself does not really need that]. That is, you can easily make the internals of a sensor asynch, but then you need a synchronous part wereby you collect all the values from all sensors, pack them together in that SensorData thing, and send it forward in the flow. You can't turn everything into an asynchronous, fully concurrent thing, except by:
- getting rid of the sensor data thing, because at that point every sample needs its own lifetime/identity (you may recognize this as a side-effect of SensorData being the hourglass bottleneck)
- you need even more infrastructure, because now the And bar has to somehow be thread-aware, and act as a true synchronization bar.
- generally speaking, you may need to be thread-safe in places that didn't need to be so before.
So what you can honestly say is that it's easy to change the FD-thing into a HS/HA thing, nothing more. Cool. Of course, you can do the same with my design, in exactly the same way. It's even *easier*. My sensor only exposes a boolean, which you get [this is the HS part] by calling IsCritical. How does it calculate that boolean is its own business. Could easily be done in its own thread [the HA part]. You don't even need synchronization primitives if your language expose something similar to the Interlocked family of memory-access function on the Intel platform.
What you're saying, it seems, is that if I want to change everything to be a fully asynchronous, concurrent, event-driven architecture I have to change things. This is true. I don't believe you can do that with Ralf thing either, without major changes as above, including even more infrastructural support. Because let's not forget this: we're comparing a no-infrastructure design with a "bring in my magic thing and everything will work" sort of design.


Btw, because you added it in a comment: SumpPump is not SoC-compliant. It observes SumpProbe, then it seems to know exactly, what to do, i.e. switching on/off a pump engine. And on top of that, it exactly knows, that it must be a SafeEngine. Thats three concerns, IMHO.
--
If you have any familiarity at all with my works, you'll proably know that I'm not very fond of contemporary design terminology, because it's so muddy that you can tweak words to mean basically whatever you want. So, for instance, in the seminal paper by Dijkstra ("On the role of scientific thought") concerns were very high-level things, like correctness and efficiency. Apparently, in your own definition, every function has at least three concerns: reading input, doing something, creating output. And yeah, if doing something amounts to a + b * c, that's two concerns, not one. Good look with that :-). Now let's see the code for SumpPump, anyway:

void Drain()
{
if( sumpProbe.MustDrain() )
engine.On();
else
engine.Off();
}

Be serious. 4 lines, because I choose to have On and Off instead of a single function taking a boolean. Would be one line in that case.

"It observers sumpprobe" means "gets input" (of course, not by being an observer, but by calling a function).

"then it seems to know exactly, what to do, i.e. switching on/off a pump engine": YES, this is exactly the purpose of a SumpPump: to turn the engine on when it must drain!

Carlo Pescio said...

[part 4]

And on top of that, it exactly knows, that it must be a SafeEngine. No, you're getting this impression because of your cursory read of my writings. In the paragraph before the SumpPump was introduced, when the Drain responsibility was still part of the Pump Controller, I wrote: We may also want to consider the option to make the PumpController working polymorphically on PumpEngine. I’ll ignore this issue, as this is already a long post.. This is of course true for the SumpPump as well. I choose not to represent polymorphic access because that would have brought in the issue of creation, which in the end I had to discuss anyway in episode 2, so perhaps it wasn't a wise choice. But there is nothing in SumpPump that requires the engine to be a safe engine. It's just the simplified diagram that makes you think so (if you come with a criticizing mindset, that is :-)

Thankfully, some modern literature (especially in the AOP side of the world) is trying to make the term "concern" a bit more useful, something like "a set of functional or non-functional requirements". Then you'll clearly see that my SumpPump only has 1 functional concern: "turn the engine on when the sump must be drained". Also the Engine has only 1 functional concern. And the SafeEngine. And the SumpProbe. That's what I was talking about.

Since you brought this issue up, you forgot to mention that I have a system with a lot of small, composable, reusable components, that can easily scale up and down. While Ralf ends up with some reusable / independent objects in the infrastructure (where his design is more OO) and then with a larger non-modular thing (the flow diagram itself). Unfortunately, he choose not to answer my questions about where he would put the logic for the pump rack, or the fail-safe pump rack. In the flow, in the infrastructure?
As you seem to appreciate the concept of gravitational center, you should easily see that the monolithic flow diagram is creating a rather big center. It's up to you to see the consequences of that. It's also up to you to appreciate that this is exactly why people, at some point in time, decided to try alternative styles to functional decomposition. Or you can say like Ralf, "I don't see what is wrong with that". It's your choice :-)

Ralf Westphal - One Man Think Tank said...

@Carlo: Just some quick remarks:

Class diagrams show allocation of responsibilities: Sure. You draw boxes and put functionality in them. It shows what goes together and what´s taken apart.

If a language provides such boxes (classes) then that´s a valuable view.

Whether or not a language should provide such boxes is an altogether different question, though. Ask the LISP guys or the Haskell guys if they deem them crucial for software development.

Class diagrams show collaboration: I guess you mean the lines between the boxes. If so, what those lines show is just relationships. Some box somehow is tied to another box.

Those relationships have a cardinality, a direction, and maybe simple semantics (e.g. aggregation vs composition).

That´s hardly more than you can read from a crow foot diagram.

At least I have a hard time to glean from such diagrams how exactly those boxes work together towards some goal - if at all. (But then, as you said, I should first read up on OO and master the necessary skills. So I´m just a lay person in your view.)

Just because some box is connected to another does not say anything at all about how the first uses the latter. It does not tell whether it uses it, or in what way. And no box itself does tell you about how it should be used.

Think of a box X with functionality A() and B(). Can I use A() before B() or the other way round? Don´t know. If box Y is connected to X does it use A() or B() or both? Don´t know.

And if you don´t believe this, try to figure out the collaboration between classes in any decent class diagram sporting 50+ classes hanging on the wall of some team room around the world.

Dependencies: What you´re alluding to with the term "collaboration" are dependencies. If Y is connected to X and uses some of its methods, then Y is depending on X. (And I don´t care whether that´s a static or dynamic dependency. In either case at runtime Y cannot do anything reasonable without holding an instance of X.)

Then what seems to be behind your argument is the belief that such functional dependencies are necessary to build decent software. Classes need to depend on each other to form a functioning whole.

And if you don´t believe this, then certainly many developers do. The result of their design attests to that: graphs of deeply nested depending classes.

Maybe that´s where you and I differ most: I do not believe in such designs anymore.

If I should name the major cause for badly maintainable code I´d say it´s way, way, way too many such "collaborative dependencies".

Out of this belief I´m building software no longer in this way. And lo and behold it´s still working and evolving nicely. How strange, isn´t it?

Which brings me to the unanswered question: In my designs I separate the concerns "integration" and "operation". Operation is, where logic is implemented using all sorts of control structures and expressions.

Integration on the other hand is where operations are put together to form larger functional units.

The point about this is: Operations do not (!) depend on other operations. There is no functional dependency in these designs.

And although integration depends on operations (and more fine grained integration) it does so only in formal terms like a DI container depends on the classes it is supposed to instanciate on demand.

So where would I put the logic for a pump rack? In operations which are independent of each other.

And then I´d integrate these operations without them knowing to larger wholes with other operations.

I´m sure you love to work with UNIX commands. String them together using pipes. None of those commands depends on another. There is no "command diagram" showing collaborations between UNIX commands. And still UNIX looks like the most successful operating system on the planet. How come?

Maybe it´s not despite the independence of it´s functional units, but because of them?

Carlo Pescio said...

Ralf, I see there is basically no way to convey my perspective in a way that makes sense to you :-)

- I never said objects are the only paradigm. See the link above for a specific reference to FP, since you talk about lisp. You keep reading my things the wrong way.

- I never said you can take a class diagram drawn from someone else, without any other knowledge, and make perfect sense out of it. Actually I said just the opposite. You just don’t listen. I can’t give someone just one single diagram of yours and call it a day either. So what. It doesn’t make them useless. Stop trashing things man.

- you don’t really read the diagrams. You expect the symbol to tell you everything. But names (in classes, methods, roles) are the key. A() and B() means nothing. MustDrain(), On(), Off(), should tell you something. You’re looking at the wrong kind of knowledge (domain-independent things like “aggregation”). You’ll never be comfortable with class diagrams that way. Which is fine, as I told you, you have every right not to learn a skill. Not so much of trashing things you don’t completely master.

- You have your own peculiar view about OO. Even the question “Can I use A() before B() or the other way round” is sort of wrong, because in a proper object, you can send messages in any order. But ok.

- You always present a skewed perspective on things. Your lines, for instance, represent data flow. But they don’t show the fracking data, so we don’t know who is using what, producing what, etc. But we don’t use that against you. We understand the role of your lines. We understand that any single view of a software system is incomplete. We learn to read you diagrams and fill up the blank with some imagination. It’s honestly tiring to see you trying to trash things while at the same time ignoring the many similar issues you have.

- You hand-wave your things, yet pretend absolute precision from others.
“So where would I put the logic for a pump rack? In operations which are independent of each other.” That means *nothing*. The only way I can read that is: you’ll have a few functions somewhere (dunno where) and then you’ll have more icons and lines in your flow diagram to integrate those things. Which is what I told you: your diagram will get more and more complex. But ok. Oh, of course, you also have to put state somewhere, and propagate that state around, but you don’t want to mention that, right?

- Once again, you come up with this damn adversarial approach. It’s ok, actually more than ok, to have your own school of thought, to create your own tools, to promote your approach. What is terribly annoying is this need you have to do that by trashing others. I already tried (failing) to handle your objections with gloves, because I respect the effort and passion you put into your things. However, if you keep coming up like this, what do you expect, really??

- It is exactly that adversarial approach that prevents you from seeing that there has been a lot of research within the OO community to separate the wiring of things from the implementation of things, which is a similar aim to yours. Ranging from architecture description languages to simpler things like the Demeter family of tools. It would be interesting to talk about that, and whether or not separation of function and data is needed for that, and why your infrastructure is OO but the rest is not, etc. But honestly, you’re just looking for a fight. I’m looking for a deeper understanding of things. This is where our roads divide. We’ll have to agree to disagree, and you’ll have to bring your fight somewhere else. Sorry.

Ralf Westphal - One Man Think Tank said...

You win. Congratulations.

Farewell.