tag:blogger.com,1999:blog-139677132024-03-14T18:41:44.040+01:00Carlo Pesciobrain-driven designCarlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.comBlogger56125tag:blogger.com,1999:blog-13967713.post-80137074114584964532013-03-07T19:33:00.001+01:002022-04-23T09:56:18.955+02:00Game Over<br />
<div class="MsoNormal">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZx6qAvsHsQtXOsl7uKxmcoUG11aIFkAT9ECtX-_676huah4kkIPV7a-XDWlM0wil48CB8HadrpDOoT3CQ0cGG9QnMB-TzEfoU5mhzlkeUPzbi-wdWF7R83816K01V93ksLbG2zQ/s1600/GameOver.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZx6qAvsHsQtXOsl7uKxmcoUG11aIFkAT9ECtX-_676huah4kkIPV7a-XDWlM0wil48CB8HadrpDOoT3CQ0cGG9QnMB-TzEfoU5mhzlkeUPzbi-wdWF7R83816K01V93ksLbG2zQ/s200/GameOver.png" width="200" /></a><span face="Arial, sans-serif" style="font-size: 12pt; line-height: 115%;"></span></div>
<div class="MsoNormal">
<span face="Arial, Helvetica, sans-serif">Every once in a
while, I take my head out the clouds and do a reality check. Today is one of
those days.</span></div>
<div class="MsoNormal">
<span face=""Arial","sans-serif"" lang="EN-US" style="font-size: 12pt; line-height: 115%; mso-ansi-language: EN-US;"><br /></span></div>
<div class="MsoNormal">
<span face="Arial, sans-serif" style="font-size: 12pt; line-height: 115%;">This blog is over for
the time being. Take care guys.</span></div>
<div class="MsoNormal">
<span face=""Arial","sans-serif"" lang="EN-US" style="font-size: 12pt; line-height: 115%; mso-ansi-language: EN-US;"><br /></span>
<br />
<div style="text-align: justify;">
<b><span face=""Arial","sans-serif"" lang="EN-US" style="font-size: 12pt; line-height: 115%; mso-ansi-language: EN-US;">Update (August 2014)</span><span face="Arial, Helvetica, sans-serif"><span lang="EN-US" style="font-size: 12pt; line-height: 115%; mso-ansi-language: EN-US;"> </span></span></b></div>
<div style="text-align: justify;">
<span face="Arial, Helvetica, sans-serif"><span lang="EN-US" style="font-size: 12pt; line-height: 115%; mso-ansi-language: EN-US;">i</span><span style="text-align: justify;">f you liked my blog, you may want to take a look at my </span><a href="http://www.eptacom.net/aspectroid" style="text-align: justify;" target="_blank">aspectroid</a><span style="text-align: justify;"> project, where I'm exploring aspect-oriented design, outside the limits of blog posts, with full source code, etc. You may also want to </span><a href="https://twitter.com/CarloPescio" style="text-align: justify;" target="_blank">follow me on twitter</a><span style="text-align: justify;">.</span></span></div>
</div>
Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.comtag:blogger.com,1999:blog-13967713.post-26461885940785244602012-12-27T16:12:00.000+01:002012-12-28T09:02:07.396+01:00Ask not what an object is, but...<br />
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span style=""><span lang="en-US">I
can barely remember the days when objects were seen like a new,
shiny, promising technology. Today, objects are </span></span><span style=""><span lang="en-US">often
positioned between mainstream and </span></span><span style=""><span lang="en-US"><i>retro</i></span></span><span style=""><span lang="en-US">,
while the functional paradigm is enjoying an interesting renaissance. Still, </span></span></span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">in the </span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">last
few months I stumbled on a couple of blog posts asking the
quintessential question, reminiscent of those dark
old days: </span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><i>“what
is an object?”</i></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">The
<a href="http://www.johndcook.com/blog/2012/09/11/what-is-an-object/" target="_blank">most recent</a> (September 2012) </span><span style=""><span lang="en-US">is
</span></span><span style=""><span lang="en-US">mostly
a pointer to a stripped-down definition provided by Brian Marick:
“</span></span><span style=""><span lang="en-US"><i>It’s
a clump of name->value mappings, some functions that take such
clumps as their first </i></span></span><span style=""><span lang="en-US"><i>arguments,
and a dispatch function that decides which function the programmer
meant to call</i></span></span><span style=""><span lang="en-US">”.
Well, honestly, this is more about a specific implementation of
objects, with a rather poor fit, for instance, with the C++
implementation. It makes sense when you’re describing a way to
implement objects (which is what Marick did) but it’s not a
particularly far-reaching definition.</span></span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">The
<a href="http://wcook.blogspot.it/2012/07/proposal-for-simplified-modern.html" target="_blank">slightly older one</a> </span><span lang="en-US">(July
2012) </span><span style=""><span lang="en-US">is
much more ambitious and comprehensive. Cook aims to provide a
“modern” definition of objects, unrestricted by specific
languages and implementations. It’s an interesting post indeed, </span></span><span style=""><span lang="en-US">and
I suggest that you take some time reading it, but in the end, it’s
still very much about the </span></span><span style=""><span lang="en-US"><i>mechanics</i></span></span><span style=""><span lang="en-US">
of objects </span></span><span style=""><span lang="en-US"><i>("</i></span></span><span style=""><span lang="en-US"><i>An
</i></span></span><span style=""><span lang="en-US"><i>object
</i></span></span><span style=""><span lang="en-US"><i>is
a first-class, dynamically dispatched behavior"</i></span></span><span style=""><span lang="en-US">).</span></span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Although
</span><span lang="en-US">it
may seem ok from a language design perspective, defining objects
through their mechanics leaves a </span><span lang="en-US"><i>vacuum</i></span><span lang="en-US">
in our collective knowledge: how do we design a </span><span lang="en-US"><i>proper</i></span><span lang="en-US">
object-oriented system? </span>
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"></span></div>
<a name='more'></a><span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span><br />
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">On
one end of the spectrum, we find people </span><span lang="en-US">quibbling
about increasingly irrelevant details about what an object is or is
not (just see some of the comments on Cook's post :-), while on the other
end, we find programmers abusing objects all the time, building
“object oriented software” based on manager classes and get/set
methods, with just a few principles as their beacon toward the
promised land.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Still,
i</span><span lang="en-US">t's
not my intention to question those definitions of objects. In a more
radical way, I say we should redefine the <b>question</b>, not the answer.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>[Optional]
A little history…</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">(</span><span lang="en-US">Feel
free to skip this and jump to the next paragraph, if you don’t care
much for the past. However, Alan Kay will promptly tell you that
without history, it's just pop culture.)</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Do
we actually need to dig deeper into the definition of object
orientation? Isn’t object oriented mainstream now? Is</span><span lang="en-US">n't
it true, as Cook says, that </span><span lang="en-US"><i>“The
fundamental nature of objects is well-known and understood by the OO
community”</i></span><span lang="en-US">?
</span>
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">I
tend to disagree. </span><span lang="en-US"><b>Bad</b></span><span lang="en-US">
object orientation is mainstream, but good object orientation is
rare. Open up the Android source code (just to mention a relatively
recent large-scale development) and you’ll see that 5000 LOC
classes are the norm, more than the exception. </span>
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">So,
m</span><span lang="en-US">aybe
the question is not really the best one. Maybe we shouldn’t ask
what objects are, but what object </span><span lang="en-US"><b>orientation</b></span><span lang="en-US">
is about. Surprisingly, or maybe not, most literature simply equates
object orientation with the adoption of an object oriented language,
adding very little to the conversation. There also a few paper
defining OO as "modeling the real world", which leaves a
lot to be desired, but overall, the literature is mostly focusing on
mechanisms (inheritance, polymorphism, classes). Sure, we have the
usual blurb on principles, but that's more of a band-aid and can only
help so much.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Facing
dissatisfaction with </span><span lang="en-US">contemporary
literature, sometimes I turn to history for inspiration. For
instance, we may just look up Alan Kay for the ultimate definition of
what object orientation is about, right? </span>
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Wrong
:-). Alan has always been elusive about defining OO. Feel free to
peruse this <a href="http://c2.com/cgi/wiki?AlanKaysDefinitionOfObjectOriented" target="_blank">comprehensive wiki page</a></span></span><span style="font-family: Arial, Helvetica, sans-serif; ">,
but don't feel ashamed if you leave with a sense of inconclusiveness. </span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">We
could also look up an <a href="http://www.purl.org/stefan_ram/pub/doc_kay_oop_en" target="_blank">interesting email exchange</a> between Stefan Ram
and Alan</span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">,
ending with "</span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><i>OOP
to me means only messaging, local retention and protection and hiding
of state-process, and extreme late-binding of all things</i></span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">",
which is better than nothing, but yeah, not really a great guidance
when you're designing a system.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Well,
maybe it's just old stuff. However, <a href="http://computinged.wordpress.com/2010/09/15/alan-kay-on-motis-objects-ever-cacm-article/" target="_blank">e</a></span><span lang="en-US"><a href="http://computinged.wordpress.com/2010/09/15/alan-kay-on-motis-objects-ever-cacm-article/" target="_blank">ven recently</a>, answering a (ridicolous) critique of OOP incredibly
featured on Communications of ACM, Alan played the "misunderstood"
card and said “</span><span lang="en-US"><i>the
remedy is to consign the current wide-spread meanings of
“object-oriented” to the rubbish heap of still taught bad ideas,
and to make up a new term for what I and my colleagues did</i></span><span lang="en-US">”.
Ouch. </span></span></div>
<div style=" margin-bottom: 0cm;">
<span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><br /></span></div>
<div style=" margin-bottom: 0cm;">
<span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">Admittedly,
</span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">he
gets around some concepts like messaging again, but honestly, we
can't blame people for going with mechanistic definitions if we
cannot provide a good alternative.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Well,
if Alan won’t cut it, perhaps his nemesis (Bjarne </span><span lang="en-US">Stroustrup)
will. After all, Bjarne wrote a promising <a href="http://cheetah.cs.virginia.edu/~weimer/2006-655/reading/stroustrup-oo.pdf" target="_blank">What is ‘‘Object-OrientedProgramming?’’</a> paper, revised in 1991. </span></span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">Still,
the paper is largely language-centric, and the general idea </span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">is
to equate ‘‘support for data abstraction’’ with the ability
to define and use new types and equate ‘‘support for
object-oriented programming’’ with the ability to express type
hierarchies. Not what I was looking for.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Well,
maybe even “what is object orientation” is simply </span><span lang="en-US"><b>not
the best question</b></span><span lang="en-US">.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>What
if…</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">I
guess you’re all familiar with this kind of </span><span lang="en-US">structure:</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="" style=" margin-bottom: 0cm;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-IzookU2r89x63dl3zCJfgRtdHm1vS2oJPNB6p5_8F1VAa1LGh9ZLQu_DNqOHCe9EIcs-wNxOu6XUn4-wSGbLrb6sTeH430YN5NdZJfcMlzPrSVDQFljggYW8kREz7YZgQdpIig/s1600/Terminal+Rooftop.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-IzookU2r89x63dl3zCJfgRtdHm1vS2oJPNB6p5_8F1VAa1LGh9ZLQu_DNqOHCe9EIcs-wNxOu6XUn4-wSGbLrb6sTeH430YN5NdZJfcMlzPrSVDQFljggYW8kREz7YZgQdpIig/s400/Terminal+Rooftop.jpg" width="400" /></a></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">It’s
that saddle-shaped thing, made of cables and membranes, often used to
cover stadiums and other large areas. The technical term is </span><span lang="en-US"><i>tensile
structure</i></span><span lang="en-US">.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Now,
if you’re in computing, you can just call that a new construction
paradigm, show a few examples of saddle-shaped things, then move to
the much more interesting field of cables and membranes, and spend
all your time debating about PTFE-coated fiberglass vs. PVC-coated
polyester and how only a complete moron would ever consider creating
a tensile structure in anything else than an ETFE film.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">But
this is not computing, so we can find a much better definition of
tensile structure. You just have to <a href="http://en.wikipedia.org/wiki/Tensile_structure" target="_blank">fire up Wikipedia</a> and step into
an entire new world. </span></span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">The
page begins with a precise definition of tensile structure, based on
forces and reactions to forces: </span></div>
<div style=" margin-bottom: 0cm;">
<span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><br /></span></div>
<div style=" margin-bottom: 0cm;">
<span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">“</span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><i>A
tensile structure is a construction of elements carrying only tension
and no compression or bending</i></span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">”. </span></div>
<div style=" margin-bottom: 0cm;">
<span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><br /></span></div>
<div style=" margin-bottom: 0cm;">
<span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">There is an immediate distinction with </span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; "><i>tensegrity</i></span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">,
which carries compression as well. </span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">Only
after a cer</span><span lang="en-US" style="font-family: Arial, Helvetica, sans-serif; ">tain
while, you'll find a discussion of materials, like membranes and
cables, and what is more appropriate when you want to build a tensile
structure.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">A
precise definition like that removes any possible source of
confusion. Being tensile is not about the saddle shape: </span><span lang="en-US">you
can't just mimic the shape of a tensile structure and claim that it
is indeed a tensile structure. Besides, tensile structures don't have
to be saddle-shaped.</span></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">It's
also not </span><span lang="en-US">about
the materials. It’s not strictly necessary to use membranes and
cables; it's just that, given our current understanding of materials,
those seem like a very good fit.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Interestingly,
the definition is short yet powerful. It provides guidance when you
design structures and guidance when you choose (or even better,
</span><span lang="en-US"><i>design</i></span><span lang="en-US">)
materials.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>The
real question</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">So
maybe we should</span><span lang="en-US">
not ask "what is an object" and then define object
orientation as "</span><span lang="en-US"><i>the
dynamic creation and use of objects</i></span><span lang="en-US">"
(the quote is a bit out of context here – Cook was talking about
languages – but you get the idea). Neither we should ask ourselves
“what is object oriented programming” and define it in terms of
language features. That would be very much like talking about cables
and membranes and then say that if you build it with cables and
membranes then it's a tensile structure. It appeals to people who
want to reason at the material level (language and constructs), but
it's neither precise nor really useful for the software architect. </span>
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">So
the real question should be: </span><span lang="en-US"><b>what
is an object oriented structure</b></span><span lang="en-US">?
Once we know the answer, we can freely discuss how different
materials (languages, constructs, etc.) can help us building an OOS,
and how they help, and how they don't. </span>
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Ideally,
the definition of OOS would also bring </span><span lang="en-US">some
clarity on what we can expect out of an OOS and what is not to be
expected. For instance, we do not expect tensile structures to
sustain heavy snow load. </span>
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">We
could also try to define the nature of a </span><span lang="en-US"><b>functional
structure</b></span><span lang="en-US">,
and compare the two without (too much) advocacy. Same thing for the
</span><span lang="en-US"><b>logic
paradigm</b></span><span lang="en-US">,
still waiting for its own little renaissance. Or for Aspect-Oriented
structures.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">The
problem, of course, is that </span><span lang="en-US"><b>we
don’t know how to do that</b></span><span lang="en-US">.
We don’t have a theory of forces. We don’t have clear notions
about how software materials react to forces. So we end up building
an entire industry based on advocacy, more than on science. It
doesn’t have to be like that, of course. Seneca used to say “</span><span lang="en-US"><i>It
is not because things are difficult that we do not dare, but because
we do not dare, things are difficult</i></span><span lang="en-US">”.
So let’s dare :-)</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Interlude
– the Three Spaces</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">I’ll
assume that most of you guys are </span><span lang="en-US"><b>not</b></span><span lang="en-US">
familiar with my work on the Physics of Software, so I’ll recap a
simple notion here, that will be useful in understanding what
follows.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Software
is shaped by <a href="http://www.carlopescio.com/2010/11/design-structure-and-decisions.html" target="_blank">decisions</a>.
Decisions about the function (what the software will do) and about
the form (how we are going to do that). Some decisions are taken
after careful consideration, some by routine, and others by feedback
(from the users, or from the material). Decisions have this nasty
habit of being </span><span lang="en-US"><b>unstable</b></span><span lang="en-US">.
Features will be added, removed, updated. We’ll change the form by
refactoring. Etc. At any given point in time, our software is just
the embodiment of a set of decisions. I consider that as a single
point into a multi-dimensional </span><span lang="en-US"><b>decision
space</b></span><span lang="en-US">.
When decisions change, we move to another point into the decision
space.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Software
is encoded in artifacts. It doesn’t really matter if you write
procedural, functional, or OO code. It doesn’t really matter if you
use models or code. We cannot manipulate information; we can only
manipulate a </span><span lang="en-US"><b>representation</b></span><span lang="en-US">
of information (artifact). Languages and paradigms define the
structure of their artifacts, and provide </span><span lang="en-US"><b>modular
units</b></span><span lang="en-US">.
A function is a modular unit in the functional paradigm. A class is a
unit in the OO paradigm. For various reasons, I tend to call those
units “centers”, which is a more encompassing concept in the
physics of software. </span>
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">In
the end, a point into the decision space is encoded into a set of
artifacts in the </span><span lang="en-US"><b>artifact
space</b></span><span lang="en-US">.
Those artifacts define a set of centers.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">As
we know, the same run-time behavior can be obtained by widely
different organizations of artifacts and centers. In the end,
however, the knowledge encoded in those artifacts will be </span><span lang="en-US"><i>executed</i></span><span lang="en-US">.
Things will happen, on real hardware, not in some theoretical
semantic space. Things like cache lines and multicore CPU etc. will
come to life and play their role. I call this the </span><span lang="en-US"><b>run-time
space</b></span><span lang="en-US">.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Now,
to cut it short, a change in the decision space will always be about
the run-time or the artifact space, but it will always be carried out
through a change in the artifact space. I want that new feature: it’s
about the run-time space, but I need to add code in the artifact
space. I want to refactor that switch-case into a polymorphic class
hierarchy. It’s about the artifact space – the run-time, visible
behavior won’t change a bit – and it’s also carried out in the
artifact space.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">In
the physics of software, we acknowledge that materials (the
artifacts) are shaped by decisions. Therefore, those decisions are
the forces acting upon the materials (following D’Arcy – it’s a
long story :-).</span><span lang="en-US">
Those forces will materialize as <a href="http://www.carlopescio.com/2011/01/notes-on-software-design-chapter-13-on.html" target="_blank">changes</a>. </span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US"><b>So,
w</b></span><span lang="en-US"><b>hat
is an Object Oriented Structure?</b></span></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Ok,
this is my best shot as of December, 2012. Maybe I’ll come up with
a better definition at some point. Or maybe you will. </span><span lang="en-US">I'm
totally interested in improving it, but not really interested in a
fight over it :-), so feel free to chime in, but please be kind. Note
that I'm not trying to say that OO is "good" or anything
like that. I'm just trying to define what an OOS is, from the
perspective of forces and reactions to forces.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Given
a Change </span><span lang="en-US">H
(in the decision space), H will occur with some probability P(H).
Generally speaking, P(H) is unknown, but in many cases we can reasonably classify P(H) in discrete sets, e.g. “unlikely”, “possible”, “very likely”. </span>
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">H</span><span lang="en-US">
will be about the Run/Time space or about the Artifact space, but
will always entail some work (a change again) in the Artifact space.
Let’s say that H will require a change (creation, deletion, or
update) in artifacts A</span><sub><span lang="en-US">1</span></sub><span lang="en-US">...A</span><sub><span lang="en-US">n</span></sub><span lang="en-US">.
</span>
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Now,
given a set of (meaningful) changes H</span><sub><span lang="en-US">1</span></sub><span lang="en-US">…H</span><sub><span lang="en-US">k</span></sub><span lang="en-US">,
occurring over time, </span><span lang="en-US"><b>an
OOS is one that minimizes the total size of the artifacts you have to
update, encouraging the creation or deletion of entire artifacts
instead</b></span><span lang="en-US">.
Slightly more formally, within the limits of HTML text, an OOS will
try to minimize:</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Sum(
j = 1…k) { </span><span lang="en-US">P(H</span><sub><span lang="en-US">j</span></sub><span lang="en-US">)
* SUM( i = 1…n ) { Size(A</span><sub><span lang="en-US">i</span></sub><span lang="en-US">)
| A</span><sub><span lang="en-US">i</span></sub><span lang="en-US">
is updated in H</span><sub><span lang="en-US">j</span></sub><span lang="en-US">
} }</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">A
couple of notes:</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<ul>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Cheating
is not allowed. If you delete a module and add it back with some
changes, it counts as an update, not as 1 delete + 1 create.</span></div>
</li>
</ul>
<ul>
<li><div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Note
the P(H</span><sub><span lang="en-US">j</span></sub><span lang="en-US">)
factor. It accounts for the fact that it’s not always possible to
balance all forces, and that a good OOS accounts for the most likely
changes.</span></span></div>
</li>
</ul>
<div lang="en-US" style="margin-bottom: 0cm; margin-left: 1.27cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Does
it work?</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">I
understand that this definition may not look like OO at all. Where is
polymorphism? Where is encapsulation? Where is inheritance? Well,
that's the material we use – the cables and membranes. A good
definition of OOS </span><span lang="en-US"><b>must</b></span><span lang="en-US">
be beyond that.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Let’s
use, once more, the familiar Shape example to explain why the
definition above is aligned with Object Oriented Structures</span><span lang="en-US">,
while other structures tend not to comply with it. Say that we have a
set of shapes: Circle, Polygon, perhaps some specialized polygons
like Square and Triangle, etc. We want the ability to create one of
those shapes, with a given position and size. We also want the
ability to calculate the bounding box of that shape. Given that, we
also want to get a bunch of those shapes, and </span><span lang="en-US"><i>distribute
them evenly in space</i></span><span lang="en-US">,
like many programs do. Of course, that's just the beginning, we know
that more features (and more shapes) are coming; we just don't know
which features and which shapes.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">What
are our basic needs? Well, any shape is defined by some data, so we
need to put those data somewhere. We also need to implement the
bounding box logic. We also need to implement the </span><span lang="en-US"><i>"distribute
evenly"</i></span><span lang="en-US">
logic. That requires the ability to </span><span lang="en-US"><i>move
shapes</i></span><span lang="en-US">
around, of course. We can shuffle around that logic in different
forms, of course. That's a design decision.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">The
decision space is usually very large, even for a small program like
this. However, a few macro-decisions can be easily identified:</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<ul>
<ol>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">We
may choose a uniform representation for all the shapes, or let
every shape have its own representation (if needed).</span></div>
</li>
</ol>
</ul>
<ul>
<ol start="2">
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">We
may have a separate implementation of the bounding box logic for
each shape, or just a single center / artifact. That single center
/ artifact may or may not have to know about every shape type.</span></div>
</li>
</ol>
</ul>
<ul>
<ol start="3">
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">We
may have to know about every shape type inside the "distribute
evenly" logic. Or not.</span></div>
</li>
</ol>
</ul>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Let's
go through this list with a little more attention to details.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">1)
</span><span style="font-family: Arial, Helvetica, sans-serif; ">One
might trivially say that every shape is just a list of points. Oh,
the magic of lists. Except that, well, we don't want to turn the
circle into a list of points, and maybe not even the regular
polygons. Of course, we can still shoehorn a circle in a list of
points. Put the center as the first point and the radius as the
second point; or maybe any given point on the circumference will do.
Or use 3 points to define the circle. But other things are easier
when you store the center and radius (think area).</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">That's
sort of misleading though. It's uniform representation with
non-uniform behavior. For instance, when you want to move a circle
you only move the center. An irregular polygon requires that you
shift all the points. Also, what if we want to add a shape that is
not easily defined by a list of points, like a fractal curve? You can
choose a uniform representation if you want, but then you also need
to store the shape type somewhere, and have some logic dealing with
specific shape types, usually by switching on the shape type.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">2)
Calculating the bounding box for a circle and for an irregular
polygon requires a different logic. We can have one big function
switching on types and then using the uniform representation to carry
out non-uniform behavior, or we can have a separate logic for each
shape type. Then it all depends on how we want to invoke that logic:
by switching (again) on shape type, or in some other way.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">3)
What is a reasonable implementation for the "distribute evenly"
thing? Well, to be precise, it's usually a "distribute
horizontally" or "distribute vertically" thing. Say
that it's horizontally. A reasonable abstract algorithm is:</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">-
take the bounding box of every shape involved</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">-
add the width of each box together so you know the required space</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">-
take the bounding box of the bounding boxes; take its width; that's
the available space</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">-
calculate free space as: available - required</span></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">-
divide by number of shapes - 1; that's the spacing.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">-
start with the leftmost and space shapes accordingly, by moving their
left side</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Most
of the logic is in terms of bounding boxes; basically, the only
things dependent on shape type are the calculation of the bounding
boxes themselves, and moving shapes by </span><span style="font-family: Arial, Helvetica, sans-serif; ">the left side </span><span style="font-family: Arial, Helvetica, sans-serif; ">(different for
circle and irregular polygon, again). Once more, we have choices: add
a switch inside the distribute function (ugly), call a move function
that is switching inside, or do it some other way that does not
require a switch.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Ok,
we're set. Let's see if the definition of OOS above will tend to
favor an OO implementation over the alternatives.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Create,
don't Update</b></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">A
very reasonable, likely Change is that we want to add a new shape
type. We may not know the probability P(H), but we can say it's
likely. If we want to do that by adding a new artifact, without
changing existing ones, any kind of switching is forbidden. That
includes the common cases of pattern matching, including case classes
and the like, as they normally require updating the list of matching
(it doesn't have to be so – more about this later).</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">So,
the definition above basically forbids the switch/based (or
match/based) implementation. There is relatively little choice: you
need some form of </span><span lang="en-US"><b>late
binding</b></span><span lang="en-US">.
You want to loop over shapes and ask for a bounding box. You can't
look inside because that requires a switching over shape type, to
make sense of the pseudo-uniform representation. You want to ask for
the bounding box (a service, not data, guess what :-), but you don't
(can't) know the function that will be called. Of course, the obvious
OO way to do that (a Shape interface, a Circle class, a
RegularPoligon class, etc) is a very reasonable way to comply with
this.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Localized
change</b></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">The
prophets of public data and uniform representation won't tell you,
but as you add new features, you may want to improve and adapt the internal representation. For instance, calculating the bounding box for a
circle is simple math and you may want to do that on the fly; for an
irregular polygon, you have to go through all the points, and you may
want to cache that thing. It's a Change you may want to take at some
point. You can't do that by merely adding a new artifact: you have to
go in and change an existing one. Or many.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">How
do you estimate the impact? A proper OOS requires that this change
will impact as little code as possible (see the artifact size in the
formula). Well, if you:</span></div>
<ul>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">expose
a BoundingBox service, not your internal data</span></div>
</li>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">actually
hide your data with some modular protection notion</span></div>
</li>
</ul>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">you're
pretty sure that there will be only one thing to change: the
IrregularPolygon abstraction, and inside that, the BoundingBox
function only. So you need some form of encapsulation, the modular
hiding of data behind stable services. Once again, the obvious
class-based design is a good way to comply. </span>
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">As
you know, most OO languages have a notion of public and private, and
sometimes protected. You can easily see those notions as ways to
limit the </span><span lang="en-US"><i>potential
impact</i></span><span lang="en-US">
of a change. Theoretically, if a Change requires an Update of a
Private part, that change will be local, which is ok with the idea
that an OOS tend to minimize the Update, not to eliminate the Update.
Of course, for that to work, you really need to create an OOS. If you
expose the private part through a Get/Set, the Update won’t be
local anymore, reflecting the fact that the structure is not really
OO.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Occam's
Razor</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Do
we need an abstraction for the pentagon, one for hexagon, one for
every regular polygon? Or is it better to have only one? Once again,
look at the reality of development, not at some philosophical idea.
The reality of development is that you may not have got the best Shape interface from the very beginning, and
that you'll change that. Maybe you didn't think of a "move left
side" service in the beginning. Now you have to go in and add it
to your shapes. That's </span><span lang="en-US"><b>a
bad change from an OOS perspective</b></span><span lang="en-US">:
you change many artifacts (more on this later). Still, the definition
is telling you to avoid unnecessary classification for the sake of
it. It's asking you to keep the number of artifacts (the cumulative
size, actually) to a bare minimum. It's actually suggesting that in
most cases, reusing a common implementation is good, because then you
have only one place to change. Some kind of implementation
inheritance will help balance this need with the need to specialize
behavior (in some cases).</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>You
don't need to "encapsulate" the Bounding Box (too much)</b></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">A
large part of the "distribute evenly" logic is based on
bounding boxes. A bounding box is an object and yeah, sure, it's nice
to expose a few methods like height() and width() but, on the other
hand, it's no big deal if you expose (perhaps read-only) the
coordinates of top / left / right / bottom corners. Sure, strictly
speaking, that would be a "violation of encapsulation",
which is the kind of primitive and limited reasoning that principles
tend to inspire. Once you look at the definition above, you'll
quickly understand that the probability of change inside the bounding
box, and especially the probability of that change having an impact
on other artifacts, is pretty much zero. It's a stable abstraction,
whereas Shape is an unstable notion (for a few hints on instability
types, look up my previous post "<a href="http://www.carlopescio.com/2012/10/dont-do-it.html" target="_blank">Don't do it</a>"). So,
let's be honest: a few methods in BoundingBox won't hurt, but
"exposing data" won't hurt much either.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>You
need to align the OOS with the force field</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">A
very reasonable request would be to have a vertical version of the
"distribute evenly" feature. The algorithm is basically the
same, except now we have to move shapes by the top, not by the left
side. That's a breaking change, and it's not localized. It's an
high-impact change, because th</span><span lang="en-US">e
obvious OOS (Shape interface + concrete classes) is well-aligned with
the need to extend shape types, but not well aligned with the need to
extend shape </span><span lang="en-US"><i>methods</i></span><span lang="en-US">.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">We
can look at this under different perspectives:</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<ol>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">We
can’t protect ourselves from every possible orthogonal change in
the multi-dimensional decision space. Not with “regular” OO
anyway (and no, not even with regular FP). So we drop in the
probability, and create a structure which is well-aligned with the
most likely changes.</span></div>
</li>
</ol>
<ol start="2">
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">We
need better materials. Open classes, selective protection, etc.
Understanding forces will guide us better. For instance, Scala-like
case classes are a bad idea if you’re aiming for an OOS (because
of the switch/case, of course, not because of the pattern matching
itself). Find the real good idea (it's rather simple).</span></div>
</li>
</ol>
<ol start="3">
<li><div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">OOP
is a scam. It doesn’t work. Let’s move </span><span lang="en-US"><i>en
masse</i></span><span lang="en-US">
to FP. Wait, what is a Functional Structure again?</span></span></div>
</li>
</ol>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Note
that it would be wrong to say that OOP can’t sustain instability on
the functional side. The well-known Command pattern, for instance, is
an OOS dealing specifically with a set of functions that is known to
be unstable (high probability of accretive change). OOP can’t deal
well with </span><span lang="en-US"><i>orthogonal</i></span><span lang="en-US">
changes, unless we improve the available materials. AOP, for
instance, was a definite step forward on that area.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">It
is worth considering that, in many cases, instability in some areas
is </span><span lang="en-US"><b>temporary</b></span><span lang="en-US">.
Very much like atoms in an annealing process, our design is often unstable
in the beginning, as we try to understand the "right"
partitioning and the "right" responsibilities. In many
cases, instability on methods does not go on forever, as we can find
a kernel of stable methods (I often say that the kernel tend to
"saturate"). After that, we can build algorithms outside
the kernel, like the "distribute evenly" thing. By the way:
where do we put those algorithm in a proper OOS?</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US"><b>Mainstream
</b></span><span lang="en-US"><b>OOP
is a good fit, but not the only possible one</b></span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Whereas
the discussion on "what is an object" tends to put strong
boundaries on our materials, focusing on forces and reactions to
forces is really liberating. The following mechanisms tend to be
useful to build an OOS:</span></div>
<ul>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">selective
protection; not necessarily the "closed" kind provided by
classes</span></div>
</li>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">late
binding; but genericity is useful as well, in some circumstances</span></div>
</li>
<li><div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">code
reuse mechanisms, like inheritance, prototypes, mix-ins, delegation
(</span><span lang="en-US">ideally,
the kind of automatic delegation that does not require manual
construction of proxies), and higher-order functions.</span></span></div>
</li>
<li><div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">generally
speaking, indirection mechanisms (I'll get back to this in a future
post), including, of course, closures and lambdas.</span></span></div>
</li>
<li><div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">add
your stuff here : )</span></span></div>
</li>
</ul>
<div lang="en-US" style=" margin-bottom: 0cm; margin-left: 0.64cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">What
hurts: whatever makes you build an <a href="http://www.carlopescio.com/2012/05/notes-on-software-design-chapter-16.html" target="_blank">hourglass shape</a>.
Switch</span><span lang="en-US">/case,
the wrong approach to pattern matching (see above), etc. Whatever
leads to duplication and non-modular decisions. Early-binding of
things (for instance, having to specify a class name in code to get
an instance). Etc.</span></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US"><br /></span></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">It also goes without saying that the adoption of a mainstream OO language gives no guarantee that we're going to build an OOS. The state of practice largely proves that.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>But…
why should we do this?</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">Sorry,
this is already a long post. Understanding why an OOS as defined above is,
let's say… </span><span lang="en-US"><i>interesting</i></span><span lang="en-US">,
is left as the dreaded exercise for the reader.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Beyond
principles</b></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">You
can easily see how many principles follow directly from this
definition:</span></div>
<ul>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Demeter's
law (trying to minimize the impact of some updates)</span></div>
</li>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Information
Hiding (same as above)</span></div>
</li>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Don't
Repeat Yourself (for unstable code, I should say)</span></div>
</li>
<li><div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Etc
etc.</span></div>
</li>
</ul>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">As
I said many times now, I think it's time to move beyond principle and
toward a better understanding of forces and properties in software
materials.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US"><b>Is
this </b></span><span lang="en-US"><b>really
the Smalltalk way?</b></span></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">No,
absolutely. Smalltalk aimed at something bigger than this. In a
sense, Smalltalk tried to apply the same rule to the language itself,
in a sort of reflective way. To really understand what Alan meant by
</span><span lang="en-US"><i>"extreme
late-binding of all things"</i></span><span lang="en-US">,
you need to realize that in Smalltalk, "if" (ifTrue
actually) <a href="http://www.gnu.org/software/smalltalk/manual/html_node/Conditions.html" target="_blank">is a method</a>, or a message in Smalltalk speaking.</span></span></div>
<div style=" margin-bottom: 0cm;">
<br /></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">The
message is sent to a Boolean object, and a code block is passed as a
parameter. Yeap. A code block. Talk about extreme late binding of
things. The meaning of that code block is up to the ifTrue
method. It's not a compile-time decision. This is way beyond what we
usually do with most mainstream OO languages, although there has been
a recent (late) move toward similar concepts.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">Building
a language this way means we can change the language itself by adding
things, instead of changing a compiler (or interpreter).
Paradoxically, languages that proclaimed to be "pure OO"
like Java were actually a step backward from this, even when compared to
C++. Funny, in its own way.</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>A
definition based on entanglement</b></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">For
those of you more familiar with my work on the Physics of Software,
here is a bonus :-) definition based on the notion of entanglement:</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><i>An
Object Oriented Structure is a construction of centers carrying
mostly */D and */C entanglement between the decision space and the
artifact space. When */U entanglement is carried, the size of the
involved artifacts is probabilistically minimized.</i></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">There
are also a few corollaries that follow directly from the definition
of OOS, but that might be worth stating:</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">An
</span><span lang="en-US">OOS
reacts to growth in the problem space with Creation in the artifact
space, not with Update in the artifact space.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">An
</span><span lang="en-US">OOS
reacts to enumerative variation in the problem space by creating an
isomorphic set of centers (often, classes) in the artifact space. This is strongly related
to the <a href="http://www.carlopescio.com/2011/02/notes-on-software-design-chapter-14.html" target="_blank">enumeration law</a>.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">An
OOS does not force a common structure when common behavior suffices.
Optimal representation can be pursued with local changes within each
center. An OOS hides the chosen representation inside the centers
whenever the representation is potentially unstable.</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">An
</span><span lang="en-US">OOS
allows the targets to make sense of the request, freeing the source
from knowledge about the target types. Hence, an OOS hides the nature
of the center outside the center itself (it's a Shape, not a
Polygon).</span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">An
OOS will react to compositional behavior by forming compositional
structures, not by creating longer functions (see some patterns:
composite, chain of responsibility, decorator, etc).</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;">There
would be more to say here, including some ideas on an economic theory
of software design, but this is a blog post, not a book :-).</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Surprise!</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">This
post is actually “Notes on Software Design, chapter 17”. Ok,
you don’t have to go back and read the other 17 chapters (there
is a chapter 0 as well). </span><span lang="en-US">But
you may want to help the world by sharing this post :-)</span></span></div>
<div style=" margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="line-height: 16px;">If you liked this post, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me</a> on twitter!</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="line-height: 16px;"><br /></span></span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<div lang="en-US" style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Acknowledgment</b></span></div>
<div style=" margin-bottom: 0cm;">
<span style="font-family: Arial, Helvetica, sans-serif;"><span lang="en-US">The
tensile structure pic</span><span lang="en-US">ture
is </span><span lang="en-US"><span style="background: #ffffff;">provided
courtesy of <a href="http://business.flydenver.com/info/news/imageLibrary.asp" target="_blank">Denver International Airport</a>.</span></span></span></div>
Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com30tag:blogger.com,1999:blog-13967713.post-76815222933107351962012-10-02T19:06:00.000+02:002012-10-03T09:16:26.904+02:00Don’t do it!<br />
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">We
used to be taught that, by spending enough time thinking about
a problem, we would come up with a "perfect" model, one
that embodies many interesting properties (often disguised as
principles). One of those properties was stability, that is, most
individual abstractions didn't need to change as requirements
evolved. Said otherwise, change was local, or even better, change
could be dealt with by adding new, small things (like new classes),
not by patching old things.</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">That
school didn't last; some would say it failed (as in "objects
have failed"). At some point in time, another school prevailed,
claiming that thinking too far into the future was bad, that it could
lead to the wrong model anyway, and that you'd better come up with
something simple that can solve today's problems, keeping the code
quality high so that you can easily evolve it later, safely protected
by a net of unit tests.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">As
is common, one school tended to mischaracterize the other (and
vice-versa), usually by pushing things to the extreme through some
cleverly designed argument, and then claiming generality. It's easy
to do so while talking about software, as we lack sound theories and
tangible forces.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Consider
this picture instead:</span></span></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzgBdZS2rPr-DzLFfL-gfT7bMiOoWy78enlFH5UZro3qh-oUd6pKVAT_X3rfE8bAxMxspakSiMjSR9N-Pdgc97HxlAB0W8u6ays692_k6l8hK9F8crvRjYjbg7rKHQWOLJ3i-I_g/s1600/unstable.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="193" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzgBdZS2rPr-DzLFfL-gfT7bMiOoWy78enlFH5UZro3qh-oUd6pKVAT_X3rfE8bAxMxspakSiMjSR9N-Pdgc97HxlAB0W8u6ays692_k6l8hK9F8crvRjYjbg7rKHQWOLJ3i-I_g/s200/unstable.png" width="200" /></a></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Even
if you don't know squat about potential energy, local minima and
local maxima, is there any doubt the ball is going to fall easily?</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br />
<a name='more'></a></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Is
there any doubt that this ball is in stable equilibrium instead?</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaBGZA_vAtmlNYWzkLtGIND2omHB2bZibWd70SYm_YDPG1pJkf6ne2mIA-hWwobXmnxG0vEgQhpEHRD2hvukSCnvJA_4_fbx5Emgl_TQWvBXiFS3-1pz5uPpHnl00GebL-3tnkXA/s1600/stable.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaBGZA_vAtmlNYWzkLtGIND2omHB2bZibWd70SYm_YDPG1pJkf6ne2mIA-hWwobXmnxG0vEgQhpEHRD2hvukSCnvJA_4_fbx5Emgl_TQWvBXiFS3-1pz5uPpHnl00GebL-3tnkXA/s200/stable.png" width="200" /></a></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">The
nice thing about mechanics is that we have both a sound theory of
forces (so that you can formally prove that some configuration is
stable, if needed) and an intuitive grasp of things (so that you
don't need big theories when things are reasonable simple).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">This
is not so in software. When the guru claims something, the "proof"
stands mostly in his argumentation ability (rhetoric). Theorems, like
the CAP theorem, are few and far between. Lacking a theory of forces,
people tend to propose oversimplified models, where (for instance)
there is always a stable configuration for every problem (the old
school) or there is never a stable configuration for any problem
(some interpretations of the modern school).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Stability</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Requirements
change; actually, our own understanding of the problem changes during
and after development. Technology shifts as we're writing our code.
All those changes in the requirement space act as forces. Those
forces materialize as changes in our artifacts (source code). Unless
we have the right shape in place, some of those forces will ripple
through our artifacts like waves. The problem, of course, is finding
(or even defining) "the right shape", one where stability
is maximized. The right shape is totally dependent on the problem
(forces), so there is no “right shape” </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>per
se</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span lang="en-US"><span style="font-family: Arial, serif;">Unfortunately,
we don't know much about forces and shapes. Sure, there is an echo of
the "stability" property in some design principles. The
"open/closed" principle is trying to keep a class stable by
moving unstable parts into polymorphically-derived classes. The
"dependency inversion" principle is trying to contain the
wave of change by avoiding dependencies on concrete things (from the
classic <a href="http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf" target="_blank">Design Principles and</a></span><a href="http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf" target="_blank"> </a></span><span style="font-family: Arial, serif;"><span lang="en-US"><a href="http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf" target="_blank">DesignPatterns</a>:
"</span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>One
motivation behind the DIP is to prevent you from depending upon
volatile modules. The DIP makes the assumption that anything concrete
is volatile</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">"
[quite an assumption anyway]). Etc. But we're far from having a sound
theory. Concrete things might be stable, and interfaces unstable, for
instance.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Still,
sometimes we can find small, "stable" abstractions.
Sometimes, apparently, we can't. It would be ok if we could at least
recognize unstable configurations, and perhaps move toward more
stable shapes. This, however, would require an even more ambitious
step: to actually </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>classify</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
the most common instabilities. Not unlike classifying mechanical
stress into tension, compression, bending, torsion, and shear (see
<a href="http://www.carlopescio.com/2011/01/notes-on-software-design-chapter-13-on.html" target="_blank">Notes on Software Design, Chapter 13: On Change</a> for a digression on
this stuff), this would bring the software design conversation to an
entirely new level.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">This
post is <b>not </b>the right place to do so, although I've spent quite some
time tinkering with the concept of stability in my exploration of the
Physics of Software. But let's at least try to classify a few common
instabilities:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">1) Instability of internal structure or observable run-time behavior,
behind a uniform interface</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">For
instance, you have many shapes, each with its own optimized
implementation, but they can all Draw() themselves. </span></span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">2) Instability in multiplicity, with uniform processing</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">You
have an unbounded collection of objects, but you treat them all the
same way.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">3) Instability in structure, with a uniform externalized behavior
(usually reflective behavior)</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">For
instance, you have widely different structures, but all you want to
do is serialize them</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">4) Instability in structure, with a non-uniform (usually external)
behavior</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">You
have different structures or fields, and you do different things with
each one</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Etc.
There aren’t many more cases anyway; for instance, encapsulation,
closures and currying can deal with another kind of instability
(guess which :-).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Progress
in programming languages brought us some helpful concepts to deal with
1, 2, and 3 (in different paradigms), but nothing to deal with 4. </span></span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">We
may try to coerce 4 into 1, 2, or 3; for instance, by pretending that
everything is just a list or a map, so that instability in structure
becomes instability in multiplicity. That does not really work well.
It may work when you try to convert 3 into 2 (because your language
lacks reflection), but 4 is a different beast altogether.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Open
sets</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">In
my <a href="http://www.carlopescio.com/2012/05/notes-on-software-design-chapter-16.html" target="_blank">latest post on the physics of software</a>, I used a visual metaphor to
show how some configurations are necessarily unstable. To recap, any
</span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>single</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
concept that is </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>U/</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
entangled with an </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>open</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
set of concepts is, by necessity, unstable. </span></span>
</div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">In
the end, I came up with the uncomfortable idea that this class is
therefore unstable by construction:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
Person</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> string
firstName;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> string
lastName;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> DateTime
dateOfBirth;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> string
phone;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> //
…</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">because, quoting myself :-), "</span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>there
is an unbounded set of attributes for a Person (mobile phone; email;
living address; working address; place of birth; height; weight;
etc), and we have a single class which is U/U entangled with that
se</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">t".
</span></span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">What
is worse, we're facing type-4 instability, as I will use the
dateOfBirth to calculate your age and email to contact you. That's
not uniform behavior.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">I
also said that the force field is suggesting a different shape, not
necessarily something you want to use, but a different shape
nonetheless. So this post is not really about instability, but about
what the force field is telling us to do.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Don't
do this at home</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">What
I'm going to show is basically a logic consequence. It doesn't mean
you have to do things that way, or that I'm proposing you do things
that way. Actually, </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>I
totally discourage you from doing things that way</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">.
It doesn't play well with the way you've been taught to structure
your code. It doesn't play well with your language and tools; with
your database; with your UI library; etc. Of course, it might just be
the opposite (UI libraries, tools, languages, etc. are not well
aligned with the real nature of things). But the net result is the
same. </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>Don't
do it</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">.
That said, here is what you could do if you were insane (yes, I’ve
done that a few times :-).</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Note:
I've removed quite a lot of details from what follows, as this post
was turning into an endless exploration of possibilities. That should
be ok since you're not supposed to use this post as a starting point
for anything practical anyway :-).</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Step
1 (simple): Decompose to small Classes with a Role</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Large
abstractions are usually unstable (I have a nice theory about this,
based on entanglement: to a screen near to you sooner or later). So
let's do the simple thing first, and re-group fields into smaller
abstractions aggregated by role:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
PhoneNumber</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
some structure and responsibility here</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><br />
</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
Address</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
some structure and responsibility here</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
possibly using fine-grained classes like</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
Country, State, City, etc</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><br />
</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
PersonalName</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
some structure and responsibility here</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
see http://en.wikipedia.org/wiki/Personal_name</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><br />
</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
Sex</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
...</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><br />
</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
DateOfBirth</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
some structure and responsibility here</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
(see also below)</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">At
this point, we could at least represent our Person as:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
Person</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> PersonalName
who;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> Sex
gender;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> DateOfBirth
born;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> Address
livingAddress;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> Address
workingAddress;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> PhoneNumber
homePhone;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> PhoneNumber
mobilePhone;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> //
etc</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">This
does </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>not</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
solve the problem, of course. But small abstractions are often more
stable, and given the proper overall design, partitioning might be
enough to prevent a ripple of changes. For instance (in theory) any
change to the structure of a phone number should be isolated into
PhoneNumber and should not ripple into Person (if you can make your
DB and UI play nice, that is).</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">This
partitioning needs careful attention. For instance, DateOfBirth is
now a class; so it’s not just a DateTime. The reason to do so is to
have a </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>place</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
where we can move a responsibility formerly allocated to Person
(calculate age). You can’t ask a DateTime to calculate Age; it
wouldn’t be appropriate for such a general class. It’s a perfect
responsibility for a DateOfBirth class, though.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<a href="http://www.blogger.com/blogger.g?blogID=13967713" name="_GoBack"></a><span style="font-family: Arial, serif;"><span lang="en-US">Note
that I probably need many more classes than you're willing to create.
Address, for instance, is a very specific abstraction, describing a
place with extreme detail. Sometimes we want a vaguer concept of
place (like "place of birth"). A structural hierarchy of
places can be in order here, and we may even find a simple way to
deal with increasingly detailed information about a place.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">One
may also want to group the </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>who</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">,
</span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>gender</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
and </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>born</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
fields into a separate Demographics sub-center. It's fine to do so,
of course.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>The
shape of step 1</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Apparently,
Step 1 is not making much of a difference. Yet there is something
here anyway. The initial shape was basically a large, big, unstable
center (the Person). Instability will occur, by necessity, inside
that center (as there is no other place). </span></span>
</div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Now
we have a number of sub-centers (the Address, the PersonalName, etc),
and instability can be localized. Oh, yes, PersonalName can easily be
unstable; move outside your familiar culture and you'll discover new
needs and new fields.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Step
1 is also suggesting a potential fractal hierarchy of concepts (see
Place) that I'm not going to investigate further, but is nonetheless
important in the pursuit of a truly object-oriented form. In a sense,
it's moving from a big circle (Person) to something like this
(represented at an intermediate step in a fractal decomposition)</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyItkOOwvr69sIOjqbOm0PPO_gQGMFb7qIDmmn_HF5I1mKQC945zBRTW4TJ3GcMQLw1wS5eaLuQXOfXGskrqozt_Id8Oz42TbyqJ6uAfyAZePh_VI7sThlHLO0U51a_gv-etqeVQ/s1600/centers.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyItkOOwvr69sIOjqbOm0PPO_gQGMFb7qIDmmn_HF5I1mKQC945zBRTW4TJ3GcMQLw1wS5eaLuQXOfXGskrqozt_Id8Oz42TbyqJ6uAfyAZePh_VI7sThlHLO0U51a_gv-etqeVQ/s320/centers.png" width="320" /></a></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Step
1 is also suggesting that sub-centers, like PhoneNumber and Address,
are full-fledged classes, with (at the very minimum) validation
behavior and probably more. It's also suggesting that we should have
a widget to edit a PhoneNumber, and a widget to edit an Address, and
that a UI technology without widgets is </span></span><strike><span style="font-family: Arial, serif;"><span lang="en-US">a
rather stupid idea</span></span></strike><span style="font-family: Arial, serif;"><span lang="en-US">
not fitting well with this. It's also suggesting that the UI should
be dynamically composed by reflectively exploring the Person
structure, looking for widget-supported concepts.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Finally,
Step 1 might suggest that we may want to have a PhoneNumberRepository
and so on to deal with persistence in a more modular way, but then,
let's wait for Step 2.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Step
2: the reversal</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Abandon
you hopes. Here is where the fun begins. Just like User shouldn't
know about Credential (see that “<a href="http://www.carlopescio.com/2012/05/notes-on-software-design-chapter-16.html" target="_blank">Chapter 16</a>” post), but
Credential should know about User, Person shouldn't know about
PhoneNumber. PhoneNumber should know about Person. (Yes, that might
require a more abstract concept than a Person, say a base class,
because companies have phone numbers too; I’ll ignore that for
simplicity).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">That's
it, nice and easy. The way to avoid instability in structure is to
</span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>disband
the structure</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">.
There is no spoon. There is no Person as you tend to think of it. The
role of Person, as we'll see, is to provide </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>identity</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
and (optionally) reflective behavior.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">So
Step 2 brings to this shape (as an object model)</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBSZFT5cMLrd-k05wiNqjd41NtpsUXI2GcoBJI30Qu9zLL6DcIaR3wc3XPSiP3l7HoIidht9Y-QXWRi1ftAq71CMPzvOmgvd0Tip30LNH7MKBOBndqIffXJQ_FdUZxE_khT1Ak4w/s1600/step2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBSZFT5cMLrd-k05wiNqjd41NtpsUXI2GcoBJI30Qu9zLL6DcIaR3wc3XPSiP3l7HoIidht9Y-QXWRi1ftAq71CMPzvOmgvd0Tip30LNH7MKBOBndqIffXJQ_FdUZxE_khT1Ak4w/s1600/step2.jpg" /></a></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Many
PhoneNumbers and Addresses can be connected to a single Person,
according to their role. If I come up with an Email class, I don't
need to change the Person. The person just provides identity. [Yes,
I’m stepping away from my usual “dependencies go up” standard
because I want to show the gravitational pattern here].</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">This
should really be the end of this post. That’s what the forcefield
is telling us to do. Everything that follows is just </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>one</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
way to make it work, with its own consequences. There might be better
ways. Actually, I hope so.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Note
that the diagram above is a </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>conceptual</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
object model. In practice, you won’t see those associations in your
classes, because they would be dealt with at the repository level.
This might be a little confusing, so let's start from the bottom: the
database.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>The
Database</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">A
truly modular system would call for separate tables for each modular
concept. Therefore, a table for PhoneNumber, a table for Address,
etc. Most of those tables would have:</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
the person ID</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
the concept Role (like "home number"). In some cases
(DateOfBirth) this is not needed.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
all the concept's fields (like country prefix and whatever)</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">(some
of the satellite classes are unstable; guess you can figure what
happens to their database counterpart)</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">There
is also a Person table. The table provides only an ID, to act as a
foreign key on the other tables. That's the persistence-level idea of
providing identity.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Of
course, that provides maximum modularity. Need a new role for an
existing concept? Nothing to change at the persistence level. Need a
new concept (email)? Just add a table.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Still,
now it’s also inefficient to build the "entire" person
(or a significant subset). Of course, it would be possible to come up
with a database (or DBMS as they used to say) where you define a
logical structure (tables) and then you specify a physical structure
(like: pack all this stuff into a single row) and the database is
smart enough to remove the logical joins when mapping to physical access. But our current toolset is
not well aligned with this. In a sense, this is basically the
opposite of views. </span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Worst
case: you do a lot of small queries. Alternative: a number of joins
and some smart code (see below). In practice, what
follows does not <i>require </i>separate tables; it’s just the shape most
aligned with the force field. As I’ll talk about persistence code,
you’ll see that the decision about database structure won’t
percolate much.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>The
Repository</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Coming
from a traditional design background, it would make sense to have a
Repository for every concept, like a PhoneNumberRepository (or
perhaps a PhoneBook class :-). After all, this would preserve
modularity as well (and avoid the hourglass shape we saw in chapter
16), again at a risk of poor performance. </span></span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Ignoring
performance for a minute, single-concept repositories would usually
be trivial. So, in practice, we could have classes like this (you may
or may not like static methods; it’s a rather irrelevant details in
this context; also, I'll assume that you can call a repository and
get back a business object. See <a href="http://www.carlopescio.com/2012/07/life-without-stupid-objects-episode-1.html" target="_blank">Life without Stupid Objects, Episode 1</a> for a way to do that while
preserving strict layering.)</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
AddressRepository</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> public
static Address GetPersonAddress( Id personIdentity, Role r );</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> public
static IEnumerable</span></span><br />
<address>
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">
GetPersonAddresses( Id personIdentity );</span></span></address>
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">
</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
…</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">The
Address object will </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>not</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
need to contain a Person object; it’s enough to have its Id inside.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">However,
this choice will force us to do a select for each and every
fine-grained concept. That’s usually unacceptable, so let’s play
with this. I’ll assume a SQL database, because it’s what I know
best and it’s the environment where I’ve actually implemented
most of this weird stuff.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>The
Statement (part 1)</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span lang="en-US"><span style="font-family: Arial, serif;">A
Repository is usually charged with quite a few responsibilities. Say
that I introduce a new class instead, that I can call <b>Statement</b>.</span> </span><span style="font-family: Arial, serif;">The
Statement, among other things, would offer the ability to:</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
specify a center / start table, providing identity</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">-
specify that we need some fields, from some other tables (joined with
the center table)</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">-
add conditions</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">- transform
all that into SQL</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">-
execute the SQL</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
help Repositories turn that stuff into objects (I'll get back to this
in a minute)</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Using
a statement, I could to things like this (at a rather low level):</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span lang="en-US"><span style="font-family: Courier New, Courier, monospace;">Statement
s = new Statement( “person”, “id” ) ; // the center table and identity field</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">s.AddFields(
“role”, “street” ).FromTable( “address” ).RelatedBy( “id”
) ; </span>
</span></div>
<div style="margin-bottom: 0cm;">
<span lang="en-US"><span style="font-family: Courier New, Courier, monospace;">s.AddFields(
“firstname” ).FromTable( “personalname” ).RelatedBy( “id”
) ;</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Which
of course would become a 3-table join with 4 fields selected. The
real syntax needs to get a bit more complicated as the concepts get
fractal, but let’s keep it simple.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">I
could also add conditions, like:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span lang="en-US"><span style="font-family: Courier New, Courier, monospace;">s.AddFields(
“role”, “street” ).FromTable( “address” ).RelatedBy( “id”
).Where( equal, “role”, “home” ) ;</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">At
this stage, the Statement is just a way to build SQL statements
compositionally, and then execute them. The trick is, of course, that
</span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>the
composition can now be spread among Repositories</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">,
each dealing only with its own table.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Note:
In .NET land, this stuff can leverage LINQ expression trees, and
improve on my syntax as well using lambdas here and there. In other
languages, it might be slightly more challenging, but still doable.</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>The
new Repository</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">The
role of a repository is now to:</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
contribute in creating a statement</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
process the result of a statement execution and build objects</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">So,
the repository does <b>not </b>own the statement, does <b>not </b>execute the
statement, does <b>not </b>entirely control the statement. It <b>participates
</b>in building a statement, and in converting results back to objects.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Traditionally,
a method like GetPersonAddresses does everything at the same time, so
let’s split this thing in two:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">class
AddressRepository</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: 'Courier New', Courier, monospace;"> // mutable object syntax</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> public
static void PrepareGetAddresses( Statement s ); </span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"><b> private</b></span><span lang="en-US">
static List</span></span><br />
<address>
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">
ProcessGetAddress( Record r );</span></span></address>
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">
</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US"> //
…</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Of
course, I can still provide the former GetPersonAddress method as a
shortcut when composition is not needed. The nice part, however, is
that now I can:</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
create a statement for a center table</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
call any number of satellite repositories to have their own tables,
fields and conditions merged in, by calling the Prepare... methods</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">- execute
the statement</span></div>
<div style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Executing
the statement, at first, will just get a bunch of records from the
DB, but I want objects. Actually, I want each repository to create
<b>its own</b> sub-object for every record that has been retrieved. That's
quite simple: during prepare, the repository will instruct the
Statement with a callback function / delegate / whatever (that's why
ProcessGetAddress can be made private). The statement, after
execution, will iterate over the retrieved records, and call </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>all</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
the callback functions in different repositories for each record.
Taken together, what we get back is the object-land equivalent of a
record. The problem, now, is where do we store that stuff.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">(Oh, Record is just a convenient class to hide the technology-specific notion of a record).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><b>The
Statement (part 2)</b></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Ideally,
I'd like to prepare my statement (by calling various repositories; an
example will follow shortly), then execute my statement, and get back
a sequence of objects.</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">If
you're in the dynamic typing side of the world, that's very simple.
The statement can simply invoke the repository callback functions for
each and every record, take the contribution from every repository,
build a dynamic object, add it to a list, and that will be it.</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">If
you're into static typing, that's a bit of a challenge. Consider a
statement that retrieves a significant subset of fields formerly
belonging to Person, like DateOfBirth, Address, Email, etc. The
statement is built from the contribution of several different
repositories, then executed, then the result is processed by
different repositories, so for every record we get back a
DateOfBirth, an Address, an Email, etc. </span><span style="font-family: Arial, serif;"><span lang="en-US">As
much as you might be tempted to bring this stuff together into a
Person class, that’s exactly what we’re trying </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>not</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
to do, so don’t :-).</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Note
how this is suggesting that when we deal with unstable structures, at
some point we may benefit from a dynamic object. Note the context.
I’m not saying that every object has to be dynamic, or that we
always need them. Just that in this case, in this specific role as a
result of statement execution for unstable structures, a dynamic
object is very useful. </span></span>
</div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">An
idea I’ve had no chance to apply so far in real projects is to
adopt Tuples (like C# type-safe tuples, that is) as type-safe transport
objects. Would be nice if it worked, although C# generics are quite
limited; for instance, something like variadic templates in C++ would
be useful to deal with fractal decomposition. </span></span>
</div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">So,
at this stage, I will assume that the Statement will return a dynamic
object, and that object will be made of typed business objects like
DateOfBirth. If your language lacks dynamic objects, an HashMap of
some sort would be ok too.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">We’re
still missing a few things:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<span style="font-family: Arial, serif;">1) Where
do I put the “coordination” logic, that is, the creation of the
statement, the Prepare calls to repositories, the statement
execution, the actual usage of the results.</span><br />
<span style="font-family: Arial, serif;">2) How
do I deal with cross-concept (business) logic.</span><br />
<span style="font-family: Arial, serif;">3) What
about the [unstable] UI?</span><br />
<span style="font-family: Arial, serif;"><br /></span>
<span style="font-family: Arial, serif;">Believe
it or not, it’s easier to start with (2) because it will pave the
way to (1) as well. I'll postpone the UI till the end, because (2) will show that it's unstable only in some use cases.</span><br />
<b style="font-family: Arial, serif;"><br /></b>
<b style="font-family: Arial, serif;">Cross-concept
logic</b><br />
<span style="font-family: Arial, serif;">Ok,
so we can extract a Person, or any projection of a Person (say, only
phone and age), as a dynamic bag of type-safe objects. What about
cross-concept logic, that is, some processing involving more than one
satellite class?</span><br />
<span style="font-family: Arial, serif;"><br /></span>
<span style="font-family: Arial, serif;">When
I talk about this, people tend to come up with unrealistic examples
because, in practice, cross-concept logic is not so common. If it
were, those concepts wouldn't be real sub-centers. I don't need your
phone number to know whether or not it's your birthday. Still, there
are realistic examples, like:</span><br />
<br />
<br />
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Data
extraction</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">:
I want to find all males over 35 living in a given town, say for
</span></span><strike><span style="font-family: Arial, serif;"><span lang="en-US">spamming</span></span></strike><span style="font-family: Arial, serif;"><span lang="en-US">
marketing purposes.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Default
values</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">:
if I know where you live and I'm asking for your phone number, I may
want to precompile the area prefix.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Business
logic</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">:
even in this trivial Person domain, we can find something meaningful.
For instance, when the user logs in, we want to check if it’s his
birthday, and if so, we want to say "happy birthday, <name>".
It borders on presentation and extraction logic, but it's not, and
it’s cross-concept.</name></span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">None
of those is particularly challenging. Actually, the new shape helps
revealing the nature of what we're doing much better than a
monolithic Person class.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Data
Extraction</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">The
basic idea for cross-concept extraction is simple: we want to resolve
a set of </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>properties</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
to a set of </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>identities</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">,
then extract the individual concepts that we’re interested in, which are connected to those identities.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Well,
that's what the Statement + Repositories can easily do. The only
question left is where do we put the Statement composition,
execution, and processing of the results. Depending on your view of
layers, that question may already contain its own answer: if you
only allow Application-level classes to access repositories, that
logic should be in an Application (or Use-Case) layer class. </span><span style="font-family: Arial, serif;">A
code sketch for this logic could be:</span></div>
<div style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">Statement
spam = PersonRepository.PrepareIdentity( id );</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">PersonalNameRepository.PrepareGet(
spam ) ;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">EmailRepository.PrepareGet(
spam ) ;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">DateOfBirth</span><span lang="en-US">Repository.SetCondition(
above, 35, spam ) ;</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;">AddressRepository.SetCondition(
"city", "some city", spam ) ;</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;">IEnumerable<
dynamic > victims = spam.Exec();</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;">//
iterate over victims and do things with email and personal name</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">What
does something like </span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;">PersonRepository.PrepareIdentity(
id );</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">actually
do? </span><span style="font-family: Arial, serif;">Quite
simple:</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">return
new Statement( “person”, “id” ).Where( equal, “id”, id )
;</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">I
guess you can figure out the rest. We're basically building a
Statement from the cooperation of different repositories. <i>PrepareGet</i>
is not the nicest name ever for a method, but it's very easy to
understand (I hope), and that's my main concern at this time (as I'm
leaving out lot of details). AddTo or IncludeIn could be nicer
versions. Also, a more fluent approach like:</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">DateOfBirthRepository.SetCondition(
above, 35 ).In( spam ) ;</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">would
improve readability quite a bit.</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">I
understand you may not like that code at first. It "looks"
like the kind of code you want to hide behind repositories. But
that's mostly about habits, not substance. The persistence stuff is
indeed inside repositories (+ Statement). This is a type-4 unstable
selection of things, and must be in the upper layers, not in the
bottom layers.</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Note: in this case, the set of fields was basically hard-coded. In other cases, it's dynamically determined by various conditions. The compositional nature of the Statement tends to play rather well with that.</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Default
Values</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">The
default value thing can happen at different levels. In the specific
case given above, it's obviously a <b>user interaction concern</b>, as there is no
business rule about users having phones within the same prefix area
where they live. It's just a convenience when there is a human on the
other side of the screen, typing data. </span></span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">So
let’s decompose this further, in two distinct responsibilities:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
Find the most likely prefix given an Address</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">exercise:
where do we put that logic? Is that about the entire Address or just
a smaller concept like Area?</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
Propose that as a default value.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">We
need a way to trigger a call to the logic above and use that as a default value; as this is a
user interaction concern (for instance, it would make no sense to do
that when we import people from a file), the right place for the
trigger is the UI itself, or if you’re stuck in MVC, the
controller. Triggering from the UI would make for a much more responsive application (think web).</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">AOP-like
interception / injection would help here. Basically, we don’t want the
PhoneNumber widget and the Address widget to know each other. We need
a place that can see both and inject an </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>aspect</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
there (a user interaction aspect). Don’t have aspects in your UI
layer? Told you :-), your UI technology isn't good for this stuff.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Business
Logic</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">In
the usual BigPerson approach, it's obvious that we can put this logic
inside the Person, because guess what, all the data you need is
there, just add methods :-). Alternatively, some would put this
inside a Controller.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">With
this design, there is no </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>natural</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
place for cross-concept logic. </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>This
is a good thing</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">.
That void is telling you something (see also the quote at the end).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Without
making a big fuss out of it:</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
at the UI level, we should be able to simply put a Welcome widget on
the home page.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
the Welcome widget should be populated using a Welcome API / service.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">-
the Welcome class, given the person identity, should gather the
necessary objects and create the right message. </span></span><span style="font-family: Arial, serif;">Something
like:</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">Statement
welcome = PersonRepository.PrepareIdentity( id );</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">PersonalNameRepository.PrepareGet(
welcome ) ;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">DateOfBirthRepository.PrepareGet(
welcome ) ;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">dynamic
res = welcome.ExecSingle();</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">PersonalName
pn = res.PersonalName;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">DateOfBirth
dob = res.DateOfBirth;</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">if(
dob.IsBirthday() ) // a stable responsibility :-)</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{
</span>
</span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
say happy birthday + name</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">else</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">{</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">//
say hello + name</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Courier New, Courier, monospace;"><span lang="en-US">}</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">(ouch,
an </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><i>if</i></span></span><span style="font-family: Arial, serif;"><span lang="en-US">;
call the anti-if police :-). </span></span>
</div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Yes,
the static + dynamic style sucks a bit. If you got time to spare and
wanna try the Tuple thing, maybe you can make it largely type-safe in
many practical cases.</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>But
it's not object oriented!</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">No,
what you mean is that it is not coarse-grained-domain-oriented, at
least not the way you want to think about your domain, that is, based
on big, fat, unstable classes with a lot of logic, like Person.
Nobody said it should be like that (well, somebody did; it's just
not <i>universally </i>right).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">I
actually have </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>lot
of objects</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
flowing around. When you decompose to small enough pieces, you'll
find stable abstractions, like DateOfBirth. They have clear
responsibilities, like Age or IsBirthday. Actually, I am allowing,
or even forcing, small abstractions like Area to emerge, instead of
disappearing into the faceless blob we usually call Person. I also have a
Welcome object. A Welcome widget too. I have many more (cooperating)
objects than the BigPerson style tends to create.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><b>What
about the UI?</b></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Well,
the UI for some use cases is pretty stable (see the Welcome widget).
In other cases, of course, it's not. The paramount case is when you
want to edit or show "the Person" itself.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">What
we need to do, of course :-), is to dynamically discover the Person
satellites, then dynamically build a UI using satellite-specific
widgets (so, a PersonalName widget, a DateOfBirth widget, etc). That
discovery should also tell us about role-based satellites like
Address, so that we can provide a way to add / display multiple
widgets of that kind (for every role)</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Note
that we cannot use regular reflection for this, as there is no Person
class with all that stuff inside. Sure, the right class to ask would
be Person. But that requires that Person implements its own
reflective behavior, and that satellite concepts </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>register</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
with Person (so that Person can expose satellites, without knowing
satellites; that's how we deal with instability here). This might easily move to a base class.</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Things
get more complicated when satellites have satellites, especially when
you want to fine-tune appearance. In some cases, a trade-off between
aesthetics and stability is necessary, and a custom widget might have
to be created instead of relying on reflective composition of
lower-level widgets. As usual, proper UI technologies might help or
hinder. A technology lacking any kind of sophisticated layout
management, for instance, is less than optimal (yet we're stuck with
HTML and CSS :-).</span></span></div>
<div style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>The
Value of Emptiness </b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">(read
this part at your own risk :-)</span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">When
I first thought about this stuff, a very old story came back to my
mind. There are many translations (see <a href="http://www.geekfarm.org/cgi-bin/tao.pl?chapter=11&translation=all" target="_blank">here</a> </span></span><span style="font-family: Arial, serif;">for
a few; pick the one you like most), but here is one I like:</span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><i>Thirty
spokes are joined together in a wheel,</i></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><i>but
it is the center hole</i></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><i>that
allows the wheel to function.</i></span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Person
is the hole. By providing identity and nothing more, Person is
leaving an empty space where things can't coalesce into a
gravitational center. Yet, that hole acts as a </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>virtual</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
gravitational center for a constellation of finer-grained, more
stable concepts. It is by being empty that Person allows the system
to grow organically, instead of monolithically.</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><b>Don’t
do it</b></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">A
promise is a promise, so I had to write this stuff, but I’m glad it's over, so I can move on to some more interesting post; </span></span><span style="font-family: Arial, serif;"><span lang="en-US"><b>way</b></span></span><span style="font-family: Arial, serif;"><span lang="en-US">
more interesting, actually. </span></span>
</div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">Please,
don’t write me in anger. It’s ok not to like this. It’s ok to
disagree. It’s probably not ok to pretend that your Person class is
stable while is not, but it’s ok to look the other way just like
everyone else is doing, keep patching it as requirements change, and
claim you're being agile. </span></span>
</div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US"><br /></span></span></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">If,
on the other hand, that blurb above seems tempting, be smart: <b>don’t
do it anyway</b>.</span></span></div>
<div style="margin-bottom: 0cm;">
<br /></div>
<div style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><span lang="en-US">There
is no invitation to follow me on twitter, because you can’t
possibly like this post. Actually, I’ve downvoted it myself :-).</span></span></div>
<div lang="en-US" style="margin-bottom: 0cm;">
<br /></div>
Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com20tag:blogger.com,1999:blog-13967713.post-70690670014912680902012-07-30T08:41:00.002+02:002012-07-30T12:08:40.456+02:00No Controller, Episode 3: Inglorious Objekts<br />
<span style="background-color: white;">A week ago or so, Ralf Westphal published yet another critique of my post on <a href="http://www.carlopescio.com/2012/03/life-without-controller-case-1.html" target="_blank">living without a controller</a>. He also proposed a different design method and therefore a different design. We also exchanged a couple of emails.</span><br />
<br />
Now, I'm not really interested in "defending" my solution, because the spirit of the post was not to show the "perfect solution", but simply how objects could solve a realistic "control" problem without needing a centralized controller.<br />
<br />
However, on one side Ralf is misrepresenting my work to the point where I have to say something, and on the other, it's an interesting chance to talk a bit more about software design.<br />
<br />
So, if you haven't read my post on the controller, I would suggest you take some time and do so. There is also an episode 2, because that post has been criticized before, but you may want to postpone reading that and spend some time <a href="http://ralfw.blogspot.de/2012/07/flow-design-contrasted-with-object.html" target="_blank">reading Ralf's post</a> instead.<br />
<br />
In the end, what I consider most interesting about Ralf's approach is the adoption of a rule-based approach, although he's omitting a lot of necessary details. So after as little fight as possible :-), I'll turn this into a chance to discuss rules and their role in OOD, because guess what, I'm using rules too when I see a good fit.<br />
<br />
I'll switch to a more conversational structure, so in what follows "you" stands for "Ralf", and when I quote him, it's in <span style="color: #38761d;">green</span>.<br />
<br />
<a name='more'></a><br />
<b>My dear Ralf...</b><br />
<span style="background-color: white;">Overall, I've found your emails to be kind and reasoned, but your post to be filled with the usual rhetoric, whereby in order to look good, you have to make the other guy look bad. In the end, I didn't like it much when you said I'm cheating or that I'm proposing a spaghetti design. Especially when you email me after the fact and ask not to turn this into a zero-sum game. Usually you reap what you sow. But let's say I don't mind too much, and move forward.</span><br />
<br />
Your post can be seen as organized around:<br />
<br />
1) A critique of my design approach and of my presentation style (like using class diagrams instead of, say, activity diagrams and use cases)<br />
<br />
2) A proposal for a design method, basically the well-known functional decomposition approach, with an interesting but underdeveloped twist toward a rule-based system.<br />
<br />
3) A final design, represented by a "flow diagram" and a sort-of class diagram.<br />
<br />
Interestingly, you didn't pay any attention to the <b>most important thing</b>, that is, <b>the properties of the results</b>. Because the result, Ralf, <b>is not the class diagram</b>. It's not even the flow diagram, of course. It's an <b>allocation of responsibilities</b> to different "things", those "things" being classes, methods, whatever, resulting in properties, like extendibility, reusability, and stuff like that.<br />
<br />
It's very much like having these two nice houses:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDGpGFBT0J_2RPe8ljs03Yk28Mw0T2sbtKrug6aCYUg3scoPszeavy0G1gMefEwW38b5j3ChZnFEDY0Vyx67YuKc0grw18nAD50HE16QyioZJ72UqihVDRI51OxvhwwvehmkLQgQ/s1600/Brick_House.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDGpGFBT0J_2RPe8ljs03Yk28Mw0T2sbtKrug6aCYUg3scoPszeavy0G1gMefEwW38b5j3ChZnFEDY0Vyx67YuKc0grw18nAD50HE16QyioZJ72UqihVDRI51OxvhwwvehmkLQgQ/s400/Brick_House.jpg" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfTRQij0df8SOPfF9fUr6-UKW7i49Sikb2GC3i6pbz8g_TDtmKG0hD3ZfU7ocFXSUDOwaX_sDhfdUrTV7aiWYn-Rjhy45MbuymyyrhCBOMxmhs07Z7CSGseOMN5diBQwPh1Tm5Iw/s1600/Podbiel_wooden_houses.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfTRQij0df8SOPfF9fUr6-UKW7i49Sikb2GC3i6pbz8g_TDtmKG0hD3ZfU7ocFXSUDOwaX_sDhfdUrTV7aiWYn-Rjhy45MbuymyyrhCBOMxmhs07Z7CSGseOMN5diBQwPh1Tm5Iw/s400/Podbiel_wooden_houses.jpg" width="400" /></a></div>
<br />
<span style="background-color: white;">and instead of discussing the properties of the houses (building and maintenance cost; serviceability; maximum load; heat transfer; etc) spending all your time in a rhetoric exercise on whether we should start with structural engineering calculations or by drawing the floor plan. Up to the point where you confuse the floor plan with the house. The product of my design process is </span><b style="background-color: white;">not</b><span style="background-color: white;"> a class diagram, just like the product of an architect is not a floor plan. Is the </span><b style="background-color: white;">set of properties</b><span style="background-color: white;"> embodied in that diagram, which will be reflected into the real thing (the code, the house).</span><br />
<br />
You spend a lot of words babbling about process, but you never talk about the properties of what I get and the properties of what you get. All you say is that you don't have all my messy connections (in your diagram), which as we're going to see is <b>incorrect</b> at best. That makes everything rather moot, and with reference to what you said via email, it's hard to move the field forward by process rhetoric.<br />
<br />
Anyway, I'll try to cover your critique as quickly as possible, I'll try not to criticize your design <b>process</b> much / at all, and I'll try instead to look at what you get in the end. Because <b>I'm not really interested in discussing process, until we compare results</b>. Results come first. Once someone can show consistently good (or bad) results, it's interesting to look at his process, to see if we can somehow extract <i>some </i>of those good or bad qualities. Babbling about process without looking at results is pointless. And I'll say that again: results are not diagrams; results are the properties of the things we build.<br />
<br />
<b>Process, process...</b><br />
As I said, I didn't like this part much, because you grossly (and seemingly intentionally) misrepresented my work, while at the same time profiting from it. You make it look to the occasional reader as if I jumped straight from a pictorial description to a "complex" class diagram, without explanation, so that you cannot even understand how / if that thing works. But you perfectly know that's not what I did. I took the problem, piece by piece, discussed how I would allocate every portion of behavior to a class, presented small diagrams, and then put them together. I discussed likely changes and how the design was ready to deal with them.<br />
<br />
For instance, you criticize me for beginning with a (simple) class diagram instead of bringing in the user. Well, your first diagram adds nothing to the pictorial description of the problem I have included, except the operator. Unfortunately, you choose to focus on the wrong user. The mine pump is not there to satisfy a data-hungry operator (a controller? :-)). It's obviously there to guarantee the safety of mine workers. Perhaps I should have gone the BDD way, adding stories like:<br />
<br />
- As a miner, I want to know when there is too much methane, so I don't blow myself up.<br />
<br />
- As a miner, I want to know when the pump is stuck, so I won't find myself drowning in mud.<br />
<br />
- As a miner, I don't want the pump to turn on when the methane concentration is high, because I'd rather come back home alive.<br />
<br />
- etc.<br />
<br />
I didn't, because it was already a long post and I thought it was kinda obvious. Maybe it was not, but anyway, the operator is not a central actor here (although, of course, in a real system there will be some operator console somewhere). Guess what, your design is fine anyway, <b>because adding that [wrong] user into the diagram provided no additional value</b>, except of course a healthy dose of process rhetoric. It's fine if you feel compelled to draw diagrams that add no value; I don't, sorry.<br />
<br />
You're also assuming / implying / suggesting that I'm drawing class diagrams because I'm focusing on data. Actually you say that the role of a class diagram is to show the structure of data, and a comment on your blog even talks about "obsession with state" on the side of OO developers. This is very funny because those class diagrams show <b>no data</b>, exactly because I was focusing on <b>allocating behavior</b>.<br />
I have relationships, but those relationships are <b>not</b> about data, but about <b>collaboration</b>. Perhaps you're looking at classes and class diagram from the wrong perspective.<br />
<br />
You're also assuming that I never consider process in my reasoning, and that I jump into classes. Of course I consider process, but instead of modeling everything top-down a-la functional decomposition, I'm looking for candidate places for that behavior, as it emerge.<br />
You already have one in mind from the very beginning - the "rules" collection, so you don't need to allocate behavior. You just have to fit every problem into a pre-cut design. Unfortunately, a plain sequence of rules won't work it in this case, and won't work it in any real-world case either. You need something more complex, like a forward-chaining rule engine, that you neglected to show, but I'll get to that in a short while.<br />
<br />
It seems to me that, being so overly focused on a universal process and perhaps a universal blueprint, you think that I too am proposing some kind of universal process. I don't. I called that post "case 1" because it's just one of many. It's a realistic example though, and that's the reasoning I used <b>in that case</b>, because I saw a good fit with the problem. I have no problems using an activity diagram when it's called for. Or a state diagram when I'm dealing with a complex state machine. I think it's wrong to propose a fixed approach, irrespective of the problem. Of course, it's your right to differ and to propose your own universal process. You won't be the first to try; you won't be the last to fail.<br />
<br />
I will gloss over the part where you say I cheated by choosing an automation problem. <span style="background-color: white;">You may think that "</span><span style="background-color: white; color: #38761d;">it´s notoriously hard to find such objects in the requirements</span><span style="background-color: white;">" outside real-world objects. Still, a</span><span style="background-color: white;">long the years, I've been working basically in every possible application domain, and I've a consistent record of finding good classes. There are many other examples in this blog as well. F</span><span style="background-color: white;">inding good classes is a skill. A skill you decided not to learn. It's ok. It's not so ok when you call those who can do it cheaters, but I understand it's part of the usual old-school tactics.</span><span style="background-color: white;"> </span><span style="background-color: white;"> </span><br />
<br />
Just a final comment on understanding the dynamics through a static view. You said: <span style="color: #38761d;">I deem it no virtue to be able to “reverse engineer” a class diagram to get an idea how things are working</span>. Of course, you're entitled to your values. However, there is a long and consistent tradition in engineering to look at a static picture and infer the dynamics.<br />
<br />
We expect mechanical engineers to look at fig. 2 and 3 <a href="http://www.esm.psu.edu/courses/emch12/intdyn/activities/kinetics_rigid/slider-crank2/default.html" target="_blank">here</a> and understand the kinetic part.<br />
<br />
We expect electronic engineers to look at <a href="http://en.wikipedia.org/wiki/File:Transistor_Multivibrator.svg" target="_blank">this picture</a> and understand it's a multivibrator.<br />
<br />
In facts, I used to repair TVs as a kid. In my lucky days, I had a schematic for that TV set. Schematics are popular, because they're a rather efficient way to transfer information to skilled people. Of course, you have any right not to learn that skill. For instance, I'm not very good at reading <a href="http://en.wikipedia.org/wiki/Musical_notation" target="_blank">musical notation</a>. However, I would never step up and say that I do not consider that valuable for a musician or a composer.<br />
<br />
So, yes, I would also expect a software engineer to look at this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhiUDEtGYOqzR3d5rQPNbfGM6ScJ7fWGhDx7usFObty-dhhTZ0B-cxBfzl0OaQqNxUMMHGoS77p0_dQ5tpV86G02_k7dSw3notkFnjRZ_uXuyBQ1nmiFx0ugYkhMeBpMfVFtOLRUQ/s1600/ClassDiagram1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhiUDEtGYOqzR3d5rQPNbfGM6ScJ7fWGhDx7usFObty-dhhTZ0B-cxBfzl0OaQqNxUMMHGoS77p0_dQ5tpV86G02_k7dSw3notkFnjRZ_uXuyBQ1nmiFx0ugYkhMeBpMfVFtOLRUQ/s320/ClassDiagram1.jpg" width="320" /></a></div>
<span style="background-color: white;">and understand that it's very likely to be a callback scenario. There would be more to say about stereotypical interactions and the use of colors, but this is not the right time, I guess.</span><br />
<br />
Note how, in my <a href="http://www.carlopescio.com/2012/03/episode-2-controller-strikes-back.html" target="_blank">second post</a> about the controller, I proposed this engineering representation of the mine pump:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzsS-Dj4anh-3YBk7yDcRhxQavjN13Z7mb37VrjlhxFhoqq-22aKf_42Xie6f58oj0otibHZIC0vRn6sUfwOPO9NfplR4n3kA9pA2MHGtS5rVG1dGU1G1apH0Hqs27OotTMPlU8Q/s1600/engineering.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="311" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzsS-Dj4anh-3YBk7yDcRhxQavjN13Z7mb37VrjlhxFhoqq-22aKf_42Xie6f58oj0otibHZIC0vRn6sUfwOPO9NfplR4n3kA9pA2MHGtS5rVG1dGU1G1apH0Hqs27OotTMPlU8Q/s400/engineering.png" width="400" /></a></div>
Once you get past the rhetoric about class diagrams being evil and sort-of-activity-diagrams being good, that picture has a 1-1 correspondence with my class diagram (polymorphism aside, which is here for non-functional reasons). Maybe you don't "get" that diagram either. Oh well. Let's focus on what you got instead.<br />
<br />
<b>What you did and what you got</b><br />
<span style="background-color: white;">Once you cut the crap and get down to business (what you call the API level) you seem to drop all the big words on design being independent on classes etc and bring in the "dirty stuff" as you call it. I'll never understand why some people think / speak this way, but ok. By the way: part of reasons I'm trying to move software design toward properties, and more broadly toward a physics of software, is exactly to avoid this kind of ivory tower thinking / talking, which benefits no one. We (the software design community) should try to collectively raise the narrative bar and leave that rhetoric behind.</span><br />
<br />
Anyway, at that point you're beginning to create what I've defined a "fake OO step 2" system, that is, you create classes for the simplest abstractions (pump and sensor) and you collapse all the other logic into a centralized place. Nothing new so far (we have to wait for rules to get something new).<br />
<br />
There, however, you benefit from my speculation on likely changes, while at the same time trying to make my design look bad. The original problem statement was based on digital water sensors. A digital sensor will only say "it's full, up to my level, dunno above". The original specification of the hysteresis cycle was therefore based on having two digital sensors. That's why I started with a SumpProbe connected to 2 LevelSensor, and then discussed how moving to a continuous LevelProbe would require changes. Changes that I've hidden behind the SumpProbe class.<br />
<br />
Now, in your design you have no visible place for a digital sensor, so I can only assume one of these two options (you didn't say; so much for your stuff being much more precise):<br />
<br />
a) you have a composite sensor somewhere, exposing two digital as a sort of simulated analog.<br />
<br />
b) you just return an integer instead of a boolean, and you deal with two sensors becoming one elsewhere (the W<span style="background-color: white;">aterLevelSensor? maybe, but you don't even mention the issue).</span><br />
<br />
If you go with (a), <span style="background-color: white;">why did you move the logic for a composite digital sensor below the API level? </span><span style="background-color: white;">The problem statement was all about digital sensors, and a</span><span style="background-color: white;">ccording to your "process", you're not trying to build new abstractions as you go. You get the barebone things at the API level and put the logic in that process thing. So doing (a) is sort of cheating, right?. Well, also (b) is sort of cheating, because you are moving part of the logic below the API level anyway, cloning parts of my design. </span><br />
<br />
<span style="background-color: white;">There is more. My SumpProbe deals with hysteresis, which is not a property of a physical sensor; it's a process thing, to avoid turning the pump on/off continuously. It's also a stateful thing, because hysteresis needs status. Where do you handle that? </span><span style="background-color: white;">Again, you don't even mention this "detail", so we're left to wonder. </span><br />
<span style="background-color: white;">Say that, being a process thing, you do what you do for other process things and move it above the API. </span><span style="background-color: white;"> Then it' just another (neglected) oval in your process. Cool. How and where is the attached state preserved? You don't say. In the Rules class? What happens if the mine gets bigger and I need to add pumps, sensors, and glue them together? I can just instantiate more objects. What do you propose?</span><br />
<span style="background-color: white;">Say that it is inside the </span><span style="background-color: white;">W</span><span style="background-color: white;">aterLevelSensor. </span><span style="background-color: white;">But then you're cheating again Ralf, moving part of the process below the API at your convenience, without any clear method, mimicking my design when it suits you. </span><br />
<span style="background-color: white;"><br /></span><br />
In the end, you waste a lot of time talking about process, but neglect to say how things <b>really</b> work in your design. You omit details and claim simplicity, and you do this over and over.<br />
<br />
<br />
<span style="background-color: white;">You also spend a lot of words on why the hell did I use different classes for different gas sensors. As you're trivializing everything into a single class, you blissfully ignore, for instance, that sensors may have to linearize the input, compensate for drifts or ageing, detect specific failures, etc etc. </span><span style="background-color: white;">Of course, in the real world we would have base classes and reusable algorithms for that, but no, it's not like we can simply put everything inside a GasSensor returning an integer and say it's done. </span><br />
<br />
<br />
<b>The worst part</b>, however, is that all this critique was unnecessary to make your point; and honestly, it seems that you're trying to criticize something you don't fully understand.<br />
<br />
Then, at some point, you propose storing all the sampled data into a large SensorData thing. It's not clear what you have in mind, as you choose not to represent that thing (sure, omitting "details" makes things look simpler, right? :-). Perhaps it is a dynamic bag. Perhaps it is a strongly typed structure. In this case, it's rather crappy, as it will break whenever you add more sensors (bye bye modularity; bye bye independent reuse). Also, who is responsible for storing each data in its slot? The individual sensor, or some kind of centralized sampl<b>ER</b>? I guess a sampl<b>ER</b>, as your sensor is just <b>returning an int</b>.<br />
Anyway, that will get you exactly into the hourglass shape I was speaking of in my recent post on <a href="http://www.carlopescio.com/2012/05/notes-on-software-design-chapter-16.html" target="_blank">the forcefield</a>. Of course, you don't seem to mind, so please, go ahead and ignore the underlying forces at your convenience. Actually, this piece alone, if you cared to explain it better, would make for a nice comparison of encapsulation vs. the typical "store all the data here where everyone can read and write everything" relics of structured programming. But you didn't provide details, so there isn't much to say.<br />
<br />
Finally, we come to an under-specified class diagram where you say: "<span style="color: #38761d;">The classes are so little connected, I hardly need a class diagram at all.</span>"<br />
<br />
This is so <b>ridiculous</b> it's almost insulting. It's not like you can just remove classes and dependencies and claim that it's simpler. It has to work. So in your case:<br />
<br />
- The clock has to call the sampling, so it has to be directly or indirectly connected to a sampler (if any) and that in turn connected to sensors, but we don't see that. Sensors hang out nowhere.<br />
<br />
- The sensors or sampler have to populate the SensorData thing, but we don't see that class at all and so we don't see any dependency.<br />
<br />
- The SensorData thing has to go to your Rules class, but we don't see that.<br />
<br />
- Somehow the PumpState has to move to the pump, but we don't see that.<br />
<br />
- etc etc.<br />
<br />
Connections in my diagram are there because data (or objects) do not flow from one place to another on the back of unicorns. Except in your diagrams, of course :-). Oh, maybe it's all in global variables. Or maybe there is a sort of workflow context that again you neglected to show to claim simplicity. Is that context type safe? A dynamic bag? Again, you never say that. Absolutely precise, sure :-).<br />
<br />
Of course, you may (will) claim that there is some magic infrastructure taking care of all that (and more, see later about the rules). Guess what, I could have done the same. In episode 2, I discussed a wiring infrastructure. I also mentioned how one could simply use an IoC in some cases (also in the first post). Don't you see I could have just "assumed" that kind of infrastructure, just like you do? I could have drawn a diagram with basically no connections, assuming things would get wired at run-time through configuration. I <b>choose</b> not to. I choose to show how things have to be wired anyway. I choose to show something that didn't need any infrastructure.<br />
<br />
It's so funny that you called me cheater, isn't it? Except that my spaghetti class diagram (spaghetti obviously meaning, for you, that I have <b>very controlled</b> dependencies toward <b>more abstract</b> classes to keep everything modular and reusable) is the diagram of a <b>working system</b>, while your diagrams is a depiction of something that <b>does not work</b> because is neglecting essential components and relationships, <b>plus the entire damn infrastructure</b> you never cared to mention. C'mon.<br />
<br />
However, you do offer an alternative to the traditional, hard-coded controller, and that's the Rules class. If you didn't bury that in a pile of <strike>bullshit</strike> rhetoric, it could have been a more interesting, focused and pleasant conversation :-). So let's focus on that.<br />
<br />
<b>The Rules</b><br />
<span style="background-color: white;">So, in the end, the only relevant difference from the usual "simple classes below, monolithic controller above" (fake OO step 2) is that you decompose your controller in rules. </span><br />
<br />
Of course, you're assuming some infrastructure here, because as presented, the Rules class is useless. Also, in your class diagram, there is no trace of the join between Assess_water_level and Assess_methane (there is also no output from that, but hey, you want to pretend you have no dependencies).<br />
<br />
Obviously, you can't just have a list of rules. You need a graph and a forward-chaining engine or something like that (been there, done that). You need a machinery to make the join, and if you support independent threads, it will also have to wait until all the inputs are present before calculating the output or the guy in the mine will die.<br />
<br />
And of course you're just <b>assuming all that</b>. I had to read some comments on your blog, follow a link to a "cheat sheet" of yours and click around for a while to discover that you're proposing your own homegrown rule engine. Phew. That took a while. Sorry, perhaps that thing is all rage in your neighbor, but I didn't know about it. It would have been nice of you to introduce that thing from the very beginning, perhaps removing some of the unnecessary process <strike>crap</strike> stuff to make some room.<br />
<br />
After spending enough time understanding what you didn't care to explain, your design started to make a little more sense. You created a rule engine. So once you write classes for the actions, you "just" have to wire those things through configuration. My "spaghetti" diagram assumed no library, no support, no nothing. Your pseudo-diagram assumed a rule engine, a wiring library, and what else (and didn't even bother to show a placeholder for some of that). Then you claim simplicity, and call me cheater. How cool is that Ralf :-). Oh, and there is still the issue with the missing SensorData thing and so on, but I'm gonna cut you some slack.<br />
<br />
The paradox here is that if you approached things another way, we would have found a lot of common ground. Because you know, I use rules and rule engines as well. For instance, I've just ported my GUI Rules library to Android. So now I can say things like (in a fluent-Java style):<br />
<br />
<span style="font-family: Verdana, sans-serif;">rules.add(</span><br />
<span style="font-family: Verdana, sans-serif;"> enabled(okButton).</span><br />
<span style="font-family: Verdana, sans-serif;"> when(</span><span style="background-color: white; font-family: Verdana, sans-serif;">notEmpty(uid).</span><span style="background-color: white; font-family: Verdana, sans-serif;">and(</span><span style="background-color: white; font-family: Verdana, sans-serif;">notEmpty(pwd)))</span><br />
<span style="font-family: Verdana, sans-serif;"> );</span><br />
<br />
It would be unfair to claim simplicity behind that thing, however. There is quite a lot of machinery. You don't see it when you combine rules. But you can't just pretend it's not there and claim that things are magically simplified by <b>process</b>. They are simplified by a <b>library</b>.<br />
<br />
Now that I have brought this thing to what I hope is a more neutral territory, I'd like to explain how my view of rules differs from yours, and why.<br />
<br />
<b>Two-levels vs Fractal</b><br />
<span style="background-color: white;">I guess you're familiar with this kind of house:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMJjmVtEu5y4rS7zhYNAq5R21ILrq0sVGH1NVJalGoszcJSi1uHquP9ob1A9r11O9xqCZpmFX6B8n71_CgO-100xrd1sXku1Qg-C0A60ZjRbSsjazIrKkhynpAENmGgZlTYYvY-w/s1600/Noetsch_Saak_altes_Haus_Kulturdenkmal_2008_0814.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMJjmVtEu5y4rS7zhYNAq5R21ILrq0sVGH1NVJalGoszcJSi1uHquP9ob1A9r11O9xqCZpmFX6B8n71_CgO-100xrd1sXku1Qg-C0A60ZjRbSsjazIrKkhynpAENmGgZlTYYvY-w/s400/Noetsch_Saak_altes_Haus_Kulturdenkmal_2008_0814.jpg" width="400" /></a></div>
<br />
It's bricks and concrete below, wood above. Of course, you have to stack them in the proper order :-). So there is a scalability issue here, because you can't just add another brick layer on top of a wood layer.<br />
<br />
So Ralf, let me ask you something. As you know, if you read <a href="http://www.carlopescio.com/2012/03/episode-2-controller-strikes-back.html" target="_blank">chapter 2</a> in the controller saga :-), I choose the mine pump because it's a well-known problem, people have already proposed meaningful changes, and I don't have to cheat, I don't have to make things up, I don't have to propose changes that I'm safely well-prepared to handle. There is literature on that.<br />
<br />
A common, meaningful change that I have already discussed is the need to have two or more pumps, and rotate among them over time to reduce wearing. Where would you introduce this change in your design? You basically have two choices (if you want to preserve the overall design):<br />
<br />
- you can introduce the change below the "api level" (bricks)<br />
<br />
- you can introduce the change above, that is, in processes and rules (wood)<br />
<br />
This is a "business rule", so to speak, so it would seem more appropriate for you to introduce the change using rules. You can add yet another oval after "switch pump", choosing which pump.<br />
<br />
Then the next meaningful change is that if one of the pumps gets stuck, as observed through current sensing, you have to switch to another pump anyway (even if it's not its time yet).<br />
If you deal with that at the process / rule level, you need an "assess current" oval, then you need to change the switch logic above, drawing another join to consider the current too, and switch to another pump when needed. You may think this is ok, because hey, it's the process.<br />
<br />
And here is where you should have your epiphany, Ralf. When you say "<span style="color: #38761d;">Well, yes, that´s a kind of functional decomposition. And I don´t see what´s wrong with that</span>" you're just ignoring 40 years of mankind experience with functional decomposition.<br />
<br />
I can tell you what's wrong with that: <b>as you scale it up, you're <i>not </i>building reusable abstractions along the way</b>. You're just making the upper part more and more complex. And after you beef it up, you can't scale it down (see Parnas), without creating a clone of your large process and pruning things here and there (remember to remove unnecessary slots in your SensorData too :-).<br />
You have only one, big gravitational center (your process), and that's it. Your rules are probably reusable (except you modeled them as method, not classes, so it's still copy&paste reuse), but the "hard knowledge" is in the wiring of the diagram, not in the rules (which, I hope you'll admit, tend / want to be very simple).<br />
<br />
You see, just like you, I wasn't born with objects in my hands. I wrote my first code back in 1978. I know what is like to create large systems using structured programming and functional decomposition.<br />
The real beauty of objects, once you really understand objects :-), is exactly in that thing you don't seem to appreciate: objects as virtual machines, things made of things made of things made of things, all of the same power.<br />
<br />
When you stack simple objects under a ton of increasingly complex procedures (whether you bundle those procedures into a controller or in a rule engine) you'll never get the reusable abstractions I'm building along the way: the pump rack, the fail-safe pump rack, the sump probe, the safe engine, etc. Abstractions that can be reused in a completely different problem, where (for instance) methane is not an issue. Too bad in your process the methane comes first.<br />
Your flow diagram is not reusable, and is bound to get bigger and bigger as you scale up requirements. You'll have to go the usual way, and adopt nested diagrams, whereby an oval on one level is an entire diagram on the next. Then you're going to discover you're passing a ton of data around (your SensorData on steroids), but wait, most of those data are only used in some places. And you're going to come closer to some form of encapsulation to solve that form of coupling (there is a long story on how the notion of coupling changed when moving from structured to OO programming, and there is something to be learnt from that. I'll talk about it in a future post). Keep going ahead, and sooner or later, you'll rediscover modules and then objects, because objects are a reasonable (although not definitive) response to some of the forces we routinely encounter in software development.<br />
<br />
Oh, by the way, you could also add those responsibilities below the API line instead. Then I really would ask you, why only that? Why don't you go further and push down more and more responsibilities? What is, in your universal process, the guiding force in placing something below or above the API level? Why do we need that distinction at all? Can you see how your "standard blueprint" is trying to squeeze every problem into the same shape?<br />
<br />
<b>So why / how do I use rules ?</b><br />
<span style="background-color: white;">I just said that I use rules too, so this may seem like a contradiction. But it is not. </span><br />
The thing is: you make process and rules the center of your system (and development process), and stack that on top of trivial objects, so you get a two-story shape that does not easily scale. I use objects as my primary decomposition technique. Inside <b>some</b> of those objects, I use rules. But it's "just" a convenient implementation detail. Let me elaborate with an example (one of many :-).<br />
<br />
I tend to design my UI as a thing made of things, so I usually have a form made of widgets, made of widgets, made of widgets. There are recurring problems in UI implementation though, like disabling some controls when there is no input or when a checkbox is not ticked, etc. The popular thing is to use standard widgets + a large, form-wide controller or view model + binding (assuming you have a binding-friendly UI library, which is not the case in Android, for instance). This is a two-level approach, with relatively stupid, standard widgets below and a more complex monolithic controller on top.<br />
<br />
I don't like monoliths, so I look for meaningful aggregates and bundle them together in a widget. So for instance I can bundle a label, a textbox, and another label or a combo box in a widget which can be [re]used to input something with a unit of measurement. Do that the right way, and you end up building an OO UI, with things made of things made of things.<br />
<br />
However, even in that case, it's hard to reuse some common code, like disabling a button when some field is empty. There is, unfortunately, some repetitive machinery to implement, especially in lambda-unfriendly Java. You have to implement an interface, add to a collection of listeners [sic], etc etc. Things get worse when you see that (for instance) in Android you can't just disable a ViewGroup to disable the internal controls; you have to do it individually. In the end, this kind of logic is not something you can easily place in a <b>single</b> reusable widget. From an AOP perspective, it's a different aspect altogether, and plain OO can't deal well with that.<br />
<br />
That same kind of repetitive logic, however, can be easily put inside rules, actions, etc. The infrastructure will then take care of implementing interfaces, adding listeners, combining results with joins, etc. That’s what my little library does. Actions and predicates, in this case, are also reusable classes (not custom methods inside a single Rules class). But here is the thing: <b>this is not the dominating structure in my code, and certainly not in my thought processes</b>.<br />
<br />
It's a small set of rules, hidden inside a widget. Some widgets don't even need rules. Some do. As the widget structure is fractal, there is no explosion of rules, because the nature of the widget is to stay small (possibly by including other smart widgets), so I never end up with a humongous set of rules in a single place. Also, data is always local (the contained widgets), so I don't suffer from a large data structure flowing around. That's it. Objects outside, a few rules inside, when / where needed.<br />
<br />
I have similar structures elsewhere. I tend to model communication protocols as workflows, and there is a workflow engine and a workflow context and all that stuff. But it's not a dominating structure. Inside (say) a Device, you'll find a workflow to handle the communication protocol. If the protocol is trivial, you may not even find a workflow. It's as simple as that. It's Device outside, workflow inside.<br />
<br />
As you asked me, via email "<span style="color: #38761d;">What´s the right way? Is there a right way? Is there overlap? Is there a continuum?</span>", I can honestly say that I don't pretend to know the "right" way, especially not some sort of universal "right" way, a good fit for every problem. But I have definitely combined objects with rules and with workflow engines, just not the way you do. You tend to create a two-layer stack. I tend to create a fractal structure, because it leaves a trail of meaningful, reusable, domain-specific abstractions I can reuse elsewhere, encourages information hiding, and can grow almost indefinitely while preserving decent maintainability. Inside some of those abstractions, I use rules and workflows.<br />
<br />
<b>Conclusions</b><br />
<span style="background-color: white;">As I said elsewhere in this blog, I value freedom, I value options, I value adaptability. I design things by looking at forces and responding to those forces, using a mixture of approaches and materials. Your post would have been more interesting if you presented it like another option, perhaps going into more details about your proposal instead of wasting way too much time trying to trash mine or promoting your own process as superior.</span><br />
<br />
While you consider your design to be self-evident, I find your explanation lacking. You don't show how / which data is supposed to flow around, how you avoid the problem of excessive data coupling typical of functional decomposition, how you intend to deal with reusability, scaling up/down, how you react to changes, etc, that is, you never talk about the properties of your code, at a level of detail where we can actually believe you, not assuming more or less magic infrastructure.<br />
<br />
That said, I guess / hope I can use your engine my own way, inside one of my classes. If I can't, then I just won't use it. I also guess / hope I don't need to become part of a method cult to use it, or to bow to your process and let it dominate the entire design. If I have to, I won't use it. However, I'm sure some people will happily surrender freedom in exchange for a standard blueprint. There is ample evidence of that (just think of J2EE :-). It's just <b>not</b> what I consider #braindrivendesign.<br />
<br />
By the way, Ralf: can't we just be friends? :-)<br />
<br />
<br />
If you liked this post, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me</a> on twitter!<br />
<br />
<b>Acknowledgements</b><br />
<span style="background-color: white;">The brick house is courtesy of </span><span style="background-color: white;">David Sawyer (</span><a href="http://commons.wikimedia.org/wiki/File:Brick_House.jpg" style="background-color: white;">http://commons.wikimedia.org/wiki/File:Brick_House.jpg</a><span style="background-color: white;">)</span><br />
<span style="background-color: white;">The wooden house is courtesy of </span><span style="background-color: white;">Robzle (</span><a href="http://commons.wikimedia.org/wiki/File:Podbiel_wooden_houses.JPG" style="background-color: white;">http://commons.wikimedia.org/wiki/File:Podbiel_wooden_houses.JPG</a><span style="background-color: white;">)</span><br />
<span style="background-color: white;">The layered :-) house is courtesy of </span><span style="background-color: white;">Joadl (</span><a href="http://commons.wikimedia.org/wiki/File:Noetsch_Saak_altes_Haus_Kulturdenkmal_2008_0814.jpg" style="background-color: white;">http://commons.wikimedia.org/wiki/File:Noetsch_Saak_altes_Haus_Kulturdenkmal_2008_0814.jpg</a><span style="background-color: white;">)</span>Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com23tag:blogger.com,1999:blog-13967713.post-87687230157726982762012-07-02T12:41:00.000+02:002013-01-06T11:07:57.725+01:00Life without Stupid Objects, Episode 1<br />
<span style="background-color: white;">So, this is not, strictly speaking, the next post to the previous post. Along the road, I realized I was using a certain style in the little code I wanted to show, and that it wasn't the style most people use, and that it would be distracting to explain that style while trying to communicate a much more important concept. </span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">So this post is about persistence and a way of writing repositories. Or it is about avoiding objects with no methods and mappers between stupid objects. Or it is about layered architectures and what constitutes a good layer, and why we shouldn't pretend we have a layered architecture when we don't. Or it is about applied physics of software, understanding our material (software) and what we can really do with it. Or why we should avoid guru-driven programming. You choose; either way, I hope you'll find it interesting.</span><br />
<br />
<a name='more'></a><br />
<b>"Line of Business" applications</b><br />
Consider the usual business application: you insert, search, edit, and delete a variety of business entities, usually storing those entities in some kind of database. There is some logic kicking in at specific times, from simple validation logic to complex calculations and workflow actions. Sometimes, data is transformed; very commonly, it is summarized in reports. In many cases, it is persisted in a form which is not really different from what you see on the screen. Needless to say, there is also a user interface to do all that, and today, it's very common to go for a web application.<br />
<br />
Let me leave the user interface aside; it's a large topic, and would (will) deserve some separate discussion. What is a reasonable shape (architecture) for all the rest? This is not a new question, and you can find many answers, picking from the various patterns in Fowler's et al. (Patterns of Enterprise Application Architecture), from the architectural suggestions in Evan's DDD book, from to the reference architectures coming with specific technologies (like the now old EJBs) or toolsets (e.g. O/R mappers), etc.<br />
<br />
The sheer volume of knowledge is so big that we face a problem of depth vs. breadth. Either we cover a lot of alternatives, but at a rather superficial level (like in Fowler's book) or we choose a particular style, and we go down deep, possibly providing an implementation as well.<br />
<br />
At this time, I'm more interested in discussing forces and how they are shaping different solutions. I'll be focusing on a rather popular style, not because it's necessarily the best, but because it's a good example of how stupid object may arise out of good intentions, and how can we cure that. In this sense, this post is more like an example of "applied physics of software" than an attempt to discuss every nuts and bolts of every possible choice. In the end we'll see some code as well, because some properties are better appreciated through a few lines of code than through endless babbling.<br />
<br />
<b>Forces, layers and decisions</b><br />
Of course, most of you guys already "know" how to do this. Sure, you may all have different architectural preferences, but there is a strong tendency to go for this overall structure:<br />
- a database of some kind (often SQL, sometimes NoSQL)<br />
- a "persistence layer"<br />
- a "business layer"<br />
- (possibly) an "application layer"<br />
- (possibly) a "service layer"<br />
- (a "gui layer", which I'll ignore)<br />
<br />
Where do those layers come from? What is a layer, exactly, anyhow? It isn't just a random grouping of classes / functions / modules, right? What is the purpose, definition, and role of a layer in a software system? You could head for "Pattern Oriented Software Architecture, Vol. 1: A System of Patterns" (Buschmann et al), chapter 2, go to the Layer pattern, and you <b>won't</b> find an exact definition of a layer :-), but you'll find its responsibilities: "Provides services used by Layer J+1. Delegates subtasks to Layer J-1". Hmmm. Well, ok.<br />
<br />
Layers, by nature, are a grouping mechanism, wereby we grow software by stacking abstractions "vertically", while grouping similar software abstractions "horizontally". Why do we give that shape to our software? In the physical world, a shape is the result of applying a force (this is an interesting insight from D'Arby, to which I'll return at some point), like compression of shear. In the software world, <b>the shape is always the result of applying a decision</b>.<br />
<br />
Many decisions are taken without even thinking: for instance, we decide that data must persist between sessions, even if we turn power off. That's sort of a given and people don't even realize they're choosing something.<br />
<br />
Some decisions are technological in nature: we decide that we want to use a SQL database. We decide that we will use Oracle. We decide that we won't use stored procedures. Etc.<br />
<br />
Some decisions are business related: we decide that we want to support multiple authentication techniques. We decide that we will greet the user with a "happy birthday" message if he logs in at the right time, etc.<br />
<br />
Decisions are unstable. Some more than others (this will largely be the subject of my next post). Now, in a 40-year old paper <b>you can't afford not to read</b> (<a href="http://www.cs.umd.edu/class/spring2003/cmsc838p/Design/criteria.pdf" target="_blank">On the Criteria To Be Used in Decomposing Systems into Module</a>s, and yes, that's forty years kids) David Parnas said: "We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others"<br />
<br />
So what is a good definition of a layer? A layer is a group of abstractions, hiding one or more internal decisions <b>plus</b> all the decisions made inside the underlying layers. Therefore, a layered system is "good" when you can actually change a decision made in one layer, and none of the layers stacked on top of it needs to change. In order for that to happen, <b>a layer should avoid replicating/exposing the responsibilities of the underlying layer</b>.<br />
<br />
Let me give a proper example. In the quintessential layered architecture (hardware abstraction in OS) at some point there is a portion of code dealing with virtual memory, and how you manipulate CPU-specific things to make that happen. This is <b>not</b> exposed to the upper layer (it could be, but it's more like a backdoor). Instead, we get a new abstraction (virtual memory). Languages like C# and Java effectively add another layer, basically hiding the concept of addressing at all (still exposed in C and C++). This in a sense is the best layering, as the concept disappears and we don't even speak of a "memory layer".<br />
<br />
Contrast this with a business layer built on top of a persistence layer, and yet exposing/replicating the persistence responsibilities (say, CRUD) for the application layer to call. This is not an ideal layered architecture (sorry), because your layers are becoming bigger as you travel up, and changes made in a lower layer may end up being reflected in an upper layer.<br />
<br />
In practice, it's quite rare to find a problem (forcefield) which lends itself well to true layering. In many cases, layers are brittle, that is, most of the decisions made inside those layers can't actually be changed without breaking all the layers above. For instance, if you aim for the Buschmann book again, you'll find an example of layered design for a chess game, explained as follows (from bottom to the top):<br />
<br />
- Elementary units of the game, such as a bishop<br />
- Basic moves, such as castling<br />
- Medium-term tactics, such as the Sicilian defense<br />
- Overall game strategies<br />
<br />
In practice, you have very little latitude for change in every layer: you can safely change only minor implementation details. Suppose that you change a "major" decision ("business rule"), like the basic moves of the games. You allow some previously illegal move, or vice versa. That requires a change in layer 2. However, that change would immediately invalidate all the upper layers: layering is not so effective here.<br />
<br />
It is also worth observing that layers, although conceptually simple, are extremely constraining, as they basically define a <b>single dimension</b> upon which you can stack your abstractions, and therefore will push you toward a design where you have to identify a "principal decomposition" (contrast this with the goal of aspect orientation to free you from the tyranny of the principal decomposition).<br />
<br />
In most cases, developers choose technology as the principal decomposition in Line of Business systems. I want to stress the fact that it is a choice (a decision), and it is not inherent in the concept of layers or in any design paradigm, but it's still the most popular choice.<br />
<br />
<b>Why technology?</b><br />
Once you see a layer as a set of artifacts that are somehow "squeezed together" by some force, you can easily see that technology is only one of the forces acting on your artifacts.<br />
<br />
Sure, you have a force saying "let's squeeze together all these classes, as they know they're dealing with SQL", the rationale being that if you change to NoSQL, you want to "change only one layer" (sooner or later, I'll come back to the concept of entanglement and explain this stuff better).<br />
<br />
Yet, you have at least another (orthogonal) force, pushing all the technologies together, and layering the system according to the domain instead, the rationale being that if you add a field to one of your abstractions (like a title, so you can say "hello <b>Dr.</b> Something, happy birthday"), you only have to change one abstraction. When you choose technology as your principal decomposition, this decision is spread among layers instead, giving place to an unfavorable entanglement field among many "distant" abstractions (I consider layers to be somehow distant in the artifact space).<br />
<br />
There is also another force. I'm sure many of you felt it, but frequently decided to look the other way :-). It’s a force pushing the logic toward the data, which gave rise over time to things like stored procedures, still considered like dirty, non-portable hacks by many. And yet, as you move toward a distributed, NoSQL solution heavily based on map-reduce, you'll see your (distributed) data attracting more logic, with nodes offering higher-level services, effectively subverting some of your layers. But we can't talk about these things in public.<br />
<br />
So, in the end, people ignore all that and favor technology as the principal decomposition. There are many valid, and many non-valid reasons to do so. I won't even try to enumerate, as reason #1 is "all the gurus are telling us this is the right decomposition" and you can't argue with that. Still, trying to list your reasons and ponder on the validity of each one is an interesting venture in self-awareness :-). For instance, division of labor and technical specialization seems like a good reason, but is not without consequences.<br />
<br />
Note that, in the typical line-of-business application, the distinction between the "application layer" (or "use case layer" as some call it) is not necessarily driven by technology. Actually, that line is blurry, and again, it's mostly being accepted as reasonable because the gurus tell us so, not because it is technically bulletproof. The typical definition is that the domain is about things that "are always valid", independently of the application / use cases, while the application layer realizes the use cases by delegating to the domain layer.<br />
Makes sense, except when you look again, and discover that "always valid" makes no sense whatsoever. For instance, my User class is offering a date of birth field (and possibly an IsBirthday() method) because my application wants to greet the user. Were I just interested (e.g.) in geolocating my user, that function wouldn't be provided, that field wouldn't even be there. So much for being part of the domain, then.<br />
In the end, many people bring in technology again as a criteria to separate the application layer and the domain layer, by stating that domain objects should not know about persistence (which de facto alters the stack, but they don't say so because the gurus never said so). This brings us to the next big question.<br />
<br />
<b>Acceptable relationships between layers</b><br />
Some relationships are basically imposed by the function (which here determines the form). Some are free for us to choose. Of course, choosing some relationship is akin to choosing a stacking order, or to abandon the stacking concept in favor of a more flexible dependency net (but hey, you can keep drawing it as if it were a stack - nobody is actually checking these things anyway :-).<br />
<br />
- nobody should know the UI. That's the generally accepted idea. It's a bit preposterous, given that in many cases the back-end exists only to support the UI, but you can't challenge the notion in guru-driven development. It does not matter that, since the UI is in a web app, and the bandwidth is limited, you need to expose paging at the service level, and probably handle it down to the database level (using your SQL dialect of choice) to keep performance reasonable. You can still pretend that your stack does not depend on the UI if you keep adding enough layers. So, let's pretend nobody knows about the UI.<br />
<br />
- the service layer, if distinct from the application layer, has to know the application layer. That's sort of a necessity, because when you split the two, it's to allocate in the service layer all the knowledge (decisions) related to the "delivery mechanism", as Martin calls it, that is, knowledge of your services being exposed as SOAP, JSON, RPC or REST style, etc. But in the end you have to invoke the damn use case / application layer, so the service knows the application.<br />
<br />
- is the persistence layer allowed to know the domain layer? Hmmm. This would be a violation of the concept of layers. The persistence layers should only know about one decision (the persistence technology). It should also hide the underlying layers. Unfortunately, to hide the domain layer, it should also expose the domain functions, because the application layer wants those functions. That's not going to work. Of course, if we give up the idea of layers, new shapes are possible. But the gurus want layers, so I won't explore the other shapes.<br />
<br />
- at this point, the application layer has to know the domain layer. That's again sort a given, because the application layer is thin and is just somewhat "orchestrating" the domain layer.<br />
<br />
- is the domain layer allowed to know the persistence layer? This has been, and in some sense still is, at the center of a long-winded debate.<br />
Proponents of techniques like Active Record and Lazy Fetching naturally think it's ok for a domain object to call into the persistence layer (perhaps without knowing so, through some ORM-generated code) when need arises.<br />
Critics have many arrows to shoot. In practice, I've often observed horrible performance in systems where access to the persistence layer is basically out of control. I'm not the only one to have observed so, of course. Some tried / are still trying to solve the problem by making the whole thing even more complicated. For instance, they try to aggressively cache / prefetch objects, which requires a proper implementation of an identity map, etc. Others have gone with a more simplified approach: the domain layer is forbidden from talking to the persistence layer. It's the application layer which is responsible to mediate between the two. As I said, this actually brings persistence and domain at the same level in the stack, but nobody wants to say so.<br />
<br />
In the end, the intended shape of many LoB systems is one of these:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUZhz9j-QRCiIxx9Ns0iNOOr50HtTftTIfhaQ_TZJxAskQBrZ_R94wSFBHFv8ONK9PTqGThNzqKX2vTnYfb4G9NvTe505WfG1NgwC9WXjwW-FjRtkN9CHQd2poi76cp0AkGyEOKw/s1600/layers.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUZhz9j-QRCiIxx9Ns0iNOOr50HtTftTIfhaQ_TZJxAskQBrZ_R94wSFBHFv8ONK9PTqGThNzqKX2vTnYfb4G9NvTe505WfG1NgwC9WXjwW-FjRtkN9CHQd2poi76cp0AkGyEOKw/s320/layers.png" width="320" /></a></div>
<br />
<span style="background-color: white;">Note that the force that is keeping the application layer and the domain layer apart is reusability (real or imaginary) of the domain layer in case 1, and a more technological separation in case 2.</span><br />
<br />
<b>Thou shall have no stupid objects</b><br />
Or perhaps thou shall. At some point people realize that we can't easily obtain the shapes above by using intelligent objects.<br />
<br />
What is coming out from the service layer, once the details of delivery (SOAP, JSON, whatever) have been removed? Not domain objects, as the service layer has to talk #only# to the application layer.<br />
<br />
What is going in / coming out of the persistence layer? Not domain objects, as persistence layer can't talk to the domain layer.<br />
<br />
What is going back to the service layer? Once again, not domain objects, because the service layer can only talk to the application layer. Not service-aware classes, as the application layer can't create service-level objects.<br />
<br />
So, most naturally, people tend do the obvious thing: they add more classes, totally devoid of behavior, because damn, we have already allocated all the possible behavior in those !@#$ layers. Those classes go by many names, like DTO (data transfer object) or even Entity in some Microsoft-oriented shops (not to be confused with Entities in DDD, of course, just like the ValueObject in DDD should not be confused with stupid DTO-like things). Some (myself included) at some point see the sadness of writing those fake classes and try to adopt hashmap-like things, or dynamic objects in .NET, or whatever. Not a quantum leap, anyway, and fiercely opposed by some people anyway.<br />
<br />
The funny thing is that this kind of architecture is often sanctioned by gurus. You can read what the ubiquitous Robert Martin has to say by starting from <a href="http://blog.8thlight.com/uncle-bob/2011/11/22/Clean-Architecture.html" target="_blank">here</a>, although it's much clearer if you watch the entire video (entertaining, albeit a bit slow) <a href="http://mchenry.softwarecraftsmanship.org/the-a-word-a-discussion-about-architecture" target="_blank">here</a>. The slides in the first post can get you up to speed quickly (if you can fill the gaps; btw, the ".key" file is actually a ".zip" file).<br />
<br />
Martin is actually right when he says that (ideally, at least) the "delivery mechanism", (aka the service layer) should be considered a detail. However, note how quickly he falls into the idea that you need stupid objects to carry around information. Indeed, at some point he declares that "it is ok for objects to simply hold some data". Hmm, ok, if you say so. Except that NO, IT'S NOT OK. Just because you're doing it, or know no better way to get what you want, does not make it ok.<br />
<br />
Guru-driven design is dangerous. Gurus have to know best, so if they can't do something, that something is irrelevant. If something they do has consequence, those consequences are ok. For instance, a guru can claim that in DDD is ok to have duplication (entanglement) between two bounded contexts, but not within a bounded context. Now, that's plain wrong, but people will accept it because you're a guru, there is no theory of forces to prove you wrong, and unlike in the physical world, the material won't react in a powerful way when you're wrong (try to negate the existence of gravity by jumping out the window instead :-). Of course, the guru could instead say that it's just the best trade/off he knows, but that's less guru-ish than ideal.<br />
<br />
The actual danger is that people will stop trying. After all, the gurus say it's ok. So there is no need to explore new materials (AOP, a mixin revival, structural conformance, whatever) or to explore new ways of shaping traditional materials (as I'll do in a moment). It's very much like having a guru saying that it's ok for a bulletproof window to be totally opaque, because he doesn't know how to make bulletproof glass. Except that I don't like the idea of people spending life in darkness. So, sorry, it's not ok.<br />
<br />
<b>In this corner... the mapper!</b><br />
Of course, when you condemn smart people to a never-ending pile of layers, all entangled on the same domain concepts, passing around stupid data structures, they can't just pretend not to see it. They won't challenge your architecture, because you're the guru, but they'll try to avoid some drudgery by writing smart code.<br />
<br />
For various reasons, the only techniques with a dim chance of success here are reflection and generative techniques, and at some point everyone comes up with a concept of Mapper (emphasis on -er), something that can somehow find "equivalent" fields in similar object and clone them. Given enough motivation, these things get beefed up and turned into full-fledged libraries, for popular joy.<br />
<br />
Introducing "mappers", hopefully semi-automatic classes, provides some relief for coders, but is only a coping technique. For instance, <a href="http://www.carlopescio.com/2010/09/notes-on-software-design-chapter-10-run.html" target="_blank">run-time friction</a> kicks in anyway. In a very interesting dissertation (<a href="http://www.ics.uci.edu/~guoqingx/papers/xu-phd11.pdf" target="_blank">Analyzing Large-Scale Object-Oriented Software to Find and Remove Runtime Bloat</a><span style="background-color: white;">), Guoqing Xu found that a lot of inefficiency stems exactly from long "copy chains". Does it ring a bell?</span><br />
<br />
Still, I don't want to challenge the established architectures, so here is the modest contribution of this long post: to tell you guys a possible way to keep those layers but let go of stupid objects (and henceforth of mappers). I don't claim originality: it's very possible that many of you guys are already using this style. I came up with this a few years ago, mostly while trying to mediate between the desire of some of my clients to go with the "standard" layered architecture and my disgust for stupid objects. I haven't seen this style adopted and pushed to its real potential elsewhere, but it's totally possible that it's commonly adopted by some and that I just had no chance to see it used. It's based on the semi-layered architecture where the domain layer <b>can't</b> see the persistence layer, but is a very general idea.<br />
<br />
<b>On truly understanding your material</b><br />
An important concept in the physics of software is that we have <b>run-time things</b> (like objects, with their memory footprint, and actual function calls, with parameters being passed and CPU cycles eaten) and <b>artifacts</b>, like source files, where we describe those things, their structure, their behavior.<br />
<br />
In practice, we expect some run-time properties and some artifact-side properties, but traditional literature has never taken time to explain <b>where</b> we want something to happen. For instance, where do we want layering to happen? Where do we want isolation of change? We want that <b>on the artifact side</b>, of course. When a decision changes, we want to change only one folder of source files (layer). We may also want to reuse an artifact elsewhere, as a source file or as a compiled library or as an executable. We actually don't care about run-time separation at all (to some extent - I'll add something about this in the "critics" section).<br />
<br />
Still, people (including gurus) tend to get trapped into a linear form of thinking. If artifact A (say, an Application-level class) can't talk to artifact P (say, a Persistence-level) class in terms of class D (say, a domain-object class), then I have to introduce another class, and therefore, a new run-time object has to be created and then "mapped".<br />
<br />
In fact, the emergence of stupid DTOs as a consequence of separation of artifacts is largely due to an underlying (wrong) assumption: <b>that artifact separation requires run-time separation</b>. This, however, is far from true, although it may require different programming techniques, depending on your language. From another perspective, the idea may suggest useful concepts we may want to be supported by our programming languages - this is akin to <a href="http://en.wikipedia.org/wiki/Materials_science" target="_blank">Materials science</a>, and the idea of creating a new material based on the forces it has to withstand; contrast this with how many languages are created, based on notions of "purity" instead of a sound theory of forces.<br />
<br />
<b>Application talking to Persistence without DTOs</b><br />
Say that you have an Application object, and this object has somehow obtained a Domain object (we'll see how in a moment). Say that you want to persist the Domain object, but unfortunately, the Persistence layer can't know about your Domain object in a layered architecture. Is there anything better than creating a DTO by basically cloning the Domain object (and then claiming that it's ok to have stupid objects)?<br />
<br />
Well, if you think about it, while it's not ok for an object to exist only for the purpose of containing data, a Domain object may want to offer an <b>interface</b> to read some of those data. After all, when we accepted technology as a principal decomposition, we also accepted some kind of entanglement on domain concepts, and that interface is there exactly to represent the R-entanglement we got.<br />
<br />
Suppose I've got a User class (a domain concept) with some domain logic, like telling me whether or not today is the user's birthday. I could easily do this (example in C#, but without the I in interfaces, so it looks like Java :-)<br />
<br />
<span style="font-family: Verdana, sans-serif;">// the R-entanglement interface</span><br />
<span style="font-family: Verdana, sans-serif;">interface UserData</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> int Key { get; }</span><br />
<span style="font-family: Verdana, sans-serif;"> DateTime dateOfBirth { get; }</span><br />
<span style="font-family: Verdana, sans-serif;">}</span><br />
<span style="font-family: Verdana, sans-serif;"><br /></span>
<span style="font-family: Verdana, sans-serif;">// the Domain/Business object</span><br />
<span style="font-family: Verdana, sans-serif;">class User : UserData</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> // UserData implementation stuff</span><br />
<span style="font-family: Verdana, sans-serif;"><br /></span>
<span style="font-family: Verdana, sans-serif;"> // some business logic</span><br />
<span style="font-family: Verdana, sans-serif;"> public bool IsBirthday()</span><br />
<span style="font-family: Verdana, sans-serif;"> {</span><br />
<span style="font-family: Verdana, sans-serif;"> // ...</span><br />
<span style="font-family: Verdana, sans-serif;"> }</span><br />
<span style="font-family: Verdana, sans-serif;">} </span><br />
<span style="font-family: Verdana, sans-serif;"><br /></span>
<span style="font-family: Verdana, sans-serif;">// the Persistence layer, using the Repository pattern</span><br />
<span style="font-family: Verdana, sans-serif;">class UserRepository</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> public void Store( UserData u )</span><br />
<span style="font-family: Verdana, sans-serif;"> {</span><br />
<span style="font-family: Verdana, sans-serif;"> // I just want to read the data and issue an insert</span><br />
<span style="font-family: Verdana, sans-serif;"> }</span><br />
<span style="font-family: Verdana, sans-serif;">}</span><br />
<br />
well, that was easy. Just because I don't want my Repository to depend on my domain object, doesn't mean I have to clone data before I pass it in. I can just use an interface (which does not need to live in the Domain layer).<br />
Things get much harder the other way, though, and I guess here is when people turn back to DTOs:<br />
<br />
<span style="font-family: Verdana, sans-serif;">class UserRepository</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> public UserData Get( int key )</span><br />
<span style="font-family: Verdana, sans-serif;"> {</span><br />
<span style="font-family: Verdana, sans-serif;"> // ????</span><br />
<span style="font-family: Verdana, sans-serif;"> }</span><br />
<span style="font-family: Verdana, sans-serif;">}</span><br />
<br />
Of course, it's not enough to give UserData the "set" method for properties. Of course we have to do that, but the real issue is that my repository has to <b>create</b> the object, and you can't create an interface, and you don't want to create a domain object inside the repository (because you don't want to know the domain object). Note that even creation alone (without usage) would break the layer shape, if you need to put a dependency between your artifacts.<br />
<br />
Now you need to know your material well enough to solve this problem, as there is no language-independent solution. I'll give you the C# solution; in C++, you can basically use the same thing (you don't even need the interface as it is inferred). In Java you can't, so you have to bring in the heavy artillery, but most likely we're gonna do it in C# as well (more in a minute).<br />
<br />
<span style="font-family: Verdana, sans-serif;">class UserRepository</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> public T Get< T >( int key ) where T : UserData, new</span><br />
<span style="font-family: Verdana, sans-serif;"> {</span><br />
<span style="font-family: Verdana, sans-serif;"> // ... get data from the database</span><br />
<span style="font-family: Verdana, sans-serif;"> T u = new T();</span><br />
<span style="font-family: Verdana, sans-serif;"> // fill u with data (needs "set" methods in UserData)</span><br />
<span style="font-family: Verdana, sans-serif;"> return u;</span><br />
<span style="font-family: Verdana, sans-serif;"> }</span><br />
<span style="font-family: Verdana, sans-serif;">}</span><br />
<br />
and usage in the application layer would just be:<br />
<br />
<span style="font-family: Verdana, sans-serif;">class MyWonderfulUseCase</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> public void DoSomethingUseful()</span><br />
<span style="font-family: Verdana, sans-serif;"> {</span><br />
<span style="font-family: Verdana, sans-serif;"> // ...</span><br />
<span style="font-family: Verdana, sans-serif;"> User u = userRepository.Get< User ></span><span style="font-family: Verdana, sans-serif;">( key ) ;</span><br />
<span style="font-family: Verdana, sans-serif;"> bool b = u.isBirthday(); // u is a full-fledged domain object :-)</span><br />
<span style="font-family: Verdana, sans-serif;"> // ...</span><br />
<span style="font-family: Verdana, sans-serif;"> }</span><br />
<span style="font-family: Verdana, sans-serif;">}</span><br />
<br />
Look ma, no stupid objects. No mappers. No unwanted dependencies between layers (as artifacts). No friction. No hashmaps. All type safe. No casts either.<br />
<br />
<b>What was that?</b><br />
Forget the language for a moment. All I did was to separate the artifacts without separating the instances (run-time things). I did that through interfaces + generics, because in C# and C++ generics / templates are powerful enough for this sort of things. But generally speaking, what I want to do is:<br />
<br />
- define an interface to access data (or two, if you want read/write separation).<br />
<br />
- communicate between layers in terms of that interface (removing the need for a DTO).<br />
<br />
- define a mechanism for the "lower layer" to create objects in the "upper layer" without having to know their classes. In this case, I used generic programming.<br />
<br />
Generally speaking, you may want to use an Inversion of Control Container instead. Besides working in Java as well, the IoC will be helpful in a variety of cases (like handling inheritance) where generics just won't work. In this case, the code above would become something like (using Unity in C#):<br />
<br />
<span style="font-family: Verdana, sans-serif;">class UserRepository</span><br />
<span style="font-family: Verdana, sans-serif;">{</span><br />
<span style="font-family: Verdana, sans-serif;"> public UserData Get( int key )</span><br />
<span style="font-family: Verdana, sans-serif;"> {</span><br />
<span style="font-family: Verdana, sans-serif;"> // ... get data from the database</span><br />
<span style="font-family: Verdana, sans-serif;"> UserData u = Container.Resolve<userdata>();</userdata></span><br />
<span style="font-family: Verdana, sans-serif;"> // fill u with data (needs "set" methods)</span><br />
<span style="font-family: Verdana, sans-serif;"> return u;</span><br />
<span style="font-family: Verdana, sans-serif;"> }</span><br />
<span style="font-family: Verdana, sans-serif;">}</span><br />
<br />
There is another reason to switch to an IoC: the generic version is a drag when your objects contain other objects which contain other objects and they all come up with a single query, perhaps through a stored procedure. The IoC has no problems with that. The generic version requires you to specify all types in the instantiation, which is not so nice (but not an insurmountable problem in many cases).<br />
<br />
Note, however, that by removing the generic parameter, I have abstracted things a little too much. I wanted a User, but I'm getting back a UserData. So my application code would become:<br />
<br />
<span style="font-family: Verdana, sans-serif;"> User u = userRepository.Get( key ) as User ;</span><br />
<span style="font-family: Verdana, sans-serif;"> bool b = u.isBirthday(); // u is a full-fledged domain object :-)</span><br />
<br />
which may not look like a cast, but is cast anyway. This can be fixed to some extent, if you add another interface on top of your domain class (which could still be useful for mocking) and combine the generic version above with the IoC. You'll still have a cast, but only in one place (the generic function).<br />
<br />
Of course, the gurus will readily tell you that your architecture should not depend on an IoC. That's a rather narrow view of things. I don't need an IoC. I could go with a factory. I also don't need polymorphism. I could go with function pointers. Except I'm using polymorphism, thank you. I'll also be happy to use a language-provided mechanism once you get me a decent one :-). In fact, the idea that most programming languages don't readily support decoupling instance creation from class name at the artifact level is appalling, and it's just another sign of why we need a physics of software. The IoC is a <a href="http://www.carlopescio.com/2006/12/infrastructure-and-superstructure.html" target="_blank">superstructure</a>. I want an infrastructure.<br />
<br />
In practice, moving outside this limited case, it would be even more beneficial to have mixins, so that we can mix behaviors into an object. I need run-time mixins though. This is a long story and perhaps I'll say something about it in the future, probably using the only widely adopted language I know where I can express those things (Javascript: ain't that ironic? :-). The design space out there is far less constrained than the usual gurus may lead you to believe. Don't fall into habit. Think think think.<br />
<br />
<b>What about the other "mappers"?</b><br />
Or about the other stupid objects? If you look at Martin's example, for instance, he has a ResponseModel class (actually, many: at least one per service) carrying only data. In practice, we tend to fall into one of these cases:<br />
<br />
- that class is basically cloning the interface of some business object. There is no smart to add because the infrastructure will take care of the rest (for instance, in a .NET WebApi service, the JSON serializer [sic] is smart enough to send back my stuff without me adding behavior. Therefore, pass back the xxxData interface instead, or the read-only version if you like so (you may recognize this as similar to storing something by passing a read-only interface, as above; after all, why should these two things be different at all?)<br />
<br />
- there is significant behaviour that could be allocated there, but it's on the service side. For instance, the class represents an error, and you want to convert it into an HTTP status + some json, and you need custom code. It's exactly the same issue we have between Persistence and Domain, except that now is the Application which has enough information (the error, the message) to build the object, but can't create the object because its class lives across the border. Just use the same trick: get a writeable interface from your IoC (or whatever), fill with data, pass the thing to the Service layer, which will happily cast it to an intelligent object. Here I think a cast is unavoidable (and no, I'm not weeping in fear).<br />
<br />
How do you pass data from the service layer to the application layer is left as an exercise for the reader :-). In practice, this depends a lot on your technology stack, how far is already trying to go (model binding) and how well does it play with interfaces, IoCs, and the like. If it doesn't, I would say it's more of a limitation of a specific implementation than a fault of the general principle. Of course, some guru will tell you that your Model is not your Domain, and that you need just another layer. Oh well. Where is YAGNI when I need it?<br />
<br />
<b>Critique</b><br />
In practice, I've heard three distinct critiques (plus one):<br />
<br />
- it requires an IoC. Funny enough, this is usually coming from people already using an IoC :-), but for other, more guru-sanctioned things like instantiating a logging service (which they never ever change). As I said, I don't need an IoC. I need a flexible way of separating creation from static knowledge of a class name. The IoC is just a reasonable way to get that.<br />
<br />
- I may then call business logic inside the persistence layer (as I'm creating a business object). Yes, but we'll spot you as you'll have a dependency on the business package. Sure, you can try to defeat the checking through reflection. But then, you can always load an assembly at run-time using reflection as well. As good Bjarne used to say, we want to protect ourselves from Murphy, not from Machiavelli.<br />
<br />
- I don't mind writing all those structures and mappers. It's simple code. It's relaxing. Now, this may sound stupid, and I can honestly say that I felt like saying something very impolite when I first heard this argument. On retrospective, I understand the feeling. Technically, it's work, but it's rote work, so you can turn off your brain for a while, and nobody can say anything because hey, someone gotta write that code. In a high-stress environment, having something brainless to do can give you a legitimate break. That said, I don't like the argument, because it's corrupting the design to cope with a problem in a very different area (management). It's not my style, sorry.<br />
<br />
- a small variant to the above is that I'm leaving some choices open (like, should I return a readable interface to the service layer, or create a service-layer instance from the application layer using an IoC). What's worse, I'm not mandating a style: you may want to use one thing or the other, depending on context. But that requires you to think, which is the opposite of the brainless coding above. Sorry. I'm into #braindrivendesign. There is no way around that :-).<br />
<br />
What about testing? Right, someone has gotta play the T card at some point (this is the "plus one"). But actually, this technique plays very well with all your testing needs, as everything is interface-based and can be easily mocked. You don't need stupid object to be test-friendly.<br />
<br />
However, I'd like this post to contribute to Brain-Driven Design, not to Guru-Driven Design. So, don't do this just because I said so. Just remember it can be done; you may not have tried this shape before: give it a try when you see a good fit, and see what happens. If, instead, you find yourself thinking "no no no no" and spending all your mental energy trying to find faults in this techniques, and not a single moment pondering on what's good, well, as I said above, it could be a nice step into self-awareness to stop and think why (I did this exercise many times in my life :-).<br />
<br />
<b>The end</b><br />
Honestly, I don't think the layered shape is a good fit for LoB applications. As I said, very few systems can be built with a <b>real</b> layered shape, and most often, there is a significant difference between what is told/drawn and what is done (code). The most natural shape for OO systems is a network of services, and a very interesting thing would be to redefine the database as a provider of services, not merely data. This would also be more aligned with the need of distributed technologies and map / reduce (and therefore, make your [ex]persistence layer more robust in face of technology changes). It's actually easier than you think, and sometimes all we need is a small change in our queries. I'll touch on this in my next post, provided I don't forget :-).<br />
<br />
Still, even with the heavy constraints of a semi-layered architecture, we can avoid stupid objects without losing separation of concerns (which is an artifact thing, not a run-time thing). Note that I didn't push (as some have done over the years) any presentation responsibility into the business objects (yes, a RenderInXml method is still a presentation responsibility). I've kept the same separation of concerns we have in traditional, DTO-based systems.<br />
<br />
In the end, although I hope you guys enjoyed this post, I wrote all this just so that next time I'll be able to write code where domain objects come freely out of repositories, without being accused of breaking separation of concerns. So, next time I'll go back to the forcefield, and see what is telling about our unstable User class :-).<br />
<br />
<span style="background-color: #fefeff; color: #303030; font-family: 'Trebuchet MS', Trebuchet, sans-serif; font-size: 16px; line-height: 22px;">If you read so far, you should </span><a href="https://twitter.com/#!/CarloPescio" style="background-color: #fefeff; color: #538cd6; font-family: 'Trebuchet MS', Trebuchet, sans-serif; font-size: 16px; font-weight: bold; line-height: 22px; text-decoration: none;" target="_blank">follow me</a><span style="background-color: #fefeff; color: #303030; font-family: 'Trebuchet MS', Trebuchet, sans-serif; font-size: 16px; line-height: 22px;"> on twitter.</span>Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com33tag:blogger.com,1999:blog-13967713.post-57254172561157049222012-05-16T11:07:00.000+02:002012-05-16T11:07:17.959+02:00Notes on Software Design, Chapter 16: Learning to see the ForcefieldWhen we interact with the physical world, we develop an intuitive understanding of some physical forces. It does not take a PhD in physics to guess what is going to happen (at a macroscopic level) when you apply a load at the end of a cantilever:<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3GmbBlzIxeFQCcapgv2boYeFKvh7uVCwxr54_S-6_T5CuAd_SRltzmTfTu4dk38vr_tyewgzM3STORppTFTjL23BGC9k0W7XIcTOFsE1UoB5aMRdePdZ7dXtMEZN-Uf9V1-MG0w/s1600/cantilever.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="190" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3GmbBlzIxeFQCcapgv2boYeFKvh7uVCwxr54_S-6_T5CuAd_SRltzmTfTu4dk38vr_tyewgzM3STORppTFTjL23BGC9k0W7XIcTOFsE1UoB5aMRdePdZ7dXtMEZN-Uf9V1-MG0w/s320/cantilever.png" width="320" /></a></div>
<br />
<br />
You can also devise a few changes (like adding a cord or a rod) to distribute forces in a different way, possibly ending up with a different structure (like a truss):<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBYKv9x6RDaPtQDx2r2sPVroa0KiQKgH8so6BqTr-jn03cRtsZnTIVND2_XXT82k6ff-anBcpzUszP4jAMnXHSLHBER26Ln3U2Gx3J_UpBjvFvkBVxzlpQHeMtup71HotT05q8BA/s1600/truss.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="157" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBYKv9x6RDaPtQDx2r2sPVroa0KiQKgH8so6BqTr-jn03cRtsZnTIVND2_XXT82k6ff-anBcpzUszP4jAMnXHSLHBER26Ln3U2Gx3J_UpBjvFvkBVxzlpQHeMtup71HotT05q8BA/s320/truss.png" width="320" /></a></div>
<br />
<br />
Software is not so straightforward. As I argued before, <a href="http://www.carlopescio.com/2010/04/notes-on-software-design-chapter-0-what.html" target="_blank">we completely lack a theory of forces</a> (and materials). Intuitive understanding is limited by the lack of correlation between form and function (see <a href="http://www.dreamsongs.com/Files/Form&Function.pdf" target="_blank">Gabriel</a>). Sure, many programmers can easily perceive some “technological” forces. They perceive the UI, business logic, and persistence to be somehow “kept apart” by different concerns, hence the popularity of layered architectures. Beyond that, however, there is a gaping void which is only partially filled by tradition, transmitted through principles and patterns.<br />
<br />
Still, I believe the modern designer should develop the ability to <b>see </b>the force field, that is, understand the real forces pulling things together or far apart, moving responsibilities around, clustering them around new concepts (centers). Part of my work on the <a href="http://physicsofsoftware.com/" target="_blank">Physics of Software</a> is to make those forces more visible and well-defined. Here is an example, inspired by a recurring problem. This post starts easy, but may end up with something unexpected.<br />
<br />
<a name='more'></a><br />
<b>I just want to log in</b><br />
Let's start with a simple problem. We want to authenticate a user through the familiar id / password combination. It may be natural to start with something like this (in a Java/C# like syntax; it doesn’t really matter, of course):<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;">class User</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> {</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> public bool Authenticate( string uid, string pwd ) { … }</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> // more stuff …</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> }</span><br />
<br />
Unfortunately, that's not a brilliant OO design. You have to create a User before you can call Authenticate. But then, that user would be invalid until authentication (and after, if it fails). Objects are not supposed to be invalid: the constructor should give you back a valid object. This may trigger another common form:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;">class User</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> {</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> public User( string uid, string pwd ) { … }</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> // more stuff …</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> }</span><br />
<br />
However, now you need to throw an exception when validation fails, which may or may not be nice, according to your view on exceptions.<br />
<br />
The problem is simple in hindsight: we’re putting a responsibility in the wrong place. I think it’s easier to see that when we look at a larger picture and realize that authentication through uid / pwd is just one of many. In the average website, you may want to add facebook authentication or twitter authentication. In a banking environment, RSA token authentication. Etc.<br />
<br />
<b>Unstable sets</b><br />
The decision of using a specific authentication technique is unstable: there is an open multiplicity of authentication techniques. Now, in the physics of software,<b> if you map an unstable set into a single software element, that element will be unstable as well</b>. In our case, that means User is going to be unstable. You have to crack it open and change it whenever you need a different authentication technique, with different parameters and different logic.<br />
<br />
More formally: authentication techniques form an unstable set, which we could write down as {UidPwd, Facebook, Twitter, RSA, ... }. User is U/U entangled with that set (when you Update the set, User must be Updated as well). Instability of a set is propagated through U/U entanglement; this is just another aspect of the <a href="http://www.carlopescio.com/2011/02/notes-on-software-design-chapter-14.html" target="_blank">Enumeration Law</a> (which in a more formal treatment could be expanded to better reflect cases like this). Moving to pictures, I tend to visualize this sort of scenario like:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif1_pbFHNZhyphenhyphenE99_8T0pnKPb0Yyh-_Aaz5GUSFSGII0jKfPgg99bNe-Hx5IR5K-v4YpGoZQYnppTkHg-H9cB6DUcoUzETymlUxbksD8uh0T5pCzEWdnSzWF5sYZlI4VNfCoT4kzA/s1600/ff1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="253" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif1_pbFHNZhyphenhyphenE99_8T0pnKPb0Yyh-_Aaz5GUSFSGII0jKfPgg99bNe-Hx5IR5K-v4YpGoZQYnppTkHg-H9cB6DUcoUzETymlUxbksD8uh0T5pCzEWdnSzWF5sYZlI4VNfCoT4kzA/s320/ff1.png" width="320" /></a></div>
<br />
Although it’s easier, once we understand each other, to depict it like this:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcUA_VicEYzXiZ5lHLHkPW44HFgApd8sPJEMidkiVCGOcfcTFSlfVhwlJsIsOryBHVIK_2R6qeJdIZQ1oxD0FOj_KhGrBrwhsgArELBujceqvyhvbYsvuHowquJZ0hUGUTA_04dA/s1600/ff2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcUA_VicEYzXiZ5lHLHkPW44HFgApd8sPJEMidkiVCGOcfcTFSlfVhwlJsIsOryBHVIK_2R6qeJdIZQ1oxD0FOj_KhGrBrwhsgArELBujceqvyhvbYsvuHowquJZ0hUGUTA_04dA/s320/ff2.png" width="320" /></a></div>
<br />
That’s not a nice shape. However, to the trained mind, the forcefield is actually suggesting a structural change:what is one must become many.<br />
<br />
<b>From Authentication to Credentials</b><br />
The decision to support multiple authentication schemes is acting as a force on our (software) material. To align the shape of our material with that force (or, as Christopher Alexander said in his Notes on the Synthesis of Form, to put it <i>in frictionless contact with the forcefield</i>), we need to separate the authentication strategy from the user. We also need to avoid the creation of a new, single center with the same kind of U-U entanglement. Fortunately, within the OO paradigm, an unstable, unbounded set suggests the usual adoption of an inheritance hierarchy:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBiSfzSRfHdLY8CrEw3i6ZsKFk7aIudsD1UCgyj6c9o0jn25yu7Ub31dwSNIvLcAf3TOJcylPKrVG0wApiHC24nH4sBFaqr8SgMazOWk_JkOehxWZsG4zvKRF7QRqNQa6ZHGwY6w/s1600/credential1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBiSfzSRfHdLY8CrEw3i6ZsKFk7aIudsD1UCgyj6c9o0jn25yu7Ub31dwSNIvLcAf3TOJcylPKrVG0wApiHC24nH4sBFaqr8SgMazOWk_JkOehxWZsG4zvKRF7QRqNQa6ZHGwY6w/s1600/credential1.png" /></a></div>
(Yeap, I'm moving to UML; it’s not a taboo :-)<br />
<br />
Although some would have called the base class Authentication, I choose Credential, which suggests an object (not an action). This will also avoid the naming antipattern whereby you have class A with an A-ish method inside, like Authentication.Authenticate.<br />
<br />
Every concrete Credential class will require a custom constructor, which is fine. After all, they need different parameters. They will also need different logic, which is fine; it’s polymorphism 101.<br />
<br />
Switching to the more formal side once again, in the Physics of Software inheritance is a <b>dampening mechanism</b> (one of the many) which can be used to prevent propagation of an unstable decision. I tend to visualize this new shape like:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR-7n_ava4qkvHh_xAwMHBUaY66HhZ0-nkLB-5YZRhV9gWPnYUixFFOEIIrzraaMzmH-qE1AsAL4AXbhnI0vKKCh-t-KKkYQDQ7wZYUXK0sIM5VfKP5Mi4JAPaVrn_sVOJ4kDELw/s1600/ff3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR-7n_ava4qkvHh_xAwMHBUaY66HhZ0-nkLB-5YZRhV9gWPnYUixFFOEIIrzraaMzmH-qE1AsAL4AXbhnI0vKKCh-t-KKkYQDQ7wZYUXK0sIM5VfKP5Mi4JAPaVrn_sVOJ4kDELw/s320/ff3.png" width="311" /></a></div>
<br />
Or, once we got the idea, like this:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhz3AG4ffEzwtA68xLPIBsoqwMU0WzWZRsEDrVH2PdvjBp7Q0JJ3EAwIwhzLCvRVmReKdUXeKGuZFGfETDn4ahT-lhYD70JwInpchyzBr0rVs_NgTW7MhKwb83LFdZwq4YE2tHZGw/s1600/ff4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhz3AG4ffEzwtA68xLPIBsoqwMU0WzWZRsEDrVH2PdvjBp7Q0JJ3EAwIwhzLCvRVmReKdUXeKGuZFGfETDn4ahT-lhYD70JwInpchyzBr0rVs_NgTW7MhKwb83LFdZwq4YE2tHZGw/s320/ff4.png" width="311" /></a></div>
<br />
this shape is ok, as it suggests that instability is dealt with by increments, not changes. We now have two U/U entangled sets (conceptual authentication techniques and materialized Credential classes). The shape of our software is aligned with the forcefield.<br />
<br />
<b>Opposites, or: your domain is lying to you</b><br />
It is now time to connect User and Credential. Now, if you think for a moment in the domain language, you have the obvious notion that<b> a User has one or more Credentials</b>, so you may end up with this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5TggO0U667pQ7SmY9WiYE4h3NEup9UpEBXCQCYrBycIsw-JA-iNMJ0h0Yeb9htXMblKDWVpD-blHnRiMHeR0WYi25OSrogFFrANbwcoM4zfE1tCIP46en-lt0SOcRUE8Yk2UnPg/s1600/credential2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5TggO0U667pQ7SmY9WiYE4h3NEup9UpEBXCQCYrBycIsw-JA-iNMJ0h0Yeb9htXMblKDWVpD-blHnRiMHeR0WYi25OSrogFFrANbwcoM4zfE1tCIP46en-lt0SOcRUE8Yk2UnPg/s1600/credential2.png" /></a></div>
<br />
I truly hope you <b>feel some discomfort</b> by looking at that shape :-). It seems like, in order to get a Credential object, we need to go through User. Doesn’t that expose us to the same validity problem we had before? Also, we’re not really expecting user to create the concrete Credential object, are we? Because that would bring us back to the same unpleasant shape above, with one center (the User) entangled with an unbounded set (the concrete Authentication classes).<br />
<br />
Fortunately, it's all very simple. At some point (I’ll get back to this in the next paragraph) the flesh-and-bones user will choose an authentication method. The actual workflow will probably be different for different authentication methods, but in the end, you’ll have the right parameters to build a concrete Credential object and invoke the Authenticate method. What is the right service you'd like the Credential to offer? Of course, <b>to either return a (valid) User, or some kind of error</b>. Right: <b>Credential has to know the User</b>, not vice versa. Actually, the user has no need whatsoever to know about the Credential.<br />
<br />
This will bring us to a favorable force field, with an unbounded set (the credentials) all entangled with a single center (the user):<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs3LHH2v3y4Kh5r5pSQW5fHSywsl1QRKiGOevdSJviKqCyoXLlj_NAGdWVYt5c7fbWGh9pRRpokxKrA-OWrxRSbt7Ltz4scmGX8Ca7aAHYTDQkGemI7OfRPC_4Ogba7qoxWoEL1w/s1600/ff5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="253" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs3LHH2v3y4Kh5r5pSQW5fHSywsl1QRKiGOevdSJviKqCyoXLlj_NAGdWVYt5c7fbWGh9pRRpokxKrA-OWrxRSbt7Ltz4scmGX8Ca7aAHYTDQkGemI7OfRPC_4Ogba7qoxWoEL1w/s320/ff5.png" width="320" /></a></div>
<br />
<br />
Please, spend a minute to consider how simple and well aligned with the problem this is:<br />
<br />
- You never have an invalid user or an exception (unless you want to; Authenticate can be exception-based)<br />
<br />
- Concrete credential classes can be added at any time, without touching the user<br />
<br />
- Actually, the user has no notion of authentication, which is a remarkable separation of concerns. Not the kind you’ve been told all the time (UI / logic / persistence). The hard kind: logic / logic / logic :-). Actually, it's more than separation of concerns. It's why I call <b>obliviousness of concerns</b>.<br />
<br />
We just need to make sure we don’t waste all this at the service level: more on this in a moment.<br />
<br />
A quick note on the “your domain is lying to you” thing, before the DDD crowd reaches for the gun. In my view, Design is about understanding forces and shaping a solution which is in frictionless contact with the forcefield. Domain modeling helps understanding <b>some</b> of those forces. But domain modeling is a beginning, not an end in itself. Just because the domain is suggesting an arrow, doesn't mean you need that arrow, or that you need it in that direction.<br />
<br />
When I teach Object Oriented Analysis (not very frequently: <a href="http://www.carlopescio.com/2011/10/youre-solving-wrong-problem.html" target="_blank">analysis is dead</a>) I recommend avoiding specifying navigability in associations. In analysis, associations are conceptual links, and conceptual links are bidirectional by definition. It used to be that navigability would be chosen during design, with a focus on the nonfunctional properties of our solution. When you compress analysis, design, and perhaps coding together, it's somewhat easier to get an arrow wrong, and then stick with it (not seeing the forest for the trees, I guess). Just be smart. Dig deeper. Use your brain. That's why I call it <b>#braindrivendesign</b>.<br />
<br />
<b>There is no Authentication Service, or: avoiding the hourglass forcefield</b><br />
OK, if you didn’t get an anaphylactic shock till now, it is time to bring this thing to its logical end. Well, almost. I’ll save the real epiphany for the conclusions.<br />
<br />
If you think about the UI for a moment, you probably need different pages, or widgets, or whatever, to carry out authentication with different credentials. After all, parameters are different; the workflow is different, etc. This is even more obvious if you also consider biometric credentials and the like. So, let’s say that you have a different widget or page for every concrete credential, possibly with some code reuse between some of them. So far, so good.<br />
<br />
Say that you have a service layer. Many modern web apps have a service layer. It might be tempting to define an Authentication service as a class and expose all the different authentications as methods. That’s plain wrong, of course. The ultimate shape would look like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgw0mdCIdv1cFf2R6TAWOFfLsR3pzC848d-0E-QRhvuTawjC3vPYQ5AVrBR1aXtQvQtxhkJF5dpywaZ30gTOxZ43Q-TB0R4sZsLqRGSTd5x2dAk709XpfH3wzCSVgKq1-tCM9TnTA/s1600/ff6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgw0mdCIdv1cFf2R6TAWOFfLsR3pzC848d-0E-QRhvuTawjC3vPYQ5AVrBR1aXtQvQtxhkJF5dpywaZ30gTOxZ43Q-TB0R4sZsLqRGSTd5x2dAk709XpfH3wzCSVgKq1-tCM9TnTA/s320/ff6.png" width="320" /></a></div>
<br />
That hourglass-like shape is a sort of antipattern again, so very common exactly because we can’t easily see the forcefield. But it should be pretty obvious by now that it would be just replicating the initial problem elsewhere. It’s trivial, and basically effortless, to overcome this issue. Just turn each authentication scheme into an independent service. That’s what the forcefield is desperately trying to tell you. Use an open grouping construct (like namespaces) if you want to keep those classes together.<br />
<br />
<b>Isn’t that just the Open/Closed Principle?</b><br />
Yeah, sure. And a little Single Responsibility Principle too. And a sprinkle of Dependency Inversion Principle. And yeah well that hierarchy calls for some Liskov Substitution Principle as well. Hmmm. I’m missing a bit of Interface Segregation, I guess.<br />
<br />
Seriously guys, I’ll probably spend an entire post about this sooner or later, but although principles are all nice, they are also keeping software design in the Stone Age. It’s time to outgrow principles and move ahead.<br />
<br />
<b>The Consequences of Truth</b><br />
If we accept the idea that this shape is undesirable:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif1_pbFHNZhyphenhyphenE99_8T0pnKPb0Yyh-_Aaz5GUSFSGII0jKfPgg99bNe-Hx5IR5K-v4YpGoZQYnppTkHg-H9cB6DUcoUzETymlUxbksD8uh0T5pCzEWdnSzWF5sYZlI4VNfCoT4kzA/s1600/ff1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="158" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif1_pbFHNZhyphenhyphenE99_8T0pnKPb0Yyh-_Aaz5GUSFSGII0jKfPgg99bNe-Hx5IR5K-v4YpGoZQYnppTkHg-H9cB6DUcoUzETymlUxbksD8uh0T5pCzEWdnSzWF5sYZlI4VNfCoT4kzA/s200/ff1.png" width="200" /></a></div>
<br />
Then we have no choice but to conclude that, moving from User to Person, this code is highly undesirable as well:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;">class Person</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">{</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">string firstName;</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">string lastName;</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">DateTime dateOfBirth;</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">string phone;</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">// …</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">}</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
There is an unbounded set of attributes for a Person (mobile phone; email; living address; working address; place of birth; height; weight; etc), and we have a single class which is U/U entangled with that set. It’s worse than that: even if we do the right thing, group some concepts together and form new classes like Place and Biometrics, Person will still be an unstable center (albeit slightly less so). Note how this instability is also closely correlated with the practical impossibility of building a reusable Person class, which after the initial hype discredited the entire concept of buying (instead of making) your business objects.<br />
<br />
Of course, we all know this. Some (most) choose to ignore the issue. Some try to deal with instability using some coping technique: dynamic objects, hashmaps, etc. Been there, done that. It works, somehow, but it ain’t perfect, and won’t solve some issues anyway.<br />
<br />
However, once we decide to look at the forcefield, there is an obvious answer in there. It’s a bit unsettling, and goes against everything we’ve being doing so far, which is why I’m sure most of you guys will reject it at once.<br />
<br />
Click one of the like/share buttons below if you want to know the entire story (in my next post, that is). Seriously. Stop lurking and click the damn button :-).<br />
<br />
If you liked this post, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me</a> on twitter!<br />
<br />
<br />Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com13tag:blogger.com,1999:blog-13967713.post-11984501058194779002012-03-23T20:27:00.001+01:002012-05-13T16:14:49.788+02:00Episode 2: the Controller Strikes Back<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFImubX6qMohAB1EujENFNJ7QInGukb-l_XGeyuV72gAOKXNorPepAJfkU8xPSCMV7m-FTfSkrlzFYkiD8hyjfNKbg9ZWEJNOKLppU7b5iop5OzLM0CSKKGMmRHwsttrgngRRD9w/s1600/emperor.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFImubX6qMohAB1EujENFNJ7QInGukb-l_XGeyuV72gAOKXNorPepAJfkU8xPSCMV7m-FTfSkrlzFYkiD8hyjfNKbg9ZWEJNOKLppU7b5iop5OzLM0CSKKGMmRHwsttrgngRRD9w/s1600/emperor.png" /></a></div>
This post should have been about power law distribution of class / method sizes, organic growth of software and living organisms, Alexandrian level of scales, and a few more things.<br />
<br />
Then the unthinkable happened. Somebody actually came up with a comment to <a href="http://www.carlopescio.com/2012/03/life-without-controller-case-1.html" target="_blank">Life without a controller, case 1</a>, said my design was crappy and proposed an alternative based on a centralized, monolithic approach, claiming miraculous properties. I couldn’t just sit here and do nothing.<br />
<br />
Besides, I wrote that post in a very busy week, leaving a few issues unexplored, and this is a good chance to get back to them.<br />
<br />
I suggest that you go read <a href="http://www.eptacom.net/blog/pump_controller_post.zibibbo.txt" rel="nofollow" target="_blank">the whole thing</a>, as it’s quite interesting, and adds the necessary context to the following. Then please come back for the bloodshed. (Note: the <a href="http://dl.dropbox.com/u/1839854/pump_controller_post.txt" rel="nofollow" target="_blank">original link</a> is broken, as the file was removed; the "whole thing" link now points to a copy hosted on my website).<br />
<br />
<b>The short version</b><br />
If you’re the TL;DR kind and don’t want to read Zibibbo’s post and my answer, here is the short version:<br />
A caveman is talking to an architect.<br />
<b>Caveman</b>: I don’t really like the architecture of your house.<br />
<b>Architect</b>: why?<br />
<br />
<a name='more'></a><br />
<b>Caveman</b>: you have specialized places I have to go. If I need to take a dump, I have to go to that place called bathroom. If I need to get some food, I need to go to that place called kitchen. What if I want to take a dump while getting some food? I have to move around! <i>Don't you ever get that problem?</i> In my cave, it’s all in one place and I do whatever I want where I want. <i>I really can't see what's wrong with</i> that.<br />
<b>Architect</b>: uh, ok, but please get out of my kitchen.<br />
(Sorry for the caveman language; he’s a caveman).<br />
<br />
<br />
<b>The long version</b><br />
My dear Zibibbo, please allow me to say that I’m actually thankful for your critics. You’re giving me a chance to clarify my thoughts better, and to deepen my analysis of the centralized / monolithic approach. That said, I’m sorry, but I’m going to slaughter your design and code for public amusement.<br />
In what follows, <span style="color: #b45f06;">text in brown</span> :-)) is always a quote from Zibibbo. Whenever possible, I’ll try to quote him in the same order as it appears in the original text. I have quoted small portions, so it would help to read what follows with a copy of his text in sight.<br />
<br />
At the end of this long section, you’ll find a more articulated, less fragmented discussion of some general design issues related to the centralized, monolithic approach. If you want to skip the gruesome argumentation, you can jump down to “Back in reflective mode”.<br />
<br />
<br />
<span style="color: #b45f06;">You yourself mentioned a number of criticisms of your design</span><br />
Yeap. I consider the ability to see any kind of weakness (especially in our own proposals) and strength (especially in other’s proposals) a necessary condition to call yourself a professional designer. I also consider the honesty to disclose them a sign of intellectual integrity, again a trait of professionalism.<br />
<br />
<span style="color: #b45f06;">It basically contains only the abstract mapping from inputs to outputs your control engineer was thinking about. It doesn't implement any main loop, nor it does any I/O.</span><br />
You keep using the term “abstract” all around your text, but in practice the only thing you got is isolation from I/O.<br />
In your perspective, your mapping is “abstract” because it’s defined in terms of “math-like” things like booleans and floats instead of “concrete” things like sensors and pumps. From a methodological perspective, this is very 1980. We used to learn that kind of stuff in CS101 in the past century. Still, the world is moving elsewhere, for a few good reasons:<br />
<br />
- Requirements come in domain terms. Someone is going to ask you to add a redundant pump. Nobody will come asking you to add one more hidden variable to your Markov chain.<br />
<br />
- Math, in the sense of the number-crunching stuff, is largely commoditized. Nobody is writing an FFT from scratch anymore, except the few who are pioneering new technologies (see cuda), waiting to be commoditized anyway. So, <b>math is the low-level stuff in 2012</b>.<br />
<br />
- <b>Domain abstractions</b>, however, are <b>not </b>going to be commoditized. That’s where you should be spending your time, creating meaningful, reusable classes.<br />
<br />
What you don’t seem to get is that a Sensor class <b>is</b> an abstraction (<a href="http://en.wikipedia.org/wiki/Abstraction" rel="nofollow" target="_blank">Abstraction </a>is a reasonably well-defined concept), and that your process is totally about sensors and pumps, not about booleans and floats. You propose exactly the kind of “bad abstraction” that I’ve been talking about more than a few times now. But enough theory. Let’s see what you get <b>in practice</b>.<br />
<br />
Well, in practice there is nothing “abstract” in that mapping. It’s a very <b>concrete</b>, flat mapping between (input + state) to output. Not having to know (inside your logic) if you have two floating sensors or a single probe is an act of abstraction. You don’t do that (until you are forced to); actually, you work against it. As we’ll see, your “abstract logic” is also <b>not </b>decoupled from “low level” concerns, as you call them, like threading. Actually, it’s dictating them (details follow).<br />
<br />
That said, technically speaking, your class is <b>not </b>a controller, as it’s not controlling squat, as you said yourself. It’s best defined as a <b>monolithic model</b>, or monolithic process model to be more precise. So, technically speaking again, what follows is not about controllers, but about monolithic software.<br />
<br />
Indeed, your approach shares several of the disadvantages of a monolithic controller, adds some of its own (as we’ll see), and is no more testable than what I discussed under “the fake-oo way, step 1”, as you get I/O mocking with that simple step as well. It’s an interesting approach, anyway, which I haven’t seen frequently in practice, probably because it’s less safe than a controller (I’ll show you near the end).<br />
<br />
I’ll take this opportunity to highlight the fact that my design was not, as some said, an example of “fat model, light controller”. It’s a <b>no-controller architecture</b>. A fat model, as you perfectly demonstrated, is not a guarantee of proper OO design.<br />
<br />
<span style="color: #b45f06;">Then I would have all the sensor objects call the appropriate methods on the controller each time they detect a change in the input value, and the objects controlling the output devices (motor and alarms) periodically poll the controller to check if the corresponding devices should be active. The corresponding classes can be decoupled from the controller class by using closures, or, equivalently, an event-driven approach</span><br />
This requires some machinery to connect things together, that you neglected to show. Yet that portion requires changes when the wiring changes. Just like the builder in the OO version. Remember that when I get back to this in a little while.<br />
<br />
<span style="color: #b45f06;">The controller class holds all the abstract logic of the system in just one place. If I want to understand at a high level how the system works, I only need to check this one class</span><br />
This is sort of a tautology, which I hear quite often from the monolithic-minded. If you write 100 KLOC in a single function you have a single place to understand and change. Unfortunately, it’s not a genius strategy, but this is a mindset problem, and I’m not planning to change your mind. I understand your position very well. I just don’t agree with it. No large system can be (safely) built without breaking it into small, cooperating parts. Your approach just does not scale (you show that yourself, and I’ll prove it again and again below).<br />
<br />
The problem here is that you’re confusing your personal preference for understanding things by linear reading of intertwined code with a general truth of which you have no proof, while the rest of the world is betting on modularity (not just in software, by the way. Modularity is of key economic value in manufacturing as well, etc.).<br />
<br />
<span style="color: #b45f06;">I'm particularly suspicious, for example, of your GasAlarm class, which, as I understand it, would mix the abstract logic (triggering an alarm when the gas level is critical) with a very low-level concern like the threading model.</span><br />
No, it’s just you trying to read my words in the worst possible way, for your own convenience. I said: <i>“GasAlarm is entirely autonomous, with a Watch() responsibility which may actually be run in its own thread”</i>. I didn’t say (a) that GasAlarm owns the thread, (b) that it’s mandatory to do so. I just said that I have <b>the option</b> to do so.<br />
<br />
You have that option too: you may have your sensors and actuators polled from a single thread, or you could actually have multiple threads. Oh, wait, you can’t :-). Not on the sensor / setXXX side, at least. You would need some kind of synchronized access to the whole <strike>mess</strike> state to prevent inconsistencies. So much for “abstraction from low level details”, right? Your “abstract logic” pretty much <b>decided</b> the threading model, by not being thread safe to begin with. Great job. And it’s not even about thread safety alone: your “abstract logic” breaks if you call the setXXX in a different sequence. I’ll show that later.<br />
<br />
On my side, <b>I like having options</b>. I like the idea that I can reuse a GasAlarm in a different design and provide it with an independent thread. Oh, but you don’t believe in reusability of domain logic anyway. Bummer.<br />
<br />
<span style="color: #b45f06;">The controller class is reasonably small, and it's very simple conceptually. It could be made even smaller and simpler by creating, for example, a tiny helper template class that encapsulates the concept of hysteresis.</span><br />
This is your style again :-), to extract only the mathematical notions. You did that with the bowling too. When <a href="http://www.carlopescio.com/2007/07/get-ball-rolling-part-3-of-4-pretty.html" target="_blank">I pointed out issues</a> with your procedural solution, you answered by proposing a state-machine framework, totally removed from problem domain. I told you what I thought in the <a href="http://www.carlopescio.com/2007/08/get-ball-rolling-part-4-of-4-told-ya.html" target="_blank">subsequent post</a>. It’s better to accommodate domain-related abstraction through proper form than abstracting out the domain into some form of math. I repeated this in my previous post too, and believe it or not, when I wrote about “another path not taken” I was talking to you. You keep looking for abstractions in the wrong place.<br />
<br />
Again, I’ll take this chance to highlight another common mistake in naming classes (no, you didn’t do that; I just forgot to mention it in my previous post). I always try to find meaningful names in the problem domain. Some, for instance, would have called SafePump simply <b>SmartPump</b>. Except Smart means nothing, <b>it’s just another naming antipattern</b>. When someone inherits our code, Smart is telling him nothing; Safe is telling something.<br />
This is also the case when someone inherits a bowling framework modeled as an “abstract” state machine, instead of a framework based on domain concepts with meaningful, domain-related extension points. Faced with the need to implement new requirements, expressed in domain terms, he now has to learn your code first, expressed in some other terminology which makes sense only to you.<br />
<br />
<span style="color: #b45f06;">The controller class knows absolutely nothing about the rest of the system. A nice consequence is that it can very easily be tested in isolation.</span><br />
Again, you keep saying that, but it’s just isolation from I/O, like in the first-cut fake OO version I presented. You got nothing more than that. Besides, your “easily tested” completely ignore the state space combinatorial explosion. More on this later.<br />
<br />
<span style="color: #b45f06;">The rest of the system (with the exception of the builder class that creates the whole object graph) also doesn't know anything about the controller (thanks to the use of closures, or the event-driven approach), which means that it can easily be made reusable.</span><br />
Except that:<br />
<br />
-<span style="white-space: pre;"> </span>There is <b>not </b>much else in the mythical “rest of the system”, except I/O, which anybody can make reusable (I didn’t call it “low hanging fruit” by chance). The juicy part is in your controller, which is not reusable, not extendible, and not adaptable except by tweaking. Great job :-).<br />
<br />
-<span style="white-space: pre;"> </span>Why is your builder ok while my builder is not? (See below).<br />
<br />
<span style="color: #b45f06;">The abstract logic can be changed easily, both because it's separated from the rest of the system and because it's all in one place. In particular, all the high-level data (input signals and abstract state) in the application are kept together, so when the mapping between inputs and outputs changes, or when new outputs are added, updating the code tends to be trivial. You never have to find your way around the object graph to reach the information you need, like you do in a "real" OO design. All the information is at your fingertips.</span><br />
This is really ludicrous. You’re passing your personal opinion as truth, with your argument being that everything is easy to change when it’s all in one place. You’re completely ignoring the explosion of complexity as the state space and logic grow (again, see below). You’re also actually ignoring what the entire world has learnt in the past 40 years or so: that without modular reasoning and encapsulation, things just don’t scale.<br />
<br />
Now, youngsters don’t read the classics, and as Santayana said, those who don’t know history are condemned to get a D- :-), so here is a little quote. Back in the 70s, Fred Brooks, father of the IBM OS/360, wrote a critic of Parnas’ concept of Information Hiding in his well-known “The Mythical Man-Month”. 20 years later, in the anniversary edition, Brooks reflected on what was right and what was wrong. Here is Brooks: <i>”I dismissed Parnas's concept as a "recipe for disaster" in Chapter 7. Parnas was right, and I was wrong. I am now convinced that information hiding, today often embodied in object oriented programming, is the only way of raising the level of software design.”</i><br />
<i><br /></i><br />
I’m counting on being still alive 20 years from now. If you happen to have an epiphany, let me know :-).<br />
<br />
<span style="color: #b45f06;">But the abstract logic is a very different beast. First, it is (by definition) much more application-specific, so I don't think reusability is something worth striving for. It also tends to change a lot, in ways that I find difficult to predict. And, worst of all, I think it's much more difficult to partition into reasonably indipendent subsets, because of all the data dependencies in there.</span><br />
(Funny how the “abstract logic” is now “application specific”. What happened to abstraction? :-).<br />
I can only hope you can see the self-fulfilling expectations here:<br />
<br />
a)<span style="white-space: pre;"> </span>You don’t believe in reuse of application logic, so you don’t even try to break it into reusable pieces. Then you claim it can’t be done. Wake up man, in my version <b>every single piece is reusable</b>.<br />
<br />
b)<span style="white-space: pre;"> </span>You think it’s hard to partition the system, so you don’t learn to see the natural seams in the forcefield. I know it’s hard for some and natural for others. I’ve seen it time and time again mentoring people. But beyond talent, it’s also a matter of application and training. You’re trapping yourself in the monolithic view.<br />
<br />
<span style="color: #b45f06;">Say that the requirement that the engine must be switched off when the methane level is high is not present in the original specification, but it's only added at a later time. In that case, your initial design would not, I presume, contain a SafeEngine class, and the PumpEngine would be operated directy by the SumpPump. When this new safety requirement comes up, you face a problem; the SumpPump has no way to know whether the methane level is critical because it is not "linked" to the object that holds that particular piece of information. So you not only have to change the SumpPump (and maybe add a new SafeEngine class), but you also have to rewire the object graph. Contrast this with what happens in the controller-based approach, where all you have to do is change one (one!) line of code.</span><br />
Now, this is an interesting argument, which allows me to explain the wiring thing. Weird enough, in your case you have to wire things together anyway (and I’ll get more specific in a moment) but you choose not to see the problem (very professional :-).<br />
<br />
Of course, as I add new abstractions, I occasionally have to re-route things. This is why I left to the well-inclined readers to understand the consequences of attaching the SumpPump to the SafeEngine, instead of polymorphically to the PumpEngine.<br />
<br />
But you seem to think that since the SafeEngine is used by the SumpPump, it must be <b>created</b> by the SumpPump, and that to do so, the SumpPump has to know the GasSensor. Well, if you do OO this way, it’s no wonder it does not work for you. <b>The MinePlant is the builder</b>. I also suggested that an IoC could make some things simpler.<br />
<br />
In a real-life, large-scale system we use configuration-driven wiring, based on catalogs of I/O, sensors, actuators, etc. That allows us to plug things in at any time, adding stuff without modifying existing code, build product families with <b>ease of extension and contraction</b> (as good ol’ Parnas said, but you weren’t listening; more on this in the second part). Theoretically, I can even rewire a live system, although I never had this requirement, so I can’t claim I did it.<br />
<br />
Oh, by the way: I have to bring water to the kitchen <b>and</b> to the bathroom, and electricity as well. I don’t mind the wiring, because I enjoy the separation of functions. Of course, you’re free to use a single place for all functions :-).<br />
<br />
<span style="color: #b45f06;">It's not just that you have to think harder, it's also that the changes you have to make are way more complex and no longer localized in a single place.</span><br />
C’mon. You’re ridiculing yourself. My changes are “way more complex” because I have to rewire a builder? While tweaking a larger and larger controller is fine? Sure. Whatever :-).<br />
<br />
<span style="color: #b45f06;">Now, this is a trivial example, and doing the rewiring here is easy. But in real-world applications, this particular issue drives me nuts all the time (Don't you ever get that problem?).</span><br />
Let me tell you something that we in the gray hairs club :-) usually know. When you implement a non-trivial system, no matter the initial design, <b>at some point you’ll face some issues</b>.<br />
<br />
If you believe in the design vision, you’ll squeeze your brain, come up with a nice solution (that is, an improvement over the initial design), and proceed. If you don’t, you’ll claim the design is a failure, and proceed doing what you always wanted to do.<br />
<br />
In your case, you need to create sensors and actuators and connect them to the centralized <strike>monster</strike> model, and you assume you’ll have some machinery in place, because you like this <strike>shit</strike> thing, and you want to make it work.<br />
<br />
Did it ever occur to you that in the OO version you could have a machinery to connect things by <b>name</b>, describing the wiring outside the application logic, so that (e.g.) every sensor has a name, and every part (pump, smart pump, etc) can be attached to any other thing <b>by configuration</b>, using a catalog, and not by hard-wiring things in code?<br />
<br />
Can you see how this would completely solve your issues with getting hold of things? Can you see how this will expand reusability, composability, etc? How this could help a system grow over time using “modern” techniques (dynamic libraries) where modules are independently written, tested, and deployed? How this would beautifully link with diagnostics, synoptic, etc, with the ability to drill down everything by name?<br />
<br />
Of course, if all you can see is a monolith, you’ll never invest in the necessary infrastructure, because you just don’t believe in modularity. So when you face problems with a modular system, you don’t squeeze your brain to solve it by increasing modularity even more. You fall back to the monolith. Well, it’s your choice :-).<br />
<br />
<span style="color: #b45f06;">So I generally prefer, when coding the abstract logic, to end up with "fatter" classes, that hold all the data that is likely to be used together. And I think a controller/manager is a very natural place to encode this highly cohesive knowledge.</span><br />
--<br />
Please. <a href="http://www.carlopescio.com/2008/09/and-found.html" target="_blank">I’ve actually defined the concept</a> of center in software as a locus of highly cohesive knowledge. And your controller (model) <b>is not</b>. It is not cohesive by any of the known software engineering metrics, and is not cohesive by my (yet to be explained) definition based on entanglement.<br />
<br />
But leaving the big theories aside, your fat model fails the pragmatic test for cohesion, which is, can I break it apart so that the communication between parts does not require exposure of internal details? Of course I can. I did. You did it too when forced by a new sensor type. So that knowledge is not cohesive. QED.<br />
<br />
You see it as highly cohesive because you choose <b>not to see</b> the natural seam in it; you choose to ignore the natural decomposition as suggested by the forcefield; you actually choose to ignore the forcefield and design a fixed-shape system (the monolith), irrespective of the problem. Sorry, as a designer, I have to say that it is a crappy design strategy.<br />
<br />
<span style="color: #b45f06;">As another example, I've also implemented the evacuation rule; all I had to do in the controller was to add a tiny new method, is_water_level_critical() (two lines of code in total)</span><br />
This just ain’t true. Even for this two-line change, you also messed up with <span style="font-family: 'Courier New', Courier, monospace;">set_high_sensor_state</span>, to set <span style="font-family: 'Courier New', Courier, monospace;">last_time_water_level_became_too_high</span>.<br />
<br />
This is a perfect case in point. Your coding style will always require you to patch things around, particularly so in the various setXXX, since those are the natural sub-gravitational centers in your design, <b>where stuff happens and state is updated</b>.<br />
<br />
So you add feature X not by adding code in a new modular unit, but by making your class larger and tweaking the existing code, at risk of breaking it. And since you have a combinatorial explosion of the state space, we just have to trust you to have enough test cases to cover for the occasional errors. Way to go, man :-). Compare that with the “way more complex” job of just rewiring things to connect <b>new code</b>, leaving existing code alone.<br />
<br />
<span style="color: #b45f06;">Outside the controller, I would have to create another alarm object, and link it to this new method. But that would be very straightforward too, the architecture would already be able to handle that, as it would already be doing the very same thing for other types of alarms.</span><br />
I’ll show you in a moment why this is a very biased view, but should be pretty obvious now that you trivialize the wiring problem on your side and make a big fuss out of it on the other.<br />
<br />
<span style="color: #b45f06;">The invariant is hardly unfathomable, and in fact, I think the controller is an ideal location to place this kind of integrity checks, because it can implement complex invariants on data that in a pure OO architecture would be scattered throughout the entire object graph, and that would be difficult to reach from a single place.</span><br />
You’re contradicting yourself. If the invariant is simple, how can it be complex? The problem is that you’re again trivializing the issue of state space explosion, and ignoring the fundamental principle of modern [software] design (modularity). We want to check parts independently (simple invariant) and compose them in a safe way. You want to work as a caveman. Your choice.<br />
<br />
<span style="color: #b45f06;">In a real application it's of course going to be larger that it is in this toy example. But even if it were, say, 4 or 5 times larger, I still would find that perfectly acceptable, given the fact that its structure is so simple. If it got much larger than that, I would of course think about splitting it into several pieces (as few as possible, though). In practice I find that that can usually be done while at the same time avoiding the problems of the pure OO approach. I think one nice way to do it is by using (abusing?) multiple inheritance.</span><br />
You see, besides the obvious mockery for abusing a design concept for its implementation consequences, you just told everybody that your design strategy does not scale. At some point, despite your effort, you’ll be forced to split your class into more manageable pieces.<br />
<br />
Even at that time, you’ll try to resist, ignore the natural seams in the forcefield and keep it as monolithic as possible. Then you want to use multiple inheritance so that everything is still visible in case you wanna mess with it (see the quote by Brooks again man – <b>you’re doing it wrong</b>).<br />
<br />
Funny enough, when you had to split your class, you didn’t use MI, so you now have distinct objects (controllers). You didn’t say that, but <b>now you have to change the damn wiring</b>. The input is no longer connected to the MineController: is connected to the LevelSensorController. The LevelSensorController must be connected to the MineController, and it’s not a sensor or an actuator, so you need some machinery in place to connect arbitrary objects.<br />
<br />
Why is that not a problem in your design, but it’s supposed to be a problem in the OO design? Because you either did the OO thing wrong, or you can’t see you have the same wiring problem, or don’t want to say that you have the same wiring problem.<br />
<br />
<span style="color: #b45f06;">So, to wrap up, I really can't see what's wrong with keeping all the high-level logic and data in one place</span><br />
Well, honestly, just get out of my kitchen :-))<br />
<br />
<span style="color: #b45f06;">On the other hand it seems to me that the pure OO architecture has a couple of pretty obvious flaws;</span><br />
<span style="color: #b45f06;">the first one is that the high-level application logic is more obscure</span><br />
To you, because you want to find it all in one place. Modularity requires the ability to understand a system a piece at a time, after which I personally think it’s easier to understand the overall behavior from a diagram.<br />
You may want to try an exercise. What if I draw a portion of my system like this, translating the UML into a more pictorial form (using a bunch of engineering symbols):<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9pxvQjUVQMde9HWSAip2pcdkdxwgAoq_nPqYKmGZCx7BlkffXsxdR3WabBhkv5-pUBlDOkvuollwrWs2dFkBZebkPrA8DtGEbxaYrVBjAT4SaaudHvYkmjCJcxQsWKhajtyVRww/s1600/engineering.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="311" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9pxvQjUVQMde9HWSAip2pcdkdxwgAoq_nPqYKmGZCx7BlkffXsxdR3WabBhkv5-pUBlDOkvuollwrWs2dFkBZebkPrA8DtGEbxaYrVBjAT4SaaudHvYkmjCJcxQsWKhajtyVRww/s400/engineering.png" width="400" /></a></div>
Do you get it better now? Well, that’s the <b>function </b>of the system, expressed pictorially. It’s also in <b>frictionless contact</b> with the shape of my design (as Alexander would say).<br />
<br />
The problem is that you’re shape-blind. You’ll never be happy with OO until you develop the ability to see function through shape.<br />
<br />
Of course, you don’t have to like OO. Remember what I said at the beginning of my previous post: <i>"Don’t get me wrong. There is nothing intrinsically wrong with the monolithic function. It works. We just get into troubles when we expect some non-functional properties which are at odd with that shape. Indeed, many cultural antipatterns I’ve observed in the industrial automation field can be mapped directly to the mismatch between monolithic software and some desirable non-functional properties".</i><br />
<i><br /></i><br />
You’re just epitomizing that statement. Actually, you’re making things worse, by pretending your software has properties which, in fact, does not possess.<br />
<br />
<span style="color: #b45f06;">and the second one is that the application is more brittle and difficult to change</span><br />
You must live in your own reality distortion field, because it’s <b>exactly the opposite</b>. You’re blinded by the wiring thing and your own preference for monolithic logic.<br />
<br />
OO software like mine is <b>easily extended and contracted</b> by adding/removing small, domain-inspired, independently testable abstractions (I’ll show you a couple of examples below). Your code requires tweaking an ever-growing class, adding methods and changing existing ones, leading to a combinatorial explosion of states which have to be tested together, etc. etc. It’s error prone and does not prevent common mistakes like using the wrong variable out of a large set (back to the 70s, man). It even fails if you set the sensors in the “wrong” order (out of a gazillion; see below). Talk about brittle. You gotta be kidding.<br />
<br />
<span style="color: #b45f06;">I would also add that I believe this is a signature problem of "real" OO solutions.</span><br />
I would add that your stance is the signature problem of those who can’t understand the concept of form, modularity, modern programming practices, yet think that the problem stays with the rest of the world and therefore dig deeper into their cave, leaving a few math symbols behind to look cool. But that would be rude. So I won’t.<br />
<br />
<b>Back into reflective mode</b><br />
<br />
Ok, now that I’ve provided some (undoubtedly unsatisfactory :-) answers to most of your points, why don’t we lay back and try to add some perspective to the whole thing.<br />
<br />
<b>You don’t want to see reusability</b><br />
I choose the mine pump instead of making up a something exactly because it is a well-known problem. People have already proposed reasonable requirement changes in existing literature, and therefore I don’t have to invent things purposely now, which is part of the aforementioned intellectual integrity.<br />
<br />
One of the proposed requirements I’ve seen in some papers is to have two pumps, and alternate between them after some time to avoid wearing the pump out (makes lot of sense too).<br />
<br />
In your programming style, that’s just <b>more stuff inside the controller</b> (and more wiring, since now you have multiple outputs). Except that in your existing controller <b>you don’t have a place</b> for time-driven logic. So you can either make up an artificial “time” input or you can try to put that time-checking logic inside some existing setXXX function which you assume will be called frequently enough. What you get, of course, is just a fatter, less reusable controller.<br />
<br />
In my design approach, I see this as a <b>great chance</b> to create yet another reusable, application-level class (abstraction!): the pump rack. A PumpRack will keep track of time and switch pump after a while. Do I need to change wiring? Well, from the outside, the PumpRack is just a PumpEngine, so I do have to create a different object, but the wiring is the same (this is where an IoC will give you everything for free).<br />
<br />
Now, before we move to the next change in requirements, let me talk about your style again.<br />
<br />
<br />
<b>Your design style does not scale up</b><br />
When I introduced the concept of <a href="http://www.carlopescio.com/2008/12/notes-on-software-design-chapter-2-mass.html" target="_blank">mass and gravity</a>, a smart reader pointed out that if you set up the right gravitational field in the beginning, gravity will be your friend, attracting things in the right place. I further explored the issue in <a href="http://www.carlopescio.com/2009/02/notes-on-software-design-chapter-4.html" target="_blank">Notes on Software Design, Chapter 4: Gravity and Architecture</a>.<br />
<br />
Of course, once you have a monolith, gravity will do the only natural thing: bring in more stuff. However, at some point you won’t cope with complexity anymore, and, as you admitted, you’ll be forced to spin off a few pieces. You can only delay the inevitable by abusing multiple inheritance. Sooner or later, you won’t have the entire state space “at your fingertips” (of which I’m grateful :-).<br />
<br />
So, the next requirement change is that if a pump gets clogged, I need to switch to the next pump even if it’s not its time yet (again, makes a lot of sense). Clogging will be detected by monitoring power.<br />
<br />
Once again, you have only one place to put that logic: the mine controller. Two more inputs (one for each pump; what if I have 3, 4 or 10 pumps?), more logic into that poor class.<br />
<br />
On the other hand, I have<b> a nice gravitational field set up</b>. That portion of the logic is naturally attracted to the PumpRack (I truly hope you can see why; for the interested readers: it’s entanglement manifesting as attraction). I now have the choice to put it in there, or create a subclass (say, FailSafePumpRack) to preserve a particular form of modularity (and reusability) that people tend to forget.<br />
<br />
<b>Your design style does not scale down</b><br />
In the post above on gravity and architecture, I mentioned how one of Parnas’ best papers on modular design was cleverly called "Designing Software for Ease of Extension and Contraction", and how people tend to forget the “contraction” part.<br />
<br />
In fact, your design cannot scale down. Once you get beefed up by new requirements, you have all sort of complexity in your controller. You cannot trim it down except by copy and paste (caveman :-). Sure, you can avoid wiring some I/O (perhaps; see also below on safety), but your stuff is all there.<br />
<br />
That's another <b>antipattern</b>: thinking that you cannot have a scaled-down version of your software (not to mention a software product line) because it’s too hard, and that you can only “disable” things and leave them in.<br />
<br />
My design has <b>no problems</b> scaling down. I can remove the FailSafePumpRack. I can remove the PumpRack. I can remove the SafeEngine if not needed, thanks to polymorphism. I don’t have to disable things. I can bring in the necessary modules and nothing else. This is the beauty of designing a system with emerging behavior in mind.<br />
<br />
About scaling and complexity, let me tell you another story. About 15 years ago, an industrial automation company asked for my help in the design of a new system. They had a few OO “pioneers” inside, but basically, the rest of the company was stuck in “controller mode”. They had troubles when the I/O space became larger, say over 300 I/O, and serious troubles over 500 I/O.<br />
<br />
We built a first-generation system which was reasonably OO, and they scavenged a small framework out of it. Five years later they called me back. A good guy said “we now routinely build systems with over 1000 I/O without problems” (I was actually quite proud of him :-).<br />
<br />
We did a quite few improvements here and there, using more modern programming techniques and bringing in more domain concepts (abstractions!). I didn’t hear from them for about 10 years, until recently. They are now working in the 10.000 I/O range and think they may need something more sophisticated (which is fine, technology and languages are much better now).<br />
<br />
Don’t tell me my approach gets too complex, because I’ve proven over and over in real-life systems that it doesn’t. On the contrary, it makes previously impossible systems <b>possible</b>. Sure, it requires skilled people. I don’t believe in deskilling. I don’t believe in designing systems for cavemen to maintain. That’s a heck of a bias, I know :-).<br />
<br />
<b>Safety</b><br />
The mine controller is a simple, yet life-critical system. Now, suppose you have to bet your life on a software system. You can choose between two implementations, coming from people of equivalent IQ, experience, dedication, etc.<br />
<br />
Company #1 says: “we believe that we can build a safe system by creating small, safe components which can be tested thoroughly in isolation, with a small state space. The components hide their internal state space to prevent accidental changes. Components are organically assembled in larger components (pump / safe pump / rack / etc), still with minimal behavior, and so on, until the complete safe system is created. We develop highly reusable components and we tend to reuse proven modules whenever is possible”.<br />
<br />
Company #2 says: “we believe it’s better to put all your state space in a controller, and don’t break it up until you are forced to. When we are forced to split, we abuse multiple inheritance, so the whole state is still available to the central core routines. That way, we can access whatever we want from wherever we want, and we don’t have routing or visibility problem. We don’t believe in reuse because you can’t have two identical large controllers. We also don’t believe in programming errors and we don’t think we’re going to mess up and write the wrong output or read the wrong input. We’re real men. We don’t believe in state space explosion either, and we think we can easily understand what is going on because all the logic is in one place. We can easily test the whole thing because the sensors/actuators can be mocked; hey, we use booleans and floats! Oh, did I mention we don’t believe in state space explosion, which is why we believe we can actually test the whole thing?”<br />
<br />
I don’t know about you, but I know which one I would choose. Oh, since you don’t believe in state space explosion :-), in your implementation you have 5 inputs, 2 internal states, 4 outputs (represented by isXXX functions). Let’s multiply that by 5, as you say that you don’t see a problem with that. You now have 25 inputs, 10 internal states, 20 outputs. Assuming for simplicity that every input, state and output is a Boolean (which is not), you’re mapping 2^35 states into 2^20 states, within a single monolithic model class. If, as you said, you don’t see a problem with that, well, please stay away from safety critical systems :-).<br />
<br />
Seriously: go through any decent, modern literature on building safety-critical systems, and you’ll see modularity and state space separation as key components. Some even suggest using hardware separation of the state space. Having adequate hardware support, in life-critical systems I would be glad to physically isolate the internal state of my components from each other, to prevent any accidental change (e.g. through wild pointers). Your “global mess at your fingertips” approach is just antithetical to safety.<br />
<br />
Oh, by the way, if you’re thinking that “in practice, you don’t have to test all those combinations”, you know what? You’re right. Because despite what you think, that information is <b>not highly cohesive</b>, so it will move inside a small subset of the state space. Ain’t that funny? The only thing that makes your technique barely acceptable is that <b>your hypothesis are wrong</b> :-). Unfortunately, having “all the information at your fingertips” means you’ll never know if you got it right, so it’s not really a safe bet.<br />
<br />
Contrast this with my design, which recognizes the natural partitioning and models a corresponding class structure, where the state space is partitioned as well. Can’t you really see the light?<br />
<br />
<b>Predictability</b><br />
One of the reasons your style is seldom adopted (not even by cavemen stuck in controller mode) is that it’s much weaker than you seem to believe. The controller style, as ugly as it is, comes with a strong guarantee of sequential execution. <b>The sequence is hard-coded in the controller itself</b>. Your fat model does not come with that guarantee. The order of execution is in the sequence of calls to the various setXXX functions, which lies elsewhere.<br />
<br />
In general, however, there is no guarantee whatsoever that you can just swap the order of setXXX calls and still have a correct behavior.<br />
<br />
In your code, for instance, if I call <span style="font-family: 'Courier New', Courier, monospace;">set_low after set_high</span>, and I have a failure in the low sensor (being false when high is true), <span style="font-family: 'Courier New', Courier, monospace;">needs_draining</span> will become false. And since you put in a safety check (in another function) only because I said that (and you know it :-), people will just die. Good job :-).<br />
<br />
By the way, it’s a crappy implementation: in case of sensor failure, unless there is methane present, it’s better to pump out water. You don’t if I call set_low after set_high, but you do if I call set_high after set_low. This is just a simple example of why, even in a trivial case, your design strategy is too weak for professional programming.<br />
<br />
By the way: with 25 inputs, you have 25! possible call sequences to your various input functions (that’s about 10^25). Since only some of them work, make sure you’re using the right one :-). As should the lucky one who inherits your code, too.<br />
<br />
<b>There is something good, of course</b><br />
I could go on and on, but let’s switch to the bright side. Your design has a few good properties, just not the one you’re thinking of.<br />
<br />
-<span style="white-space: pre;"> </span>Observability: it’s easier to observe the global state with a software probe when state is not scattered in memory. We had this requirement in a project a few years ago, but being in C++ we solved it beautifully at the allocator level.<br />
<br />
-<span style="white-space: pre;"> </span>Patchability of state. A byproduct of the above, seldom used but extremely useful if you’re building a rover to mars. Suppose you have a bug and your state gets messed up. Thanks to observability, a remote team can <i>potentially</i> devise and send a state patch.<br />
<br />
Now, I want to stress the intellectual integrity thing again. I’m pointing out any strength I can see in your design out of honesty. Overall, they don’t balance the negative side, and there are different ways to obtain those properties without crippling the entire design.<br />
<br />
<b>Keep it up!</b><br />
Ok, I’ve been trashing you for a while now, but the truth is, I’m not really interested in proving you’re wrong and I’m right. I don’t think that I can (and I don’t want to) change your mind. What we see here is a belief system in action. You believe some things are true. It is only natural to defend your belief systems against my words, and try to crush my antagonist vision.<br />
<br />
I also have a belief system. It’s called freedom :-). Within that belief system, you have any right to explore your own programming style (as ugly as it is :-), as long as you don’t try to push it down my throat. I’d rather not having you around safety-critical systems, but ok, life is dangerous anyway :-)<br />
<br />
Maybe one day your ideas will evolve into something different, better than what we both do today. Nature values diversity. Especially in caves :-).<br />
<br />
<b>Bonus quote</b><br />
I retweeted this the other day, and it seems <b>so</b> appropriate here: <i>The truth will set you free, but first it will piss you off. ~ Gloria Steinem</i>.<br />
<br />
<br />
If you liked this post, you should <a href="https://twitter.com/#!/CarloPescio">follow me</a> on twitter!<br />
<br />
<br />
<br />Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com16tag:blogger.com,1999:blog-13967713.post-44501225692831999832012-03-12T19:19:00.000+01:002012-07-30T08:49:31.387+02:00Life without a controller, case 1In my <a href="http://www.carlopescio.com/2011/04/your-coding-conventions-are-hurting-you.html" target="_blank">most popular post ever</a>, I argued (quoting Alan Kay and Peter Coad along the way) that classes ending in –er (like the infamous Controller, Manager, etc) are usually an indication that our software is not really OO (built from cooperating objects), but still based on the centralized style of procedural programming.<br />
<br />
That post was very well received, with a few exceptions. One of the critics said something like “I’ll keep calling my controllers controllers, and my presenters presenters”. This is fine, of course. If you have a controller, why not calling it “controller”? Or, as Shakespeare said, a controller by any other name would smell as bad :-) . The entire idea was to <b>get rid of controllers</b>, not changing their name!<br />
<br />
Now, getting rid of controllers is damn hard in 2012. Almost every UI framework is based on some variation of MVC. Most server-side web-app frameworks are based on MVC as well. Proposing to drop the controller would be just as useful as proposing that we drop html and javascript and start using something serious to write web apps :-).<br />
<br />
Still, I can’t really sit here and do nothing while innocent objects get slaughtered, so I decided to write a few posts on living without a controller. I’ll start far away from UI, to avoid rocking the boat. I’ll move closer in some future post.<br />
<br />
<a name='more'></a><br />
<br />
<b>A well-known problem</b><br />
I could easily come up with some problem, cleverly designed to prove my point, but I won’t. Instead, I’ll adopt a well-known problem, dating back to 1983, which has been used in numerous articles and books about process control, industrial automation, etc: the mine pump problem (a couple of references: <a href="http://comjnl.oxfordjournals.org/content/34/2/173.full.pdf+html" target="_blank">A Framework for Building Dependable Systems</a> by Burns and Lister, and <a href="http://i.unu.edu/media/publication/000/001/566/report323.pdf" target="_blank">Real-time Systems Specification, Verification and Analysis</a>, edited by Mathai Joseph).<br />
<br />
The overall problem is represented here:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7bnnaiAQpCmGbBXJYANkaeZsiZZBUoD9NhYkOKrWIz8lcxyCq3-HctoR_0Iv0CRTbiLmk7GuJKakHlocApLyvdHrRo6zVTqnCebnVBjOJ57UyYV5L-Uhwl5uY_k6KsSWY0er0hg/s1600/mine.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="356" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7bnnaiAQpCmGbBXJYANkaeZsiZZBUoD9NhYkOKrWIz8lcxyCq3-HctoR_0Iv0CRTbiLmk7GuJKakHlocApLyvdHrRo6zVTqnCebnVBjOJ57UyYV5L-Uhwl5uY_k6KsSWY0er0hg/s400/mine.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.1</td></tr>
</tbody></table>
<br />
We want to pump water out of a mine sump. We have two water level sensors (D, E). When D goes on, we pump out water until E goes off (this is to realize a form of <a href="http://en.wikipedia.org/wiki/Hysteresis#Control_systems" target="_blank">hysteresis</a> and avoid “bouncing” around a given level).<br />
There are also a few gas sensors for carbon monoxide, methane and airflow levels (A,B,C). If any of those becomes critical, an alarm must be raised.<br />
Finally, to prevent explosions, the pump must not be operated when methane is above a certain level.<br />
<br />
Existing literature is mostly concerned with specification, timing and modeling issues. Software design issues are largely ignored, and Fig.1 already seems to suggest a centralized controller style.<br />
<br />
I’m mostly interested in exploring design choices, so I’ll forego timing issues (which ain’t really that hard for this problem, anyway). I’ll try to evolve a design starting from the most primitive style, and highlight what needs to be done to break out of a centralized controller style.<br />
<br />
If you got some time, you may want to explore a few design alternatives on your own: there are many sensible (and less sensible :-) alternatives to what I’m going to present here.<br />
<br />
<br />
<b>The control engineer way </b><br />
(Apologies to control engineers, I’m mostly thinking of a friend of mine here).<br />
Control engineers will follow a simple, 4-step, repeatable process for system design:<br />
<br />
- write 10 pages of differential equations to prove that if water is coming in faster than you can get it out, you’ll drown.<br />
<br />
- assume that once you got your equations right, it’s just a matter of mapping some inputs (bits) to outputs (bits).<br />
<br />
- start with a simple MainLoop function, reading bits, going through a sequence of conditionals and switch cases, adding state variables as needed (hysteresis requires a simple state machine), until you can write bits.
<br />
<br />
- add a Kalman filter at some point. If you ain’t gonna use a Kalman filter, you can’t call yourself a control engineer.<br />
<br />
More seriously, what’s really common about control engineers is to spend a lot of time modeling the function and then proceed in total disregard of shape<sup>*</sup>. Therefore, we usually get a monolithic function. Sometimes, a few functional abstractions are indeed identified and extracted, but more often than not, you simply get a long sequence of statements (sometimes justified by “performance”, usually without any data to back up that claim).<br />
<br />
[<span style="font-size: 13px;">*</span>I’m moving from using “form” to using “shape” because “form” is often confused with “ceremony”, which is definitely not what I mean.]<br />
<br />
Don’t get me wrong. There is nothing intrinsically wrong with the monolithic function. It works. We just get into troubles when we expect some non-functional properties which are at odd with that shape.<br />
Indeed, many cultural antipatterns I’ve observed in the industrial automation field can be mapped directly to the mismatch between monolithic software and some desirable non-functional properties.<br />
<br />
Consider testability: if your entire process is implemented as one large function, it will be hard to test thoroughly. Antipattern: claim that faults can only be found in the field and neglect testing. You may recognize this as a <a href="http://www.carlopescio.com/2006/07/self-fulfilling-expectations.html" target="_blank">self-fulfilling expectation</a>: if you don’t test because faults can only be found in the field, you’ll only find them on the field, thereby reinforcing your belief.<br />
<br />
<br />
<b>The fake-OO way, step 1</b><br />
There is a wide spectrum from the monolith to what I’m discussing here, including some entirely functional or procedural solutions, but my destination is a true OO design, so let’s take a first step toward objects.<br />
At some point, people get usually tempted by a few low-hanging fruits and introduce some abstractions:<br />
<br />
- the DigitalInput, to read from the water sensors<br />
<br />
- the AnalogInput, to read from the gas sensors<br />
<br />
- the DigitalOutput, to control the pump and the alarm<br />
<br />
In hubris, they may even “hide” those abstractions behind an interface, because (and that’s right) it may help to abstract away a few hw/sw details.<br />
In the end, you get something like this (ah, the beauty of a picture, conveying the entire structure with a few strokes; how far is the day when the agile police will allow picture-based conversations again?)<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoULs3_1gsfGpG7wV6bQoDD8tqXWbkCG6k5CvtWsSgKKAo5VezAyt1DV-3luqOSSASKM6t8eWwlTwds_qBEsJjFg0oH7rmvwPJQ_14A86X2MI4MuOr2vwgcqBMI8xY7Q7W-iQbBA/s1600/Mine1.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="163" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoULs3_1gsfGpG7wV6bQoDD8tqXWbkCG6k5CvtWsSgKKAo5VezAyt1DV-3luqOSSASKM6t8eWwlTwds_qBEsJjFg0oH7rmvwPJQ_14A86X2MI4MuOr2vwgcqBMI8xY7Q7W-iQbBA/s320/Mine1.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.2</td></tr>
</tbody></table>
I’ve used letters as roles to reflect Fig.1.<br />
<br />
Of course, we still have the dreaded controller: all the logic is sitting there. Still, there has been a slight fluctuation in the non-functional properties: now we don’t really care about the details of (e.g.) a digital input. Is it being mapped into a register? Is it coming from some kind of serial bus? Who cares (to some extent; the devil is in the details).<br />
<br />
If we take the extra step and introduce interfaces, we actually gain something on the testability side as well: we can then mock I/O (depending on where/how creation is actually implemented).<br />
<br />
Those of you who are [still] looking at design from the “principles” perspective will see this tiny step as perhaps inspired by the Single Responsibility Principle and by the Open/Closed Principle.
Maintainability, however, is still pretty low. We have one, big <a href="http://www.carlopescio.com/2008/12/notes-on-software-design-chapter-2-mass.html" target="_blank">gravitational center</a>, and that’s where we’re gonna spend most of our time adding, removing, and tweaking functionality. Reusability is also limited to the I/O. Everything else is copy / paste / tweak.<br />
<br />
<br />
<b>The fake-OO way, step 2 </b><br />
At this stage, people realize that OO is about “modeling the world”, and perhaps parroting the Domain Driven Design guys, start to babble about the ubiquitous language and introduce a few more abstractions:<br />
<br />
- various type of sensors<br />
<br />
- the pump<br />
<br />
- the alarm<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1OJM0C_jKyAtrIOeuudKyy-tRhYopBFmyRTDSb6hY34CZ_MkAqHB26dXjs8yvblOW-J6VvyEhfJjxUDh-4itNFnRxt5Sl4hx9pHXmsPvkgslGbOhbj6Y5nD8dSwFaRknwRwYF-g/s1600/Mine2.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1OJM0C_jKyAtrIOeuudKyy-tRhYopBFmyRTDSb6hY34CZ_MkAqHB26dXjs8yvblOW-J6VvyEhfJjxUDh-4itNFnRxt5Sl4hx9pHXmsPvkgslGbOhbj6Y5nD8dSwFaRknwRwYF-g/s400/Mine2.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.3</td></tr>
</tbody></table>
Note: I’m leaving out of the diagram the obvious interfaces for the Analog/Digital Input/Output classes. They add little to the discussion and waste quite a bit of space.<br />
<br />
This shape is not wrong, of course. Actually, it’s ok, sort of. What is wrong, at this stage, is usually how <b>responsibilities </b>are distributed (or not, meaning: centralized). When you have a controller in your mind, it’s all too easy to end up with stupid classes, because stupid are easier to control.<br />
<br />
As you can see, the responsibilities in layer 2 (sensors and actuators) have a 1:1 mapping with responsibilities in layer 1 (I/O). Again, this is not necessarily wrong: layer 2 is closer to problem domain. However, layer2 does precious nothing as far as the controller is concerned.<br />
The entire logic is still in the controller. Writing <span style="font-family: 'Courier New', Courier, monospace;">pump.On()</span> instead of <span style="font-family: 'Courier New', Courier, monospace;">pumpDigitalOutput.Write(1)</span>is an improvement, but it’s not changing the overall design (architecture).<br />
<br />
<br />
<b>The path not taken </b><br />
It is perhaps tempting, given the similarities among concrete Sensors (and between Pump and Alarm), to introduce yet another abstraction layer (the Sensor/Actuator as hinted above). Again, this is not wrong, but you shouldn’t take this path too early.<br />
<br />
Object oriented design is <b>not </b>about building a deep chain of domain-inspired “abstractions” with little or no behavior. It’s about decomposing the problem so that the intended function <b>emerges</b> from the interaction between intelligent, specialized mini-machines (objects, as seen from Alan Kay).<br />
<br />
Adding layers for mere classification is not a terribly useful step, although I understand the intellectual satisfaction :-) in doing that.
<br />
<br />
<br />
<b>Toward "real" OO</b><br />
A key realization is that to move toward a different set of non-functional properties we must find better abstractions. Those abstractions will eat away the controller, until in the end there will be no controller left.<br />
<br />
There is not a single, deterministic process to find those abstractions. We may end up there through domain knowledge, through experience in <a href="http://www.carlopescio.com/2011/10/youre-solving-wrong-problem.html" target="_blank">asking the right questions</a>, through commonality analysis of past implementations, through sheer brightness, and so on. I’m a bit skeptical about the possibility of finding all those abstractions through test-driven design, but some may definitely emerge that way as well. I don’t want to suggest that a specific design process is superior to another, so I’d rather move on and introduce the first abstraction.<br />
<br />
<br />
<b>The sump probe </b><br />
We can blame the specification, so to speak. As it usually happens, it does not describe the real problem. It’s more about a specific incarnation of the problem, or about the invariably biased perception of whoever is describing the problem (which is why we used to consider analysis important ;-). The presence in Fig.1 of two stylized sensors was enough to bring into the software counterpart the digital input, and then a level sensor (which is actually a level switch, hence digital).<br />
<br />
What if we change sensor technology, and we bring in a submersible level probe, which has an analog output and can tell the actual depth? Well, what we need is to:<br />
<br />
- drop the level sensor (ok)<br />
<br />
- add a LevelProbe, connected to an AnalogInput (ok)<br />
<br />
- tweak the Pump Controller; remove the switch-based level detection logic, replace it to deal with analog measurement. Maybe tweak the state machine too. (not ok).<br />
<br />
An alternative would be to introduce, from the very beginning, the concept of SumpProbe. Its sole responsibility is to know when it’s time to drain the sump.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiClz2IOF8nDKvetIw404NEDQVnk-vtcuYHZ4IhKafz_hlfwIIh0hwcn9VWohFL-YSGvpP19CQhk8BwhrkIRiRN9ELYE5aR38Dxk4PFl4XH88U4ZKw0CMP0a5_B36wdowAAdRWL9g/s1600/Mine3.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiClz2IOF8nDKvetIw404NEDQVnk-vtcuYHZ4IhKafz_hlfwIIh0hwcn9VWohFL-YSGvpP19CQhk8BwhrkIRiRN9ELYE5aR38Dxk4PFl4XH88U4ZKw0CMP0a5_B36wdowAAdRWL9g/s320/Mine3.jpg" width="131" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.4</td></tr>
</tbody></table>
<br />
In an early implementation, the SumpProbe could be based on Level Sensors. We don’t need an interface yet; we’re just moving some logic outside the controller and into SumpProbe: generalization can wait. In a LevelSensor-based implementation, SumpProbe will sample two sensors and implement the hysteresis state machine.<br />
<br />
Now, this change may seem like nothing, but it’s actually a <b>pivotal moment</b>. For the first time so far, we have a class that is actually <b>doing </b>something. The SumpProbe is not exposing data. It’s exposing a service.<br />
<br />
If we want to use a LevelProbe in place of a LevelSensor, we have to change the SumpProbe, but the PumpController won’t need to change, because that logic is no longer there. Of course, now polymorphism can easily kick in, and we can turn SumpProbe into an interface with two concrete implementations, one based on LevelSensor and another on LevelProbe. In a LevelProbe-based implementation, SumpProbe will poll just one sensor. It may or may not be worth sharing the state machine between the two implementations. That’s truly a minor point. I won’t show the polymorphic version in the diagram, keeping only the main structure in place (as my goal is to get rid of the controller, and I want to focus on that).<br />
<br />
Interestingly, the SumpProbe, in its
LevelSensor-based implementation, would also be the perfect place to add some fault-detection code. If the higher sensor is triggered, the lower sensor <b>must</b> be triggered. That’s a nice invariant for the
LevelSensor-based SumpProbe class, which reminds me I seldom talk about Design by Contract, so here is another hint that you’re on the right path: <b>you can actually think of an invariant for your class</b>.<br />
<br />
Stupid classes have empty invariant; controller classes have unfathomable invariant (which adds to the difficulty of testing).<br />
<br />
<br />
<b>A useful Gas Sensor</b><br />
I warned against introducing a Sensor abstraction before. I want to stress that point a little: the wrong abstraction can actually lead you the wrong way. Now, what is a meaningful, little responsibility for a <b>gas </b>sensor (as opposed to a generic sensor)?<br />
Of course (?!), it’s telling you when a critical level has been reached. The beauty of this responsibility is that, for instance, a Methane or CO Sensor will be triggered by an upper bound, while AirFlow by a lower bound, but that's invisible from the outside. A different type of sensor may be triggered by a window, or by any given law.<br />
<br />
Note: the actual beauty is not really in respecting the Open/Closed Principle. It’s about having found a <b>natural place for knowledge</b>, more specifically, knowledge about gas type. GasSensor is an excellent <a href="http://www.carlopescio.com/2008/09/and-found.html" target="_blank">gravitational center for highly cohesive knowledge</a>.<br />
<br />
This is a first-cut diagram:
again, we’re eating a small portion out of PumpController (checking thresholds). It’s ok to eat a small bit: it means we’re creating small abstractions.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-tziZGKxctendQgPtncnRsw6UK6U20VvSUM-MkMOvruFK-GqGH87OsKQW7D6Q9dj0mdaYEEdFQumHFXMT0H6xVSIugd3wr4vpJLX45_stojbm8IeVstwikFDQlNNha-EX2jbR0g/s1600/Mine4.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-tziZGKxctendQgPtncnRsw6UK6U20VvSUM-MkMOvruFK-GqGH87OsKQW7D6Q9dj0mdaYEEdFQumHFXMT0H6xVSIugd3wr4vpJLX45_stojbm8IeVstwikFDQlNNha-EX2jbR0g/s320/Mine4.jpg" width="290" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.5</td></tr>
</tbody></table>
<br />
As simple as it is, this change may generate some discomfort. I am making the sensors “less reusable”, because some domain (hence, application) knowledge has got into it. For instance, a CO sensor may need to operate on a lower bound, instead of an upper bound, in an entirely different problem.<br />
<br />
Here is where the designer needs to understand what he’s doing, and avoid being a blind follower of some principles or prescriptions:<br />
<br />
- Reusability is just one of the many forces we have to consider.<br />
<br />
- The new sensors are perhaps less reusable, but they’re more useful. An empty class is absolutely reusable, but also completely useless.<br />
<br />
- Before you discard the smarter sensor, consider making it even smarter, and maybe get a smaller codebase in exchange. If we implement just one super-policy (the high/low window) everything else is just a matter of configuration (a threshold being a special case of a window).<br />
<br />
<br />
<b>Real OO</b><br />
Ok, so those were still pretty low hanging fruits. The pump controller is still sitting there, asking sensors, deciding whether or not it’s safe to operate the pump, and whether or not to trigger the alarm.<br />
<br />
To the experienced designer, these two responsibilities will eventually suggest two new abstractions.<br />
<br />
<br />
<b>The Safe Engine </b><br />
I’ll rename Pump into PumpEngine, because that’s what we’re really operating. Now, the problem with the “standard” engine is that it’s dumb: it doesn’t know whether or not it’s safe to turn on. Well, that’s a SafeEngine job:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUb3efcQ4ajymYbpQPThUHA5Vu4wiGQI3Su_uaOtm1chN4PL-ZLoNxJf9REFSH3VPsagaqEUaW-g7qNijmuv-OrzFk4cNXEA4Erwa2Dp5Ld73r9hlwAqeFXTifh0mx-zB7ZjH4qQ/s1600/Mine5.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUb3efcQ4ajymYbpQPThUHA5Vu4wiGQI3Su_uaOtm1chN4PL-ZLoNxJf9REFSH3VPsagaqEUaW-g7qNijmuv-OrzFk4cNXEA4Erwa2Dp5Ld73r9hlwAqeFXTifh0mx-zB7ZjH4qQ/s320/Mine5.jpg" width="274" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.6</td></tr>
</tbody></table>
<br />
I’ve brought in multiplicities (which I’ve left at the implicit level so far) just to point out how easy it would be to make the SafeEngine watch more than one sensor. Just use a list, or perhaps a composite. Once you have the right structure in place, things get simpler, not harder.<br />
We may also want to consider the option to make the PumpController working polymorphically on PumpEngine. I’ll ignore this issue, as this is already a long post.<br />
<br />
<br />
<b>The Gas Alarm</b><br />
It may follow with ease now that we also want to trigger an alarm whenever a Gas Sensor triggers a critical condition.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip-9t4Hz929jzkA1ZMprE3DRzTL94hCP9g4kugY1OrQTxVsmD_9cK6IQFCM4nbgzbSybgGFDzkCbXoHHxsLDDY3s4AVVWWcS1WqcVH0WfHn48YGLKfvNt2o386RCvnfCdJCciDDw/s1600/Mine6.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip-9t4Hz929jzkA1ZMprE3DRzTL94hCP9g4kugY1OrQTxVsmD_9cK6IQFCM4nbgzbSybgGFDzkCbXoHHxsLDDY3s4AVVWWcS1WqcVH0WfHn48YGLKfvNt2o386RCvnfCdJCciDDw/s320/Mine6.jpg" width="272" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.7</td></tr>
</tbody></table>
<br />
You can see similarities with PumpEngine becoming smarter, except that:<br />
<br />
- Here I’m going for 0..n multiplicity from the very beginning. The composite would still be an alternative.<br />
<br />
- GasAlarm has-a, not is-a, Alarm. That’s because a GasAlarm is entirely autonomous, with a Watch() responsibility which may actually be run in its own thread (I’m ignoring all the threading and timing issues here, much to the scorn of the existing Mine Pump literature :-).<br />
<br />
<br />
<b>The End (sort of)</b><br />
What is left in the controller now? Three distinct responsibilities:<br />
<br />
- Creation and configuration of all the other objects<br />
<br />
- Watching the SumpProbe and activating the SafePump when MustDrain (now literally a one liner :-)<br />
<br />
- Defining the threading model (I’ll ignore this; trust me, it’s a rather marginal issue now).<br />
<br />
The second responsibility can be easily factored out into a SumpPump class. After all, it’s all about draining the sump.<br />
I would leave the first to a MinePlant class, which is no longer a controller, but just a builder (in the pattern sense). If you’re in the IoC camp, the MinePlant is particularly trivial to write.<br />
<br />
Here is a final diagram. I’m still neglecting a few interfaces which could be useful for mocking and extendibility (like SumpProbe, still a concrete class here), and I’m also leaving some multiplicities implicit, both for sake of space and terseness.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii2arMylR2YwbNPfnbYlR_KAcI0_SusrNtFt_jNWnZj3_jkg2DBfJjOsysG38ScSI9Pl5oumZ7zCZvcrF6dK6sEJGOS2D9cWbM1x8MjmJaJToPxEjMYjMt9Y65SPCE0BYyzOPcYQ/s1600/Mine7.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii2arMylR2YwbNPfnbYlR_KAcI0_SusrNtFt_jNWnZj3_jkg2DBfJjOsysG38ScSI9Pl5oumZ7zCZvcrF6dK6sEJGOS2D9cWbM1x8MjmJaJToPxEjMYjMt9Y65SPCE0BYyzOPcYQ/s320/Mine7.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fig.8</td></tr>
</tbody></table>
In practice, it’s very likely that the three concrete gas sensors classes could be reduced to 3 instances of ThresholdGasSensor.<br />
<br />
<br />
<b>Another path not taken </b><br />
Some of you may recognize the SumpPump / SafeEngine chain as an "and" chain: we pump when we must drain <b>and </b>it’s safe to do so. As usual, if you work with this kind of systems on a regular basis, it could be tempting to generalize on that and build some kind of rule-based infrastructure.<br />
Again, it is not wrong to do so, but always remember the difference between a domain-based abstraction and a math-based abstraction. In my experience (and as I told before, I used to love math-based abstractions) the former is more easily understood and adopted, the latter is often misunderstood and abused. It’s not a general rule or “principle”. Just experience.<br />
<br />
<br />
<b>The inevitable extension</b><br />
Ok, so, I didn’t tell you the whole story. If you read the literature on the mine pump problem, you’ll find one extra requirement: <i>“if, due to a failure of the pump, the water cannot be pumped out, the mine must be evacuated”</i> (which I read as: if the water level does not fall after X minutes, the alarm must be triggered).<br />
<br />
In a centralized controller style, we know what to do: we add more stuff to the controller. Is the shape above suggesting a natural place for this responsibility? Is it ok to put it there? What about coupling and reuse? Where would you encode that knowledge? Is it better to change a few dependencies?<br />
<br />
Note: I’ve ignored the logging requirement as well. Logging is a cross-cutting concern in the structure above, which is kinda obvious, since I’ve been breaking apart a large class. There is little to be said/added here, if not that this is exactly why AOP should be taken more seriously.<br />
<br />
<br />
<b>Critics </b><br />
This is just one among many shapes. I took a few design choices, and this is the final result. You may take different choices, still leading to a controller-free shape. For instance, everything above is based on a polling loop (easier to deal with in safety critical stuff), but an event-based approach is ok too, and may lead you to a different structure.<br />
No shape is perfect. So, while I know I shouldn't be ending my post with a critic of my work, I'll ignore the good advice and do it anyway:<br />
<br />
- Structure is more complex. I have many classes, and quite a few dependencies. It is worth noting that in practice, most of those classes would be 5 to 10 lines long. Easy to write (bug-free), easy to test, and easy to understand. But the overall shape may not be easy. Here is where I see the value of a diagram as a communication tool.<br />
<br />
- The centralized controller is easier for the junior guy. Which is another way to say Worse is Better. Yeah, well, it’s true. Beginners are usually better at understanding complex logic than complex structure and interactions. There is also some evidence that a diagram can help there. But overall, it’s a non-trivial choice about the kind of software you want to write (for beginners or for expert programmers).<br />
<br />
- The process is no longer “visible”. This is true, as there is no longer a central piece of software encoding the entire process. It has been scattered among cooperating objects. This is another facet of the same problem: for those who don’t get OO, this shape is harder to understand. For those who do, it provides a number of non-functional benefits.<br />
<br />
- It’s easier to add stuff in the centralized controller. Again, this is a facet of the same problem, with a different slant. Every once in a while, someone tells me that when he’s working with a sophisticated structure, he needs to actually <b>think </b>before adding stuff. Where do I put this logic without breaking the conceptual integrity of the whole? When you have little or no structure to begin with, you just don’t care, and you can take the path of least resistance.<br />
Of course we know where this leads: to the big ball of mud, the natural destiny of those who surrender to gravity. Depending on your business model and professionalism, you may still not care.<br />
<br />
<br />
<b>Class size </b><br />
I mentioned that most (all) classes above would be 5 to 10 LOC. This is not what happens in most software, where there is a power law distribution for LOCs. Today, I got why. I’ll cover that in my next blog post.<br />
<br />
<br />
If you liked this post, you should <a href="https://twitter.com/#!/CarloPescio">follow me</a> on twitter!<br />
<br />
<br />
There is a follow-up to this post in "<a href="http://www.carlopescio.com/2012/03/episode-2-controller-strikes-back.html" target="_blank">Episode 2: the Controller Strikes Back</a>"<br />
<br />
... And another in "<a href="http://www.carlopescio.com/2012/07/no-controller-episode-3-inglorious.html" target="_blank">No Controller, Episode 3: Inglorious Objekts</a>"<br />
<br />Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com20tag:blogger.com,1999:blog-13967713.post-21225654823581093832012-02-01T15:12:00.001+01:002012-02-01T15:18:27.878+01:00Turn that If into a WhenI've been absent for a few months now. The thing is, I'm spending my little free time writing a non-trivial Android app. Of course, I'm also taking thousands of notes on software design, ready for a comeback :-).
Anyway, staying away from my blog so long is kinda painful :-), so here I am with a short post, inspired by a very simple <a href="http://www.carlopescio.com/2010/11/design-structure-and-decisions.html" target="_blank">design choice</a> I had to make.
<br/><br/>
When you start my little app for the first time, you get an EULA screen. If you accept the agreement, I'll initialize an internal database with some data and bring you the main screen. Ouch, did I say "if"?
<br/><br/>
Object-oriented programming was supposed to save us from switch/cases (replaced by polymorphism), and an if/else is a switch/case by another name. However, things like "if you accept the agreement" don't fit so well with polymorphism. You can make them fit, but it's not their natural shape. So we end up with just another (fat) controller.
<br/><br/>
On the other hand, perhaps I'm just saying it wrong. What if I change it into "<b>When</b> you accept the agreement..."? I don't know about you, but that instantly speaks "event" to me. Now, once we stop thinking about "conditions" and we think about "events", a few things happen:
<br/><br/>
- We can actually implement the concept on top of events if our language/library is event-based.
<br/><br/>
- We can fall back to the inner interface idiom, if we're in Java-land.
<br/><br/>
- We may even think <b>aspect</b> if we're bold enough.
<br/><br/>
Indeed, back in 2008, I wrote a post about reasoning in aspects and then implementing in objects (<a href="http://www.carlopescio.com/2008/04/can-aop-inform-oop-toward-soa-too-part.html" target="_blank">Can AOP inform OOP (toward SOA, too? :-) [part 1]</a>). The idea was exactly that in some cases (cross-cutting business rules) an event-based implementation could be a poor man's alternative to an aspect-based interception (the <a href="http://www.carlopescio.com/2008/04/can-aop-inform-oop-toward-soa-too-part_25.html" target="_blank">subsequent post</a> delved a bit more into some obliviousness issues and into the concept of cross-cutting business rules as candidate aspects).
<br/><br/>
Of course, we have to <b>recognize</b> the opportunity for an aspect (or event) based design. Sometimes, changing the words we use to describe [ourselves] the problem makes that easier. Next time you're facing an <b>if</b>, try changing it into a <b>when</b>. It won't always work, but it's worth trying :-).
<br/><br/>
That's it for now. Time to hit the road (snow actually :-). Places to go, clients to meet. Next time, I'm probably going to start a new series here, "life without a controller", because yeah, controll<b><a href="http://www.carlopescio.com/2011/04/your-coding-conventions-are-hurting-you.html" target="_blank">ers</a></b> ain't OO.
<br/><br/>
If you liked this post, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me</a> on twitter!Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com7tag:blogger.com,1999:blog-13967713.post-54633639190445620142011-10-17T18:56:00.000+02:002011-10-17T18:56:32.000+02:00You're solving the wrong problem<img src="http://www.eptacom.net/blog/wrong.jpg" align="left" style="margin-right:16px;"> It had to happen. After my post on <a href="http://www.carlopescio.com/2011/06/cut-red-wire.html" target="_blank">Yahtzee</a>, my agile friend (Marco) has come back with more code katas, adding to the pile we already discussed. His latest shot was <a href="http://codekata.pragprog.com/2007/01/kata_nine_back_.html" target="_blank">Kata Nine</a>, a.k.a. "Back to the CheckOut".<br />
His code was fine, really. Nothing fancy, basically a straightforward application of the Strategy pattern. You can find many similar implementations on the net (one of which prompted my tweet about ruby on training wheels, but it was more about the development style than the final result). Of course, my friend would have been disappointed if I had just said "yeah, well done!", so I had to be honest (where is the point of being friends if you can't be honest) and say "yeah, but you're solving the wrong problem".<br />
<br />
A short conversation followed, centered on the need to question requirements (as articulated by users and customers) and come up with the real, hidden requirements, the value of domain knowledge and the skills one would need to develop to become an effective analyst or "requirement engineer" (not that I suggest you do :-), with the usual drift into agility and such.<br />
<br />
I'll recommend that you read the problem statement for kata nine. It's not really necessary that you solve the problem, but it wouldn't hurt to think about it a little. I'm not going to present any code this time.<br />
<br />
<b>Kata Nine through the eye of a requirements engineer</b><br />
The naive analyst would just take a requirement statement from the user/customer, clean it up a little, and pass it downstream. That's pretty much useless. The skilled analyst will form a coherent [mental] model of the world, and move toward a deeper understanding of the problem. The quintessential tool to get there is simple, and we all learn to use it around age 4: it's called <b>questions</b> :-). Of course, it’s all about the quality of the questions, and the quality of the mental model we're building . In the end, it has a lot to do with the way we look at the world, as explained by the concept of <a href="http://en.wikipedia.org/wiki/Framing_(social_sciences)" target="_blank">framing</a>.<br />
<br />
The problem presented in kata nine can be framed as a set of billing rules. Rules, as formulated, are based on product type (SKU), product price (per SKU) and a discount concept based on grouping a fixed quantity of items (quantity depending on SKU) and assigning a fixed price to that group (3 items 'A' for $1.30). It might be tempting for an analyst to draw a class diagram here, and for a designer to introduce an interface over the concept of rule, so that the applicability and the effects of a rule become just an implementation detail (you wish :-).<br />
<br />
Note how quickly we're moving from the real-world issues of products, prices, discounts into the computing world, made of classes, interfaces, information hiding. However, the skilled analyst will recognize some underlying assumptions and ask for more details. Here are a few questions he may want to ask, and some of the reasoning that may follow. Some are trivial, some are not. Sometimes, two trivial questions conjure to uncover a nontrivial interplay.<br />
<br />
- rules may intersect on a single SKU (3 items 'A' for $1.30, 6 for $2.50). Is that the case? Is there a simplified/simplifying assumption that grouping by larger quantities will always lead to a more convenient price per unit? <br />
<br />
- without discounts, billing is a so-called <i>online process</i>, that is, you scan one item, and you immediately get to know the total (so far). You just add the SKU price, and never have to re-process the previous items. <br />
If you have <b>at most</b> one grouping-based discount rule per SKU, you still have a <i>linear process</i>. You add one item, and you either form a new group, or add to the spare items. Forming a new group may require to recalculate the billing (just for the spares).<br />
If you allow more than one rule for the same SKU, well, it depends. Under the assumption above about convenience for larger groups, you basically have a <i>prioritized list of rules</i>. You add one item, and you have to reprocess the entire set of items with the same SKU. You apply the largest possible quantity rule, get a spare set, apply the larger quantity rule to that spare set, and so on, until only the default "quantity 1" rule applies. This is a linear, deterministic process, repeated every time you add a new item.<br />
In practice, however, you may have a rule for 3, 4, 5 items of the same SKU. A group of 8 items could then be partitioned in 4+4 or 5+3. Maybe 5+3 (which would be chosen by a priority list) is more convenient. Maybe not. Legislation tends to favor the buyer, so you should look for the most favorable partitioning. This is no longer a linear process, and the corresponding code will increase in complexity, testing will be more difficult, etc. <br />
Perhaps we should exclude these cases. Perhaps we just discovered a new requirement for another part of the systems, where rules are interactively created (we don't expect to add rules by writing code, do we :-). We need to dig deeper.<br />
Note: understanding when a requirement is moving the problem to an entirely new level of complexity is one of the most important moments in requirement analysis. As I said before, I often use the term "nonlinear decision" here, implying that there is a choice: to include that requirement or not, to renegotiate the requirement somehow, to learn more about the problem and maybe discover that it's our lucky day and that the real problem is not what we thought. This happens quite often, especially when requirements are gathered from legacy systems. I discussed this in an old (but still relevant) paper: <a href="http://www.eptacom.net/pubblicazioni/pub_eng/probsol.pdf" target="_blank">When Past Solutions Cause Future Problems</a>, IEEE Software, 1997.<br />
<br />
- grouping by SKU is still a relatively simple policy. At worst, you recalculate the partitioning for all the items of the same SKU that you're scanning, and deal with a localized combinatorial problem. Is that the only kind of rule we need? What about the popular "buy 3 items, get the cheapest for $1", that is not based on SKU? What about larger categories, like "buy 3 items in the clothing department, get the cheapest for $1"? A naive analyst may ignore the issue, and a naive designer may think that dropping an interface on top of a rule will make this a simple extension. That's not the case.<br />
Once you move beyond grouping by SKU, you have to recalculate the entire partitioning for <b>all</b> scanned products every time a new product is scanned. You add a product B priced as $5. Which combination of products and rules results in the most favorable price? Grouping with the other 3 items with the same SKU, or with other 2 in the clothing department? Finding the most favorable partitioning is now a rule-based combinatorial process. New rules can be introduced at any time, so it's hard to come up with an optimization scheme. Beware the large family with two full carts :-). <br />
Your code is now two orders of magnitude harder than initially expected. Perhaps we should really renegotiate requirements, or plan a staged development within a larger timeframe than expected. What, you didn't investigate requirements, promised a working system in a week, and are now swamped? Well, you can always join the recurrent twitter riot against estimates :-).<br />
<br />
- Oh, of course you also have fidelity cards, don't you. Some rules apply only if you have a card. In practice, people may hand their card over only after a few items have been scanned. So you need to go back anyway, but that wouldn't be a combinatorial process per se, just a need to start over.<br />
<br />
<b>It gets much worse...</b><br />
There is an underlying, barely visible concept of time in the problem statement: <i>"this week we have a special offer"</i>. So, rules apply only during a time interval.<br />
<br />
Time-dependent rules are tricky. The skilled analyst knows that it's quite simple to verify if a specific <i>instant</i> (event) falls within an interval, and then apply the rules that happen to be effective at that instant. However, he also understands that a checkout is not an instantaneous process. It takes time, perhaps a few seconds, perhaps a few minutes. What if the two intervals overlap, that is, a rule is effective when checkout starts, but not when it ends (or vice versa)? The notion of "correctness", here, is relatively hard to define, and the store manager must be involved in this kind of decision. The interplay with the need to recalculate everything whenever a new item is scanned should be obvious: if you don't do anything about it, the implementation is going to choose for you.<br />
Note: traditionally, many businesses have tried to sidestep this sort of problem by applying changes to business rules "after hours". Banks, for instance, still rely a lot on nightly jobs, but even more so on the <i>concept</i> of night. This safe harbor, however, is being constantly challenged by a 24/7 world. Your online promotion may end at midnight in your state, but that could be morning in your customer's state.<br />
<br />
<b>... before it gets better :-)</b><br />
Of course, it's not just about finding more and more issues. The skilled analyst will quickly see that a fixed price discount policy (3 x $1.30) could easily be changed into a proportional discount ("20% off"). He may want to look into that at some point, but that's a secondary detail, because it's not going to alter the nature of the problem in any relevant way. Happy day :-).<br />
<br />
<b>Are we supposed to ask those questions?</b><br />
Shouldn't we just start coding? It's agile 2011, man. Implement the first user story, get feedback, improve, etc. It's about individual and interactions, tacit knowledge, breaking new ground, going into the unknown, fail fast, pivot, respond to change. Well, at least, it's just like that on Hacker News, so it must be true, right? <br />
I'm not going to argue with that, if not by pointing out an old post of mine on <a href="http://www.carlopescio.com/2006/07/self-fulfilling-expectations.html" target="_blank">self-fulfilling expectations</a> (quite a few people thought it was just about UML vs. coding; it's the old "look at the moon, not at the finger" thing). How much of the rework (change) is actually due to changing business needs, and how much is due to lack of understanding of the <b>current</b> business needs? We need to learn balance. Analysis paralysis means wasting time and opportunities. So does the opposite approach of rushing into coding.<br />
<br />
<b>We tried having an analyst, but it didn't work</b><br />
My pre-canned, context-free :-) answer to this is: "no, your obsolete coder is not a skilled analyst". Here are some common traits of the <b>unskilled</b> analyst:<br />
- ex coder, left himself become obsolete but knows something about the problem domain and the legacy system, so the company is trying to squeeze a little more value out of him.<br />
- writes lengthy papers called "functional specifications" that nobody wants to read.<br />
- got a whole :-) 3 days training on use cases, but never got the difference with a functional spec.<br />
- kinda knows entity-relationship, and can sort-of read a class diagram. However, he would rather fill 10 pages of unfathomable prose than draw the simplest diagram.<br />
- talks about web applications using CICS terminology.<br />
- the only Michael Jackson he ever heard about was a singer.<br />
- after becoming agile :-), he writes user stories like "as a customer, I want to pay my bill" (yeah, sure, whatever).<br />
- actually believes he can use a set of examples (or test cases) as a specification (good luck with combinatorial problems).<br />
<br />
More recurrent ineffective analysts: the marketer who couldn't market, the salesman who couldn't sale, the customer care who didn't care. Analysis is though. Just because Moron Joe couldn't do it, however, it doesn't mean it can't be done.<br />
<br />
<b>How to become a skilled analyst</b><br />
The answer is surprisingly simple:<br />
- you can't. It's too late. The discipline has been killed and history has been erased.<br />
- you shouldn't. Nobody is looking for a skilled analyst anyway. Go learn a fashionable language or technology instead.<br />
<br />
More seriously (well, <i>slightly</i> more seriously), the times of functional specifications are gone for good, and nobody is mourning. Use cases stifled innovation for a long while, then faded into background because writing effective use cases was hard, writing crappy use cases was simple, and people usually went for simple. Users stories are a joke, but who cares. <br />
<br />
What if you want to learn the way of the analyst, anyway? Remember, a skilled analyst is <b>not</b> an expert in a <b>specific</b> domain (although he tends to grow into an expert in several domains). He's not the person you pay to get answers. He's the person you pay to get <b>better questions</b> (and so, indirectly, better answers). <br />
The skilled analyst will ask different questions, explore different avenues, and find out different facts. So the essential skills would be observation, knowledge of large class of problems, classification, and description. Description. Michael Jackson vehemently advocated the need for a discipline of description (see "Defining a Discipline of Description," IEEE Software, Sep./Oct. 1998; it's behind pay walls, but I managed to find a free copy <a href="http://mcs.open.ac.uk/mj665/Missing6.pdf" target="_blank">here</a>, until it lasts; please go read it). He noted, however, that we lacked suck a discipline; that was 1998, and no, we ain't got one meanwhile.<br />
<br />
Now, consider the patterns of thought I just used above to investigate potential issues: <br />
Rules -> overlapping rules. <br />
Incremental problem -> Priority list -> Selective combinatorial problem -> Global combinatorial problem. <br />
Time interval -> overlapping intervals -> effective rules. <br />
The problem is, that kind of knowledge is nowhere to be learnt. You get it from the field, when it dawns on you. Or maybe you don't. Maybe you get exposed to it, but never manage to give it enough structure, so you can't really form a proto-pattern and reuse that knowledge elsewhere. Then you won't have 20 years of experience, but 1 year of experience 20 times over.<br />
<br />
Contrast this with more mature fields. Any half-decent mechanical engineer would immediately recognize cyclic stress and point out a potential <a href=" http://en.wikipedia.org/wiki/Fatigue_(material)" target="_blank">fatigue</a> issue. It's part of the basic training. It's not (yet) about solving the problem. It's about recognizing large families of recurrent issues.<br />
<br />
<b>C'mon, we got Analysis Patterns!</b><br />
Sort of. I often recommend Fowler's book, and even more so the lesser known volumes from David Hay (Data Model Patterns) and a few more, similar works. But honestly, those are not analysis patterns. They are "just" models. Some better and more refined than others. But still, they are just models. A pattern is a different beast altogether, and I'm not even sure we can have analysis patterns (since a pattern includes a solution resolving forces, and analysis is not about finding a solution). <br />
<br />
A few years ago <a href="http://www.carlopescio.com/2007/12/problem-frames.html" target="_blank">I argued</a> that <i>problem frame patterns</i> were closer to the spirit of patterns than the more widely known work from Fowler, and <a href="http://www.carlopescio.com/2008/02/problem-frames-and-dnc.html" target="_blank">later</a> used them to investigate the applicability of Coad's Domain Neutral Component (which is another useful weapon in the analyst arsenal).<br />
<br />
However, in a maturing discipline of requirements engineering, we would need to go at a much finer granularity, and build a true catalog of recurring real-world problems, complete with questions to ask, angles to investigate, subtleties to work out. No solutions. It's not about design. It's not about process reengineering. It's about learning the problem.<br />
<br />
Again, I don't think we will, but here is a simplified attempt at documenting the Rules/Timing stuff, in a format somewhat inspired by patterns, but still new (and therefore, tentative).<br />
<br />
<ul><li><b>Name</b></li></ul>Time-dependent Rules and Transactions<br />
<br />
<ul><li><b>Context</b></li></ul>We have a set of Rules that must be checked during a Transaction. Rules are effective from time T1 to time T2. Time can be expressed in absolute terms (e.g. December 25, 2011, 12:00 AM), recurring terms (every Friday, 12:00 to 17:00), etc. A Transaction is a set of actions, taking place over time, with a non-negligible duration (that is, a transaction is not an instantaneous event). Rules may have to be checked several times during the lifetime of a single Transaction.<br />
<br />
<ul><li><b>Problem</b></li></ul>The interval during which any given Rule is effective may overlap with the time interval required to carry over a transaction, that is, a rule may be effective when the transactions starts, but not when it ends, or vice versa, or may be valid for an interval beginning and ending inside the transaction time span (for long-lived transactions). Still, we need to guarantee some form of coherence in the observed behavior.<br />
<br />
<ul><li><b>Example</b></li></ul>[... the checkout problem would be a perfect fit here...]<br />
<br />
<ul><li><b>Issues and Questions</b></li></ul>- Is there any kind of regulation dictating the system behavior?<br />
- Would the effect of simply applying rules effective at different times be noticeable?<br />
- Can the transaction be simplified into an instantaneous event?<br />
- Can we just "freeze" the rule set when the transaction starts? (there are two facets here: the real-world implications of doing so, and the machine-side implications of doing so).<br />
- Can we replay the entire transaction every time a new event is processed (e.g. a new item is added to the cart), basically moving everything forward in time?<br />
- etc.<br />
<br />
I don't know about you, but I would love to have a place where this kind of knowledge could be stored, cleaned up, structured, and curated. <br />
<br />
<b>Conclusions</b><br />
It would be easy to blame it all on agility. After all, a lot of people have taken agility as an excuse for "if it's hard, don't do it". Design has been trivialized to SOLID and Beck's 4 principles. Analysis is just gone. Still, I'm not going to. I've seen companies stuck in waterfall. It ain't pretty. But it's more and more obvious that we're stuck in just another tar pit, where a bunch of gurus keep repeating the same mantras, and the discipline is not moving forward. <br />
<br />
Coding is fascinating. A passionate programmer will stay up late to work on some interesting problem and <i>watch his code work</i>. I did, many times, and I'm sure I'll still do for a long while. Creating models and descriptions does not seem to be on a par with that. There is some intellectual satisfaction in there too, but not near as much as seeing something actually <i>run</i>. In the end, this might be the simplest, most honest explanation of why, in the end, we always come back to code.<br />
<br />
Still, we need a place, a format, a curating community for long-term, widely useful knowledge that transcends nitty-gritty coding issues (see also my concept of <a href="http://www.carlopescio.com/2010/03/where-is-your-knowledge.html" target="_blank">Half-Life of a Knowledge Repository</a>). Perhaps we just need to make it fun. I like the way the guys at <a href="http://thefuntheory.com/" target="_blank">The Fun Theory</a> can turn everything into something fun (take a look at the piano stairs). Perhaps we should give requirements engineering another shot :-).<br />
<br />
<br />
If you read so far, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me</a> on twitter.<br />
<br />
<b>Acknowledgement</b><br />
The image on top is a copy of <a href="http://www.flickr.com/photos/johnnyjet/3290272880/" target="_blank">this picture</a> from Johnny Jet, released under a creative common license with permission to share for commercial use, with attribution.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com7tag:blogger.com,1999:blog-13967713.post-20700948959478977182011-09-02T16:52:00.001+02:002011-09-02T16:58:48.974+02:00Notes on Software Design, Chapter 15: Run-Time EntanglementSo, here I am, back to my unpopular :-) series on the Physics of Software. It's time to explore the run-time side of entanglement, or more exactly, begin to explore, as I'm trying to write shorter posts, hoping to increase frequency as well. <br />
A lot of time has gone by, so here is a recap from <a href="http://www.carlopescio.com/2010/11/notes-on-software-design-chapter-12.html" target="blank">Chapter 12</a>: <i>Two clusters of information are entangled when performing a change on one immediately requires a change on the other</i> (to maintain overall consistency). If you're new to this, reading chapter 12, 13 and 14 (dealing with the artifact-side of entanglement) won't hurt.<br />
<br />
Moving to the run-time side, some forms of entanglement are obvious (like redundant data). Some are not. Some are rather narrow in scope. Some are far-reaching. I'll cover quite a few cases in this post, although I'm sure it's not an exhaustive list (yet).<br />
<br />
<ul><li><b>Redundancy</b></li>
</ul>This is the most obvious form of run-time entanglement: duplicated information. <br />
Redundancy occurs even without your intervention: swap-file vs. RAM contents, 3rd-2nd-1st level cache vs. RAM and vs. caches in other cores/processors, etc; at this level, entanglement satisfaction is usually called <b>coherence</b>. <br />
In other cases, redundancy is an explicit design decision: database replicas, mem-cached contents, etc. At a smaller scale, we also have redundancy whenever we store the same value in two distinct variables. Redundancy means our system is always on the verge of violating the underlying information entanglement. That's why caches are notoriously hard to "get right" in heavily concurrent environments without massive locking (which is a form of <a href="http://www.carlopescio.com/2010/09/notes-on-software-design-chapter-10-run.html" target="blank">friction</a>). This is strictly related with the <a href="http://www.carlopescio.com/2011/05/cap-theorem-memristor-and-physics-of.html" target="blank">CAP Theorem</a>, which I have discussed months ago, but I'll come back to it in the future.<br />
Just like we often try (through hard-won experience) to minimize redundancy in artifacts (the <a href="http://media.pragprog.com/articles/may_04_oo1.pdf" target="blank">Keep it dry</a> principle), we learnt to minimize redundancy in data as well (through normalization, see below), but also to somehow control redundancy through design patterns like the observer. I'll get back to patterns in the context of entanglement in a future post.<br />
<br />
<ul><li><b>Referential Integrity</b></li>
</ul>This is perhaps less obvious, but easy to understand. <a href="http://en.wikipedia.org/wiki/Referential_integrity " target="blank">Referential integrity</a>, as known within the relational database culture, requires (quoting Wikipedia) that <i>any field in a table that is declared a foreign key can contain only values from a parent table's primary key or a candidate key.</i> That means you cannot remove some records unless you update others, or that you cannot create some records unless some other is in place. That's a form of entanglement, and now we can start to grasp another concept: the idea of transaction is born out of the need to have intermediate micro-states that do not respect the entanglement constraint, yet make those micro-states invisible. As far as the external observer is concerned, time is quantized, and transactions are the ticks.<br />
Interestingly, <a href="http://en.wikipedia.org/wiki/Database_normalization" target="blank">database normalization</a> is all (just? :-) about imposing a <b>specific form</b> on data entanglement. Consider a database that is not in <a href="http://en.wikipedia.org/wiki/Second_normal_form" target="blank">second normal form</a>; I'll use the same example of the Wikipedia page for simplicity. Several records contain the same information (work location). If we change the work location for Jones, we have to update three records. This is entanglement through redundancy, of course. Normalization <b>does not</b> remove entanglement. What it does is to impose a particular form on it: through foreign keys. That makes it easier to deal with entanglement within the relational paradigm, but it is not without consequences for modularity (which would require an entire post to explore). In a sense, this "standardization" of entanglement form could be seen as a form of dampening. Views, too, are a practical dampening tool (on capable hands). More on this in a future post.<br />
<br />
<ul><li><b>Class Invariant</b></li>
</ul>This, I must confess, was not obvious at all to recognize at first, which is weird in insight. A <a href="http://en.wikipedia.org/wiki/Class_invariant" target="blank">class invariant</a> is a constraint on the internal state (data) of any object of that class. That means, of course, that those data <b>cannot</b> change independently. Therefore, they are entangled by definition. More precisely, a class invariant can often be stated as a set of predicates, and each predicate is revealing some form of entanglement between two or more data members.<br />
Curiously enough (or perhaps not :-), invariant must hold at the beginning/end of externally observable methods, though it can be broken during the actual execution of those methods. That is to say, public methods play the role of transactions, making time quantized once again.<br />
<br />
<ul><li><b>Object Lifetime</b></li>
</ul>Although this can be seen as a special case of class invariant (just like a cascade delete is ascribed to referential integrity) I'd like to make this explicit, for reasons we'll understand better in a future post. Some objects cannot outlive others. Some objects cannot be created without others. Reusing the terminology of <a href="http://www.carlopescio.com/2011/01/notes-on-software-design-chapter-13-on.html" target="blank">chapter 13</a>, this is a D-D or C-C form of entanglement. The same form of entanglement can be found underneath several DB normalization concepts, and thanks to the RT/Artifact dualism, will turn out to be a useful concept in the artifact world as well.<br />
<br />
<b>Toward a concept of tangling through procedural knowledge</b><br />
All the above, I guess, it's pretty obvious in hindsight. It borrows heavily on the existing literature, which always considered data as something that can be subject to constraints. The risk here is to fall into the "information as data" tar pit. Information is not data. Information includes procedural knowledge (see also my old posts <a href="http://www.carlopescio.com/2008/09/lost.html" target="blank">Lost</a> <a href="http://www.carlopescio.com/2008/09/and-found.html" target="blank">... and Found?</a> for more on the concept of Information and Alexandrian centers). Is there a concept of entanglement for procedural knowledge?<br />
<br />
Consider this simple spreadsheet:<br />
<br />
<table border="1" cellspacing="0" cellpadding="4"><tr style="background-color:#a0a0a0;"><td></td><td>A</td><td>B</td></tr>
<tr><td style="background-color:#a0a0a0;">1</td><td>10</td><td>20</td></tr>
<tr><td style="background-color:#a0a0a0;">2</td><td>200</td><td>240</td></tr>
</table><br />
That may look like plain data, but behind the curtain we have two formulas (procedural knowledge):<br />
<br />
A2=A1*B1<br />
B2=A2*1.2<br />
<br />
In the picture above, the system is stable, balanced, or as I will call it, in a <b>steady state</b>. Change a value in A1 or B1, however, and procedural knowledge will kick in. After a short interval (during which the system is unbalanced, or as I'll call it, in a <b>transient state</b>), balance will be restored, and <b>all data entangled through procedural knowledge</b> will be updated.<br />
A spreadsheet, of course, is the quintessential <a href="http://en.wikipedia.org/wiki/Dataflow" target="_blank" >Dataflow</a> system, so it's also the natural <i>trait d'union</i> between knowledge-as-data and procedural knowledge. It should be rather obvious, however, that a similar reasoning can be applied to any kind of event-driven or reactive system. That kind of system is sitting idle (steady state) until something happens (new information entering the system and moving it to a transient state). That unleashes a chain of reactions, until balance has been restored.<br />
<br />
Baby steps: now, what was the old-school view of objects (before the watering down took place)? Objects were like tiny little machines, communicating through messages. Right. So invoking a method on an object is not really different than sending the object a message. We're still upsetting its steady state, and the procedural knowledge inside the method is restoring balance (preserving the invariant, etc.)<br />
<br />
So, what about the traditional, top-down, imperative batch system? It's actually quite straightforward now: invoking a function (even the infamous main function) means pushing some information inside the system. The system reacts by moving information around, creating new information, etc, until the output is ready (the new steady state). The glue between all that information is the procedural knowledge stored inside the system. Procedural knowledge is basically encoding the entanglement path for dynamic information.<br />
<br />
<b>Say that again?</b><br />
This is beginning to look too much like philosophy, so let's move to code once again. In any C-like languages, given this code:<br />
<pre>int a2 = a1 * b1;
int b2 = a2 * 1.2;
int a3 = a1 + b1;</pre>we're expecting the so-called control-flow to move forward, initializing a2, then b2, then a3. In practice, the compiler could reorder the statements and calculate a3 first, or in between, and even the processor could execute the two instructions "out of order" (see the concept of "observable behavior" in the C and C++ standard). What we are actually saying is that a2 can be calculated given a1 and b1, and how; that b2 can be calculated given a2, and how; and that a3 can be calculated given a1 and b1, and how. The "can be calculated given ..." is an abstract interpretation of the code. That abstract interpretation provides the entanglement graph for run-time entities. The "how" part, as it happens, is more germane to the function than to the form, and in this context, is somehow irrelevant. <br />
<br />
What about the control-flow primitives, like iteration, choice, call, etc? Iteration and choice are nothing special: at run-time, a specific branch will be taken, for a specific number of times, resulting in a specific run-time entanglement between run-time entities. Call (procedure call, function call) is somewhat different, because it reveals the fractal nature of software: whatever happens inside the function is hidden (except via side effects) from the caller. The function will carry out its job, that is, will restore the fine-grained input-output balance upon invocation.<br />
<br />
This is the general idea of procedural knowledge: given an input, control-flow goes on, until the program reaches a steady state, where output has been produced. As noted above, due to the fractal nature of information, this process takes place at different levels. A component instance may still be in a transient state while some object contained in that instance is already back to a steady state.<br />
<br />
In this sense, a procedure (as an artifact) is how we teach a machine to calculate an output given an input, through the stepwise creation and destruction of entangled run-time information. Phew :-).<br />
<br />
<b>Next steps</b><br />
The beauty of the Run-Time/Artifact dualism is that whatever we learn on one side, we can often apply on the other side (with the added benefit that many things are easier to "get" on one side or another). <br />
Here, for instance, I've introduced the concept of steady and transient state for run-time information. The same reasoning can be applied to the artifact side. Whenever you create, update or delete an artifact A1, if there is any other artifact that is entangled with A1, the system goes into an unbalanced, transient state. You have to apply work (energy) to bring it back to the steady state, where all the entangled information is properly updated (balanced). A nice quote from Leonardo da Vinci could apply here: "Motion is created by the destruction of balance" (unfortunately, I can't find an authoritative source for the quote :-). Motion in the information space is created by unsettling the system (moving to a transient state) until entanglement is finally satisfied again, and a steady state is reached. This is as true in the artifact space as in the run-time space.<br />
Overall, this has been more like a trampoline chapter, where a few notions are introduced for the benefits of forthcoming chapters. Here is a short list of topics that I'll try to explore at some point in the future:<br />
<br />
- The truth about multiplicity, frequency, and attraction/rejection in the run-time world.<br />
- As above, in the artifact world.<br />
- Moving entanglement between run-time and artifacts (we do this a lot while we design).<br />
- The Entanglement Diagram.<br />
- A few examples from well-known Design Patterns.<br />
- Cross-cutting concerns and Delocalized plans: relationships with entanglement.<br />
- More on distance, entanglement, probability of failure, and the CAP theorem.<br />
- Dampening (isolation). This will open a new macro-chapter in the Physics of Software series.<br />
<br />
If you read so far, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me</a> on twitter, or perhaps even share this post (I dare you :-)<br />
<br />
Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com2tag:blogger.com,1999:blog-13967713.post-54596607488564962342011-06-29T17:18:00.008+02:002012-09-20T07:36:23.480+02:00Cut the red wire!<img align="left" src="http://www.eptacom.net/blog/redWire.jpg" style="margin-right: 16px;" />We're all familiar with this scene: the good guys trying to save the day (or the world) by defusing a bomb. Quickly running out of time, they have to cut a wire. Failure is not an option. The red bar won't turn into a green bar.<br />
<br />
What if you had to write code that works <b>the very first time</b>? What if you had <b>no chance to test</b> your code, let alone fixing bugs? You got only one chance to run your code and save the world. Would that change the way you write? The way you think? How? What can you learn from that?<br />
<br />
Lest you think I'm kinda crazy: I'm not actually suggesting that you shouldn't test your code. I'm not actually suggesting that you stop doing your test-driven thing (if you do) or that you forget your test plan (if you have one), etc etc. <br />
Consider this like a stretching exercise. We may learn something useful just by going outside our comfort zone. Give it a try.<br />
<div style="clear: both;">
</div>
<br />
<a name='more'></a><br /><br />
<b>Yahtzee</b><br />
[sorry guys, gotta go into Dr. Mallard mode for a while; you can breeze through this section, if you're the TL;DR type]<br />
<br />
I came to write this post, and to choose Yahtzee as the underlying problem, mostly because I like coincidences :-). <br />
<br />
A friend of mine, a relatively young programmer in love with everything agile, every once in a while shows me some "code kata" that he has completed. Not long ago, he came up with the <a href="http://codingdojo.org/cgi-bin/wiki.pl?KataYahtzee" target="_blank">KataYahtzee</a>. His code was, to use his words "completely covered by tests". After all, that's what today's gurus are <b>constantly</b> talking about. Tests, tests, more tests (yawn). However, the code emerging from all those tests was in my opinion rather crappy (one of the benefits of friendship is that you can say this sort of things in a benign way and have them interpreted in a benign way :-). I muttered something about a better alternative, promised I would send him my code, then got caught in other stuff and never actually implemented a thing.<br />
<br />
Still, shortly after, I was visiting the blog of an acquaintance of mine (xpmatteo), and after leaving a <a href="http://matteo.vaccari.name/blog/archives/602" target="_blank">comment</a>, I read a few more things including a post about Yahtzee and the removal of conditionals and loops.<br />
<br />
That was yet another coincidence: I joked with my agile friend before, telling him that sooner or later, they would have raised the bar from anti-if to anti-while, anti-for, anti-pattern-matching, anti-list-comprehension, anti-recursion, etc :-). But I also truly liked the idea that by changing the underlying data structure you could make some rules extremely simple to check (I'm a big fan of the "smart data – stupid procedures" approach). Anyway, that reminded me of my promise, so I fired up my editor and then I closed it :-), because I'd better read the actual rules first (some would call me anti-agile for that, but I don't see why I can't leverage existing knowledge, when available).<br />
<br />
Having never played Yahtzee before, I quickly discovered that the <a href="http://en.wikipedia.org/wiki/Yahtzee" target="_blank">standard rules</a> didn't match the rules in the kata page. However, you'll find them under "scoring variations". Reading wikipedia sorted out some of my doubts (is "13333" a double pair? No), so I was ready to actually write some code. <br />
<br />
<b>Before you get any further</b><br />
This is where I should ask you to try it on your own. You probably won't, but if you do, remember: you want your code to run the very first time. You can write a set of test cases, but if one fails, the game is over (unless the test case is wrong).<br />
<br />
Just to add a little juice: the modern twist on the opening scene is that shortly after the heroes manage to defuse the bomb, the countdown is restarted. So, here is your second little bomb coming: if it actually works the first time, change the rules from the "kata version" to the traditional Yahtzee rules, quickly :-), and run a new set of test cases. We're counting on you.<br />
<br />
<b>Picking my tools</b><br />
My <i>alma mater</i> was completely sold on formal methods, and I used to like them too. In practice, over a number of years now, I had very few chances (and then reasons) to apply formal methods in the real world. Therefore, I <b>won't suggest</b> that you develop a formal proof of correctness for your code before you run it. <br />
<br />
The problem with a formal proof is that (in many cases) the proof is at least as complex as the code, sometimes more. I won't trust my only chance on a proof that is potentially more obscure than my code. After all, Donald Knuth <a href="http://www-cs-faculty.stanford.edu/~knuth/faq.html" target="_blank">notoriously said</a> <i>"Beware of bugs in the above code; I have only proved it correct, not tried it"</i>. Sure, I may try to check my proof :-) with the help a theorem prover, but that would be cheating: in a sense, the prover would be <i>testing</i> my proof. <br />
<br />
Another well-known technique to develop mission-critical code is the use of inspections and reviews. But then again, this is sort of cheating, because I would be using people to test my code (in their heads). That's a no-no. <br />
<br />
So, having ruled out old-school stuff like <a href="http://en.wikipedia.org/wiki/Cleanroom_Software_Engineering" target="_blank">Cleanroom Software Engineering</a>, and having removed TDD from the table at very beginning, let's move on. <br />
<br />
<b>Is that even possible?</b><br />
Of course, before we even start trying, it makes sense to ask if there is actually any hope to get a program working the very first time. Well, for simple problems, it's certainly possible. For instance, if you ask me to calculate the area of a circle with (say) 5 digits accuracy given the radius, I may just write something like:<br />
<pre>double Area( double radius )
{
return radius * radius * 3.1415926 ;
}</pre>
<br />
I would probably use a statically checked language (which is a bit like cheating, but not too much :-), just to catch any typo. After that, I would feel rather confident.<br />
<br />
Ok, that was easy. Indeed, it would have been even simpler/safer if my language had a PI constant. The language would have been closer to the problem I was trying to solve. Which is exactly what we need: a language where our problem can be explained in a simple, straightforward way. Here, I think, is the key to my approach to writing code that runs the first time (I actually do that in real life, just not every time – more on this later on). I could say it shortly like this: <b>Everything is simple when it has a direct mapping to your language</b>. <br />
Quoting Ward Cunningham (from "Clean Code" by Bob Martin), "You know you are working on clean code when each routine you read turns out to be pretty much what you expected. You can call it beautiful code when the code also makes it look like the language was made for the problem". I won't trust my only chance on nothing less than beautiful code :-).<br />
<br />
Now, we can read this as the frequently preached "look for the best language for the problem at hand", and in a sense it's true, but truth is, once you get past a few obvious cases, you ain't gonna find a language that was made exactly for your problem. The alternative approach, of course, is to create a language that is made exactly for your problem. No, I'm not suggesting that you design a language and write a compiler first: remember, you can only run your code once. That includes your brand-new compiler. Do it if you want, but don't blame me if you manage to annihilate our civilization.<br />
<br />
Looking back at the trivial Area function, you can easily see that my language had a useful abstraction (a "double", meaning IEEE 754 floating-point math) together with a few basic operations. Say that I trust those abstractions and the underlying hardware. I also trust my Area function, simple and small as it is. Area, therefore, could be thought of as a <b>language extension</b>, a new trusted abstraction.<br />
<br />
I could simply call this <b>"Compositional Correctness"</b>: get your low-level abstractions right. Then build more abstractions on top of those, each so small and simple that it is trivially correct. Of course, you can do it the other way too (top-down), or you can mix the strategies. In a sense, it's just an application of C. A. R. Hoare's principle: <i>"there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult"</i>. Well, sometimes, it doesn't have to be difficult.<br />
<br />
<b>Listen to Your Problem</b><br />
I can almost hear the word "DSL" in the background, and indeed, a <a href="http://en.wikipedia.org/wiki/Domain-specific_language" target="_blank">Domain Specific Language</a> may turn a complex problem into a trivial one. For instance, HTML is a pretty good DSL to create formatted, hyperlinked documents. <br />
DSLs are often associated with humongous graphical / modeling tools (and I'm not gonna trust my only chance with one of those) or with flexible languages like Ruby. Still, remember the old Bell Labs motto: "library design is language design". You don't really need to write your language: you can extend your language just by writing a library (like with Area). <br />
Of course, you need a programming language with some built-in flexibility (which is why I dislike overly restrictive languages), but you don't need to get too fancy. To prove my point, I'll use (modern) C# in what follows. I could have used good ol' C++ as well. Java would make a few things a little more cumbersome than I'd like to, but it could still be a viable option. Dynamic languages would do just fine. As usual, design decisions are way more important than little language-specific details.<br />
<br />
A DSL should speak the language of your problem. Therefore, the first step toward the creation of a DSL should be to carefully listen to your problem and understand the inner nature, the basic concepts upon which the problem structure is built. In a better world, this would be a widely researched, frequently applied process, but in this world, it's not (although you can find a few noteworthy works on DLSs). In practice, I first try to understand whether the problem "fits" with some of the categories I'm familiar with, like:<br />
<br />
- is it a unification / pattern matching problem? For instance, I could model a double pair like (x,x,y,y,z), and leave it as an implicit convention of the language that different letters will necessarily match different faces. This looks promising, until you look at some rules (like Large straight) that don't really fit that model. Sure, you can stretch the language a little, and allow literals: by the way, this is the approach taken by the original <a href="http://rubyquiz.com/quiz19.html" target="_blank">Ruby Quiz</a> solution, which inspired the kata, which inspired this post. However, if you look at the original Yahtzee rules for a Large straight, they won't fit with literals either (it's just five sequential dice). Pattern matching is not the real Yahtzee language. As an aside, having read the Ruby code, it just doesn't like like the kind of code I would trust to run flawlessly the first time :-).<br />
<br />
- is it an incremental counting problem? What I really loved in Matteo's post was the idea that you could simply keep a count of faces, and that would easily provide a score for several rules. However, he already showed that such technique won't cover all rules.<br />
<br />
- is it a map/reduce kind of problem? It doesn't really look like that, because the "map" is supposed to take a (k1,v1) pair and return a list of (k2,v2) pairs, to be grouped and reduced. I could shoehorn Yahtzee into this, but it doesn't look like a natural fit for its rules.<br />
<br />
- is it a filter and projection problem (a-la SQL)? This may actually work. Conditions can be expressed as filters. I can start with a sequence and, through a series of trivial filters, end up with an empty sequence (if there is no match for the rule) or with the surviving dice. At that point, it's basically a matter of taking the sum. Hmm, perhaps this could work. Time to dig a little deeper.<br />
<br />
<b>Why not SQL, then?</b><br />
At this point, it might be tempting to simply use SQL as a language. Alternatively, if you're familiar with .NET, you may want to try LINQ, which is basically a SQL-like syntax to play with collections. Well, why not. Here is a Pair scoring expressed as a LINQ statement:<br />
<pre>2 *
(from d in dice group d by d into g
where g.Count() >= 2 orderby g.Key
descending select g.Key).FirstOrDefault();</pre>
<br />
Part of the trick here is that the select expression returns a collection of integers, and that FirstOrDefault will return the first element (the highest face with a pair in the original sequence) or 0, which is the default value for int. It works with any kind of sequence (IEnumerable). In my tests (yes, I tested this :-), I just used an array of integers to represent dice.<br />
<br />
The problem is, however, that this code has to be carefully read. I have seen production code full of statements like that, and to put it gently, it sucks. <br />
On the upside, it's control-flow free, so it should please a few people out there :-). More seriously, the absence of control flow should probably by a byproduct of the "right" Yahtzee language. Control flow makes code harder to read in one pass. If I have to trust the code, I'd like to read it and "get it" in one pass. Note, however, that absence of control flow is not a guarantee of readability (or testability, or whatever else). Standard SQL, for instance, is control-flow free, but I can hardly say that a lengthy query mixing inner and outer joins can be easily read and trusted. The LINQ query above has no control flow, but I wouldn't like my code to read like that. Part of the problem lies with the LINQ syntax, but another part of the problem is that we're relying on standard data structures (collections) and standard SQL-like instructions, and they don't speak the <b>problem language</b>.<br />
<br />
<b> Sketching the Yahtzee language</b><br />
Language design is difficult and requires that we pay attention to many details and facets. We want the "procedural" part to become trivial: this is the point of the entire exercise. Sketching the language requires that we focus on a few rules, while keeping the others (or at least some of them) in the back of our mind, because the "right" language must be a good fit for most of them (ideally, all of them). However, picking two similar rules to begin with allows us to focus slightly better, without being too narrow. I'll pick the Pair and Double Pair rules.<br />
<br />
We also have to choose a language style. The "language style" inside Area was the familiar "infix operations" borrowed from math. Personally, I can't see that as a good fit for the Yahtzee rules. I'm pretty sure, however, that I'd like my rules to become one-liners, something you can understand by reading it once. I think a <a href="http://en.wikipedia.org/wiki/Fluent_interface" target="_blank">Fluent Interface</a> style will fit that role better.<br />
<br />
At this stage, I would not focus too much on [domain] classes. Sure, some classes are there just for the picking, as Bertrand Meyer said. Die, Face, Roll, Rule, Score, etc. However, when you design a small DSL, starting with classes is not necessarily the right approach. More often than not, classes will emerge as the (Alexandrian) <b>centers</b> of the conversations we're building. Still, we have to start somewhere, so I'll have an imaginary "roll" object (of unknown class) representing the roll we want to score against a rule.<br />
<br />
Let's give it a try. A trivial pair of one-liners for our rules could be:<br />
<pre>roll.GetHighestPair().Score();
roll.GetTwoHighestPairs().Score();</pre>
<br />
however, this is <b>wrong</b>. Sure, it looks like good old stepwise refinement, but in fact I'm just pushing the burden of Yahtzee rules on roll's shoulders. Rules can grow and change, roll should be a stable abstraction. <br />
Even this little step in the wrong direction, however, can teach us something: what if there is no pair? I surely don't want my rules to be choke-full of conditionals, as that would wreck my "read it once and see it's right" principle. So, we can already give some structure to our language:<br />
<br />
we are manipulating a collection of die (roll)<br />
the collection may go through several "stages" (the "filters" or "selections")<br />
at every stage, the collection may get empty, but never a null reference (that will avoid conditionals)<br />
the score of an empty collection is zero (which is consistent with Yahtzee)<br />
<br />
Ok, let's give it another try:<br />
<pre>roll.GetPairs().GetHighest().Score();
roll.GetPairs().GetTwoHighest().Score();</pre>
<br />
this is marginally better, because we formed a few smaller concepts. We can ask a roll for pairs; we can ask a pairs collection for the highest (by face: that should be made more explicit in our language), or the two highest. However, it's still very tailored, and it does not support any other rule. As we progress, the other rules have to get slightly more in the foreground to help us shaping our language (yeah, it's really a <a href="http://en.wikipedia.org/wiki/Gestalt_psychology" target="_blank">Gestalt</a> process, but don't get me started on this :-). <br />
<br />
The problem with the concept of pair is that it's not general enough to appear in our language. We need something more general. In the end, Yahtzee as a selection language is mostly about filtering by face or by occurrences of dice, grouped by faces. Perhaps a class would help here, a group of dice with the same face:<br />
<pre>class DiceGroup
{
public DiceGroup(int f, int c)
{
Face = f;
Count = c;
}
public int Face
{
get;
private set;
}
public int Count
{
get;
private set;
}
public int Score()
{
return Face * Count;
}
}</pre>
<br />
Now, I know some of you have been brainwashed :-) to the point that you feel the need to write a load of test cases for this class, but let's face it: it's basically as complex as Area, actually even less. I made it more complex than strictly needed, by using properties instead of plain public data, but that's giving me something back: immutability. A DiceGroup cannot be changed after construction. Not having to reason about mutable objects helps a lot in writing safe code, and Yahtzee is not a performance-critical problem, so I can afford it. DiceGroup is also a stable abstraction. Either you use it as-is, or you scrap it and use something else. There is no point in tweaking DiceGroup. Hence, a <b>real</b> economic theory for software design (as opposed to babbling about the need to refactor every single line you ever wrote) would tell us that there is little value in drowning <b>this specific class</b> in trivial test cases. <br />
Provocation: what if the best way to minimize the cost of change was to find a number of useful, small abstractions that <b>do not change</b> at all, or rarely do, because they're in frictionless contact with the nature of the problem?<br />
<br />
Now that I have a DiceGroup, what is a Roll? Well, a Roll might be just a sequence of DiceGroup. I can take my 5 dice, organize them in 1 to 5 DiceGroup objects (depending on the actual faces) and call that sequence a Roll. I'll ignore the details of how to do that for now, and get back to it later. Let's say that a Roll will be immutable too, so whenever we filter a Roll, we get a new Roll.<br />
<br />
So, say that I roll my dice and I get 4, 4, 1, 4, 3. I'll organize them into three DiceGroup objects: {4,3}, {1,1}, {3,1} (read them as {Face,Count}). To score them as Pair, I would have to remove the two DiceGroup with Count < 2, but that's not enough: {4,3} must also be turned into {4,2}, because the idea of Pair is that you max out at 2 occurrences. Hmm. That's just like trimming. I could write Pairs just like:<br />
<pre>roll.TrimCountTo(2).TopFace().Score();</pre>
<br />
where both TrimCountTo and TopFace return a new Roll, filtered and changed as needed. The rule is pretty readable, and given the language conventions above about returning an empty sequence if there is no match for the filter, I'll just get 0 when there are no pairs.<br />
<br />
TwoPairs would require a little extension to the language, because TopFace is again a bit too tailored to the Pair rule. The language itself is also telling us that (if we listen closely) because a TopFace method should return a DiceGroup, not a Roll. That makes the language asymmetric, so operations are harder to combine, and I don't want that. Lucky enough, this is easy to fix: why not having a TopByFace(n) method returning a roll with the n best DiceGroup (by face)? Now I can write Pair and TwoPairs like:<br />
<pre>roll.TrimCountTo(2).TopByFace(1).Score();
roll.TrimCountTo(2).TopByFace(2).Score();</pre>
<br />
[Un]surprisingly enough, those two little functions will get us many other rules for free:<br />
ThreeOfAKind: roll.TrimCountTo(3).Score();<br />
FourOfAKind: roll.TrimCountTo(4).Score();<br />
<strike>Yahtzee: r.TrimCountTo(5).Score(); </strike> <b>[Edit: wrong, see comment]</b><br />
Chance: roll.Score();<br />
<br />
Yo, I like this. Rules are trivially correct. They can be created by combining a few operators. This looks like the right DSL, or part of it. You can't express all the Yahtzee rules with just two operators (TrimCountTo and TopByFace). Ones, Twos, ..., Sixes are out. Well, they can be easily brought in by mirroring TrimCountTo with TrimFaceTo. So, I will have:<br />
<br />
Ones: roll.TrimFaceTo(1).Score();<br />
etc...<br />
<br />
This is almost <b>too</b> simple, isn't it? Well, when it gets <b>that</b> easy, it's because the language "was made for the problem", to quote Ward again, or because our solution is in frictionless contact with the forcefield, to bring in Alexander once more.<br />
<br />
As usual, the thinking process is more important than code here. I <b>am</b> getting feedback. Backtalk, actually. I'm just not getting it the way most people are used / taught to (by executing code). I'm getting feedback through a <a href="http://www.eptacom.net/pubblicazioni/pub_eng/ListenToYourToolsAndMaterials.pdf" target="_blank">reflective conversation with my material</a>. I know, this is not what everyone is telling you. And please understand I'm not implying that you should stop testing your code. I'm just taking the path less traveled by to see what happens.<br />
<br />
What's left? Small straight (1,2,3,4,5), Large straight (2,3,4,5,6), Full house (two of a kind + three of a kind, exactly: as per instructions, (4,4,4,4,4) is not a Full house). These rules cannot be expressed yet. You may see a pattern here, however. The straights are basically trimming by face, but with multiple faces. Full house is trimming by count, but with multiple counts. This may suggest an improvement to our language: turn TrimCountTo into TrimCountsTo, and TrimeFaceTo into TrimFacesTo. Care must be taken, however, not to break the simplicity of the language by creating too powerful instructions. Perhaps an orthogonal operator should be brought in. This is already a very long post, and I still have a lot of code to show, so I'll leave this extension out. <br />
<br />
<b>The Structure of the Yahtzee Language</b><br />
DiceGroup was trivial. I basically did that bottom-up. Roll might not be just as trivial, and so far we only know that we need a TrimCountTo, a TrimFaceTo, and a Score method. Now we have two forces to balance:<br />
We want Roll to be as trivial as possible. It has to work the first time! Honestly, this is the keystone now. DiceGroup was so trivial to be obviously correct. Rules are now so obvious that we can happily trust them. We need Roll to be so simple that we can trust it too.<br />
We want our DSL to be extendible. We're missing a few rules, and the "standard" Yahtzee rules were different as well. I don't want to tweak Roll to change or extend the rules: remember what I said about finding <b>stable</b> abstractions.<br />
<br />
I've been talking about [programming] language design in the past, and how "good" languages should be largely extendible. A few built-in core concepts supporting most of the language itself, which would then largely be in a library. It's time to apply the same principle to the Yahtzee Language.<br />
<br />
Here is where our actual implementation language can help or hinder. C# is almost helpful here. A dynamic language, or a language with structural conformance for interfaces, would make the following code much shorter and easier to read. Anyway, the key concept here is that of an <b>open class</b>, something that we can extend without breaking the basic abstraction and without creating subtypes (while still not breaking encapsulation). What we need is a "simple" Roll class with the core methods and a set of Extension Methods (in C# parlance) which implements the "upper layer" of our language. Score() will end up in Roll, while TrimCountTo, TrimFaceTo, and TopByFace will land in a RollLanguage class. Therefore, adding support for more rules won't require any change to existing classes. Just implement new extension methods. Of course, that in turn requires Roll to be flexible enough to support basically all the desirable extension methods.<br />
<br />
In practice, it's not that hard. Let's start with RollLanguage. Ideally, it would be something like this:<br />
<pre>static class RollLanguage
{
public static Roll TopByFace(this Roll groups, int minCount)
{
//trivial code here
}
public static Roll TrimCountTo(this Roll groups, int minCount)
{
//trivial code here
}
public static Roll TrimFaceTo(this Roll groups, int face)
{
//trivial code here
}
}</pre>
<br />
Let's go ahead and implement TrimFaceTo. This would be my first shot:<br />
<pre>public static Roll TrimFaceTo(this Roll groups, int face)
{
return groups.Where(g => g.Face == face);
}</pre>
<br />
note that I'm using LINQ (not exactly, I'm using the Linq extension method Where, not the LINQ syntax). However, this is a very simple expression. Sure, you have to know C# and .NET, but that's basically returning a sequence with all the DiceGroup which have the specified face. Unfortunately that won't compile :-), for two simple reasons:<br />
<br />
1) Roll must be an IEnumerable<DiceGroup>, because that's what Where expects.<br />
2) Where will then return an IEnumerable<DiceGroup>, not a Roll.<br />
<br />
(1) Is relatively trivial to fix, and while doing so, we'll sketch Roll as well:<br />
<pre>class Roll : IEnumerable<DiceGroup>
{
public IEnumerator<DiceGroup> GetEnumerator()
{
return diceByFace.GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return diceByFace.GetEnumerator();
}
private IEnumerable<DiceGroup> diceByFace;
}</pre>
<br />
This is a boilerplate code (partially generated by Visual Studio), that I wouldn't need with dynamic languages (or with structural conformance). It's pretty trivial though: Roll has-a and is-a IEnumerable<DiceGroup>. It implements the interface by forwarding to the data member. I'm rather confident that it's correct.<br />
<br />
(2) Requires that we tweak our code a little (again, this wouldn't be necessary in a more flexible programming language). Code speaks better than words here:<br />
<pre>public static Roll TrimFaceTo(this Roll groups, int face)
{
return groups.Where(g => g.Face == face).AsRoll();
}
public static Roll AsRoll(this IEnumerable<DiceGroup> r)
{
return new Roll(r);
}</pre>
<br />
All methods in RollLanguage can be implemented over Linq, but have to end with AsRoll() to convert the IEnumerable into an actual Roll. Yeah, it sucks, but the alternative (getting rid of DiceGroup and Roll altogether) is even worse (I'll touch on this later on). An implicit conversion operator would be ok, but C# does not allow implicit conversion from interfaces.<br />
<br />
So, here is TrimToCount:<br />
<pre>public static Roll TrimCountTo(this Roll groups, int minCount)
{
return groups.Where(g => g.Count >= minCount).Select(
g => new DiceGroup(g.Face, minCount)).AsRoll();
}</pre>
<br />
Honestly, this is a borderline function. You need to be somewhat confident with Linq to get it the first time through. Basically, I'm filtering by count (that's the easy part) and then I'm capping the count by creating a new DiceGroup with the same face and minCount as the actual count. I can trust this, but if you don't, I'll understand :-). Simpler code is welcome!<br />
<br />
So far I managed to avoid any kind of conditional. I didn't want any IF in my rules, and I didn't need any in RollLanguage thus far. However, my most readable TopByFace implementation is if-based:<br />
<pre>public static Roll TopByFace(this Roll groups, int minCount)
{
if( groups.Count() < minCount )
return Roll.Empty();
else
return groups.OrderByDescending(g => g.Face).Take(minCount).AsRoll();
}</pre>
<br />
The idea is simple: if I don't have at least minCount groups, the filter fails and an empty roll must be returned. Otherwise, I can simply use Linq again, sort my groups by face (descending) and take the first minCount groups. Again, this might be borderline for someone uncomfortable with Linq. Looks pretty trivial to me, but simpler code would be welcome; actually, if you are an anti-if devotee, feel <b>morally obliged</b> to show me how to improve that code <b>and</b> get rid of the if (using a ternary operator is not a real alternative, of course). Pushing the IF in a custom version of Take is possible, but that would be just moving it around (with an interesting trade/off in efficiency Vs. reusability).<br />
<br />
By writing this code, we already got a set of requirements for Roll. Must be (and have) an IEnumerable<DiceGroup>; must have a Score() method; must have an Empty() static method returning an empty sequence; must be constructable from a IEnumerable<DiceGroup> (trivial); we also need the "real" constructor, taking a sequence (or array) of integers (the 5 dice) and doing the actual grouping. We have already seen part of the code above, so I'll add the trivial parts here and leave the "hard" constructor last:<br />
<br />
<pre>class Roll : IEnumerable<DiceGroup>
{
public Roll(IEnumerable<DiceGroup> dice)
{
diceByFace = dice;
}
public int Score()
{
return diceByFace.Sum(g => g.Score());
}
public static Roll Empty()
{
return empty;
}
private IEnumerable<DiceGroup> diceByFace;
private static int[] emptyArray = new int[0];
private static Roll empty = new Roll(emptyArray);
}</pre>
<br />
OK. Once again, this is so trivial that I can trust it. Now let's get over with the grouping; lucky me, I can simply use Linq once again:<br />
<pre>public Roll(int[] dice)
{
diceByFace = dice.GroupBy(x => x).Select(
g => new DiceGroup(g.First(), g.Count()));
}</pre>
<br />
Once again, this is borderline if you aren't familiar with Linq. I'm grouping by faces, and for each group I'm building a DiceGroup object with face and count. I'd like a simpler syntax (g.First() is not exactly obvious) but this is what Linq provides me with. Usually, I'd also add an assertion that dice has 5 elements. I left that out to keep my code shorter.<br />
<br />
Interestingly, although I ignored performance, concurrency, etc so far, it would be rather simple to translate all this code in high-performance C++, using mutable stack-based objects, because in Yahtzee we're always dealing with (at most) 5 dice, so everything could be pre-allocated at fixed size. I would have to re-create some of the Linq extension methods, but C++11 has got lambda, so the overall style could stay the same.<br />
<br />
<b>Wrap it up (and extend it)</b><br />
We're basically through. At this point, everything else is cosmetic. How do I group rules? Do I need a Rule interface? Well, in C#, I could just use a delegate for that. Can I just use member functions in a Yahtzee class and get over with it? Sure, unless we're concerned about a hierarchy of Yahtzee games (standard, non-standard, etc). Also, organizing rules into an array or dictionary might be the simplest way to glue them to the UI (which I'm not going to write). Since this is all basically irrelevant to our quest, I just wrote a set of one-line member functions.<br />
<br />
Part 2 of the problem was to change rules to the original Yahtzee scoring. For instance, adopting the original rules a Three-Of-A-Kind will get you the sum of all dice (not just the 3 of a kind). The simplest way, perhaps a bit too technical, would be to do it like this:<br />
<pre>Math.Sign( ScoreThreeOfAKind( roll ) ) * roll.Score()</pre>
<br />
Alternatively, you can create new filters, returning the original roll (if the filter is passed) or an empty roll. It's basically trivial now. <br />
<br />
<b>[Edit: see comment and code to see how I did it in the end]</b>
<br />
<b><br /></b>
In the end, I didn't create classes like Die or Face. They would add some value (by forcing constraints) but I don't want to get too serious with this code :-). <br />
<br />
The entire code is <a href="http://www.eptacom.net/blog/Yahtzee.cs" target="_blank">here</a>. For simplicity, I used a single file. I added a few tests in the end, so that I could either succeed or fail when I finally ran the code (it worked!). In the end, I sort of like it. It's not perfect, but is pretty good. Feel free to improve it, in which case, I'd like to know!<br />
<br />
<b>Winding down</b><br />
Even though I've written so much, there would be much more to say. I'll give you the short version:<br />
<br />
<b>Bad abstraction vs. Good abstraction</b><br />
strictly speaking, I wouldn't need any "base" abstraction like DiceGroup and Roll in my Yahtzee language. I could write every single function as an extension method of standard library classes. Why should I use concrete, non-reusable terms like Roll and DiceGroup when I could just use <b>IEnumerable<IGrouping<int, int>></b>? Arguably, my code would be "more abstract", "more mathematic", and therefore "better" by sticking to abstract, domain-independent interfaces. Here, I think, my programming style differs from some. I used to like the idea of abstracting the domain <i>away</i> from my code, perhaps 20 years ago or so. I guess it was some sort of academic imprinting. However, over time I moved into a different coding style, where abstractions are taken from the problem domain, not from abstract math. This allows me to focus on the real-world, concrete problem I want to solve; to speak the language of the problem, with people and with my code; and to leave behind me code that can be more easily understood by people who know about the problem domain, and perhaps a little less about computer science. I hinted at this issue in the past, as in my <a href="http://www.carlopescio.com/2008/10/microblogging-is-not-my-thing.html" target="_blank">mind map</a> about habitable software, and even before, without ever taking time to do it justice. Perhaps this post will fill part of that gap.<br />
<br />
<b>Code quality</b><br />
As I discussed my code with my friend, he sort of liked it. He couldn't fully admit it, because hey, I had no tests in place, but he obviously felt it was much easier to read <b>and</b> to evolve. Of course, popular authors would say that it's bad code anyway: "Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well-encapsulated it is" (Michael C. Feathers, "Working Effectively with Legacy Code"). This is pretty harsh, because if I had to choose between the code above (with this blog post as the dreaded "comprehensive documentation") and some crappy code entirely covered by tests, I would have a hard time choosing the latter. <br />
Curiously enough, for a very long time testability has been like the stepchild of -ilities. In the past few years, the pendulum swung too far on the other side, and now it's all about tests. <br />
Since it's way too easy to babble and pontificate, I'd like someone to write much better code than mine (and yeah, <b>completely different</b>, thank you) using TDD, and show me the process through which tests are driving him toward that code. If you ain't got code, please don't bother leaving a comment about the greatness of TDD and how dumb I am for even trying to write code without using tests. OTOH, if you got code to back up your claims, you're more than welcome. <br />
<br />
<b>Limits and confessions</b><br />
I'm making a habit to conclude with a few confessions, so here I come. I don't always write code this way. I tend to: I like my code to work the first time. However, it took me more than the allotted time of 1h to complete my mission. Writing the code was easy. Thinking about the language, trying out a few different styles, rejecting options, etc, was not, and almost never is. <br />
The "little DSL" approach requires a clarity of thought that we can't always afford, and it does not necessarily pay off. Yahtzee is also quite simple, and requirements were well-known. Again, that's a luxury we don't face so often (although I don't buy the idea that we never know what we're doing; deep inside uncertainty there is often a core of stable concepts, if you take the time to dig it out).<br />
There are also limits to the applicability of these techniques. Not every single problem lends itself well to the DSL approach. As usual, we have to be flexible, understand the nature of the problem, and choose the best design <b>strategy</b>. A DSL was just particularly well-suited for Yahtzee.<br />
Indeed, a few years ago I spent some time doing a [little] upfront design, followed by implementation, for the <a href="http://www.carlopescio.com/2007/07/get-ball-rolling-part-1-of-4-i-guess.html" target="_blank">bowling problem</a>. Back then, my aim was to show that you don't have to give up on OOP and OOD so easily, and that by sticking to OOD you won't end up with a LOC monster as it was suggested elsewhere. I didn't use a DSL back then, but of course, it's quite possible to combine a little upfront OOD with a little DSL-like library with a little emergent design. This is very close to what I usually do when I have to code something on my own. Group dynamics are different, and I tend to adapt my techniques to the context.<br />
<br />
<b>What can we learn from this stuff?</b><br />
A lot, I hope :-). Software development is about acquiring and encoding knowledge into an executable format (my usual quote from Phillip Armour). Our code can be extremely simplified through a careful selection of the language we use to encode that knowledge: when the language is aligned with the problem domain, many layers of complexity simply fall down. Your programming language is usually <b>not</b> aligned with your problem. It's your job to make it so, by introducing the right abstractions.<br />
So, next time you're facing some complex issue, ask yourself: what if this code had to run the very first time?<br />
<br />
<b>Acknowledgement</b><br />
The image on top is adapted from <a href="http://www.flickr.com/photos/wstryder/5643458755/" target="_blank">this picture</a> from Lauri Rantala, released under a creative common license with permission to modify the picture and for commercial use.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com14tag:blogger.com,1999:blog-13967713.post-72941083465405708692011-05-21T11:58:00.002+02:002011-05-21T12:23:51.872+02:00The CAP Theorem, the Memristor, and the Physics of SoftwareWhere I present seemingly unrelated facts that are, in fact, not unrelated at all :-).<br /><br /><b>The CAP Theorem</b><br />If you keep current on technology, you're probably familiar with the proliferation of <a href="http://en.wikipedia.org/wiki/NoSQL " target="_blank" >NoSQL</a> , a large family of non-relational data stores. Most NoSQL stores have been designed for internet-scale applications, where large data stores are preferably deployed using an array of loosely connected low-cost servers. That brings a set of well-known issues to the table:<br /><br />- we'd like data to be consistent (that is, to respect all the underlying constraints at any time, especially replication constraints). We call this property <b>C</b>onsistency.<br /><br />- we'd like fast response time, even when some server is down or under heavy load. We call this property <b>A</b>vailability.<br /><br />- we'd like to keep working even if the system gets partitioned (a connection between servers fails). We call this property <b>P</b>artition tolerance.<br /><br />The CAP Theorem (formerly the Brewer's Conjecture) says that we can only choose two properties out of three.<br /><br />There is a lot of literature on the CAP Theorem, so I don't need to go in further details here: if you want a very readable paper covering the basics, you can go for <a href="http://www.julianbrowne.com/article/viewer/brewers-cap-theorem" target="_blank" >Brewer's CAP Theorem</a> by Julian Browne, while if you're more interested in a formal paper, you can refer to <a href="http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf " target="_blank" >Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services</a> by Seth Gilbert and Nancy Lynch.<br /><br />There is, however, a relatively subtle point that is often brought up and seldom clarified, so I'll give it a shot. Most people realize that there is some kind of "connection" between the concept of availability and the concept of partition. If a server is partitioned away, it is by definition not available. In that sense, it may seem like there is some kind of overlapping between the two concepts, and overlapped concepts are bad (orthogonal concepts should be preferred). However, it is not so:<br /><br />- Availability is an <b>external</b> property, something the clients of the data store can see. They either can get data when they need it, or they can't.<br /><br />- Partitioning is an <b>internal</b> property, something clients are totally unaware of. The data store has to deal with that on the inside.<br /><br />Of course, given the fractal nature of software, in a system of systems we may see lack of availability at one level, because of the lack of partition tolerance at a lower level (or vice-versa, as an unavailable node may not be distinguishable from a partitioned node).<br /><br />To sum up, while the RDBMS world has usually approached the problem by choosing Consistency and Availability over Partition tolerance, the NoSQL world has often chosen Availability and Partition tolerance, leading to the now famous Eventually Consistent model.<br /><br /><b>The Memristor, or the Value of a Theory</b><br />As a kid, I spent quite a bit of time hacking electronic stuff. I used to scavenge old TV sets and the like, and recover resistors, capacitors, and inductors to use on very simple projects. Little I knew that, theoretically, there was a missing passive element: the <a href="http://en.wikipedia.org/wiki/Memristor" target="_blank" >memristor</a>. <br /><br />Leon Chua developed <a href="http://www.cpmt.org/scv/meetings/chua.pdf" target="_blank" >the theory</a> in 1971. It took until 2008 before someone could finally create a working memristor, using nanoscale techniques that would have looked like science fiction in 1971. Still, the theory was there, long before, pointing toward the future. At the bottom of the theory was the realization that the three known passive elements could be defined by a relationship between four concepts (current, voltage, charge, and flux-linkage). By investigating all possible relationships, he found a missing element, one that was theoretically possible, but yet undiscovered. He used the term <i>completeness</i> to indicate this line of reasoning, although we often call this kind of property <i>symmetry</i>.<br /><br /><b>Software Entanglement</b><br />In <a href="http://www.carlopescio.com/2010/11/notes-on-software-design-chapter-12.html" target="_blank" >Chapter 12</a> of my ongoing series on the nature of software, I introduced the concept of Entanglement, further discusses in <a href="http://www.carlopescio.com/2011/01/notes-on-software-design-chapter-13-on.html" target="_blank" >Chapter 13</a> and <a href="http://www.carlopescio.com/2011/02/notes-on-software-design-chapter-14.html" target="_blank" >Chapter 14</a> (for the artifact world).<br /><br />Here, I'll reuse my early definition from Chapter 12: <b>Two clusters of information are entangled when performing a change on one immediately requires a change on the other.</b> <br /><br />Entanglement is constantly at work in the artifact world: change a class name, and you'll have to fix the name everywhere; add a member to an enumeration, and you'll have to update any switch/case statement over the enumeration; etc. It's also constantly at work in the <b>run-time</b> world. <br /><br />Although I haven't formally introduced entanglement for the run-time world (that would/will be the subject of the forthcoming Chapter 15), you can easily see that maintaining a database replica creates a run-time entanglement between data: update one, and the other must be immediately (atomically, as we use to say) updated. Perhaps slightly more subtle is the fact that referential integrity is strongly related to run-time entanglement as well. Consistency in the CAP theorem, therefore, is basically Entanglement satisfaction.<br /><br />Once we understand that, it's perhaps easier to see that the CAP theorem applies well beyond our traditional definition of distributed system. Consider a multi-core CPU with independent L1 caches. Cache contents become entangled whenever they map to the same address. CPU designers usually go with CA, forgetting P. That makes a lot of sense, of course, because we're talking about in-chip connections.<br /><br />That's sort of obvious, though. Things get more interesting when we start to consider the run-time/artifact symmetry. That's part of the value of a [good] theory.<br /><br /><b>A CAP Theorem for Artifacts</b><br />My perspective on the <a href="http://www.PhysicsOfSoftware.com" target="_blank" >Physics of Software</a> is strongly based on the run-time/artifact dualism. Most forces apply in both worlds, so it is just natural to bring ideas from one world to another, once we know how to translate concepts. Just like symmetry allowed Chua to conceive the memristor, symmetry may allow us to extend concepts from the run-time world to the artifact world (and vice-versa). Let's give the CAP theorem a shot, by moving each run-time concept to the corresponding notion in the artifact world:<br /><br />Consistency: just like in the run-time world, it's about satisfying Entanglement. While change in the run-time world is a C/U/D action on data, here is a C/U/D action on artifacts. If you add a member to an enumerated type, your artifacts are consistent when you have updated every portion of code that was enumerating over the extension of that type, etc.<br /><br />Availability: just like in the run-time world, is the ability to access a working, up-to-date system. If you can't deploy your new artifacts, your system is not available (as far as the artifact world is concerned). The old system may be up and running, but your new system is not available to the user.<br /><br />Partition tolerance: just like in the run-time world, it means you want your system to be usable even if some changes can't be propagated. That is, some artifacts have been partitioned away, and cannot be reached. Therefore, some sort of partial deployment will take place.<br /><br />Well, indeed, <b>you can only have two</b> :-). Consider the artifacts involved in the usual distributed system:<br /><br />- one or more servers<br />- one or more clients<br />- a contract in between<br /><br />The clients and the server (as artifacts – think source code not processes) are U/D entangled over the contract: change a method signature (in RPC speak), or the WSDL, or whatever represents your contract, and you have to change both to maintain consistency.<br /><br />What happens if you update [the source code of] a server (say, to fix a serious bug), and by doing so you need to update the contract, but cannot update [the source code of] a client [timely]? This is just like a partition. Think of a distributed development team: the team working on the client, for a reason or another, has been partitioned away.<br /><br />You only have two choices:<br /><br />- let go of Availability: you won't deploy your new server until you can update the client. This is the artifact side of not having your database available until all the replicas are connected (in the run-time world).<br /><br />- let go of Consistency: you'll have to live with an inconsistent client, using a different contract.<br /><br /><b>Consequences</b><br />Chances are that you:<br />- Shiver at the idea of letting go of Consistency.<br />- Think versioning may solve the problem.<br /><br />Of course, versioning is a coping strategy, not a solution. In fact, versioning is usually proposed in the run-time world of data stores as well. However, versioning your contract is like pretending that the server has never been updated. It may or may not work (what if your previous contract had major security flaws that cannot be plugged unless you change the contract? What if you changed your server in a way that makes the old contract impossible to keep? etc). It also puts a significant burden on the development team (maintaining more source code than strictly needed) and may significantly slow down development, which is another way to say it's impacting availability (of artifacts).<br /><br />It is important, however, that we thoroughly understand <b>context</b>. After all, any given function call is a contract, and we surely want to maintain consistency as much as we can. So, when does the CAP Theorem apply to Artifacts? Partition tolerance and Availability must be part of the equation. That requires one of the following conditions to apply:<br /><br />- Independent development teams. The Facebook ecosystem, for instance, is definitely prone to the artifact side of the CAP Theorem, just like any other system offering a public API to the world at large. You just can't control the clients.<br /><br />- A large system that cannot be updated in every single part (unless you accept to slow down deployment/forgo availability). A proliferation of clients (web based, rich, mobile, etc) may also bring you into partitioning problems.<br /><br />If you work on a relatively small system, and you have control of all clients, you're basically on a safe field – you can go with Consistency and Availability, because you just don't have significant partitions. Your development team is more like a multi-core CPU than a large distributed system.<br /><br />Once we understand that versioning is basically a way to pretend changes never happened on the server side, is there something similar to <a href="http://en.wikipedia.org/wiki/Eventual_consistency" target="_blank" >Eventual Consistency</a> for the artifact side? <br /><br /><a href="http://martinfowler.com/bliki/TolerantReader.html" target="_blank" >TolerantReader</a> may help. If you put extra effort into your clients, you can actually have a working system that will eventually be consistent (when source code for clients will be finally updated) but still be functional during transition times, without delaying the release of server updated. Of course, everything that requires discipline on the client side is rather useless in a Facebook-like ecosystem, but might be an excellent strategy for a large system where you control the clients.<br /><br />Interestingly, Fowler talks about the virtues of TolerantReader as if it was always warranted for a distributed system. I tend to disagree: it makes sense only when some form of development-side partitioning can be expected. In many other cases, an enforced contract through XSD and code generation is exactly what we need to guarantee Consistency and Availability (because code generation helps to quickly align the syntactical bits – you still have to work on the semantic parts on both sides). Actually, the extra time required to create a TolerantReader will impact Availability on the short term (the server is ready, the client is not). Context, context, context. This is one more reason why I think we need a Physics of Software: without a clear understanding of forces and materials, it's too easy to slip into the fallacy that just because something worked very well for me [in some cases], it should work very well for you as well. <br /><br />In fact, once you recognize the real forces, it's easy to understand that the problem is not limited to service-oriented software, XML contracts, and the like. You have the same problem between a database schema and a number of different clients. Whenever you have artifact entanglement <b>and</b> the potential for development partitioning (see above) you have to deal with the artifact-side CAP Theorem.<br /><br />Moving beyond TolerantReader (which is not helping much when your client is <b>sending</b> the wrong data anyway :-), when development partitioning is expected, you need to think about a flexible contract from the very beginning. Facebook, for instance, allows clients to specify which fields they want to receive, but is still pretty rigid in the fields they have to send.<br /><br />This is one more interesting R&D challenge that is not easily solved with today's technology. Curiously enough, in the physical world we have long understood the need to use soft materials at the interface level to compensate for imperfections and unexpected variations of contact surfaces. Look at any recent, thermally insulated window, and most likely you're gonna find a seal between the sash and the frame. Why are seals still needed in a world of high-precision manufacturing? ;-)<br /><br /><b>Conclusions</b><br />Time for a confession: when I first thought about all the above, I had already read quite a bit on the CAP Theorem, but not the <a href="http://www.cs.berkeley.edu/~brewer/cs262b-2004/PODC-keynote.pdf" target="_blank" >original presentation</a> from Eric Brewer where the conjecture (it wasn't a theorem back then) was first introduced. The presentation is about a larger topic: challenges in the development of large-scale distributed systems.<br /><br />As I started to write this post, I decided to do my homework and go through Brewer's slides. He identified three issues: state distribution, consistency vs. availability, and boundaries. The first two are about the run-time world, and ultimately lead to the CAP theorem. While talking about Borders, however, Brewers begins with run-time issues (independent failure) and then moves into the artifact world, talking (guess what!) about contracts, boundary evolution, XML, etc. Interestingly, nothing in the presentation suggests any relationship between the problems. Here, I think, is the value of a good theory: as for the memristor, it provides us with leverage, moving what we know to a higher level.<br /><br />In fact, another pillar of the Physics of Software is the Decision Space. I'm pretty sure the CAP theorem can be applied in the decision space as well, but that will have to wait.<br /><br />Well, if you liked this, stay tuned for Chapter 15 of my <a href="http://www.carlopescio.com/search/label/NOSD" target="_blank" >NOSD</a> series, share this post, <a href="https://twitter.com/#!/CarloPescio" target="_blank" >follow me</a> on twitter, etc.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com0tag:blogger.com,1999:blog-13967713.post-49675243775274293172011-04-20T15:06:00.008+02:002012-03-23T22:44:03.296+01:00Your coding conventions are hurting youIf you take a walk in my hometown, and head for the center, you may stumble into a building like this:<br />
<br />
<img src="http://www.eptacom.net/blog/names2.jpg" /><br />
<br />
from a distance, it's a relatively pompous edifice, but as you get closer, you realize that the columns, the ornaments, everything is fake, carefully painted to look like the real thing. Even shadows have been painted to deceive the eye. Here is another picture from the same neighbor: from this angle, it's easier to see that everything has just been painted on a flat surface:<br />
<br />
<img src="http://www.eptacom.net/blog/names1.jpg" /><br />
<br />
Curiously enough, as I wander through the architecture and code of many systems and libraries, I sometimes get the same feeling. From a distance, everything is object oriented, extra-cool, modern-flexible-etc, but as you get closer, you realize it's just a thin veneer over procedural thinking (and don't even get me started about being "modern").<br />
<br />
<b>Objects, as they were meant to be</b><br />
Unlike other disciplines, software development shows little interest for classics. Most people are more attracted by recent works. Who cares about some 20 years old paper when you can play with Node.js? Still, if you never took the time to read <a href="http://www.smalltalk.org/downloads/papers/SmalltalkHistoryHOPL.pdf" target="_blank">The Early History of Smalltalk</a>, I'd urge you to. Besides the chronicles of some of the most interesting times in hw/sw design ever, you'll get little gems like this: <i>"The basic principal of recursive design is to make the parts have the same power as the whole." For the first time I thought of the whole as the entire computer and wondered why anyone would want to divide it up into weaker things called data structures and procedures. Why not divide it up into little computers, as time sharing was starting to? But not in dozens. Why not thousands of them, each simulating a useful structure? </i><br />
<br />
That's brilliant. It's a vision of objects like little virtual machines, offering specialized services. Objects were meant to be smart. Hide data, expose behavior. It's more than that: Alan is very explicit about the idea of methods as <b>goals</b>, something you want to happen, unconcerned about how it is going to happen.<br />
<br />
Now, I'd be very tempted to write: "unfortunately, most so-called object-oriented code is not written that way", but I don't have to :-), because I can just quote Alan, from the same paper: <i>The last thing you wanted any programmer to do is mess with internal state even if presented figuratively. Instead, the objects should be presented as <i>sites of higher level behaviors more appropriate for use as dynamic components</i>. [...]It is unfortunate that much of what is called “object-oriented programming” today is simply old style programming with fancier constructs. Many programs are loaded with “assignment-style” operations now done by more expensive attached procedures. </i><br />
<br />
That was 1993. Things haven't changed much since then, if not for the worse :-). Lot of programmers have learnt the <i>mechanics</i> of objects, and forgot (or ignored) the underlying <i>concepts</i>. As you get closer to their code, you'll see procedural thinking oozing out. In many cases, you can see that just by looking at class names.<br />
<br />
<br />
<b>Names are a thinking device</b><br />
Software development is about discovering and encoding knowledge. Now, humans have relatively few ways to encode knowledge: a fundamental strategy is to <b>name</b> things and concepts. Coming up with a good name is hard, yet programming requires us to devise names for:<br />
- components<br />
- namespaces / packages<br />
- classes<br />
- members (data and functions)<br />
- parameters<br />
- local variables<br />
- etc<br />
People are basically lazy, and in the end the compiler/interpreter doesn't care about our beautiful names, so why bother? Because finding good names is a journey of discovery. The names we choose shape the dictionary we use to talk and think about our software. If we can't find a good name, we obviously don't know enough about either the problem domain or the solution domain. Our code (or our model) is telling us something is wrong. Perhaps the metaphor we choose is not properly aligned with the problem we're trying to solve. Perhaps there are just a few misleading abstractions, leading us astray. Still, we better <a href="http://www.eptacom.net/pubblicazioni/pub_eng/ListenToYourToolsAndMaterials.pdf" target="_blank">listen</a>, because we are doing it wrong.<br />
<br />
As usual, balance is key, and focus is a necessity, because clock is ticking as we're thinking. I would suggest that you focus on class/interface names first. If you can't find a proper name for the class, try naming functions. Look at those functions. What is keeping them together? You can apply them to... that's the class name :-). Can't find it? Are you sure those functions belong together? Are you thinking in concepts or just slapping an implementation together? What is that code <b>really</b> doing? Think method-as-a-goal, class-as-a-virtual-machine. Sometimes, I've found useful to think about the opposite concept, and move from there.<br />
<br />
<b>Fake OO names and harmful conventions</b><br />
That's rather bread-and-butter, yet it's way more difficult than it seems, so people often tend to give up, think procedurally, and use procedural names as well. Unfortunately, procedural names have been institutionalized in patterns, libraries and coding conventions, therefore turning them into a major issue. <br />
<br />
In practice, a few widely used conventions can seriously stifle object thinking:<br />
- the -er suffix<br />
- the -able suffix<br />
- the -Object suffix<br />
- the I- prefix<br />
<br />
of these, the I- prefix could be the most harmless in theory, except that in practice, it's not :-). Tons of ink has been wasted on the -er suffix, so I'll cover that part quickly, and move to the rest, with a few examples from widely used libraries.<br />
<br />
<b>Manager, Helper, Handler...</b><br />
Good ol' Peter Coad used to say: <i>Challenge any class name that ends in "-er" (e.g. Manager or Controller). If it has no parts, change the name of the class to what each object is managing. If it has parts, put as much work in the parts that the parts know enough to do themselves</i> (that was the "er-er Principle"). That's central to object thinking, because when you need a Manager, it's often a sign that the Managed are just plain old data structures, and that the Manager is the smart procedure doing the real work.<br />
<br />
When Peter wrote that (1993), the idea of an Helper class was mostly unheard of. But as more people got into the OOP bandwagon, they started creating larger and larger, uncohesive classes. The proper OO thing to do, of course, is to find the right cooperating, cohesive concepts. The lazy, fake OO thing to do is to take a bunch of methods, move them outside the overblown class X, and group them in XHelper. While doing so, you often have to weaken encapsulation in some way, because XHelper needs a privileged access to X. Ouch. That's just painting an OO picture of classes over old-style coding. Sadly enough, in a wikipedia article that I won't honor with a link, you'll read that "Helper Class is one of the basic programming techniques in object-oriented programming". My hope for humanity is only restored by the fact that the article is an orphan :-).<br />
<br />
I'm not going to say much about Controller, because it's so popular today (MVC rulez :-) that it would take forever to clean this mess. Sad sad sad.<br />
<br />
Handler, again, is an obvious resurrection of procedural thinking. What is an handler if not a damn procedure? Why do something need to be "handled" in the first place? Oh, I know, you're thinking of events, but even in that case, EventTarget, or even plain Target, is a much better abstraction than EventHandler.<br />
<br />
<b>Something-able</b><br />
Set your time machine to 1995, and witness the (imaginary) conversation between naïve OO Developer #1, who for some reason has been assigned to design the library of the new wonderful-language-to-be, and naïve OO Developer #2, who's pairing with him along the way.<br />
N1: So, I've got this Thread class, and it has a run() function that it's being executed into that thread... you just have to override run()...<br />
N2: So to execute a function in a new thread you have to extend Thread? That's bad design! Remember we can only extend one class!<br />
N1: Right, so I'll use the Strategy Pattern here... move the run() to the strategy, and execute the strategy in the new thread.<br />
N2: That's cool... how do you wanna call the strategy interface?<br />
N1: Let's see... there is only one method... run()... hmmm<br />
N2: Let's call it Runnable then!<br />
N1: Yes! Runnable it is!<br />
<br />
And so it began (no, I'm not serious; I don't know how it all began with the -able suffix; I'm making this up). Still, at some point people thought it was fine to look at an interface (or at a class), see if there was some kind of "main method" or "main responsibility" (which is kinda obvious if you only have one) and name the class after that. Which is a very simple way to avoid thinking, but it's hardly a good idea. It's like calling a nail "Hammerable", because you known, that's what you do with a nail, you hammer it. It encourages procedural thinking, and leads to ineffective abstractions.<br />
<br />
Let's pair our naïve OO Developer #1 with an Object Thinker and replay the conversation:<br />
<br />
N1: So, I've got this Thread class, and it has a run() function that it's being executed into that thread... you just have to override run()...<br />
OT: So to execute a function in a new thread you have to extend Thread? That's bad design! Remember we can only extend one class!<br />
N1: Right, so I'll use the Strategy Pattern here... move the run() to the strategy, and execute the strategy in the new thread.<br />
OT: OK, so what is the real abstraction behind the strategy? Don't just think about the mechanics of the pattern, think of what it really represents...<br />
N1: Hmm, it's something that can be run...<br />
OT: Or executed, or performed independently...<br />
N1: Yes<br />
OT: Like an Activity, with an Execute method, what do you think? [was our OT aware of the UML 0.8 draft? I still have a paper version :-)]<br />
N1: But I don't see the relationship with a thread...<br />
OT: And rightly so! Thread depends on Activity, but Activity is independent of Thread. It just represents something that can be executed. I may even have an ActivitySequence and it would just execute them all, sequentially. We could even add concepts like entering/exiting the Activity... <br />
(rest of the conversation elided – it would point toward a better timeline :-)<br />
<br />
Admittedly, some interfaces are hard to name. That's usually a sign that we don't really know what we're doing, and we're just coding our way out of a problem. Still, some others (like Runnable) are improperly named just because of bad habits and conventions. Watch out.<br />
<br />
<br />
<b>Something-Object</b><br />
This is similar to the above: when you don't know how to name something, pick some dominant trait and add Object to the end. Again, the problem is that the "dominant trait" is moving us away from the concept of an object as a virtual machine, and toward the object as a procedure. In other cases, Object is dropped in just to avoid more careful thinking about the underlying concept. <br />
<br />
Just like the -able suffix, sometimes it's easy to fix, sometimes is not. Let's try something not trivial, where the real concept gets obscured by adopting a bad naming convention. There are quite a few cases in the .NET framework, so I'll pick <b>MarshalByRefObject</b>. <br />
<br />
If you don't use .NET, here is what the documentation has to say:<br />
<i>Enables access to objects across application domain boundaries in applications that support remoting</i><br />
What?? Well, if you go down to the Remarks section, you get a better explanation:<br />
<i>Objects that do not inherit from MarshalByRefObject are implicitly marshal by value. […] The first time [...] a remote application domain accesses a MarshalByRefObject, a proxy is passed to the remote application</i><br />
Whoa, that's a little better, except that it's "are marshaled by value", not "are marshal by value" but then, again, the name should be MarshaledByRefObject, not MarshalByRefObject. Well, <i>all your base are belong to us</i> :-)<br />
<br />
Now, we could just drop the Object part, fix the grammar, and call it MarshaledByReference, which is readable enough. A reasonable alternative could be MarshaledByProxy (Vs. MarshaledByCopy, which would be the default).<br />
Still, we're talking more about implementation than about concepts. It's not that I want my object marshaled by proxy; actually, I don't care about marshaling at all. What I want is to keep a single object identity across appdomains, whereas with copy I would end up with distinct objects. So, a proper one-sentence definition could be:<br />
<br />
<i>Preserve object identity when passed between appdomains [by being proxied instead of copied]</i><br />
or<br />
<i>Guarantees that methods invoked in remote appdomains are served in the original appdomain</i><br />
<br />
Because if you pass such an instance across appdomains, any method call would be served by the original object, in the original appdomain. Hmm, guess what, we already have similar concepts. For instance, we have objects that, after being created in a thread, must have their methods executed only inside that thread. A process can be set to run only on one CPU/core. We can configure a load balancer so that if you land on a specific server first, you'll stay on that server for the rest of your session. We call that concept <b>affinity</b>.<br />
<br />
So, a MarshalByRefObject is something with an <b>appdomain affinity</b>. The marshaling thing is just an implementation detail that makes that happen. <b>AppDomainAffine</b>, therefore, would be a more appropriate name. Unusual perhaps, but that's because of the common drift toward the <i>mechanics</i> of things and away from <i>concepts</i> (because the mechanics are usually much easier to get for techies). And yes, it takes more clarity of thoughts to come up with the notion of appdomain affinity than just slapping an Object at the end of an implementation detail. However, clarity of thoughts is exactly what I would expect from framework designers. While we are at it, I could also add that AppDomainAffine should be an attribute, not a concrete class without methods and fields (!) like MarshalByRefObject. Perhaps I'm asking too much from those guys.<br />
<br />
<br />
<b>ISomething</b><br />
I think this convention was somewhat concocted during the early COM days, when Hungarian was widely adopted inside Microsoft, and having ugly names was therefore the norm. Somehow, it was the only convention that survived the general clean up from COM to .NET. Pity :-).<br />
<br />
Now, the problem is not that you have to type an I in front of things. It's not even that it makes names harder to read. And yes, I'll even concede that after a while, you'll find it useful, because it's easy to spot an interface just by looking at its name (which, of course, is a relevant information when your platform doesn't allow multiple inheritance). The problem is that it's too easy to fall into the trap, and just take a concrete class name, put an I in front of it, and lo and behold!, you got an interface name. Sort of calling a concept IDollar instead of Currency.<br />
<br />
Case in point: say that you are looking for an abstraction of a container. It's not just a container, it's a special container. What makes it special is that you can access items by index (a more limited container would only allow sequential access). Well, here is the (imaginary :-) conversation between naïve OO Developer #1, who for unknown reasons has been assigned to the design of the base library for the newfangled language of the largest software vendor on the planet, and himself, because even naïve OO Developer #2 would have made things better:<br />
<br />
N1: So I have this class, it's called List... it's pretty cool, because I just made it a generic, it's List<T> now!<br />
N1: Hmm, the other container classes all derive from an interface... with wonderful names like IEnumerable (back to that in a moment). I need an interface for my list too! How do I call it?<br />
N1: IListable is too long (thanks, really :-)))). What about IList? That's cool!<br />
N1: Let me add an XML comment so that we can generate the help file...<br />
<i><br />IList<T> Interface<br />Represents a collection of objects that can be individually accessed by index.</i><br />
<br />
So, say that you have another class, let's call it Array. Perhaps a SparseArray too. They both can be accessed by index. So Array IS-A IList, right? C'mon.<br />
<br />
Replay the conversation, drop in our Object Thinker:<br />
<br />
N1: So I have this class, it's called List... it's pretty cool, because I just made it a generic, it's List<T> now!<br />
N1: Hmm, the other container classes all derive from an interface... with wonderful names like IEnumerable. I need an interface for my list too! How do I call it?<br />
OT: What's special about List? What does it really add to the concept of enumeration? (I'll keep the illusion that "enumeration" is the right name so that N1's mind won't blow away)<br />
N1: Well, the fundamental idea is that you can access items by index... or ask about the index of an element... or remove the element at a given index...<br />
OT: so instead of <i>sequential access</i> you now have <i>random access</i>, as it's usually called in computer science?<br />
N1: Yes...<br />
OT: How about we call it RandomAccessContainer? It reads pretty well, like: a List IS-A RandomAccessContainer, an Array IS-A RandomAccessContainer, etc.<br />
N1: Cool... except... can we put an I in front of it?<br />
OT: Over my dead body.... hmm I mean, OK, but you know, in computer science, a List is usually thought of as a sequential access container, not a random access container. So I'll settle for the I prefix if you change List to something else.<br />
N1: yeah, it used to be called ArrayList in the non-generic version...<br />
OT: kiddo, do you think there was a reason for that?<br />
N1: Oh... (spark of light)<br />
<br />
<br />
<b>Yes, give me the worst of both worlds!</b><br />
Of course, given enough time, people will combine those two brilliant ideas, turn off their brain entirely, and create wonderful names like IEnumerable. Most .NET collections implement IEnumerable, or its generic descendant IEnumerable<T>: when a class implements IEnumerable, its instances can be used inside a foreach statement. <br />
<br />
Indeed, IEnumerable is a perfect example of how bad naming habits thwart object thinking, and more in general, abstraction. Here is what the official documentation says about IEnumerable<T>:<br />
<i>Exposes the enumerator, which supports a simple iteration over a collection of a specified type.</i><br />
So, if you subscribe to the idea that the main responsibility gives the -able part, and that the I prefix is mandatory, it's pretty obvious that IEnumerable is the name to choose. <br />
<br />
Except that's just wrong. The right abstraction, the one that is completely hidden under the IEnumerable/IEnumerator pair, is the sequential access collection, or even better, a Sequence (a Sequence is more abstract than a Collection, as a Sequence can be calculated and not stored, see yield, continuations, etc). Think about what makes more sense when you read it out loud:<br />
<br />
A List is an IEnumerable (what??)<br />
A List is a Sequence (well, all right!)<br />
<br />
Now, a Sequence (IEnumerable) in .NET is traversed ("enumerated") through an iterator-like class called an IEnumerator ("iterator" like in the iterator pattern, not like that thing they called iterator in .NET). Simple exercise: what is a better name than IEnumerator for something that can only move forward over a Sequence?. <br />
<br />
<b> Is that all you got?</b><br />
No, that's not enough! Given a little more time, someone is bound to come up with the worst of <b>all</b> worlds! What about an interface with: <br />
an I prefix<br />
an -able suffix<br />
an Object somewhere in between<br />
<br />
That's a challenging task, and strictly speaking, they failed it. They had to put -able in between, and Object at the end. But it's a pretty amazing name, fresh from the .NET Framework 4.0: IValidatableObject (I wouldn't be surprised to discover, inside their code, an IValidatableObjectManager to, you know, <i>manage</i> :-> those damn stupid validatable objects; that would really close the circle :-). <br />
<br />
We can read the documentation for some hilarious time: <br />
<i>IValidatableObject Interface - Provides a way for an object to be invalidated.</i><br />
<br />
Yes! That's what my objects really want! To be <b>invalidated</b>! C'mon :-)). I'll spare you the imaginary conversation and go straight to the point. Objects don't want to be invalidated. That's procedural thinking: "validation". Objects may be subject to <b>constraints</b>. Yahoo! <b>Constraints</b>. What about a Constraint class (to replace the Validation/Validator stuff, which is so damn <b>procedural</b>). What about a Constrained (or, if you can't help it, IConstrained) interface, to replace IValidatableObject? <br />
<br />
Microsoft (and everyone else, for that matter): what about having a few more Object Thinkers in your class library teams? Oh, while we are at it, why don't you guys consider that we may want to check constraints in any given place, not just in the front end? Why not moving all the constraint checking away from UI or Service layers and make the whole stuff available <i>everywhere</i>? It's pretty simple, trust me :-).<br />
<br />
Bonus exercise: once you have Constraint and IConstrained, you need to check all those constraints when some event happens (like you receive a message on your service layer). Come up with a better name than ConstraintChecker, that is, something that is not ending in -er. <br />
<br />
<b>And miles to go...</b><br />
There would be so much more to say about programming practices that hinder object thinking. Properties, for instance, are usually misused to expose internal state ("expressed figuratively" or not), instead of being just zero-th methods (henceforth, goals!). Maybe I'll cover that in another post.<br />
<br />
An interesting question, for which I don't really have a good answer, is: could some coding convention <b>promote</b> object thinking? Saying "don't use the -er suffix" is not quite the same as saying "do this and that". Are your conventions moving you toward object thinking?
<br />
<br />
Note: for more on the -er suffix, see: <a href="http://www.carlopescio.com/2012/03/life-without-controller-case-1.html" target="_blank">Life without a controller, case 1</a>.<br />
<span style="color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span><br />
If you read so far, you should <a href="https://twitter.com/#!/CarloPescio" target="_blank">follow me on twitter</a>.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com56tag:blogger.com,1999:blog-13967713.post-39225344677940944902011-02-27T10:53:00.003+01:002011-02-27T20:04:07.437+01:00Notes on Software Design, Chapter 14: the Enumeration LawIn my <a href="http://www.carlopescio.com/2011/01/notes-on-software-design-chapter-13-on.html">previous post</a> on this series, I used the well-known Shape problem to explain polymorphism from the entanglement perspective. We've seen that moving from a C-like approach (with a ShapeType enum and a union of structures) to an OO approach (with a Shape interface implemented by concrete shape classes) creates new centers, altering the entanglement between previous centers. <br />This time, I'll explore the landscape of entanglement a little more, using simple, well-known problems. My aim is to provide an intuitive grasp of the entanglement concept before moving to a more formal definition. In the end, I'll come up with a simple Software Law (one of many). This chapter borrows extensively from the previous one, so if you didn't read chapter 13, it's probably better to do so before reading further.<br /><br /><b>Shapes, again</b><br />Forget about the Shape interface for a while. We now have two concrete classes: Triangle and Circle. They represent geometric shapes, this time without graphical responsibilities. A triangle is defined by 3 points, a Circle by center and radius. Given a Triangle and a Circle, we want to know the area of their intersection. <br />As usual, we have two problems to solve: the actual mathematical problem and the software design problem. If you're a mathematician, you may rightly think that solving the first (the function!) is important, and the latter is sort of secondary, if not irrelevant. As a software designer, however, I'll allow myself the luxury of ignoring the gory mathematical details, and tinker with form instead.<br /><br />If you're working in a language (like C++) where functions exist [also] outside classes, a very reasonable form would be this:<br /><pre>double Intersection( const Triangle& t, const Circle& c )<br /> {<br /> // something here<br /> }<br /></pre>Note that I'm not trying to solve a more general problem. First, I said "forget the Shape interface"; second, I may not know how to solve the general problem of shapes intersection. I just want to deal with a circle and a triangle, so this signature is simple and effective.<br /><br />If you're working in a class-only language, you have several choices.<br /><br />1) Make that an instance method, possibly changing an existing class.<br /><br />2) Make that a static method, possibly changing an existing class.<br /><br />3) If your language has open classes, or extension methods, etc, you can add that method to an existing class without changing it (as an artifact).<br /><br />So, let's sort this out first. I hope you can feel the ugliness of placing that function inside Triangle or Circle, either as an instance or static method (that feeling could be made more formal by appealing to coupling, to the open/closed principle, or to entanglement itself). <br />So, let's say that we introduce a new class, and go for a static method. At that point is kinda hard to find nice names for class and method (if you're thinking about ShapeHelper or TriangleHelper, you've been living far too long in ClassNameWasteLand, sorry :-). Having asked this question a few times in real life (while teaching basic OO thinking), I'll show you a relatively decent proposal:<br /><pre>class Intersection<br /> {<br /> public static double Area( Triangle t, Circle c )<br /> {<br /> // something here<br /> }<br /> }<br /></pre>It's quite readable on the calling site:<br /><pre>double a = Intersection.Area( t, c ) ;<br /></pre>Also, the class name is ok: a proper noun, not something ending with -er / -or that would make Peter Coad (and me :-) shiver. An arguably better alternative would be to pass parameters in a constructor and have a parameterless Area() function, but let's keep it like this for now.<br /><br />Whatever you do, you end up with a method/function body that is deeply coupled with both Triangle and Circle. You need to get all their data to calculate the area of the intersection. Still, this doesn't feel wrong, does it? That's the method purpose. It's ok to manipulate center, radius, and vertexes. Weird, coupling is supposed to be bad, but it doesn't feel bad here.<br /><br /><b>Interlude</b><br />Let me change problem setting for a short while, before I come back to our beloved shapes. You're now working on yet another payroll system. You got (guess what :-) a Person class:<br /><pre>class Person<br /> {<br /> Place address;<br /> String firstName;<br /> String lastName;<br /> TelephoneNumber homePhone;<br /> // …<br /> }<br /></pre>(yeah, I know, some would have written "Address address" :-)<br /><br />You also have a Validate() member function (I'll simplify the problem to returning a bool, not an error message or set of error messages).<br /><pre>bool Validate()<br /> {<br /> return <br /> address.IsValid() && <br /> ValidateName( firstName ) && <br /> ValidateName( lastName ) && <br /> homePhone.IsValid() ;<br /> }<br /></pre>Yikes! That code sucks! It sucks because it's asymmetric (due to the two String-typed members) but also because it's fragile. If I get in there and add "Email personalEmail ;" as a field, I'll have to update Validate as well (and more member functions too, and possibly a few callers). Hmmm. That's because I'm using my data members. Well, I'm using Triangle and Circle data member as well in the function above, which is supposed to be worse; here I'm using <i>my own</i> data members. Why was that right, while Validate "feels" wrong? Don't use hand-waving explanations :-).<br /><br /><b>Back to Shapes</b><br />A very reasonable request would be to calculate the intersection of two triangles or two circles as well, so why don't we add those functions too:<br /><pre>class Intersection<br /> {<br /> public static double Area( Triangle t, Circle c )<br /> { // something here }<br /> public static double Area( Triangle t1, Triangle t2 )<br /> { // something here }<br /> public static double Area( Circle c1, Circle c2 )<br /> { // something here }<br /> }<br /></pre>We need 3 distinct algorithms anyway, so anything more abstract than that may look suspicious. It makes sense to keep the three functions in the same class: the first function is already coupled with Triangle and Circle. Adding the other two doesn't make it any worse from the coupling perspective. Yet, this is <b>wrong</b>. It's a step toward the abyss. Add a Rectangle class, and you'll have to add 4 member functions to Intersection.<br /><br />Interestingly, we came to this point without introducing a single switch/case, actually without even a single "if" in our code. Even more interestingly, our former Intersection.Area was structurally similar to Person.Validate, yet relatively good. Our latter Intersection is not structurally similar to Person, yet it's facing a similar problem of instability.<br /><br /><b>Let's stop and think :-)</b><br />A few possible reactions to the above:<br /><br /><i>- I'm too demanding. You can do that and nothing terrible will happen.</i><br />Probably true, a small scale software mess rarely kills. The same kind of problem, however, is often at the core of a large-scale mess.<br /><br /><i>- I'm throwing you a curve.</i><br />True, sort of. The intersection problem is a well-known double-dispatch issue that is not easily solved in most languages, but that's only part of the problem. <br /><br /><i>- It's just a matter of Open/Closed Principle.</i><br />Yeah, well, again, sort of. The O/C is more concerned with extension. In practice, you won't create a new Person class to add email. You'll change the existing one.<br /><br /><i>- I know a pattern / solution for that!</i><br />Me too :-). Sure, some acyclic variant of Visitor (or perhaps more sophisticated solutions) may help with shape intersection. A reflection+attribute-based approach to validation may help with Person. As usual, we should look at the moon, not at the finger. It's not the problem per se; it's not about finding an ad-hoc solution. It's more about understanding the common cause of a large set of problems.<br /><br /><i>- Ok, I want to know what is really going on here :-)</i><br />Great! Keep reading :-)<br /><br /><b>C/D-U-U Entanglement</b><br />Everything that felt wrong here, including the problem from Chapter 13, that is:<br /><br />- the ShapeType-based Draw function with a switch/case inside<br /><br />- Person.Validate<br /><br />- the latter Intersection class, dealing with more than one case<br /><br />was indeed suffering from the <b>same</b> problem, namely an entanglement chain: C/D-U-U. In plain English: Creation or Deletion of something, leading to an Update of something else, leading to an Update of something else. I think that understanding the full chain, and not simply the U-U part, makes it easier to recognize the problem. Interestingly, the former Intersection class, with just one responsibility, was immune from C/D-U-U entanglement.<br /><br />The chain is easier to follow on the Draw function in Chapter 13, so I'll start from there. ShapeType makes it easy to understand (and talk about) the problem, because it's both <a href="http://en.wikipedia.org/wiki/Nominative_type_system" target="_blank">Nominative</a> and <a href="http://en.wikipedia.org/wiki/Extensional_definition" target="_blank">Extensional</a>. Every concept is named: individual shape types and the set of those shape types (ShapeType itself). The set is also providing through its full extension, that is, by enumerating its values (that's actually why it's called an enumerated type). Also, every concrete shape is given a name (a struct where concrete shape data are stored).<br /><br />So, for the old C-like solutions, entanglement works (unsurprisingly) like this:<br /><br />- There is a C/D-C/D entanglement between members of ShapeType and a corresponding struct. Creation or deletion of a new ShapeType member requires a corresponding action on a struct.<br /><br />- There is a C/D-U entanglement between members of ShapeType and ShapeType (this is obvious: any enumeration is C/D-U entangled with its constituents).<br /><br />- There is U-U entanglement between ShapeType and Draw, or every other function that is <b>enumerating over ShapeType</b> (a switch/case being just a particular case of enumerating over names).<br /><br />Consider now the first Intersection.Area. That function is enumerating over two sets: the set of Circle data, and the set of Triangle data. There is no switch/case involved: we're just using those data, one after another. That amounts to enumeration of names. The function is C/D-U-U entangled with any circle and triangle data. We don't consider that to be a problem because we assume (quite reasonably) that those sets won't change. If I change my mind and represent a Circle using 3 points (I could) I'll have to change Area. It's just that I do not consider that change any likely.<br /><br />Consider now Person.Validate. Validate is enumerating over the set of Person data, using their names. It's not using a switch/case. It doesn't have to. Just using a name after another amounts to enumeration. Also, enumeration is not broken by a direct function call: moving portions of Validate into sub-functions wouldn't make any difference, which is why layering is ineffective here. <br />Here, Validate is U-U entangled with the (unnamed) set of Person data, or C/D-U-U entangled with any of its present or future data members. Unlike Circle or Triangle, we have all reasons to suspect the Person data to be unstable, that is, for C and/or D to occur. Therefore, Validate is unstable.<br /><br />Finally, consider the latter Intersection class. While the individual Area functions are ok, the Intersection class is not. Once we bring in more than one intersection algorithm, we entangle the class with the <i>set of shape types</i>. Now, while in the trivial C-like design we had a visible, named entity representing the set of shapes (ShapeTypes), in the intersection problem I didn't provide such an artifact (on purpose). The fact that we're not naming it, however, does not make it any less real. It's more like we're dealing with an <a href="http://en.wikipedia.org/wiki/Intensional_definition" target="_blank">Intensional</a> definition of the set itself (this would be a long story; I wrote an entire post about it, then decided to scrap it). Intersection is C/D-U-U entangled with individual shape types, and that makes it unstable.<br /><br /><b>A Software Law</b><br />One of the slides in my <a href="http://www.physicsofsoftware.com" target="_blank">Physics of Software</a> reads "software has a nature" (yeah, I need to update those slides with tons of new material). In natural sciences, we have the concept of <a href="http://en.wikipedia.org/wiki/Physical_law" target="_blank">Physical law</a>, which despite the name doesn't have to be about physics :-). In physics, we do have quite a few interesting laws, not necessarily expressed through complex formulas. The <a href="http://en.wikipedia.org/wiki/Laws_of_thermodynamics" target="_blank">Laws of thermodynamics</a> , for instance, can be stated in plain English. <br /><br />When it comes to software, you can find several "laws of software" around. However, most of them are about process (like Conway's law, Brooks' law, etc), and just a few (like Amdhal's law) are really about software. Some are just principles (like the Law of Demeter) and not true laws. Here, however, we have an opportunity to formulate a meaningful law (one of many yet to be discovered and/or documented):<br /><br /><b>- The Enumeration Law</b><br /><i>every artifact (a function, a class, whatever) that is enumerating over the members of set <b>by name</b> is U-U entangled with that set, or said otherwise, C/D-U-U entangled with any member of the set.</i><br /><br />This is not a design principle. It is not something you must aim for. It's a law. It's always true. If you don't like the consequences, you have to change the shape of your software so that the law no longer applies, or that entanglement is harmless because the set is stable.<br /><br />This law is not earth-shattering. Well, the laws of thermodynamics don't look earth-shattering either, but that doesn't make them any less important :-). Honestly, I don't know about you, but I wish someone taught me software design this way. This is basically what is keeping me going :-).<br /><br /><b>Consequences</b><br />In <a href="http://www.carlopescio.com/2010/11/notes-on-software-design-chapter-12.html">Chapter 12</a>, I warned against the natural temptation to classify entanglement in a good/bad scale, as it has been done for coupling and even for connascence. For instance, "connascence of name" is considered the best, or least dangerous, form. Yet we have just seen that entanglement with a set of names is at the root of many problems (or not, depending on the stability of that name set).<br /><br />Forces are not good or evil. They're just there. We must be able to recognize them, understand how they're influencing our material, and deal with them accordingly. For instance, there are well-known approaches to mitigate dependency on names: indirection (through function pointers, polymorphism, etc) may help in some cases; double indirection (like in double-dispatch) is required in other cases; reflection would help in others (like Validate). All those techniques share a common effect, that we'll discuss in a future post.<br /><br /><b>Conclusions</b><br />Speaking of future: recently, I've been exploring different ways to represent entanglement, as I wasn't satisfied with my forcefield diagram. I'll probably introduce an "entanglement diagram" in my next post.<br /><br />A final note: visit counters, sharing counters and general feedback tell me that this stuff is not really popular. My previous rant on design literature had way more visitors than any of my posts on the Physics of Software, and was far easier to write :-). If you're not one of the usual suspects (the handful of good guys who are already sharing this stuff around), consider spreading the word a little. You don't have to be a hero and tell a thousand people. Telling the guy in front of you is good enough :-).Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com11tag:blogger.com,1999:blog-13967713.post-85979398226144928562011-02-03T17:28:00.004+01:002011-02-26T21:35:26.981+01:00Is Software Design Literature Dead?Sometimes, my clients ask me what to read about software design. Most often than not, they don't want a list of books – many already have the classics covered. They would like to read papers, perhaps recent works, to further advance their understanding of software design, or perhaps something inspirational, to get new ideas and concepts. I have to confess I'm often at loss for suggestions.<br /><br />The sense of discomfort gets even worse when they ask me what happened to the kind of publications they used to read in the late '90s. Stuff like the C++ Report, the Journal of Object Oriented Programming, Java Report, Object Expert, etc. Most don't even know about the <a href="http://www.jot.fm/" target="_blank">Journal of Object Technology</a>, but honestly, the JOT has taken a strong academic slant lately, and I'm not sure they would find it all that interesting. I usually suggest they keep current on design patterns: for instance, the complete <a href=" http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-566/" target="_blank">EuroPLoP 2009 proceedings</a> are available online, for free. However, mentioning patterns sometimes furthers just another question: <i>what happened to the pattern movement</i>? Keep that conversation going for a while, and you get the final question: <i>so, is software design literature dead</i>?<br /><br /><b>Is it?</b><br />Note that the question is not about software design - it's about software design <b>literature</b>. Interestingly, Martin Fowler wrote about the other side of the story (<a href="http://www.martinfowler.com/articles/designDead.html" target="_blank">Is Design Dead?</a>) back in 2004. He argued that design wasn't dead, but its nature had changed (I could argue that if you change the <i>nature</i> of something, then it's perhaps inappropriate to keep using the same name :-), but ok). So perhaps software design literature isn't dead either, and has just changed nature: after all, looking for "software design" on Google yields 3,240,000 results. <br />Trying to classify what I usually find under the "software design" chapter, I came up with this list:<br /><br />- Literature on software design philosophy, hopefully with some practical applications. Early works on Information Hiding were as much about a philosophy of software design as about practical ways to structure our software. There were a lot of philosophical papers on OOP and AOP as well. Usually, you get this kind of literature when some new idea is being proposed (like my <a href="http://www.PhysicsOfSoftware.com" target="_blank">Physics of Software</a> :-). It's natural to see less and less philosophy in a maturing discipline, so perhaps a dearth of philosophical literature is not a bad sign.<br /><br />- Literature on notations, like UML or SysML. This is not really software design literature, unless the rationale for the notation is discussed.<br /><br />- Academic literature on metrics and the like. This would be interesting if those metrics addressed real design questions, but in practice, everything is always observed through the rather narrow perspective of correlation with defects or cost or something appealing for manager$. In most cases, this literature is not really about software design, and is definitely not coming from people with a strong software design background (of course, they would disagree on that :-)<br /><br />- Literature on software design principles and heuristics, or refactoring techniques. We still see some of this, but is mostly a rehashing of the same old stuff from the late '90s. The SOLID principles, the Law of Demeter, etc. In most cases, papers say nothing new, are based on toy problems, and are here just because many programmers won't read something that has been published more than a few months ago. If this is what's keeping software design literature alive, let's pull the plug.<br /><br />- Literature on methods, like Design by Contract, TDD, Domain-Driven Design, etc. Here you find the occasional must-read work (usually a book from those who actually invented the approach), but just like philosophy, you see less and less of this literature in a maturing discipline. Then you find tons of <i>advocacy</i> on methods (or against methods), which would be more interesting if they involved <b>real</b> experiments (like the ones performed by Simula labs) and not just another toy projects in the capable hands of graduate students. Besides, experiments on design methods should be evaluated [mostly] by cost of change in the next few months/years, not exclusively by design principles. Advocacy may seem to keep literature alive, but it's just noise.<br /><br />- Literature on platform-specific architectures. There is no dearth of that. From early EJB treatises to JBoss tutorials, you can find tons of papers on "enterprise architecture". Even Microsoft has some architectural literature on application blocks and stuff like that. Honestly, in most cases it looks more like Markitecture (marketing architecture as defined by Hohmann), promoting canned solutions and proposing that you adapt your problem to the architecture, which is sort of the opposite of real software design. The best works usually fall under the "design patterns" chapter (see below).<br /><br />- Literature for <a href="http://www.joelonsoftware.com/articles/fog0000000018.html" target="_blank">architecture astronauts</a>. This is a nice venue for both academics and vendors. You usually see heavily layered architectures using any possible standards and still proposing a few more, with all the right (and wrong) acronyms inside, and after a while you learn to turn quickly to the next page. It's not unusual to find papers appealing to money-saving managers who don't "get" software, proposing yet another combination of blueprint architectures and MDA tools so that you can "generate all your code" with a mouse click. Yeah, sure, whatever.<br /><br />- Literature on design patterns. Most likely, this is what's keeping software design literature alive. A well-written pattern is about a real problem, the forces shaping the context, an effective solution, its consequences, etc. This is what software design is about, <b>in practice</b>. On the other hand, a couple of conferences every year can't keep design literature in perfect health.<br /><br />- Literature on design in the context of agility (mostly about TDD). There is a lot of this on the net. Unfortunately, it's mostly about trivial problems, and even more unfortunately, there is rarely any discussion about the quality of the design itself (as if having tests was enough to declare the design "good"). The largest issue here is that it's basically impossible to talk about the design of anything non-trivial strictly from a code-based perspective. Note: I'm <b>not</b> saying that you can't design using only code as your material. I'm saying that when the problem scales slightly beyond the toy level, the amount of code that you would have to write <b>and</b> show to talk about design and <b>design alternatives</b> grows beyond the manageable. So, while code-centric practices are <b>not</b> killing design, they are nailing the coffin on design <b>literature</b>.<br /><br />Geez, I am usually an optimist :-)), but this picture is bleak. I could almost paraphrase Richard Gabriel (of "Objects have failed" fame) and say that evidently, software design has failed to speak the truth, and therefore, software design narrative (literature) is dying. But that would not be me. I'm more like the opportunity guy. Perhaps we just need a different kind of literature on software design.<br /><br /><b>Something's missing</b><br />If you walk through the aisles of a large bookstore, and go to the "design & architecture" shelves, you'll all the literary kinds above, not about software, but about real-world stuff (from chairs to buildings). except on the design of physical objects too. Interestingly, you can also find a few <b>more</b>like:<br /><br />- Anthologies (presenting the work of several designers) and monographs on the work of famous designers. We are at loss here, because software design is not immediately visible, is not interesting for the general public, is often kept as a trade secret, etc. <br />I know only one attempt to come up with something similar for software: "Beautiful Architecture" by Diomidis Spinellis. It's a nice book, but it suffers from a lack of depth. I enjoyed reading it, but didn't come back with a new perspective on something, which is what I would be looking for in this kind of literature. <br />It would be interesting, for instance, to read more about the architecture of successful open-source projects, not from a fanboy perspective but through the eyes of experienced software designers. Any takers?<br /><br />- Idea books. These may look like anthologies, but the focus is different. While an anthology is often associated with a discussion or critics of the designer's style, an idea book presents several different objects, often out of context, as a sort of creative stimulus. I don't know of anything similar for software design, though I've seen many similar book for "web design" (basically fancy web pages). In a sense, some literature on patterns comes close to being inspirational; at least, good domain-specific patterns sometimes do. But an idea book (or idea paper) would look different.<br /><br />I guess anthologies and monographs are at odd with the software culture at large, with its focus on the whizz-bang technology of the day, little interest for the past, and often bordering on religious fervor about some products. But idea books (or most likely idea papers) could work, somehow.<br /><br />Indeed, I would like to see more software design literature organized as follows:<br /><br />- A real-world, or at least realistic problem is presented.<br /><br />- Forces are discussed. Ideally, real-world forces.<br /><br />- Possibly, a subset of the whole problem is selected. Real-world problems are too big to be dissected in a paper (or two, or three). You can't discuss the detailed design of a business system in a paper (lest you appeal only to architecture astronauts). You could, however, discuss a selected, challenging portion. Ideally, the problem (or the subset) or perhaps just the approach / solution, should be of some interest even outside the specific domain. Just because your problem arose in a deeply embedded environment doesn't mean I can't learn something useful for an enterprise web application (assuming I'm open minded, of course :-). Idea books / papers should trigger some lateral thinking on the reader, therefore unusual, provocative solutions would be great (unlike literature on patterns, where you are expected to report on well-known, sort of "traditional" solutions).<br /><br />- Practical solutions are presented and scrutinized. I don't really care if you use code, UML, words, gestures, whatever. Still, I want some depth of scrutiny. I want to see more than one option discussed. I don't need working code. I'm probably working on a different language or platform, a different domain, with different constraints. I want fresh design ideas and new perspectives.<br /><br />- In practice, a good design aims at keeping the cost of change low. This is why a real-world problem is preferable. Talking about the most likely changes and how they could be addressed in different scenarios beats babbling about tests and SOLID and the like. However, "most likely changes" is meaningless if the problem is not real.<br /><br />Funny enough, there would be no natural place to publish something like that, except your own personal page. Sure, maybe JOT, maybe not. Maybe IEEE Software, maybe not. Maybe some conference, maybe not. But we don't have a software design journal with a large readers pool. This is part of the problem, of course, but also a consequence. Wrong feedback loop :-).<br /><br />Interestingly, I proposed something similar years ago, when I was part of the editorial board of a software magazine. It never made a dent. I remember that a well-known author argued against the idea, on the basis that people were not interested in all this talking about ins-and-outs, design alternatives and stuff. They would rather have a single, comprehensive design presented that they could immediately use, preferably with some source code. Well, that's entirely possible; indeed, I don't really know how many software practitioners would be interested in this kind of literature. Sure, I can bet a few of you guys would be, but overall, perhaps just a small minority is looking for inspiration, and most are just looking for canned solutions.<br /><br /><b>Or maybe something else...</b><br />On the other hand, perhaps this style is just too stuck in the '90s to be appealing in 2011. It's unidirectional (author to readers), it's not "social", it's not fun. Maybe a design challenge would be more appealing. Or perhaps the media are obsolete, and we should move away from text and graphics and toward (e.g.) videos. I actually tried to watch some code kata videos, but the guys thought they were <a href="http://www.youtube.com/watch?v=aMHTC4IqK3A" target="_blank">like this</a>, but to an experienced designer they looked more <a href="http://www.youtube.com/watch?v=Wnjb8Hx76u8" target="_blank">like this</a>. (and not even <b>that</b> funny). Maybe the next generation of software designers will come up with a better narrative style or media.<br /><br /><b>Do something!</b><br />I'm usually the "let's do something" kind of guy, and I've entertained the idea of starting a Software Design Gallery, or a Software Design Idea Book, or something, but honestly, it's a damn lot of work, and I'm a bit skeptical about the market size (even for a free book/paper/website/whatever). As usual, any feedback is welcome, and any action on your part even more :-)Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com8tag:blogger.com,1999:blog-13967713.post-46111406717501031252011-01-19T22:38:00.002+01:002011-01-19T23:09:24.897+01:00Can we really control the quality/cost trade-off?An old rule in software project management (also known as the <a href="http://en.wikipedia.org/wiki/Project_triangle" target="_blank"> Project triangle</a> is that you can have a system that:<br />- has low development cost (Cheap)<br />- has short development time (Fast)<br />- has good quality (Good)<br />except you can only choose two :-). That seem to suggest that if you want to go fast, and don't want to spend more, you can only compromise on quality. Actually, you can also compromise on feature set, but in many real-world cases, that happens only after cost, time and quality have already been compromised.<br /><br />So, say that you want to compromise on quality. Ok, I know, most engineers don't want to compromise on quality; it's almost like compromising on ethics. Still, we actually <b>do</b>, more often that we'd like to, so let's face the monster. We can:<br /><br />- Compromise on external quality, or quality as perceived by the user. <br />Ideally, this would be simply defined as MTBF (Mean Time Between Failures), although for many users, the perceived "quality" is more akin to the "user experience". In practice, it's quite hard to predict MTBF and say "ok, we won't do error checking here because that will save us X development days, and only decrease MTBF by Y". Most companies don't even measure MTBF. Those who do, use it just as a post-facto quality goal, that is, they fix bugs until they reach the target MTBF. We just don't know enough, in the software engineering community, to actually <b>plan</b> the MTBF of our product.<br /><br />- Compromise on internal quality, or quality as perceived by developers. <br />Say goodbye to clarity of names, separation of concerns, little or no duplication of logic, extendibility in the right places, low coupling, etc. In some cases, poor internal quality may resurface as poor external quality (say, there is no error handling in place, so the code just crashes on unexpected input), but in many cases, the code can be rather crappy but appear to work fine.<br /><br />- Compromise on process, or quality as perceived by the Software Engineering Institute Police :-).<br />What usually happens is that people start bypassing any activity that may seem to slow them down, like committing changes into a version control system :-) or being thorough about testing. Again, some issues may resurface as external quality problems (we don't test, so the user gets the thrill of discovering our bugs), but users won't notice if you cut corners in many process areas (well, they won't notice short-term, at least).<br /><br />Of course, a "no compromises" mindset is dangerous anyway, as it may lead to an excess of ancillary activities not directly related to providing value (to the project and to users). So, ideally, we would choose to work at the best point in the quality/time and/or quality/cost continuum (the "best point" would depend on a number of factors: more on this later). Assuming, however, that there is indeed a continuum. Experience taught me the opposite: we can only move in discrete steps, and those are few and rather far apart.<br /><br /><b>Quality and Cost</b><br />Suppose I'm buying a large quantity of forks, wholesale. Choices are almost limitless, but overall the <b>real</b> choice is between:<br /><br />- the plastic kind (throwaway)<br /><img src="http://www.eptacom.net/blog/plasticFork.jpg"><br />it works, sort of, but you can't use it for all kinds of food, you can't use it while cooking, and so on.<br /><br />- the "cheap steel" kind. <br /><img src="http://www.eptacom.net/blog/cheapFork.jpg"><br />it's surely more resistant than plastic: you can actually <b>use</b> it. Still, the tines are irregular and feel uncomfortable in your mouth. Weight isn't properly balanced. And I wouldn't bet on that steel actually being stainless.<br /><br />- the good stainless steel kind. <br /><img src="http://www.eptacom.net/blog/stainlessFork.jpg"><br />Properly shaped, properly balanced, good tasting :-), truly stainless, easy to clean up. You may or may not have decorations like that: price doesn't change so much (more on this below).<br /><br />- the gold-plated kind. <br /><img src="http://www.eptacom.net/blog/goldFork.jpg"><br />it shows that you got money, or bad taste, or both :-). Is not really better than the good stainless fork, and must be handled more carefully. It's a real example of "worse is better".<br /><br />Of course, there are other kind of forks, like those with resin or wood handles (which ain't really cheaper than the equivalent steel kind, and much worse on maintenance), or the sturdy plastic kind for babies (which leverage parental anxiety and charge a premium). So, the quality/cost landscape does not get much richer than that. Which brings us to prices. I did a little research (not really exhaustive – it would take forever :-), but overall it seems like, normalizing on the price of the plastic kind, we have this price scale:<br /><br />Plastic: 1<br />Cheap steel: 10<br />Stainless steel: 20<br />Gold-plated: 80<br /><br />The range is huge (1-80). Still, we have a big jump from throwaway to cheap (10x), a relatively small jump from cheap to good (2x), and another significant jump between good and gold-plated (4x). I think software is not really different. Except that a 2x jump is usually considered huge :-).<br /><br />Note that I'm focusing on <b>building</b> cost, which is the significant factor on market price for a commodity like forks. Once you consider operating costs, the stainless steel is the obvious winner.<br /><br /><b>The Quality/Cost landscape for Software</b><br />Again, I'll consider only building costs here, and leave maintenance and operation costs aside for a moment.<br /><br />If you build throwaway software, you can get really cheap. You don't care about error handling (which is a huge time sink). You copy&paste code all over. You proceed by trial and error, and sometimes patch code without understanding what is broken, until it seems to work. You keep no track of what you're doing. You use function names like foo and bar. Etc. After all, you just want to run that code once, and throw it away (well, that's the theory, anyway :-).<br />We may think that there is no place at all for that kind of code, but actually, end-user programming often falls into that case. I've also seen system administration scripts, lot of scientific code, and quite a bit of legacy business code that would easily fall into this category. They were written as throwaways, or by someone who only had experience writing throwaways, and somehow they didn't get thrown away.<br /><br />The next step is very far apart: if you want your code to be <b>any</b> good, you have to spend much more. It's like the plastic/"cheap steel" jump. Just put decent error handling in place, and you'll spend way more. Do a reasonable amount of systematic testing (automated or not) and you're gonna spend more. Aim for your code to be reasonably readable, and you'll have to think more, go back and refactor as you learn more, etc. At this stage, internal quality is still rather low; code is not really stainless. External quality is acceptable. It doesn't feel good, but it gets [some] work done. A lot of "industrial" code is actually at this level. Again, maintenance may not be cheap, but I'm only considering building costs here; a 5x to 10x jump compared to throwaway code is quite reasonable.<br /><br />The next step is when you invest more in -ilities, as well as in external quality through more systematic and deeper testing. You're building something that you and your users like. It's truly stainless. It feels good. However, it's hard to be "just a little stainless". It's hard to be "somehow modular" or "sort of thread-safe". Either you're on one side, or you're on the other. If you want to be on the right side, you're gonna spend more. Note that just like with fork, long-term cost is lower for good, stainless code, but as far as building cost is concerned, a 2X increase compared to "cheap steel" code is not surprising.<br /><br />Gold-plating is obvious: grandiose design that serves no purpose (and that would appear as gratuitous complexity to a good designer), coding standards that add no value (like requiring a comment on each and every parameter of every function), etc. I've seen systems spending a lot of energy trying to be as scalable as Amazon, even though they only had to serve a few hundred concurrent clients. Goldplated systems spend more on building, and may also incur higher maintenance costs afterward (be careful not to scratch the gold :-). Of course, it takes experience to recognize goldplating. To someone that has never designed a system for testability, interfaces for mocking may seem like a waste. To an experienced designer, they tell a different story.<br /><br /><b>But software is... soft!</b><br />As usual, a metaphor can be stretched only so far. Software quality can be easily altered over time. Plastic can't be transformed into gold (if you can, give me a call :-), but we can release a bug-ridden version and fix issues later. We may incur intentional technical debt and pay it back in the next release. This could be the right strategy in the right market at the right time, but that's not always the case.<br /><br />Indeed, although start-ups, customer discovery, innovative products, etc seem to get all the attention lately, if you happen to work in a mature market you may want to focus on early-on quality. Some customers are known to be forgiving about quality issues, in exchange for early access to innovative products. It's simply not the general case.<br /><br /><b>Modular Quality?</b><br />Theoretically, we could invest more in some "critical" components and less in others. After all, we don't need the same quality (external and internal) in every single portion. We can accept quirks here and there. Due to the fractal nature of software, this can be extended to the entire hierarchy of artifacts. It's surely possible. However, it takes a lot of maturity to develop on the edge of quality. Having to constantly reason about the quality/cost trade off can easily cost you more than you're saving: I'd rather spend that reasoning to create quality code. <br /><br />Once again, however, we may have a plan to start with plastic and grow into stainless steel over time. We must be aware that it is easier said than done. I've seen it many times over my long career. Improving software quality is just another case of moving our software in the decision space. We decided to make software out of plastic early on, and now we want to move it to another place, say the cheap steel spot. Some work is required. Work is proportional to the distance (which is large – see above), but also to the opposing force. Whatever form that force takes, a significant factor is always the mass to be moved, and here lies the key. It's hard to move a monolithic mass of code into another place (in the decision space). It's easier if you can move it a module at time. <br /><br />Within a small modular unit, you can easily improve local quality. Change a switch/case into polymorphism. Add an interface to decouple two concrete classes. It's easy and it's cheap, because mass is small. Say, however, that you got your modular structure wrong: an important concept has not been identified, and is now scattered all around. You have to tweak a lot of code. While doing so, you'll find that you're not just fighting the mass you have to move: you have to fight the gravitational attraction of the surrounding code. That may reveal more scattered concepts. It's a damn lot of work, possibly more than it's worth. Software is not necessarily soft.<br /><br />So, can we really move software around? Yes, in the small. Yes, if you got the modular structure right, and you're only cleaning up within that modular structure. Once again, the fractal nature of software can be our best friend or our worst enemy. Get the right partitioning between applications / services, and you only have to work inside. Get the right partitioning between components, and you only have to work inside. And so on. It's fine to have a plastic component, inasmuch as the interface is right. Get the modular structure wrong, and sooner or later you'll need to make wide-range changes, or accept a "cheap steel" quality. If we got it wrong at a small scale, it's no big deal. At a large scale, it can be a massacre.<br /><br /><b>Conclusions</b><br />Once again, modularity is the key. Not just "any" modularity, but a modularity aligned with the underlying forcefield. If you get that right, you can build plastic components and evolve them to stainless steel quality over time. Been there, done that. Get your modular structure wrong, and you'll lack locality of action. In other words: if you plan on cutting corners inside a module, you have to invest at the interface level. That's simpler if you design top-down than if you're in the "emergent design" camp, because the module will be underengineered inside, and we can't expect a clean interface to emerge from underengineered code.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com5tag:blogger.com,1999:blog-13967713.post-36491063393382912232011-01-02T21:40:00.002+01:002011-01-02T22:35:05.394+01:00Notes on Software Design, Chapter 13: On ChangeWe have a long history of manipulating physical materials. Cavemen built primitive tools out of rocks and sticks. I would speculate :-) that they didn't know much about physics, yet through observation, serendipity and good old trial and error they managed to survive pretty well, or we wouldn't be here today talking about software.<br /><br />For centuries, we only had empirical observations and haphazard theories about the real nature of the physical world. The <a href="http://en.wikipedia.org/wiki/Classical_element" target="_blank">Greek Classical Element</a> were Earth, Water, Air, Fire, and Aether, which is not exactly the way we look at materials today. Yet the Greeks managed to build large and incredibly beautiful temples.<br /><br />Of course, today we know a quite a bit more. Take any kind of physical material, like a copper wire. In the real world, you can:<br /><br />- apply thermal stress to the material, e.g. by warming it. Different materials react differently.<br /><br />- apply mechanical stress to the material, e.g. by pulling it on both ends. Different materials react differently.<br /><br />- apply electromagnetic forces, e.g. by moving the wire inside a magnetic field. Different materials react differently.<br /><br />- etc.<br /><br />To get just a little more into specifics, if we restrict ourselves to mechanical stress, and focus on the so-called <i>static loading</i>, we end up with just a few kind of stress: tension, compression, bending, torsion, and shear (see <a href="http://books.google.com/books?id=qbdnVEd5nukC&lpg=PT41&ots=l_YyA4IsJM&dq=tension%2C%20compression%2C%20bending%2C%20shear%2C%20torsion&pg=PT41" target="_blank">here</a> and also <a href="http://www.allstar.fiu.edu/aero/flight11.htm" target="_blank">here</a>).<br /><br />Although people have built large, beautiful structures without any formal knowledge of material science, it is hard to deny the huge progress made possible by a deeper understanding of what is actually going on with materials. Knowledge of forces and reactions allows us to choose the best material, the best shape, and even to create new, better materials.<br /><br /><b>The software world</b><br />Although we can build large, even beautiful systems, it is hard to deny that we are still in the dark ages. We have vague principles that usually can't be formally defined. We have different materials, like statically typed and dynamically typed languages, yet we don't have a sound theory of forces, so we end up with the usual evangelist trying to convert the world to its material of choice. It's kinda funny, because it's just as sensible as having a Brick Evangelist trying to convince you to use bricks instead of glass for your windows ("it's more robust!"), and a Glass Evangelist trying to convince you to build your entire house out of glass ("you'll get more light!"). <br /><br />Indeed, a key realization in my exploration on the nature of software was that we completely lack the notion of <i>force</i>. Sure, we use a bunch of terms like "flexibility", "heavy load", "fast", "fragile", etc., and they all seem to be somehow inspired by physics, but in the end it's just vernacular usage without any kind of theory behind.<br /><br />As I realized that, I began looking for a small, general yet useful set of "software forces". Nothing too fancy. Tension, compression, bending, torsion, and shear are not fancy, but are incredibly useful for physical materials (no, I wasn't looking for similar concepts in software, but for something just as useful). <br /><br />Initially, I focused my attention on artifacts, because design is very much about form, and form is made visible through artifacts. I got many ideas and explored several avenues, but most seemed more like ad hoc concepts or turned out to be dead ends. Eventually, I realized we already knew the fundamental forces, though we never used the term "force" to characterize them. <br /><br />Interestingly, those forces are better known in the run-time world, and are usually restricted to <i>data</i>. Upon closer scrutiny, it turned out they also apply to procedural knowledge, and to the artifact world as well. Also, it turned out that those forces were intimately connected with the concept of entanglement, and could actually shed some light on entanglement itself. More than that, they could be used to find entangled information, in practice, no more philosophy, thank you :-).<br /><br />So, let's take the easy road and see what we can do with run-time knowledge expressed through <b>data</b>, and build from there.<br /><br /><b>It's almost too simple</b><br />Think about a simple data-oriented application, like a book library. What can you do with a Book record? It's rather obvious: create, read, update, delete. That's it. CRUD. (I'm still thinking about the need for an identity-preserving Move outside the database realm, but let's focus on CRUD right now). CRUD for run-time data is trivial, and is a well-known concept. <br /><br />CRUD in the artifact world is also relatively simple and intuitive:<br /><br />Create: we create a new artifact, being it a component, a class, a function.<br /><br />Read: this got me thinking for a while. In the end, my best shot is also the simplest: we (the contingent intepreter) read the function, as an act of understanding. More on the idea of contingent/essential interpreter in a rather whimsical :-) <a href="http://www.dreamsongs.com/Files/Form&Function.pdf" target="_blank">presentation</a> by Richard Gabriel, that I already suggested a few years ago.<br /><br />Update: we change an artifact; for instance, we change the source code of some function.<br /><br />Delete: we remove an artifact; for instance, we remove a member function from a class.<br /><br />Note that given the fractal nature of software, some operations are recursively identical (by deleting a component we delete all the sub-artifacts), while others change nature as we move into the fractal hierarchy (by creating a new class inside a component we're also updating the component). This is true in the run-time world of data as well.<br /><br />Extending CRUD to run-time procedural knowledge is not necessarily trivial. Can we talk about CRUD for functions (as run-time, not compile-time, concepts)? We usually don't, but that's because the most common material and tools (programming languages) we're using are largely based on the idea that procedural knowledge is created once (through artifacts), then compiled and executed.<br /><br />However, interpreted languages always had the possibility of creating, updating and deleting functions at run-time. Some compiled languages allow dynamic function creation (like C#, through Reflection.Emit) but not update/delete. The ability to update a function through reflection would be an important step toward AOP-like possibilities, but most languages have no support for that. This is fine, though: liquids can't be compressed, but that didn't prevent us from defining the concept of compression. I'm looking for concepts that apply in both worlds (artifacts and run-time) but not necessarily to all materials (languages, etc.). So, to summarize:<br /><br />- Create, Update, Delete retain the usual meaning also for the run-time concept of functions. It means we're literally creating, updating and deleting a function at run-time. The same applies to the (run-time) concept of class (not object). We can Create, Update, Delete classes at run-time as well (given the appropriate language/material).<br /><br />- Read, in the artifact world, is the human act of reading, or better, is the contingent interpreter (us) giving meaning to (interpreting) the function. At run-time, the equivalent notion is execution, with the essential interpreter (the machine) giving meaning to the function. Therefore, reading a function in the run-time world simply means executing the function. Executing a function may also update some data, but this is fine: the function is read, data are updated; there is no contradiction.<br /><br /><b>Where do we go from here?</b><br />I'm trying to keep this post short, so I'll necessarily leave a lot of stuff for next time. However, I really want to anticipate something. In my previous post I said:<br /><br /><i>Two clusters of information are entangled when performing a change on one immediately requires a change on the other. </i><br /><br />I can now state that better as:<br /><br /><b>Two clusters of information are entangled when a C/R/U/D on one immediately requires a C/R/U/D on the other. </b><br /><br />Yes, the two definitions are not equivalent, as Read is not a Change. The second definition, however, is better, for reasons we'll explore in the next few posts. One reason is that it provides guidance on the kind of "change" you need to consider. Just restrict yourself to CRUD, and you'll be fine. As we'll see next, we can usually group CD and RU, and in fact I've defined two concepts representing those groupings.<br /><br />Entanglement through CD and/or RU is not necessarily a bad thing. It means that the entangled clusters are attracting each other, and, inasmuch as that specific force is concerned, rejecting others. In other words, they want to form a single cluster, possibly a composite cluster in the next hierarchical level (like two functions attracting themselves and creating a class). Of course, it might also be the case that the two clusters should be merged into a single cluster.<br /><br />We'll see more about different form of CRUD-induced entanglement in the next few posts. We'll also see how many concepts (from references to database normalization) are actually just ways to deal with the many forms of entanglement, or to move entanglement from the artifact space to the run-time space. Right now, I'll use my remaining time to explain polymorphism through the entanglement perspective. <br /><br /><b>Polymorphism explained</b><br />Consider the usual idea of multiple shapes (circle, triangle, rectangle, etc.) that can be drawn on a screen. In the pre-object era (say, in ANSI C) we could have defined a ShapeType enum, a Shape struct with a ShapeType field and possibly a ShapeData field, where ShapeData would have probably been a union (with all fields for all different shapes). Draw would have been implemented as a switch/case over the ShapeType field.<br /><br />We have U/U entanglement between ShapeType and ShapeData, and between ShapeType and Draw. U/U means that whenever we update ShapeType we need to update ShapeData as well. I add the Spiral type, so I need to add the relevant fields in the ShapeData union, and add logic to Draw. We also have U/U entanglement between ShapeData and Draw: if we change the way we store shape data, we have to change the Draw function as it depends on those data. We could live with that, but is not pretty.<br /><br />What if we have another function that depends on ShapeType, like Area? Well, we need another switch-case, and Area will have the same kind of U/U entanglement with ShapeType and ShapeData as Draw. It is that entanglement that is bringing all those concepts together. ShapeType, ShapeData, Draw and Area want to form a cluster.<br /><br />Note that without polymorphism (language-supported or simulated through function pointers) the code mass is organized into a potentially degenerative form: every function (Draw, Area, etc) is U/U-entangled with ShapeType, and is therefore attracting new procedural knowledge related to each and every ShapeType value. The forcefield will make those functions bigger and bigger, unless of course ShapeType is no longer extended.<br /><br />Object-oriented polymorphism "works" because it changes the gravitational centers. Each concrete class (Circle, Triangle, etc) becomes a gravitational center for procedural and structural knowledge entangled with a single (ex)ShapeType value. In fact, ShapeType no longer needs to exist at all. There is still U/U-entanglement between the (ex)ShapeData portion related to Circle and the procedural knowledge in Circle.Draw and Circle.Surface (in a Java/C#ish syntax). But this is fine, because they have now been brought together into a new cluster (the Circle class) that then rejected the other artifacts (the rest of the ShapeData fields, the other functions). As I said in my previous post, entanglement between distant element is bad.<br /><br />Note that we don't really change the mass of code. We just organize it differently, around different gravitational centers. If you draw functions as tall boxes in the switch-case form, you can see that classes are just "wide" boxes cutting through functions. The area of the boxes (the amount of code) does not change.<br /><br />Object-oriented polymorphism in a statically-typed language also requires [interface] inheritance. This brings in a new form of U/U-entanglement, this time between the interface and each and every concrete class. Whenever we change the interface, we have to change all the concrete classes.<br /><br />Interestingly, some of those functions might be entangled in the run-time world: they tend to be called together (R/R-entanglement in the run-time space). That would suggest yet another clustering (in the artifact space), like separating all the drawing operations from the geometrical operations in different interfaces.<br /><br />Note that the polymorphic solution is not necessarily better: it's just a matter of probability. If the set of functions is unstable, plain polymorphism is not really a good answer. We may want to explore a more complex solution based on the Visitor pattern, or some other clustering of artifacts. Understanding entanglement for Visitor is indeed an interesting exercise. <br /><br /><b>Coming soon (I hope :-)</b><br />CD and RU-entanglement (with their real names :-), paving our way to the concept of isolation, or shielding, or whatever I'm gonna settle for. Oh guys, by the way: happy new year!Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com0tag:blogger.com,1999:blog-13967713.post-61011761455972456222010-11-19T17:44:00.002+01:002010-11-19T17:48:05.601+01:00Notes on Software Design, Chapter 12: EntanglementMechanical devices require maintenance, some more than others. Maintenance is intended to keep them working, because mechanical parts tend to wear out. Maintenance is required so that they can still work "as new", that is, do the same thing they were built to do.<br /><br />Software programs require maintenance, some more than others. Maintenance is intended to keep them useful, because software parts don't wear out, but they get obsolete. They solve yesterday's problems, not today's problems. Or perhaps they didn't even solve yesterday's problems right (bugs). Maintenance is required so that software can so something <i>different</i> from what it was built to do.<br /><br />Looking from a slightly different perspective, software is encoded knowledge, and the value of knowledge is constantly decaying (see the concept of <a href="http://www.carlopescio.com/2010/03/where-is-your-knowledge.html">half-life of knowledge</a>). Maintenance is required to restore value. Energy (human work) must be supplied to keep encoded knowledge current, that is, valuable.<br /><br />Either way, software developers spend a large amount of their time <i>changing</i> stuff.<br /><br /><b>Change</b><br />Ideally, changes would be purely <i>additional</i>. We would write a new software entity (e.g. a class) inside a new artifact (a source file). We would transform that artifact into something executable (if needed; interpreted languages don't need this step). We would deploy just the new executable artifact. The system would automagically discover the new artifact and start using it.<br /><br />Although it's possible to create systems like that (I've done it many times through plug-in architectures), programs are not written this way from the ground up. Plug-ins, assuming they exist at all, usually appears at a relatively coarse granularity. Below that level, maintenance is rarely additional. Besides, additional maintenance is only possible when the change request is aligned with the underlying architecture.<br /><br />So, the truth is, most often we can't simply add new knowledge to the system. We also have to change and remove existing parts. A single change request then results in a <b>wave</b> of changes, propagating through the system. Depending on some factors (that I'll discuss later on) the wave could be dampened pretty soon, or even amplified. In that case, we'll have to change more and more parts as the wave propagates through the system. If we change a <a href="http://www.carlopescio.com/2010/11/design-structure-and-decisions.html">modularized decisions</a>, the change wave is dampened at module boundary.<br /><br /><b>The nature of change</b><br />Consider a system with two hardware nodes. The nodes are based on completely different hardware architectures: for instance, node 1 is a regular PC, node 2 is a DSP+ASIC embedded device. They are connected through standard transport protocols. Above transport, they don't share a single line of code, because they don't calculate the same function. <br />Yet, if you change the code deployed in node 1, you have to change some (different) code in node 2 as well, or the system won't work. Yikes! Aren't those two systems <b>loosely coupled</b> according to the traditional coupling theory? Sure. But still, changes need to happen on both sides.<br /><br />The example above may seem paradoxical, or academic, but is not. Most likely, you own one of those devices. It's a decoder, like those for DVB-T, sat TV, or inside DivX players. The decoder is not computing the same function as the encoder (in a sense, it's computing the inverse function). The decoder may not share any code at all with the encoder. Still, they must be constantly aligned, or you won't see squat.<br /><br />This is the real nature of change in software: you change something, and a number of things must be changed at the same time to preserve correctness. In a sense, change is affecting very distant, even physically disconnected components, and must be simultaneous: change X, and Y, Z, K must all change, at once. <br /><br />At this point, we may think that all the physical analogies are breaking down, because this is not how the physical world behaves. Except it does :-). You just have to look at the right scale, and move from the familiar Newtonian physics to the slightly less comfortable (for me, at least) quantum physics.<br /><br /><b>Quantum Entanglement</b><br />Note: if you really, really, really hate physics, you can skip this part. Still, if you never came across the concept of Quantum Entanglement, you may find it interesting. Well, when I first heard of it, I thought it was amazing (though I didn't see the connection with software back then).<br /><br />You probably know about the <a href="http://en.wikipedia.org/wiki/Uncertainty_principle" target="_blank">Heisenberg Uncertainty Principle</a>. Briefly, it says that when you go down to particles like photons, you can't measure (e.g.) both position and wavelength at arbitrarily high precision. It's not a problem of measurement techniques. It's the nature of things. <br /><br />Turns out, however, that we can create <b>entangled particles</b>. For instance, we can create two particles A and B that share the same exact wavelenght. Therefore, we can try to circumvent the uncertainty principle by measuring particle A wavelength, and particle B position. Now, and this is absolutely mind-blowing :-), <b>it does not work</b>. As soon as you try to measure particle B position, particle A reacts, by collapsing its wave function, immediately, even at an arbitrary distance. You cannot observe A without changing B. <br /><br />Quoting <a href="http://en.wikipedia.org/wiki/Quantum_entanglement" target="_blank">wikipedia</a>: <i>Quantum entanglement [..] is a property of certain states of a quantum system containing two or more distinct objects, in which the information describing the objects is inextricably linked such that performing a measurement on one immediately alters properties of the other, even when separated at arbitrary distances.</i><br /><br />Replace measurement with change, and that's exactly like software :-).<br /><br /><b>Software Entanglement</b><br />The parallel between entangled particles and entangled information is relatively simple. What is even more interesting, it works both in the artifact world and in the run-time world, at every level in the corresponding hierarchy (the run-time/artifact distinction is becoming more and more central, and was sorely missing in most previous works on related concepts). On the other hand, the concept of entanglement has far-reaching consequences, and is also a trampoline to new concepts. This post, therefore, is more like a broad introduction than an in-depth scrutiny.<br /><br />Borrowing from quantum physics, we can define software entanglement as follows:<br /><br /><b>Two clusters of information are entangled when performing a change on one immediately requires a change on the other.</b><br /><br />A simple example in the artifact space is renaming a function. All (by-name) callers must be changed, immediately. Callers are entangled with the callee. A simple example in the run-time space is caching. Caching requires cache coherence mechanisms, exactly because of entanglement.<br /><br />Note 1: I said "by name", because callers by reference are not affected. This is strongly related with the idea of dampening, and we'll explore it in a future post.<br /><br />Note 2: for a while, I've been leaning on using "tangling" instead of "entanglement", as it is a more familiar word. So, in some previous posts, you'll find mentions to "tangling". In the end, I decided to go with "entanglement" because "tangling" has already being used (with different meaning) in AOP literature, and also because entanglement, although perhaps less familiar, is simply more precise.<br /><br /><b>Not your granpa coupling</b><br /><a href="http://en.wikipedia.org/wiki/Coupling_(computer_science)" target="_blank">Coupling</a> is a time-honored concept, born in the 70s and survived to this day. Originally, coupling was mostly concerned with <i>data</i>. Content coupling took place when a module was tweaking another module's internal data; common coupling was about sharing a global variable; etc. Some forms of coupling were considered stronger than others.<br /><br />Most of those concepts can be applied to OO software as well, although most metrics for OO coupling takes a more simplified approach and mostly consider <i>dependencies</i> as coupling. Still, some forms of dependency are considered stronger than others. Inheritance is considered stronger than composition; dependency on a concrete class is considered stronger than dependency on an interface; etc. Therefore, the mere presence of an interface between two classes is assumed to reduce coupling. If class A doesn't talk to class B, and doesn't even know about class B existence, they are considered uncoupled.<br /><br />Of course, lack of coupling in the traditional sense does not imply lack of entanglement. This is why so many attempts to decouple systems through layers are so ineffective. As I said many times, all those layers in business systems can't usually dampen the wave of changes coming from the (seemingly) innocent need to add a field to a database table. In every layer, we usually have some information node that is tangled with that table. Data access; business logic; user interface; they all need to change, instantly, so that this new field will be put to use. That's entanglement, and it's not going away by layering. <br /><br />So, isn't entanglement just coupling? No, not really. Many systems that would be defined as "loosely coupled" are indeed "heavily entangled" (the example above with the encoder/decoder is the poster child of a loosely coupled / heavily entangled system). <br /><br />To give honor to whom honor is due, the closest thing I've encountered in my research is the concept of <i>connascence</i>, introduced by Meilir Page-Jones in a little known book from mid-90s ("What Every Programmer Should Know About Object-Oriented Design"). Curiously enough, I came to know connascence after I conceived entanglement, while looking for previous literature on the subject. Stille, there are some notable differences between connascence and entanglement, that I'll explore in the forthcoming posts (for instance, Page-Jones didn't consider the RT/artifact separation). <br /><br /><b>Implications</b><br />I'll explore the implications of entanglement in future posts, where I'll also try to delve deeper into the nature of change, as we can't fully understand entanglement until we fully understand change.<br /><br />Right now, I'd like to highlight something obvious :-), that is, having distant yet entangled information is dangerous, and having too much tangled information is either maintenance hell or performance hell (artifact vs run-time).<br /><br />Indeed, it's easy to classify entanglement as an attractive force. This is an oversimplification, however, so I'll leave the real meat for my next posts.<br /><br /><b>Beyond Good and Evil</b><br />There is a strong urge inside the human being to classify things as "good" and "bad". Within each category, we further classify things by degree of goodness or badness. Unsurprisingly, we bring this habit into computer science.<br /><br />Coupling has long been sub-classified by "strength", with content coupling being stronger than stamp coupling, in turn stronger than data coupling. Even connascence has been classified by strength, with e.g. connascence of position being stronger than connascence of name. The general consensus is that we should aim for the weakest possible form of coupling / connascence.<br /><br />I'm going to say something <b>huge</b> now, so be prepared :-). This is <b>all wrong</b>. When two information nodes are entangled, changes will propagate, period. The safest entanglement is the one with the minimum likelihood of occurrence. Of course, that must be weighted with the number of entangled nodes. It's very similar to <b>risk exposure</b> (believe or not, I couldn't find a decent link explaining the concept to the uninitiated :-).<br /><br />There is more. We can dampen the effects of some forms of entanglement. That requires human work, that is, energy. It's usually upfront work (sorry guys :-), although that's not always the case. Therefore, even if I came up with some classification of good/bad entanglement, it would be rather pointless, because dampening is totally context-dependent.<br /><br />I'll explore dampening in future posts, of course. But just to avoid excessive vagueness, think about polymorphism: it's about dampening the effect of a change (which change? Which other similar change is not dampened at all by polymorphism?). Think about the concept of reference. It's about dampening the effect of a change (in the run-time space). Etc.<br /><br />Your system is not really aligned with the underlying forcefield unless you have strategies in place to dampen the effect of entanglement with high risk exposure. Conversely, a deep understanding of entanglement may highlight different solutions, where entangled information is kept together, as to minimize the cost of change. Examples will follow.<br /><br /><b>What's next?</b><br />Before we can talk about dampening, we really need to understand the nature of change better. As we do that, we'll learn more about entanglement as well, and how many programming and design concepts have been devised to deal with entanglement, while some concepts are still missing (but theoretically possible). It's a rather long trip, but the sky is clear :-).Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com4tag:blogger.com,1999:blog-13967713.post-8848518233721767312010-11-01T15:42:00.002+01:002010-11-01T15:46:11.996+01:00Design, Structure, and DecisionsJoe is a junior programmer in the IT department of SomeLargeCompany, Inc. His company is not in the business of software; however, they have a lot of custom programs that are constantly developed, adapted, tweaked to follow the ever-changing landscape of products, sales, regulations, etc.<br /><br />Joe is largely self-taught. He doesn't know much about software engineering, but in the end, he gets things done.<br /><br />Right now, Joe is writing some code dealing with money. He's using lots of floating points variables ("double" in his language) to store money.<br />Joe does not know about the inherent rounding error of decimal-to-binary conversion (and back). He doesn't know about a decimal floating point type. He doesn't know, and didn't look for, a Money class that could also store the currency. He's just coding his way out of his problem. He didn't <b>choose</b> to use a double after pondering on the alternatives, or based on previous experiences. It wasn't a choice at all. He just didn't know any better.<br /><br />Joe is actually in the middle of a rather long function, doing some database lookup, some calculations, some reporting. He learned, the hard way, that is better to check for errors. He will handle any error in the simplest possible way: by opening an error box in the middle of the function, where the error can be diagnosed.<br />Joe doesn't know about design principles. He doesn't know that it's usually better to keep business logic decoupled from the user interface. He didn't <b>choose</b> to put some GUI code inside the business logic after thinking about alternatives. He just did it. He didn't know any better.<br /><br />That function is so long because at some point Joe realized he had to handle several different cases. He did that the usual way: with a switch/case. He never really "got" objects and couldn't even think about using polymorphism there. He uses classes, somehow, but not at that fine granularity. So, again, he didn't <b>choose</b> the switch/case over something else. He just didn't know any better.<br /><br /><b>Structure vs. Design</b><br />Joe is a good guy, really; and his code kinda works, most of the times. But this story is not about Joe. It's about code, structure, design, and decisions: because in the end, Joe will write some serious amount of code. Code has an inner <b>structure</b>: in this case, structure will reveal heavy reliance on primitive types (double), will reveal that business logic is tangled with database access and with user interface concerns, and will also reveal that no extension mechanism is in place. Code never lies. <br /><br />Now, the modern (dare I say post-agile?) way of thinking is that in the end, <i>the code is the design</i>. That is, if you want to know the <i>real</i> design, you have to look at the code. You'll find a lot of literature (not to mention blog posts), some from well-known authors, promoting or just holding on this idea. At some point, just about everybody gave in and accepted this as a truism. As you might guess, I didn't.<br /><br /><b>A process or a thing?</b><br />Software development is a young discipline, yet extremely dynamic. It's hardly surprising to find confusion and disagreement even on the fundamentals. That includes, of course, having several hundred different definitions of "software design". For some, it's a process. For some, it's the result of that process. We then have endless debates, like "is coding a form of design?", which again are often confusing "design" with "modeling" and wasting lot of ink about nothing.<br /><br />It doesn't have to be that hard. Let's start with the high-level question: design as a process vs. design as a thing. It's a rather simple choice; we may even benefit from the digital version of some old, dusty <a href="http://www.etymonline.com/index.php?term=design" target="_blank">etymology dictionary</a>, where design is commonly defined as "mark out, devise, choose, designate, appoint". Design is an act, and therefore a process. Please don't read "process" as "a series of mechanical acts"; just consider a process as something you carry out over a period of time.<br />More exactly, design is <b>decision</b> process. We choose between alternatives, balancing forces as we go. We make decisions through a <a href="http://www.eptacom.net/pubblicazioni/pub_eng/ListenToYourToolsAndMaterials.pdf" target="_blank">reflective conversation</a> with our materials, taking place all the time, from early requirements to bug fixing.<br /><br />Design, however, is <b>not</b> the code. It is <b>not</b> the diagram. It is <b>not</b> an artifact. If we really want to transform design into "a thing", then it is best defined as <i>the set of choices that brought us to the artifact</i>; assuming, of course, that we actually made any choice at all.<br /><br />Side note: not every decision is a design decision. Design is concerned with form, that is, with the shape of our material. Including or excluding a feature from a product, for instance, is not a design decision; it is best characterized as a marketing decision. In practice, a product often results from the interplay of marketing and design decisions.<br /><br /><b>Design as a decision process</b><br />Back in <a href="http://www.carlopescio.com/2006/12/infrastructure-and-superstructure.html">2006</a> I suggested that we shouldn't use terms like "accidental architecture" because architecture (which is just another form of design) is always intentional: it's a set of choices we <b>make</b>. I suggested using "structure" instead.<br /><br />Artifacts <b>always</b> have structure. Every artifact we use in software development is made of things (depending on our paradigm) that are somehow related (again, depending on our paradigm). That's the structure. Look at a diagram, and you'll see structure. Look at the code, and you'll see structure. Look at the executable, and if you can make sense of it, you'll see structure.<br /><br />There is a fundamental difference between <b>structure</b> and <b>design</b>. Structure can be the result of an intentional process (design) or the result of an accidental process (coding your way out of something). Design is always intentional; if it's not intentional, it's not design. <br /><br />So, if you want to know the real <b>structure</b> (and yes, usually you want to), you have to look at the code. You may also look at diagrams (I would). But diagrams will show an abstract, idealized, and yes, possibly obsolete high-level structure. The "real" structure is the code structure. It would be naive, however, to confuse structure with design.<br /><br />Consider Joe's code. We can see some structure in that code. That structure, however, is not design. There is no rationale behind that structure, except "I didn't know any better". Joe did not <i>design</i> his code. He just wrote it. He didn't make a single <i>decision</i>. He did the only thing he knew. That's not design. Sorry.<br /><br /><b>What is design, again?</b><br />Once we agree that design is a decision process, we can see different design approaches for what they are: different ways to come to a decision.<br /><br />The upfront school claims that some decisions can be made before writing code, perhaps by drawing models. Taken to the extreme (which is always naive), it would require that we make all design decisions before writing code. <br /><br />The emergent design school claims that system-level decisions should emerge from the self-organization of lower-level decision, mostly taken by writing code according to "good practices" like Don't Repeat Yourself. Taken to the extreme (which is always naive), it would require that we make no system-level choices at all, and wait for code to jell into a well-formed structure.<br /><br />The pattern school claims that we can recognize forces and adopt pre-packaged decisions, embodied into a pattern.<br /><br />Systematic techniques like my good old SysOOD claim that we can systematically recognize some problems and choose from a set of sound transformations.<br /><br />The MDA school claims that we can organize decisions along two axes: decisions that we store in models, and decisions that we store in transformation tools. This would be a long story in itself.<br /><br />Etc.<br /><br />If you have a favorite design approach (say, TDD or Design by Contract) you may consider spending a little time pondering on which kind of decisions are better supported by that approach. <br /><br /><b>There is no choice</b><br />If there is no choice to be made, or if you can't see that there is a choice to be made, you are not doing design. This is so <b>big</b> that I'll have to say it again. <b>If there is no choice to be made, you're not doing design</b>. You might be drawing some fancy diagram, but it's not design anyway. I can draw a detailed diagram of the I/O virtualization layer for an embedded device in 10 minutes. I've done that many times. I'm not choosing anything, just replicating something. That's not design. It's "just" modeling.<br /><br />On the other end of the spectrum, I've been writing quite a bit of code lately, based on various Facebook APIs (the graph api, the javascript sdk, fbml, whatever). Like many other APIs, things never really work as documented (or lacking documentation, as it would be reasonable to assume). Here and there, in my code, I had to do things not the way I wanted, but the only way that actually worked. In light of the above, I can honestly say that I did <b>not</b> design those portions. I often couldn't make a choice, although I wish I could (in this sense, it would be terrible to have someone look at that code and think that it represents "my design").<br /><br />Of course, that's not my first web application ever. Over time, I've written many, and I've created a lot of small reusable components. I know them well, I trust them, I have the source code, and I'm familiar with it. When I see a problem that I can easily solve by reusing one, I tend to do it on the spot. <br />Now, this is a borderline case, as I'm not really making a choice. The component is just there, crying to be reused. I'm just going with the flow. It's perhaps 5% design (recognizing the component as a good candidate and choosing to use it) and 95% habits. <br />Reusing standard library classes is probably 1% design, 99% habits. We used to write our own containers and even string classes back in the early 90s. Today, I must have a very compelling case to do so (the 1% design is recognizing I don't have a very compelling case :-).<br /><br />Trying to generalize a little, we may like to think that we're making decisions all the time, but this is far from true. In many cases, we don't, because:<br /><br />- Like Joe, we don't know any better. <br /><br />- Like above, we're working with an unstable, poorly documented, rather crappy third party library. It's basically trial and error, with a little clean-up in the end (so, let's say 2-5% design). The library is making the choices (95-98%).<br /><br />- We've done this several times in the past, and we're just repeating ourselves again. It's a habit. Perhaps we should question our habit, but we don't. Perhaps we should see that we're writing the same code over and over and <b>design</b> a reusable component, but we don't.<br /><br />- We (the company, the team, the community) have a standard way to do this.<br /><br />- The library/framework/language we're using has already made that choice.<br /><br />- We took a high-level decision before, and the rest follows naturally. <br /><br />After pondering on this for a while, I came to the conclusion that in the past I've been mixing two aspects of the decision making process. The <b>need</b> to make a decision, and the <b>freedom</b> to make a decision. You can plot this as a quadrant, if you want.<br /><br />Sometimes, there is just no need to make a choice. This can actually be a good thing. It could be a highly productive state where you just have to write down your logic. There might be short "design moments" where we make low-scale decisions (the fractal nature of software makes the decision process fractal as well), but overall we just go with the flow. As usual, the issue is <b>context</b>. Not making a decision because there is a natural solution is quite different from not making a decision because we can't even see the possibilities, or the need. <br /><br />Sometimes, we need to make a choice. At that point, we need the freedom to do so. If the code is so brittle that you have to find your way by trial and error, if company standards are overly restrictive, if our language is too limited, if the architecture is too constraining, we lack freedom. That leads to sub-optimal local and global choices. Here is the challenge of architecture: make the right decisions, so that you offer a reasonable structure for growth, yet leave enough freedom for local choices, and a feedback loop from local to global.<br /><br />Design, as a decision process, is better experienced in bursts. You make a set of choices, and then you let the natural consequences unravel themselves. You may unravel some consequences using models, if you like it. In some cases, it's very effective. Some consequences are best unraveled by writing code. Be smart, not religious.<br /><br />Be aware when you have to make too many choices, too often. It's fine in the beginning, as the decision space is huge. But as you progress, the need for new choices should naturally decrease, in frequency and in scale. Think of coding as a conversation with your artifacts. Just like any conversation, it can turn into an argument and then into a fight. Your code just does not want to be shaped that way; it was never meant to be shaped that way; it is resisting your change. You change something here, just to find out that you really need to patch that other part, and so on. You have to think, think, think. Lot of effort, little progress. On a deeper level, you're <a href="http://www.carlopescio.com/2009/04/bad-luck-or-fighting-forcefield.html">fighting the forcefield</a>. The design is not aligned with the forcefield. There is friction (in the decision space!) and friction energy is wasted energy.<br /><br />Finally, if you draw a quadrant, you'll see an interesting spot where there is no need to make choices, but there is freedom to. I like that spot: it's where I know how to do it, because I've done it before, but it's not the only way to do it, and I might try something new this time. Some people don't take the opportunity, and always take the safe, known way. Depending on the project, I don't mind taking a risk if I see the potential for a sizeable reward.<br /><br /><b>What about <i>good</i> design?</b><br />Back in 1972, David Parnas wrote one of the most influential papers ever ("On the criteria to be used in decomposing systems into modules"), where two alternative modular structures were proposed for the same system. The first used the familiar functional decomposition. The second was based on the concept of Information Hiding. Later on, that paper has been heavily quoted in the OOP literature as an inspiration for encapsulation. Indeed, Parnas suggests hiding data structure details inside modules, but that was just an example. The real message was that modules should encapsulate <b>decisions</b>: <i>every module [...] is characterized by its knowledge of a design decision which it hides from all others"</i>; again: <i>"we propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others"</i>.<br /><br />Think about it, and it makes lot of sense. Design is a decision-making process. Decisions may turn out to be wrong, or may become obsolete as business and technology change. Some decisions are also meant to be temporary from the very beginning. In all cases, we would like to modularize those decisions, so that we can change them without global impact. <br /><br />Unfortunately, this is not how most people design software. Most decisions are not modularized at all. Worse yet, most decisions are not made <b>explicit</b>, either in code, diagrams, sketchy notes, whatever. And worst of all, a lot of code is written without taking any decision at all, just like good ol' Joe does. "Seat of your pants" is not a design strategy.<br /><br />Note: given current programming technology and tools, modularizing decisions is easier said than done. For instance, I decided to use AJAX in the web application I mentioned above. I would love to modularize that decision away. But it would be damn hard. Too hard to be worth it. So I let this decision influence the overall structure. There are many cases like this. Quite often, during a joint design session, I would say something like - guys, here we're making a pervasive, non-linear choice. This is like a magic door. Once we choose, we enter into a different world.<br /><br />So, we're doing a good design job when we make modular decisions, that we can change later without global impact. I think this may shed a different light on delaying decisions (design decisions, not (e.g.) marketing decision). <br /><br />The folklore is that if you wait, you can make a more informed decision. Of course, if you wait too much, you reach a state of paralysis where you can't progress anymore, and you may also miss some opportunity along the road.<br /><br />From the decision modularization perspective, things are slightly more precise:<br /><br />- If, given our current knowledge, the decision can't be modularized, it's better to wait, because we might find a way to make it modular, or get a better picture of our problem, and hopefully make the "right" nonmodular decision. Note that when a decision is not modular, undoing/changing it will incur a substantial cost: the mass to move in the decision space will be high, and that's our measure of work.<br /><br />- If the decision can be modularized, but you think that given more knowledge you could modularize it better, it makes sense to wait. Otherwise, there is little or no value in waiting. Note that postponing a decision and postponing implementation are two different things (unless your only way to make a decision is by writing code). There is a strong connection with my concept of <a href="http://www.carlopescio.com/2010/01/delaying-decisions.html">invariant decision</a>, but I'll leave it up to you to dig around.<br /><br />As an aside, I'm under the impression that most people expect to just "learn the right answer" by waiting. And by "right" they mean that there is no need to modularize the decision anymore (because it's right :-). This is quite simplistic, as there is no guarantee whatsoever that a delayed decision will be particularly stable. <br />So, again, the value of delaying should be in learning new ways to modularize away the decision. Learning more about the risk/opportunity trade-off of <b>not</b> modularizing away the decision is second best.<br /><br /><b>Conclusions</b><br />In a sense, software development is all about decisions. Yet, the concept is completely absent from programming and modeling languages, and not particularly explicit in most design methods as well.<br />In the Physics of Software I'm trying to give Decisions a central role. Decisions are gateways to different forcefields. Conversely, we shape the forcefield by making decisions. Decisions live in the Decision Space, and software evolution is basically a process of moving our software to another position in the Decision Space. Decisions are also a key ingredient to link software design and software economics. I have many interesting ideas here, mostly based on real option theories, but that's another story.<br /><br />Next time: tangling.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com3tag:blogger.com,1999:blog-13967713.post-4471514061393207022010-10-04T15:34:00.003+02:002010-10-04T15:45:14.096+02:00Notes on Software Design, Chapter 11: Friction in the Artifacts worldWhen I first "got" the concept of run-time friction, I thought it made sense only in the run-time world. I was thinking of friction as "everything that gets in the way as we process the Function", and since there is no Function in the artifact world, there can be no friction as well. That disturbed me a little, because every other concept was present in both worlds.<br /><br />Later on, I realized I could extend that notion to the artifact side in two distinct ways. One didn't survive scrutiny. It was too informal, although somehow I'd like to bring back some of the underlying reasoning, probably as part of a different property. The other proved more solid. Since a few people asked me (in real life) how do I get these ideas, I think it might be interesting to tell the story behind the concept; after all, a blog ought to be a... log :-). The story is not really linear, but then, very few things in life are linear.<br /><br /><b>Step 1</b><br />It was an early morning back in July. I was running. At some point (within the first few kilometers) I had a flash that perhaps the notion of <a href="http://www.carlopescio.com/2008/12/notes-on-software-design-chapter-2-mass.html">mass</a> was not a primitive concept. Perhaps there was a concept of volume (and LOC would give volume, not mass) and a concept of density (after all, lines can be quite different). I spent a few minutes thinking about density (the simplest idea being that perhaps something like cyclomatic complexity could explain density), then thinking that I didn't like volume because it reminded me of Halstead's Software Science, and I didn't want my work to be so disconnected from practice. Then I started to think about a concept of surface; maybe there was an ideal volume / surface ratio too? Then the zen effect of running took over and I blissfully stopped thinking :-)). [most of those ideas were good, and at some point I'll have to reconsider a few things].<br /><br /><b>Step 2</b><br />Days later, I was thinking about giving a name to this stuff I'm writing. I came up with a few ideas, and also run a trademark / domain name search, because you never know. Looking for "Physics of Software", I found a remotely related entry in the C2 wiki: <a href="http://c2.com/cgi/wiki?PhysicalCuesInSoftwareDevelopment" target="_blank">Physical Cues In Software Development</a>. <br />Now, that stuff seems more concerned with the geometry/topology of code than with the physics of software, but while reading that page I was slightly tantalized by this sentence: "Too dense to refactor easily". Interesting. I did some literature research on density vs. refactoring, but nothing substantial came up.<br /><br /><b>Step 3</b><br />Days later again. I was writing some code (yeah, I write real world code too :-). I tend to write short methods: I practice what I preach. Still, I was in the middle of a rather long function (by my standards, that is, about 100 lines). I was looking at it, trying to understand how it got to be that big. I could see the gravitational effect of having used a massive third party component; that was consistent with my current understanding of the scale-free nature of software (more on this another time).<br />I could also see a few small, simple improvements, but in the end it was not trivial to refactor that method into a few shorter functions. Sure, it could be done, just not by selecting a few lines and choosing "extract method" from the refactoring menu. I had to create new classes, shuffle responsibilities around. Overall, it was a large effort (given the relatively small mass). I contemplated the idea of leaving that function alone :-). Too dense to refactor easily? Hmm.<br /><br /><b>Step 4</b><br />Perhaps a couple of weeks later. This thing kept bouncing in my head. I always assumed I could refactor every single method into a set of smaller methods, perhaps introducing new classes. Sure, run-time friction could grow as a result, but that's just something to be balanced. But was that actually true? Could I write a function that was basically impossible to refactor, that is, where extracting a few lines required either a huge effort or, even better, where to move N symbols outside, you basically have to add N symbols inside (to call the extracted method)? Having too much to do, I left the question unanswered.<br /><br /><b>Step 5</b><br />Just a few days later. Late evening, but unwilling to call it a day :-), I sat down trying to write a gordian function :-), a simple sequence of lines that couldn't easily be refactored. This is what I end up with:<pre><br />void f( int a, int b )<br /> {<br /> int c = a + b;<br /> int d = a + c;<br /> int e = b + c;<br /> int f = a + d;<br /> int g = b + d;<br /> int h = c + d;<br /> int i = c + e;<br /> int j = b + e;<br /> int k = a + e;<br /> // ... use f, g, h, i, j, k as above<br /> }<br /></pre><br />It may be easier to visualize the pattern through a graphical view:<br /><br /><img src="http://www.eptacom.net/blog/gordian.jpg"/><br /><br />the idea is pretty simple: at every level, I'm using nearby concepts and distant concepts; I'm also creating nodes for subsequent use in lower levels. Now, this function is trivial. Cyclomatic complexity is just 1. Yet is hard to refactor. It is hard to move things (lines, symbols, concepts) around. So I thought perhaps this was "density".<br /><br /><b>Step 6</b><br />Out of nowhere, a few days later I realize that density was not the right name. When you have troubles moving things around, we call it viscosity, not density. That triggered an internal alert: viscosity has already been used in some computing literature, and I hate to redefine existing terms, so let's check literature again.<br /><br /><b>Step 7</b><br />To my knowledge, "viscosity" has been used in:<br />- The <a href="http://www.cl.cam.ac.uk/~afb21/publications/CT2001.pdf" target="_blank">Cognitive Dimensions of Notations</a> literature, where it is defined as "the difficulty of making small changes to the information structure" or "resistance to change", both of which are rather similar to what I'm thinking. Note that although the papers above are about the notation, not the code, I discussed extending those concepts from tools to materials almost <a href="http://www.carlopescio.com/2008/02/cognitive-dimensions-of-notations.html">three years ago</a>.<br />- The well known <a href="http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf" target="_blank">Design Principles and Design Patterns</a> paper from Robert Martin, where it is defined as "When the design preserving methods are harder to employ than the hacks, then the viscosity of the design is high". This is completely unrelated, and honestly I'm not sure that "viscosity" is the right term. Indeed, Martin is using several physics-lookalike properties (immobility, fragility, rigidity), but it seems like they've been adopted on the basis of some vernacular usage of terms, not on the basis of strong correlation between the software world and the physical world (there is certainly no notion of "preserving a design vs. hacks" in viscosity as defined in physics).<br /><br />In the end, I considered viscosity as a good choice for the difficulty of moving knowledge around in the artifact world.<br /><br /><b>Step 8</b><br />Guess what. <a href="http://en.wikipedia.org/wiki/Viscosity" target="_blank">Viscosity</a> is basically <b>friction</b> (in fluids). Ok, I got it :-). <br />Just like run-time friction kicks in when you try to move run-time knowledge around, artifact-side friction kicks in when you try to move artifact-side knowledge around. Code is one way of representing artifact-side knowledge. Diagrams are another way. They both manifest some resistance to change.<br />Once you get the concept (almost) right, it's time to clean things up, come up with a more precise definition, see if it's useful and what you can learn from it.<br /><br /><b>Defining viscosity</b><br />Why is the artifact in step 5 viscous, that is, what makes it difficult to move knowledge around? The main issue is that we can't find sub-centers, because every line has local interactions (with nearby knowledge) and non-local interaction (with distant knowledge). So although every single line is using just a few symbols, you can't simply find a subset of lines that is relatively isolated from the rest. Every line is also very simple on its own, and unworthy to be moved outside alone.<br /><br />So, I could define a viscous artifact A like this:<br /><br /><i>A is viscous when given any subset B of A, the mass of knowledge inside B is not significantly higher than the amount of knowledge exchanged between B and A-B</i><br /><br />I crafted this definition rather carefully :-). In fact, the problem with the function above is not just that every line depends on nearby and distant knowledge. It's also that it's doing very little. Otherwise, we could move a significant portion of code (doing "lot" of stuff) outside the function, of course by passing parameters. But then, the mass of knowledge inside the subset B would be higher than the mass of knowledge exchanged (the parameters). That is not the case in function f.<br /><br />Viscous artifacts resist shear/tensile stress, that is, they resist extraction of knowledge. As we try to move the knowledge in B outside A, the knowledge exchanged with A-B is opposing the movement. The effort we have to spend to reorganize viscous artifacts is the equivalent of friction energy in the run-time world. Only, this time, it's human work, not CPU work.<br /><br />Note: I could come up with some kind of formula for a viscosity coefficient. I did not because I feel I'm not yet at that stage. Still, the minimum ratio between exchanged and internal knowledge (quantified over the subsets B, not over A) seems like a good candidate.<br /><br />It is important to understand that viscosity is an <i>internal</i> property of an artifact. It has nothing to do with the artifact interface. It's about the artifact internals. Of course, given the hierarchy of artifacts, an artifact may have low [internal] viscosity, yet be part of a viscous higher-level artifact. For instance, a low-viscosity function can be part of an high-viscosity class. In that case, it would be easy to move some portions of code outside the function, but not outside the class.<br /><br /><b>Consequences</b><br />Once again, we have to resist the temptation to define some property as "bad". In the physical world, viscosity is not "bad". It can be useful, or it can be a problem: it's a matter of context.<br /><br />Besides, when you look at the definition above, you may see some relationship with a vague notion of cohesion: a viscous artifact is "more cohesive". Cohesion is usually considered a good property. What's wrong here?<br /><br />My current understanding is that it's fine to be viscous when the mass is small. Actually, it's good to be viscous when the mass is small. A small viscous chunk of knowledge is a good center: it stands on its own, and can't be easily broken. However, <b>viscosity should decrease as mass increases</b>. That gives an opportunity for extraction of knowledge, thereby forming new centers. Note that by saying so, I'm implicitly accepting the existence of high-mass artifacts. Still:<br /><br />- high-mass, low-viscosity artifacts can be considered as rather innocent underengineering, a form of technical debt that can be easily repaid (see also the latest comments to <a href="http://www.carlopescio.com/2010/04/notes-on-software-design-chapter-0-what.html">Chapter 0</a>). The high gravity of the artifact will bring in more stuff: that would be the ideal time to refactor the code, which will be easy, since viscosity is low.<br /><br />- high-mass, high-viscosity artifacts are a serious design weakness that should be dealt with as soon as you notice, possibly while you're still creating the artifact. Gravity will bring in more stuff, and things can only get worse.<br /><br />Note: a trivial refactoring of a viscous artifact will significantly increase run-time friction, as we'll have to pass a lot of parameters around. In most cases, we'll have to rethink the artifact and perhaps a significant portion of the surroundings, as we may have chosen the wrong centers.<br /><br /><b>Conclusions</b><br />It's sort of revealing that I started with a notion of density but ended up with a notion of viscosity, and had to reject an existing definition of viscosity in the process. Labeling software phenomena after physical phenomena is simple. Doing so in a meaningful way is not so trivial. <br /><br />I think that keeping the duality run-time / artifact in mind is helping me a lot in this journey. Things are much easier once you can clearly see that you're dealing with two different worlds. A <a href="http://blogs.msdn.com/b/ricom/archive/2010/09/27/less-loosely-coupled-than-meets-the-eye.aspx" target="_blank">recent post</a> by Rico Mariani, for instance, raises an interesting point, which is trivially explained within my frame of reasoning, but seems unnecessarily obscure when you simply talk about "coupling" and ignore the artifact/run-time duality.<br /><br />As I progress in cleaning up some ideas on the decision space, I hope to bring in even more clarity. Which reminds me that I have something really important to say about design and decisions. It's short, and I'll let it preempt :-) the next chapter on tangling.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com3tag:blogger.com,1999:blog-13967713.post-22905051281852890582010-09-12T18:40:00.002+02:002010-09-12T20:13:22.356+02:00Notes on Software Design, Chapter 10: Run-Time FrictionSo, here is the story. I keep a lot of notes. Some are text files with relevant links and organized ideas. Most are rather embarrassing scribbles on just about any piece of paper that is lying around when I need it. From time to time, I move some notes from paper to files, discard concepts that didn't prove themselves, and rearrange paragraphs to fake some kind of logical, sequential reasoning over a process that was, in fact, rather chaotic. Not surprisingly (since software is just another way to encode knowledge), David Parnas suggested long ago that we could do the same while documenting software design (see <a href="http://web.cs.wpi.edu/~gpollice/cs3733-b05/Readings/FAKE-IT.pdf" target="_blank">"A rational design process: How and why to fake it"</a>). <br />Well, it's not always easy. Sometimes, I try to approach the storytelling from an angle, see that it doesn't work out so well, and look for another. Sometimes I succeed, sometimes I don't (although, of course, the reader is the ultimate judge). This time, I have to confess, I feel like I couldn't find the right angle, the right way to start, to unfold a concept in a way that makes it look simple and natural. So I'll trust you to be smart enough to make sense of what follows :-). It's a very long post, and you may want to digest it in more than one session.<br /><br /><b>The physical world</b><br />I guess you all had to push some furniture around at one time or another. You have probably felt a stronger resistance in the beginning, followed by a milder form of resistance as soon as you got some movement. <br />The mild resistance is due to kinetic friction, while the initial, stronger resistance is usually due to static friction, that you have to overcome before moving the object (if you're not familiar with kinetic and static friction, <a href="http://en.wikipedia.org/wiki/Friction" target="_blank">wikipedia</a>will tell you more than you want to know :-).<br /><br />As you move your stuff around, friction makes you <b>waste some energy</b>, in a way that is basically proportional to the normal force, the distance, and the coefficient of kinetic friction (see the page above for the actual equation). I'll get back to this later, but if you move a constant mass on a flat surface, the energy you waste is proportional to the mass you move, the distance you go, and the magic coefficient of friction.<br /><br />The beauty of all this is that it's simple and rather unambiguous. Friction is always present in mechanical engineering, but it's a well understood concept (as far as engineering is concerned; it's still blurry at the quantum level, at least for the uninitiated like myself), and there is usually no wishy-washy talking about friction. It's not a broad concept, that is, you won't be able to design the next-generation jet engine if all you have in your conceptual toolbox is friction, yet you won't be able to design an engine at all without an appreciation of friction.<br /><br /><b>The software world</b><br />I'm first and foremost a software design practitioner: I design software, almost every day. Sometimes by myself, most often with other people; therefore, I do a lot of "design talk". In many cases, at one point or another, someone is going to bring in "performance" or "efficiency" to support (or reject) a design decision. <br />We use those words a lot, with different meaning depending on who's saying it and why he's saying it. It seems like I'm never tired of linking wikipedia, so here is a page on <a href="http://en.wikipedia.org/wiki/Computer_performance" target="_blank">computer performance</a >. Just look at the initial list of different, context-dependent meanings. It's not surprising, then, to find out some people have very peculiar views of performance. "I use arrays because they're more efficient". Sure, except that then you do a linear search because you need multiple indexes; say "efficient" again :-)? <br /><br />One might expect Computer Science (with capital letters :-) to come to the rescue and define terms more precisely, and hopefully with some relevance for practice. However, computer science is more concerned with <a href="http://en.wikipedia.org/wiki/Algorithmic_complexity" target="_blank">computational complexity theory</a> than with the nitty-gritty details of being "fast".<br />Now, don't get me wrong. You won't get too far as a programmer (and definitely not as a software designer) if you don't get the concept of complexity classes, if you can't see that an algorithm is O( n^2 ) and another is O( n log n ), or if you don't even know what the <a href="http://en.wikipedia.org/wiki/Big_Oh_notation" target="_blank">Big Oh notation</a> is all about. You <b>have</b> to know this stuff, period. In a sense, complexity theory is part of the <i>math of software</i>, and there is little point in investigating a <a href="http://www.physicsofsoftware.com" target="_blank">physics of software</a> if you don't get the math first. But math alone won't cut it. However, once we get past the complexity class we get very little assistance from computer science (and I'm purposely ignoring the fact that just because an algorithm is in the O( n log n ) class in the average case doesn't mean I can't beat it with an O( n^2 ) algorithm in my <i>practical</i> cases). <br /><br />On the "software engineering" side, the usual advice is to build the program and then use a profiler. Yeah, well, sure, beats banging your head against the wall :-), but it's not exactly like knowing what you're doing all along. Still, we make a lot of low-level design decisions while coding, and many of them will ultimately impact "performance". Lacking the basic terminology to think (and talk) about this kind of stuff is rather depressing, so why don't we try to move just a tiny step forward?<br /><br /><b>Wasting energy in software</b><br />So, here I have this piece of software (executable knowledge). For most practical applications, what I need is to get some data (interactively, from a DB, through some kind of device, whatever), transform it in a meaningful way (which could be a complex process encoded in thousands of lines), and spit out some results (which is still data, anyway). The transformation is the <i>Function</i>.<br /><br />On the artifact side, our software may be using global variables all around, or be based on a nice polymorphic structure, yet the Function doesn't care. The structure we provide on the artifact side is the domain of Form (by now, you're probably familiar with all this stuff). <br /><br />Now, transformation is a process, and no real-world process is 100% efficient; it's always going to waste something. Perhaps we should look better at that "waste" part. Something I learnt a long time ago, while pondering on <a href="http://www.eptacom.net/pubblicazioni/pub_eng/sysdes.pdf" target="_blank">principles and patterns</a>, is that overly general concepts (like "performance") must give way to more specialized notions.<br /><br /><b>Some code, please :-)</b><br />Consider this short portion of C code. I'm using C because it's a low-level language, where the implications of any given choice are relatively easy to understand.<br /><pre>double max( double x, double y )<br /> {<br /> if( x > y )<br /> return x;<br /> else<br /> return y;<br /> }<br /><br />double max3( double x, double y, double z )<br /> {<br /> double d = max( x, y );<br /> d = max( d, z );<br /> return d;<br /> }<br /></pre>The code is pretty obvious. In a common, stack-based CPU architecture, max3 will copy x and y on the stack and call max; then it will copy d and z and call max again. The return value might be stored in a CPU register or in RAM, depending on the compiler. <br /><br />Copying those values is a waste of energy, of course. I could manually inline max inside max3 and get rid of that waste. I would sacrifice reusability and perhaps clarity for "higher performance" or "higher efficiency" or "reduced waste". Alternatively, the compiler could inline the function on my behalf (see <a href="http://www.carlopescio.com/2010/06/notes-on-software-design-chapter-6.html">Chapter 6</a> for the role of languages on balancing the two worlds).<br /><br />What if I'm working on some large data structure? The Fortran guy down the corner will suggest that by keeping your structures in the common area / global memory, you won't even have to pass parameters around: every function knows exactly where to get input and where to store output! Again, we'll sacrifice reusability, and perhaps duplicate large portions of code, for sake of efficiency. As you go through most literature on High Performance Computing (see, for instance, <a href="http://queue.acm.org/detail.cfm?id=1820518" target="_blank">The Ideal HPC Programming Language</a>, recently reprinted in Communications of ACM), you'll see that the HPC community is constantly facing the problem of wasted cycles, and is wasting a lot of LOC to prevent that.<br /><br />Moving data around is not the only way to waste energy. Consider this portion of real-world code, written by a (supposedly) performance-conscious "little meritocracy":<br /><pre>static int is_rfc2822_header(char *line)<br />{<br />int ch;<br />char *cp = line; <br />if (!memcmp(line, "From ", 5) || !memcmp(line, ">From ", 6))<br /> return 1;<br />while ((ch = *cp++)) { <br /> if (ch == ':')<br /> return cp != line;<br /> if ((33 <= ch && ch <= 57) ||<br /> (59 <= ch && ch <= 126))<br /> continue;<br /> break;<br /> }<br />return 0;<br />} <br /></pre>yeah, it's ugly as hell, but it's also wasteful (which is funny, for reasons that are too long to explain here). If you read it carefully, you'll find a way to optimize the "while" body quite a bit, and while you're at it, you can easily make it more readable. Also, the two memcmp in the beginning are wasting cycles (going through the first 5 characters twice), but just like the coefficient of friction must be measured in practice, at this level any alternative should really be measured on a real-world CPU.<br /><br />Note: as we optimize code, we have to assume that is correct. We don't change Form unless we know that the Function is right. Before posting this, I checked for any update to the codebase (I got that code a few years ago, and it never looked right to me). The code is still the same, but now there is also a comment explaining what the function is intended to do. Unfortunately, it's not what it's doing, which is even more ironic, for the same unspoken reasons above. Anyway, we could easily fix the bug <b>and</b> still optimize the code.<br /><br /><b>Run-time Friction</b><br />Just like in the physical world we move object around, in the run-time world of software we move knowledge around. More exactly, we move data around (we call it data flow) and we move the execution point around (we call it control flow). We move that stuff around to calculate some Function. In the process of calculating the Function, we usually waste some cycles. We waste cycles because we have to copy data on the stack, or from one data structure to another. We waste cycles because we do unnecessary comparison, computations, jumps. We waste cycles because we process the same data more than once. Most often, we waste those cycles because we get something in exchange in the Form (artifact) domain. Sometimes, we waste cycles just because of bad coding.<br /><br />The energy waste is not a constant: copying an integer is different from copying an array of integers (that's weight, of course). Also, if your array has been swapped out to the paging file, the copy is going to cost you more: that's the contribution of distance, and I'll get back to this later. Right now, remember that wasted energy is a consequence of friction, but <b>is not</b> friction. <br /><br /><b>Causes and types of software friction</b><br />We have already seen a few cases of software friction: when you copy data, you waste cycles. Max3 didn't strictly need to copy data: the Function didn't care about reusing max, only Form did. Before we try do define friction more precisely, it's interesting to see how deep the analogy with real-world friction really is. Indeed, we even have static software friction, and kinetic software friction!<br /><br />Consider a Java (or .NET) virtual machine. When you hit a function for the first time, the code is compiled just in time. This has nothing to do with Function. It is a byproduct of a technological choice. It will cost you some cycles: that's friction. Also, it happens only once, to "put things in motion": that's <i>static friction</i>. In general, <b>static friction will increase latency</b>, while <b>kinetic friction will reduce throughput</b>. Good: we just sorted out the two main components of "performance".<br /><br />Consider a web service. Before you can call the server, you go through a relatively lengthy process, from high-level stuff (marshaling your data) to low level stuff (establishing a network connection). This is all friction: the Function is happening on the other side, inside the service code. Here we see both static and kinetic friction at play: establishing a connection adds latency, exchanging data over the network reduces throughput. <br /><br />Consider stored procedures. The ideal stored procedure takes little data in input, does significant CRUD inside, and returns little. This way, we have minimal waste due to kinetic friction, as we exchange little data with the database. Of course, this is not the only way to minimize energy waste: another approach would be to reduce distance, by bringing the database itself in-process. Interestingly, most real-time databases use the second approach.<br /><br />So, what is causing friction in software? Friction is caused by:<br /><ul><li>A copy of data from one place to another (e.g. parameter passing, temporary variables, etc), as this adds no meaning to data, and therefore is useless as far as Function is concerned.</li><br /><li>Syntactical transformation of data (e.g. marshaling) which adds no semantics (as above: this processing is not part of the Function). This includes any form of data transformation needed to talk over a non-native protocol.</li><br /><li>Unnecessary statements (like those that could be removed in the C function above). </li><br /><li>Redundant access / processing (some will be removed by the compiler, but some won't)</li><br /><li>Bookkeeping (allocation, deallocation, reference counting, heap defragmentation, garbage collection, paging, etc). All this adds no semantics, and it's irrelevant for the Function: indeed, a well-written garbage collected program should behave properly under the so-called <a href="http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx" target="blank">null garbage collector</a>.</li><br /><li>Unnecessary indirection. This is a long story and I'll leave for another time, as I've yet to talk about indirection in the physics of software.</li><br /><li>In general, <b>everything that is not strictly necessary to calculate the Function</b>, but has been added because of Form, or because of the programmer's inability to streamline the code to the mere Function, is a source of friction and will waste run-time energy.</li></ul><br /><br /><b>Defining friction</b><br />At this stage in my understanding of the physics of software, it's still hard to come up with numbers, coefficients, sometimes even formulas. Actually, I'm usually happy when I get some concept right. Still, let's look at a simplified formula for the energy wasted through friction (in the real world):<br /><br />Normal Force * Coefficient of Friction * Distance.<br /><br />That would hold pretty well in the software world as well, both at the qualitative (easier) and probably quantitative (not there yet) level. At the qualitative level, it tells us what we can control and perhaps leverage. I'll explore this in the next paragraph. At the quantitative level, it could help to evaluate low-level choices. First, however, we have to define Normal Force, Coefficient of Friction, and Distance.<br /><br />I've defined distance in the run-time world in <a href="http://www.carlopescio.com/2010/08/notes-on-software-design-chapter-9.html">Chapter 9</a>. Unfortunately, it's an ordinal scale, so we can't do math with distance. This sort of rules out any chance to have a quantitative definition of friction, but we can also look at it from the other side: a better understanding of friction energy (like: wasted cycles) could shed light on the right measurement scale for distance!<br /><br />Assuming a flat world (I have no reason to think otherwise) the Normal Force is just weight. Weight could be easily defined as the number of bytes involved. For instance, the cost of a copy is linear with the number of bytes you copy.<br /><br />The coefficient of friction is a dimensionless parameter. Interestingly, if we decide to measure energy in cycles (which makes some sense, although we usually think of cycles as time, not energy) that would imply that unit of measurement for Distance is cycles/byte. I'll have to think more about this.<br /><br />Although the coefficient of friction, in the real world, cannot be predicted but only measured, we have some intuitive grasp of it being related to the materials. As the aforementioned wikipedia page explains, it's a relatively complex "system property", depending on many factors. The same applies in the software world. The cost to move a bunch of bytes from one position to another dependes on a bunch of factors. If we want to raise the abstraction level and think in terms of objects, and not bytes, things become more complex. The exact copy semantics (reference, shallow, deep) kicks in. That's fine: a software material with shallow copy semantics would have a different coefficient of friction than one with reference copy semantics. <br /><br />Overall, I think we have little control over the coefficient of friction (I might be wrong), so for any practical purpose, distance and weight are the most interesting parameters.<br /><br /><b>Is it useful, anyway?</b><br />A good theory, and a good concept, must have a good explanatory power, that is, we should be able to use them to explain known phenomena, explain why something works, rationalize widespread practice or beliefs, etc.<br /><br />As I've already discussed, the evolution of programming languages can be largely seen as an attempt to balance the world of artifact / form with the run-time / function world. In this sense, we can look for instance at the <a href="http://thbecker.net/articles/rvalue_references/section_07.html" target="blank">perfect forwarding</a> problem, solved by right value references in the next C++ standard, as a further attempt to remove some energy waste, by avoiding unnecessary copy of data. C++ provides many ways to control friction energy, mostly in the area of generic programming and also template metaprogramming. The <a href="http://en.wikipedia.org/wiki/Curiously_recurring_template_pattern" target="blank">Curiously Recurring Template Pattern</a>, for instance, provides a form of static polymorphism exactly to avoid some friction due to unnecessary indirection (virtual dispatch).<br /><br />More generally, the simple equation for energy waste provides a clue on what we can actually control: weight, distance, coefficient of friction. This is it. As we shape software, this is what we can actually change if we want to reduce friction energy.<br /><br />Consider <a href="http://en.wikipedia.org/wiki/HTTP_compression" target="blank">HTTP compression</a>: distance couldn't be changed, so we had to change weight.<br /><br />Also, understanding the difference between static and kinetic friction explains a lot of existing practices. Think of the <a href="http://en.wikipedia.org/wiki/Nagle's_algorith" target="blank">Nagle's algorithm</a>. It works by increasing static friction (therefore latency) in exchange for lower kinetic friction (therefore throughput). Once you get your concepts right, so many things unfold so easily :-).<br /><br />Finally, the analogy holds to the extremes: just like excessive friction in mechanical systems can lead to jam, excessive friction due to paging can jam a software system. This is commonly known as <a href="http://en.wikipedia.org/wiki/Thrashing_(computer_science)" target="blank"> Trashing</a>.<br /><br />I think a caveat is in order: friction in the physical world is not necessarily evil. Wasn't it for friction, we couldn't even walk. Mechanical devices have to deal with friction all the time, but they also <i>exploit</i> friction all the time. It's harder to exploit friction in software (although the Nagle's algorithm does). Most often, we must see friction as a trade/off with other properties, mostly in the artifact side. Still, an understanding of the different types of friction, and of the constituents of friction energy, can help evaluate alternatives and even generate new, better ideas in a more systematic and (dare I say it :-) scientific way.<br /><br /><b>A different angle</b><br />I choose friction as a physical analogy because it's a simple, familiar concept. Intuition and everyday experience can easily compensate any lack of engineering knowledge. Still, I've been tempted to use different analogies, like hydraulic or electrical analogies. Indeed, there are several analogies between electrical, mechanical, hydraulic and even acoustic and optical systems (see <a href="http://en.wikipedia.org/wiki/Hydraulic_analogy" target="_blank">here</a> for a start), so it's always possible to choose a different reference system.<br /><br />Anyway, my alternative would have been to model everything after resistance and current. Current would be the equivalent of throughput, or "performance", and resistance would cause thermal dissipation. In the end, I didn't go this way for a number of reasons; for instance, one-shot stuff like JIT would require something like a thermistor (think of a PTC in CRT degaussing), but I would lose a few readers that way :-). <br /><br />Still, if you followed so far, there is an interesting result I'd like to share. Consider a trivial circuit where we apply 1V to a 1 ohm resistor, resulting in 1A current. Now, I'll replace the resistor with a series of 2, with resistance (1-P) and P ohms. Nothing changes, same current. Resistors represent processes.<br /><img src="http://www.eptacom.net/blog/amdahl.png"/><br />Now say that we have this concept of parallel execution, so the process carried out by P can be parallelized. By way of the analogy, to increase throughput (current) I can simply add up to N resistors in parallel. Now the circulating current is obviously 1 / (1-P + P/N) A. Guess what, I just rediscovered <a href="http://en.wikipedia.org/wiki/Amdahl's_law" target="_blank">Amdahl's Law</a> using Ohm's Law. That's cute :-).<br /><br />Ok guys, next time I'll have a much shorter post on the artifact-side notion of friction. If we survive that, we'll be ready for tangling.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com0tag:blogger.com,1999:blog-13967713.post-13559844743318433072010-08-19T14:56:00.003+02:002010-08-19T18:06:53.621+02:00Notes on Software Design, Chapter 9. A simple property: Distance (part 2)What is Distance in the run-time world? As I began pondering on this, it turned out I could define distance in several meaningful ways. Some were redundant with other notions I have yet to present. Some were useless for a theory of software design. In the end, I picked up a very simple definition, with some interesting ramifications.<br /><br />Just like the notion of distance in the artifact world is based on the hierarchy of artifacts, the notion of distance in the run-time world is based on a hierarchy of locations. These are the locations where executable knowledge is located at any given time. So, given two pieces of executable knowledge P1 and P2, we can define an ordinal scale:<br /><br />P1 and P2 are inside the CPU registers or the CPU execution pipeline - that's minimum distance.<br />P1 and P2 are inside the same L1 cache line<br />P1 and P2 are inside the L1 cache<br />… etc<br /><br />for the full scale, see the (updated) <a href="http://www.PhysicsOfSoftware.com/Summary.pdf">Summary</a> at the <a href="http://www.PhysicsOfSoftware.com/">Physics of Software</a> website. <br /><br /><b>Dude, I don't care about cache lines!</b><br />Yeah, well, but the processor does, and I'm looking for a good model of the real-world, not for an abstract model of computing disconnected from reality (which would be more like the math of software, not the physics). <br />You may be writing your code in Java or C#, or even in C++, and be blissfully unaware of what is going on under the hood. But the caches are there, and their effect is absolutely visible. For a simple, experimental, and well-written summary, see Igor Ostrovsky's <a href="http://igoro.com/archive/gallery-of-processor-cache-effects/" target="_blank">Gallery of Processor Cache Effects</a> (the source code is in C#, but results wouldn't be different in Java or C++).<br />Interestingly enough, most algorithms and data structures are <b>not</b> optimized for modern processors with N levels of cache. Still, there is an active area of research on <a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.44.5650" target="_blank">cache-oblivious algorithms</a> which, despite the name, are supposed to perform well with any cache line size across any number of cache levels (you can find a few links to specialized algorithms <a href="http://blogs.msdn.com/b/devdev/archive/2007/06/12/cache-oblivious-data-structures.aspx" target="_blank">here</a> but you'll have to work around broken links).<br /><br />What about virtual memory? Again, we can ignore the magic most of the times, but when we're looking for high-performance solutions, we have to deal with it. Unconvinced? Take a look at <a href="http://queue.acm.org/detail.cfm?id=1814327" target="_blank">You're Doing It Wrong</a>, where Poul-Henning Kamp explains (perhaps with a bit too much "I know it all and you don't" attitude :-)) why textbooks are not really talking about real-world computers [anymore].<br /><br /><b>Consequences</b><br />What happens when run-time distance grows? We're bound to see a staircase-like behavior, as in the second picture in the gallery above, just with more risers/treads. When you move outside your process, you have context switching. When you move outside your computer, you have network latency. When you move outside your LAN, you also have name lookup and <a href="http://compnetworking.about.com/od/networkdesign/l/bldef_hop.htm" target="_blank">packet hops</a>. We'll understand all this stuff much better as we get to the concept of friction.<br /><br />There is more. When talking about artifact distance, I said that coupling (between artifacts) should decrease as distance increases. In the run-time world, coupling <b>to the underlying platform</b> should decrease as distance increases. This must be partially reflected in the artifacts themselves, but it is also a language / platform / transformation concern.<br />Knowledge at short distance can be tightly coupled to a specific hw / sw platform. For instance, all code inside one component can be tightly bound to:<br /><ul><li>An internal object model, say the C++ object model of a specific compiler version, or the C# or Java object model for a specific version of the virtual machine.</li><br /><li>A specific operating system, if not virtualized (native code).</li><br /><li>A specific hardware, if not virtualized.</li></ul><br />This is fine at some level. I can even accept the idea that all the components inside a single application have to share some underlying assumptions. Sure, it would be better if components were relatively immune from binary issues (a plague in the C++ world). But overall (depending on the size of the application) I can "control" things and make sure everything is aligned.<br />But when I'm talking to another service / application over the network, my degree of control is much smaller. If everything is platform-dependent (with a broad definition of platform, mind you: Java is a platform), we're in for major deployment / maintenance issues. Even worse, it would represent a huge platform lock-in (I can't use a different technology for a new service). Things get just worse on a global network scale. This is why XML took the world by storm, why web services have been rather successful in the real world, and so on. This is also why I like technologies / languages that take integration with other technologies / languages seriously, and not <a href="http://www.carlopescio.com/2005/07/pure-something-considered-harmful.html ">religiously</a>.<br />As usual, the statement above is bi-directional. That is, it makes very little sense to pursue strong decoupling from the underlying platforms at micro-level. Having a class talking to itself in XML is not a brilliant strategy. Again, design is about balance: in this case, balance between efficiency and convenience on one side, and flexibility and evolvability on the other. Balance is obtained when you can depend on your platform locally, and be increasingly independent as you move farther. <br /><br /><b>Run-time Distance is not a constant</b><br />Not necessarily, anyway; it depends on small-scale technical choices. In C++, for instance, once you get two objects in the same cache line, they will stay there for their entire life, because identity in C++ is address-based. In a garbage collected environment, this is not true anymore: objects can move freely during collection. <br />Moreover, once we move from the cache line to the entire cache, things come and go, they become near and distant along time. This contributes to complex performance patterns, and indeed, modern hardware makes accurate performance prediction almost impossible. I'm pretty sure there are some interesting phenomena to be studied here - a concept of oscillating distance, perhaps the equivalent of a performance <a href="http://en.wikipedia.org/wiki/Beat_(acoustics)" target="_blank">beat</a> when two concurrent threads have slightly different oscillating frequency, and so on, but I'm not currently investigating any of this - it's just too early.<br />At some point, distance becomes more and more "constant". Sure, a local service may migrate to LAN and then to WAN, but usually it does so because of human intervention (decisions!), and may require changes on the artifact side as well. Short-range distance is a fluid notion, changing as executable knowledge is, well, executed :-).<br /><br />By the way: distance in the artifact world is not a constant, either. It is constant when the code is frozen. As soon as we change it, we change some distance relationships. In other words, when the computer is processing executable knowledge, run-time distance changes. When <b>we</b> process encoded knowledge (artifacts), artifact distance changes.<br /><br /><b>Distance-preserving transformations</b><br />Knowledge encoded in artifacts can be transformed several times before becoming executable knowledge. Most of those transformations are distance-preserving, that is, they map nearby knowledge to nearby knowledge (although with some jumps here and there).<br /><br />For instance, pure sequences of statements (without choices, iterations, calls) are "naturally" converted into sequential machine-level instructions that not only will likely sit in the same cache line, but won't break the prefetch pipeline either. Therefore, code that is near in the artifact world will end up near in the run-time world as well.<br />In the C/C++ family, data that is sequentially declared in a structure (POD) is sequential in memory as well. Therefore, there is a good chance of sharing the same cache line if you keep your structures small.<br />Conversely, data and code in different components are normally mapped to different pages (in virtual memory systems). They won't share the same cache line (they may <i>compete</i> for the same cache line, but won't be present in the same line at once). So distant things will be distant.<br /><br />Even "recent" advances in processor architecture are increasing the similarity between the artifact and run-time class of distance. Consider <a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.154.7435" target="_blank">predicated execution</a>: it's commonly used to remove branches at machine-level for short sequence of statements in if/else conditionals (see Fig. 1 in the linked paper). In terms of distance, it allows nearby code in the artifact space to stay close in the run-time space, by eliminating branches and therefore maximizing proximity in the execution pipeline.<br /><br />Some transformations, however, are not distance-preserving. Inlining of code (think of C++ inline functions, for instance) will <i>shrink</i> distance while moving from the artifact world to the run-time world. <br />Aspect Oriented Programming is particularly interesting from the point of view of distance. On the artifact side, aspects allow to isolate cross-cutting concerns. Therefore, they allow to increase distance between the advice, that is factored out, and the join points. A non-distance-preserving transformation (weaving) brings the two concepts back together as we move toward execution. <br /><br />Curiously enough, some non-preserving transformations work in the opposite way: they allow things to be near in the artifact space, yet be distant in the run-time world. Consider the numerous technologies (dating back to remote procedure calls) that allow you to code as if you were invoking a local function, or a method of a local object, while in fact you are executing a remote function, or a method of a remote object (through some kind of local proxy). This is creating an illusion of short distance (in the artifact world) while in fact maintaining high distance in the run-time world. As usual, whenever we create software over a thin layer of illusion, there is a potential for problems. When you look at <a href="http://www.rgoarchitects.com/Files/fallacies.pdf" target="_blank">The 8 fallacies of distributed computing </a>, you can immediately recognize that dealing with remote objects as if they were local objects can be rather dangerous. Said otherwise, the illusion of short distance is a <a href="http://www.carlopescio.com/2007/01/leaky-abstractions-and-tacit.html ">leaky abstraction</a>. More on distributed computing when I'll get to the concept of friction.<br /><br />A closing remark: in my previous post, I said that gravity (in the artifact world) tends to increase performance (a run-time property). We can now understand that better, and say that it is largely (not entirely) because:<br />- the most common transformations are distance-preserving.<br />- performance increases as the run-time distance decreases.<br />Again, <i>friction</i> is also at play here, but I have yet to introduce the concept.<br /><br /><b>Addenda on the artifact side</b><br />Some concepts on the artifact side are meant to give the illusion of a shorter distance, while maintaining separation. Consider <a href="http://msdn.microsoft.com/en-us/library/bb308966.aspx#csharp3.0overview_topic3" target="_blank">extension methods</a> in .NET or the more powerful concept of <a href="http://en.wikibooks.org/wiki/Objective-C_Programming/in_depth#Categories" target="_blank">category</a> in objective C. They both give the illusion of being very close to a class (when you use them) while in fact they are just as distant as any other class. (By the way: I've been playing with extension methods [in C#] as a way to get something like partial specialization in C++; it kinda works, but not inside generics, which is exactly were I would need it).<br /><br /><b>Distance in the Decision Space</b><br />While thinking about distance in the artifact and in the run-time world, I realized that the <a href="http://www.carlopescio.com/2009/01/notes-on-software-design-chapter-3-mass.html ">very first</a> notion of distance I introduced was in the decision space. Still, I haven't defined that notion at the detail level (or lack thereof :-) at which I've defined distance in the artifact and run-time world. I have a few ideas, of course, the simplest definition being "the number of decision that must be undone + the number of [irreversible?] decisions that must be taken to move your artifacts". Those decisions would involve some mass of code. Moving that mass for that "space" would give a notion of necessary work. Anyway, it's probably too early to say more, as I have to understand the decision space better. <br /><br />Coming soon, I hope, the notion of friction.Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com1tag:blogger.com,1999:blog-13967713.post-90219148187081177422010-08-05T17:26:00.005+02:002010-08-05T18:00:30.327+02:00Notes on Software Design, Chapter 8. A simple property: Distance (part 1)You're writing a function in your preferred language. At some point, you look at your code and you see that a portion just does not belong there. You move it outside, creating another function. Asked why, you may answer that by doing so you made it reusable; that you want to respect the Single Responsibility Principle; that you want to increase cohesion; that code just looks "cleaner" and "easier to read" that way; and so on. I guess it sounds rather familiar. <br /><br />By moving the code outside the function, you also increased its <b>distance</b> with the code that is left <b>inside</b>. This is probably not so familiar: distance is not a textbook property of software. However, the notion of distance is extremely important. Fortunately, distance is a very simple concept. It has an immediate intuitive meaning, and although it's still rather informal, we can already use it to articulate some non-trivial reasoning.<br /><br />I'm trying to keep this post reasonably short, so I'll cover only the artifact side of the story here. I'll talk about the run-time world next time.<br /><br /><b>A Concept of Distance</b><br />Consider this simple function (it's C#, but that's obviously irrelevant)<br /><pre><br />int sum( int[] a )<br /> {<br /> int s = 0;<br /><br /> foreach( int v in a )<br /> s += v ;<br /> <br /> return s;<br /> }<br /></pre><br />I used two blank lines to split the function in three smaller portions: initialization - computation - return. I didn't use comments - the code is quite readable as it is, and you know the adage: if you see a comment, make a function. It would be unnatural (and with dubious benefits) to split that function into sub-functions. Still, I wanted to highlight the three tiny yet distinct procedural portions (centers) within that function, so I used empty lines. I guess most of you have done the same at one point or another, perhaps on a larger scale.<br /><br />Said otherwise, I wanted to show that some statements were conceptually closer than others. They don't have to be procedural statements. I have seen people "grouping" variable declarations in the same way, to show that some variables sort of "lump together" without creating a structure or a class. I did that by increasing their physical distance in the artifact space.<br /><br /><b>A Measure of Distance</b><br />Given two pieces of information P1, P2, encoded in some artifacts A1, A2, we can define their distance D( P1, P2 ) using an <a href="http://en.wikipedia.org/wiki/Level_of_measurement" target="_blank">ordinal scale</a>, that is, a totally ordered set:<br /><br />P1 and P2 appear in the same statement - that's minimum distance<br />P1 and P2 appear in the same group of consecutive statements <br />P1 and P2 appear in the same function<br />… etc<br /><br />for the full scale, including the data counterpart, see my <a href="http://www.PhysicsOfSoftware.com/Summary.pdf">Summary</a> at the <a href="http://www.PhysicsOfSoftware.com/">Physics of Software</a> website.<br /><br />Note that Distance is a <b>relative property</b>. You cannot take a particular piece of information and identify its distance (as you could do, for instance, with mass). You need two. <br />Also, the ordinal scale is rather limiting: you can do no math with it. It would be nice to turn it into a meaningful interval or ratio scale, but I'm not there yet.<br /><br /><b>Is Distance useful?</b><br />As in most theories, individual concepts may seem rather moot, but once you have enough concepts you can solve interesting problems or gain better understanding of complex phenomena. Right now, I haven't introduced the concept of tangling yet, so Distance may seem rather moot on itself. Still, we can temporarily use a vague notion of coupling to explore the value of distance. It will get better in the [near] future, trust me :-).<br /><br />Consider a few consecutive statements inside a function. It's ok if they share intimate knowledge. The three segments in <i>sum</i> are rather strongly coupled, to the point that it's ineffective to split them in subfunctions, but that doesn't bother me much. It's fine to be tightly coupled at small distance. As we'll see, it's more than fine: it's expected.<br /><br />Functions within a class are still close together, but farther apart. Again, it's ok if they share some knowledge. Ideally, that knowledge is embodied in the class invariant, but private functions are commonly tied with calling functions in a rather strong way. They often assume to be called in specific states (that could be captured in elaborated preconditions), and the caller is responsible to guarantee such preconditions. Sequence of calls are also expected to happen in specific orders, so that preconditions are met. Again, that doesn't bother me much. That's why the class exists in the first place: to provide a place where I can group together "closely related" functions and data.<br /><br />Distinct classes are even more distant. Ideally, they won't share much. In practice, classes inside the same component often end up having some acquaintance with each other. For instance, widgets inside a widget library may work well together, but may not work at all with widgets inside a different library. Still, they're distant enough to be used individually.<br /><br />We expect components / services to be lightly coupled. They can share some high-level contract, but that should be all.<br /><br />Applications shouldn't be coupled at all – any coupling should appear at a lower level (components).<br /><br />The logical consequence here is that <b>coupling must decrease as distance increases</b>. There is more to this statement than is immediately obvious. The real meaning is:<br />a) large distance requires low coupling<br />b) small distance requires high coupling<br /><br />When I explain the concept, most people immediately think of (a) and ignore (b). Yet (b) is very important, because it says: <br />1) <i>if coupling with the surroundings is not strong enough, you should move that portion elsewhere</i>. <br />2) <i>the code should go where the coupling is stronger</i> (that is, if code is attracted elsewhere, consider moving it elsewhere :-)). That's basically why <a href='http://books.google.com/books?id=1MsETFPD3I0C&pg=PA80&lpg=PA80&dq="feature+envy"' target="_blank">feature envy</a> is considered a bad smell – the code is in the wrong place.<br /><br /><b>Cohesion as an emergent property</b><br />Cohesion has always been a more elusive concept than coupling. Looking at literature, you'll find dozens of different definitions and metrics for cohesion (early works like Myers' Composite/Structured Design used to call it "strength"). I've struggled with the concept for a while, because it didn't fit too well with other parts of my theory, but then I realized that cohesion is not a property <i>per se</i>. <br /><br /><b>Cohesion is a byproduct of attraction and distance</b>: an artifact is cohesive if its constituents are at the right distance, considering the forces of attraction and rejection acting upon that artifact. If the resulting attraction is too strong or too weak, parts of that artifact want to move either down or up in the distance hierarchy, or into another site at the same level. <br /><br />Attraction is too weak: the forces keeping that code together are not strong enough to warrant the short distance at which we placed the code. For instance, a long function with well-identified segments sharing little data. We can take that sequence of statements and move it up in the hierarchy - forming a new function.<br /><br />Attraction is too strong: for instance, we put code in different classes, but those classes are intimately connected. The easier thing is to demote one class to a set of functions (down in the hierarchy) and merge those functions with the other class. But perhaps the entire shape is wrong, at odd with the forcefield. Perhaps new abstractions (centers) must be found, and functions, or even statements, moved into new places.<br /><br />This is closing the circle, so to speak. <b>Good software is in a state of equilibrium: attraction and rejection are balanced with proper distance between elements</b>.<br /><br />Note: I'm talking about attraction and rejection, but I have yet to present most attractive / repulsive forces. Still, somehow I hope most of you can grasp the concepts anyway. <br /><br /><b>An Alexandrian look on the notion of distance</b><br />I've quoted Christopher Alexander several time in an early discussion on the concept of form. Now, you may know that Alexander's most recent theory is explained in 4 tomes (which I haven't deeply read yet) collectively known as "The Nature of Order". A few people have tried to relate some of his concepts with the software world, but so far the results have been rather unimpressive (I'm probably biased in my judgment :-).<br /><br />On my side, I see a very strong connection between the concept of equilibrium as an interplay between distance and the artifact hierarchy and the Alexandrian concept of levels of scale: “A balanced range of sizes is pleasing and beautiful”. <br />Which is not to say that you should have long functions, average functions, small functions :-). I would translate that notion in the software world as: <i>a balanced use of the artifact hierarchy is pleasing and beautiful</i>. That is:<br />Don't use long function: use multiple functions in a class instead.<br />Don't use long classes: use multiple classes in a component instead.<br />Don't create huge components: use multiple components inside an [application/service/program] instead<br /><br />This is routinely ignored (which, I think, contributes to the freescale nature of most source code) but it's also the very first reason why those concepts have been introduced in the first place! Actually, we are probably still missing a few levels in the hierarchy, as required for instance to describe systems of systems. <br /><br /><b>Gravity, efficiency, and the run-time distance</b><br />Remember <a href="http://www.carlopescio.com/2009/01/notes-on-software-design-chapter-3-mass.html">gravity</a>? Gravity (in the artifact world) provides a path of least resistance for the programmer: just add stuff where there is other vaguely related related stuff. <b>Gravity works to minimize distance</b>, but in a kind of piecemeal, <a href="http://en.wikipedia.org/wiki/Maxima_and_minima" target="_blank">local minimum</a> way. It's easy to get trapped into local minimum. The minimum is local when we add code that is not tightly connected with the surroundings, so that other forces at play (not yet discussed) will reject it.<br /><br />When you point out incoherent, long functions, quite a few programmers bring in "efficiency" as an excuse (the other most common excuse being that it's easier to follow your code when you can just read it sequentially, which is another way to say "I don't understand abstraction" :-).<br />Now, efficiency is a run-time concept, and I haven't explained the corresponding concept in my theory yet. Still, using again the informal notion of efficiency we all have, we can already see that efficiency [in the run-time world] tends to decrease as distance [in the artifact world] increases. For instance, moving lines into another function requires passing parameters around. This is a first-cut, rough explanation of the well-known trade-off between run-time efficiency and artifact quality (maintainability, readability, reusability). <br /><br />Coming soon:<br />the concept of distance in the run-time world, and distance-preserving transformations<br />efficiency in the physics of software<br />tangling<br />not sure yet : ), but probably isolation and densityCarlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com5tag:blogger.com,1999:blog-13967713.post-47628318237309791372010-07-27T09:50:00.003+02:002010-07-27T14:39:17.440+02:00On Kent Beck's Responsive DesignI try to keep an eye on software design literature. I subscribe to relevant publications from IEEE and ACM; I get a copy of conference proceedings; I read blogs and, just like everybody else, I follow links and suggestions. Being aware of the potential risk of filtering out information that is not aligned with the way I think, I'm also following a few blogs that are definitely not aligned with my belief system, just to make sure I don't miss too much. I'm bound to miss something, anyway, but I can live with that :-).<br /><br />A few weeks ago I realized that I wasn't following <a href="http://www.threeriversinstitute.org/blog/" target="_blank">Kent Beck's blog</a>. I don't know why: I'm not a fan of XP, but I'm rather fond of Kent. He's an experienced designer with many good ideas. So, I took a tour of his posts and discovered that I had completely missed the concept of Responsive Design. That's weird, because I'm reading/scanning quite a few agile-oriented blogs, and I've never seen any mention of it. Oh, well; time to catch up.<br /><br />I did my homework and spent some time reading all posts in the <a href="http://www.threeriversinstitute.org/blog/?cat=6" target="_blank">Responsive Design category</a> and watching Kent's interesting <a href="http://www.infoq.com/presentations/responsive-design" target="_blank">QCon presentation</a>. Looking at blog dates, it seems like activity peaked in April 2009, but rapidly declined during 2009 to almost nothing in 2010. So, apparently, Responsive Design hasn't taken the world by storm yet. Well, design wasn't so popular in the pre-agile era, and it's not going to be popular in the post-agile era either :-).<br /><br />Anyway, the presentation inspired me with a stream of reflections that I'd like to share with you guys. I would recommend that you watch the presentation first (I know, it takes about 1 hour, but I think it's worth it). What follows is not intended to be a critic. I'm mostly trying to relate Kent's perspective on "what software design is about" with mine. It's a rather long post, so I split it in three paragraphs. The first two are about concepts in the presentation itself. The third is about an interesting difference in perspective, and how it affects our thinking.<br /><br /><b>Reflections on the introductory part</b><br />The idea of taking notes as we design, to uncover our own strategies and tactics, sounded <b>so</b> <a href="http://www.carlopescio.com/2008/11/tale-of-two-methods.html">familiar</a>. I guess at some point some of us feel an urge to understand what we're <b>really</b> doing. Although I'm looking for something different from Kent, I share some of his worries: trivial or extremely complicated insights might be just around the corner :-)<br /><br />The talk about the meaning of "responsive" around 0:13 is resonating very well with the concepts of forcefield, design as "shaping a material", and backtalk. From my perspective, Kent is mostly saying: am I going to look at the forcefield, and design accordingly, or am I going to force a preconceived set of design decisions upon the problem? (see also my <a href="http://www.carlopescio.com/2009/07/when-in-doubt-do-right.html">empty cup</a> post). <br /><br />Steady flow of features. We all love that, of course, and in a sense it's the most agile, XP-ish concept in the entire talk. The part about "the right time to design" seems rather connected with the Least Responsible Moment, so I can't really spare you a link to my own little idea of <a href="http://www.carlopescio.com/2010/01/delaying-decisions.html">Invariant Decisions</a>.<br />Again, I have a feeling that there is never enough context when talking about timing. I've certainly designed systems with a large separation (in time) between design and implementation. In many cases, it worked out quite well (of course, we always considered design as something fluid, not cast in stone). I guess it has a lot to do with how much domain knowledge you can leverage. Not every project is about doing something nobody has done before. Many are "just" about doing something in a much better way. Context, context, context...<br /><br />Starting with values: it's something I have to improve in my <a href="http://www.physicsofsoftware.com">Physics of Software</a> work. Although I've said several times that all new methods should start with a declaration of values and beliefs, so far I haven't been thorough on this side. For instance, I value honest, unbiased, free, creative communication and thinking about software design issues, where the best ideas, and not the best talkers, get to win (just because I'm a good talker doesn't mean I want to win that way :-)). That's why I'm trying to come up with a less ambiguous, wishy-washy way to think and talk about design.<br /><br /><i>"Most of the decisions I make while designing have nothing to do with the problem domain [… but ...] are shaped by the fact that I'm instructing a computer</i>. Weird. This is not my experience. I surely reuse solutions across domains, but the problem domain is heavily shaping the forcefield. When you go down to small-scale decisions, yeah, it gets more domain-independent, but high-level design is heavily problem-dependent. For instance, while you can usually (but not always) implement a chosen data structure in a rather domain-independent way, choosing the right data structure is usually a domain-dependent choice. <br />Don't get me wrong: I know I can safely ignore some portions of the problem domain when I design - I do that all the time. My view is that I can ignore the (possibly very complex) functional issues that have little impact on the forcefield, and therefore on the form. This is a complex subject that would require an in-depth study.<br /><br />Principles. I don't quite like the term "principle" because is so wide in meaning that you can use it for just about everything. So, while the "don't repeat yourself" is a commonly accepted design principle, Kent's examples of principles from the insurance world or from Dynamo looks much more like <b>pre-made [meta] decisions</b> to me. <br /><i>"You should never to lose a write"</i>. Fine. It's a decision, you can basically consider that a requirement. <br /><i>"We're there to aid human decision making"</i>. Fine. It's a meta-decision, meaning, it's not just a design decision: it will influence everything, from your value proposition down to functional requirements and so on. It's not really a design principle, not even a project-specific design principle: it's more akin to a metaphor or to a goal.<br />Aside from that distinction, I agree that sometimes constraints, whatever form they take, are actually <i>helpful</i> during design, because they prune out a lot of the decision space (again, see my post above on the empty cup). Of course, the wrong constraints can prevent you from finding the sweet spot, so timing is an issue here as well - some constraints should be considered fluid early on, because you may learn that they're not the right constraints for your project.<br /><br />The short part about patterns and principles reminded me of my early work on SysOOD, some of which got published in IEEE Software exactly as <a href="http://www.eptacom.net/pubblicazioni/pub_eng/sysdes.html">Principles Vs. Patterns</a>. So, while Kent wants to understand principles better, I've always been trying to eliminate principles altogether (universal principles, that is). Still, I'd like to see the "non-universal" or "project-specific" principles recast under some other name and further explored, because I agree that having a shared set of values / goals / constraints can help <i>tremendously</i> in any significant project. Oh, I still have the Byte smalltalk balloon issue somewhere : ).<br /><br /><b>Reflection on the 4 strategies</b><br />I see the "meat" of the talk as being about [meta] strategies to <b>move in the decision space</b>. Your software is at some point A in the multi-dimensional decision space; for instance, you decided early on that you would use filenames and not streams. Now you want to move your software to another point B in the decision space (where, for instance, streams are used instead of filenames). How do you go from A to B? <br />That's an interesting issue, where experienced designers/programmers routinely adopt different approaches compared to novices (see the empty cup post again), so it's both interesting and promising to see Kent articulate his reasoning. In a sense, it's completely orthogonal to both patterns/antipatterns (which may point you to B / away from A) and to my work (which is more concerned with understanding and articulating why B is a more desirable point than A, or to find B in the first place). <br />Actually, strictly speaking I'm not even sure this stuff is really about "design". In my view, design is more about finding B. The execution/implementation of design, however, is about moving to B. Therefore, Kent's strategies looks more like "design execution strategies" than "design strategies".<br /><br />That said, there is a subtle, yet striking difference between Kent's perspective and mine. Kent is talking about / looking to software design in a "first-person perspective", while I'm looking through the "I'm shaping a material" metaphor. He says things like: "I'm here; how do I get there?", and he's literally mimicking small steps. I'm saying things like: "my software is here; how do I move it there?"; it may seem like a small, irrelevant distinction, but I think it's shaping our thoughts rather deeply. It surely shapes the names we use: I would have never used "stepping stone", because I'm not thinking of myself going anywhere : ). More on this on the third paragraph below.<br /><br />Although this is probably the most “practical” part of the talk, concepts are a little fuzzy. Leap can be seen as a strategy, actually the simplest strategy of all: you just move there. Safe Steps is more like a meta-strategy to move inside the decision space (indeed, around 42:00 Kent talks about Safe/Small Steps as a "principle"). Once you choose Safe Steps, you have many options left, Parallel, Stepping Stones and Simplifications being 3 of them. Still, the granularity of those strategies is so coarse that it's rather surprising Kent found only 3 of them. <br /><br />For instance, I probably wouldn't have used Parallel to deal with the filenames/streams issues. I can't really say without looking at the code, but assuming I wanted to take small steps, I could have created a new class, which IS-A stream and HAS-A filename, moved the existing code to use that class (a very simple and safe change, because both the stream and the filename are there), gradually moved all the filename-dependent code inside that class, removed the filename from the interface (as it's no longer used), moved the filename-dependent code in a factory (so that I could make a choice on the stream type). At that point I could have killed the temporary class, using a plain stream again. This strategy (I could call it wrap/extract/unwrap) is quite simple and effective when dealing with legacy code, and doesn't really fit in any of the strategies Kent is proposing. <br /><br />Stepping Stone. While Leap and Parallel are more like strategies to <b>implement</b> a decision you have already taken, it seems to me that Stepping Stone is different. Although this might be my biased understanding of it, it's also the only interpretation I can give that makes Stepping Stone different from good old top-down decomposition (which Kent makes clear is different, at about 50:30).<br />I would say that Stepping Stone is about making progress where you can, and while doing so, shed some light on the surroundings as well. This is not very clear in the beginning, but as Kent moves forward, it's more and more obvious that he's thinking about novel situations, where you don't know exactly what you want to build.<br />Indeed, sometimes the forcefield is just too blurry. Still, we often can see a portion of it rather clearly, and we may have a sensible idea about the form we want to shape <b>locally</b>. Stepping Stone is a good strategy here: build (or design) what you understand, get feedback (or backtalk), get progress, etc. I usually keep exploring the dark areas while I'm building stepping stones. I'm also rather careful not to overcommit on stepping stones, as they may turn out not to fit perfectly with what's coming next. That's fine, inasmuch as we understand that design is a process, not a phase with a beginning and an end. <br />I can also build "stepping stones" when I know where I'm going to, but as I said, at that point it's hard to tell the difference between building stepping stones and executing top-down design. Oh, by the way, for the agile guys who hated me so much for saying that TDD is a limited design strategy: move around 50:20. Play. Rewind. Play again. Repeat till necessary :-). But back to Stepping Stones: I think the key phrase here is around 55:40, when Kent says that by standing on a stepping stone he can see where to go next, which I read like: a previously blurry portion of the forcefield is now visible and clear, and I can move further.<br /><br />Simplification. This is more akin to a design strategy, not to an execution strategy. I could see Simplification as a way to create small steps (perhaps even stepping stones). Indeed, the difference between Stepping Stones and Simplification is not really obvious (around 1:02:00 a participant asks about the similarity and doesn't seem like Kent can explain the difference so well either). I guess simplification is about to solve one single problem by solving a simpler version of the same problem first, whereas Stepping Stones are about solving a complex problem by solving <b>another</b> problem first (one that, however, brings us closer to solving the initial problem). In this sense, Simplification can be used where you can see where you wanna go (but it's too big / risky / ambitious) but also where you can't see exactly where you want to go, but you can clearly see a smaller version of it. Or, from my "I'm shaping a material" perspective, you can't really see the form of the real thing, but you can see the form of a smaller version of it. As I said, this stuff is interesting but rather fuzzy, and honestly, I'm not really sold on those 4 strategies.<br /><br />Overall, I'd like to see Responsive Design further developed. It would probably benefit from a better distinction between design, design execution, principles, values, goals, and so on, but it's interesting stuff in the almost flat landscape of contemporary software design literature.<br /><br /><b>Perspective Matters</b><br />As I said, Kent is looking at the decision space in first-person perspective, while I see myself moving other things around. There is an interesting implication: when you think about moving yourself, you mostly consider distance. You are who you are. You want to go elsewhere. Distance, and therefore Leaps, Small Steps, Stepping Stones, it's all that matters.<br /><br />When you think about moving something else, you have to consider two more factors. That, in my opinion, allows better reasoning.<br /><br />Sure, distance is an important factor. But, as I explained in my post about <a href="http://www.carlopescio.com/2009/01/notes-on-software-design-chapter-3-mass.html">Inertia</a>, another important factor is Mass. You cannot ignore mass, but it's unnatural to consider a varying mass from a first-person perspective.<br /><br />A simple example: I use Visual Studio (yeah yeah I know, some of you don't like it : ). Sometimes, as I'm creating a new project, I click in the wrong place (yeah yeah I'm kinda dumb :-) and I get the wrong project type. I want a C# project and I get a VB.NET project (yikes!! :-)). That's quite a distance in the decision space. Yet there is a very simple, safe step (Leap): delete the project and start from scratch. The distance is not a problem, because mass is basically null. I can use Leap not because the distance is small, but because the mass is small. I wouldn't do that with an existing, large project (I may consider going through an IL -> C# transformation though :-).<br /><br />There is more: when I discussed Inertia, I talked about software being in a state of rest or <b>motion</b> in the decision space. This is the third factor, and again, it's hard to think about it in first-person perspective. You want to go somewhere, but you're already going elsewhere. It's not truly natural. Deflecting a moving object, however, is quite ordinary (most real-life games involving a ball, for instance, involve deflecting a moving object).<br /><br />Consider a large project. Maybe a team is busy moving a subsystem S1 to another place in the decision space. Now you want to move subsystem S2 from A to B. It's not enough to consider distance and mass. You have to consider whether or not S2 is already moving, perhaps by attraction from S1. So S2 may be small (in mass) and the distance A-B may be small too, but if S2 is already moving into a different direction your job is harder, more risky, and in need of a careful execution strategy.<br /><br />A real-world example: you have a legacy C++/MFC application. Currently, the UI is talking with the application through a small, homebrew messaging component (MC), of which you are in charge. You just found a better messaging component (BMC) on the internet, say something that is more efficient, portable, and actively developed by a community. You want to move your subsystem to another place in the decision space, from MC to BMC. Part of BMC being "portable" is based on the reasonable assumption that messages are strictly C++ objects. <br /><br />Meanwhile, another team is moving the UI toward .NET (that's a big mass - big distance issue, but it's their problem, not yours :-). Now, there is an attraction between your messaging component and the UI. A .NET UI may have some issues dealing with native C++ messages. So, like it or not, there is another force at play, trying to move your component to a different place (say, NMC). Now, the mass of the messaging component might be small. The distance between MC and BMC may be small too - perhaps something you can deal with by using an adapter. But probably BMC is in a very different direction than NMC, where the UI team is pushing your component to move. So it's not just a matter of distance. It's not even enough to consider mass. You need to consider distance + full-blown Inertia. Or, to cut it short, <a href="http://en.wikipedia.org/wiki/Work_(physics)" target="_blank">work</a><br /><br />Of course, depending on your project, some factor may dominate over the others, and a simpler model might be enough, but usually, distance alone won't cut it.<br /><br /><b>There is more...</b><br />At the end of the presentation, Kent mentions power law distribution. This is something that has got me interested too in the past few months (I mentioned that in Facebook), but it's too early to say something about it, and I want to check a few ideas with the author of an interesting paper first, anyway.<br /><br />See you guys soon with a post on Distance (not in the decision space, but in the artifact space!)Carlo Pesciohttp://www.blogger.com/profile/12652284939993729858noreply@blogger.com0