I've been absent for a few months now. The thing is, I'm spending my little free time writing a non-trivial Android app. Of course, I'm also taking thousands of notes on software design, ready for a comeback :-). Anyway, staying away from my blog so long is kinda painful :-), so here I am with a short post, inspired by a very simple design choice I had to make.

When you start my little app for the first time, you get an EULA screen. If you accept the agreement, I'll initialize an internal database with some data and bring you the main screen. Ouch, did I say "if"?

Object-oriented programming was supposed to save us from switch/cases (replaced by polymorphism), and an if/else is a switch/case by another name. However, things like "if you accept the agreement" don't fit so well with polymorphism. You can make them fit, but it's not their natural shape. So we end up with just another (fat) controller.

On the other hand, perhaps I'm just saying it wrong. What if I change it into "When you accept the agreement..."? I don't know about you, but that instantly speaks "event" to me. Now, once we stop thinking about "conditions" and we think about "events", a few things happen:

- We can actually implement the concept on top of events if our language/library is event-based.

- We can fall back to the inner interface idiom, if we're in Java-land.

- We may even think aspect if we're bold enough.

Indeed, back in 2008, I wrote a post about reasoning in aspects and then implementing in objects (Can AOP inform OOP (toward SOA, too? :-) [part 1]). The idea was exactly that in some cases (cross-cutting business rules) an event-based implementation could be a poor man's alternative to an aspect-based interception (the subsequent post delved a bit more into some obliviousness issues and into the concept of cross-cutting business rules as candidate aspects).

Of course, we have to recognize the opportunity for an aspect (or event) based design. Sometimes, changing the words we use to describe [ourselves] the problem makes that easier. Next time you're facing an if, try changing it into a when. It won't always work, but it's worth trying :-).

That's it for now. Time to hit the road (snow actually :-). Places to go, clients to meet. Next time, I'm probably going to start a new series here, "life without a controller", because yeah, controllers ain't OO.

If you liked this post, you should follow me on twitter!

Labels: AOP, coding, design, oop

It had to happen. After my post on Yahtzee, my agile friend (Marco) has come back with more code katas, adding to the pile we already discussed. His latest shot was Kata Nine, a.k.a. "Back to the CheckOut".
His code was fine, really. Nothing fancy, basically a straightforward application of the Strategy pattern. You can find many similar implementations on the net (one of which prompted my tweet about ruby on training wheels, but it was more about the development style than the final result). Of course, my friend would have been disappointed if I had just said "yeah, well done!", so I had to be honest (where is the point of being friends if you can't be honest) and say "yeah, but you're solving the wrong problem".

A short conversation followed, centered on the need to question requirements (as articulated by users and customers) and come up with the real, hidden requirements, the value of domain knowledge and the skills one would need to develop to become an effective analyst or "requirement engineer" (not that I suggest you do :-), with the usual drift into agility and such.

I'll recommend that you read the problem statement for kata nine. It's not really necessary that you solve the problem, but it wouldn't hurt to think about it a little. I'm not going to present any code this time.

Kata Nine through the eye of a requirements engineer
The naive analyst would just take a requirement statement from the user/customer, clean it up a little, and pass it downstream. That's pretty much useless. The skilled analyst will form a coherent [mental] model of the world, and move toward a deeper understanding of the problem. The quintessential tool to get there is simple, and we all learn to use it around age 4: it's called questions :-). Of course, it’s all about the quality of the questions, and the quality of the mental model we're building . In the end, it has a lot to do with the way we look at the world, as explained by the concept of framing.

The problem presented in kata nine can be framed as a set of billing rules. Rules, as formulated, are based on product type (SKU), product price (per SKU) and a discount concept based on grouping a fixed quantity of items (quantity depending on SKU) and assigning a fixed price to that group (3 items 'A' for $1.30). It might be tempting for an analyst to draw a class diagram here, and for a designer to introduce an interface over the concept of rule, so that the applicability and the effects of a rule become just an implementation detail (you wish :-).

Note how quickly we're moving from the real-world issues of products, prices, discounts into the computing world, made of classes, interfaces, information hiding. However, the skilled analyst will recognize some underlying assumptions and ask for more details. Here are a few questions he may want to ask, and some of the reasoning that may follow. Some are trivial, some are not. Sometimes, two trivial questions conjure to uncover a nontrivial interplay.

- rules may intersect on a single SKU (3 items 'A' for $1.30, 6 for $2.50). Is that the case? Is there a simplified/simplifying assumption that grouping by larger quantities will always lead to a more convenient price per unit?

- without discounts, billing is a so-called online process, that is, you scan one item, and you immediately get to know the total (so far). You just add the SKU price, and never have to re-process the previous items.
If you have at most one grouping-based discount rule per SKU, you still have a linear process. You add one item, and you either form a new group, or add to the spare items. Forming a new group may require to recalculate the billing (just for the spares).
If you allow more than one rule for the same SKU, well, it depends. Under the assumption above about convenience for larger groups, you basically have a prioritized list of rules. You add one item, and you have to reprocess the entire set of items with the same SKU. You apply the largest possible quantity rule, get a spare set, apply the larger quantity rule to that spare set, and so on, until only the default "quantity 1" rule applies. This is a linear, deterministic process, repeated every time you add a new item.
In practice, however, you may have a rule for 3, 4, 5 items of the same SKU. A group of 8 items could then be partitioned in 4+4 or 5+3. Maybe 5+3 (which would be chosen by a priority list) is more convenient. Maybe not. Legislation tends to favor the buyer, so you should look for the most favorable partitioning. This is no longer a linear process, and the corresponding code will increase in complexity, testing will be more difficult, etc.
Perhaps we should exclude these cases. Perhaps we just discovered a new requirement for another part of the systems, where rules are interactively created (we don't expect to add rules by writing code, do we :-). We need to dig deeper.
Note: understanding when a requirement is moving the problem to an entirely new level of complexity is one of the most important moments in requirement analysis. As I said before, I often use the term "nonlinear decision" here, implying that there is a choice: to include that requirement or not, to renegotiate the requirement somehow, to learn more about the problem and maybe discover that it's our lucky day and that the real problem is not what we thought. This happens quite often, especially when requirements are gathered from legacy systems. I discussed this in an old (but still relevant) paper: When Past Solutions Cause Future Problems, IEEE Software, 1997.

- grouping by SKU is still a relatively simple policy. At worst, you recalculate the partitioning for all the items of the same SKU that you're scanning, and deal with a localized combinatorial problem. Is that the only kind of rule we need? What about the popular "buy 3 items, get the cheapest for $1", that is not based on SKU? What about larger categories, like "buy 3 items in the clothing department, get the cheapest for $1"? A naive analyst may ignore the issue, and a naive designer may think that dropping an interface on top of a rule will make this a simple extension. That's not the case.
Once you move beyond grouping by SKU, you have to recalculate the entire partitioning for all scanned products every time a new product is scanned. You add a product B priced as $5. Which combination of products and rules results in the most favorable price? Grouping with the other 3 items with the same SKU, or with other 2 in the clothing department? Finding the most favorable partitioning is now a rule-based combinatorial process. New rules can be introduced at any time, so it's hard to come up with an optimization scheme. Beware the large family with two full carts :-).
Your code is now two orders of magnitude harder than initially expected. Perhaps we should really renegotiate requirements, or plan a staged development within a larger timeframe than expected. What, you didn't investigate requirements, promised a working system in a week, and are now swamped? Well, you can always join the recurrent twitter riot against estimates :-).

- Oh, of course you also have fidelity cards, don't you. Some rules apply only if you have a card. In practice, people may hand their card over only after a few items have been scanned. So you need to go back anyway, but that wouldn't be a combinatorial process per se, just a need to start over.

It gets much worse...
There is an underlying, barely visible concept of time in the problem statement: "this week we have a special offer". So, rules apply only during a time interval.

Time-dependent rules are tricky. The skilled analyst knows that it's quite simple to verify if a specific instant (event) falls within an interval, and then apply the rules that happen to be effective at that instant. However, he also understands that a checkout is not an instantaneous process. It takes time, perhaps a few seconds, perhaps a few minutes. What if the two intervals overlap, that is, a rule is effective when checkout starts, but not when it ends (or vice versa)? The notion of "correctness", here, is relatively hard to define, and the store manager must be involved in this kind of decision. The interplay with the need to recalculate everything whenever a new item is scanned should be obvious: if you don't do anything about it, the implementation is going to choose for you.
Note: traditionally, many businesses have tried to sidestep this sort of problem by applying changes to business rules "after hours". Banks, for instance, still rely a lot on nightly jobs, but even more so on the concept of night. This safe harbor, however, is being constantly challenged by a 24/7 world. Your online promotion may end at midnight in your state, but that could be morning in your customer's state.

... before it gets better :-)
Of course, it's not just about finding more and more issues. The skilled analyst will quickly see that a fixed price discount policy (3 x $1.30) could easily be changed into a proportional discount ("20% off"). He may want to look into that at some point, but that's a secondary detail, because it's not going to alter the nature of the problem in any relevant way. Happy day :-).

Are we supposed to ask those questions?
Shouldn't we just start coding? It's agile 2011, man. Implement the first user story, get feedback, improve, etc. It's about individual and interactions, tacit knowledge, breaking new ground, going into the unknown, fail fast, pivot, respond to change. Well, at least, it's just like that on Hacker News, so it must be true, right?
I'm not going to argue with that, if not by pointing out an old post of mine on self-fulfilling expectations (quite a few people thought it was just about UML vs. coding; it's the old "look at the moon, not at the finger" thing). How much of the rework (change) is actually due to changing business needs, and how much is due to lack of understanding of the current business needs? We need to learn balance. Analysis paralysis means wasting time and opportunities. So does the opposite approach of rushing into coding.

We tried having an analyst, but it didn't work
My pre-canned, context-free :-) answer to this is: "no, your obsolete coder is not a skilled analyst". Here are some common traits of the unskilled analyst:
- ex coder, left himself become obsolete but knows something about the problem domain and the legacy system, so the company is trying to squeeze a little more value out of him.
- writes lengthy papers called "functional specifications" that nobody wants to read.
- got a whole :-) 3 days training on use cases, but never got the difference with a functional spec.
- kinda knows entity-relationship, and can sort-of read a class diagram. However, he would rather fill 10 pages of unfathomable prose than draw the simplest diagram.
- talks about web applications using CICS terminology.
- the only Michael Jackson he ever heard about was a singer.
- after becoming agile :-), he writes user stories like "as a customer, I want to pay my bill" (yeah, sure, whatever).
- actually believes he can use a set of examples (or test cases) as a specification (good luck with combinatorial problems).

More recurrent ineffective analysts: the marketer who couldn't market, the salesman who couldn't sale, the customer care who didn't care. Analysis is though. Just because Moron Joe couldn't do it, however, it doesn't mean it can't be done.

How to become a skilled analyst
The answer is surprisingly simple:
- you can't. It's too late. The discipline has been killed and history has been erased.
- you shouldn't. Nobody is looking for a skilled analyst anyway. Go learn a fashionable language or technology instead.

More seriously (well, slightly more seriously), the times of functional specifications are gone for good, and nobody is mourning. Use cases stifled innovation for a long while, then faded into background because writing effective use cases was hard, writing crappy use cases was simple, and people usually went for simple. Users stories are a joke, but who cares.

What if you want to learn the way of the analyst, anyway? Remember, a skilled analyst is not an expert in a specific domain (although he tends to grow into an expert in several domains). He's not the person you pay to get answers. He's the person you pay to get better questions (and so, indirectly, better answers).
The skilled analyst will ask different questions, explore different avenues, and find out different facts. So the essential skills would be observation, knowledge of large class of problems, classification, and description. Description. Michael Jackson vehemently advocated the need for a discipline of description (see "Defining a Discipline of Description," IEEE Software, Sep./Oct. 1998; it's behind pay walls, but I managed to find a free copy here, until it lasts; please go read it). He noted, however, that we lacked suck a discipline; that was 1998, and no, we ain't got one meanwhile.

Now, consider the patterns of thought I just used above to investigate potential issues:
Rules -> overlapping rules.
Incremental problem -> Priority list -> Selective combinatorial problem -> Global combinatorial problem.
Time interval -> overlapping intervals -> effective rules.
The problem is, that kind of knowledge is nowhere to be learnt. You get it from the field, when it dawns on you. Or maybe you don't. Maybe you get exposed to it, but never manage to give it enough structure, so you can't really form a proto-pattern and reuse that knowledge elsewhere. Then you won't have 20 years of experience, but 1 year of experience 20 times over.

Contrast this with more mature fields. Any half-decent mechanical engineer would immediately recognize cyclic stress and point out a potential fatigue issue. It's part of the basic training. It's not (yet) about solving the problem. It's about recognizing large families of recurrent issues.

C'mon, we got Analysis Patterns!
Sort of. I often recommend Fowler's book, and even more so the lesser known volumes from David Hay (Data Model Patterns) and a few more, similar works. But honestly, those are not analysis patterns. They are "just" models. Some better and more refined than others. But still, they are just models. A pattern is a different beast altogether, and I'm not even sure we can have analysis patterns (since a pattern includes a solution resolving forces, and analysis is not about finding a solution).

A few years ago I argued that problem frame patterns were closer to the spirit of patterns than the more widely known work from Fowler, and later used them to investigate the applicability of Coad's Domain Neutral Component (which is another useful weapon in the analyst arsenal).

However, in a maturing discipline of requirements engineering, we would need to go at a much finer granularity, and build a true catalog of recurring real-world problems, complete with questions to ask, angles to investigate, subtleties to work out. No solutions. It's not about design. It's not about process reengineering. It's about learning the problem.

Again, I don't think we will, but here is a simplified attempt at documenting the Rules/Timing stuff, in a format somewhat inspired by patterns, but still new (and therefore, tentative).

• Name

Time-dependent Rules and Transactions

• Context

We have a set of Rules that must be checked during a Transaction. Rules are effective from time T1 to time T2. Time can be expressed in absolute terms (e.g. December 25, 2011, 12:00 AM), recurring terms (every Friday, 12:00 to 17:00), etc. A Transaction is a set of actions, taking place over time, with a non-negligible duration (that is, a transaction is not an instantaneous event). Rules may have to be checked several times during the lifetime of a single Transaction.

• Problem

The interval during which any given Rule is effective may overlap with the time interval required to carry over a transaction, that is, a rule may be effective when the transactions starts, but not when it ends, or vice versa, or may be valid for an interval beginning and ending inside the transaction time span (for long-lived transactions). Still, we need to guarantee some form of coherence in the observed behavior.

• Example

[... the checkout problem would be a perfect fit here...]

• Issues and Questions

- Is there any kind of regulation dictating the system behavior?
- Would the effect of simply applying rules effective at different times be noticeable?
- Can the transaction be simplified into an instantaneous event?
- Can we just "freeze" the rule set when the transaction starts? (there are two facets here: the real-world implications of doing so, and the machine-side implications of doing so).
- Can we replay the entire transaction every time a new event is processed (e.g. a new item is added to the cart), basically moving everything forward in time?
- etc.

I don't know about you, but I would love to have a place where this kind of knowledge could be stored, cleaned up, structured, and curated.

Conclusions
It would be easy to blame it all on agility. After all, a lot of people have taken agility as an excuse for "if it's hard, don't do it". Design has been trivialized to SOLID and Beck's 4 principles. Analysis is just gone. Still, I'm not going to. I've seen companies stuck in waterfall. It ain't pretty. But it's more and more obvious that we're stuck in just another tar pit, where a bunch of gurus keep repeating the same mantras, and the discipline is not moving forward.

Coding is fascinating. A passionate programmer will stay up late to work on some interesting problem and watch his code work. I did, many times, and I'm sure I'll still do for a long while. Creating models and descriptions does not seem to be on a par with that. There is some intellectual satisfaction in there too, but not near as much as seeing something actually run. In the end, this might be the simplest, most honest explanation of why, in the end, we always come back to code.

Still, we need a place, a format, a curating community for long-term, widely useful knowledge that transcends nitty-gritty coding issues (see also my concept of Half-Life of a Knowledge Repository). Perhaps we just need to make it fun. I like the way the guys at The Fun Theory can turn everything into something fun (take a look at the piano stairs). Perhaps we should give requirements engineering another shot :-).


If you read so far, you should follow me on twitter.

Acknowledgement
The image on top is a copy of this picture from Johnny Jet, released under a creative common license with permission to share for commercial use, with attribution.

Labels: agile, analysis, requirements

Un mio cliente (centro di ricerca in zona La Spezia) cerca uno sviluppatore freelance per un progetto di 3-5 mesi.

 Conoscenze richieste:
- silverlight, html5, css, javascript
- C#, WCF, Service Oriented Architecture
- programmazione ad oggetti

Desiderabili:
- ASP.NET
- SQL Server
- Sistemi GIS
- Inglese ragionevole

 Il lavoro sara' da svolgere in sede. E' ovviamente previsto un affiancamento, ma e' richiesta una buona autonomia nell'affrontare e risolvere i problemi. Il trattamento economico e' interessante. Mandatemi un CV all'indirizzo jobs@eptacom.net, faro' una prima scrematura e poi vi mettero' in contatto diretto con il team di sviluppo.

So, here I am, back to my unpopular :-) series on the Physics of Software. It's time to explore the run-time side of entanglement, or more exactly, begin to explore, as I'm trying to write shorter posts, hoping to increase frequency as well.
A lot of time has gone by, so here is a recap from Chapter 12: Two clusters of information are entangled when performing a change on one immediately requires a change on the other (to maintain overall consistency). If you're new to this, reading chapter 12, 13 and 14 (dealing with the artifact-side of entanglement) won't hurt.

Moving to the run-time side, some forms of entanglement are obvious (like redundant data). Some are not. Some are rather narrow in scope. Some are far-reaching. I'll cover quite a few cases in this post, although I'm sure it's not an exhaustive list (yet).

• Redundancy

This is the most obvious form of run-time entanglement: duplicated information.
Redundancy occurs even without your intervention: swap-file vs. RAM contents, 3rd-2nd-1st level cache vs. RAM and vs. caches in other cores/processors, etc; at this level, entanglement satisfaction is usually called coherence.
In other cases, redundancy is an explicit design decision: database replicas, mem-cached contents, etc. At a smaller scale, we also have redundancy whenever we store the same value in two distinct variables. Redundancy means our system is always on the verge of violating the underlying information entanglement. That's why caches are notoriously hard to "get right" in heavily concurrent environments without massive locking (which is a form of friction). This is strictly related with the CAP Theorem, which I have discussed months ago, but I'll come back to it in the future.
Just like we often try (through hard-won experience) to minimize redundancy in artifacts (the Keep it dry principle), we learnt to minimize redundancy in data as well (through normalization, see below), but also to somehow control redundancy through design patterns like the observer. I'll get back to patterns in the context of entanglement in a future post.

• Referential Integrity

This is perhaps less obvious, but easy to understand. Referential integrity, as known within the relational database culture, requires (quoting Wikipedia) that any field in a table that is declared a foreign key can contain only values from a parent table's primary key or a candidate key. That means you cannot remove some records unless you update others, or that you cannot create some records unless some other is in place. That's a form of entanglement, and now we can start to grasp another concept: the idea of transaction is born out of the need to have intermediate micro-states that do not respect the entanglement constraint, yet make those micro-states invisible. As far as the external observer is concerned, time is quantized, and transactions are the ticks.
Interestingly, database normalization is all (just? :-) about imposing a specific form on data entanglement. Consider a database that is not in second normal form; I'll use the same example of the Wikipedia page for simplicity. Several records contain the same information (work location). If we change the work location for Jones, we have to update three records. This is entanglement through redundancy, of course. Normalization does not remove entanglement. What it does is to impose a particular form on it: through foreign keys. That makes it easier to deal with entanglement within the relational paradigm, but it is not without consequences for modularity (which would require an entire post to explore). In a sense, this "standardization" of entanglement form could be seen as a form of dampening. Views, too, are a practical dampening tool (on capable hands). More on this in a future post.

• Class Invariant

This, I must confess, was not obvious at all to recognize at first, which is weird in insight. A class invariant is a constraint on the internal state (data) of any object of that class. That means, of course, that those data cannot change independently. Therefore, they are entangled by definition. More precisely, a class invariant can often be stated as a set of predicates, and each predicate is revealing some form of entanglement between two or more data members.
Curiously enough (or perhaps not :-), invariant must hold at the beginning/end of externally observable methods, though it can be broken during the actual execution of those methods. That is to say, public methods play the role of transactions, making time quantized once again.

• Object Lifetime

Although this can be seen as a special case of class invariant (just like a cascade delete is ascribed to referential integrity) I'd like to make this explicit, for reasons we'll understand better in a future post. Some objects cannot outlive others. Some objects cannot be created without others. Reusing the terminology of chapter 13, this is a D-D or C-C form of entanglement. The same form of entanglement can be found underneath several DB normalization concepts, and thanks to the RT/Artifact dualism, will turn out to be a useful concept in the artifact world as well.

Toward a concept of tangling through procedural knowledge
All the above, I guess, it's pretty obvious in hindsight. It borrows heavily on the existing literature, which always considered data as something that can be subject to constraints. The risk here is to fall into the "information as data" tar pit. Information is not data. Information includes procedural knowledge (see also my old posts Lost ... and Found? for more on the concept of Information and Alexandrian centers). Is there a concept of entanglement for procedural knowledge?

Consider this simple spreadsheet:

	A	B
1	10	20
2	200	240

That may look like plain data, but behind the curtain we have two formulas (procedural knowledge):

A2=A1*B1
B2=A2*1.2

In the picture above, the system is stable, balanced, or as I will call it, in a steady state. Change a value in A1 or B1, however, and procedural knowledge will kick in. After a short interval (during which the system is unbalanced, or as I'll call it, in a transient state), balance will be restored, and all data entangled through procedural knowledge will be updated.
A spreadsheet, of course, is the quintessential Dataflow system, so it's also the natural trait d'union between knowledge-as-data and procedural knowledge. It should be rather obvious, however, that a similar reasoning can be applied to any kind of event-driven or reactive system. That kind of system is sitting idle (steady state) until something happens (new information entering the system and moving it to a transient state). That unleashes a chain of reactions, until balance has been restored.

Baby steps: now, what was the old-school view of objects (before the watering down took place)? Objects were like tiny little machines, communicating through messages. Right. So invoking a method on an object is not really different than sending the object a message. We're still upsetting its steady state, and the procedural knowledge inside the method is restoring balance (preserving the invariant, etc.)

So, what about the traditional, top-down, imperative batch system? It's actually quite straightforward now: invoking a function (even the infamous main function) means pushing some information inside the system. The system reacts by moving information around, creating new information, etc, until the output is ready (the new steady state). The glue between all that information is the procedural knowledge stored inside the system. Procedural knowledge is basically encoding the entanglement path for dynamic information.

Say that again?
This is beginning to look too much like philosophy, so let's move to code once again. In any C-like languages, given this code:

int a2 = a1 * b1;
int b2 = a2 * 1.2;
int a3 = a1 + b1;

we're expecting the so-called control-flow to move forward, initializing a2, then b2, then a3. In practice, the compiler could reorder the statements and calculate a3 first, or in between, and even the processor could execute the two instructions "out of order" (see the concept of "observable behavior" in the C and C++ standard). What we are actually saying is that a2 can be calculated given a1 and b1, and how; that b2 can be calculated given a2, and how; and that a3 can be calculated given a1 and b1, and how. The "can be calculated given ..." is an abstract interpretation of the code. That abstract interpretation provides the entanglement graph for run-time entities. The "how" part, as it happens, is more germane to the function than to the form, and in this context, is somehow irrelevant.

What about the control-flow primitives, like iteration, choice, call, etc? Iteration and choice are nothing special: at run-time, a specific branch will be taken, for a specific number of times, resulting in a specific run-time entanglement between run-time entities. Call (procedure call, function call) is somewhat different, because it reveals the fractal nature of software: whatever happens inside the function is hidden (except via side effects) from the caller. The function will carry out its job, that is, will restore the fine-grained input-output balance upon invocation.

This is the general idea of procedural knowledge: given an input, control-flow goes on, until the program reaches a steady state, where output has been produced. As noted above, due to the fractal nature of information, this process takes place at different levels. A component instance may still be in a transient state while some object contained in that instance is already back to a steady state.

In this sense, a procedure (as an artifact) is how we teach a machine to calculate an output given an input, through the stepwise creation and destruction of entangled run-time information. Phew :-).

Next steps
The beauty of the Run-Time/Artifact dualism is that whatever we learn on one side, we can often apply on the other side (with the added benefit that many things are easier to "get" on one side or another).
Here, for instance, I've introduced the concept of steady and transient state for run-time information. The same reasoning can be applied to the artifact side. Whenever you create, update or delete an artifact A1, if there is any other artifact that is entangled with A1, the system goes into an unbalanced, transient state. You have to apply work (energy) to bring it back to the steady state, where all the entangled information is properly updated (balanced). A nice quote from Leonardo da Vinci could apply here: "Motion is created by the destruction of balance" (unfortunately, I can't find an authoritative source for the quote :-). Motion in the information space is created by unsettling the system (moving to a transient state) until entanglement is finally satisfied again, and a steady state is reached. This is as true in the artifact space as in the run-time space.
Overall, this has been more like a trampoline chapter, where a few notions are introduced for the benefits of forthcoming chapters. Here is a short list of topics that I'll try to explore at some point in the future:

- The truth about multiplicity, frequency, and attraction/rejection in the run-time world.
- As above, in the artifact world.
- Moving entanglement between run-time and artifacts (we do this a lot while we design).
- The Entanglement Diagram.
- A few examples from well-known Design Patterns.
- Cross-cutting concerns and Delocalized plans: relationships with entanglement.
- More on distance, entanglement, probability of failure, and the CAP theorem.
- Dampening (isolation). This will open a new macro-chapter in the Physics of Software series.

If you read so far, you should follow me on twitter, or perhaps even share this post (I dare you :-)

Labels: form, NOSD

We're all familiar with this scene: the good guys trying to save the day (or the world) by defusing a bomb. Quickly running out of time, they have to cut a wire. Failure is not an option. The red bar won't turn into a green bar.

What if you had to write code that works the very first time? What if you had no chance to test your code, let alone fixing bugs? You got only one chance to run your code and save the world. Would that change the way you write? The way you think? How? What can you learn from that?

Lest you think I'm kinda crazy: I'm not actually suggesting that you shouldn't test your code. I'm not actually suggesting that you stop doing your test-driven thing (if you do) or that you forget your test plan (if you have one), etc etc.
Consider this like a stretching exercise. We may learn something useful just by going outside our comfort zone. Give it a try.

Yahtzee
[sorry guys, gotta go into Dr. Mallard mode for a while; you can breeze through this section, if you're the TL;DR type]

I came to write this post, and to choose Yahtzee as the underlying problem, mostly because I like coincidences :-).

A friend of mine, a relatively young programmer in love with everything agile, every once in a while shows me some "code kata" that he has completed. Not long ago, he came up with the KataYahtzee. His code was, to use his words "completely covered by tests". After all, that's what today's gurus are constantly talking about. Tests, tests, more tests (yawn). However, the code emerging from all those tests was in my opinion rather crappy (one of the benefits of friendship is that you can say this sort of things in a benign way and have them interpreted in a benign way :-). I muttered something about a better alternative, promised I would send him my code, then got caught in other stuff and never actually implemented a thing.

Still, shortly after, I was visiting the blog of an acquaintance of mine (xpmatteo), and after leaving a comment, I read a few more things including a post about Yahtzee and the removal of conditionals and loops.

That was yet another coincidence: I joked with my agile friend before, telling him that sooner or later, they would have raised the bar from anti-if to anti-while, anti-for, anti-pattern-matching, anti-list-comprehension, anti-recursion, etc :-). But I also truly liked the idea that by changing the underlying data structure you could make some rules extremely simple to check (I'm a big fan of the "smart data – stupid procedures" approach). Anyway, that reminded me of my promise, so I fired up my editor and then I closed it :-), because I'd better read the actual rules first (some would call me anti-agile for that, but I don't see why I can't leverage existing knowledge, when available).

Having never played Yahtzee before, I quickly discovered that the standard rules didn't match the rules in the kata page. However, you'll find them under "scoring variations". Reading wikipedia sorted out some of my doubts (is "13333" a double pair? No), so I was ready to actually write some code.

Before you get any further
This is where I should ask you to try it on your own. You probably won't, but if you do, remember: you want your code to run the very first time. You can write a set of test cases, but if one fails, the game is over (unless the test case is wrong).

Just to add a little juice: the modern twist on the opening scene is that shortly after the heroes manage to defuse the bomb, the countdown is restarted. So, here is your second little bomb coming: if it actually works the first time, change the rules from the "kata version" to the traditional Yahtzee rules, quickly :-), and run a new set of test cases. We're counting on you.

Picking my tools
My alma mater was completely sold on formal methods, and I used to like them too. In practice, over a number of years now, I had very few chances (and then reasons) to apply formal methods in the real world. Therefore, I won't suggest that you develop a formal proof of correctness for your code before you run it.

The problem with a formal proof is that (in many cases) the proof is at least as complex as the code, sometimes more. I won't trust my only chance on a proof that is potentially more obscure than my code. After all, Donald Knuth notoriously said "Beware of bugs in the above code; I have only proved it correct, not tried it". Sure, I may try to check my proof :-) with the help a theorem prover, but that would be cheating: in a sense, the prover would be testing my proof.

Another well-known technique to develop mission-critical code is the use of inspections and reviews. But then again, this is sort of cheating, because I would be using people to test my code (in their heads). That's a no-no.

So, having ruled out old-school stuff like Cleanroom Software Engineering, and having removed TDD from the table at very beginning, let's move on.

Is that even possible?
Of course, before we even start trying, it makes sense to ask if there is actually any hope to get a program working the very first time. Well, for simple problems, it's certainly possible. For instance, if you ask me to calculate the area of a circle with (say) 5 digits accuracy given the radius, I may just write something like:

double Area( double radius )
  {
  return radius * radius * 3.1415926 ;
  }

I would probably use a statically checked language (which is a bit like cheating, but not too much :-), just to catch any typo. After that, I would feel rather confident.

Ok, that was easy. Indeed, it would have been even simpler/safer if my language had a PI constant. The language would have been closer to the problem I was trying to solve. Which is exactly what we need: a language where our problem can be explained in a simple, straightforward way. Here, I think, is the key to my approach to writing code that runs the first time (I actually do that in real life, just not every time – more on this later on). I could say it shortly like this: Everything is simple when it has a direct mapping to your language.
Quoting Ward Cunningham (from "Clean Code" by Bob Martin), "You know you are working on clean code when each routine you read turns out to be pretty much what you expected. You can call it beautiful code when the code also makes it look like the language was made for the problem". I won't trust my only chance on nothing less than beautiful code :-).

Now, we can read this as the frequently preached "look for the best language for the problem at hand", and in a sense it's true, but truth is, once you get past a few obvious cases, you ain't gonna find a language that was made exactly for your problem. The alternative approach, of course, is to create a language that is made exactly for your problem. No, I'm not suggesting that you design a language and write a compiler first: remember, you can only run your code once. That includes your brand-new compiler. Do it if you want, but don't blame me if you manage to annihilate our civilization.

Looking back at the trivial Area function, you can easily see that my language had a useful abstraction (a "double", meaning IEEE 754 floating-point math) together with a few basic operations. Say that I trust those abstractions and the underlying hardware. I also trust my Area function, simple and small as it is. Area, therefore, could be thought of as a language extension, a new trusted abstraction.

I could simply call this "Compositional Correctness": get your low-level abstractions right. Then build more abstractions on top of those, each so small and simple that it is trivially correct. Of course, you can do it the other way too (top-down), or you can mix the strategies. In a sense, it's just an application of C. A. R. Hoare's principle: "there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult". Well, sometimes, it doesn't have to be difficult.

Listen to Your Problem
I can almost hear the word "DSL" in the background, and indeed, a Domain Specific Language may turn a complex problem into a trivial one. For instance, HTML is a pretty good DSL to create formatted, hyperlinked documents.
DSLs are often associated with humongous graphical / modeling tools (and I'm not gonna trust my only chance with one of those) or with flexible languages like Ruby. Still, remember the old Bell Labs motto: "library design is language design". You don't really need to write your language: you can extend your language just by writing a library (like with Area).
Of course, you need a programming language with some built-in flexibility (which is why I dislike overly restrictive languages), but you don't need to get too fancy. To prove my point, I'll use (modern) C# in what follows. I could have used good ol' C++ as well. Java would make a few things a little more cumbersome than I'd like to, but it could still be a viable option. Dynamic languages would do just fine. As usual, design decisions are way more important than little language-specific details.

A DSL should speak the language of your problem. Therefore, the first step toward the creation of a DSL should be to carefully listen to your problem and understand the inner nature, the basic concepts upon which the problem structure is built. In a better world, this would be a widely researched, frequently applied process, but in this world, it's not (although you can find a few noteworthy works on DLSs). In practice, I first try to understand whether the problem "fits" with some of the categories I'm familiar with, like:

- is it a unification / pattern matching problem? For instance, I could model a double pair like (x,x,y,y,z), and leave it as an implicit convention of the language that different letters will necessarily match different faces. This looks promising, until you look at some rules (like Large straight) that don't really fit that model. Sure, you can stretch the language a little, and allow literals: by the way, this is the approach taken by the original Ruby Quiz solution, which inspired the kata, which inspired this post. However, if you look at the original Yahtzee rules for a Large straight, they won't fit with literals either (it's just five sequential dice). Pattern matching is not the real Yahtzee language. As an aside, having read the Ruby code, it just doesn't like like the kind of code I would trust to run flawlessly the first time :-).

- is it an incremental counting problem? What I really loved in Matteo's post was the idea that you could simply keep a count of faces, and that would easily provide a score for several rules. However, he already showed that such technique won't cover all rules.

- is it a map/reduce kind of problem? It doesn't really look like that, because the "map" is supposed to take a (k1,v1) pair and return a list of (k2,v2) pairs, to be grouped and reduced. I could shoehorn Yahtzee into this, but it doesn't look like a natural fit for its rules.

- is it a filter and projection problem (a-la SQL)? This may actually work. Conditions can be expressed as filters. I can start with a sequence and, through a series of trivial filters, end up with an empty sequence (if there is no match for the rule) or with the surviving dice. At that point, it's basically a matter of taking the sum. Hmm, perhaps this could work. Time to dig a little deeper.

Why not SQL, then?
At this point, it might be tempting to simply use SQL as a language. Alternatively, if you're familiar with .NET, you may want to try LINQ, which is basically a SQL-like syntax to play with collections. Well, why not. Here is a Pair scoring expressed as a LINQ statement:

2 * 
(from d in dice group d by d into g 
 where g.Count() >= 2 orderby g.Key
 descending select g.Key).FirstOrDefault();

Part of the trick here is that the select expression returns a collection of integers, and that FirstOrDefault will return the first element (the highest face with a pair in the original sequence) or 0, which is the default value for int. It works with any kind of sequence (IEnumerable). In my tests (yes, I tested this :-), I just used an array of integers to represent dice.

The problem is, however, that this code has to be carefully read. I have seen production code full of statements like that, and to put it gently, it sucks.
On the upside, it's control-flow free, so it should please a few people out there :-). More seriously, the absence of control flow should probably by a byproduct of the "right" Yahtzee language. Control flow makes code harder to read in one pass. If I have to trust the code, I'd like to read it and "get it" in one pass. Note, however, that absence of control flow is not a guarantee of readability (or testability, or whatever else). Standard SQL, for instance, is control-flow free, but I can hardly say that a lengthy query mixing inner and outer joins can be easily read and trusted. The LINQ query above has no control flow, but I wouldn't like my code to read like that. Part of the problem lies with the LINQ syntax, but another part of the problem is that we're relying on standard data structures (collections) and standard SQL-like instructions, and they don't speak the problem language.

Sketching the Yahtzee language
Language design is difficult and requires that we pay attention to many details and facets. We want the "procedural" part to become trivial: this is the point of the entire exercise. Sketching the language requires that we focus on a few rules, while keeping the others (or at least some of them) in the back of our mind, because the "right" language must be a good fit for most of them (ideally, all of them). However, picking two similar rules to begin with allows us to focus slightly better, without being too narrow. I'll pick the Pair and Double Pair rules.

We also have to choose a language style. The "language style" inside Area was the familiar "infix operations" borrowed from math. Personally, I can't see that as a good fit for the Yahtzee rules. I'm pretty sure, however, that I'd like my rules to become one-liners, something you can understand by reading it once. I think a Fluent Interface style will fit that role better.

At this stage, I would not focus too much on [domain] classes. Sure, some classes are there just for the picking, as Bertrand Meyer said. Dice, Face, Roll, Rule, Score, etc. However, when you design a small DSL, starting with classes is not necessarily the right approach. More often than not, classes will emerge as the (Alexandrian) centers of the conversations we're building. Still, we have to start somewhere, so I'll have an imaginary "roll" object (of unknown class) representing the roll we want to score against a rule.

Let's give it a try. A trivial pair of one-liners for our rules could be:

roll.GetHighestPair().Score();
roll.GetTwoHighestPairs().Score();

however, this is wrong. Sure, it looks like good old stepwise refinement, but in fact I'm just pushing the burden of Yahtzee rules on roll's shoulders. Rules can grow and change, roll should be a stable abstraction.
Even this little step in the wrong direction, however, can teach us something: what if there is no pair? I surely don't want my rules to be choke-full of conditionals, as that would wreck my "read it once and see it's right" principle. So, we can already give some structure to our language:

we are manipulating a collection of die (roll)
the collection may go through several "stages" (the "filters" or "selections")
at every stage, the collection may get empty, but never a null reference (that will avoid conditionals)
the score of an empty collection is zero (which is consistent with Yahtzee)

Ok, let's give it another try:

roll.GetPairs().GetHighest().Score();
roll.GetPairs().GetTwoHighest().Score();

this is marginally better, because we formed a few smaller concepts. We can ask a roll for pairs; we can ask a pairs collection for the highest (by face: that should be made more explicit in our language), or the two highest. However, it's still very tailored, and it does not support any other rule. As we progress, the other rules have to get slightly more in the foreground to help us shaping our language (yeah, it's really a Gestalt process, but don't get me started on this :-).

The problem with the concept of pair is that it's not general enough to appear in our language. We need something more general. In the end, Yahtzee as a selection language is mostly about filtering by face or by occurrences of dice, grouped by faces. Perhaps a class would help here, a group of dice with the same face:

class DiceGroup
  {
    public DiceGroup(int f, int c)
    {
      Face = f;
      Count = c;
    }

    public int Face
    {
      get;
      private set;
    }

    public int Count
    {
      get;
      private set;
    }

    public int Score()
    {
      return Face * Count;
    }
  }

Now, I know some of you have been brainwashed :-) to the point that you feel the need to write a load of test cases for this class, but let's face it: it's basically as complex as Area, actually even less. I made it more complex than strictly needed, by using properties instead of plain public data, but that's giving me something back: immutability. A DiceGroup cannot be changed after construction. Not having to reason about mutable objects helps a lot in writing safe code, and Yahtzee is not a performance-critical problem, so I can afford it. DiceGroup is also a stable abstraction. Either you use it as-is, or you scrap it and use something else. There is no point in tweaking DiceGroup. Hence, a real economic theory for software design (as opposed to babbling about the need to refactor every single line you ever wrote) would tell us that there is little value in drowning this specific class in trivial test cases.
Provocation: what if the best way to minimize the cost of change was to find a number of useful, small abstractions that do not change at all, or rarely do, because they're in frictionless contact with the nature of the problem?

Now that I have a DiceGroup, what is a Roll? Well, a Roll might be just a sequence of DiceGroup. I can take my 5 dice, organize them in 1 to 5 DiceGroup objects (depending on the actual faces) and call that sequence a Roll. I'll ignore the details of how to do that for now, and get back to it later. Let's say that a Roll will be immutable too, so whenever we filter a Roll, we get a new Roll.

So, say that I roll my dice and I get 4, 4, 1, 4, 3. I'll organize them into three DiceGroup objects: {4,3}, {1,1}, {3,1} (read them as {Face,Count}). To score them as Pair, I would have to remove the two DiceGroup with Count < 2, but that's not enough: {4,3} must also be turned into {4,2}, because the idea of Pair is that you max out at 2 occurrences. Hmm. That's just like trimming. I could write Pairs just like:

roll.TrimCountTo(2).TopFace().Score();

where both TrimCountTo and TopFace return a new Roll, filtered and changed as needed. The rule is pretty readable, and given the language conventions above about returning an empty sequence if there is no match for the filter, I'll just get 0 when there are no pairs.

TwoPairs would require a little extension to the language, because TopFace is again a bit too tailored to the Pair rule. The language itself is also telling us that (if we listen closely) because a TopFace method should return a DiceGroup, not a Roll. That makes the language asymmetric, so operations are harder to combine, and I don't want that. Lucky enough, this is easy to fix: why not having a TopByFace(n) method returning a roll with the n best DiceGroup (by face)? Now I can write Pair and TwoPairs like:

roll.TrimCountTo(2).TopByFace(1).Score();
roll.TrimCountTo(2).TopByFace(2).Score();

[Un]surprisingly enough, those two little functions will get us many other rules for free:
ThreeOfAKind: roll.TrimCountTo(3).Score();
FourOfAKind: roll.TrimCountTo(4).Score();
Yahtzee: r.TrimCountTo(5).Score();
Chance: roll.Score();

Yo, I like this. Rules are trivially correct. They can be created by combining a few operators. This looks like the right DSL, or part of it. You can't express all the Yahtzee rules with just two operators (TrimCountTo and TopByFace). Ones, Twos, ..., Sixes are out. Well, they can be easily brought in by mirroring TrimCountTo with TrimFaceTo. So, I will have:

Ones: roll.TrimFaceTo(1).Score();
etc...

This is almost too simple, isn't it? Well, when it gets that easy, it's because the language "was made for the problem", to quote Ward again, or because our solution is in frictionless contact with the forcefield, to bring in Alexander once more.

As usual, the thinking process is more important than code here. I am getting feedback. Backtalk, actually. I'm just not getting it the way most people are used / taught to (by executing code). I'm getting feedback through a reflective conversation with my material. I know, this is not what everyone is telling you. And please understand I'm not implying that you should stop testing your code. I'm just taking the path less traveled by to see what happens.

What's left? Small straight (1,2,3,4,5), Large straight (2,3,4,5,6), Full house (two of a kind + three of a kind, exactly: as per instructions, (4,4,4,4,4) is not a Full house). These rules cannot be expressed yet. You may see a pattern here, however. The straights are basically trimming by face, but with multiple faces. Full house is trimming by count, but with multiple counts. This may suggest an improvement to our language: turn TrimCountTo into TrimCountsTo, and TrimeFaceTo into TrimFacesTo. Care must be taken, however, not to break the simplicity of the language by creating too powerful instructions. Perhaps an orthogonal operator should be brought in. This is already a very long post, and I still have a lot of code to show, so I'll leave this extension out.

The Structure of the Yahtzee Language
DiceGroup was trivial. I basically did that bottom-up. Roll might not be just as trivial, and so far we only know that we need a TrimCountTo, a TrimFaceTo, and a Score method. Now we have two forces to balance:
We want Roll to be as trivial as possible. It has to work the first time! Honestly, this is the keystone now. DiceGroup was so trivial to be obviously correct. Rules are now so obvious that we can happily trust them. We need Roll to be so simple that we can trust it too.
We want our DSL to be extendible. We're missing a few rules, and the "standard" Yahtzee rules were different as well. I don't want to tweak Roll to change or extend the rules: remember what I said about finding stable abstractions.

I've been talking about [programming] language design in the past, and how "good" languages should be largely extendible. A few built-in core concepts supporting most of the language itself, which would then largely be in a library. It's time to apply the same principle to the Yahtzee Language.

Here is where our actual implementation language can help or hinder. C# is almost helpful here. A dynamic language, or a language with structural conformance for interfaces, would make the following code much shorter and easier to read. Anyway, the key concept here is that of an open class, something that we can extend without breaking the basic abstraction and without creating subtypes (while still not breaking encapsulation). What we need is a "simple" Roll class with the core methods and a set of Extension Methods (in C# parlance) which implements the "upper layer" of our language. Score() will end up in Roll, while TrimCountTo, TrimFaceTo, and TopByFace will land in a RollLanguage class. Therefore, adding support for more rules won't require any change to existing classes. Just implement new extension methods. Of course, that in turn requires Roll to be flexible enough to support basically all the desirable extension methods.

In practice, it's not that hard. Let's start with RollLanguage. Ideally, it would be something like this:

static class RollLanguage
  {
    public static Roll TopByFace(this Roll groups, int minCount)
    {
    //trivial code here
    }

    public static Roll TrimCountTo(this Roll groups, int minCount)
    {
    //trivial code here
    }

    public static Roll TrimFaceTo(this Roll groups, int face)
    {
    //trivial code here
    }
  }

Let's go ahead and implement TrimFaceTo. This would be my first shot:

public static Roll TrimFaceTo(this Roll groups, int face)
{
  return groups.Where(g => g.Face == face);
}

note that I'm using LINQ (not exactly, I'm using the Linq extension method Where, not the LINQ syntax). However, this is a very simple expression. Sure, you have to know C# and .NET, but that's basically returning a sequence with all the DiceGroup which have the specified face. Unfortunately that won't compile :-), for two simple reasons:

1) Roll must be an IEnumerable<DiceGroup>, because that's what Where expects.
2) Where will then return an IEnumerable<DiceGroup>, not a Roll.

(1) Is relatively trivial to fix, and while doing so, we'll sketch Roll as well:

class Roll : IEnumerable<DiceGroup>
{
  public IEnumerator<DiceGroup> GetEnumerator()
  {
    return diceByFace.GetEnumerator();
  }

  System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
  {
    return diceByFace.GetEnumerator();
  }

  private IEnumerable<DiceGroup> diceByFace;
}

This is a boilerplate code (partially generated by Visual Studio), that I wouldn't need with dynamic languages (or with structural conformance). It's pretty trivial though: Roll has-a and is-a IEnumerable<DiceGroup>. It implements the interface by forwarding to the data member. I'm rather confident that it's correct.

(2) Requires that we tweak our code a little (again, this wouldn't be necessary in a more flexible programming language). Code speaks better than words here:

public static Roll TrimFaceTo(this Roll groups, int face)
{
  return groups.Where(g => g.Face == face).AsRoll();
}

public static Roll AsRoll(this IEnumerable<DiceGroup> r)
{
  return new Roll(r);
}

All methods in RollLanguage can be implemented over Linq, but have to end with AsRoll() to convert the IEnumerable into an actual Roll. Yeah, it sucks, but the alternative (getting rid of DiceGroup and Roll altogether) is even worse (I'll touch on this later on). An implicit conversion operator would be ok, but C# does not allow implicit conversion from interfaces.

So, here is TrimToCount:

public static Roll TrimCountTo(this Roll groups, int minCount)
{
  return groups.Where(g => g.Count >= minCount).Select(
    g => new DiceGroup(g.Face, minCount)).AsRoll();
}

Honestly, this is a borderline function. You need to be somewhat confident with Linq to get it the first time through. Basically, I'm filtering by count (that's the easy part) and then I'm capping the count by creating a new DiceGroup with the same face and minCount as the actual count. I can trust this, but if you don't, I'll understand :-). Simpler code is welcome!

So far I managed to avoid any kind of conditional. I didn't want any IF in my rules, and I didn't need any in RollLanguage thus far. However, my most readable TopByFace implementation is if-based:

public static Roll TopByFace(this Roll groups, int minCount)
{
  if( groups.Count() < minCount )
    return Roll.Empty();
  else
    return groups.OrderByDescending(g => g.Face).Take(minCount).AsRoll();
}

The idea is simple: if I don't have at least minCount groups, the filter fails and an empty roll must be returned. Otherwise, I can simply use Linq again, sort my groups by face (descending) and take the first minCount groups. Again, this might be borderline for someone uncomfortable with Linq. Looks pretty trivial to me, but simpler code would be welcome; actually, if you are an anti-if devotee, feel morally obliged to show me how to improve that code and get rid of the if (using a ternary operator is not a real alternative, of course). Pushing the IF in a custom version of Take is possible, but that would be just moving it around (with an interesting trade/off in efficiency Vs. reusability).

By writing this code, we already got a set of requirements for Roll. Must be (and have) an IEnumerable<DiceGroup>; must have a Score() method; must have an Empty() static method returning an empty sequence; must be constructable from a IEnumerable<DiceGroup> (trivial); we also need the "real" constructor, taking a sequence (or array) of integers (the 5 dice) and doing the actual grouping. We have already seen part of the code above, so I'll add the trivial parts here and leave the "hard" constructor last:

class Roll : IEnumerable<DiceGroup>
{
  public Roll(IEnumerable<DiceGroup> dice)
  {
    diceByFace = dice;
  }

  public int Score()
  {
    return diceByFace.Sum(g => g.Score());
  }

  public static Roll Empty()
  {
    return empty;
  }

  private IEnumerable<DiceGroup> diceByFace;
  private static int[] emptyArray = new int[0];
  private static Roll empty = new Roll(emptyArray);
}

OK. Once again, this is so trivial that I can trust it. Now let's get over with the grouping; lucky me, I can simply use Linq once again:

public Roll(int[] dice)
{
  diceByFace = dice.GroupBy(x => x).Select(
    g => new DiceGroup(g.First(), g.Count()));
}

Once again, this is borderline if you aren't familiar with Linq. I'm grouping by faces, and for each group I'm building a DiceGroup object with face and count. I'd like a simpler syntax (g.First() is not exactly obvious) but this is what Linq provides me with. Usually, I'd also add an assertion that dice has 5 elements. I left that out to keep my code shorter.

Interestingly, although I ignored performance, concurrency, etc so far, it would be rather simple to translate all this code in high-performance C++, using mutable stack-based objects, because in Yahtzee we're always dealing with (at most) 5 dice, so everything could be pre-allocated at fixed size. I would have to re-create some of the Linq extension methods, but C++11 has got lambda, so the overall style could stay the same.

Wrap it up (and extend it)
We're basically through. At this point, everything else is cosmetic. How do I group rules? Do I need a Rule interface? Well, in C#, I could just use a delegate for that. Can I just use member functions in a Yahtzee class and get over with it? Sure, unless we're concerned about a hierarchy of Yahtzee games (standard, non-standard, etc). Also, organizing rules into an array or dictionary might be the simplest way to glue them to the UI (which I'm not going to write). Since this is all basically irrelevant to our quest, I just wrote a set of one-line member functions.

Part 2 of the problem was to change rules to the original Yahtzee scoring. For instance, adopting the original rules a Three-Of-A-Kind will get you the sum of all dice (not just the 3 of a kind). The simplest way, perhaps a bit too technical, would be to do it like this:

Math.Sign( ScoreThreeOfAKind( roll ) ) * roll.Score()

Alternatively, you can create new filters, returning the original roll (if the filter is passed) or an empty roll. It's basically trivial now.

In the end, I didn't create classes like Dice or Face. They would add some value (by forcing constraints) but I don't want to get too serious with this code :-).

The entire code is here. For simplicity, I used a single file. I added a few tests in the end, so that I could either succeed or fail when I finally ran the code (it worked!). In the end, I sort of like it. It's not perfect, but is pretty good. Feel free to improve it, in which case, I'd like to know!

Winding down
Even though I've written so much, there would be much more to say. I'll give you the short version:

Bad abstraction vs. Good abstraction
strictly speaking, I wouldn't need any "base" abstraction like DiceGroup and Roll in my Yahtzee language. I could write every single function as an extension method of standard library classes. Why should I use concrete, non-reusable terms like Roll and DiceGroup when I could just use IEnumerable<IGrouping<int, int>>? Arguably, my code would be "more abstract", "more mathematic", and therefore "better" by sticking to abstract, domain-independent interfaces. Here, I think, my programming style differs from some. I used to like the idea of abstracting the domain away from my code, perhaps 20 years ago or so. I guess it was some sort of academic imprinting. However, over time I moved into a different coding style, where abstractions are taken from the problem domain, not from abstract math. This allows me to focus on the real-world, concrete problem I want to solve; to speak the language of the problem, with people and with my code; and to leave behind me code that can be more easily understood by people who know about the problem domain, and perhaps a little less about computer science. I hinted at this issue in the past, as in my mind map about habitable software, and even before, without ever taking time to do it justice. Perhaps this post will fill part of that gap.

Code quality
As I discussed my code with my friend, he sort of liked it. He couldn't fully admit it, because hey, I had no tests in place, but he obviously felt it was much easier to read and to evolve. Of course, popular authors would say that it's bad code anyway: "Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well-encapsulated it is" (Michael C. Feathers, "Working Effectively with Legacy Code"). This is pretty harsh, because if I had to choose between the code above (with this blog post as the dreaded "comprehensive documentation") and some crappy code entirely covered by tests, I would have a hard time choosing the latter.
Curiously enough, for a very long time testability has been like the stepchild of -ilities. In the past few years, the pendulum swung too far on the other side, and now it's all about tests.
Since it's way too easy to babble and pontificate, I'd like someone to write much better code than mine (and yeah, completely different, thank you) using TDD, and show me the process through which tests are driving him toward that code. If you ain't got code, please don't bother leaving a comment about the greatness of TDD and how dumb I am for even trying to write code without using tests. OTOH, if you got code to back up your claims, you're more than welcome.

Limits and confessions
I'm making a habit to conclude with a few confessions, so here I come. I don't always write code this way. I tend to: I like my code to work the first time. However, it took me more than the allotted time of 1h to complete my mission. Writing the code was easy. Thinking about the language, trying out a few different styles, rejecting options, etc, was not, and almost never is.
The "little DSL" approach requires a clarity of thought that we can't always afford, and it does not necessarily pay off. Yahtzee is also quite simple, and requirements were well-known. Again, that's a luxury we don't face so often (although I don't buy the idea that we never know what we're doing; deep inside uncertainty there is often a core of stable concepts, if you take the time to dig it out).
There are also limits to the applicability of these techniques. Not every single problem lends itself well to the DSL approach. As usual, we have to be flexible, understand the nature of the problem, and choose the best design strategy. A DSL was just particularly well-suited for Yahtzee.
Indeed, a few years ago I spent some time doing a [little] upfront design, followed by implementation, for the bowling problem. Back then, my aim was to show that you don't have to give up on OOP and OOD so easily, and that by sticking to OOD you won't end up with a LOC monster as it was suggested elsewhere. I didn't use a DSL back then, but of course, it's quite possible to combine a little upfront OOD with a little DSL-like library with a little emergent design. This is very close to what I usually do when I have to code something on my own. Group dynamics are different, and I tend to adapt my techniques to the context.

What can we learn from this stuff?
A lot, I hope :-). Software development is about acquiring and encoding knowledge into an executable format (my usual quote from Phillip Armour). Our code can be extremely simplified through a careful selection of the language we use to encode that knowledge: when the language is aligned with the problem domain, many layers of complexity simply fall down. Your programming language is usually not aligned with your problem. It's your job to make it so, by introducing the right abstractions.
So, next time you're facing some complex issue, ask yourself: what if this code had to run the very first time?

Acknowledgement
The image on top is adapted from this picture from Lauri Rantala, released under a creative common license with permission to modify the picture and for commercial use.

Labels: .NET, C#, design, language design

Where I present seemingly unrelated facts that are, in fact, not unrelated at all :-).

The CAP Theorem
If you keep current on technology, you're probably familiar with the proliferation of NoSQL , a large family of non-relational data stores. Most NoSQL stores have been designed for internet-scale applications, where large data stores are preferably deployed using an array of loosely connected low-cost servers. That brings a set of well-known issues to the table:

- we'd like data to be consistent (that is, to respect all the underlying constraints at any time, especially replication constraints). We call this property Consistency.

- we'd like fast response time, even when some server is down or under heavy load. We call this property Availability.

- we'd like to keep working even if the system gets partitioned (a connection between servers fails). We call this property Partition tolerance.

The CAP Theorem (formerly the Brewer's Conjecture) says that we can only choose two properties out of three.

There is a lot of literature on the CAP Theorem, so I don't need to go in further details here: if you want a very readable paper covering the basics, you can go for Brewer's CAP Theorem by Julian Browne, while if you're more interested in a formal paper, you can refer to Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services by Seth Gilbert and Nancy Lynch.

There is, however, a relatively subtle point that is often brought up and seldom clarified, so I'll give it a shot. Most people realize that there is some kind of "connection" between the concept of availability and the concept of partition. If a server is partitioned away, it is by definition not available. In that sense, it may seem like there is some kind of overlapping between the two concepts, and overlapped concepts are bad (orthogonal concepts should be preferred). However, it is not so:

- Availability is an external property, something the clients of the data store can see. They either can get data when they need it, or they can't.

- Partitioning is an internal property, something clients are totally unaware of. The data store has to deal with that on the inside.

Of course, given the fractal nature of software, in a system of systems we may see lack of availability at one level, because of the lack of partition tolerance at a lower level (or vice-versa, as an unavailable node may not be distinguishable from a partitioned node).

To sum up, while the RDBMS world has usually approached the problem by choosing Consistency and Availability over Partition tolerance, the NoSQL world has often chosen Availability and Partition tolerance, leading to the now famous Eventually Consistent model.

The Memristor, or the Value of a Theory
As a kid, I spent quite a bit of time hacking electronic stuff. I used to scavenge old TV sets and the like, and recover resistors, capacitors, and inductors to use on very simple projects. Little I knew that, theoretically, there was a missing passive element: the memristor.

Leon Chua developed the theory in 1971. It took until 2008 before someone could finally create a working memristor, using nanoscale techniques that would have looked like science fiction in 1971. Still, the theory was there, long before, pointing toward the future. At the bottom of the theory was the realization that the three known passive elements could be defined by a relationship between four concepts (current, voltage, charge, and flux-linkage). By investigating all possible relationships, he found a missing element, one that was theoretically possible, but yet undiscovered. He used the term completeness to indicate this line of reasoning, although we often call this kind of property symmetry.

Software Entanglement
In Chapter 12 of my ongoing series on the nature of software, I introduced the concept of Entanglement, further discusses in Chapter 13 and Chapter 14 (for the artifact world).

Here, I'll reuse my early definition from Chapter 12: Two clusters of information are entangled when performing a change on one immediately requires a change on the other.

Entanglement is constantly at work in the artifact world: change a class name, and you'll have to fix the name everywhere; add a member to an enumeration, and you'll have to update any switch/case statement over the enumeration; etc. It's also constantly at work in the run-time world.

Although I haven't formally introduced entanglement for the run-time world (that would/will be the subject of the forthcoming Chapter 15), you can easily see that maintaining a database replica creates a run-time entanglement between data: update one, and the other must be immediately (atomically, as we use to say) updated. Perhaps slightly more subtle is the fact that referential integrity is strongly related to run-time entanglement as well. Consistency in the CAP theorem, therefore, is basically Entanglement satisfaction.

Once we understand that, it's perhaps easier to see that the CAP theorem applies well beyond our traditional definition of distributed system. Consider a multi-core CPU with independent L1 caches. Cache contents become entangled whenever they map to the same address. CPU designers usually go with CA, forgetting P. That makes a lot of sense, of course, because we're talking about in-chip connections.

That's sort of obvious, though. Things get more interesting when we start to consider the run-time/artifact symmetry. That's part of the value of a [good] theory.

A CAP Theorem for Artifacts
My perspective on the Physics of Software is strongly based on the run-time/artifact dualism. Most forces apply in both worlds, so it is just natural to bring ideas from one world to another, once we know how to translate concepts. Just like symmetry allowed Chua to conceive the memristor, symmetry may allow us to extend concepts from the run-time world to the artifact world (and vice-versa). Let's give the CAP theorem a shot, by moving each run-time concept to the corresponding notion in the artifact world:

Consistency: just like in the run-time world, it's about satisfying Entanglement. While change in the run-time world is a C/U/D action on data, here is a C/U/D action on artifacts. If you add a member to an enumerated type, your artifacts are consistent when you have updated every portion of code that was enumerating over the extension of that type, etc.

Availability: just like in the run-time world, is the ability to access a working, up-to-date system. If you can't deploy your new artifacts, your system is not available (as far as the artifact world is concerned). The old system may be up and running, but your new system is not available to the user.

Partition tolerance: just like in the run-time world, it means you want your system to be usable even if some changes can't be propagated. That is, some artifacts have been partitioned away, and cannot be reached. Therefore, some sort of partial deployment will take place.

Well, indeed, you can only have two :-). Consider the artifacts involved in the usual distributed system:

- one or more servers
- one or more clients
- a contract in between

The clients and the server (as artifacts – think source code not processes) are U/D entangled over the contract: change a method signature (in RPC speak), or the WSDL, or whatever represents your contract, and you have to change both to maintain consistency.

What happens if you update [the source code of] a server (say, to fix a serious bug), and by doing so you need to update the contract, but cannot update [the source code of] a client [timely]? This is just like a partition. Think of a distributed development team: the team working on the client, for a reason or another, has been partitioned away.

You only have two choices:

- let go of Availability: you won't deploy your new server until you can update the client. This is the artifact side of not having your database available until all the replicas are connected (in the run-time world).

- let go of Consistency: you'll have to live with an inconsistent client, using a different contract.

Consequences
Chances are that you:
- Shiver at the idea of letting go of Consistency.
- Think versioning may solve the problem.

Of course, versioning is a coping strategy, not a solution. In fact, versioning is usually proposed in the run-time world of data stores as well. However, versioning your contract is like pretending that the server has never been updated. It may or may not work (what if your previous contract had major security flaws that cannot be plugged unless you change the contract? What if you changed your server in a way that makes the old contract impossible to keep? etc). It also puts a significant burden on the development team (maintaining more source code than strictly needed) and may significantly slow down development, which is another way to say it's impacting availability (of artifacts).

It is important, however, that we thoroughly understand context. After all, any given function call is a contract, and we surely want to maintain consistency as much as we can. So, when does the CAP Theorem apply to Artifacts? Partition tolerance and Availability must be part of the equation. That requires one of the following conditions to apply:

- Independent development teams. The Facebook ecosystem, for instance, is definitely prone to the artifact side of the CAP Theorem, just like any other system offering a public API to the world at large. You just can't control the clients.

- A large system that cannot be updated in every single part (unless you accept to slow down deployment/forgo availability). A proliferation of clients (web based, rich, mobile, etc) may also bring you into partitioning problems.

If you work on a relatively small system, and you have control of all clients, you're basically on a safe field – you can go with Consistency and Availability, because you just don't have significant partitions. Your development team is more like a multi-core CPU than a large distributed system.

Once we understand that versioning is basically a way to pretend changes never happened on the server side, is there something similar to Eventual Consistency for the artifact side?

TolerantReader may help. If you put extra effort into your clients, you can actually have a working system that will eventually be consistent (when source code for clients will be finally updated) but still be functional during transition times, without delaying the release of server updated. Of course, everything that requires discipline on the client side is rather useless in a Facebook-like ecosystem, but might be an excellent strategy for a large system where you control the clients.

Interestingly, Fowler talks about the virtues of TolerantReader as if it was always warranted for a distributed system. I tend to disagree: it makes sense only when some form of development-side partitioning can be expected. In many other cases, an enforced contract through XSD and code generation is exactly what we need to guarantee Consistency and Availability (because code generation helps to quickly align the syntactical bits – you still have to work on the semantic parts on both sides). Actually, the extra time required to create a TolerantReader will impact Availability on the short term (the server is ready, the client is not). Context, context, context. This is one more reason why I think we need a Physics of Software: without a clear understanding of forces and materials, it's too easy to slip into the fallacy that just because something worked very well for me [in some cases], it should work very well for you as well.

In fact, once you recognize the real forces, it's easy to understand that the problem is not limited to service-oriented software, XML contracts, and the like. You have the same problem between a database schema and a number of different clients. Whenever you have artifact entanglement and the potential for development partitioning (see above) you have to deal with the artifact-side CAP Theorem.

Moving beyond TolerantReader (which is not helping much when your client is sending the wrong data anyway :-), when development partitioning is expected, you need to think about a flexible contract from the very beginning. Facebook, for instance, allows clients to specify which fields they want to receive, but is still pretty rigid in the fields they have to send.

This is one more interesting R&D challenge that is not easily solved with today's technology. Curiously enough, in the physical world we have long understood the need to use soft materials at the interface level to compensate for imperfections and unexpected variations of contact surfaces. Look at any recent, thermally insulated window, and most likely you're gonna find a seal between the sash and the frame. Why are seals still needed in a world of high-precision manufacturing? ;-)

Conclusions
Time for a confession: when I first thought about all the above, I had already read quite a bit on the CAP Theorem, but not the original presentation from Eric Brewer where the conjecture (it wasn't a theorem back then) was first introduced. The presentation is about a larger topic: challenges in the development of large-scale distributed systems.

As I started to write this post, I decided to do my homework and go through Brewer's slides. He identified three issues: state distribution, consistency vs. availability, and boundaries. The first two are about the run-time world, and ultimately lead to the CAP theorem. While talking about Borders, however, Brewers begins with run-time issues (independent failure) and then moves into the artifact world, talking (guess what!) about contracts, boundary evolution, XML, etc. Interestingly, nothing in the presentation suggests any relationship between the problems. Here, I think, is the value of a good theory: as for the memristor, it provides us with leverage, moving what we know to a higher level.

In fact, another pillar of the Physics of Software is the Decision Space. I'm pretty sure the CAP theorem can be applied in the decision space as well, but that will have to wait.

Well, if you liked this, stay tuned for Chapter 15 of my NOSD series, share this post, follow me on twitter, etc.

Labels: architecture, article reference, NOSD

If you take a walk in my hometown, and head for the center, you may stumble into a building like this:



from a distance, it's a relatively pompous edifice, but as you get closer, you realize that the columns, the ornaments, everything is fake, carefully painted to look like the real thing. Even shadows have been painted to deceive the eye. Here is another picture from the same neighbor: from this angle, it's easier to see that everything has just been painted on a flat surface:



Curiously enough, as I wander through the architecture and code of many systems and libraries, I sometimes get the same feeling. From a distance, everything is object oriented, extra-cool, modern-flexible-etc, but as you get closer, you realize it's just a thin veneer over procedural thinking (and don't even get me started about being "modern").

Objects, as they were meant to be
Unlike other disciplines, software development shows little interest for classics. Most people are more attracted by recent works. Who cares about some 20 years old paper when you can play with Node.js? Still, if you never took the time to read The Early History of Smalltalk, I'd urge you to. Besides the chronicles of some of the most interesting times in hw/sw design ever, you'll get little gems like this: "The basic principal of recursive design is to make the parts have the same power as the whole." For the first time I thought of the whole as the entire computer and wondered why anyone would want to divide it up into weaker things called data structures and procedures. Why not divide it up into little computers, as time sharing was starting to? But not in dozens. Why not thousands of them, each simulating a useful structure?

That's brilliant. It's a vision of objects like little virtual machines, offering specialized services. Objects were meant to be smart. Hide data, expose behavior. It's more than that: Alan is very explicit about the idea of methods as goals, something you want to happen, unconcerned about how it is going to happen.

Now, I'd be very tempted to write: "unfortunately, most so-called object-oriented code is not written that way", but I don't have to :-), because I can just quote Alan, from the same paper: The last thing you wanted any programmer to do is mess with internal state even if presented figuratively. Instead, the objects should be presented as sites of higher level behaviors more appropriate for use as dynamic components. [...]It is unfortunate that much of what is called “object-oriented programming” today is simply old style programming with fancier constructs. Many programs are loaded with “assignment-style” operations now done by more expensive attached procedures.

That was 1993. Things haven't changed much since then, if not for the worse :-). Lot of programmers have learnt the mechanics of objects, and forgot (or ignored) the underlying concepts. As you get closer to their code, you'll see procedural thinking oozing out. In many cases, you can see that just by looking at class names.


Names are a thinking device
Software development is about discovering and encoding knowledge. Now, humans have relatively few ways to encode knowledge: a fundamental strategy is to name things and concepts. Coming up with a good name is hard, yet programming requires us to devise names for:
- components
- namespaces / packages
- classes
- members (data and functions)
- parameters
- local variables
- etc
People are basically lazy, and in the end the compiler/interpreter doesn't care about our beautiful names, so why bother? Because finding good names is a journey of discovery. The names we choose shape the dictionary we use to talk and think about our software. If we can't find a good name, we obviously don't know enough about either the problem domain or the solution domain. Our code (or our model) is telling us something is wrong. Perhaps the metaphor we choose is not properly aligned with the problem we're trying to solve. Perhaps there are just a few misleading abstractions, leading us astray. Still, we better listen, because we are doing it wrong.

As usual, balance is key, and focus is a necessity, because clock is ticking as we're thinking. I would suggest that you focus on class/interface names first. If you can't find a proper name for the class, try naming functions. Look at those functions. What is keeping them together? You can apply them to... that's the class name :-). Can't find it? Are you sure those functions belong together? Are you thinking in concepts or just slapping an implementation together? What is that code really doing? Think method-as-a-goal, class-as-a-virtual-machine. Sometimes, I've found useful to think about the opposite concept, and move from there.

Fake OO names and harmful conventions
That's rather bread-and-butter, yet it's way more difficult than it seems, so people often tend to give up, think procedurally, and use procedural names as well. Unfortunately, procedural names have been institutionalized in patterns, libraries and coding conventions, therefore turning them into a major issue.

In practice, a few widely used conventions can seriously stifle object thinking:
- the -er suffix
- the -able suffix
- the -Object suffix
- the I- prefix

of these, the I- prefix could be the most harmless in theory, except that in practice, it's not :-). Tons of ink has been wasted on the -er suffix, so I'll cover that part quickly, and move to the rest, with a few examples from widely used libraries.

Manager, Helper, Handler...
Good ol' Peter Coad used to say: Challenge any class name that ends in "-er" (e.g. Manager or Controller). If it has no parts, change the name of the class to what each object is managing. If it has parts, put as much work in the parts that the parts know enough to do themselves (that was the "er-er Principle"). That's central to object thinking, because when you need a Manager, it's often a sign that the Managed are just plain old data structures, and that the Manager is the smart procedure doing the real work.

When Peter wrote that (1993), the idea of an Helper class was mostly unheard of. But as more people got into the OOP bandwagon, they started creating larger and larger, uncohesive classes. The proper OO thing to do, of course, is to find the right cooperating, cohesive concepts. The lazy, fake OO thing to do is to take a bunch of methods, move them outside the overblown class X, and group them in XHelper. While doing so, you often have to weaken encapsulation in some way, because XHelper needs a privileged access to X. Ouch. That's just painting an OO picture of classes over old-style coding. Sadly enough, in a wikipedia article that I won't honor with a link, you'll read that "Helper Class is one of the basic programming techniques in object-oriented programming". My hope for humanity is only restored by the fact that the article is an orphan :-).

I'm not going to say much about Controller, because it's so popular today (MVC rulez :-) that it would take forever to clean this mess. Sad sad sad.

Handler, again, is an obvious resurrection of procedural thinking. What is an handler if not a damn procedure? Why do something need to be "handled" in the first place? Oh, I know, you're thinking of events, but even in that case, EventTarget, or even plain Target, is a much better abstraction than EventHandler.

Something-able
Set your time machine to 1995, and witness the (imaginary) conversation between naïve OO Developer #1, who for some reason has been assigned to design the library of the new wonderful-language-to-be, and naïve OO Developer #2, who's pairing with him along the way.
N1: So, I've got this Thread class, and it has a run() function that it's being executed into that thread... you just have to override run()...
N2: So to execute a function in a new thread you have to extend Thread? That's bad design! Remember we can only extend one class!
N1: Right, so I'll use the Strategy Pattern here... move the run() to the strategy, and execute the strategy in the new thread.
N2: That's cool... how do you wanna call the strategy interface?
N1: Let's see... there is only one method... run()... hmmm
N2: Let's call it Runnable then!
N1: Yes! Runnable it is!

And so it began (no, I'm not serious; I don't know how it all began with the -able suffix; I'm making this up). Still, at some point people thought it was fine to look at an interface (or at a class), see if there was some kind of "main method" or "main responsibility" (which is kinda obvious if you only have one) and name the class after that. Which is a very simple way to avoid thinking, but it's hardly a good idea. It's like calling a nail "Hammerable", because you known, that's what you do with a nail, you hammer it. It encourages procedural thinking, and leads to ineffective abstractions.

Let's pair our naïve OO Developer #1 with an Object Thinker and replay the conversation:

N1: So, I've got this Thread class, and it has a run() function that it's being executed into that thread... you just have to override run()...
OT: So to execute a function in a new thread you have to extend Thread? That's bad design! Remember we can only extend one class!
N1: Right, so I'll use the Strategy Pattern here... move the run() to the strategy, and execute the strategy in the new thread.
OT: OK, so what is the real abstraction behind the strategy? Don't just think about the mechanics of the pattern, think of what it really represents...
N1: Hmm, it's something that can be run...
OT: Or executed, or performed independently...
N1: Yes
OT: Like an Activity, with an Execute method, what do you think? [was our OT aware of the UML 0.8 draft? I still have a paper version :-)]
N1: But I don't see the relationship with a thread...
OT: And rightly so! Thread depends on Activity, but Activity is independent of Thread. It just represents something that can be executed. I may even have an ActivitySequence and it would just execute them all, sequentially. We could even add concepts like entering/exiting the Activity...
(rest of the conversation elided – it would point toward a better timeline :-)

Admittedly, some interfaces are hard to name. That's usually a sign that we don't really know what we're doing, and we're just coding our way out of a problem. Still, some others (like Runnable) are improperly named just because of bad habits and conventions. Watch out.


Something-Object
This is similar to the above: when you don't know how to name something, pick some dominant trait and add Object to the end. Again, the problem is that the "dominant trait" is moving us away from the concept of an object as a virtual machine, and toward the object as a procedure. In other cases, Object is dropped in just to avoid more careful thinking about the underlying concept.

Just like the -able suffix, sometimes it's easy to fix, sometimes is not. Let's try something not trivial, where the real concept gets obscured by adopting a bad naming convention. There are quite a few cases in the .NET framework, so I'll pick MarshalByRefObject.

If you don't use .NET, here is what the documentation has to say:
Enables access to objects across application domain boundaries in applications that support remoting
What?? Well, if you go down to the Remarks section, you get a better explanation:
Objects that do not inherit from MarshalByRefObject are implicitly marshal by value. […] The first time [...] a remote application domain accesses a MarshalByRefObject, a proxy is passed to the remote application
Whoa, that's a little better, except that it's "are marshaled by value", not "are marshal by value" but then, again, the name should be MarshaledByRefObject, not MarshalByRefObject. Well, all your base are belong to us :-)

Now, we could just drop the Object part, fix the grammar, and call it MarshaledByReference, which is readable enough. A reasonable alternative could be MarshaledByProxy (Vs. MarshaledByCopy, which would be the default).
Still, we're talking more about implementation than about concepts. It's not that I want my object marshaled by proxy; actually, I don't care about marshaling at all. What I want is to keep a single object identity across appdomains, whereas with copy I would end up with distinct objects. So, a proper one-sentence definition could be:

Preserve object identity when passed between appdomains [by being proxied instead of copied]
or
Guarantees that methods invoked in remote appdomains are served in the original appdomain

Because if you pass such an instance across appdomains, any method call would be served by the original object, in the original appdomain. Hmm, guess what, we already have similar concepts. For instance, we have objects that, after being created in a thread, must have their methods executed only inside that thread. A process can be set to run only on one CPU/core. We can configure a load balancer so that if you land on a specific server first, you'll stay on that server for the rest of your session. We call that concept affinity.

So, a MarshalByRefObject is something with an appdomain affinity. The marshaling thing is just an implementation detail that makes that happen. AppDomainAffine, therefore, would be a more appropriate name. Unusual perhaps, but that's because of the common drift toward the mechanics of things and away from concepts (because the mechanics are usually much easier to get for techies). And yes, it takes more clarity of thoughts to come up with the notion of appdomain affinity than just slapping an Object at the end of an implementation detail. However, clarity of thoughts is exactly what I would expect from framework designers. While we are at it, I could also add that AppDomainAffine should be an attribute, not a concrete class without methods and fields (!) like MarshalByRefObject. Perhaps I'm asking too much from those guys.


ISomething
I think this convention was somewhat concocted during the early COM days, when Hungarian was widely adopted inside Microsoft, and having ugly names was therefore the norm. Somehow, it was the only convention that survived the general clean up from COM to .NET. Pity :-).

Now, the problem is not that you have to type an I in front of things. It's not even that it makes names harder to read. And yes, I'll even concede that after a while, you'll find it useful, because it's easy to spot an interface just by looking at its name (which, of course, is a relevant information when your platform doesn't allow multiple inheritance). The problem is that it's too easy to fall into the trap, and just take a concrete class name, put an I in front of it, and lo and behold!, you got an interface name. Sort of calling a concept IDollar instead of Currency.

Case in point: say that you are looking for an abstraction of a container. It's not just a container, it's a special container. What makes it special is that you can access items by index (a more limited container would only allow sequential access). Well, here is the (imaginary :-) conversation between naïve OO Developer #1, who for unknown reasons has been assigned to the design of the base library for the newfangled language of the largest software vendor on the planet, and himself, because even naïve OO Developer #2 would have made things better:

N1: So I have this class, it's called List... it's pretty cool, because I just made it a generic, it's List<T> now!
N1: Hmm, the other container classes all derive from an interface... with wonderful names like IEnumerable (back to that in a moment). I need an interface for my list too! How do I call it?
N1: IListable is too long (thanks, really :-)))). What about IList? That's cool!
N1: Let me add an XML comment so that we can generate the help file...

IList<T> Interface
Represents a collection of objects that can be individually accessed by index.

So, say that you have another class, let's call it Array. Perhaps a SparseArray too. They both can be accessed by index. So Array IS-A IList, right? C'mon.

Replay the conversation, drop in our Object Thinker:

N1: So I have this class, it's called List... it's pretty cool, because I just made it a generic, it's List<T> now!
N1: Hmm, the other container classes all derive from an interface... with wonderful names like IEnumerable. I need an interface for my list too! How do I call it?
OT: What's special about List? What does it really add to the concept of enumeration? (I'll keep the illusion that "enumeration" is the right name so that N1's mind won't blow away)
N1: Well, the fundamental idea is that you can access items by index... or ask about the index of an element... or remove the element at a given index...
OT: so instead of sequential access you now have random access, as it's usually called in computer science?
N1: Yes...
OT: How about we call it RandomAccessContainer? It reads pretty well, like: a List IS-A RandomAccessContainer, an Array IS-A RandomAccessContainer, etc.
N1: Cool... except... can we put an I in front of it?
OT: Over my dead body.... hmm I mean, OK, but you know, in computer science, a List is usually thought of as a sequential access container, not a random access container. So I'll settle for the I prefix if you change List to something else.
N1: yeah, it used to be called ArrayList in the non-generic version...
OT: kiddo, do you think there was a reason for that?
N1: Oh... (spark of light)


Yes, give me the worst of both worlds!
Of course, given enough time, people will combine those two brilliant ideas, turn off their brain entirely, and create wonderful names like IEnumerable. Most .NET collections implement IEnumerable, or its generic descendant IEnumerable<T>: when a class implements IEnumerable, its instances can be used inside a foreach statement.

Indeed, IEnumerable is a perfect example of how bad naming habits thwart object thinking, and more in general, abstraction. Here is what the official documentation says about IEnumerable<T>:
Exposes the enumerator, which supports a simple iteration over a collection of a specified type.
So, if you subscribe to the idea that the main responsibility gives the -able part, and that the I prefix is mandatory, it's pretty obvious that IEnumerable is the name to choose.

Except that's just wrong. The right abstraction, the one that is completely hidden under the IEnumerable/IEnumerator pair, is the sequential access collection, or even better, a Sequence (a Sequence is more abstract than a Collection, as a Sequence can be calculated and not stored, see yield, continuations, etc). Think about what makes more sense when you read it out loud:

A List is an IEnumerable (what??)
A List is a Sequence (well, all right!)

Now, a Sequence (IEnumerable) in .NET is traversed ("enumerated") through an iterator-like class called an IEnumerator ("iterator" like in the iterator pattern, not like that thing they called iterator in .NET). Simple exercise: what is a better name than IEnumerator for something that can only move forward over a Sequence?.

Is that all you got?
No, that's not enough! Given a little more time, someone is bound to come up with the worst of all worlds! What about an interface with:
an I prefix
an -able suffix
an Object somewhere in between

That's a challenging task, and strictly speaking, they failed it. They had to put -able in between, and Object at the end. But it's a pretty amazing name, fresh from the .NET Framework 4.0: IValidatableObject (I wouldn't be surprised to discover, inside their code, an IValidatableObjectManager to, you know, manage :-> those damn stupid validatable objects; that would really close the circle :-).

We can read the documentation for some hilarious time:
IValidatableObject Interface - Provides a way for an object to be invalidated.

Yes! That's what my objects really want! To be invalidated! C'mon :-)). I'll spare you the imaginary conversation and go straight to the point. Objects don't want to be invalidated. That's procedural thinking: "validation". Objects may be subject to constraints. Yahoo! Constraints. What about a Constraint class (to replace the Validation/Validator stuff, which is so damn procedural). What about a Constrained (or, if you can't help it, IConstrained) interface, to replace IValidatableObject?

Microsoft (and everyone else, for that matter): what about having a few more Object Thinkers in your class library teams? Oh, while we are at it, why don't you guys consider that we may want to check constraints in any given place, not just in the front end? Why not moving all the constraint checking away from UI or Service layers and make the whole stuff available everywhere? It's pretty simple, trust me :-).

Bonus exercise: once you have Constraint and IConstrained, you need to check all those constraints when some event happens (like you receive a message on your service layer). Come up with a better name than ConstraintChecker, that is, something that is not ending in -er.

And miles to go...
There would be so much more to say about programming practices that hinder object thinking. Properties, for instance, are usually misused to expose internal state ("expressed figuratively" or not), instead of being just zero-th methods (henceforth, goals!). Maybe I'll cover that in another post.

An interesting question, for which I don't really have a good answer, is: could some coding convention promote object thinking? Saying "don't use the -er suffix" is not quite the same as saying "do this and that". Are your conventions moving you toward object thinking?

If you read so far, you should follow me on twitter.

Labels: .NET, coding, oop