Thursday, February 03, 2011

Is Software Design Literature Dead?

Sometimes, my clients ask me what to read about software design. Most often than not, they don't want a list of books – many already have the classics covered. They would like to read papers, perhaps recent works, to further advance their understanding of software design, or perhaps something inspirational, to get new ideas and concepts. I have to confess I'm often at loss for suggestions.

The sense of discomfort gets even worse when they ask me what happened to the kind of publications they used to read in the late '90s. Stuff like the C++ Report, the Journal of Object Oriented Programming, Java Report, Object Expert, etc. Most don't even know about the Journal of Object Technology, but honestly, the JOT has taken a strong academic slant lately, and I'm not sure they would find it all that interesting. I usually suggest they keep current on design patterns: for instance, the complete EuroPLoP 2009 proceedings are available online, for free. However, mentioning patterns sometimes furthers just another question: what happened to the pattern movement? Keep that conversation going for a while, and you get the final question: so, is software design literature dead?

Is it?
Note that the question is not about software design - it's about software design literature. Interestingly, Martin Fowler wrote about the other side of the story (Is Design Dead?) back in 2004. He argued that design wasn't dead, but its nature had changed (I could argue that if you change the nature of something, then it's perhaps inappropriate to keep using the same name :-), but ok). So perhaps software design literature isn't dead either, and has just changed nature: after all, looking for "software design" on Google yields 3,240,000 results.
Trying to classify what I usually find under the "software design" chapter, I came up with this list:

- Literature on software design philosophy, hopefully with some practical applications. Early works on Information Hiding were as much about a philosophy of software design as about practical ways to structure our software. There were a lot of philosophical papers on OOP and AOP as well. Usually, you get this kind of literature when some new idea is being proposed (like my Physics of Software :-). It's natural to see less and less philosophy in a maturing discipline, so perhaps a dearth of philosophical literature is not a bad sign.

- Literature on notations, like UML or SysML. This is not really software design literature, unless the rationale for the notation is discussed.

- Academic literature on metrics and the like. This would be interesting if those metrics addressed real design questions, but in practice, everything is always observed through the rather narrow perspective of correlation with defects or cost or something appealing for manager$. In most cases, this literature is not really about software design, and is definitely not coming from people with a strong software design background (of course, they would disagree on that :-)

- Literature on software design principles and heuristics, or refactoring techniques. We still see some of this, but is mostly a rehashing of the same old stuff from the late '90s. The SOLID principles, the Law of Demeter, etc. In most cases, papers say nothing new, are based on toy problems, and are here just because many programmers won't read something that has been published more than a few months ago. If this is what's keeping software design literature alive, let's pull the plug.

- Literature on methods, like Design by Contract, TDD, Domain-Driven Design, etc. Here you find the occasional must-read work (usually a book from those who actually invented the approach), but just like philosophy, you see less and less of this literature in a maturing discipline. Then you find tons of advocacy on methods (or against methods), which would be more interesting if they involved real experiments (like the ones performed by Simula labs) and not just another toy projects in the capable hands of graduate students. Besides, experiments on design methods should be evaluated [mostly] by cost of change in the next few months/years, not exclusively by design principles. Advocacy may seem to keep literature alive, but it's just noise.

- Literature on platform-specific architectures. There is no dearth of that. From early EJB treatises to JBoss tutorials, you can find tons of papers on "enterprise architecture". Even Microsoft has some architectural literature on application blocks and stuff like that. Honestly, in most cases it looks more like Markitecture (marketing architecture as defined by Hohmann), promoting canned solutions and proposing that you adapt your problem to the architecture, which is sort of the opposite of real software design. The best works usually fall under the "design patterns" chapter (see below).

- Literature for architecture astronauts. This is a nice venue for both academics and vendors. You usually see heavily layered architectures using any possible standards and still proposing a few more, with all the right (and wrong) acronyms inside, and after a while you learn to turn quickly to the next page. It's not unusual to find papers appealing to money-saving managers who don't "get" software, proposing yet another combination of blueprint architectures and MDA tools so that you can "generate all your code" with a mouse click. Yeah, sure, whatever.

- Literature on design patterns. Most likely, this is what's keeping software design literature alive. A well-written pattern is about a real problem, the forces shaping the context, an effective solution, its consequences, etc. This is what software design is about, in practice. On the other hand, a couple of conferences every year can't keep design literature in perfect health.

- Literature on design in the context of agility (mostly about TDD). There is a lot of this on the net. Unfortunately, it's mostly about trivial problems, and even more unfortunately, there is rarely any discussion about the quality of the design itself (as if having tests was enough to declare the design "good"). The largest issue here is that it's basically impossible to talk about the design of anything non-trivial strictly from a code-based perspective. Note: I'm not saying that you can't design using only code as your material. I'm saying that when the problem scales slightly beyond the toy level, the amount of code that you would have to write and show to talk about design and design alternatives grows beyond the manageable. So, while code-centric practices are not killing design, they are nailing the coffin on design literature.

Geez, I am usually an optimist :-)), but this picture is bleak. I could almost paraphrase Richard Gabriel (of "Objects have failed" fame) and say that evidently, software design has failed to speak the truth, and therefore, software design narrative (literature) is dying. But that would not be me. I'm more like the opportunity guy. Perhaps we just need a different kind of literature on software design.

Something's missing
If you walk through the aisles of a large bookstore, and go to the "design & architecture" shelves, you'll all the literary kinds above, not about software, but about real-world stuff (from chairs to buildings). except on the design of physical objects too. Interestingly, you can also find a few morelike:

- Anthologies (presenting the work of several designers) and monographs on the work of famous designers. We are at loss here, because software design is not immediately visible, is not interesting for the general public, is often kept as a trade secret, etc.
I know only one attempt to come up with something similar for software: "Beautiful Architecture" by Diomidis Spinellis. It's a nice book, but it suffers from a lack of depth. I enjoyed reading it, but didn't come back with a new perspective on something, which is what I would be looking for in this kind of literature.
It would be interesting, for instance, to read more about the architecture of successful open-source projects, not from a fanboy perspective but through the eyes of experienced software designers. Any takers?

- Idea books. These may look like anthologies, but the focus is different. While an anthology is often associated with a discussion or critics of the designer's style, an idea book presents several different objects, often out of context, as a sort of creative stimulus. I don't know of anything similar for software design, though I've seen many similar book for "web design" (basically fancy web pages). In a sense, some literature on patterns comes close to being inspirational; at least, good domain-specific patterns sometimes do. But an idea book (or idea paper) would look different.

I guess anthologies and monographs are at odd with the software culture at large, with its focus on the whizz-bang technology of the day, little interest for the past, and often bordering on religious fervor about some products. But idea books (or most likely idea papers) could work, somehow.

Indeed, I would like to see more software design literature organized as follows:

- A real-world, or at least realistic problem is presented.

- Forces are discussed. Ideally, real-world forces.

- Possibly, a subset of the whole problem is selected. Real-world problems are too big to be dissected in a paper (or two, or three). You can't discuss the detailed design of a business system in a paper (lest you appeal only to architecture astronauts). You could, however, discuss a selected, challenging portion. Ideally, the problem (or the subset) or perhaps just the approach / solution, should be of some interest even outside the specific domain. Just because your problem arose in a deeply embedded environment doesn't mean I can't learn something useful for an enterprise web application (assuming I'm open minded, of course :-). Idea books / papers should trigger some lateral thinking on the reader, therefore unusual, provocative solutions would be great (unlike literature on patterns, where you are expected to report on well-known, sort of "traditional" solutions).

- Practical solutions are presented and scrutinized. I don't really care if you use code, UML, words, gestures, whatever. Still, I want some depth of scrutiny. I want to see more than one option discussed. I don't need working code. I'm probably working on a different language or platform, a different domain, with different constraints. I want fresh design ideas and new perspectives.

- In practice, a good design aims at keeping the cost of change low. This is why a real-world problem is preferable. Talking about the most likely changes and how they could be addressed in different scenarios beats babbling about tests and SOLID and the like. However, "most likely changes" is meaningless if the problem is not real.

Funny enough, there would be no natural place to publish something like that, except your own personal page. Sure, maybe JOT, maybe not. Maybe IEEE Software, maybe not. Maybe some conference, maybe not. But we don't have a software design journal with a large readers pool. This is part of the problem, of course, but also a consequence. Wrong feedback loop :-).

Interestingly, I proposed something similar years ago, when I was part of the editorial board of a software magazine. It never made a dent. I remember that a well-known author argued against the idea, on the basis that people were not interested in all this talking about ins-and-outs, design alternatives and stuff. They would rather have a single, comprehensive design presented that they could immediately use, preferably with some source code. Well, that's entirely possible; indeed, I don't really know how many software practitioners would be interested in this kind of literature. Sure, I can bet a few of you guys would be, but overall, perhaps just a small minority is looking for inspiration, and most are just looking for canned solutions.

Or maybe something else...
On the other hand, perhaps this style is just too stuck in the '90s to be appealing in 2011. It's unidirectional (author to readers), it's not "social", it's not fun. Maybe a design challenge would be more appealing. Or perhaps the media are obsolete, and we should move away from text and graphics and toward (e.g.) videos. I actually tried to watch some code kata videos, but the guys thought they were like this, but to an experienced designer they looked more like this. (and not even that funny). Maybe the next generation of software designers will come up with a better narrative style or media.

Do something!
I'm usually the "let's do something" kind of guy, and I've entertained the idea of starting a Software Design Gallery, or a Software Design Idea Book, or something, but honestly, it's a damn lot of work, and I'm a bit skeptical about the market size (even for a free book/paper/website/whatever). As usual, any feedback is welcome, and any action on your part even more :-)


michelangelo said...

Concordo completamente con quanto hai scritto. Mi trovo spesso ad “invidiare” gli architetti per la facilità con cui accedono ai progetti dei colleghi: semplicemente sfogliando una rivista o un libro del settore e guardando foto e schizzi. Tra l’altro in questa attività il senso estetico non solo spesso ne gode ma viene “alimentato” quasi senza sforzo!

Questo almeno per un accesso superficiale. Ma anche andare in profondità per vedere le motivazioni di ogni decisione e' altrettanto facile.

Detto questo ti dico che sarei quindi disposto a pagare un biglietto salatissimo per accedere ad una tua eventuale “Software Design Gallery”! ;-) L’idea poi di un’antologia o di un Idea Book e’ bellissima. La carenza di opere simili in giro non credo sia dovuta ad una questione di tecnologia di “presentazione”. Credo invece che occorrano due skill insieme: capire a fondo il software (e quello di qualcun altro) e saperlo “raccontare”. Due skill forse difficili da trovare insieme.

Ma ti chiedo, secondo te ci sono o ci sono stati progettisti nel campo del software paragonabili ai grandi maestri dell’architettura? un Le Corbusier, un Wright, un Gropius, un Gaudi, ecc. solo per citare i moderni? e che per giunta ci abbiano lasciato il loro codice aperto? Insomma detto in altri termini, nella tua antologia chi metteresti?

C’e’ infine un’ultima questione. Probabilmente nel nostro campo c’e’ poco l’allenamento a cercare o vedere le “ragioni” dietro ogni scelta. Anche nel campo dell’architettura c’e’ questa tendenza, forse anche più marcata perché si sconfina più facilmente nel campo del gusto. Ci sono le eccezioni ovviamente. Conosco architetti che non approvano una soluzione se “e' bella” ma se non c’e’ una vera ragione pratica che la giustifichi.

Romano Scuri said...

Questa volta mi sono preso la briga di leggerlo tutto.

Caspita, fa molto "crisi di mezza età".

Dopo anni di TV commerciale, ci domandiamo come mai la gente ragiona in un certo modo?

Dopo anni di strumenti di sviluppo visuale ci domandiamo come mai scompare l'analisi?

Ed anche tu mi sembri come lo slogan di quella maglietta: Dio è morto, Marx è morto e neanch'io mi sento tanto bene...

Ma le crisi sono spesso salutari.

Carlo.Pescio said...

Michelangelo: che tu fossi ben disposto verso iniziative di questo tipo lo davo praticamente per scontato :-), il problema appunto e' se esiste un numero sufficiente di persone interessate. Tanto per fare un esempio, a grandi linee il post ha avuto sino ad oggi 1000 visite, in larga misura grazie ad una brava persona :-) che spesso mette i miei post su dzone. Ha avuto *2* retweet, i conti si fanno in fretta :-) (inclusa la possibilita' che tutti e 1000 fossero blog-free, twitter-free, facebook-free, ecc, il che ha altre interessanti implicazioni :-).

Sicuramente ci sono stati grandi progettisti, semplicemente non lo sapremo mai, o quasi. Ripensandoci, Grady Booch aveva iniziato un lavoro di catalogazione, ma spesso non andava oltre il recupero di un diagramma di livello cosi' alto da non dire granche', e mancava un esame approfondito. Rimangono alcuni lavori sul language design, da cui si puo' capire molto su come ragionava il progettista. Sono lavori di nicchia ma comunque interessanti.

Purtroppo, proprio per la separazione forma-funzione, che come dice Gabriel e' piu' forte nel sw che in altri artefatti, anche il semplice fatto che un sw "funzioni bene" non significa che la sua architettura interna sia meritevole. Il lavoro e' improbo, insomma, e in mancanza di un target ampio vedo difficile che si muova qualcosa. Gli idea book / idea paper sarebbero probabilmente piu' semplici da realizzare, e comunque non meno interessanti...

Carlo.Pescio said...

Romano: come dicevo io sono piu' il tipo "ok, facciamo qualcosa". In pratica poi mi trovo ad avere un argomento long-term a cui vorrei dedicare un po' di energia (quella physics of software che non ti piace molto :-), un altro argomento long-term del tutto diverso (una teoria sul sw project management che purtroppo ho parcheggiato perche' mi manca il tempo di svilupparla), diverse bozze di idea paper sul design che non vedranno la luce perche' c'e' un tot di lavoro dietro e se poi le leggono in 4 un po' mi dispiace :-).

Non mi vedo particolarmente "in crisi". Vedo una comunita' "in crisi", questo si. Vedo che si sta perdendo una cultura ed una sensibilita, che viene trivializzata nel rispetto di 4-5 principi e nell'anteposizione delle tecniche ai risultati.
Purtroppo vedere la cosa come un *mio* problema, mentre in realta' e' un *nostro* problema, non porta a quello che speravo, ovvero che qualcuno iniziasse a rimboccarsi le maniche, e non per elargirmi perle di saggezza :-) ma per iniziare a scrivere qualche idea paper...

Romano Scuri said...


credo che tu mi stia sopravvalutando. E poi per le perle (del pirla :-) è giusto quanto mi riesce di fare...

Continuo a definirmi un artigiano del software, ormai in via d'estinzione (io, non la categoria).

Michele Arpaia said...

just a bit OT: Have you read "The design of design" (Fred Brooks)? Thoughts? Got it shipped but didn't have time to open it just yet.

Also, from a different perspective (i.e. the societal impact of IT) - I started a series of posts on some recommended readings. It is turning out to be a very basic (and biased :-) catalog of books that I've been reading over the years. Have a look.

Carlo.Pescio said...

Michele: Brooks' book is quite interesting, and a recommended reading for software designers. It leans on the "philosophy" side quite a bit, but Fred's experience makes it extremely valuable.
Some sections (like the one on design rationale) are a bit simplistic/outdated. The case studies at the end of the book are almost like an anthology, too bad only 2 out of 7 are actually about software :-). Anyway, a nice read!

Erich Ocean said...

Physically Based Rendering by Matt Phar is pretty interesting, and presents the full design (including rational) for a complete 3D renderer using the ray tracing algorithm.

At least in that small subset of software, it's had a huge influence.