Perceptions of Quality

Quite some ago Intergen's Development Steering Group (DSG, a small group of senior developers/architects) went on a 'retreat' to discuss issues affected the development team as a whole and plan some strategy. Among the items up for discussion was the issue of quality - how to achieve it, and, in order to decide that, what on earth quality actually is.

You might be thinking: "That's easy. Quality is making something good" which is all very well until you are asked to define what makes a piece of software 'good'. Is it that the code is well written, well documented, that the design is clean, tidy and understandable, that the GUI looks pretty, that the GUI is easy to use, that the system meets user requirements? All of the above?

What the DSG guys did was to take a list of eleven fairly well-accepted aspects of software quality and rank them - not at all an easy task but one that generated lots of discussion. Ranking was done by a combination of a client's view of quality and a developer's view. While the client view tended to trump all (we are a consulting organisation after all) considerations such as the ease of updates/fixes also came into the equation. Here, in alphabetical order, are the aspects they tried to rank:

• Conciseness
• Consistency
• Efficiency
  
• Maintainability
  
• Portability
• Reliability
• Security
  
• Structuredness
  
• Testability
  
• Understandability
• Usability

Before I tell you how they ranked them, let me add another interesting piece to the puzzle. After the DSG retreat, the same list was given to a group of less senior team leads (which is where I come in) and we ranked them ourselves before we saw what the DSG had decided. The resulting rankings were similar, but not entirely in agreement.

The DSG guys are more experienced, but generally far less hands-on than the Team Lead group. The Team Lead group is also generally younger and less experienced; and therefore completed their training more recently. DSGers usually work at a high level on many projects at once, providing guidance and support; while Team Leads usually work in-depth on one project at a time, dealing with the day to day issues of the team.

So... without any further suspense, here are the results:

			DSG		Team Leads
1			Security		Security
2			Reliability		Reliability
3			Usability		Usability
4			Portability		Testability
5			Understandability		Maintainability
6			Efficiency		Portability
7			Maintainability		Structuredness
8			Testability		Understandability
9			Structuredness		Consistency
10			Consistency		Conciseness
11			Conciseness		Efficiency

We obviously all agree that Security, Reliability and Usability come out on top without too much bother. In our discussion we had next to no difficulty with those three being at the top, in that order. It was a big jump down in certainty to number four - and that's where we start to see differences.

The Portability/Testability disagreement at that point is one we spent some time discussing. We thought it related mostly to a difference in perspective. The DSG see a variety of projects and are usually called in when something goes majorly wrong. Since deployment can easily go wrong, and is urgent when it does, I wouldn't be surprised if those guys see a lot of deployment issues. Also, when looking at a high level timeline of a project (or an iteration), deployment stands out as it's own phase so its importance is highlighted: Plan, Build, Deploy. Team Leads, on the other hand, tend to spend most of their time trying to get things built and resolving issues. Since Testability makes getting something built and stable enormously easier, its importance is clear.

I can't help but wonder if there's a little bit of a generational difference in there too - Test Driven Design being still a recent-ish concept, the Team Leads may have formally studied unit testing at least a little (I know it was part of my university courses, at least) while the DSG are less likely to have done so. Just guessing here.

I find the difference in where we place Efficiency interesting too: sixth for the DSG, simply last for the Team Leads. Why? I know efficiency certainly seems important when looked at on the surface, so placing it last, at first glance seems strange. However, when we (the Team Leads) were discussing our rankings, each time we compared Efficiency to one of the other aspects we came to the conclusion that Efficiency was something that could be added in later. Perhaps even added in relatively easily if everything else had been done right - reliable, testable, understandable, well structured, consistent code being easier to optimise. On the other hand, if you got the rest wrong, well, good luck getting it to be efficient! And would it still be efficient if it ran fast but it took six months to make a change? I don't know what discussions took place for the DSG, but that was our train of thought at least.

One last point I'll pull out - both groups ranked Consistency and Conciseness low (and both in that order). Obviously we all love consistent and concise code, and I hope we all strive to write it, but, at the end of the day, the other aspects did seem more important. If the code is structured well (so you can find things), and understandable, and maintainable (you can make changes in one part with breaking the others), and there are good unit tests... does it really matter if the naming or style is not entirely consistent or if some things are rather verbose? It would be annoying I'll grant you, but not critical.

To a certain extent the discussion of rankings is academic - all eleven aspects are really important and we should be trying to ensure our projects have all bases covered. However, pulling out what we believe to be most important can help us to focus and to ensure that we don't get caught up in something relatively minor and forget what really matters. I also think it's a great discussion-starter to get people talking about quality (which we all should be) and also about the different perspectives we can have that cause us to value different things for different reasons.

What do you think?