Understanding Quality

Quality can be a tough concept to get a handle on, especially when you are dealing with a web site. Many different roles and positions take part In running a typical large commerce web site, and they all have different agendas, so what they mean by "quality" is likely to differ.

Working from a quality assurance outlook, you must be clear on what you understand quality to mean. You may need to declare or defend your views on the quality of your site to people who see quality differently.

The following definitions reflect different ways of looking at what quality might mean. I find the last definition – that quality is meeting requirements – the most useful approach here.

You Know Quality When You See It

You probably know of some things – products, services, etc. – that you feel are excellent, because they meet your needs, do what they are supposed to do, make your life easier, taste great, whatever. These things are identified, by you, as being very positive, and if you stop to think about why they are positive, the word "quality" probably comes to mind. You fire up your brand new computer, and it won't boot, you're probably not thinking to yourself, "hmmm, that's a damned fine piece of machinery I have here." You inherit your grandfather's pocketknife that's fifty years old and still holds a fine edge, you're probably thinking "that's quality".

This view of quality lacks any concrete way of measuring the quality of something, because it is based on a specific person's judgment. If you build a web site, and you think it is a quality site because you know quality when you see it, you have no assurance that anybody else will see the site the same way. When you build a site for an audience consisting of more than just you, you cannot rely on this definition of quality. People judge quality on many factors; most of these factors are idiosyncratic, and many of those are probably being used without the person ever being aware of it. Furthermore, advertising shapes perceptions and expectations to the extent that people may be taught the wrong signals of quality – these paper towels are decorated with detailed drawings native wildflowers.

This definition doesn't address issues like the suitability of the site for its purpose; for example, if your site provides information on a topic, doesn't its ability to perform this duty count for the quality of the site? It also doesn't address the service to the audience: if you create the site to meet the needs of a particular audience, doesn't success in meeting these needs affect quality?

Quality is a Function of Brand

Brand names can be a sign of quality, either as a result of personal experience with the brand or from judgments driven by advertising. Most people have faith in some brands or companies: "The best truck is a Ford truck, and the best cookie is an Oreo cookie." A brand is a great way to set expectations, so for example you can go to any McDonald's restaurant in the United States and get a Big Mac that's going to taste just about exactly the same; look for those Golden Arches, and you know with certainty the kind of experience you will receive.

The problem with this definition of quality is that you may have a hard time understanding just what about the brand marks quality. If you don't love Oreo cookies, you will be hard-pressed to explain just what it is about them that makes Oreos such a good cookie…if it is a good cookie, in your view. And if you disagree with the quality signaled by a specific brand, perhaps because of bad experiences with the product or brand, then that brand sure won't signal quality to you.

If your web site acquires branded data from a data vendor, you cannot simply rely on the brand's quality and skip testing the data. Brands may be very important to typical consumers – and so are probably important to your site's customer – you can never rely on a brand to establish the quality of what you serve to your customers.

Quality is a Passing Grade

Perhaps the most common understanding of quality is that quality corresponds to a passing grade. If you've ever bought a piece of clothing that had one of those little paper tags saying that this piece had passed inspector something-or-other, you know that your clothes were tested and received a passing grade. If you've every taken home a school report that had a smileyface sticker or maybe some gold stars, you know the utter pride of showing that quality report to your parents.

Your experience with class grades are similar. You spend a semester studying a topic, attending lectures, doing your reading, turning in papers, and taking tests, and your understanding of the topic is evaluated by the teacher. Your teacher assigns you a grade, and this grades is used to help determine whether you pass the class and move ahead.

This definition has several problems, though, when it comes to understanding the meaning of quality. First, you can't always be certain about what exactly the test – and the passing grade – actually measure. With the class example, the teacher could be measuring your comprehension of the class's topic OR your ability to master the class itself; haven't you ever taken a class where somebody who doesn't understand the material passes because they meet the minimum work requirement?

Second, the teacher is often using his/her subjective views to set the milestones measured by the tests, so the tests are not necessarily objective evaluations.

Third, this view doesn't help you understand the differences in grades that pass. Say you have two students who pass a class, one of whom gets a grade of "A", the other gets a "C". Both pass the class, but one apparently did better than the other. If quality is passing this class, is there any difference between the performance and understanding of the C student and that of the A student? From a testing point of view, passing a test would seem to imply quality, but you need to know a lot more about the big picture to get any value from a passing grade.

Quality is Perfection

Perfection is good; actually, perfection is better than good, it's the best. If something is the best, then it must be overflowing with quality, right?

But who decides that something is perfect, and who decides what perfection means? Who or what is it perfect for? The problem with this definition is that it tells you where you want to go, but not how to get there.

Quality is the Absence of Problems

Something doesn't cause me any problems, and I have no complaints about it, it doesn't get in the way of work or play, it's always held up, never breaks, never dies, it's "Old Reliable" – that's a sure sign of quality, right?

Saying that quality is the absence of problems doesn't go far enough, because it doesn't address the big picture around problems: problems for whom? Take something like a software program – you can expect different people to be different kinds of users, with different needs that involve using the software with different goals in mind, and probably using different tools within the software. Say the software has a function that most people will never need to use, like the old "convert the file to Sanskrit" command; if that function does not work, but most users won't come across that failure, does the problem nonetheless exist?

Quality is Zero Defect Code

Quality is code that has no bugs – that's a great goal to have, but it's usually impractical. Most software, and your website is a kind of software product, must be developed and released quickly to meet marketplace demands. Covering every line of code is time consuming, and at some point the development team will have to decide the benefits of releasing versus continuing to tweak the code.

Quality is Acceptable Performance

You have something that overall does what it's supposed to do, and failures are within acceptable limits. The majority of students pass the class, the spread of grades looks like it's supposed to. It may not be perfect, but it's certainly good enough. Thinking of quality as being acceptable performance is a qualitatively different definition than the previous ones, because you are looking at what something does, rather than what it is. You may know quality when you see it, but that's a hard thing to share among multiple points of view short of persuading other people to agree to your understanding of quality, and using persuasion to shape others' perception of quality seems beside the point.

When you look at what something does, you have performance, which is measurable and trackable. Instead of saying that something is good because you like it, or because it's perfect, or because it doesn't have any problems, you can say something is good because it does X, Y and Z, and it does them like this.

Quality is Meeting Goals

This definition also describes quality as reflecting what something does. You set goals for something, say a web site, and if the web site meets these goals then it has quality. But is there a scale, so that there is an amount or level of quality that is a function of how much or how well the goals are met? If the goals are just not meet-able – every person who sees our ad will buy the product – then does the site lack quality? Or what if the site performs superbly except when trying to meet one goal, for example, say your new commerce site is immensely popular, garnering millions of new users each day, racking up the "cool site" awards, even growing a large and active community of users, but people just are not buying your products. A big goal, if not the biggest goal, for your site is that it sells product, but the site fails to meet this goal; is this a quality site?

Quality Is Meeting Requirements

If you define quality as meeting requirements, then you have specific indicators of quality. If the requirements are testable, then you can test the success of meeting the requirements. And you can verify the quality repeatedly, by testing at intervals. You can say that whatever you're testing is good, and that it was good the last time you ran the tests, and the time before that, etc. You can measure quality, and measure it over time.

Requirements can describe the attributes of something, such as the extension and simple characteristics, as well as how it should function, behave, perform, respond, etc. These requirements can be determined in many different ways, but usually the requirements are logically derived from the purpose or function of the subject and from the design intentions: if a television set is designed to receive certain frequency ranges, then clearly part of the television's measure of quality is its ability to correctly fulfill this requirement.

Individual quality requirements can take different forms:

• attributes – the subject is supposed to have measurements x, y and z; it's supposed to be
• does the subject meet the specified measurements? Does the subject have all of its parts? Is the subject the correct color?
• functionality – does the subject do what it's supposed to?