Quality Assurance: The Comfort Syndrome


I sent out a new build of a web application this morning and just before branching the source tree was told that there was another bug in the site. The bug was minor, but that is not the point. We have been in QA for a couple of weeks and it seems every time we think we have something completed and are ready to call it a gold build, something else comes along.
 
This is not a unique problem. In fact, it is quite common. In organizations like Microsoft, the reduction of bugs to zero is not the release point. They, instead, wait for the "zero bug bounce". Shortly after the number of bugs goes to zero, more are found. Why?
 
My theory is the car theory. Remember the last time you bought a car. Prior to buying it, it seemed like the type of car you wanted was rare. Then, you buy one and they are everywhere. Everyone suddenly rushed out and bought the car you wanted to buy. Or, more likely, you were being myopic and not seeing what was plainly in front of you.
 
THe same is true for QA testing. When a certain type of bug is found, every single page (website) or form (windows app) is checked for that type of bug, and rechecked, ad nauseum. But, at the same time, other types of bugs are missed. Myopia strikes. After all of one type of bug are wiped out, the fog lifts and other types of bugs are found. And the cycle repeats until you either wipe out enough bugs to have a quality product (not perfect, but perhaps excellent) or you run past the quality of your QA staff.
 
The more experienced your QA staff, the more likely you will limit the number of cycles, as experience helps cure some of the myopia. But, it has been a fact everywhere I have worked, that more bugs will be found after you think you have every bug wiped out.
 
A few suggestions for fewer bugs and higher quality:
 
1. Have written standards, including User Interface checks, constraints and validations.
    a) If the field in the database is 100 in length, make sure they cannot go past 100
    b) If the field is a date field, ensure the textbox is a date
2. Work with testing frameworks.
     a) nUnit and Team Test are both good for the unit testing
     b) Use mocks on true unit tests
     c) Fit makes a nice acceptance framework
3. Aim for 100% coverage on tests – there are cases where you will not achieve this (serialization of custom exception objects comes to mind), but make the attempt. It pays off
4. Write a confirmatory test for EVERY bug you find
 
Until next time
— GB
Advertisements

One Response to Quality Assurance: The Comfort Syndrome

  1. Kevin says:

    Another problem occurs when, in the process of fixing bugs, new bugs emerge. This is particularly true of complex applications. It can be very difficult to deal with, from personal experience.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: