Zero Defect Policy
The 4th criteria of the Joel Test of quality on a software team is:
- Do you have a bug database?
You probably do. The test was written almost 20 years ago and if I recall correctly almost everyone had one anyway. Joel was even selling one.
On the surface it seems like common sense. Quality is something you want to manage. So naturally you’ll also tend to want to log things… measure things… track things.
I want to propose a better way for most situations though. It’s simpler, your long-term speed will be faster, and your product quality will be higher:
The Zero Defect Policy
Simply put: Prioritize every defect above feature work or close the issue as a WONTFIX. I’m not suggesting you interrupt any work already in progress, but once that’s done, burn your defect list down to 0.
Why in the world?
Bugs are much cheaper to fix immediately. You know the product requirements most clearly at that time. You were touching that code most recently at that time and it’s still fresh in your mind. You might even remember a related commit that could have been the cause. The person who wrote it is still on the team.
Most things we’d classify as bugs are really some of the most obvious improvements you could make. You probably don’t need to a/b test a typo fix. If your android app is crashing on the latest Samsung phone, you don’t need a focus group.
Managing bugs is a huge effort. If you’re not going to immediately fix them, you have to tag them, categorize them, prioritize them, deduplicate them, work around them, do preliminary investigations on them, revisit them, and have meetings about them.
The development team needs the proper immediate feedback and back-pressure to know when to speed up and when to slow down.
Say What now?
Defects are perfectly normal. If you don’t have them, you’re either NASA or you’re developing too slowly. However, if you have too many, or if they’re particularly costly, you absolutely need to slow down and turn them into learning opportunities. In those times, the fix isn’t enough. The fix with an automated test isn’t even enough. You’ll want to look into: * other prevention measures * faster detection measures * faster ways to fix * better ways to limit impact
This is the REAL definition of Quality Assurance. If you do this thoughtfully, and don’t try to make an army of manual testers the best solution you could come up with, over the long term you’ll be much much faster. You’ll be the kind of product development team that the company actually believes when you say something is “done”.
What about the low value, high cost bugs?
Delete them. If they become worthwhile later, you’ll hear about them again. If you can’t bring yourself to delete it, you probably value it too much to not fix it. Just fix it.
What about when my team ends up with a week’s worth of defects and can’t get any feature work through?
There will definitely be dark times. Slow down and learn from them. It’s the best time to talk about speed improvements, because speed and quality are interdependent. You can move much faster in mistake-proof environments. In most cases, nobody cares how fast you ship broken software.
Sounds great, but what about our existing bug database of hundreds of bugs?
Many are probably not even bugs anymore if you’ve been collecting long enough to get hundreds. How many of those old things can you even reproduce? How many have clear enough descriptions that you can even still understand the problem? Is the original submitter still around? Here’s my solution: delete all but the highest priority ones and immediately schedule the high priority ones above other features. Allow people to file reports again if they really care about a particular defect and the defect still exists.
The bug database is great in theory, but in practice, it’s often an aging garbage heap of ignored customer frustrations and a /dev/null for opportunities for improvement.