When to add QA to your project
Chris Vannoy wrote an insightful post titled “Save QA for when the fall is deadly” in which he argues against prematurely adding a dedicated QA role. He argues that giving developers direct ownership over their work quality leads to higher quality at a lower cost. I agree with him about waiting for QA to test software slows the process down. Where I disagree is in the implicit assumption that developers are as good at testing software as they are at writing it. Good QA engineers are magicians in their ability to find issues in software before it ships to the customer. And Chris, to his credit, never says “you don’t need QA ever,” which got me to think about when an open source project might need to add a QA team.
The bar Chris sets for companies makes sense: don’t add the QA until the expense is less than the cost of not having QA. But open source projects are different. For example, no one should ever hire me to be a developer, but I’ve contributed code to a variety of open source projects and written my own as well. I understand the concepts behind unit tests and integration tests, but I have basically no experience writing or running them. My “testing” is essentially “what are a few ways that I can think of where this might break. Does it break in those situations? No? Ship it!” For me to ship quality code, it has to be trivial or lucky. QA would help everyone feel a lot more comfortable running my code.
But QA is more than just running tests. QA engineers design tests. They come up with scenarios that can break software. They build and run infrastructure for managing those tests. Plus, by not being the author of the code, QA can bring fresh eyes and see things that the developer wouldn’t spot.
So to get back to the question at hand: when do you add QA to your open source project? The easy answer is “whenever someone shows up wanting to do QA work.” Open source projects run on the idea of “the work that gets done is the work that people show up to do.” If someone wants to help with testing, why turn them away?
When should you actively recruit and build QA roles into your project? When the amount of time you spend fixing user-reported bugs becomes overwhelming. QA will help you find — and therefore fix — the bugs before you release them. Working together, you can make your software better.
This post’s featured photo by Neringa Hünnefeld on Unsplash.
