Do release-blocking bugs have to be new?

A do not enter sign on metal posts at the edge of a cliff overlooking an evergreen forest.

As part of your project’s release process, you may evaluate known bugs to determine if they should block the release. You do this, of course, by looking at your release criteria. If the bug violates a criterion, it blocks the release. Easy peasy.

Of course, real life is rarely that simple. You can determine a seeming-blocker isn’t truly a blocker for a variety of reasons. Maybe it only affects a limited set of hardware. Maybe the user has to do some really funky deviation from the defaults to trigger it. One consideration I see come up in conversations is of the “well, this bug existed in the previous release, too.” Should you consider those pre-existing bugs release blockers?

The case for “yes”

If a bug violates a release criterion, then it violates a release criterion. You have release criteria for a reason: you want to ensure a certain level of functionality in the software. Just because the bug is old, that doesn’t mean the loss of functionality ceases to exist. You wouldn’t accept “well we went ahead and release this year’s car model because the ‘the battery might spontaneously explode as you drive down the road’ issue was present in the previous model year” from a car manufacturer.

The case for “no”

If a bug exists in software and no one files it, does it make an impact? More to the point: if the behavior has existed for multiple releases and you’ve just now caught it, does it actually matter? Either it’s so minor that no one ever bothered to report it or it’s such an edge case that it’s never been triggered. Since the point of release criteria is to define a minimum level of functionality, not fix every bug, it seems reasonable to exclude a bug on this basis.

Which to pick?

As usual, there’s no straightforward answer. Both arguments have merit. You probably don’t want to make a blanket decision to always accept pre-existing bugs as blockers, and you definitely don’t want to always reject pre-existing bugs. In general, I favor ignoring the age of the misbehavior. After all, you may have missed it before, but you caught it now; you might as well fix it. But in the case of minor impacts, the fact that no one has caught it before means you can either ignore it — or modify the criteria.

You can read more about release criteria in chapter 11 of Program Management for Open Source Projects.

This post’s featured photo by Dim Hou on Unsplash.

Ben works on open source strategy at Docker and was previously the Fedora Program Manager. He is the author of Program Management for Open Source Projects. Ben is an Open Organization Ambassador and frequent conference speaker. His personal website is Funnel Fiasco.

Share

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.