Combinatorial releases won’t help

A gold combination lock and gold credit cards lying on a white laptop keyboard.

Kurt Seifried recently wondered why we produce software in a way that “would be pretty much immediately recognizable to Gutenberg as the process and workflow for releasing a printed piece of paper.” Specifically, he asks

What if for example a project released multiple versions of the software with slightly different patch sets or commits or an end user ran the software and pulled in different patch sets and so on and had some sort of feedback mechanism the people could choose to participate in to figure out which one was better more stable had less flaws and so on?

Kurt Seifried

The pat answer is “because nobody wants what that would mean in practice.” From the end user perspective, people don’t want to fill out a character sheet or assemble a kit just to use software. They just want to use the damn software. With some software delivery models (particularly software-as-a-service), it’s certainly possible to offer slight variations to different users. This works well in A-B testing to see if a change has a desired impact. It’s less useful when it’s A-B-C-D-E-F… testing because the number of possible combinations makes it hard to keep track of what actually works better.

This leads us to the maintainer’s perspective. For maintainers, such a combinatorial approach to software releases is a maintenance nightmare. It’s difficult to get useful bug reports from end users, reproduce the issue, and develop a fix. It becomes even harder when the user doesn’t know what combination they’re using. If they do, the maintainer still has to replicate that exact combination, which adds complexity to the process. If the project supports multiple major versions at once, the burden grows larger. Open source maintainers already have more work to do than time to do it, so making the process more complex only makes the situation worse.

The general workflow looks like something Johannes Gutenberg would recognize from 1440 because it works pretty well. There are plenty of ways we can improve it, of course, but at a conceptual level, we keep doing it because it works. As Seifried noted, a new system would necessarily be a radical change. He likened it from inventing a bulldozer instead of hiring more people to dig. But the practical implementation of what he proposes makes more work for everyone.

This post’s featured photo by Towfiqu barbhuiya on Unsplash

Ben formerly led open source messaging at Docker and was the Fedora Program Manager. He is the author of Program Management for Open Source Projects. Ben is an Open Organization Ambassador and frequent conference speaker. His personal website is Funnel Fiasco.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.