Combinatorial releases won’t help

Kurt Seifried recently wondered why we produce software in a way that “would be pretty much immediately recognizable to Gutenberg as the process and workflow for releasing a printed piece of paper.” Specifically, he asks

What if for example a project released multiple versions of the software with slightly different patch sets or commits or an end user ran the software and pulled in different patch sets and so on and had some sort of feedback mechanism the people could choose to participate in to figure out which one was better more stable had less flaws and so on?

Kurt Seifried

The pat answer is “because nobody wants what that would mean in practice.” From the end user perspective, people don’t want to fill out a character sheet or assemble a kit just to use software. They just want to use the damn software. With some software delivery models (particularly software-as-a-service), it’s certainly possible to offer slight variations to different users. This works well in A-B testing to see if a change has a desired impact. It’s less useful when it’s A-B-C-D-E-F… testing because the number of possible combinations makes it hard to keep track of what actually works better.

This leads us to the maintainer’s perspective. For maintainers, such a combinatorial approach to software releases is a maintenance nightmare. It’s difficult to get useful bug reports from end users, reproduce the issue, and develop a fix. It becomes even harder when the user doesn’t know what combination they’re using. If they do, the maintainer still has to replicate that exact combination, which adds complexity to the process. If the project supports multiple major versions at once, the burden grows larger. Open source maintainers already have more work to do than time to do it, so making the process more complex only makes the situation worse.

The general workflow looks like something Johannes Gutenberg would recognize from 1440 because it works pretty well. There are plenty of ways we can improve it, of course, but at a conceptual level, we keep doing it because it works. As Seifried noted, a new system would necessarily be a radical change. He likened it from inventing a bulldozer instead of hiring more people to dig. But the practical implementation of what he proposes makes more work for everyone.

This post’s featured photo by Towfiqu barbhuiya on Unsplash