How to re-write your project
If you’re lucky, you know a lot more now than you did when you started your project. The natural inclination is to want to rewrite the project. Sometimes it’s because you did things in a way that isn’t as good as you could do now. Sometimes you made a decision that paints yourself into a corner — or the decisions that made sense in the landscape of the past no longer do. So how do you re-write your project?
Don’t
In a lot of cases, the best way to rewrite a project is to not rewrite it. When I was active in the HTCondor community, I’d occasionally see suggestions to rewrite the project. HTCondor was written primarily in C++. Why not rewrite it in Python, someone would ask. (They’d probably suggest Go or Rust now.)
Here’s the thing: HTCondor is a 35 year old project. It has multiple daemons and client tools. It’s used to manage research computing workloads in the private sector, at the Large Hadron Collider, NASA, and so on. It’s kind of a big deal. How many bugs would we introduce, the developers asked, by trying to reimplement this project in a new language?
Rewriting a software project is risky. It’s also rewarding, but do the rewards truly outweigh the risks? Early in a project’s life, perhaps, or if there are few users. But once you have a broad user base, it becomes riskier to introduce new breakages.
But if you must
In some cases, you’ll decide the risk is worth the reward. So how do you go about rewriting the project? It turns out, as is often the case, that the technical part is easy. The hard part is managing people.
Understand the landscape and motivation
Before your community writes a single line of the new code, you have to understand what’s going on. You’ve learned a lot from the previous version of your project. Document that. Clearly identify the benefits and drawbacks to the current approach. You also need to make clear why you’re rewriting the project. Is it because a key library is no longer getting updates? Is it because the existing doesn’t allow for the features you want to add? Maybe the current version is impossibly slow. You should have this clearly documented and you should be able to explain it without feeling silly. (If you feel silly that your answer is “…so we’re rewriting the project from scratch,” then you probably shouldn’t do it.)
Understand how your community — and the ecosystem more broadly — have changed since the current version began. What languages, frameworks, etc. do your current contributors know? And what do your desired contributors know?
List the features and their tests
In most cases, a rewrite involves changing how features work. You might drop old features that are hard to maintain or get in the way of adding the features you really want. This is fine, but you have to be clear about it. No sense in pretending that it will do everything the way the old version did.
Also, you need to prepare yourself to lose some users. The new version won’t work for everyone. Again, that’s fine. Stick to your plan.
Then write tests for everything you’ll keep. You should have test cases for all of the things that you plan on keeping the same. Unit tests may or may not work, depending on how different the new version will be internally. But you can plan for integration tests, manual testing, or whatever will work. This is where your user community can make great contributions. They can tell you all of the weird, unexpected ways they use your software, and you can use that to make sure your tests have enough coverage.
Allocate your effort
Bad news: you can’t just abandon the old version to work on the new version. Okay, you can, but that’s not going to win you any friends. While you’re writing the new version — and for some time after — you need to maintain the old version. You don’t necessarily need to put effort into adding new features, but you do need to continue to fix bugs and provide whatever level of support that your project provides.
The easiest way is to have some people working on the old version while others work on the new version. That, of course, implies that there are contributors who want to work on the old version. If no one does, it’s hard to force them, so you need to try to make sure you’re collectively putting some effort into the old code.