How to triage security reports

In chapter 10 of Program Management for Open Source Projects, I talk about triaging bug reports. One of the questions in that process is “is it a security bug?” That’s good for helping you sort the security reports from the non-security reports, but what do you do when the answer is “yes”?

The crushing weight of security reports

In the first third of this year, we’ve seen a remarkable increase in security reports, particularly against popular open source projects. This isn’t (just) AI slop, it’s also AI quality. Daniel Stenberg says we’re in a “high-quality chaos era.” GitHub is tightening the rules on its bug bounty program because the staff can’t keep up.

Despite the hype about Claude Mythos Preview, I’m not particularly worried about the severity of security reports; I’m worried about the volume. There’s slop coming, yes, but also a lot of medium-to-high quality reports. And maintainers are going to have to deal with them.

How to triage security reports

The vast majority of open source projects are too small to provide the sort of reputational boost that finding a vulnerability in, say, the Linux kernel would provide. But even if you’re not get deluged with security reports (and even if you may never be), it’s good to start planning now.

Create a security policy

A good first step is to create a security policy and store it in a place like SECURITY.md in your repo. This doesn’t have to be long and complex. Even a few sentences is better than nothing.

At a minimum, tell people how to submit security reports. Do you want them sent through some private mechanism for coordinated disclosure? Do you want to take libxml2’s approach and treat them like any other bug report? If you are doing the private reporting mechanism, what should the reporter expect next? Do you have a response timeline? A default coordinated disclosure timeline? Do you offer a bug bounty program? These can all be “no”, of course. The goal here is not to place unwanted obligations on you as a maintainer; the goal is to set expectations in advance.

You may also want to add things that come up in the following section to your security policy as well.

And of course, the security policy is not just for reporters — it’s also for users. You may include information about what releases get security fixes and how those are handled. When do you bundle certain security fixes into a planned release and when do you do a special release? Do you even say which releases are “security releases”?

The Eclipse CSI project has a good example policy.

Questions to ask

As you triage the reports you get, you should ask yourself some questions. There’s no scoring system here (although you could try to devise one if that makes you happy). Instead, this is about rejecting bad reports and moving a particular good report up and down the stack. If you only have one report to deal with at a time, you don’t need to worry about prioritizing against other vulnerabilities, only against your regular work.

Quick-reject questions

• Is this in your software? If it’s a security issue in one of your dependencies, the reporter should file it upstream. (If you can mitigate it in your software, you still should). If it’s a security issue in a downstream user of your software, the reporter should file it there. Unless there’s something you can do directly about it, close the report.
• Is this legitimately a vulnerability or is it just a dangerous tool? If I filed a security issue that said “when I run rm --no-preserve-root / it deletes my data and makes the system unbootable”, I’d be rightfully banned from the project. Of course it can do that because rm is for deleting files. Reports like that can be closed immediately. On the other hand, if passing a specific, carefully crafted file name to rm as an unprivileged user resulted in me getting privileged access, that’s a legitimate vulnerability.
• Is this documented behavior? Similar to the above, if the report is about something that you document to be risky or unsuited for production use, then you can close the report. Many static website builders, for example, have a command to run a local webserver for testing. These almost never have any real security because they’re not intended to.
• Does the reporter seem to understand what they’re talking about? You’ll probably have follow-up questions. Does the reporter answer them like they know what’s going on or are they just copy/pasting out of their LLM. If there’s no human judgment happening here, you can reject it (and may consider banning future reports from that user).
• Does the alleged vulnerability cause harm? Can an attacker actually exploit this? If there’s no obvious harm that comes from it, it’s just a bug.

Prioritization questions

• Does the report include key metadata like version, OS, hardware, etc? This is a good candidate for “stuff to include in the security policy.” If the report is for an unsupported setup (especially an unsupported version), then you may choose to reject it. If the reporter didn’t provide the information, you can wait until until they do.
• Does the report include a proof of concept? If the alleged vulnerability is highly theoretical, that doesn’t mean it’s not real, but you will probably move it lower in the pile. If the report includes specific steps or code that shows the vulnerability in action, move the report up. Apart from showing that it’s a real thing that an attacker could exploit, having a proof of concept enables you to test if your fix is correct.
• Can the vulnerability be triggered remotely? If a vulnerability can only be performed by someone who already has access to the system, you can move it down the pile. If anyone passing by on the Internet can trigger it, then it moves up the pile.
• Does the vulnerability require privileged access? Similarly, vulnerabilities that require privileged access have a smaller set of possible attack vectors and might move down the list a bit.
• Does the vulnerability require specific user action? Anything that self-propagates or is exploited through automated means (like receiving a specially crafted text message or opening a file) should move way up the list.
• Is the vulnerability in the default configuration of the software? Most people will use the defaults you provide. If a vulnerability only exists when one strays from the defaults, you can move that priority down the list.
• What’s the harm? If a vulnerability causes an application to crash, that can be inconvenient, and maybe even costly to a business. If a vulnerability causes data loss or corruption, that’s worse.

Your obligations

Good news: you have no obligations other than what you choose to put on yourself! Your project is provided “as-is” after all, and no matter who thinks you should do things a certain way, you don’t have to do anything you don’t want to do. That said, having a clear security policy is part of being a good member of the ecosystem. Fixing security issues as quickly as your time permits is also good — you’d want the same from the maintainers of software you use.

I hope this post helps you manage incoming security reports. If there’s something you’d add, let me know! The Open Source Security Foundation has a ton of tools, training, guides, and other resources to help you survive the security tidal wave.

This post’s featured photo by FlyD on Unsplash.