Add separate field for withdrawn reason & add guidelines for withdrawing

[Marcono1234]

Hello,
for a university project a fellow student and I had a look in December 2022 at the back then 141 GitHub-reviewed withdrawn advisories in the GitHub Advisory Database, which uses the OSV schema. Back then we noticed the following main issues:

• for 23% no reason for withdrawing was obvious at all
• no uniform way to describe the withdrawing reason; the OSV schema documentation just says "should go into the summary text"¹
  • if we could determine the reason for withdrawing, we found it in 95% in the description; in the other cases we had to go through the references or perform external searches to find more information
  • sometimes the reason was mentioned at the start of the description, sometimes at the end; the format was not uniform at all
  • in some cases GitHub issues or discussions were referenced, but often no direct link to a specific comment so you would have to go through the whole conversation to understand the reason
  • sometimes the complete description and title were replaced with something like "** REJECT ** DO NOT USE THIS CANDIDATE NUMBER. ..." or similar text; this makes it difficult to understand what the advisory was about in the first place
• 56% of the advisories were withdrawn because they are duplicates, but only for ~50% of them the duplicated advisory was mentioned

Based on this we suggest:

• To introduce an additional field, such as withdrawn_reason (Markdown text field), which is mandatory when an advisory is withdrawn
• To specify guidelines how withdrawn advisories should be represented:
  • When an advisory is withdrawn, only a withdrawn_reason should be set. The original summary and details should remain unchanged. Vulnerability database UIs should properly indicate that an advisory is withdrawn without having to rely on special (non-standard) summary texts.
  • The withdrawn_reason should shortly describe the reason and optionally link to discussions (e.g. GitHub issues) for additional information. Only referring to a GitHub issue (especially without referring to a specific summarizing comment) should be avoided.
  • If possible the reason should explain why the advisory was withdrawn and not be something as generic as "This issue was not considered a vulnerability", but instead for example "Withdrawn because product XYZ is intended as command line program run with trusted user input". This way users might notice if the vulnerability actually applies to them because they are using a program incorrectly and in an unsafe way.
  • If an advisory is withdrawn because it is a duplicate, the duplicated advisory ID should be mentioned in the withdrawn_reason field. Or alternatively as proposed in Expand vuln id relationships #53, there should be standard relationship types to indicate the duplicated advisory.

We hope this information is helpful. What do you think?

Here is the discussion specific to the GitHub Advisory Database: github/advisory-database#2420

Footnotes

1. Maybe this is also a bug / ambiguity in the documentation, because I assume you mean with "summary" here the details and not the summary field, since the summary is the plaintext title and you can hardly describe the reason there in much detail. ↩

[oliverchang]

Hi there, thanks for the discussion! Would you be able to speak a bit more around the use case for a more detailed withdrawn field?

The OSV Schema is primarily meant to help with enabling automation for common vulnerability management workflows (e.g. vulnerability scanning, vulnerability interchange between databases), and to be as minimal as possible. Is there a strong case to be made for a dedicated withdrawn reason and how a user would make use of this?

[Marcono1234]

First of all, as short disclaimer, at least for the GitHub Advisory Database the number of withdrawn advisories only represented a small percentage (~1,4% of all GitHub-reviewed advisories were withdrawn, see github/advisory-database#2420), so it is understandable if you think some of this is not worth the effort. For this university project we simply decided to explicitly look at the withdrawn advisories, which is why this issue is so focused on that.

Would you be able to speak a bit more around the use case for a more detailed withdrawn field?
[...]
how a user would make use of this?

I can imagine at least two situations where this would be useful:
Let's say you noticed an advisory (without any tooling) for one of your dependencies and started to investigate it, but the next day the advisory is withdrawn. Or you are using automated tooling which previously reported a vulnerability and now stopped reporting it because the advisory is withdrawn. Then you might want to understand why the advisory was withdrawn, especially if you already investigating it and came to the conclusion that you are affected. Maybe the advisory was withdrawn because it covered a rare specific corner case or unintended usage, but your application just happens to use that dependency in this unsafe way.

The other situation is when there are multiple vulnerability databases involved and the advisory is withdrawn in one (but without proper reason) but not the other. You might then wonder which information is correct, and the detailed withdrawal information might make this easier for you. Let's take for example GHSA-7mg7-m5c3-3hqj, as you see the advisory is withdrawn but it links to a RustSec advisory which has not been withdrawn. It turns out that first GitHub advisory is actually a duplicate of another one, but this is not obvious from the first GitHub advisory at all (this is also mentioned in github/advisory-database#2420).

These are cases where I imagine more detailed withdrawal information would be useful, there might be more cases. Though the question is how often users actually encounter such situations, and what experience other users have made so far with withdrawn advisories.