rfd: Introduce a Validation Library

rfds/0000-validator-package.md

@@ -0,0 +1,215 @@
+# Creating a CVE Record Validator Package
+
+| Field            | Value  |
+|:-----------------|:-------|
+| RFD Submitter    | Andrew Lilley Brinker |
+| RFD Pull Request | [RFD #0000](https://github.com/CVEProject/cve-schema/pull/1234) |
+
+## Summary
+[summary]: #summary
+
+The CVE Quality Working Group (QWG) should create a JavaScript package that
+provides an API for validating CVE Records. This package would be used within
+CVE Services (the application providing the CVE API to CVE Numbering Authorities
+\[CNAs\] and Authorized Data Publishers \[ADPs\]) and would be published for
+others to use as well. It would perform both the basic validation of CVE Records
+against the CVE Record Format itself, as well as any validation necessary which
+is not expressible using a JSON Schema.
+
+## Problem Statement
+[problem-statement]: #problem-statement
+
+With the introduction of the `packageURL` field for objects within the
+`affected` array, the validity of CVE Records is no longer checkable solely by
+validating records for compliance with the CVE Record Format. The Record Format,
+written as a JSON Schema, is not sufficiently expressive to handle _all_
+validation cases. In particular, Package URL parsing is sufficiently complicated
+to not be conveniently expressible with JSON Schemas' support for regular
+expressions.
+
+This raises the need for CNAs and ADPs, who submit CVE Records or CVE Record
+fragments to CVE Services for publication, to have some mechanism to validate
+records which applies _all_ of the checks which would be used at submission-time
+by CVE Services. This mechanism must also meet some additional constraints,
+including that it be runnable locally, without transmitting any information to
+an external service. While all CVE Record materials that get published will of
+course become public, some CVE stakeholders demand the ability to validate
+records within their own environments, including ensuring that no proprietary
+information would be published, before sending those records to CVE Services
+for publication. Validation is part of that checking, and so needs to be able
+to be performed locally.
+
+There is also the possibility that future constraints on CVE Records may be
+added which are not expressible, or not conveniently expressible, within the
+JSON Schema. For example, there are currently limited constraints on the
+expression of version ranges in the Record Format, which permits version ranges
+which are nonsensical. While the onus is and will always remain on CNAs to
+publish valid, sensible, high-quality information in their CVE Records, it would
+be useful for CVE Services and for CNAs themselves to be held to a higher
+standard with automated checking that disallows nonsensical version ranges.
+
+If nothing is done to adopt some mechanism for complete validation of CVE
+Records, then stakeholders would continue to rely on the Record Format to
+provide _most_ validation, but would still need to handle the possibility that
+CVE Records which pass Record Format validation would be rejected by CVE
+Services which implements additional checks not expressible in the Record
+Format.
+
+## Proposed Solution
+[proposed-solution]: #proposed-solution
+
+This RFD proposes creating a JavaScript package, to be published on the NPM
+package registry, which implements an API for validating CVE Records and
+CVE Record Fragments, according to the requirements of both the CVE Record
+Format and any additional rules not expressible with the Record Format.
+
+This validation library would be used by CVE Services and by external
+stakeholders such as CNAs. This would be done to ensure that the validation
+performed by CVE Services would not drift apart from the validation logic
+provided to stakeholders.
+
+The versioning of this library would, for clarity and simplicity, be matched
+to the versioning of the CVE Record Format. Whenever new versions of the Record
+Format are published, a new release of the validation library with a matching
+version number would also be published. This would ensure users have a clear
+answer for what version of the validation library to use, as it will always
+be the version that matches the current version of the Record Format used by
+CVE Services.
+
+To indicate that the library is officially maintained and provided by the CVE
+Project through the Quality Working Group, the package name should be scoped
+to an organization name which includes the word "cve" on the NPM package
+registry. The choice of a specific organization name is left unspecified here,
+as it will depend in part on name availability on the registry.
+
+The name of the package would be `cve-record-validator`, which is intended to be
+a clear and unambiguous name which reflects the purpose of the library. Coupled
+with the scope recommended by this RFD, the full package name would be
+`@<org>/cve-record-validator`. If the CVE Project is able to secure the
+organization name `cve` on NPM, then the name could be modified to
+`record-validator`, giving the full name `@cve/record-validator`.
+
+The specifics of what this API would look like are still to be determined. While
+possibilities have been raised of specific libraries in JavaScript which could
+be useful for expressing validation rules, such decisions are premature. Once
+consensus is reached among the QWG on the need to develop this library, the
+next step would be to work with both the Automation Working Group (AWG), which
+owns the development of CVE Services, and with the Community Working Group
+(CWG), which interfaces with and represents the interest of CVE stakeholders,
+to understand the set of requirements which must be met for the API of this
+validation library.
+
+With the creation of this library, two major procedural changes would also need
+to be made. First, the RFD template must be amended to include a requirement
+that new proposals be assessed for any necessary updates to the validator
+library. This will ensure that future improvements to the Record Format have
+their validation rules consistently applied to the validation library. The
+second procedural change would be to update the process of releasing new
+versions of the Record Format to include also releasing a new version of the
+validation library with a matching version number. These releases should
+include a changelog which describes the modifications made since the last
+version.
+
+## Examples
+[examples]: #examples
+
+Not applicable, as this is a procedural and goal-oriented RFD which does not
+describe changes to the Record Format.
+
+## Impact Assessment
+[impact-assessment]: #impact-assessment
+
+The benefits of the creation of a CVE Record validation library include:
+
+- CVE stakeholders, including CNAs and ADPs, having the ability to fully
+  validate CVE Records before submitting them to CVE Services, ensuring they
+  can have confidence in submissions to CVE Services while keeping potentially
+  sensitive data from spilling to CVE Services unintentionally.
+- CVE Services being able to keep its validation logic in sync with the full
+  set of rules required by the CVE Record Format, even as validation logic
+  exceeds what is expressable with JSON Schemas.
+- The QWG becoming responsible for the maintenance of validation rules, which
+  places responsibility for this work on the organization within CVE's structure
+  most equipped to understand and implement the rules.
+
+The risks associated with the production of this validation library include:
+
+- The QWG may fail to maintain the library over time.
+- The need to maintain the library adds some additional overhead to the work
+  of the QWG, as updates to the schema must be assessed for additional
+  validation requirements, and any new requirements must be implemented by the
+  QWG prior to shipping the next release of the Record Format.
+
+## Compatibility and Migration
+[compatibility-and-migration]: #compatibility-and-migration
+
+This change does not affect the CVE Record Format, and so does not impact the
+compatibility of the Format itself.
+
+For CVE Services, the introduction of this validation library will require some
+work on their part to replace any existing validation logic with uses of the
+library.
+
+For CVE stakeholders, such as CNAs or ADPs, they would need to transition their
+own existing validation logic which makes use of the Record Format to instead
+use this library. As we've already adopted the introduction of `packageURL`s as
+a field within the objects of the `affected` array, this kind of additional
+validation work is already set to be demanded of these stakeholders, but it
+continues under this proposal. Once such a transition is complete, stakeholders
+could continue to use this validation library, which may then be updated across
+subsequent versions of the Record Format to reflect additional validation rules,
+reducing maintenance burden for stakeholders over time, as they would not need
+to implement additional validation rules themselves, except for any they
+desire to implement beyond those required by CVE Services.
+
+## Success Metrics
+[success-metrics]: #success-metrics
+
+The RFD will be considered to be successful if, six months after the first
+release of the validation package, it is in use by two or more CVE stakeholders,
+as assessed by the Consumer Working Group, and if the validation library is also
+in use and considered valuable and maintainable by the CVE Services team.
+
+If these metrics are not met, the validation package will be discontinued and
+alternatives solicited by the QWG for consideration.
+
+## Supporting Data or Research
+[supporting-data-or-research]: #supporting-data-or-research
+
+The need for this or some alternative solution to the validation problem
+described is inherent in the adoption of the `packageURL` field and the new
+need to perform validation beyond what is expressible in JSON Schema.
+
+## Related Issues or Proposals
+[related-issues-or-proposals]: #related-issues-or-proposals
+
+There are no other open proposals currently available.
+
+One alternative considered was to offer some API on CVE Services which
+_only_ performs validation without publishing. However, this was rejected as
+it would require CVE stakeholders to share CVE Records with CVE Services before
+they are considered ready for publication. While CVE Services could establish
+policies to protect the confidentiality of these pre-publication records,
+such as not logging their contents and not storing them, such an approach would
+be riskier than one which does not involve the transmission of pre-publication
+CVE Records at all.
+
+## Recommended Priority
+[recommended-priority]: #recommended-priority
+
+This has a recommended priority of __High__, as it interacts with the already-
+approved `packageURL` field slated to be published in version 5.2.0 of the
+CVE Record Format.
+
+## Unresolved Questions
+[unresolved-questions]: #unresolved-questions
+
+The design and implementation of the API of this library remains open for
+determination, as do the specific set of requirements which are needed to
+support both CVE Services and external CVE stakeholders.
+
+## Future Possibilities
+[future-possibilities]: #future-possibilities
+
+We anticipate a follow-on RFD which addresses requirements, the design of the
+API, and any salient details about the implementation.