cveproject / cve-services

This repo contains the source for the CVE Services API.

CVE-API

Table of contents

• Project
• Contributing
  • Security
    • Reporting a Vulnerability
• Development
  • Technologies
  • Style Guidelines
  • Directory Layout
  • Setup
  • API Documentation
  • Unit Testing

The CVE Services Project

This repository contains services that support the CVE Program's mission to "identify, define, and catalog publicly disclosed cybersecurity vulnerabilities."

There are many ways one can assist:

OSS Contributor

Developers can contribute code directly. Getting started can be as fast as choosing an issue on our board.

Please read our contributor's guide for more details. We welcome all contributions!

Working Groups

The CVE project operates as multiple focused working groups. Visit the CVE Website working groups page for more information.

Security

Reporting a Vulnerability

Warning Do not put vulnerability information in a GitHub issue.

Please consult our SECURITY.md for specific instructions on reporting a vulnerability that exists in the CVE Services.

Development

Technologies

This project uses or depends on software from

• NodeJS
• Express
• MongoDB for locally run instances
• Mongoose.js

Style Guidelines

This project follows the JavaScript Standard Style.

Setup

Docker

See the Docker README found in the repo here: https://github.com/CVEProject/cve-services/blob/dev/docker/README.md

Local Development

Warning

DO NOT use the dev configuration on a public network. The dev environment includes credentials to enable rapid development and is not secure for public deployment.

1. Install required node modules

This assumes node 16.14.2 and the latest npm are installed.

cd cve-services
npm install

2. Setup and start MongoDB locally

Install MongoDB locally

• https://docs.mongodb.com/manual/administration/install-community/

Download MongoDB Compass (MongoDB GUI)

• https://www.mongodb.com/download-center/compass

Create a cve_dev database in Compass. The collections will be automatically created when the API starts storing documents.

You can populate the database with test data using:

npm run populate:dev

3. Start the node application

In order to start a dev environment:

npm run start:dev

API Documentation

API documentation is generated using swagger-autogen which ensures that we keep the API specification up to date with any major changes to API routes. Extra information for each API route is defined as a comment in the index.js files under the respective controller and all request and response schemas are stored under the schemas folder served up by schemas.controller.

To ensure you are using the correct API specification the following endpoints can be used:

• Test Instance
• Production

Note: The specification file stored in GitHub will only be correct for that branch; there could be differences between branches and production.

If you are developer and want to test changes to the API specification you can generate a specification in one of two ways:

1. Preferred

When you start your local development server using npm run start:dev the specification file will be generated. Subsequent changes require reloading the server.

2. Manual

You can use npm run swagger-autogen to generate a new specification file.

Unit Testing

This project uses the following for unit testing

• https://mochajs.org/
• https://www.chaijs.com/

In order to run the unit tests:

npm run start:test

cve-services's Issues

Non-Sequential IDs

User story: As a user from a CNA or CNA-LR, I want to be able to reserve multiple IDs and for them to be non-sequential (random assortment) so that others cannot speculate as to which CVE IDs I have reserved at any given time.

AC:
Given that a user specifies an amount greater than 1 and for the batch to be non-sequential, then the system reserves an assortment of CVE IDs equal to the amount specified and includes them in the response.

Historical Requirement Mapping:
R3: Reservations can be requested and assigned in either a random (default) or sequential based on the request. Note: The Random selection is not the default in this user story, user must specify if the request is sequential/non-sequential.
R4: Reservations can be requested and assigned as a single (default) or block of CVE IDs
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization

Changing ID Quota

User story: As a Root CNA , I want to be able change the quota on behalf of a CNA under my administrative control.
AC:
When a Root CNA initiates a quota modification on behalf of a CNA, the quota modification is reflected in the available CVE IDs that are available to the CNA.

Historical Requirements:
R7: The Requestor and Managing Root needs to be notified (logging?) when any error condition occurs during ID Reservation requested operations. See Note* above. There is only 1 “managing root” which is the Secretariat for Phase 1. Note: The need to inform the “managing root” is not part of the user story. Logging is not part of the user story.
R12: Requests are checked against preset organizational values (e.g., ID quota) on a pre-request basis
R13: A quota is enforced based on hard limits. The quota usage will follow the formula:
available_ids = hard_limit - reserved_but_not_published_id_count
R14: Published IDs will not count against the quota.
R15: Reservation of IDs up to the hard limit will be allowed. For example, if a CNA has quota of 100 CVE IDs with 90 reserved but not published, they then have 10 CVE IDs available for future reservations.
R16: Requests for IDs will be rejected once the hard limit is reached.
The hard and soft limits need to be dynamically resized based on an algorithm that considers the organization's historic population stats, highest usage within a useful timeframe, and projected growth. (optional)
R17: The Requestor and Managing Root needs to be notified when soft and/or hard limits are reached. See Note* above. There is only 1 “managing root” which is the Secretariat for Phase 1.
R20: Any response should return the requested number of CVE IDs and balance information on the quota status after this request is completed. (Email receipt as well) Note: There is no email notification as part of this User Story.

User Registry for ID Allocation

Org MongoDB Document

Example of an Org document stored in MongoDB:

{
    "_id": ObjectID(),
    "name": String,
    "short_name": String,
    "UUID": String, // A UUID used for provider_data_meta in cve records
    "authority": {
        "active_roles": [String]
    },
    "policies": {
        "id_quota": Number
    },
    "time": {
        "created": Date,
        "modified": Date
    }
}

Note: We are abstaining from using CNA terminology in this collection to emphasize that a CNA is just one of the types of organizations that participate in the CVE system and that an organization does not have to be a CNA to be in the User Registry. Being a Secretariat or an ADP would also warrant an organization to be in the User Registry, but they do not have to be CNAs.

User MongoDB Document

Example of a User document stored in MongoDB:

{
    "_id": ObjectID(),
    "username": String,
    "cna_short_name": String,
    "secret": String,
    "UUID": String, // A UUID used for protecting PII in cve record
    "active": Boolean // Indicates whether the account is deactivated
    "name": {
        "first": String,
        "last": String,
        "middle": String,
        "surname": String,
        "suffix": String
    },
    "time": {
        "created": Date,
        "modified": Date
    }
}

CNA Detail Endpoints

GET /cna/:shortname/id_quota

This is an endpoint for CNAs to see details about their ID quota.
CNAs cannot see other CNA's information, if requested a 403 Forbidden will be returned.
The Secretariat can see anyone's quota and details.

Response when the requester's CNA owns the CVE ID or is the Secretariat

{
    "id_quota": Number,
    "total_reserved": Number,
    "available": Number // id_quota - total_reserved (for convenience)
    // Status: 200 (OK)
}

Response when the user's CNA does not own the CVE ID and is not the Secretariat

{
    "message": 'This information can only be viewed by the owning CNA or the secretariat.'
    // Status: 403 (Forbidden)
}

Total reserved should not be stored, it is calculated upon request. Storing this number will cause unnecessary complexity in trying to keep it accurate as the IDR reserves IDs.

Payload Definitions

Payload	Description
id_quota	The hard limit on how many IDs the CNA can have in the Reserved state. All CVE-IDs from all years count against this quota.
total_reserved	The number of CVE-IDs from all years that the CNA has at the time of the request in the Reserved state.
available	The number of IDs the CNA can currently reserve before they hit their quota. Provided for convenience, as it is id_quota - total_reserved.

Admin/Manipulation Endpoints

POST /cna/:shortname

Endpoint for Secretariat to change details of the organization. Community will need to use current channels to update their details.

Query Parameters	Values	Description
id_quota	Number (i.e., 300)	Allows Secretariat to change id_quota for an Org.
active_roles.add	String (i.e., CNA)	Allows Secretariat to add the CNA role from an Org's active roles, effectively enabling that Org's User's from performing tasks as that role.
active_roles.remove	String (i.e., CNA)	Allows Secretariat to remove the CNA role from an Org's active roles, effectively disabling that Org's User's from performing tasks as that role.
shortname	String (i.e., mitre)	Allows Secretariat to change the CNA short name.
name	String (i.e., The MITRE Corporation)	Allows Secretariat to change the CNA formal name.

Examples
POST /cna/:shortname?id_quota="300"
POST /cna/:shortname?active_roles.add="CNA"
POST /cna/:shortname?active_roles.remove="CAT"&active_roles.remove="DOG"
POST /cna/:shortname?shortname="cve"
POST /cna/:shortname?name="CVE Program"
POST /cna/:shortname?id_quota="300"&active_roles.add="CNA"&active_roles.add="SECRETARIAT"&active_roles.remove="CAT"&shortname="cve-program"&name="CVE Program"

Note: /cna/:shortname?active_roles.[add|remove] endpoint is extensible for when additional roles are added to the system.

Response when the user’s CNA is not the Secretariat

{
    "message": 'A CNA cannot be updated by other than the secretariat.'
    // Status: 403 (Forbidden)
}

Response when the user’s CNA is the Secretariat

{
    "message": 'cve-program CNA was successfully updated.',
    "updated": {
        "authority": {
            "active_roles": [
                "CNA",
                "SECRETARIAT"
            ]
        },
        "policies": {
            "id_quota": 300
        },
        "short_name": "cve-program",
        "name": "CVE Program"
    }
    // Status: 200 (OK)
}

POST /cna/:shortname/user/:username

Endpoint for Secretariat to change details of the user. Community will need to use current channels to update these details.

Query Parameters	Values	Description
new_username	String (i.e., tsmith)	Allows Secretariat to change the userrname of a specific User.
new_cna_shortname	String (i.e., cve-program)	Allows Secretariat to change the CNA to whom a specific User belongs to.
active	true or false	Allows Secretariat to activate/deactivate a specific User.
name.first	String (i.e., Ted)	Allows Secretariat to change the first name of a specific User.
name.last	String (i.e., Smith)	Allows Secretariat to change the last name of a specific User.
name.middle	String (i.e., John)	Allows Secretariat to change the middle name of a specific User.
name.surname	String (i.e., Mr.)	Allows Secretariat to change the surname name of a specific User.
name.suffix	String (i.e., Jr.)	Allows Secretariat to change the suffix name of a specific User.

Examples
POST /cna/:shortname/user/:username?new_username="tsmith"&active="false"
POST /cna/:shortname/user/:username?name.first="Ted"&name.last="Smith"&name.middle="John"
POST /cna/:shortname/user/:username?new_cna_shortname="cve-program"

Response when the user’s CNA is not the Secretariat

{
    "message": 'A user cannot be updated by other than the secretariat.'
    // Status: 403 (Forbidden)
}

Response when the user’s CNA is the Secretariat

{
    "message": "tsmith was successfully updated.",
    "updated": {
        "username": "tsmith",
        "cna_short_name": "cve-program",
        "name": {
            "first": "Ted",
            "last": "Smith",
            "suffix": "Jr.",
            "surname": "Mr."
        },
        "active": false
    }
    // Status: 200 (OK)
}

POST /cna/:shortname/user/:username/reset_secret

An action endpoint that allows the Secretariat to reset the User's api secret. The old one is removed and no longer to be used. Payload will be the new api secret.

Response when the user’s CNA is not the Secretariat

{
    "message": 'The user's secret cannot be updated by other than the secretariat.'
    // Status: 403 (Forbidden)
}

Response when the user’s CNA is the Secretariat

{
    "API-secret": <secret>
    // Status: 200 (OK)
}

test note

Establish a CI pipeline to run the repository's unit tests

Motivation

Having a CI/CD pipeline improves a development team's productivity by bringing bugs and interoperability issues to the team's attention early. It also will run against PRs as they're opened which will reduce effort needed by maintainers to review code contributions.

Work Necessary

Investigate GitHub Actions. This is a new service that might be the most convenient as there seems to already be workflows that can test Node.js code with no set up.

Prepare the test suite. After the pipeline is established, the tests need to be clean and up to date in order to be used throughout the project.

Describe alternatives you've considered

There are many tools and cloud services that can provide CI pipelines. GitHub Actions seems the most convenient for now and allows outside contributors to see the results of their changes within the project without any additional accounts or permissions.

Complete internal code to register users

Summary

In order to make sure registering users goes smoothly, the ESUS should have internal code to allow IT admins to easily register users and have the service create user's API secrets. This way, nothing is done manually to allow opportunity for human error.

This code can then be migrated to the User Registry and overall make the transition of the Secretariat role smoother.

Motivation

Hand registering people can be error prone and slower.

Describe alternatives you've considered

None

Docker environment printout

Summary

Print out some environment variables before starting the application in the docker container

Motivation

Debug database connectivity during deployment to our development, staging and production environments

Initial ID Owner

User story: As a user, I want the ID owner to be set to my CNA organization UUID so that my CNA can manage it.
AC:
Scenario 1:
When a CNA requests for any ID to be reserved the owner is set to their CNA organization UUID.

Scenario 2:
Given that a Root CNA requests for an ID to be reserved for another CNA the owner is set to that CNA.

Historical Requirement Mapping:
None

Define push contact methods that don't require an account with a specific company

We need push contact methods for CNAs that don't require a specific account/ToU/etc (e.g. Facebook only or Linkedin). Some possible options:

1. Email
2. Web Form on CNA web site or third party that does not require a login/ToU that is heavy handed (note: most sites now have a ToU even if static)
3. Phone (with voicemail required because timezones?)
4. Postal mail (included here for completeness

Other push options?

Secretariat is not limited by id_quota

While CNAs and Root CNAs are limited by quotas, the Secretariat should not.

Sequential IDs

User story: As a user from a CNA or CNA-LR, I want to be able to reserve multiple IDs and for them to be in sequential order so that I can publish related CVEs with sequential IDs for my customers.
AC:
Given that a user specifies an amount greater than 1 and for the batch to be sequential, then the system reserves a sequence of CVE IDs equal to the amount specified and includes them in the response.

Historical Requirements Mapping
R3: Reservations can be requested and assigned in either a random (default) or sequential based on the request. Note: Random is not the default, user must specify if the request is sequential/non-sequential
R4: Reservations can be requested and assigned as a single (default) or block of CVE IDs
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization

Access Control for Root CNA

User story: As a CNA, I want only users that are authorized to reserve IDs on my behalf.
AC:
Scenario 1:
Given that a user that belongs to a CNA makes a request to reserve IDs for that CNA to the IDR, the service proceeds with processing the reservation request.

Scenario 2
Given that a user makes a request to reserve IDs and they are not a user for the Root CNA, CNA-LR or the CNA specified, the user will receive an error response (HTTP 401 Unauthorized).

*Note: For Phase 1 implementation, the Root CNA, and CNA-LR role is filled by MITRE, hence the implementation is limited to CNAs and MITRE as the only Root CNA and CNA-LR.

Historical Requirements Mapping
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization

Modify User Registry endpoints to follow a RESTful model

User Registry routes should be laid out like so:

/cna GET [List all CNAs]
/cna POST [Register a CNA] (limited to the Secretariat)
/cna/:id GET [List a CNA]
/cna/:id POST [Update a CNA] (limited to Secretariat & the CNA being updated or its managing root)

Using ESUS' data flow diagram, develop specific API errors to return to a client

Summary

Now that we have a flow diagram for how client's requests will be handled, we can develop an initial set of errors at the points identified where faults can occur. This document will contain that error's code along with a specific message and any other data found necessary to return to the client by the developer.

Motivation

As clients integrate with the service, having the specific errors documented with additional context and suggestions for how to resolve them will reduce pain points.

CVE ID Requested by User

User story: As a CNA or CNA-LR, I want the user’s UUID stored with the IDs that the user reserved so that I can track which users have reserved which IDs.
AC:
Scenario 1:
When a CNA requests for any ID to be reserved the user’s UUID is stored with that ID.

Scenario 2:
Given that a Secretariat requests for an ID to be reserved for another CNA the user’s UUID is set to the Secretariat user.

Historical Requirement Mapping:
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization

Create Unit Tests for CVE Controller and Middleware

Unit tests should be developed for testing the middleware including the JSON Schema v.5.0 validator and the user authentication. In addition, tests for ESUS and CPS endpoints should be provided.

The unit tests will be created using mocha and chai and will be needed in order to establish the CI pipeline as described in issue 32.

Clarify which branch to contribute code to

Summary

Clarify which branch contains the latest development code. Currently the master and dev branches exist, and it is not clear which one PRs should be submitted to. Maybe update the CONTRIBUTING.md with this info?

Motivation

To help potential contributors understand where to contribute code.

Describe alternatives you've considered

None.

Additional context

None.

Specific user input can create orphan users and cnas in the local MongoDB

In the cna controller, the createCna, updateCna, createUser, and updateUser functions can create/update a cna and create/update a user in the local MongoDB without a cna's shortname or user's username when specific data is provided by the user in the request.

For example:

A request to http://localhost:3000/api/cna with body:
{
	"name": "Red Cat Holdings",
	"short_name": { "red": "cat" }
}

will respond with:
{
    "message": "undefined CNA was successfully created in MongoDB.",
    "created": {
        "name": "Red Cat Holdings"
    }
}

and create the following entry in the Cna collection in cve_dev of the local MongoDB:
{
	"_id": { 
		"$oid": "5eb1ccd1c18558e6c3c5d88c"
	},
	"name":"Red Cat Holdings"
}

However, all entries in the Cna collection must have a short_name field since this is the identifier used to find and uniquely identify a cna in MongoDB.

Reserving CVE ID for Next Year

User story: As a user, I want to be able to reserve CVE IDs for the next year so that I can better prepare.
AC:
When a user specifies the next year in their request, the CVE IDs returned have that next year as the CVE ID year.

Historical Requirements Mapping:
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization
R6: Reservations need to be able to optional specify the “year’ component of the requested CVE. Minimally for a three-year period

JavaScript code does not follow the declared coding standard

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Checked the FAQs on the message board for common solutions: (TBD)
  • Checked that your issue isn't already filed.

Description

The code in this repo does not follow https://github.com/standard/standard as the README.md states.

This is a low-priority defect. Didn't fit as a feature request though.

Steps to Reproduce

Requires installation of the JavaScript Standard Style.

Run $ standard or $ npm test.

Versions

Current version in master.

Additional Information

None.

CC-BY-3.0 license can be problematic

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Checked the FAQs on the message board for common solutions: (TBD)
  • Checked that your issue isn't already filed.

Description

The LICENSE file seems to have text for CC-BY-3.0 license which may have some unnecessary legal baggage as cited here: http://apache.org/legal/resolved.html#cc-by

Expected behavior:

Consider changing the license to a less controversial license such as MIT/BSD-3 clause or CC0.
Better to use SPDX-License-Identifier tag such as

   SPDX-License-Identifier: CC0-1.0
   or 
   SPDX-License-Identifier: MIT

Actual behavior:

• No "SPDX-License-Identifier" in code
• LICENSE contains phrase "Effective Technological Measures" which some people like find objectionable.

Change the use of cna in API endpoints to org

In the org controller package, update index.js to use org instead of cna.

Update the open-api.yml file to accurately reflect these changes.

Remove docker-env as it's not longer needed

Initial ID State

User story: As a Secretariat, I want the initial ID state to be set as “Reserved” so that it adheres to the new CVE record states (CVE states defined on Terminology page).
AC:
When a user requests for any ID to be reserved the initial state is set as “Reserved”.

Historical Requirement Mapping:
None

CVE-ID Lookup endpoints

CVE-ID Data Representations

MongoDB Document

Example of a Cve-Id document stored in MongoDB:

{
    "_id": ObjectID(),
    "cve_id": "CVE-2019-8888",
    "cve_year": "2019", // same year as the CVE ID
    "owning_cna": "apple"
    "state": "PUBLIC" // can be PUBLIC, REJECT, or RESERVED
    "requested_by": {
	   "cna": "apple", // CNA's shortname that reserved the ID
	   "user": "[email protected]" // user that reserved the ID
     },
     "time": {
	   "created": "2020-06-24T21:08:42.937+00:00",
	   "modified": "2020-06-29T23:12:46.576+00:00",
           "reserved": "2020-08-09T04:10:16.116+00:00"
     }
}	

CVE-ID Lookup Endpoints

GET cve-id/:id

If the requester does not belong to the CNA that owns the ID and the state of the ID is reserved, they get 404 in order to not reveal a reserved ID they do not own.

If the requester does not belong to the CNA that owns the ID, they only get the state (PUBLIC or REJECT). Not the user who allocated the ID.

Response when the requester's CNA owns the CVE ID or is the Secretariat:

{
	"cve_id": "CVE-2019-8888",
	"cve_year": "2019", // same year as the CVE ID
        "owning_cna": "apple",
        "state": "PUBLIC" // can be PUBLIC, REJECT, or RESERVED
	"requested_by": {
		"cna": "apple", // CNA's shortname that reserved the ID
		"user": "[email protected]" // user that reserved the ID
	},
	"time": {
		"created": "2020-06-24T21:08:42.937+00:00",
		"modified": "2020-06-29T23:12:46.576+00:00",
                "reserved": "2020-08-09T04:10:16.116+00:00"
	}
        // Status: 200 (OK)
}

Response when the user's CNA does not own the CVE ID and the ID is RESERVED:

{
    "message": id + " not found"
    // Status: 404 (NOT FOUND)
}

Response when the user's CNA does not own the CVE ID and the ID is not RESERVED:

{
	"cve_id": "CVE-2019-8888",
	"cve_year": "2019", // same year as the CVE ID
        "owning_cna": "apple"
	"state": "PUBLIC" // can be PUBLIC or REJECT only
        // Status: 200 (OK)
}

GET cve-id

This endpoint will only return the CVE IDs that are owned by the requester's CNA.

Query Parameters	Values	Description
state	PUBLIC, RESERVED, or REJECT	N/A
cve_id_year	Year of the CVE ID (i.e., 2019)	Year the CVE ID was allocated for.
time_reserved.gt	Timestamp (i.e., 2020-06-24T21:10:00)	Formats allowed: YYYY-MM-DDTHH:mm:ss or YYYY-MM-DD
time_reserved.lt	Timestamp (i.e., 2020-06-30T12:40:00)	Formats allowed: YYYY-MM-DDTHH:mm:ss or YYYY-MM-DD

Allows CNAs to get their IDs, be it reserved, populated, or rejected. Can filter by year and time created. Time created allows clients to reduce payload when trying to update lists since the last GET request.

Examples
GET cve-id?state="RESERVED"
GET cve-id?cve_id_year="YYYY"
GET cve-id?time_reserved.gt="timestamp"
GET cve-id?time_reserved.lt="timestamp"
GET cve-id?time_reserved.gt="timestamp"&time_reserved.lt="timestamp"
GET cve-id?state="PUBLIC"&cve_id_year="YYYY"&time_reserved.gt="timestamp"&time_reserved.lt="timestamp"

Response when no CVE IDs are found for the given query parameters:

{ 
    "message": "No CVE IDs were found for the specified query parameters."
    // Status 404 (NOT FOUND)
}

Response when CVE IDs are found for the given query parameters:

The response will be an array of Cve-Id documents with the exception of the `_id` field. See "Example of a Cve-Id document stored in MongoDB" at the top.

CVE-ID Admin/Manipulation Endpoints

POST cve-id/:id

Query Parameters	Values	Description
state	PUBLIC or REJECT	Can only be done by the Secretariat. The Secretariat can change CVE ID state for specific administration cases. More self service capabilities will come with updates, but for now a CNA will need to go through proper, existing channels to have the CVE ID state changed.
cna	CNA's shortname (i.e., apple)	Can only be done by the Secretariat. Allows Secretariat to transition the ID to another CNA.

Examples
POST cve-id/:id?state="PUBLIC"
POST cve-id/:id?cna="new_cna"
POST cve-id/:id?state="PUBLIC"&cna="new_cna"

Response when the requester wants to update the CVE ID state and is not the Secretariat:

{ 
    "message": id + " cannot be updated by other than the secretariat."
    // Status 403 (FORBIDDEN)
}

Response when the Secretariat wants to update the CVE ID state to RESERVED:

{ 
    "message": "Cannot change the state to RESERVED."
    // Status 400 (BAD REQUEST)
}

Response when the Secretariat wants to update the CVE ID state to PUBLIC or REJECT:

{
    "message: "CVE-2019-8888 was successfully updated.",
    "updated": {
        "cve_id": "CVE-2019-8888",
    	"cve_year": "2019", 
        "owning_cna": "apple"
        "state": "PUBLIC" // can be PUBLIC, REJECT, or RESERVED
    	"requested_by": {
    		"cna": "apple", 
    		"user": "[email protected]" 
    	},
    	"time": {
    		"created": "2020-06-24T21:08:42.937+00:00",
    		"modified": "2020-06-29T23:12:46.576+00:00",
                "reserved": "2020-08-09T04:10:16.116+00:00"
    	}
    }
    // Status 200 (OK)
}

Add support for hard allocation limits in the CVE ID Allocation Service

Summary

There is a need to establish and enforce hard limit for the number of CVE IDs a given CNA can reserve at any point in time. This will prevent a CNA from over reserving CVE IDs beyond their established needs. This also provides a means to manage Reserved but Public (RPB) CVE IDs.

• Hard limits for each CNA need to be derived from their current block allocation numbers. [The CNA coordination team has these details.]
• Populated CVE IDs must not count towards a CNA's hard limit. This allows the CNA to reserve more CVE IDs once they populate their existing reservations.
• The maximum allocation hard limit must be changeable for a CNA. This will allow adjustments to be made if a CNA's needs change. This is equivalent to handing out a new, additional block today.
• A cap needs to be implemented for the number of IDs that can be requested in a single request. This should be something like: minimum_of(hard_quota, 1000). 1000 is in the service description from the CVE services CONOPS. This number could be lower, maybe 100 or 150? [The specific value needs to be confirmed with the SPWG.]
• The specific hard limit for a CNA needs to be definable in the User Registry Service and needs to be manageable by their responsible root CNA.

Motivation

The SPWG has identified the need to establish and enforce hard limits for the number of CVE IDs a given CNA can reserve at any point in time. This replaces the block allocation model currently used by the CVE Program. Per CNA hard limits allow each CNA's limit to be tuned based on their volume of assignment. This feature is an important management tool for addressing RBPs.

Limits will need to be managed in a way that allows a CNA working in good faith to request new CVE IDs as needed. Such requests should not be refused as long as the CNA is populating previously requested CVE IDs once they have published their advisory for the associated vulnerability through the CVE Entry Submission Service.

Describe alternatives you've considered

The SPWG has not identified any alternatives.

Additional context

None.

Establish live API documentation with the use of the Open API yml file

With an openapi.yml file in the repo, an individual can now use that file to generate interactive, living web documentation for the API without any necessary interaction with the current development work for the API.

This task is ideal for someone wanting to get familiar with Open API and the concept of live documentation. It is also a great way to get familiar with the project and how to contribute.

ID Quota Limit

User story: As a Root CNA, I want the IDR to not allow CNAs or CNA-LRs under my administrative control to have more IDs in the Reserved state than their ID quota (hard limit) so that CNAs or CNA-LRs do not reserve too many IDs and allow them to become stale or cause RBP problems.
AC:
When a user for a CNA or CNA-LR requests to reserve an amount of IDs that would exceed that CNA's ID quota, the IDR returns an error message (HTTP 403 Forbidden). This error message informs the user/consuming client that the amount would exceed the CNA’s quota and include useful information like the current ID quota and how many IDs the CNA could currently reserve.

*Note: For Phase 1 implementation, the Root CNA, and CNA-LR role is filled by MITRE, hence the implementation is limited to CNAs and MITRE as the only Root CNA and CNA-LR.

Historical Requirements:
R7: The Requestor and Managing Root needs to be notified (logging?) when any error condition occurs during ID Reservation requested operations. See Note* above. There is only 1 “managing root” which is the Secretariat for Phase 1. Note: The need to inform the “managing root” is not part of the user story. Logging is not part of the user story.
R12: Requests are checked against preset organizational values (e.g., ID quota) on a pre-request basis
R13: A quota is enforced based on hard limits. The quota usage will follow the formula:
available_ids = hard_limit - reserved_but_not_published_id_count
R14: Published IDs will not count against the quota.
R15: Reservation of IDs up to the hard limit will be allowed. For example, if a CNA has quota of 100 CVE IDs with 90 reserved but not published, they then have 10 CVE IDs available for future reservations.
R16: Requests for IDs will be rejected once the hard limit is reached.
The hard and soft limits need to be dynamically resized based on an algorithm that considers the organization's historic population stats, highest usage within a useful timeframe, and projected growth. (optional)
R17: The Requestor and Managing Root needs to be notified when soft and/or hard limits are reached. See Note* above. There is only 1 “managing root” which is the Secretariat for Phase 1.
R20: Any response should return the requested number of CVE IDs and balance information on the quota status after this request is completed. (Email receipt as well) Note: This User Story does not include email notification of the response.

Modify CVE ID Allocation endpoints to follow a RESTful model

Routes should be laid out like so:

/cve-id GET [List all CVE IDs]
/cve-id POST [Allocate new CVE ID]
/cve-id/:id GET [List status of a CVE ID]

ESUS request body and parameter sanitation

The CNA and CVE controllers use req.body and req.params without sanitation, which can lead to multiple security related issues. Different alternatives should be considered to safely use user provided data.

The following npm modules are options to consider:

• express-validator
• sanitize-html
• express-sanitizer
• sanitize

Reserving a Single CVE ID

User story: As a Secretariat, I want the IDR to only use the numbers 1-20,000 for requests that only ask for one CVE ID to be reserved so that the CVE Program incentivizes the behavior of asking for CVE IDs on demand.

AC:
Scenario 1:
Given that all IDs from 1-20,000 are not yet reserved, when a user asks for a single CVE ID to be reserved, they get the next available ID from 1-20,000.

Scenario 2:
Given that all IDs from 1-20,000 are reserved, when a user asks for a single CVE ID to be reserved, they get the next available ID from the regular range of IDs starting from 20,001.

Parking Lot:
P6: In future sprints, the 20,000 will be a configurable value.

Historical Requirements Mapping:
R3: Reservations can be requested and assigned in either a random (default) or sequential based on the request. Note: In Phase 1 implementation, there is no “default”. The “user” must specify which they prefer.
R4: Reservations can be requested and assigned as a single (default) or block of CVE IDs (Note: Block ID (i.e., non-sequential ID are referenced in #86 )
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization. (Note: Block ID (i.e., non-sequential ID are referenced in #86 )
R11: A range of CVE IDs (initially 20K) will be set aside as reserved for single on-demand CVE ID reservations. Settable value on an as needed basis.

Default CVE ID Year

User story: As a user, I want to the default CVE ID year to be set to the current year so that I can reserve an ID for the current year.
AC:
When a user requests for an ID to be reserved and the year is not specified then the current year will be used.

Historical Requirement Mapping:
R6: Reservations need to be able to optional specify the “year’ component of the requested CVE. Minimally for a three-year period

Backlog

This backlog is used to track feature requests to be considered post phase 1 release.

#78:

1. Ability to delineate different types of CNAs (e.g., Root CNA, CNA of Last Resort (CNA-LR) mirroring the fact that a an organization can have more than one CNA within their organization.
2. An updated web interface which will directly access the IDR (as opposed to go through the traditional MITRE form request process)

#82:
3. Change in the implementation to accommodate new Root CNA’s and CNA-LR (related to #78)

#83:
4. Accurate tracking of CNA’s and sub-CNA’s. (i.e., Root CNAs and their relationships to their CNAs)

#84:
5. Add soft quota requirement and also include a scheme to maintain ID quota over time with hard and soft limits.
6. Separate the Secretariat Role and the CNA, Root CNA and CNA-LR roles (related to item 2. & 3 from above).

7. R8: Secretariat needs to be notified of any IDR Service outage or operational failures. (historical requirement from Kent L.)

8. R9: Out-of-band notification will be done via a specified email address for each authorized CVE Program organizations (logging function?) (historical requirement from Kent L.)

9. R19: A quota balance request in the API / web would enable the requesting organization to “check their quota status”. (historical requirement from Kent L.)

Implement internal (admin) endpoints to register a user for the Submission Service

Summary

Develop an endpoint that eases the process for an admin to register a new user. Specifically in the creation of a user's secret key.

Motivation

When rolling out the submission service to initially replace the GitHub process, we will need to register users in order to protect the service from open submissions. In order to ease that process, it would be helpful for admins to be able to provide their identification & CNA and have the service generate their secret key.

Describe alternatives you've considered

An admin could use cli commands to create an API key and insert all the user information into the service's MongoDB. Many things could go wrong with the steps and inserting it into the DB. Very bad alternative that hasn't really been considered but is an alternative.

Authentication

User story: As a Secretariat, I want only registered users to make requests to the system.
AC:
Scenario 1:
Given the user makes a request IDR, when a user provides invalid or no credentials then the user receive an error response (HTTP 401 Unauthorized).

Scenario 2:
Given the user makes a request IDR, when a user provides valid credentials then the service will continue servicing the request.

Historical Requirements Mapping:
R1: Requesters are validated via the User Registry as authorized CVE stakeholders able to request CVE IDS from the service

Create a github.io project site

A project website is a great way to continue strengthening the project as it's effective at orienting developers with the project and about the efforts going into making it.

The content part of this task would be perfect for anyone who enjoys the community side of CVE while someone interested in simple web styling and design can also contribute to the look of the project page to make sure it compliments the future CVE website.

Finally, anyone looking for a great, engaging way to familiarize themselves with the repository and get some easy repetitions in for contributing to the project are welcome to start here.

Integrate library or tool to generate Open API documentation from API code

Summary

In order to ease development against the ESUS and future AWG services, set up a tool/library that can auto-generate an Open API compliant file from API code and code comments.

Motivation

Hand writing yaml files is tedious and error prone. Open API compliance is even more tedious. A few tools exist in the wild and will be used and updated more frequently than a manual process.

Additional context

Open API is a format originating from Swagger specifications. Developers can use tools to ingest an Open API yaml file and have those tools generate code stubs for clients in various languages. For shops that use such tools, it's very beneficial to not have to hand code these and start with a client skeleton they're familiar with.

Reserving CVE ID for Last Year

User story: As a User, I want to be able to reserve CVE IDs from the year before so that I can publish CVEs, that according to program customs, should have a CVE ID from last year.
AC:
When a user specifies the previous year in their request, the CVE IDs returned have that previous year as the CVE ID year.

Historical Requirement Mapping:
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization
R6: Reservations need to be able to optional specify the “year’ component of the requested CVE. Minimally for a three-year period

Replace Secretariat env variable with a list of organizations with the Secretariat role from the database

Summary

Currently, the IDR service uses an environment variable to determine which organization is the Secretariat. Now that roles have been added to the org model we can use a MongoDB query to determine the organizations with the Secretariat role.

Motivation

Is bad practice to hardcode the secretariat organization in an env variable.

Which static site generator serves us best?

The most common approach to building a project website with GitHub is through the use of static site generators, but which one to use?

In the 8/4/20 AWG meeting Hugo and Jekyll were suggested, both with admirable traits.

Jekyll, as the chosen one by GitHub, boasts possibly the best integration with GitHub pages along with one of the largest adoption rates—increasing our chances of contributors being familiar with the platform. But as the older player, it has shown to be troublesome to work with and slow to compile templates.

Hugo, still somewhat new but established, has been taking "market" share in the site generation space due to its fast compilation and rich feature set including internationalization, SEO, and the ability to have almost instant previews as you work.

This issue serves as a discussion board for the community to present cases for either tool or even another static site generator they believe will better serve our use.

Provide a release branch of ESUS for alpha testing

Summary

Provide a release of ESUS for community beta testing. This release can be pulled down by community developers and easily ran in order for them to execute requests against it.

Motivation

Currently, MITRE has some red tape around publishing code to production for public access. In order to not have this cause a delay development, we can provide a modified version of the ESUS that does not make external requests in order to give the community a head start on integrating with the API.

Additional Context

This will effectively only validate that a client can submit a correct request to the ESUS and that their JSON passed the CVE JSON schema v5. The service will not be making requests to CPS or storing the data.

Modify CVE Submission endpoints to follow a RESTful model

Summary

The current API follows some "custom" practices that aren't standard to REST APIs. In order to promote openness in the community, we want to start as close to standard REST practices as possible.

CVE Submission endpoints should be laid out like so:
/cve GET [List all CVEs]
/cve POST [Submit a new CVE] (limited to the CNA the CVE corresponds to or Last Resort)
/cve/:id GET [List a CVE]
/cve/:id POST [Update a CVE] (limited to the CNA the CVE corresponds to or Last Resort)

Implement GraphQL interface for querying CVEs

Summary

For GET CVE functionality, provide an endpoint that can accept GraphQL queries and properly execute them over the collection of CVEs.

Motivation

Instead of having to predict the needs of clients and implementing either endpoints or query parameters that clients have to use in order to obtain the exact information needed, the API can provide a GraphQL interface that allows clients to construct the exact query to execute against the collection of CVEs. With a GraphQL interface, the API and the development team's timelines do not become obstacles from being able to query the collection in a new way.

Describe alternatives you've considered

The development team isn't aware of a technology similar to GraphQL. The alternative to support queries on data would be to document the ways query parameters could be passed in order to construct a proper query to execute against the collection of CVEs. This would need to be done "by hand" in code inside the API and if another way of querying the data was needed, that would also have to be written before it would be available.

Close off all User Registry endpoints to only the Secretariat

Summary

Only the Secretariat will be allowed to make changes to Organizations and accounts for the time being. Right now, the endpoints haven't had middleware authentication to allow developers to insert data when developing. With ways to have data populated automatically in dev and staging environments, we should finally be able to update these endpoints with proper authentication.

Create initial Open API yml file

Until a tool is found to allow us to auto-generate the file from code, manually create the Open API file to assist in documentation of the services API. Use PR #17 as a reference but update to reflect the current endpoints.

ID Reservation (IDR) Service Phase 1 Epic

User story:
As a CNA*, Root CNA or CNA-LR I want to use an on-demand IDR service so that I can reserve CVE ID(s) without human intervention.
*Note: For Phase 1 implementation, the Root CNA, and CNA-LR role is filled by MITRE, hence the implementation is limited to CNAs and MITRE

AC:

• Coding has been complete to implement all functionality noted in the these User Stories
• All source code for deployed services is available in Github for public review
• Unit Tests are written and pass for new code to insure good test coverage across the code base
• User Acceptance Tests are written and pass
• User Acceptance Tests are reviewed by the community
• Publicly available REST API

DoD:

• IDR service is deployed in a production environment and usable by all stakeholders to address their operational role in the CVE Program related to ID reservation as defined by the user stories.
• All defects identified during community testing have been fixed and unit tested. (@mattrbianchi : please revise as discussed in 9/15/2020 AWG meeting)

Historical Requirements Mapping:
R2: Reservation service requests need to be available via an API for automation and a web page utility. Note: the web utility does not map to this Epic Description.
R10: Once a CVE ID is reserved, they are released forever and can only be REJECTed by the CNA in the publication service.

Clean source code for ESUS and remove unused dependencies

Clean unused dependencies to reduce the attack surface for ESUS. Clean dead code and unused code for better readability, maintenance, and to reduce security-related issues.

Pre-populate MongoDB in dev and int

Summary

Create a script that allows users to pre-populate the MongoDB database with dummy data for the Cve-Id, User, and Org collections. The users will still have the option to run the project without pre-populating the db in case they are already using a populated database.

Motivation

The purpose is to allow users to run the project with a pre-populated database of dummy data in a development or integration environment for easier and faster project setup.

Boundless sequential and non-sequential CVE ID Reservations

Summary

This proposal aims to support the ability to allocate IDs in both sequential and non-sequential order. Through the method described in the technical description section below, this approach will keep the bounds of both sequential and non-sequential IDs limitless and not subjugate one option to longer ID lengths than the other.

Motivation

In the community, some find it practical to obtain a sequence of IDs for closely related vulnerabilities. Their various customers and other related parties find it jarring when this group of vulnerabilities do not possess a closely grouped sequence of CVE IDs. Allowing these CNAs to obtain sequential numbers for this purpose allows a smooth transition from manual ID allocation to an automated service by retaining a normality some find customary.

Others in the community find only being able to obtain sequential IDs problematic for various reasons. Sequential IDs in large batches bring forth predictability and allows people to assume a precedent that isn't supported (nor should be). Properly random, non-sequential IDs allow these CNAs to break that precedent and avoid predictability in the publishing of their CVEs.

Additionally, there is a strategic desire to motivate the community to allocate IDs singularly and on demand instead of requesting IDs to be allocated in large batches. In order to foster this movement, the proposed algorithm can easily configure itself to give out a set range of the shorter, more aesthetically desirable IDs to requests that only ask for a single ID as proposed by the SPWG (Strategic Planning Working Group).

Technical Description

First, some definitions:

batch - amount of IDs asked for in a single request to the CVE ID Allocator and thus allocated to the requesting CNA.
block - used to refer to a specific number range within the range of available IDs.

Caution should be used to not confuse the use of block with the current use among some in the community, which is to refer to a range of IDs that have been allocated to a CNA. The use in this document is to refer to a number range with no indication of status of allocation at all.

In order to have sequential IDs, there needs to be a form of reserved space amongst the available range of numbers in the CVE ID format to ensure a proper sequence of unallocated IDs can be obtained. In order to have non-sequential IDs, another form of reserved space needs to exist as a pool from which random IDs can be selected without being easily predictable while not encroaching on the needed sequential ranges.

Note in the formal definitions, N represents a number within the range of natural numbers, 0 to infinity.

For the reserved space for sequential IDs, an allocator selects only the first 500 numbers of each thousand and starts from the lowest number possible. This range can be formally described as CVE-YYYY-[(N)001-(N)500]. Following this practice, the first sequential numbers to be allocated would be CVE-YYYY-0001 to CVE-YYYY-0500 and then the service would move on to CVE-YYYY-1001 to CVE-YYYY-1500, etc. A sequential block of 500 IDs is more than any CNA has evidently used for a single disclosure and therefore should suffice in needing sequential IDs when disclosing closely related CVEs.

This leaves the entire pool of numbers CVE-YYYY-[(N)501-(N+1)000] available for a random number generator to produce non-sequential numbers. While the current CVE ID format allows for an unlimited number of digits, this implementation can initially bound the pool to 5 digits, allotting for 50,000 non-sequential IDs and increase that bound should the pool ever threaten to dry up. This way, the length of non-sequential IDs stays as short as is available.

As noted in the motivation section, this approach can be adjusted to set specific bounds on the lower range of IDs (say, the first 20k IDs) to only be allocated when a request asks for a single ID. This whole lower range would act as its own separate pool for the service to allocate in a non-sequential fashion—as requesting only one ID makes a sequential batch algorithm moot. Once this lower range pool is exhausted, requests for a singular ID would join the current non-sequential pool that was currently serving batched non-sequential ID requests.

Practical Example

If the above proposal was implemented with the lower range of IDs contained under 10,000 reserved for singular ID allocation requests, the whole number space of IDs could be placed in a table like so:

sequential	non-sequential	singular non-sequential
		0001-10000
10001-10500	10501-20000
20001-20500	20501-30000
(∞)0001-(∞)0500	(∞)0501-(∞+1)0000

Alternatives Considered

Initially, an idea of bounding sequential IDs to the space of 4 digits was suggested (being 1-9999) and having non-sequential IDs start past that bound with 5 digits. The first major concern for this implementation is that the range for sequential IDs would be limited at 10k numbers. While that could work currently, as the number of CVEs disclosed per year increases, this solution would require revisiting. On the other end, if sequential ID use becomes unpopular, another concern would be that a significant amount of the first 10k IDs would be wasted.

Access Control for CNA

User story: As a CNA, I want only users that are authorized to reserve IDs on my behalf.
AC:
Scenario 1:
Given that a user that belongs to a CNA makes a request to reserve IDs for that CNA to the IDR, the service proceeds with processing the reservation request.

Scenario 2
Given that a user makes a request to reserve IDs and they are not a user for the Root CNA, CNA-LR or the CNA specified, the user will receive an error response (HTTP 401 Unauthorized).

*Note: For Phase 1 implementation, the Root CNA, and CNA-LR role is filled by MITRE, hence the implementation is limited to CNAs and MITRE as the only Root CNA and CNA-LR and in the meantime any other Root CNAs and CNA-LR will leverage other existing MITRE infrastructure services.

Historical Requirements Mapping
R5: Reservations are tracked to the requesting authorized CVE user and CNA organization