18f / tts-buy-bug-bounty

Solicitation and acquisition documents created for the TTS Bug Bounty program that can be reused by other government agencies and organizations.

TTS Bug Bounty

Background

As part of its programmatic focus on security, the Technology Transformation Services (TTS) had to purchase access to a pre-existing, commercially available Bug Bounty SaaS Platform that would allow it to manage the TTS Bug Bounty program. The purpose of this acquisition is to give TTS access to a large network of security researchers, people who have an interest in helping to find and address bugs and other technical issues within TTS-owned web applications.

What we're hoping to end up with

The purpose of this solicitation is for the contractor to deliver a Bug Bounty program which TTS will utilize for TTS-owned web applications. The contractor will provide access to their Bug Bounty SaaS Platform for researchers to report vulnerabilities (“Platform/Network Access”) and allow TTS to manage and track issues across multiple public web applications, triage services for those reported vulnerabilities, disburse rewards for effective vulnerabilities, and explain the reasons behind rejections (“Vulnerability Report Triage Services”).

Contributing

See CONTRIBUTING for additional information.

Public domain

This project is in the worldwide public domain. As stated in CONTRIBUTING:

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

tts-buy-bug-bounty's Issues

Type of Contract

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section Section 7.0 Type of Contract - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#70-type-of-contract
It states the following “Based on the nature of this requirement, the government intends to award a hybrid Firm-Fixed-Price (FFP) and Firm-Fixed-Price Not-To-Exceed (NTE) contract type. The contract will include a FFP CLIN for access to the platform and triage services. The bounty pool will be NTE, with varying vulnerability levels but with all costs paid directly to the researchers.”

Question/Comment

Can the vendor respond with maintaining and coordinating the Bug Bounty program as a complete Fix Firm Price Model?

Would this disqualify the vendor if submitting pricing as a complete Fixed Firm Price model, to include the platform, vulnerability management and triage, vulnerability value management, and vulnerability management for all bug bounty challenges?

Phase 2 Price Evaluation

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 5.2. - Phase 2 Price Evaluation - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#52---phase-2-price-evaluation
it states “Evaluation of options under FAR 52.217-8” Within FAR 52.217-8, in section 52.212-5:

Question/Comment

-Is the Contractor required to comply with the FAR clauses set forth under 52.212-5(e)?

-Has 52.212-5 been constructed that as Contractor we need to comply with the FAR clauses listed in 52.212-5(a)-(d)?

-Does the FAR clauses listed under 52.212-5(e) have the requirements for any subcontractors that are appointed?

Background

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 2.0 - Background - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#20-background Third paragraph in this section there is a statement that states “a contractor provides a Bug Bounty SaaS platform that can achieve the goals of the TTS while providing the best value to the government must be one that is well-established.”

Question/Comment

What metrics will the government use to define a well established Bug Bounty SaaS platform besides the size of the pool of researchers in the community that would use the platform?

Performance requirements

Question/Comment on TTS Bug Bounty draft RFO

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFO documents

Section 2.1 Performance Requirements Matrix

Question/Comment

Are the three listed performance requirements the full and complete list, or will TTS add more later?

Background Program Management

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 2.0 - Background - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#20-background
Fifth paragraph in this section states “Program management services include services related to promotion of the program, tracking and workflow, and payouts”.

Question/Comment

Does the vendor have to specifically publicly disclose tracking, workflow and payout?

Phase 2 Price Evaluation

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 5.2. - Phase 2 Price Evaluation - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#52---phase-2-price-evaluation
it states “Prices shall be submitted via the Price Evaluation Form” there is a link to the price evaluation form.

Question/Comment

How can vendors get access to this page? When you click on it you receive the following message: "You can't respond to RFQ Pricing Response Form - Bug Bounty because you don't have permission to share documents outside of your domain. Contact your domain administrator if you think this is a mistake”.

Phase 2 Price Evaluation

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 5.2. - Phase 2 Price Evaluation - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#52---phase-2-price-evaluation and https://www.acquisition.gov/far/html/52_217_221.html#wp1135887
it states “Evaluation of options under FAR 52.217-8” Within FAR 52.217-8, in section 52.212-5(e)”

Question/Comment

Are the flow-down FAR clauses set forth under 52.212-5(e) apply to the researchers participating in the bug bounty research project?

Requirements on Disclosure of Researchers

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 3.0 - Requirements on disclosure of researchers- https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#30-requirements
Within Bounty Pool Management under sub bullet four it states - “Forward to TTS the vulnerability reports, the names of the researchers, and the award amounts.”

Question/Comment

Would the government require the name of the researcher if the vendor provides protection for the researchers and considers this information confidential and provides confidentiality assurances for researchers?

Addendum

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 12 - Addendum - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#120-attachments https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/Addendum%20-%20Commercial%20Contract%20Clauses.md
Commercial Contract Clauses - it states reviewing the GSA IT Security Procedural Guide 17-75.

Question/Comment

Can the vendor have the GSA IT Security Procedural Guide 17-75 disclosed prior to the RFQ submission for review?

Vulnerability Reports

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 3.2.1 - Vulnerability Reports. https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#321-vulnerability-reports
It states “The contractor will submit through their security disclosure platform vulnerability reports for those on the TTS application list. These vulnerabilities will be triaged and classified based on the severity of the vulnerability before being submitted to TTS.”

Question/Comment

Does the 1 business day requirement require that from the disclosure of vulnerability discovery to the vendor include triage and providing a complete vulnerability report including remediation steps to the vulnerability and submit the entire report TTS?

RFO response deadline

Question/Comment on TTS Bug Bounty draft RFO

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFO documents

N/A

Question/Comment

When do we need to submit the response to the RFO by? No possible date is mentioned, but any estimates on a deadline would be helpful for planning purposes.

Phase 1 Technical Evaluation

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 5.1 - Phase 1 Technical Evaluation - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#51---phase-1-technical-evaluation
The Technical Evaluation Google Form - contains three questions. Each question has a field to provide answers.

Question/Comment

Can the government provide any limitations imposed within this form? I.e. What are the maximum character lengths for submitting documentation on the form for each section?

Elaborate on "all costs paid directly to the researchers"

Question/Comment on TTS Bug Bounty draft RFO

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFO documents

Section 4.0 Type of Contract

Question/Comment

Can you please elaborate on what you mean by "all costs paid directly to the researchers"?

Requirement Metrics

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack

Section of RFQ documents

RFQ Section 3 - Requirements. https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#30-requirements
It states “The contractor will provide a Software-as-a-Service platform, with a publicly-available website, for researchers to report security vulnerabilities on publicly available government websites in a manner consistent with the TTS vulnerability disclosure policy.”

Question/Comment

Does the vendor have to disclose the following information based on the 2017 Solicitation under the technical_file.yaml under Service_Platform_Metrics::

• The number of security researchers on the SaaS platform?
• The number of companies using the platform for bug bounty?
• Average times for triage an initial vulnerability report?
• Average times for responses of researcher questions and follow ups?

FedRAMP impact level

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFQ documents

Addendum - Commercial Contract Clauses
Low Impact Software as a Service (LiSaaS) – IT Security and Privacy Requirements
https://github.com/18F/tts-buy-bug-bounty/blob/c0f3f6f4ad32be445694b45933621fb78da13c9f/2018-procurement/Addendum%20-%20Commercial%20Contract%20Clauses.md#low-impact-software-as-a-service-lisaas--it-security-and-privacy-requirements

Question/Comment

Even though security vulnerability data is in-scope, only FedRAMP Tailored LI-SaaS or FedRAMP Low is required in order to meet the FedRAMP compliance requirement (within 1-year after contract date), correct? Just want to make sure we understand the impact level required for this project, as per FIPS PUB 199.

Phase 1 Technical Evaluation Platform Requirements

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 5.1 - Phase 1 Technical Evaluation platform requirements - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#51---phase-1-technical-evaluation
it states in sub bullet two “Maintaining a reliable, secure bug bounty SaaS platform.

Question/Comment

Can the government define the requirements the solution must meet in order to be compliant with the reference of Maintaining a reliable, secure bug bounty SaaS platform”?

Impact Reports

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 3.2.2 - Impact Reports https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#322-impact-reports
It states “The contractor shall be responsible for providing timely notification to the CO/COR and the TTS Product Owner when activities or issues outside of the contractor’s control may directly impact the contractor’s performance.”

Question/Comment

Can the government specify what the desired time frame is for this notification?

Background on Researchers

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 2.0 - Background - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#20-background
Third paragraph in this section states “The larger the community of security researchers in the Bug Bounty SaaS Platform provider’s network, the better the chance TTS has of finding bugs and technical issues within their web applications.”

Question/Comment

Specific to the network of security researchers, can the government confirm they are expecting quality over quantity?

Is there an expectation that allowed researchers have been properly vetted for trust and skill prior to being included in any test?

Key Personnel

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 3.1 - Key Personnel - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#31-key-personnel
point number one it states “The contractor shall provide a Technical Account Manager (TAM) as the primary point of contact for the government’s program office to enable timely problem resolution, reporting in a timely manner, and properly aligning staffing requirements. The contractor will be expected to work with the CO/COR and the TTS Product Owner.”

Question/Comment

Does the technical account manager (TAM) need to have Public Trust, or go through the SF85P process?

Quality Assurance

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 3.2.4 - Quality Assurance - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#324-quality-assurance
within the Quality Assurance Surveillance Plan form - Section 2.0 - Standard - it states “The contractor shall perform all work required in a satisfactory manner in accordance with the requirements of the PWS” and further in section 2.3 - Acceptance of Services - it states “Acceptance of services shall be based upon compliance with performance standards described in the PWS”.

Question/Comment

The 2018 solicitation does not make reference to an existing or award. Can the PWS for the 2018 RFQ be provided?

Response length limit for technical evaluation

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFQ documents

RFQ
5.1 - Phase 1 Technical Evaluation
https://github.com/18F/tts-buy-bug-bounty/blob/d6dcb9f669d4da26ed1488dd275e3325bbbbc729/2018-procurement/RFQ.md#51---phase-1-technical-evaluation

Question/Comment

Is there a response length limit for the technical evaluation?

Requirements Pricing

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 3.0 - Requirements https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#30-requirements
Within Bug Bounty pool management - under sub bullet three it states “Once classified and deemed within the scope of the vulnerabilities, the vendor will manage payout to the reporter based on the agreed up bounty reward tiers by the contractor and TTS”.

Question/Comment

Can the vendor/contractor manage the payout directly without TTS when a Firm Fixed Price Model is used?

Where to submit responses to questions for potential vendors

Question/Comment on TTS Bug Bounty draft RFO

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFO documents

Non-Applicable: Guideline to submit responses to questions for potential vendors.

Question/Comment

Should we submit answers to the questions to potential vendors in an issue via GitHub (due date 01/30) or to the provided email address to the TTS contracting officer?

FedRAMP assessor

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Reed Loden
Director of Security
HackerOne, Inc.

Section of RFQ documents

Addendum - Commercial Contract Clauses
Low Impact Software as a Service (LiSaaS) – IT Security and Privacy Requirements
https://github.com/18F/tts-buy-bug-bounty/blob/c0f3f6f4ad32be445694b45933621fb78da13c9f/2018-procurement/Addendum%20-%20Commercial%20Contract%20Clauses.md#low-impact-software-as-a-service-lisaas--it-security-and-privacy-requirements

Question/Comment

Is it expected that we use an external 3PAO for FedRAMP assessment, or would the GSA be our independent assessor?

FedRamp Certification

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 12 - Addendum - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#120-attachments https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/Addendum%20-%20Commercial%20Contract%20Clauses.md
The Commercial Contract Clauses document calls for the vendor to obtain FedRamp certification for their platform.

Question/Comment

Can the government confirm the type of certification that is expected (i.e. PaaS, SaaS)?

Is it the intent of the government to sponsor the vendor in their certification?

Is there any other support provided by the Government for the vendor throughout this process?

Quality Assurance Surveillance Plan

Question/Comment on TTS Bug Bounty RFQ

Name and affiliation

Brett Kozisek
Director
Synack Inc.

Section of RFQ documents

RFQ Section 12 - https://github.com/18F/tts-buy-bug-bounty/blob/master/2018-procurement/RFQ.md#120-attachments
Quality Assurance Surveillance Plan QASP Section 2.4 - it states “This document specifies all contractor key personnel, employees, and subcontractors shall execute the attached non-disclosure statement.”

Question/Comment

If the vendor already requires all participants to sign their own Non-Disclosure Agreement can this be used in-lieu of the Government NDA from a government review of the vendors NDA?

Can the government please identify where to find the Government NDA?