ossf / wg-metrics-and-metadata

The purpose of the Metrics & Metadata (formerly Identifying Security Threats) working group is to enable stakeholders to have informed confidence in the security of open source projects. We do this by collecting, curating, and communicating relevant metrics and metadata from open source projects and the ecosystems of which they are a part.

Metrics and Metadata in Open Source Projects

The purpose of this working group is to enable stakeholders to have informed confidence in the security of open source projects. We do this by collecting, curating, and communicating relevant metrics and metadata from open source projects and the ecosystems of which they are a part.

Motivation

Open source software is an essential part of modern software development, and of practically all technology solutions. Adoption of open source software has grown over the past two decades, powering everything from tiny "Internet of Things" devices to the most advanced supercomputers in the world. This has led to enormous productivity gains, allowing software engineers to focus more on solving business problems and less on creating and re-creating the same building blocks needed in many situations.

With these benefits, however, comes some risk. Attackers frequently target open source projects and the ecosystems they are a part of in order to compromise the organizations or users that use those projects. It's essential that we understand these threats and work to build defenses against them.

Objective

Our objective is to enable stakeholders to have informed confidence in the security of open source projects. This includes identifying threats to the open source ecosystem and recommending practical mitigations. We will also identify a set of key metrics and build tooling to communicate those metrics to stakeholders, enabling a better understanding of the security posture of individual open source software components.

Scope

The scope of this working group includes "security", as opposed to privacy, resiliency, or other related areas. We also consider the broad open source ecosystem, as opposed to focusing exclusively on critical open source projects.

Active Projects

• Security Insights - Provides a mechanism for projects to report information about their security practices in a machine-readable way.

  • Lead: Luigi Gubello

• Security Risk Dashboard - This project's purpose is to collect, organize, and provide interesting security metrics for open source projects to stakeholders, including users.

  • Lead: Jay White

• Security Reviews - This repository contains a collection of security reviews of open source software.

• Threats, Risks, and Mitigations in the Open Source Ecosystem

Get Involved

• Please get involved with our specific projects, e.g,.
• Mailing List and Security Reviews. (Manage your subscriptions to OpenSSF mailing lists)
• OpenSSF Community Calendar
• Join us on Slack

Related Work

• OpenSSF Best Practices Badge Program - an input to the metrics dashboard generated by the Security Metrics project (formerly named CII Best Practices Badge Program).

• OpenSSF Scorecard - another input to the metrics dashboard

• CHAOSS - develops definitions of metrics

Quick Start

The best way to get started is to simply join a working group meeting. You can also read our Meeting Minutes to get up to speed with what we're up to.

Meeting Times

• We meet every other week on Wednesdays. See the OpenSSF Community Calendar.

Meeting Notes

Meeting Minutes If attending please add your name, and if a returning attendee, please change the color of your name from gray to black.

Antitrust Policy Notice

Linux Foundation meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

Examples of types of actions that are prohibited at Linux Foundation meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at http://www.linuxfoundation.org/antitrust-policy. If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation.

Governance

The CHARTER document outlines the scope and governance of our group activities.

The workgroup leads are:

• Michael Scovetta
• Luigi Gubello

wg-metrics-and-metadata's Issues

Update the threat paper with additional threats/commentaries

Some sources:

• Dependency Confusion
• https://daniel.haxx.se/blog/2021/03/30/howto-backdoor-curl/

Change name in charter

Create requirements document for Best Practices Badge, Security Metrics and Security Scorecards

Let's document the goals, customer scenarios and requirements from three separate yet related workstreams and see if it makes sense to pull these together under common infrastructure as time goes on.

Here is a document to begin capturing our thoughts.

https://docs.google.com/document/d/1qoCnDW6eo17pZv09P114D07YpgFD45AEB0lgY8ixOyY/edit#

WG Charter updates

Hello. The OSSF TAC is seeking to get an issue(1) closed out. We want to ensure all working groups have a complete charter.md file and as I reviewed this group's file I noticed a few items that should be addressed please:

• charter community name (replace the template items with WG name)
• date approved - the WG should officially review and approve its charter and note that date
• mission statement - enter the WG's mission into Section 1
• link TSC to contributors under Section 2 (most likely the WG's contributors list - https://github.com/ossf/wg-identifying-security-threats/graphs/contributors)

Thank you for attending to this matter!

(1) - ossf/tac#9

Proposal: Move CII Best Practices Badge to this "Identifying Security Threats" WG

I propose that the "CII Best Practices badge" project be moved into this "Identifying Security Threats" WG as a sub-project, because this WG has a strong focus on metrics. I would continue to maintain the badge project on behalf of this WG. I also propose that changes to the criteria be coordinated between this WG and the best practices WG. Finally, I propose that the next WG meeting vote on this.

As many of you know, the CII Best Practices badge (website, repo) identifies a set of best practices for open source software (focusing on security) and provides badges to projects meeting various criteria. There are over 3,300 particiapting projects and over 400 projects with a badge.

The badging project was created as part of CII, but CII had a 3-year term that has since expired. The LF has continued to fund some work, such as the Best Practices badge work, because it seemed desirable to keep them going. Now that the OpenSSF exists, it seems reasonable to move such projects into the OpenSSF if the OpenSSF wants them. The OpenSSF TAC (which met today) seemed to think it was reasonable, but wanted to make sure that the "receiving" WG was okay with it. I know of nothing that prevents the LF from transferring it to some OpenSSF WG, but that WG must be okay with it!

One complication is that there are really two WGs that would be sensible OpenSSF homes for the CII Best Practices badge: The Best Practices (including education) WG and the Identifying Security Threats (including metrics) WG. I think it's important that it have a single home, and this issue proposes moving it to the "Security Threats" WG because of its metrics focus. However, it's very important that both WGs coordinate. Therefore, I recommend that proposed criteria changes be voted on by both groups (combined), to ensure that everyone's viewpoints are considered. (An alternative would be to move the CII Best Practices Badge into the Best Practices WG, and then both groups vote together. But no matter what, I think it's important to ensure both WGs work together on this, no matter what form it takes.)

I don't believe this directly affects the OpenSSF budget. It takes some money to keep the website running and do occasional maintenance (updating vulnerable libraries, responding to GDPR requests, etc.). But it's relatively small, so I believe the LF will just continue to fund it at that level. If there's a significant increase in effort (e.g., a huge new scope for the project), then that would need a separate discussion.

Alternative approaches welcome!

@kaywilliams @rhaning - I believe this issue, along with others I'm filing, meets the assignment to me at the TAC meeting today 2020-09-22.

Note that this issue partially supports Strategy committee #8 and TAC issue #26, and is consistent with the proposal to the GB that I developed on how to integrate CII work into the OpenSSF should the OpenSSF choose to do so.

Metadata need by Ortelius

Attribute	Implmented?
Security Insights Verified
Open Source Project (Y/N)
Open Source Foundation (CNCF, Apache, CDF)
License File
Readme File
OWNERS File
MAINTAINERS File
SECURITY File
Governance Doc
Contributor Doc
Code of Conduct
CLA Required
CLA URL
OpenAPI/Swagger
Repository Access Definitions as Code
Project Contact
Project Website
Project Issue Tracking
Project Documentation
Security Contact
Harassment Reporting Contac
Git Repo 2FA
SCM Repo Type
SCM Repo Url
CodeQL
Dependency Tool (Dependabot, Renovate)
Build SBOM Generation
Post Build SBOM Generation
SBOM File
SBOM Signing
SBOM Signing Method
SBOM Signing Public Key
SBOM Signing Valid
Artifact Publishing Location (PURL)
Artifact Mirrors
Artifact Signing
Artifact Signing Method
Artifact Signing Public Key
Artifact Signing Valid
Provenance
Attestation
SonarQube
VeraCode
Linting (Mega/Super Linters)
SAST
DAST
OpenSSF Scorecard

Consider Translating the Threats Document to Markdown

Consider translating the published threats doc to Markdown or another plain text format. That would allow others to contribute and make edits here in the repo.

Update description in .github/settings.yml

@scovetta @rhaning

We will be using the description listed in the .github/settings.yml for each project to create a summary list for the OpenSSF website getting started page and an upcoming press release. Please edit the description field to include a short (1-2 sentence) description. Due Date - Thursday 10/8/2020

See example here:
https://github.com/ossf/gb-planning-committee/blob/master/.github/settings.yml

nullptr NULL denial of service attacks

There's one constant source of crashes, SEGV due to NULL pointer de-references in APIs. They are literally everywhere. eg glibc

#include <stdio.h>
int main()
{
puts(NULL);
return 0;
}

C11 had Annex K functions which avoid many crashes, but annex K has not been adopted widely.

Create "Metrics explainer": Document for each metric how it is computed and how it impacts security

I am seeking to write a "metrics explainer" in which, for each metric on the dashboard, we document things like:

• Security impact: Details about how a given metric relates to the security of software in general; rationale for its' inclusion and interpretation in the dashboard; other context that can help a user make an informed interpretation of a given metric
• Computation of metric: How the metric was obtained or computed, including whether this is something our backend computes (and based upon what input) or something pulled from a third-party source
• Interpretation of scores: What is the potential range of scores for the metric, and how to interpret a score

We will eventually publish this as a resource for users of the dashboard. TBD how we will integrate this (will discuss in a future meeting with @rhaning as we get closer to knowing the overall dashboard UX of the version for initial release)

In terms of next steps:

1. @rhaning / @scovetta can you share (here or by email) a list of metrics which are intended to be present in the initial release?
2. I will create a working draft, and share it here
3. Anyone who would like to help should let me know - currently, the collaborators are me and @mayakacz

Feedback welcome!

Update to the Charter is needed

The Charter needs to be updated to reflect the new goals and aspirations of the new "Metrics and Metadata" working group. The shared google 'working doc' can be found at https://docs.google.com/document/d/13-rJh63bkWBC1xJQ48bkGnpTJTsVKziCez8OqqmzvR0/edit

Structured SECURITY.md - Brainstorming

Based on 7/21/2021 WG meeting, we should consider advocating for a structured security.md file. Not necessarily Markdown, but something that can be introspected and validated.

A small repeated paragraph in the official website

Hi all, I don't know where to report, but I only know Alpha-Omega project is in the work of this group. So I submit an issue here. Let me know if I am wrong.

In the page of Alpha-Omega in the official site, the last paragraph is repeated to the paragraph above.

Omega community members will provide suggestions on how to automate detection of security vulnerabilities in the future and more generally on efficient ways to implement security best practices.

CII Best Practices badge: Improved mechanism for dashboards

FYI, CII Best Practices badge has a new mechanism to make its results easier to integrate in to dashboards:
coreinfrastructure/best-practices-badge#1460

You can easily see the passing criteria here:
https://bestpractices.coreinfrastructure.org/criteria/0

Renaming our working group

We'd like to change the name of our working group to better reflect what we do. Please suggest options and vote for the ones you'd like. We'll stop voting on 12/6.