ossf / allstar

GitHub App to set and enforce security policies

Allstar

Overview

• What Is Allstar?

What's new with Allstar

• whats-new.md

Disabling Unwanted Issues

• Help! I'm getting issues created by Allstar and I don't want them!

Getting Started

• Background
• Org-Level Options
• Installation Options
  • Quickstart Installation
  • Manual Installation

Policies and Actions

• Actions
• Policies

Advanced

• Configuration Definitions
• Example Configurations
• Run Your Own Instance of Allstar

Contribute

• Contributing

---

---

Overview

What is Allstar?

Allstar is a GitHub App that continuously monitors GitHub organizations or repositories for adherence to security best practices. If Allstar detects a security policy violation, it creates an issue to alert the repository or organization owner. For some security policies, Allstar can also automatically change the project setting that caused the violation, reverting it to the expected state.

Allstar’s goal is to give you finely tuned control over the files and settings that affect the security of your projects. You can choose which security policies to monitor at both the organization and repository level, and how to handle policy violations. You can also develop or contribute new policies.

Allstar is developed as a part of the OpenSSF Scorecard project.

What's new with Allstar

Disabling Unwanted Issues

If you're getting unwanted issues created by Allstar, follow these directions to opt out.

Getting Started

Background

Allstar is highly configurable. There are three main levels of controls:

• Org level: Organization administrators can choose to enable Allstar on:
  • all repositories in the org;
  • most repositories, except some that are opted out;
  • just a few repositories that are opted in.

These configurations are done in the organization's .allstar repository.

• Repo level: Repository maintainers in an organization that uses Allstar can choose to opt their repository in or out of organization-level enforcements. Note: these repo-level controls are only functional when "repo override" is allowed in the org-level settings. These configurations are done in the repository's .allstar directory.

• Policy level: Administrators or maintainers can choose which policies are enabled on specific repos and which actions Allstar takes when a policy is violated. These configurations are done in a policy yaml file in either the organization's .allstar repository (admins), or the repository's .allstar directory (maintainers).

Org-Level Options

Before installing Allstar at the org level, you should decide approximately how many repositories you want Allstar to run on. This will help you choose between the Opt-In and Opt-Out strategies.

• The Opt In strategy allows you to manually add the repositories you'd like Allstar to run on. If you do not specify any repositories, Allstar will not run despite being installed. Choose the Opt In strategy if you want to enforce policies on only a small number of your total repositories, or want to try out Allstar on a single repository before enabling it on more.

• The Opt Out strategy (recommended) enables Allstar on all repositories and allows you to manually select the repositories to opt out of Allstar enforcements. You can also choose to opt out all public repos, or all private repos. Choose this option if you want to run Allstar on all repositories in an organization, or want to opt out only a small number of repositories or specific type (i.e., public vs. private) of repository.

	Opt Out (Recommended) optOutStrategy = true	Opt In optOutStrategy = false
Default behavior	All repos are enabled	No repos are enabled
Manually adding repositories	Manually adding repos disables Allstar on those repos	Manually adding repos enables Allstar on those repos
Additional configurations	optOutRepos: Allstar will be disabled on the listed repos optOutPrivateRepos: if true, Allstar will be disabled on all private repos optOutPublicRepos: if true, Allstar will be disabled on all public repos (optInRepos: this setting will be ignored)	optInRepos: Allstar will be enabled on the listed repos (optOutRepos: this setting will be ignored)
Repo Override	If true: Repos can opt out of their organization's Allstar enforcements using the settings in their own repo file. Org level opt-in settings that apply to that repository are ignored. If false: repos cannot opt out of Allstar enforcements as configured at the org level.	If true: Repos can opt in to their organization's Allstar enforcements even if they are not configured for the repo at the org level. Org level opt-out settings that apply to that repository are ignored. If false: Repos cannot opt into Allstar enforcements if they are not configured at the org level.

Installation Options

Both the Quickstart and Manual Installation options involve installing the Allstar app. You may review the permissions requested. The app asks for read access to most settings and file contents to detect security compliance. It requests write access to issues and checks so that it can create issues and allow the block action.

Quickstart Installation

This installation option will enable Allstar using the Opt Out strategy on all repositories in your organization. All current policies will be enabled, and Allstar will alert you of policy violations by filing an issue. This is the quickest and easiest way to start using Allstar, and you can still change any configurations later.

Effort: very easy

Steps:

1. Install the Allstar app
   1. Open the installation page and click Configure
   2. If you have multiple organizations, select the one you want to install Allstar on
   3. Select "All Repositories" under Repository Access, even if you plan to disable Allstar on some repositories later
2. Fork the sample repository
   1. Open the sample repository and click the "Use this template" button
   2. In the field for Repository Name, type .allstar
   3. Click "Create repository from template"

That's it! All current Allstar policies are now enabled on all your repositories. Allstar will create an issue if a policy is violated.

To change any configurations, see the manual installation directions.

Manual Installation

This installation option will walk you through creating configuration files according to either the Opt In or Opt Out strategy. This option provides more granular control over configurations right from the start.

Effort: moderate

Steps:

1. Install the Allstar app (choose "All Repositories" under Repository Access, even if you don't plan to use Allstar on all your repositories)
2. Follow the manual installation directions to create org-level or repository-level Allstar config files and individual policy files.

Policies and Actions

Actions

Each policy can be configured with an action that Allstar will take when it detects a repository to be out of compliance.

• log: This is the default action, and actually takes place for all actions. All policy run results and details are logged. Logs are currently only visible to the app operator, plans to expose these are under discussion.
• issue: This action creates a GitHub issue. Only one issue is created per policy, and the text describes the details of the policy violation. If the issue is already open, it is pinged with a comment every 24 hours without updates (not currently user configurable). If the policy result changes, a new comment will be left on the issue and linked in the issue body. Once the violation is addressed, the issue will be automatically closed by Allstar within 5-10 minutes.
• fix: This action is policy specific. The policy will make the changes to the GitHub settings to correct the policy violation. Not all policies will be able to support this (see below).

Proposed, but not yet implemented actions. Definitions will be added in the future.

• block: Allstar can set a GitHub Status Check and block any PR in the repository from being merged if the check fails.
• email: Allstar would send an email to the repository administrator(s).
• rpc: Allstar would send an rpc to some organization-specific system.

Action configuration

Two settings are available to configure the issue action:

• issueLabel is available at the organization and repository level. Setting it will override the default allstar label used by Allstar to identify its issues.

• issueRepo is available at the organization level. Setting it will force all issues created in the organization to be created in the repository specified.

Policies

Similar to the Allstar app enable configuration, all policies are enabled and configured with a yaml file in either the organization's .allstar repository, or the repository's .allstar directory. As with the app, policies are opt-in by default, also the default log action won't produce visible results. A simple way to enable all policies is to create a yaml file for each policy with the contents:

optConfig:
  optOutStrategy: true
action: issue

The details of how the fix action works for each policy is detailed below. If omitted below, the fix action is not applicable.

Branch Protection

This policy's config file is named branch_protection.yaml, and the config definitions are here.

The branch protection policy checks that GitHub's branch protection settings are setup correctly according to the specified configuration. The issue text will describe which setting is incorrect. See GitHub's documentation for correcting settings.

The fix action will change the branch protection settings to be in compliance with the specified policy configuration.

Binary Artifacts

This policy's config file is named binary_artifacts.yaml, and the config definitions are here.

This policy incorporates the check from scorecard. Remove the binary artifact from the repository to achieve compliance. As the scorecard results can be verbose, you may need to run scorecard itself to see all the detailed information.

Outside Collaborators

This policy's config file is named outside.yaml, and the config definitions are here.

This policy checks if any Outside Collaborators have either administrator(default) or push(optional) access to the repository. Only organization members should have this access, as otherwise untrusted members can change admin level settings and commit malicious code.

SECURITY.md

This policy's config file is named security.yaml, and the config definitions are here.

This policy checks that the repository has a security policy file in SECURITY.md and that it is not empty. The created issue will have a link to the GitHub tab that helps you commit a security policy to your repository.

Dangerous Workflow

This policy's config file is named dangerous_workflow.yaml, and the config definitions are here.

This policy checks the GitHub Actions workflow configuration files (.github/workflows), for any patterns that match known dangerous behavior. See the Security Scorecards Documentation for more information on this check.

Generic Scorecard Check

This policy's config file is named scorecard.yaml, and the config definitions are here.

This policy runs any scorecard check listed in the checks configuration. All checks run must have a score equal or above the threshold setting. Please see the Security Scorecards Documentation for more information on each check.

GitHub Actions

This policy's config file is named actions.yaml, and the config definitions are here.

This policy checks the GitHub Actions workflow configuration files (.github/workflows) (and workflow runs in some cases) in each repo to ensure they are in line with rules (eg. require, deny) defined in the organization-level config for the policy.

Repository Administrators

This policy's config file is named admin.yaml, and the config definitions are here.

This policy checks that by default all repositories must have a user or group assigned as an Administrator. It allows you to optionally configure if users are allowed to be administrators (as opposed to teams).

Future Policies

• Ensure dependabot is enabled.
• Check that dependencies are pinned/frozen.

Example Config Repository

See this repo as an example of Allstar config being used. As the organization administrator, consider a README.md with some information on how Allstar is being used in your organization.

Advanced

Configuration Definitions

• Organization level enable configuration
• Repository Override enable configuration

Secondary Org-Level configuration location

By default, org-level configuration files, such as the allstar.yaml file above, are expected to be in a .allstar repository. If this repository does not exist, then the .github repository allstar directory is used as a secondary location. To clarify, for allstar.yaml:

Precedence	Repository	Path
Primary	.allstar	allstar.yaml
Secondary	.github	allstar/allstar.yaml

This is also true for the org-level configuration files for the individual policies, as described below.

Repo policy configurations in the Org Repo

Allstar will also look for repo-level policy configurations in the organization's .allstar repository, under the directory with the same name as the repository. This configuration is used regardless of whether "repo override" is disabled.

For example, Allstar will lookup the policy configuration for a given repo myapp in the following order:

Repository	Path	Condition
myapp	.allstar/branch_protection.yaml	When "repo override" is allowed.
.allstar	myapp/branch_protection.yaml	All times.
.allstar	branch_protection.yaml	All times.
.github	allstar/myapp/branch_protection.yaml	If .allstar repo does not exist.
.github	allstar/branch_protection.yaml	If .allstar repo does not exist.

Org-level Base and Merge Configuration Location

For org-level Allstar and policy configuration files, you may specify the field baseConfig to specify another repository that contains base Allstar configuration. This is best explained with an example.

Suppose you have multiple GitHub organizations, but want to maintain a single Allstar configuration. Your main organization is "acme", and the repository acme/.allstar contains allstar.yaml:

optConfig:
  optOutStrategy: true
issueLabel: allstar-acme
issueFooter: Issue created by Acme security team.

You also have a satellite GitHub organization named "acme-sat". You want to re-use the main config, but apply some changes on top by disabling Allstar on certain repositories. The repository acme-sat/.allstar contains allstar.yaml:

baseConfig: acme/.allstar
optConfig:
  optOutRepos:
  - acmesat-one
  - acmesat-two

This will use all the config from acme/.allstar as the base config, but then apply any changes in the current file on top of the base configuration. The method this is applied is described as a JSON Merge Patch. The baseConfig must be a GitHub <org>/<repository>.

Contributing

See CONTRIBUTING.md

allstar's Issues

Organization repo name and config... match probot?

create a repository named .allstar in your organization

Many GitHub apps are based on Probot and default to the .github repository for organization config. The default setting of this app requires splitting config into yet another repository.

At a minimum it would be nice if this app allowed specifying the org config location.

It would be nice to mirror https://probot.github.io/docs/best-practices/#configuration.

How to use allstar to actually look into a specific branch in a repository

Hello @jeffmendoza,

Is there a possibility that I can look for a specific branch which using allstar?
Assume i have two different branch ,
Main and dev

Main branch as Security.md file and dev doesnt have.
Does allstar detect this and raise an issue?

Please help me understand

Thanks in advance

New to Allstar

Hello Team,

I am new to allstar and tried to use it in my GitHub repository. I have few questions

1. Is it specific only to the GitHub repo? Does it have a docker image or CLI to use it in local/ other git remote repositories

2. I have configured .allstar folder wiht the specified 4 policy yaml file. I dont see any issue or any process running to indicate a check is running and i need to wait for the result. Where can i see the result?

3. Does allstar use scorecard internally?

Thanks in advance.

Update wording and formatting of Binary Artifacts issue

Proposal is to look like:

(No change to title)

Security Policy violation Binary Artifacts

This issue was automatically created by Allstar

Security policy violation
Security policy Binary Artifacts failed: binaries present in source code

Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see Scorecards Documentation.

Remediation Steps
To remediate, remove the generated executable artifacts from the repository.

Artifacts found

• dummy.dll
• dummy.exe
• dummy.jar
• dummy.so

Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores security best practices for a project. You may wish to run a Scorecards scan directly on this repository for more details.

---

Staging deploy of Allstar for testing, see https://github.com/ossf-tests/.allstar for config. (This is the custom footer)

This issue will auto resolve when the policy is in compliance. (This is current Allstar footer)

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

---

Currently looks like: https://github.com/ossf-tests/scorecard-check-binary-artifacts-e2e-4-binaries/issues/4

Where will issues be submitted?

For the flutter org, we have lots of repos but (with a few small exceptions) we use flutter/flutter/issues for all of them. Where will allstar try to submit issues if I enable that?

Alerting for repos missing dependabot config (`dependabot.yml`)

Similar to the SECURITY.md community health check, for organizations that are interested in having dependabot automated updates enabled on their repositories, it'd be great to have a check that alerts when a .github/dependabot.yml config is not present within the repo.

Ideally, it'd be cool to have this baked into https://github.com/organizations/<org-name>/settings/security_analysis.
cc: @jhutchings1

Allstar Integration on Enterprise version of GitHub

Hello Team,

I have followed your documentation and enabled allstart github app as below.

1. Create .allstar repo under the org as internal (public is not allowed in our enterprise either private or internal)
2. Create all the four config files and opted for scanning all repositories

Do not see any issues being created automatically, Does it support GitHub enterprise with allstar as internal repository type?
Is there anything else to configure to get the issues created in respective repositories?

Thanks

Enforce certain status checks in branch protections

Accept an optional yaml list of status check names for the branch protection policy to enforce repos have enabled an organization's expected checks. The yaml list should accept objects containing at least the fields name and required (to enforce whether a particular status check should be marked as required).

I'm happy to take on this pull request myself if the maintainers approve of the issue.

OptConfig should have a setting "optOutArchivedRepos"

When we enable allstar on the flutter org, it filed about 37 issues complaining that branch protection wasn't enabled on various repos.

All of these were archived, read-only repos. These don't need branch protection, since they have the ultimate protection of being entirely unmodifiable.

Add retry to transport

We get 502 and 504 occasionally from GitHub. These should be retried.

Update Scorecard Interface

After discussing with scorecard folks, we will try to use checker.Runner https://github.com/ossf/scorecard/blob/main/checker/check_runner.go as the interface, update: https://github.com/ossf/allstar/blob/main/pkg/policies/binary/binary.go#L117-L137 to use that.

Policies documentation is incomplete

https://github.com/ossf/allstar#policies links to golang documentation as part of its Yaml documentation. It's quite confusing. I was unable to determine at a glance what form the .yaml files should take and what options were available.

Allow enabling / disabling on all private / public repos

Right now the opt in/out config must list all repos to opt in or out. It would be good to be able to opt in/out all public or private repos.

Feature: require GitHub Actions to be present

Would be great if allstar could verify whether certain GitHub Actions exist and a minimum version.
Often GitHub Actions contain a set of security screenings to block the PR.

bug: typo in RepoConfig struct

it seems that requireAppproval has a typo (should it be requireApproval?)

allstar/pkg/policies/branch/branch.go

Line 79 in 84b402d

RequireApproval *bool `yaml:"requireAppproval"`

SECURITY.md not found under .github/ folder

example issue:

stackables/cloudevents-router-gcp#3

Security Policy SECURITY.md is out of compliance, status:
SECURITY.md not found.
Go to https://github.com/stackables/cloudevents-router-gcp/security/policy to enable.

github docs:
https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository

Show SECURITY.md can be under root, docs, or .github

Allstar frequency

Hello Team,

I am curious to understand the frequency in which allstar scans the repositories to identify security policy violations?

I believe allstar doesn't use webhooks, so how will it know if a push or any action that happens in the GitHub repository?

Thanks in advance

Organization wide SECURITY.md file does not fulfill SECURITY.md policy

If an organization has an organization level SECURITY.md file defined as outline in About default community health files, I would expect that this would satisfy the SECURITY.md policy check.

Instead, an issue was created with a link to create this policy, but this link is actually just to our policy itself, which is defined here.

Support creating JIRA tickets with the findings

It would be great if AllStar supported creating JIRA tickets instead of just GitHub issues, as many projects use JIRA instead.

I might find time to contribute this next year, if no-one else jumps in before then.

Handle 451

https://developer.github.com/changes/19/#--the-451-status-code-is-now-supported
repo cannot be accessed: GET https://api.github.com/repos/...: 451 Repository access blocked []

Looks like repos can be listed, apps installed, but unavailable due to DMCA takedown request. Allstar should catch this abort this repo, and continue enforcing on other repos in the installation.

Currently we are aborting the whole installation/org when hitting this.
https://github.com/ossf/allstar/blob/main/pkg/enforce/enforce.go#L107

Documentation is confusing because of non-existent features being documented

It would be significantly clearer if the documentation only included actual implemented features rather than being aspirational.

For example, "fix" is listed as a possible action, separate from the "Proposed, but not yet implemented actions", but later is revealed to do nothing yet. It would be better to just not list it until it does something.

Allow custom label on raised issue

Allstar always creates it's issue(s) with a gray label called allstar. We use an organization-wide label associated with security / risk that we'd like it to use. I've read through the code and config files and I don't see how to configure allstar to use a custom label, but it seems like something that would be possible based on the way the config works currently.

Thanks for the great tool!

GitHub Issue ping duration should be user configurable.

Currently the bot posts an update to each issue every few days. Since we have around a hundred issues open (primarily because of #108) this causes a huge amount of bugspam to anyone following the repository.

In pondering this I realised that technically the bot is violating our contribution guidelines, which say that a comment shouldn't be added if it doesn't add any new information. In general, if a bug is open, it is taken that it is still valid, and nothing need be said if nothing has changed.

Write DEVELOPER.md

Include Go requirements, how to clone, build, test. Install linter and run lint.

Also include how to setup a test app in GitHub and run locally.

Binary artifact issue should list the actual artifacts

"Allstar has detected that this repository’s Binary Artifacts security policy is out of compliance. Status: Scorecard Check Binary Artifacts failed: binaries present in source code" is what the message says today.

This is not super helpful. It would be nice if the message was more specific about exactly which files it was referring to.

It's unclear how "disableRepoOverride: true" in the allstar.yaml file interacts with policy files

The readme (https://github.com/ossf/allstar) links to a sample config repository whose policy files contain "disableRepoOverride: true". It's not clear from the documentation whether this overrides, combines with, or is overridden by the global "disableRepoOverride: true" in the "allstar.yaml" (which also has it set in that sample repo).

Handle BP on free-tier private repos

Branch Protection is not enabled on free-tier private repos: https://docs.github.com/en/github/administering-a-repository/defining-the-mergeability-of-pull-requests/about-protected-branches
GitHub responds with:
GET https://api.github.com/repos/org/repo/branches/master/protection: 403 Upgrade to GitHub Pro or make this repository public to enable this feature. []
Currently we error out of the whole installation/org.

Change to fail BP policy, notify text should include the message from GitHub "Upgrade to GitHub Pro or make this repository public to enable this feature."

Update linter config

Look at : https://github.com/ossf/scorecard/blob/main/.golangci.yml
See if there are any more linters we want to enable over default. Only include additions to config file necessary over default settings.

Administrator requirement for repo default setting

Forking from: #103 (comment)

My thoughts:

it is also a goal for me to have Allstar's default settings represent the highest level of security best practices that we (OpenSSF) can recommend. ...
That said, I also acknowledge that my point of view is from a large-org perspective, and while the org admins do have transient permissions on all repos, they can't be expected to be responsible for all repos, and a direct admin should be required. I see how this would not be the case for a small org. I'd like to bring this up at the next "Best Practices" WG meeting to discuss further.

In-memory cache could cause OOM

The in-memory cache is great for simple workflows. But for large repositories it usually causes OOM.

A sample repository that was causing OOM was github.com/ApolloAuto/apollo

related issue ossf/scorecard#589

allstar/pkg/ghclients/ghclients.go

Line 83 in 70d3e36

Cache: httpcache.NewMemoryCache(),

This might not be applicable for allstar but wanted to point out the issues it could run into.

Support the ability to define the repository types for security issues

It would be nice to be able to define the repository types to include for the security issues. For example, we might want a SECURITY.md for all public and internal repos, but not care about whether private repos have it or not.

Convert Binary Artifacts policy to be a generic scorecard policy

Should have multiple checks enabled (3-5 to start). Default config will be to run all enabled checks, but can override in config to only run certain ones. Should have a "success value" to decide what score (0-10) is passing.

Preview mode

I administer an org with many, many repositories with varying levels of security (and varying levels of importance). I would love to enable allstar globally, but I am concerned that it will file hundreds of issues on various repos that may be secret, archived, unimportant, or some combination of the above, as well as filing issues that are known to be bogus (for example some of our repos are intentionally for storing binary files so the whole "don't store binary files" thing won't work in those repos).

What I would like is a way to get a preview of what allstar is going to complain about ahead of me doing something with public visibility.

Run scorecard on allstar

Eat your own dog food. Run scorecard on allstar.

Own instance of allstar

Hello @jeffmendoza,

I am trying to create an own instance of allstar.

• I cloned the repository
• build the cmd/allstar
• tried running it locally using the "allstar" command
  Got the below error,
  {"severity":"CRITICAL","error":"open variable gcpsecretmanager://projects/allstar-ossf/secrets/allstar-private-key?decoder=bytes: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.","time":"2021-08-23T15:16:18+05:30","message":"Could not load app secret, shutting down"}
  Is it because allstar can not be run locally?

Include scorecard details in created issue

When creating an issue for scorecard, include the details from:
https://github.com/ossf/scorecard/blob/main/checker/check_result.go#L45
in the issue. Only include DetailWarn and above, only include first ~10 lines and note if some lines were cut.

Administrator requirement for outside collaborators does not seem to work with an admin team

Hi,

We got a recent issue opened using the allstar-app:

"Allstar has detected that this repository’s Outside Collaborators security policy is out of compliance. Status:
Did not find any owners of this repository
This policy requires all repositories to have an organization member or team assigned as an administrator. Either there are no administrators, or all administrators are outside collaborators."

I guess due to this change: 01e4892

However, the repo has a team with admin access, and a member of the org is on this team. Perhaps the change didn't take into account the scenario of an admin team as opposed to a member being an admin directly on the team?

Issues should include footer text with information about opt-in/opt-out strategy configuration

Users who want to opt out of or disable Allstar need to know the strategy configuration (opt-in vs opt-out vs. repo-level), which involves looking in several places to find certain files or settings. Is it possible to add a feature that tells users the configuration in each issue they receive? For example, a line in the issue footer could say:

Allstar is enabled on your repository
Allstar is enabled on your organization in the opt-in strategy
Allstar is enabled on your organization in the opt-out strategy with/without repository override.

Fix the security advisories

https://deps.dev/go/github.com%2Fossf%2Fallstar

Readme doesn't describe priority order for opt-out features

There are various ways to opt repos in and out, including these:

optConfig:
  optOutStrategy: true
  optOutRepos:
  - ...
  optInRepos:
  - ...
  optOutPrivateRepos: true
  optOutPublicRepos: false

...and including a per-repo file and an org-wide .allstar repo file.

It is highly unclear from the documentation what will win if there is a conflict (additionally it seems there's an undocumented feature that changes which will win, as mentioned briefly in opt-out.md, which makes this even more unclear). (edit: found the documentation for this feature)

For example, does optOutPrivateRepos win vs an explicit optInRepos? What if optOutStrategy is true, but optOutPrivateRepos and optOutPublicRepos are also true? What if a repo opts out locally but is listed explicitly in the global .allstar file?

Allow pattern matching for repos on which to `enforceBranches`

It would be great if the enforceBranches configuration for branch_protection.yaml could be set to apply to any repository which matches a pattern.

Example of usage:
In our repositories across our organization we perform our releases from branches which match *.*.x, we would like to enforce our branch protections on multiple repositories for those branches, but would love to have this requirement centralized in our .allstar repository.

You could just include it in the yaml as the branch name:

enforceBranches:
  "*-releasable":
    - "*.*.x"

For our usage we would have:

enforceBranches:
  "*":
    - "*.*.x"

Change "optOutStrategy" to "useOptOutStrategy"

Improved readability.

https://github.com/ossf/allstar/blob/main/pkg/config/config.go#L40

Branch protection out of compliance even tough codereview requirements is set to 1

Dart and Flutter Orgs branch protection requires 1 codereview but we allow some bot accounts to bypass the requirement. This is causing the allstar application to detect that the repositories as out of compliance:

Allstar has detected that this repository’s Branch Protection security policy is out of compliance. Status:
PR Approvals not configured for branch master

Could this be related to ossf/scorecard#1614?

Search .github repo for allstar org level config as a second priority

To allow flexibility, we should allow org-level config in the .github repo and allstar sub-directory. This should be searched only if the .allstar repo does not exist in the org. Note, we must check for repo existance of .allstar, and not just try to get the particular config file (ex: .allstar/foo.yaml), as some config files will not exist on purpose. If the .allstar repo exists at all, then .github should not be used at all.

This will address #27

Handle 410 when trying to create issue

POST https://api.github.com/repos/org/repo/issues: 410 Issues are disabled for this repo
If Allstar is configured to create an issue, but issues are disabled, fail silently, I guess, and continue.

Upgrade to v3 of scorecard

allstar/go.mod

Line 13 in 1c2befa

github.com/ossf/scorecard v1.2.1-0.20210722153731-89c8e2af3131

The scorecard version v3 has more reliable features.

Issues should link to the repositories

Since issues for an org can all be sent to one issue database, it would be nice if the issues themselves linked to the offending repositories, so that the admin didn't have to copy-paste the name into a URL to go to the settings of that repo to fix the issue.

Issues created by allstar-app could be improved

I think that it's crucial to include some actionable information when creating an issue of any kind. This is a follow up from this issue: envoyproxy/java-control-plane#173.

I think that a well created ticket should at least include the following information:

1. Why is this ticked created?
2. Why is the current situation bad?
3. What are the risks?
4. How can it be fixed? (step by step guide)
5. How to know it's fixed.

Let's look at the message in the linked issue:

Security Policy Outside Collaborators is out of compliance, status:
Found 1 outside collaborators with admin access.

In my opinion it does not inform why it's an issue and how to resolve it. A better message would be:

Security Policy "Outside Collaborators" is out of compliance.

This policy requires all users with admin access to be members of the organisation. That way if an account is compromised it can be quickly denied access to all resources of the organisation. To fix this you should:
1. Remove the user from the repository based access by going to settings -> permissions -> ...
2. Add the user to the organisation by going to -> ...

If you don't see the settings tab you probably don't have administrative access. Reach out to the administrators of the organisation to fix this issue.

This issue will auto resolve when the policy is in compliance.

If you have any questions - please reach out to XYZ.

If you have any questions or you'd like to agree/disagree please comment below. Also, I'm creating this issue in good faith, I really think better issue contents will improve this project.

Best regards,
slonka

When is the issue submitted?

I've created a sample repository to see how Allstar works. It was 9PM JST, and an issue is created in 6AM in the next day.
I couldn't find documents about when Allstar app is triggered. Is it once a day?
Can we configure when it's triggered, or manually trigger?

Document necessary access rights for repos

It's not immediately clear to me whether I need to give special rights to any particular user for allstar to be able to see the .allstar repo if it's private, or file issues in private repos. The documentation seems to be silent about this. I'm guessing that apps get magic access to everything somehow?