wospm / checker

A checker for project owners to measure their open source project if welcoming or not.

WOSPM Checker

A checker for project owners to measure their open source project if it is a welcoming project for contributors or not.

---

Table Of Contents

• Introduction
• How To Install And Use
  • Github Action
• To Be Considered
• WOSPM Metrics
  • Metric Rules
  • List of Existing Metrics
  • Badges
• Contributing
• Code of Conduct

---

Introduction

wospm-checker is a commandline tool to measure how an open source project welcomes users and possible contributors. The tool checks the repository against a list of metrics. The metrics are mostly inspired by Github's Open Source Guides.

How To Install And Use

Install the package with composer.

composer global require wospm/checker

You can use --help parameter to how the options and other information of the command.

wospm-checker --help

WOSPM Checker version: 0.0.1
Options:
    --output            The format of output. JSON, READABLE (Default), NO, HTML.
    --verbose           Show the progress or not. (0 => No, 1 => Detailed,
                        2 => Dots)
    --no-colors         Disable the console colors. It is enabled by default.
    --strict            Enable strict mode. The script will have a higher
                        bound to give success code.
    --version           Show version.
    --help              Print this help.

You can check your project by running the wospm-checker command in the root folder of your repo.

/full/path/to/wospm-checker

Github Action

It is also ready on Github marketplace as an action with name "WOSPM Checker Github Action" to use in your pipeline.

To Be Considered

GitHub Rest API Rate Limit ❗❗❗

wospm-checker uses GitHub Rest API to fecth repository information in some of the metric checks. There some limits in using this API. When you use wospm-checker very frequently, you may hit the wall of anonymous limit of the API.

Client error: `GET https://api.github.com/repos/user/repo/labels` resulted in a `403 Forbidden` response:
{
  "message": "API rate limit exceeded for XX.XX.XX.XXX. (But here's the good news: Authenticated requests get a higher (truncated...)

You can use your personal access token to have a bigger rate limit. In order to do this, you need to create a YML file with name .wospm under the root folder of the repository to be checked.

github:
    auth_token: PERSONAL_ACCESS_TOKEN

WOSPM Metrics

WOSPM metrics are measures to make quantitative assessments about the open sourse projects if they are contributor friendly or not. They are not scientific values which are mostly derived from Open Source Guides.

Metric Rules

1. Every metric should check only one simple case
2. Metrics can be dependent to each other (If there is no README, no need to make any check in README content etc.)
3. Every metric should have a unique WOSPMXXX number and a unique title (uppercase and snake-case).

For more information about adding a new metric to the project, please Add New Metric section.

List of Existing Metrics

To see the details of the metrics, click the metric code for detailed document.

Code	Title	Description
WOSPM0001	USING_WOSPM	You are using WOSPM checker. It is a good start.
WOSPM0002	README	Every open source project should have a README file.
WOSPM0003	LICENSE	Every open source project should have a LICENSE file.
WOSPM0004	CONTRIBUTING	Every open source project should have a CONTRIBUTING file.
WOSPM0005	CODE_OF_CONDUCT	Every open source project should have a CODE_OF_CONDUCT file.
WOSPM0006	README_TOC	Every README file should have table of contents list.
WOSPM0007	LINK_TO_CONTRIBUTE	README should have a link to CONTRIBUTING file.
WOSPM0008	LINK_TO_CODE_OF_CONDUCT	README should have a link to CODE_OF_CONDUCT file.
WOSPM0009	README_ADEQUATE	README should have atleast 200 words.
WOSPM0010	INSTALLATION_SECTION	README file should have an installation section.
WOSPM0011	GITHUB_ISSUE_TEMPLATE	You should have issue templates on Github.
WOSPM0012	GITHUB_PR_TEMPLATE	You should have PR template on Github.
WOSPM0013	GITHUB_SHORT_DESCRIPTION	Project should have a short description on Github.
WOSPM0014	GITHUB_TOPICS	Related Github topics should be added to the repository.
WOSPM0015	GITHUB_LABELS	Project should have issue labels.
WOSPM0016	GITHUB_LABELS_USED	Labels should be used to highlight the issues.
WOSPM0017	GITHUB_CUSTOM_LABELS	Creating custom labels is a good practice.
WOSPM0018	GITHUB_LABELS_GFI_HW	good first issue and help wanted labels should exist.
WOSPM0019	GITHUB_CUSTOM_LABELS_USED	At least one custom label should be associated to an issue.
WOSPM0020	GITHUB_LABELS_GFI_HW_USED	good first issue and "help wanted" labels should be used.
WOSPM0021	GITHUB_RESPONSIVENESS	Responsive owners encourage users to be contributors.
WOSPM0022	GITHUB_CUSTOM_LABEL_DESCRIPTON	Custom labels should have descriptions.
WOSPM0023	NO_BROKEN_LINKS_IN_README	No broken links exists in README.
WOSPM0024	CONTRIBUTORS_SECTION	README file should have a contributors section.
WOSPM0025	CHANGELOG	Project should have a changelog published.

Badges

After the check, the checker will generate a overall status for the project. There types of status are considered;

• Perfect: It means that the porject covers 100 percent of the metrics.
• Welcoming: It means that the project covers atleast 90 percent of the metrics.
• Not Ready: It means that the project is not ready to be accepted as Welcoming. The project covers between 50 and 90 percents of the metrics.
• Bad: I means that the project is in very bad status. The coverage is under 50 percent.

The checker generates the badge code of the project at the end of the the execution.

Contributing

See CONTRIBUTING.md for information.

Code of Conduct

See CODE_OF_CONDUCT for information.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Tom Mens 🤔

Mohammad Reza 💻

By all-contributors.

Contributions of any kind welcome!

checker's Issues

Online Checks

Is your feature request related to a problem? Please describe.
There are some metrics that can be derived from actions taken on Github like replying the issues. The checker should be able to query Github.

Describe the solution you'd like
Github API can be used to fetch the metrics. A HTTP client is to be added. composer.json should be updated accordingly.

Describe alternatives you've considered
NA

Additional context
NA

WOSPM0011: Check if there are issue templates for Github

Describe the metric you'd have
Check the .github/ISSUE_TEMPLATE folder for issue templates

$this->code        = "WOSPM0011";
$this->title          = "GITHUB_ISSUE_TEMPLATE";
$this->message = "You should have issue templates on Github.";
$this->type         = MetricType::ERROR;

Additional context
Add any other context or screenshots about the feature request here.

Document How To Add A Metric

For possible future contributions, a how-to document should be ready to explain adding a metric to the checker.

Status and Badge Generation

An overall status (percent of the metrics which are ok) and a Mardown code for the corresponding badge should be generated at the end of the execution.

Presence of changelog

I would suggest that the presence of a changelog in some form is also a criterion that tremendously helps in welcoming users or contributors of open source projects. changelogs can of course take many forms, and there are specific guidelines for it, for example:
https://keepachangelog.com/en/1.0.0/ and even automatic changelog generators, for example https://github.com/skywinder/gitlab-changelog-generator

Contributions And Code Of Conduct Metrics

Metrics WOSPM0004 and WOSPM0005 checks the existence of files. Some projects are using external files correspondingly. WOSPM0007 and WOSPM0008 checks if there are links to the files from README. These checks should be adopted to handle this situation.

WOSPM0020: GITHUB_LABELS_GFI_HW_USED

Having "good first issue" and "help wanted" labels is not enough, they should also be associated with the cases

$this->code        = "WOSPM0020";
$this->title          = "GITHUB_LABELS_GFI_HW_USED";
$this->message = '"good first issue" and "help wanted" should be used.';
$this->type         = MetricType::ERROR;

Additional context
Add any other context or screenshots about the feature request here.

Full Verbose

A new verbose mode to show the metric details is needed.

For example;
While checking the responsiveness, all steps (issue by issue) should be verbosed.

WOSPM0013: Short description on GITHUB

Describe the metric you'd have
Project has a short description on version control system. This issue depends on #19 in order to fetch the description.

$this->code        = "WOSPM0013";
$this->title          = "GITHUB_SHORT_DESCRIPTION";
$this->message = "Project should have a short description on Github";
$this->type         = MetricType::ERROR;

Additional context
NA

Docker-Slim ToC Issue

Docker-Slim ToC Issue
The checker fails in WOSPM0006 - README_TOC of repo Docker-Slim

To Reproduce
Use checker for the repo

Expected behavior
Should not fail as there is a ToC section on README.

Screenshots

WOSPM0025: CHANGELOG

In the discussion done at issue #43, we decided to check the existence of a changelog as a metric.

$this->code        = "WOSPM0025";
$this->title          = "CHANGELOG";
$this->message = "Every open source project should have a changelog published.";
$this->type         = MetricType::ERROR;

Additional context
A changelog is a file which contains a curated, chronologically ordered list of notable changes for each version of a project. To make it easier for users and contributors to see precisely what notable changes have been made between each release (or version) of the project.

Link to https://keepachangelog.com/en/1.0.0/

Metric Dependency

If a metric is dependent to another, It will not checked if the dependency fails. For example if there is no CONTRIBUTING file, there is no need to do checks related to the file.

WOSPM0024 alternative

Is your feature request related to a problem? Please describe.
WOSPM0024 is a good guideline to list contributors, but it is not the only solution. Hence some projects may be welcoming because they provide a list of contributors, but will not be detected as such by the current WOSPM0024.

Describe the solution you'd like
Some projects prefer not to list the list of all their contributors as part of the README file (because it might bee too long, or for maintainability reasons). Instead, they may use a separate file to store the list of past and current contributors. This could for example take the form of a file CONTRIBUTORS.md

Describe alternatives you've considered
I cannot tell if there is a common standard format or name of such a contributors file, for those cases where it is not part of the readme.

WOSPM0024: CONTRIBUTORS

Describe the metric you'd have
A clear and concise description of what you want to happen.

$this->code        = "WOSPM0024";
$this->title          = "CONTRIBUTORS_SECTION";
$this->message = "README should have a contributor section or a link.";
$this->type         = MetricType::ERROR;

Be nice with people that helped you! Some open source projects like gatsby reward contributors with goodies. If you can't afford that, do a public post (on twitter or other platforms) about the contribution and mention the author (here is an example of public thanks). Open a Contributors section in your README to publicly thank them or showcase them on your project documentation or website.

WOSPM Badges

We want to create badges like below so that maintainers can use to show WOSPM status of their projects.

WOSPM0021: GITHUB_RESPONSIVENESS

Describe the metric you'd have
According to the Open Source Survey 2017 by Github, 80 percent of the people thinks that unresponsiveness is a problem encountered in open source.

• Check every issue opened by users (not by owner). If there is no issue opened yet, the metirc will be though to be success.
• Check if all the user issues is answered in 24 hours by owner or any body other than issue creator
• Apply same rule to pull requests

$this->code        = "WOSPM0021";
$this->title          = "GITHUB_RESPONSIVENESS";
$this->message = "Responsive owners encourage users to be contributors.";
$this->type         = MetricType::ERROR;

WOSPM0014: Related topics should be added to the repository

Describe the metric you'd have
At least one topics should be added to the repository. Topics helps people to find your repo.

$this->code        = "WOSPM0014";
$this->title          = "GITHUB_TOPICS";
$this->message = "Related topics should be added to the repository.";
$this->type         = MetricType::ERROR;

Additional context
NA

Add PHP 8 Support

• Check if the command is compatible with PHP 8.
• Fix the issues.

Add Version Info To The HTML and JSON results

Headline To Link is not working correctly.

Describe the bug
Headline to link converter cannot convert the headlines containing Turkish characters correctly.

To Reproduce
Use https://github.com/umutphp/startup-sozlugu for test.

Double Quote Problem

Describe the bug
The " character in the messages and titles creates problem during escaping and json encode/decode.

Expected behavior
Do not use :)

Add cmohammadc for code

@all-contributors please add @cmohammadc for code

WOSPM0007: README has link to CONTRIBUTING file

Check if the README has link to CONTRIBUTING file.

Definition of done;

• Metric code will be WOSPM0007
• Find a link to CONTRIBUTING from README
• It will be an ERROR
• An MD file to explain the metric should be created under mdocs folder using mdocs/WOSPM0000.sample.md template file.

$this->code        = "WOSPM0007";
$this->title          = "NO_LINK_TO_CONTRIBUTE";
$this->message = "Every README file should have a link to CONTRIBUTING file.";
$this->type         = MetricType::ERROR;

Improve WOSPM0010 INSTALLATION_SECTION

Update the metric so that it will pass if there is a link to an installation document.

WOSPM0006: README has a table of contents (ToC) at very top of the file

Check if the README file has ToC at the top of the file.

Definition of done;

• Metric code will be WOSPM0006
• Find a pattern of ToC
• It will be an ERROR
• An MD file to explain the metric should be created under mdocs folder using mdocs/WOSPM0000.sample.md template file.

$this->code        = "WOSPM0006";
$this->title          = "NO_README_TOC";
$this->message = "Every README file should have a ToC.";
$this->type         = MetricType::ERROR;

WOSPM0015: Labels are used to highlight the issues

Describe the metric you'd have
Labels are used to highlight the issues. If you use labels, they give brief information about the issues and your priorities which is very helpful to contributors.

$this->code        = "WOSPM0015";
$this->title          = "GITHUB_LABELS";
$this->message = "Labels should be used to highlight the issues.";
$this->type         = MetricType::ERROR;

There should be labels and at least one label should be assigned to an issue.

Additional context
Add any other context or screenshots about the feature request here.

Complete documentation of existing Metrics

Four metrics are already implemented without documentation under mdocs folder.

• WOSPM0002 - NO_README
• WOSPM0003 - NO_LICENSE
• WOSPM0004 - NO_CONTRIBUTING
• WOSPM0005 - NO_CODE_OF_CONDUCT

Readme

We need to create a good README to represent the WOSPM values :)

Small README but A Good Website

Some of projects (for example Pest) has a detailed web site to publish the documentation containing installation and other sections that we check to find on README files.

Solution

The metrics against README can be applied to the web site of the project if failed on README.

WOSPM0012: Check if there are PR templates for Github

Describe the metric you'd have
A clear and concise description of what you want to happen.

$this->code        = "WOSPM0012";
$this->title          = "GITHUB_PR_TEMPLATE";
$this->message = "You should have PR on Github.";
$this->type         = MetricType::ERROR;

Additional context
Add any other context or screenshots about the feature request here.

WOSPM0022: GITHUB_CUSTOM_LABEL_DESCRIPTON Custom labels should have description

Using custom label is a good practice but you should describe the custom tags.

$this->code        = "WOSPM0022";
$this->title          = "GITHUB_CUSTOM_LABEL_DESCRIPTON";
$this->message = "Custom labels should have description.";
$this->type         = MetricType::ERROR;

Use thephpleague/climate For Console Coloring

thephpleague/climate

nette/safe-stream Project As Tester

Bug Description

When we try to calculate the metrics for nette/safe-stream project, we see the following output.

To Reproduce

Clone the nette/safe-stream repo and execute the checker.

Expected behavior

• WOSPM0003 - LICENSE check should not fail. As there is already a license.
• WOSPM0010 - INSTALLATION_SECTION check should not fail.
• WOSPM0012 - GITHUB_PR_TEMPLATE check should not fail.

Use All-Contributors

WOSPM0023: NO_BROKEN_LINKS_IN_README

README documents can mostly contain internal or external links. We should check if all the links are valid and existing.

$this->code        = "WOSPM0023";
$this->title          = "NO_BROKEN_LINKS_IN_README";
$this->message = "No broken links exists in README";
$this->type         = MetricType::ERROR;

HTML Output

HTML output may be needed.

Place of the files

Describe the bug
CONTRIBUTING and CODE_OF_CONDUCT files may be under folder other than root folder. Possible candidates are docs and .github folders. Please check Adding a CONTRIBUTING file

To Reproduce
Steps to reproduce the behavior:

1. Move one of the files to under .github or docs folders (create if not exists).
2. Execute the checker.

Expected behavior
The metric will be ok if files are under folders other than root folder.

WOSPM0008: README has link to CODE_OF_CONDUCT file

Check if the README has link to CONTRIBUTING file.

Definition of done;

• Metric code will be WOSPM0008
• Find a link to CODE_OF_CONDUCT file from README
• It will be an ERROR
• An MD file to explain the metric should be created under mdocs folder using mdocs/WOSPM0000.sample.md template file.

$this->code        = "WOSPM0008";
$this->title          = "NO_LINK_TO_CODE_OF_CONDUCT";
$this->message = "Every README file should have a link to CODE_OF_CONDUCT.";
$this->type         = MetricType::ERROR;

Check if README has Installation section

$this->code        = "WOSPM0010";
$this->title          = "NO_INSTALLATION_SECTION";
$this->message = "Every README file should have a how to install section.";
$this->type         = MetricType::ERROR;

Exit Level Option

Currently, the exit code of the comment is 1 (fail) if only the project covers less than 70 percent which means that some of the metrics can fail.

It is better if there is an cli option to make it strict to 100 percent or maybe a less value if needed.

WOSPM0019: GITHUB_CUSTOM_LABELS_USED

Describe the metric you'd have
Creating custom labels is not enough, they should also be used in issues.

$this->code        = "WOSPM0019";
$this->title          = "GITHUB_CUSTOM_LABELS_USED";
$this->message = "At least one custom label should be associated to an issue.";
$this->type         = MetricType::ERROR;