tooangel / worlddriven

Automatic and well-defined pull request merged based on contribution-based weighted voting

world driven

World driven introduces a contribution based weighted voting system for time-based auto-merges. It defines a process and automates the process of merging Pull Requests based on Reviews.

World driven automatically merges Pull Requests after a certain amount of time, this gives interested people time to review and approve or request changes (in world driven speech: vote). Positive votes reduce the time to merge to speed up the release. Previous contributions are the factor of this merge boost (think of number of commits). Voting against the Pull Request increases the time to merge. Reaching a certain threshold blocks the merge.

The configuration for the repository is:

• Merge duration for a pull request: 10 days
• Per commit time: 0 days
• Merge method: squash

Read more on https://www.worlddriven.org

Setup

Run with docker compose

Copy .env-example to .env and add your environment variables.

docker compose up

Manual

The application uses MongoDB as back end, the default configuration is localhost:27017 This can be overwritten with the environment variable: MONGODB_URI

Gunicorn serves as the server, checkout the Procfile or use gunicorn --workers=1 --worker-class=flask_sockets.worker server:app --chdir src.

The authentication and authorization uses GitHub OAuth and API.

For OAuth and merging of pull requests create an GitHub OAuth App (https://docs.github.com/en/free-pro-team@latest/developers/apps/creating-an-oauth-app) and set GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET as environment variable. The GitHub callback path is: /github-callback.

For commenting create and set a personal token (https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token) as: GITHUB_USER_TOKEN

Set Session secret as SESSION_SECRET initial a random string.

To disable https in a local environment set DEBUG=true as environment variable.

Prepare PostgreSQL server

sudo -u postgres psql
postgres=# create database mydb;
postgres=# create user myuser with encrypted password 'mypass';
postgres=# grant all privileges on database mydb to myuser;

Testing

Run JavaScript tests with: npm run test Run python tests with: pytest

Configuration

To allow different configurations on different repositories you can place a .worlddriven.ini (see .worlddriven.ini-example) in the root of your repository and adapt the values. When World Driven checks new Pull Requests it fetches the .worlddriven.ini from your default branch (fetching from the Pull Request could be a security issue) and applies the configuration.

Front end

The Front end has three views:

• / the front page
• /dashboard a dashboard with an overview of the repositories and a button to enable World Driven for the repository
• /:org/:repo/pull/:pull_number A detailed calculation breakdown for the pull request

Use the following endpoints to more easily work on the dashboard with mock data:

• /test/dashboard - for the dashboard
• /test/:org/:repo/pull/:pull_number - for the pull request view

Production environment

The World Driven auto-merge service is:

• hosted on Heroku
• tested via CircleCI,
• merged via World Driven and automatically deployed.

worlddriven's Issues

500 HTTP error on dashboard

1. I click "Sign in with GitHub" on worlddriven.org
2. I allowed it on my account, but not on an organization.
3. worlddriven.org/dashboard gives me a 500 Internal Server Error.
   This occurs every time I log in to the website, including from other browsers and PCs.

Likely related to:
https://www.worlddriven.org/v1/tooangel/worlddriven/pull/44/ (the example API for a pull request) also returns a 500 error.

Repository documentation

The documentation seems a bit out of date. The API responses can be removed or updated to the current version. Running the application (also for production which is on heroku) is outdated.

Migrate to World Driven organization account

The repository should be moved out of my private GitHub account to the World Driven organization account.

My main blocker: What should be the name? :-)

TooAngel/worlddriven --> worlddriven/worlddriven (nah)
TooAngel/worlddriven --> worlddriven/auto-merger (better)

Any ideas?

Cookie message and approval

AFAIK before cookies are stored a message needs to be displayed and the user needs to approve that they allow to store cookies.

My Idea

Have cookie cookies_allowed. If this is not set, show some message box at the bottom:
The message something like:

This sites uses cookies only for session authentication and this message [OK Button]_

When hitting the [OK Button], the cookies_allowed cookie is set.

Before storing the session cookie it is checked if the cookies_allowed cookie is set, otherwise display somehow that this needs to be agreed on first.

There are some libraries and services out there which does that, I don't think the complexity is very high, so I would do it on my own.

Weighings

Suggestion: add author's commit weighting, otherwise total is not achievable, as authors can't review their own PRs.

Second suggestion: require at least 1 reviewer, so 2 people will have seen the code.

Suggestion: Add optional REVIEWERS file

Rather than implicitly determining who should review pull requests, perhaps it's better to have a REVIEWERS file that lists who wants to review code out of all the collaborators.

Use case 1: Someone contributed in the past but no longer wants to or have time to contribute/review now. They can remove themselves from REVIEWERS.

Use case 2: New people who contribute just a couple times don't block the review process since they likely don't understand the code anyways.

Use case 3: Only experienced contributors review code, ensuring a higher code quality, and new contributors can add themselves to REVIEWERS when they feel ready instead of immediately whether they want to or not.

Add imprint and data security

AFAIK a website need to have an imprint and a data security policy.

Anything else?
What needs to be in there?

Added hours to post-review incorrectly

Didn't spot it until I saw the latest post-review message and not sure how I did it - I suspect it was when I redid the changes to get everything into one commit after the rebase went weird - but the floor division is being done to pr.days_to_merge as oppose to pr.days.merge.seconds. This change, which I will submit in a moment, should fix it. My apologies.

Increasing code coverage

Only basic parts of the code are test covered. Shouldn't be too difficult to increase. Including a code coverage in the CI server is a plus.

Do not write a comment in Draft pull requests

When opening a Draft Pull Request the world driven message is commented with a time to merge.
This message should only happen on non Draft Pull Request creation, or changing the status from Draft to Open.

Admin area

To be able to maintain the World Driven auto-merge certain 'admin' functionality needs to be properly exposed.
I started already with /admin/logs which shows the logs from heroku.

My idea

Adding an admin link to the footer which opens a view that shows some statistics like a graph with requests, the current logs from heroku and everything else we need to maintain the service without superpowers.
Everything needs to be read-only of course, because we only do changes via Pull Request.

Caching of pull request data

The frontend loads pretty slow, especially because a lot of data needs to be fetched from the github api.

Instead of fetching and calculating the data on each request, the data can be easily cached in the database backend. On webhooks and the regular worker this cache can be updated.

Improve Logo

The current logo is more thought of as a placeholder.

It would be nice to have a better logo, the logo should be done as svg and should be either derived from the name (as the current one) or somehow visualize the idea.

Adding a config file for repository based configuration

Currently the configuration (time per commit, auto merge without reviews, ...) is hardcoded. Different projects have different needs.
A config file which is read from the master branch could allow different projects to have different configurations.

Automatic closing of stale Pull Requests

The current logic only merges Pull Requests after a certain amount of time.

I think the same logic could be applied for closing Pull Requests. So if the majority of votes (Request changes) are against merging the PR, the PR should be closed at some point in time. Maybe doing the same type of calculation but inverted and also multiplied by some factor (maybe 10). When that point in time is reached the PR gets closed automatically.
I think the factor makes sense, to give the contributor time to improve on their PR.

E.g. Instead of 10 days time to merge, we would have 100 days time to close. If 80% of the votes are against that PR, it will be closed in 20 days.

Update the status check more often and write a comment

When a github webhook is triggered the github commit status should be updated and a comment should be written to keep the contributor informed.

When a PR is created a comment is already created, but the status check can be set, too.
When a Review is done a comment is already created, but the status check can be set, too.
When the Pull Request is updated (due to a push / force push), a comment should be written that the time to merge is reset and the status check can be set accordingly.

This will make sure that the information on the Pull Request is more up to date and contributors are better informed about the current status.
...

Post-review worlddrivenbot message doesn't mention the hours until merge

I noticed that the post-review message from worlddrivenbot only mentions the days until merge not the hours. I think adding the hours is a good idea.

I believe this section is the one that need updating:

worlddriven/src/routes/githubWebHook.py

Lines 139 to 143 in cc450b3

	This pull request will be automatically merged by [worlddriven](https://www.worlddriven.org) in {} days, votes {}/{}.
	Check the `worlddriven` status checks or the [dashboard]({}) for actual stats.
	'''.format(pr.days_to_merge.days, pr.votes, pr.votes_total, pr.url))
	return {'info': 'All fine, thanks'}

The hours can be calculated the same way as it is above, i.e math.floor(pr.days_to_merge.seconds / 3600), but we can also get rid of the math import by using pr.days_to_merge.seconds // 3600 instead.

Any thoughts?

Review design and UI

The current UI is very (very very) minimalistic and mostly from a perspective from someone who knows how all of this is working.

A better and more explanatory UI would help a lot.

Enable https on the server

Just use letsencrypt

Include new logo into the frontpage and adapt according

Recently a new logo was proposed and merged (/static/images/[favicon-16x16.png, favicon-32x32.png, globe_logo.svg, logo_world_driven.svg]), these need to be used on the frontpage (/static/index.html). .
The style/colors should be adapted to the color palette https://user-images.githubusercontent.com/44964878/94834270-755dc700-0410-11eb-9b8f-b78775ba4cc9.png

Change from draft to open

Currently, only the Pull Request open date, the commit date and the latest push (pull request update) date is taken in account.

To trick in a Pull Request you can open a Draft PR, wait the time to merge and change to open and it will be merged directly.

=> For the calculations the date when a PR is changed to open needs to be included.

Improve the PR merged message from a new contributor

When a PR request is merged from a new contributor, the comment of the Pull Request should contain more information about the consequences.

Due to the world driven system contributions on a repository also lead to a certain, let's say, responsibility for the progress of the project.

When a PR is merged and the owner does not have any commits on the repository, the message could be something like:

This pull request was merged by [worlddriven](https://www.worlddriven.org).

Thank you {} for contributing. This repository is world driven and is driven by all contributors, now you are part of us, we are glad you are here.
From now on you can speed up or slow down future Pull Requests by reviewing these.

Encrypt the github tokens in the database

The github keys currently are stored in plain text in the database. To make sure these are not accessible if someone gets access to the database, the keys needs to be encrypted with a secret which is available in the application via environment variable.

Should adding time until auto merge per commit be removed

I understand using the number of commits in a pull request as a metric for how complex the PR is and therefore how long reviewers should have to evaluate it but I'm not convinced it is a good idea. Here's some of my thoughts on why:

• Number of commits isn't a great indication of complexity as people develop in different ways: some like to be very granular with their commits while others like to code the entire feature before making a commit.

• It encourages, if not demands, squashes and rebases to get it down to 1 commit which masks the degree of complexity entirely.

• Squashes and rebases can be difficult to get right at first and it can be frustrating having to struggle with this so that an otherwise fine PR can be accepted in a reasonable amount of time. This might discourage future contributions.

• If the branch is already pushed online and someone else is working from it - perhaps it already implements a feature they want to build from but other aspects of the PR aren't done yet - messing around with the history via rebasing complicates things considerably.

Overall, I'm personally not sure that even if the number of commits was an accurate measure of how much time is needed to review the code that it would be worth it having considered and experienced some of the downsides; the fact that it doesn't actually indicate complexity reliably at all means that I think it should be removed.

Thoughts @TooAngel ?

Votes are counted for the merge commits.

Perhaps making a seperate acc for bot with write access would be good?

Discounting merge commits would also work but from the beginning I think the bot should have its own account :)

Add current time to merge in status comment? :)

Github error regarding status check

Some pull requests weren't merged because of following error.
In the GitHub webinterface the status checks were set and green, I will disable the required status check for now.

2020-10-09T06:24:08.666212+00:00 app[web.1]: 2020-10-09 06:24:08,665 PullRequest(title="Add issue templates", number=191)
2020-10-09T06:24:08.666224+00:00 app[web.1]: Traceback (most recent call last):
2020-10-09T06:24:08.666225+00:00 app[web.1]: File "/app/src/PullRequest.py", line 131, in check_for_merge
2020-10-09T06:24:08.666226+00:00 app[web.1]: self.pull_request.merge()
2020-10-09T06:24:08.666227+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/github/PullRequest.py", line 866, in merge
2020-10-09T06:24:08.666227+00:00 app[web.1]: "PUT", self.url + "/merge", input=post_parameters
2020-10-09T06:24:08.666227+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/github/Requester.py", line 319, in requestJsonAndCheck
2020-10-09T06:24:08.666228+00:00 app[web.1]: verb, url, parameters, headers, input, self.__customConnection(url)
2020-10-09T06:24:08.666228+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/github/Requester.py", line 342, in __check
2020-10-09T06:24:08.666228+00:00 app[web.1]: raise self.__createException(status, responseHeaders, output)
2020-10-09T06:24:08.666229+00:00 app[web.1]: github.GithubException.GithubException: 405 {"message": "Required status check \"ci/circleci: test\" is expected.", "documentation_url": "https://docs.github.com/articles/about-protected-branches"}

gitlab integration

Right now only github is integated as VCS.
I think it would be good to integrate gitlab, too.

For user authentication and automatic merge OAuth is used.
For commenting on issues a personal access token is used.

For the calculations it is necessary to access repository, pull request and review data.

Handling of github Issues

The idea of world driven is to remove the owner of a (github) repository, this works out good in regards to Pull Requests and also due to the github issue feature on closing issues, too.

But how to handle new issues? Creating is easy. How about assigning and labels?

First ideas which come to my mind:
As soon as a PR is opened which related to an issue, the contributor is assigned with that issue.

For labels, I don't have a very good idea yet. Best idea so far: Have some kind of keywords: label: bug/unlabel: bug which someone can write and you can vote via 👍 and 👎. Similar to PR merge (maybe faster) the tag is added.

WDYT?

Project documentation

I'm still struggling with how to properly explain the idea behind the project.
The last attempt can be found under https://www.worlddriven.org/ or in the index.html, reviewing this and improving (not only the english) would be a huge help.

Store web sessions in mongodb

Docker / docker-compose setup

To ease up the development environment a container setup would be nice.

The service is using mongodb as a backend, so having a container for that is necessary => docker-compose

For login the service also talks to the github API, best would be if this could be mocked somehow, too (as a bonus)

Update HTML line breaks to paragraph tags

Modern HTML design uses <p> tags instead of <br /> for line breaks. Updating all of the static files in this way should make further styling to the pages easier.

Mock mongodb backend in the test

Currently, the python tests are trying to connect to the MongoDB backend. AFAIK the tests don't use the backend and they shouldn't.
For faster and easier running of the tests, the MongoDB backend needs to be completely mocked.

New Review message buggy

The Pull Request comment when someone reviews a Pull Request is wrong: (#159)

Check the worlddriven status checks or the [dashboard](https://www.worlddriven.org/TooAngel/worlddriven/pull/159) for actual stats. - shouldn't be formatted as code.

The time calculation does not count in the current review. I think the underlying is:
The message is triggered by the github webhook Pull_Request_Review, within that requests the reviews are fetched and the current one is not yet included.
Solution: Extract the necessary data out of the webhook data and include it in the calculations.

Project Visibility

I think the project slowly reaches a point where it can be shown to a bigger audience. (I'm really not good at that)

I used the #hacktoberfest a bit to get some more tracking - that was good

My ideas so far:

1. Reddit post /r/opensource - This needs to be written (maybe as MarkDown added to the repository, so that we can decide via Pull Request)
2. HackerNews post - Prepare via PullRequest, too

Any other ideas?

What are the blocker?

• Migrate project to worlddriven organization #194
• Change mongodb backend #192
• Frontend (#189 satisfies me for the frontpage, dashboard and pull request view need some more love)
• Cookie, Imprint, Data Privacy missing #193 #188
• Caching of data would be nice, because of the load times #184
• Fix Draft bug #161

Anything else?

Check when the countdown should start

E.g. TooAngel/screeps#201 shouldn't happen.

maximum time of

• github syncronize events
• commits date
• PR opened
• label WIP removed events

more?

Change the database backend

The current MongoDB provider mLab will shutdown by 2020-11-01. Another MongoDB addon provider needs to be choose and the database needs to be migrated.

How (I would sketch right now how) it should be done for a real World Driven service

The add-on configuration is somewhere stored in the repository (maybe app.json).
A new Pull Request is created adding the new addon. When this is merged the CI server will read that config file and provisions the new database

A second Pull Request is created indicates that the data needs to be migrated and the system should use the new database afterward. (Maybe with a maintenance page in between, would be cool if we don have to)

A third Pull Request removes the old database and the CI server will make sure the database is removed.

How I would do it now

Yeah, do it manually. (If someone is interested in preparing the proper process, we could do the first steps)

Add a header to the `/dashboard` and the pull request view

The /dashboard and the pull request view should have a small header with e.g.

• A small version of the logo as link to the first page
• The current login status (Either the option to login or to logout)