feature: add a GitHub action to quell spam PRs about express

[UlisesGascon]

This is why I am hesitant do add an action for this as I would rather ask GitHub for better spam management tooling.

Agree with @wesleytodd. The issue is quite complex because the moderation in GitHub is not an easy thing:

• Many people (those that are watching the repo) will receive a notification in their email or in the app/web once a new PR is created. This will occur even before a Github Action is triggered.
• Closing PRs is an easy action (1 click), so that is not a big time consumer
• Detecting invalid PRs is not the big issue because you can easily to spot them with the practice.
• Reporting issues/users is complex because the UI requires many steps to do it for each user, so that is a blocker for many maintainers.
• Nuke button like @CBID2 suggested is great when the community needs to slow down due a discussion/flame in certain moments, but is not the long term solution as it will prevent other users from contributing (PRs) that are legit or need help (issues) while using Express.

So, I think that we are quite limited on how much we can do with GitHub actions in this case. There are other scenarios that are less frequent but more prone to use GitHub Actions to moderate, for example when people do comments that include offensive content or heavily language. In most projects that I am involved the moderation is done by the humans behind the project or a specific team that volunteer to do it, it is a heavy job. The same way as it is hard to keep a slack/discord/gitter community channel a safe space by moderating content.

[wesleytodd]

I agree and we are actively removing their comments and blocking them. I wish that work was not necessary, and I appreciate y'all working to help with suggestions and feedback. We used to have a pretty active triage team in place, maybe we can revive that to be more active, and if we do you all are welcome to help!

[wesleytodd]

Yeah this is a one-off issue of the day. Spam PRs have been a problem for years and they come in different varieties. This is why I am hesitant do add an action for this as I would rather ask GitHub for better spam management tooling.

[krzysdz]

Maybe a pull reqest template would make some people think before creating a PR?
On the other hand, it doesn't look like those who spam with pull requests will even bother to read it and they'll probably just click "Create pull reqest" with unedited description.

[Pratik-Kumar-621]

And if word count check is added, they must find some other way to open spam prs. The problem here is those who make spam prs, they just started their development journey and when they got introduced to github (by some youtubers or some articles), they tried to test it by themselves. They didn't have any idea that the thing they are doing is a headace of someone else.

The only way we can stop this is by creating awarness.

[gaurishhs]

A workflow could be an option too, These spam PRs generally don't have more than 2-3 words. Closing PRs with less than 3 words sounds reasonable. GitHub too does something similar in it's documentation repository. GitHub's workflow

[krzysdz]

unless there is an action which does more than that one (like spam detection with ai or something)

Not sure about AI spam detection, but almost all of these PRs update the readme, have default name (Update filename), change only a single line and have no description, so it should be rather easy to close them automatically.

Most authors of these PRs have a repsitory called "localrepo", so that's another rule that could be used to detect spam from this source (Apna College).

[jonchurch]

Closing as not planned.

Many words have been written now in this issue and others about auto-closing spam PRs, and we have come to the same conclusion every time. It is trivial to implement, but is a sledgehammer approach for what is now a mosquitto level annoyance. Closing and locking a handful of spam issues/PRs now and then is the easiest thing a maitainer will be doing that day, so it doesn't save much by labor.

There is very little benefit in having an auto-close action. It will still spam people's Github notifications no matter if it's closed and locked at the speed of a human or a machine. The only thing we'd prevent there is people commenting disparagingly on the PR before it gets locked by a human, generating more notifications. We haven't had any of that in the past couple weeks (we did get some hostile comments when this topic was trending originally, which it no longer is).

The PRs are still coming, but intermittently. The storm has passed, this can be reevaluated if another flood happens in the future from the previous source or any other.

Thank you all for your eagerness to help. If you want an ear to the ground on things that the project is prioritizing and possibly looking for help with, peruse the Discussions repo https://github.com/expressjs/discussions/issues

[wesleytodd]

I am not sure we need an action for this unless there is an action which does more than that one (like spam detection with ai or something). It is easy to delete/close/lock with as many clicks as it is to comment and we don't need an additional third party action (added security risk and maintenance) to do it right?

[gaurishhs]

I think the workflow should be something like this:

The expressjs member check can be omitted / replaced with a collaborator check

[wesleytodd]

Hey Everyone, lots going on in here but I suggest we take a beat and pause here. I don't think anyone currently on the contributor, triager, or TC roles really want's an action for this (if anyone in those groups disagrees please correct me). Closing PRs from legitimate users is a bad experience, and avoiding that should be a goal which is hard to do with any automated tooling. This idea is more of a "last resort" than an ideal solution.

To update folks on the status:

1. We added Issue templates and will be adding PR templates which will hopefully warn folks against opening spam prs
2. We have reached out through more official ways to the video creators about editing the video to reduce the likelyhood of causing accidental spam
3. Medium term we will likely attempt to get the Triage Team's help for this, but we are not going to do much of that until we sort out a few other things
4. We are continuing to close and locking the spam PRs

All that said, if folks could please STOP directly mentioning maintainers in these PRs (or in any other fashion) that would be great. We are watching the repos and are well aware without the direct ping's, they are only causing more notification spam.

[QuantGeekDev]

I keep getting email notifications so I figured I would contribute something to the conversation. From my reading of the spam PR's, a lot of them add their own name to the commit. Perhaps a regex to detect if the diff consists of only any variation of username, along with the title "Update README.md" would filter at least 60% of the spam PRs. You don't need to use AI for this

[ServerDeveloper9447]

Yep. I agree, too. A pull request template would be best to fight these spam pull requests.

[dougwilson]

They have slowed down too for the time being, the last one already is two days old.

[CBID2]

Hey @wesleytodd. I found this on X: https://twitter.com/github/status/1311772722234560517
Maybe that could work?

[CBID2]

And if word count check is added, they must find some other way to open spam prs. The problem here is those who make spam prs, they just started their development journey and when they got introduced to github (by some youtubers or some articles), they tried to test it by themselves. They didn't have any idea that the thing they are doing is a headace of someone else.

The only way we can stop this is by creating awarness.

True, but knowing that there those who create spam PRs out of malice, raising awareness is not enough. Protective measures are a must too.

[CBID2]

Hey Everyone, lots going on in here but I suggest we take a beat and pause here. I don't think anyone currently on the contributor, triager, or TC roles really want's an action for this (if anyone in those groups disagrees please correct me). Closing PRs from legitimate users is a bad experience, and avoiding that should be a goal which is hard to do with any automated tooling. This idea is more of a "last resort" than an ideal solution.

To update folks on the status:

1. We added Issue templates and will be adding PR templates which will hopefully warn folks against opening spam prs
2. We have reached out through more official ways to the video creators about editing the video to reduce the likelyhood of causing accidental spam
3. Medium term we will likely attempt to get the Triage Team's help for this, but we are not going to do much of that until we sort out a few other things
4. We are continuing to close and locking the spam PRs

All that said, if folks could please STOP directly mentioning maintainers in these PRs (or in any other fashion) that would be great. We are watching the repos and are well aware without the direct ping's, they are only causing more notification spam.

Sounds wonderful @wesleytodd! :) I'm going to join the triage team to offer more ideas

[singhmeharjeet]

Could Github not implement something like stack overflow? (like you need some amount of rating before you could file a PR on a Repo which is a little old, maybe 10-20 commits)

[sudo-Mystic]

A PR template with some automation seems good to me . We can check for the change in description . Most of these student dont care to change description , so if a PR is opened with the template as it is without any change then most probably it is spam and a action or any automation could be used to close it .

[nicholasgriffintn]

Really, I think you could probably just reject anything with "update: ''" as a PR title, I don't think that would be too disruptive for normal PRs.

[ADTC]

Most of them can be auto-closed if they meet all of the below criteria:

1. PR has only one commit.
2. Commit changes only one file.
3. Commit has a message subject matching regex Update [^ ]*\.[^ ]+ (match is for all file names generally).
4. Commit has no message body.

This should auto-close almost all of them. Maybe 1 or 2 every month may not match, but it's much less work to manually check and close them.

PS: The regex match should be on commit subject, not PR title because the newbie spammers are more likely to change the title.

For fun: Close them with a cheeky comment: Congrats! You are now an open source spam contributor. Now learn how to make real contributions and your training will be complete. [Add a Hindi translation of the same.]

[SakuraBlossomTree]

I think we can get auther (who made PR) total lifetime PR count if it is more then 10 or whatever we can prevent that ?

They already made a github action which takes care of this

[TimGels]

I reported a couple users already for spam. But this is just mopping with the tap wide open. I hate how this impacts contributors their time.

[wesleytodd]

In most projects that I am involved the moderation is done by the humans behind the project or a specific team that volunteer to do it, it is a heavy job. The same way as it is hard to keep a slack/discord/gitter community channel a safe space by moderating content.

I think historically the express project has not needed the same style of moderation as things like Node.js does. There have been less contentious discussions and mostly the CoC violations have come from folks outside the project so it was relatively simple to ban and move on. I don't think we need to immediately spin up a moderation team but I do think that the Triage team and TC should have the tools to properly moderate. Right now I don't think we have that well in hand. I believe that we can add this to the list of TODOs to address after we can get next weeks TC meeting organized and finished.

Are we in agreement that a GH Action is most likely not the direction we would want to take to solve this problem?

[UlisesGascon]

I believe that we can add this to the list of TODOs to address after we can get next weeks TC meeting organized and finished.

Yes, we can add it to the TODO list and start to work on it in few weeks

Are we in agreement that a GH Action is most likely not the direction we would want to take to solve this problem?

+1 from me

[wesleytodd]

I do agree that workflow is a bit more well suited IMO. I am still hesitant and I would like to also look at other ways but yeah in the mean time lets re-open the issue so we don't end up having multiple on the same topic or miss out on good ideas like @gaurishhs'.

[deepak4566]

even one more better thing is maximum spam pr revolve around README.md ,if we can specify to block those pr updating readme for now (upto the spam pr's get less) we can handle this through github actions for this so that good pr's may not effect.

[Pratik-Kumar-621]

And if word count check is added, they must find some other way to open spam prs. The problem here is those who make spam prs, they just started their development journey and when they got introduced to github (by some youtubers or some articles), they tried to test it by themselves. They didn't have any idea that the thing they are doing is a headace of someone else.
The only way we can stop this is by creating awarness.

True, but knowing that there those who create spam PRs out of malice, raising awareness is not enough. Protective measures are a must too.

There are Rest APIs for getting the info of prs and also for actions. Making a bot which can automate spam pr closing using those rest apis can help. But not sure how to detect spam.

[CBID2]

I think we should try my suggestion for now and then do more afterwards. We must take some form of action quickly

[Romakita]

Hello team!
A quick action could be to limit who can submit a PR while there is no action to filter PR spamming.

github provide a settings to limit temporarly the submitted PR for a determined duration (maybe one week) for an external contributor:
https://twitter.com/github/status/1311772722234560517
https://github.com/orgs/community/discussions/22804
(in this case only express team member can submit a PR)

I know isn’t aligned with open source phylosophy but it can stop this PR spamming cycle quickly while waiting for a real solution. Maybe when they see that it no longer works, the followers of the video will ask the YouTuber for explanations :)

Another solution, is to check if a PR have a commit related to an open issue on the same repo. If not the github action xill close automatically the PR.

see you

[animesh-chouhan]

@animesh-chouhan The action certainly is a decent idea, however I don't think a simple check of whether a default README.md change commit exists in a PR would be enough to prevent such spam. I feel like we should have a greater set of rules to check against, such as how active the said user is on GitHub, whether they've contributed to different repositories, etc.

@CompeyDev Yeah it is upto the repository owner to configure rules. I have just provide a template and added basic rules just to give an idea. More complex rules can be added depending on the requirement.

[mydeveloperday]

I'm an open source contributor on other projects, I stumbled upon a video explaining what was going on. Unfortunately this project has become part of a video to teach people how to make github pull requests to open source projects (https://www.youtube.com/watch?v=Ez8F0nW6S-w), Do as I did and report this video to try and get it taken down. Whilst not harmful content its damaging "misinformation" after which I gave a description why. (if enough people complain then maybe it can be forced to remove it and that might help, but this video has 1.4M views so be prepared for the number of PRs to keep coming in (pretty much forever and fast)

I'm not a github expert maybe there is a way to prevent submission of the PR ahead of time if it doesn't contain a github issue, or maybe a "bug/enhancement/contributor label" , other projects I've worked on you need to be a member of a "virtual" organization (e.g. expressjs community), but you can easily join that organization by raising a ticket, a moderator can adds people to that orginization. Nothing would stop contributors creating their own commits in their own forks, but they couldn't then send you their PRs unless they were part of your organization.

Sorry just trying to help, feel bad for you all.

name: Prevent PR submission from non-organization members

on:
  pull_request:
    types:
      - opened
      - synchronize

jobs:
  validate_pr:
    runs-on: ubuntu-latest
    steps:
      - name: Check if PR author is organization member
        uses: actions/github-script@v4
        with:
          script: |
            const { owner, repo } = context.repo;
            const { login } = context.payload.pull_request.user;

            const { data: orgMembers } = await github.rest.orgs.listMembers({
              org: owner,
            });

            const isMember = orgMembers.some((member) => member.login === login);

            if (!isMember) {
              core.setFailed(`PR author is not an organization member: ${login}`);
            }
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}```

[CompeyDev]

Heya! An update on my last comment: I think I'll be working on an AI based spam detector for use-cases like this, which would analyze diff files of PRs and conclude whether it is worthwhile to keep or not. Will update with progress on that here if y'all would like that.

[animesh-chouhan]

Heya! An update on my last comment: I think I'll be working on an AI based spam detector for use-cases like this, which would analyze diff files of PRs and conclude whether it is worthwhile to keep or not. Will update with progress on that here if y'all would like that.

You don't need AI for this. How are you going to approach this?

I agree with @QuantGeekDev, detecting this kind of spam is relatively easy and using AI would be an overkill. A simple rule-based approach will work well in this case: https://github.com/animesh-chouhan/github-spamstop/blob/main/index.js#L48-L66

This uses a pull-request template which needs to be edited with the details of the change which would easily deter a large amount of such PRs.

PR template: https://github.com/animesh-chouhan/github-spamstop/blob/main/.github/pull_request_template.md

[ayush7]

Honestly, the only way to qwell this issue is to ask the person who originally posted a github tutorial on youtube to pull down her video and reupload it without the ExpressJS bits in the very end. Otherwise, it may keep happening in the future. Honestly not a big ask. I think the express team should get in contact and ask them to do so. They are a medium size coding tutor platform so it should not be a crazy ask. Their contact and other details are available on their website.

[wesleytodd]

Thanks for the help, we reached out via more formal methods and have not heard anything back afaik so just wanted to check if this had any added context on the situation.

I am going to take a pass pretty soon on this thread to mark a lot of the comments in here as "off topic", so please don't take offense, but this is all really off topic conversation.

[SakuraBlossomTree]

Maybe we can add a PR template when people want to issue a PR

It is probably as this people won't be able to know how to change basic things in the PR template

[iammohan01]

It's kinda scary that, Those idiots spamming PR's

[QuantGeekDev]

I agree and we are actively removing their comments and blocking them. I wish that work was not necessary, and I appreciate y'all working to help with suggestions and feedback. We used to have a pretty active triage team in place, maybe we can revive that to be more active, and if we do you all are welcome to help!

How can we join the triage team?

[krzysdz]

How can we join the triage team?

https://github.com/expressjs/express/blob/master/Contributing.md#becoming-a-triager

[wesleytodd]

Doesn't need to stop the conversation, but since we specifically don't want to use a GHA to do this then this issue is complete. There are more threads to follow up on, but I think we can do that in other discussions more specific to those.

[CBID2]

A workflow could be an option too, These spam PRs generally don't have more than 2-3 words. Closing PRs with less than 3 words sounds reasonable. GitHub too does something similar in it's documentation repository. GitHub's workflow

@wesleytodd, I think you should reopen this issue. @gaurishhs made another point about using GitHub actions

[TimGels]

Is it possible to have the workflow also incorporate a check for new contributors when doing the x words check? I feel like that people who contributed, and already have code merged, in the past do not pose much of a threat in regards to spam. Or am I overthinking it?

[CBID2]

Is it possible to have the workflow also incorporate a check for new contributors when doing the x words check? I feel like that people who contributed, and already have code merged, in the past do not pose much of a threat in regards to spam. Or am I overthinking it?

I don't think you're overthinking it @TimGels. Previous contributors should be spared in some way.

[Pratik-Kumar-621]

I think the workflow should be something like this:

The expressjs member check can be omitted / replaced with a collaborator check

I don't think word count matters because some bugs only requires a minimal changes.

[JavidSumra]

Hello! While I may not possess your level of genius, I've got an idea. Can we utilize NLP in Python code to detect significant changes in a Readme file solely through a GitHub Action?

[Kishlay-notabot]

I really feel pensive seeing all these PRs come from literally "tech youtubers" who don't use basic common senses before uploading a video, I apologize on their behalf 🙏 .

[CompeyDev]

spam detection with ai

This seems like a pretty cool project idea, someone should DEFINITELY put something like this into place to help with spam hell for open-source maintainers.

[animesh-chouhan]

Hello team,

Spent few hours learning about GitHub actions and came up with this rule-based spam PR detection action which automatically closes the PR based on user-configurable rules.

I have setup few rules like detecting the default commit message for README.md and also if the pull request template has been updated with the details. It can be easily extended to prevent this PR havoc and future misuses.

Check it out here: https://github.com/animesh-chouhan/github-spamstop

[CompeyDev]

@animesh-chouhan The action certainly is a decent idea, however I don't think a simple check of whether a default README.md change commit exists in a PR would be enough to prevent such spam. I feel like we should have a greater set of rules to check against, such as how active the said user is on GitHub, whether they've contributed to different repositories, etc.

[CBID2]

So can we get started?

[CBID2]

Heya! An update on my last comment: I think I'll be working on an AI based spam detector for use-cases like this, which would analyze diff files of PRs and conclude whether it is worthwhile to keep or not. Will update with progress on that here if y'all would like that.

I'd like that

[QuantGeekDev]

Heya! An update on my last comment: I think I'll be working on an AI based spam detector for use-cases like this, which would analyze diff files of PRs and conclude whether it is worthwhile to keep or not. Will update with progress on that here if y'all would like that.

You don't need AI for this. How are you going to approach this?

[Usmanzahidcode]

I just stumbled upon a tweet about a problem that seems identical to the Binod virus that surfaced a few years ago. It's high time that GitHub takes notice of this new issue. Repositories like these should have a strict set of criteria or credibility requirements for contributors. It's imperative to understand that libraries like these are fundamental building blocks of the web ecosystem, and such issues only hinder progress.

[matthewelmer]

You could probably block 99% of the spam by checking the IP address from which it came. Dunno how to do that, though.

[prakhar-pal]

You could probably block 99% of the spam by checking the IP address from which it came. Dunno how to do that, though.

Blocking PRs based on IP address isn't a solution because 1. Different people are making these spam PRs so IP addresses are different, 2. If you meant IP range that targets some region, then it can result in legit PR getting blocked.

[ADTC]

We have reached out through more official ways to the video creators about editing the video to reduce the likelyhood of causing accidental spam.

the only way to qwell this issue is to ask the person who originally posted a github tutorial on youtube to pull down her video and reupload it without the ExpressJS bits in the very end.

Probably not going to happen. Based on their post in their Instagram broadcast channel, they feel they did everything right, and they are calling us a bunch of bullies and whiners.

Click here to see a screenshot of the channel.

This is so unfortunate for the open source community.

without substantial changes

One of the spam PRs replaced the entire Readme file with junk text. It is a substantial change.

[wesleytodd]

@ADTC is any of this on a public link? I don't have an Instagram account but would be interested in reading links if they are available. Screenshots just don't seem to provide enough context to interpret what she is saying in that post.

[ADTC]

@wesleytodd there isn't much on the channel related to this topic. She posted a copy of the tweet, made her comment and started the poll. That's all.

Here's the screenshot with the tweet.

You can't access the channel without an Instagram account. Here's a link though: https://ig.me/j/AbagxilRJ7KwLyaD/

PS: last ditch effort would be to create a new repo and lock this repo as read only. Post a notice sign posting the new repo.

[Kamleshpaul]

I think we can get auther (who made PR) total lifetime PR count if it is more then 10 or whatever we can prevent that ?

like this

name: Spam Detect

on:
  pull_request:
    types:
      - opened

jobs:
  check_author_pr_count:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Set up Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '14'

    - name: Install dependencies
      run: npm install github-api

    - name: Count Merged PRs
      id: pr_count
      run: |
        AUTHOR=$(curl -sSL https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }} | jq -r '.user.login')

        MERGED_PR_COUNT=$(curl -sSL -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" "https://api.github.com/search/issues?q=is:pr+author:$AUTHOR+is:merged" | jq -r '.total_count')
        
        echo "merged_pr_count=$MERGED_PR_COUNT" >> $GITHUB_OUTPUT

    - name: Check PR Count
      run: |
        if [[ "${{ steps.pr_count.outputs.merged_pr_count }}" -gt 20 ]]; then
          echo "PR count is greater than 20. Proceed with the workflow."
        else
          echo "PR count is not greater than 20. Blocking the PR."
          exit 1  
        fi

this is it's test https://github.com/Kamleshpaul/github-spam-action-test/pulls

[ServerDeveloper9447]

Well, it looks like the issue will be solved when the pr is merged

[wesleytodd]

Hey folks, just to update you: We have a thread open with GitHub support where they are passing along our requests to the various product teams on how to improve the moderation features. I personally am still 👎 on an action which auto closes PRs as it is both a maintenance burden and something best handled by features on GitHub's side. We are focused right now on landing some governance changes and will be following up with re-constituting the triage team. I think ideally this decision is left up to the triage team members. So likely there will not be action on this or the PR until we can get that team organized again. If you are interested in helping that please follow the instructions to get involved there (as some of you already have, thank you very much for that).

[nkroker]

unless there is an action which does more than that one (like spam detection with ai or something)

Not sure about AI spam detection, but almost all of these PRs update the readme, have default name (Update filename), change only a single line and have no description, so it should be rather easy to close them automatically.

Most authors of these PRs have a repsitory called "localrepo", so that's another rule that could be used to detect spam from this source (Apna College).

The flood of pull requests (PRs) occurred due to individuals learning about Git and GitHub from a specific YouTube video

https://youtu.be/Ez8F0nW6S-w?t=4323

In the video, the presenter demonstrated how to contribute to open-source projects using the Express repository as an example. While the video emphasized creating PRs for meaningful changes only, the majority of the audience consisted of beginners unfamiliar with GitHub and community guidelines. Consequently, many blindly followed the steps outlined in the tutorial, resulting in an overwhelming influx of PRs.

[wesleytodd]

Thanks @nkroker, please read above as that topic has been covered quite a few times. I will mark this as off topic but thanks for the well intentioned help.