nodejs / automation Goto Github PK

Hello.
I propose to create a project with common resources and utils for all the @nodejs/automation projects.

It would contains .editorconfig, eslint configuration files, gql queries and fixtures...
With this we get a centralized management of common resources and we avoid repeating files with the same content.

It could work like a dependency with a postinstall script that makes symlinks.

What do you think? I you give me the approval I start working on it.

Please comment your impressions! 😃

https://github.com/nodejs/node/releases

There is an api for editing a release... so this seems pretty reasonable
https://developer.github.com/v3/repos/releases/#edit-a-release

Taken from nodejs/TSC#392 (comment)

• The Github bot
• node-review (See discussion on moving them into the foundation)
• branch-diff and changelog-maker (See discussion on moving them into the foundation)
• core-validate-commit if @evanlucas agrees to move it into the foundation
• node-core-utils (still in rapid development, can be transferred when it's ready)
• lts
• IRC/gitter/slack Slurp (current)

Like nodejs/node README does.

Wanted to ask if the make-node-meeting repo is within the scope of automation. My thought is yes - it's a tool we use for automation as WG organizers/team members. It was just recently pulled in like several other tools, and I think it fits very well within the scope of Automation.

  - version: REPLACEME
    pr-url: REPLACEME

When originally authoring a PR that has API doc changes, you can't know the pr-url apriori. So this issue can't be trivially linted.
We should either:

1. Lint only via ncu before landing
2. Get Jenkins to suggest change (if PR ID is available)
3. Fail lint so PRs don't get green CI until this is fixed
4. Create a REPLACEME replace tool for releasers, that deduces the pr-url

There was a discussion in the moderation repo about managing the bans (reminders etc).

I proposed that if needed, I can help in building an app to (un)block users, set reminders, maybe later automatically detect bad behaviours etc.

GitHub API supports organization ban related CRUD operations, but as @ChALkeR and @ryanmurakami pointed out, we have to figure out what are the security implications (e.g. @ChALkeR noticed that such operations need the admin:org scope which is quite powerful).

I opened this issue for discussion. I worked with GitHub API in the past and help if needed, maybe I can help. 🍀

https://github.com/nodejs/core-validate-commit and https://github.com/nodejs/node-review (EDIT: also https://github.com/nodejs/branch-diff and https://github.com/nodejs/changelog-maker) have already been moved into the Node.js orgnization and there will be more repos get transferred later. (also the github bot is probably going to be handed over to the automation team as well). We need to figure out how to organize the accesses of these projects.

Repository access

Including write access(commit) and admin access(add new people).

A few options that I have in mind:

1. Form subteams that have write/admin accesses for each repo
2. Gives @nodejs/automation write access for all the repos, form a subteam of automation that has admin accesses (similar to how working groups organize things).
   • P.S. I noticed that at the moment not everyone in the automation team has turned on 2 factor authentication...personally I want to see people with write access turns it on, just to be safe.

If we do want to adopt the monorepo idea(see #1 (comment)), then after the transition we can probably just go with option 2.

NPM publish access

So first thing to do is to add the foundation account to the collaborator list as a safety net. We should keep the existing npm collaborators, other than that, I can think of a few options:

1. Add people from the subteams to the npm collaborator list
2. If we want a bigger team that works on code, then we can assign 1-2 people for each project that handle releases. (basically from nodejs/TSC#406 (comment))

Personally I would prefer to see the releasers turn on 2FA on their npm accounts as well.

So...

👍 if you want subteams with repo/npm accesses
😄 if you want a big team with repo accesses, and small teams of releasers for each project

Or discuss in this thread about any other ideas that you have!

Hey team!

I'm currently working on the translation of nodejs.org to french, and I came across the description of the build WG.
It says it's purpose is "to create and maintain a distributed automation infrastructure."

Given the recent founding of our team, I know things aren't always pretty clear, but I think I remember that we are a separate team, not a subteam of the Build WG. Am I correct?

If so, shouldn't we try and see with them if we can rework their description to better suit their purpose?

And on a side note: if we're not a WG, how do we stand in the organization chart? And do we want to promote our work, and ask for a section on the WG page?
I'd say yes, but hey, I'm just one in a team! ✨

Hope I'm clear enough...
Cheers!

Small suggestion:

The npm install issue we had earlier today had me thinking:
Shouldn't we test our releases in some way with earlier Node releases?

I mean, I understand there's a great tool for that in the project, so maybe we could take advantage?
Not necessarily for really old versions, but at least current lts + latest version?

Maybe dumb idea, but at least we can talk about it ^^

The earliest I remember this being brought up was by @refack

A commit queue is a process for automating the landing of commits and managing of multiple branches.

This is a potential way to fix #2

For inspiration here is the design doc for the Chromium Commit Queue

The CQ polls rietveld for CLs that are ready to be committed. The Chromium's fork of Rietveld has a 'Commit' checkbox. When the author or the reviewer checks it, it will be included in the next poll. Once the CQ learns about the CL, it verifies the author or one of the reviewers approving this CL is a full committer, then runs the presubmit checks, then runs try jobs and then commit the patch on the behalf of the author, faking its credential. The whole project is written in python.

One small automation idea: when a PR lands from a brand new contributor, it would be cool to have a Twitter bot that automatically posts a thank you tweet with a link to the PR. Might be a cool way to give some recognition.

I've searched the organization for "testing" repos and have not found one. We should have a repo forking from the nodejs/node repo for testing bots/scripts. Any other ideas on how this should be managed?

So, this happened: nodejs/node#19438 (comment)

(Someone accidentally used the wrong PR number when starting a CI job.)

Surely this isn't the first time this has happened nor the last time this will happen.

Would be awesome if we could either:

• have git node land check the CI parameters to make sure they match the PR number? (That's a big request and would probably be sloooooow.)

• have a tool to start CI jobs that would somehow reduce the probability of this sort of thing. Not sure what that would look like to be honest.

This may be Too Big To Solve. Feel free to close. Just wanted to put it out there for consideration. Thanks.

1. Anyone interested in joining but does not have the team admin access, please indicate your interest in this thread. (BTW Node.js organization owners should have the access)
2. I did not make this team a child team of build because that would grant access to repos that contain secrets and stuff. I am not that well-versed with Github teams so let me know if there is a better way to organize things better

IMPORTANT: if you are not already a member of the Node.js organization, please read #4 (comment) about the purpose of this team, make sure to read our Code of Conduct and agree on it before you reply.

How could we automate the process of creating the staging branches?

An idea to start:

perhaps we could use labels to mark issues that are approved for backporting and a bot could automate cherry-picking it to a staging branch. If it does not land cleanly it adds a label / pings a backporting team

edit:

I've pivoted this issue to be about the LTS branches rather than the release process, as they are slightly different problems

https://github.com/Azure/brigade

This is really interesting tool for writing CI / CD pipelines using k8s with JavaScript (node...ish).

I wonder if it could be a cool way to build some tools. I can likely stand up a test cluster for us to experiment with

See nodejs/TSC#491 (comment), it would be great if mhdawson/create-node-meeting-artifacts could pull the team lists from GitHub, probably by using ncu-team.? The current lists is here.

cc/ @mhdawson @richardlau

This idea started in #1 (comment), and seems to receive some popularity within our team, so I think we can open an issue dedicated to this topic.

The current proposal

1. Put all automation related projects into a single repo. We can keep the existing git history with git subtrees.
2. At the moment the most viable solution for managing a monorepo seems to be Lerna, there is a WIP using it in #9 to move branch-diff into this repo.

The upside of a monorepo

(...that I can think of)

1. More code-sharing, we don't have to define a formal API and can test every dependencies to make sure one change in a tool does not break others
2. More visibility, can get more people involved into the maintenance of these projects
3. Easier to uniform the practices and environment setup for these tools, including contributing, governance, coding styles, Github API authentication, Jenkins token setup
4. Easier to bundle them into one toolkit and distribute/update together

The downside of a monorepo

1. Complicates release and commit processes, we are very likely to need to rely on a tool for that
2. More messages for the collaborators because they all live in one repo (though I don't think this would be a big problem considering the sum of existing traffic is not that terrifying)
3. Need to figure out how to get infrastructure e.g. CI working
4. Would need to audit/transfer existing issues/PRs of these projects.

Things to discuss if we do adopt the monorepo

1. Should the github bot be placed there? It's more like a server-side application, not an one-off command. #12 (comment)
2. Should we use the automation repo, or another repo, to place the code? #12 (comment)
3. How should we handle the transition? Maybe progressively adding them to the monorepo instead of doing it at once to avoid premature decisions. #12 (comment)

Please indicate your approval/objection towards this idea in this thread, and discuss about how we are going to structure the monorepo if this idea sounds good to you.

We could use the github bot to listen to the org's membership event and automatically open a PR to update the member list in readmes.

Just an idea, implementation still should thought of (how would the bot know which teams are relevant and which readmes need update)

In the core repo we are experimenting with the use of CODEOWNERS file. It's going to require some maintenance and iteration to get correct. At some point, we may want to have some tooling to automate maintenance and auditing of the file so that it is correct.

There are also other ways the CODEOWNERS can be leveraged... For instance, ncu could include a git command that uses the CODEOWNERS file to identify who would be notified for any given set of commits.

https://github.com/kubernetes/test-infra/tree/master/prow#prow

Prow is a Kubernetes based CI/CD system. Jobs can be triggered by various types of events and report their status to many different services. In addition to job execution, Prow provides GitHub automation in the form of policy enforcement, chat-ops via /foo style commands, and automatic PR merging. See the GoDoc for library docs. Please note that these libraries are intended for use by prow only, and we do not make any attempt to preserve backwards compatibility.