opensearch-project / .github Goto Github PK

What is the bug?

There are dozens of issues labeled beta. These are usually inside the but_template.md.

OpenSearch beta has long sailed.

https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+user%3Aopensearch-project+label%3ABeta

What is the expected behavior?

Beta tag needs to be removed from everywhere.

• opensearch-project/common-utils#154
• opensearch-project/ml-commons#273
• opensearch-project/alerting-dashboards-plugin#205
• opensearch-project/alerting#382
• opensearch-project/anomaly-detection-dashboards-plugin#226
• opensearch-project/anomaly-detection#488
• opensearch-project/asynchronous-search#126
• opensearch-project/reporting#328
• opensearch-project/dashboards-visualizations#59
• opensearch-project/index-management-dashboards-plugin#168
• opensearch-project/index-management#324
• opensearch-project/job-scheduler#165
• opensearch-project/k-NN#351
• opensearch-project/notifications#396
• opensearch-project/performance-analyzer#177
• opensearch-project/performance-analyzer-rca#154
• opensearch-project/security-dashboards-plugin#931
• opensearch-project/security#1740
• opensearch-project/sql#531
• opensearch-project/cross-cluster-replication#349
• opensearch-project/observability#662

Is your feature request related to a problem?

In https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md#triage-open-issues we talk about triaging issues. But what is it?

What solution would you like?

Add to https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md#triage-open-issues. I think we want to say that:

1. Triage is a regular thing that teams do (with an example like security - provide link - that does it in the open).
2. All new issues get an untriaged label.
3. Triage means looking at all issues that say "untriaged" are looked at. Confirmed as valid as feature requests or bugs, belonging to this repo, etc.
4. Untriaged label removed.
5. Doesn't mean anyone will ever promise to work on this issue.

What are you proposing?

In opensearch-project/opensearch-build#2051 (comment) we communicated that part of the release of .NET client we performed a security review. Add a section to RELEASING.md describing this process, including when it's performed, and for which released components.

What kind of business use case are you trying to solve? What are your requirements?

The OpenSearch-Project needs language to provide guidance on moving a project to maintenance mode. (see need in #59).

What is the problem? What is preventing you from meeting the requirements?
No guidance yet and this will be useful if a repo becomes stale / unmaintained

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
Something along the lines of The Linux Foundation

What are your assumptions or prerequisites?
Project no longer maintained

What are remaining open questions?
What's a good process for OpenSearch-Project

When I go to create an issue in opensearch-project/OpenSearch it looks like the organization templates are not all showing up. Specifically I don't see the proposal template.

What is the bug?

https://github.com/opensearch-project/.github/blob/main/RELEASING.md#releasing has a link to bundle-workflow that is broken. This also shows up in other repos that have copies of the RELEASING.md file.

How can one reproduce the bug?

Go to https://github.com/opensearch-project/.github/blob/main/RELEASING.md#releasing. In the Releasing section, the first bullet has "bundle-workflow" linked to https://github.com/opensearch-project/opensearch-build/blob/main/bundle-workflow/README.md which produces a 404.

What is the expected behavior?

Not sure what a valid link is here.

What is your host/environment?

N/A

What kind of business use case are you trying to solve? What are your requirements?

What are the processes that each repo in this org can modify, and which processes can't they?

What is the problem? What is preventing you from meeting the requirements?

Coming from #59 (comment) where someone asked whether maintainers of the project can modify the maintainer rules in their own repo.

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
A doc of what it means to be part of the org, e.g. "must adopt code of conduct", "must adopt maintainers.md", "free to adopt whatever programming language".

Standardize release notes across all products that we release in this org.

• Should one include every PR or remove some?
• Should these be edited to be humanly readable?
• Where can I find the standard format?
• ...

Probably should go into https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md

Describe the bug
Lots of issues labeled "beta".
https://github.com/opensearch-project/sql/issues?q=is%3Aissue+is%3Aopen+label%3ABeta

When I open a new issue it still says beta.

We have branch protection for merging code to a dev branch. Can we remove branch protection on -dev branch where an "" can represent the name of a feature - e.g bucket-level-alerting-dev

Apply branch protection rules only to
main
opensearch-[1-9].[0-9].[0-9]*

Is your feature request related to a problem?

It's unclear how to get started with creating new repos in this organization, what the repo standards may be, or guidelines for specific types of repos such as plugins (should they be monorepo vs. not for example)?

What solution would you like?

A top level entry in README about new repos and paths to answering questions about more specific repos, such as new plugins.

Coming from opensearch-project/opensearch-plugins#35, and #11, but that merges OpenSearch, OpenSearch Dashboards and plugins.

• OpenSearch - opensearch-project/OpenSearch#853
• OpenSearch Dashboards - opensearch-project/OpenSearch-Dashboards#527
• Plugins - opensearch-project/opensearch-plugins#41

• Externally MAINTAINERS.md, such as OpenSearch MAINTAINERS.md lists 13 maintainers and points to this file which lists 2. If there is a difference we should make it clear, if not we should should correct.
• RELEASING.md has blank/empty TODO at the bottom.

We should update .MAINTAINERS to clarify mechanics for adding features to the roadmap. We have an FAQ which says "the maintainers of the repo will also work with you to incorporate it into the roadmap." However, it isn't super clear how this works in MAINTAINERS.md.

What is the bug?

Because of the way we're creating repos, our templates are copied from https://github.com/amazon-archives (specifically Apache-2.0), not this .github. That means our files like maintainers, etc, are not getting copied over when repos are created. (see opensearch-project/opensearch-testcontainers#1).

How can one reproduce the bug?

Create a new repo.

What is the expected behavior?

.github files would be copied over. They are not.

Do you have any additional context?

I'm proposing we update the issue proposal template as follows:

What/Why

What are you proposing?

In a few sentences, describe the feature and its core capabilities.

What users have sked for this feature?

Include links to GitHub Issues, Forums, Stack Overflow, Twitter, Etc

What problems are you trying to solve?

Summarize the core use cases and user problems and needs you are trying to solve. Describe the most important user needs, pain points and jobs as expressed by the user asks above. Template: When <a situation arises> , a <type of user> wants to <do something>, so they can <expected outcome>. (Example: When searching by postal code, a buyer wants to be required to enter a valid code so they don’t waste time searching for a clearly invalid postal code.)

What is the developer experience going to be?

Does this have a REST API? If so, please describe the API and any impact it may have to existing APIs. In a brief summary (not a spec), highlight what new REST APIs or changes to REST APIs are planned. as well as any other API, CLI or Configuration changes that are planned as part of this feature.

Are there any security considerations?

Describe if the feature has any security considerations or impact. What is the security model of the new APIs? Features should be integrated into the OpenSearch security suite and so if they are not, we should highlight the reasons here.

Are there any breaking changes to the API

If this feature will require breaking changes to any APIs, ouline what those are and why they are needed

What is the user experience going to be?

Describe the feature requirements and or user stories. You may include low-fidelity sketches, wireframes, APIs stubs, or other examples of how a user would use the feature via CLI, OpenSearch Dashboards, REST API, etc. Using a bulleted list or simple diagrams to outline features is okay. If this is net new functionality, call this out as well.

Are there breaking changes to the User Experience?

Will this change the existing user experience? Will this be a breaking change from a user flow or user experience perspective?

Why should it be built? Any reason not to?

Describe the value that this feature will bring to the OpenSearch community, as well as what impact it has if it isn't built, or new risks if it is. Highlight any research, proposals, requests, issues, forum posts, anecdotes that signal this is the right thing to build. Highlight opportunities for additional research.

What will it take to execute?

Describe what it will take to build this feature. Are there any assumptions you may be making that could limit scope or add limitations? Are there performance, cost, or technical constraints that may impact the user experience? Does this feature depend on other feature work? What additional risks are there?

What are the remaining open questions?

What are known enhancements to this feature? Any enhancements that may be out of scope but that we will want to track long term? List any other open questions that may need to be answered before proceeding with an implementation.

The current scheme (n.n.n.n) makes it difficult if not impossible (for GitHub CD pipeline in the security plugin) to distinguish release tag from a tag. That pipeline today triggers builds on tags starting with v. See opensearch-project/security#1406 for the original issue.

We will need to change/re-tag 1.0 and 1.0.1 and possibly 1.1 if we do this.

What/Why

What are you proposing?

Let's earn the OpenSSF Best Practices Passing Badge!

“A CII Best Practices badge, especially a gold badge, shows that an OSS project has implemented a large number of good practices to keep the project sustainable, counter vulnerabilities from entering their software, and address vulnerabilities when found.” – David A. Wheeler, Director of Open Source Supply Chain Security. See full article.

References

• https://bestpractices.coreinfrastructure.org/en
• https://bestpractices.coreinfrastructure.org/en/criteria/0?details=true&rationale=true
• https://www.linuxfoundation.org/blog/blog/why-cii-best-practices-gold-badges-are-important

The CONTRIBUTING.md file in this repository specifies a longer license header.

This conflicts with apparent (authoritative?) guidance posted in a comment on the alerting project repo here, although the qualification of "like this one" isn't clear:
opensearch-project/alerting#136 (comment)

There was an apparently more authoritative version in issue #21 and PR #29 where the CONTRIBUTING file was updated with a shorter version.
@dblock asked:

is the short version the preferred version, @hyandell?

@hyandell replied:

Yes, that's the preferred version.

But then @dblock updated the header in #76 to include the more expansive language presently in the header based on this issue: opensearch-project/opensearch-java#178

That update stated that the change was "confirmed" but did not cite a reference.

But we still have some PRs updating per the older guidance such as opensearch-project/OpenSearch#4657

My ask:

1. Please confirm that the header in this repo's CONTRIBUTING file is the latest official guidance
2. Suggest someone with the admin-powers to do so edit the conflicting guidance in the linked issues above to add an "update" which points to the latest guidance

Is your feature request related to a problem?

In opensearch-project/opensearch-plugin-template-java#4 we proposed moving the repo into the OpenSearch GitHub organization. That required some support from org admins, and was done preserving the original admin/maintainers.

What solution would you like?

Documentation on how one would start moving/proposing moving a repo into opensearch-project. A superset of #79.

What kind of business use case are you trying to solve? What are your requirements?

Markdown files could use some standardization.

• TOCs
• Ends of files should have a line ending
• Capitalization in sections

What is the problem? What is preventing you from meeting the requirements?
Human have to remind humans to add TOCs.

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
Add a linter.

Is your feature request related to a problem? Please describe.
Create opensearch-sdk for opensearch-project/OpenSearch#1619.

What is the bug?

There are multiple repos with MAINTAINERS.md that were inherited through various processes that have never committed to a repo. For example, in opensearch-project/opensearch-java@53dc6ba we had asked some folks to volunteer to be maintainers, but that was about it. We can now remove them from the repos.

Do you have any screenshots?

Using project-tools.

$ ./bin/project maintainers audit
https://github.com/opensearch-project/.github: 2
https://github.com/opensearch-project/OpenSearch: 22
https://github.com/opensearch-project/OpenSearch-Dashboards: 12
  boktorbb-amzn: 0
https://github.com/opensearch-project/alerting: 7
  tlfeng: 0
  leeyun-amzn: 0
https://github.com/opensearch-project/alerting-dashboards-plugin: 7
  skkosuri-amzn: 0
  rishabhmaurya: 0
  leeyun-amzn: 0
https://github.com/opensearch-project/anomaly-detection: 11
  ohtyler: 0
https://github.com/opensearch-project/anomaly-detection-dashboards-plugin: 5
  VijayanB: 0
https://github.com/opensearch-project/ansible-playbook: 5
  bbarani: 0
https://github.com/opensearch-project/asynchronous-search: 
https://github.com/opensearch-project/common-utils: 8
  rishabhmaurya: 0
  tlfeng: 0
  leeyun-amzn: 0
https://github.com/opensearch-project/cross-cluster-replication: 8
  rushiagr: 0
https://github.com/opensearch-project/dashboards-anywhere: 7
  zengyan-amazon: 0
https://github.com/opensearch-project/dashboards-desktop: 3
  zhongnansu: 0
https://github.com/opensearch-project/dashboards-docker-images: 6
  peterzhuamazon: 0
  bbarani: 0
  gaiksaya: 0
  zelinh: 0
  prudhvigodithi: 0
https://github.com/opensearch-project/dashboards-maps: 3
  VijayanB: 0
  vamshin: 0
https://github.com/opensearch-project/dashboards-notebooks: 
https://github.com/opensearch-project/dashboards-reports: 4
  davidcui-amzn: 0
https://github.com/opensearch-project/dashboards-search-relevance: 2
https://github.com/opensearch-project/dashboards-visualizations: 2
https://github.com/opensearch-project/data-prepper: 10
  treddeni-amazon: 0
https://github.com/opensearch-project/docker-images: 6
  peterzhuamazon: 0
  bbarani: 0
  gaiksaya: 0
  zelinh: 0
  prudhvigodithi: 0
https://github.com/opensearch-project/documentation-website: 
https://github.com/opensearch-project/geospatial: 3
  jmazanec15: 0
  vamshin: 0
https://github.com/opensearch-project/helm-charts: 6
https://github.com/opensearch-project/i18n-plugin: 7
  boktorbb-amzn: 0
  kavilla: 0
  seanneumann: 0
  tmarkley: 0
  joshuarrrr: 0
  ashwin-pc: 0
https://github.com/opensearch-project/index-management: 9
  CEHENKLE: 0
  nknize: 0
  praveensameneni: 0
  skkosuri-amzn: 0
https://github.com/opensearch-project/index-management-dashboards-plugin: 4
  leeyun-amzn: 0
https://github.com/opensearch-project/job-scheduler: 5
https://github.com/opensearch-project/k-NN: 3
https://github.com/opensearch-project/logstash-input-opensearch: 2
  vamshin: 0
https://github.com/opensearch-project/logstash-output-opensearch: 6
  jmazanec15: 0
  vamshin: 0
  dlvenable: 0
  sshivanii: 0
https://github.com/opensearch-project/maps: 4
  Shivamdhar: 0
  vamshin: 0
  VijayanB: 0
https://github.com/opensearch-project/ml-commons: 3
https://github.com/opensearch-project/neural-search: 9
  vamshin: 0
  sean-zheng-amazon: 0
  model-collapse: 0
  wujunshen: 0
  ylwu-amzn: 0
  jngz-es : 0
https://github.com/opensearch-project/notifications: 3
https://github.com/opensearch-project/observability: 6
https://github.com/opensearch-project/opensearch-api-specification: 
https://github.com/opensearch-project/opensearch-benchmark: 6
  gkamat: 0
  treddeni-amazon: 0
https://github.com/opensearch-project/opensearch-benchmark-workloads: 6
  gkamat: 0
https://github.com/opensearch-project/opensearch-build: 6
https://github.com/opensearch-project/opensearch-build-libraries: 3
https://github.com/opensearch-project/opensearch-catalog: 4
  bowenlan-amzn: 0
  praveensameneni: 0
https://github.com/opensearch-project/opensearch-ci: 5
  bbarani: 0
  zelinh: 0
https://github.com/opensearch-project/opensearch-cli: 5
  CEHENKLE: 0
  jmazanec15: 0
  nknize: 0
  vamshin: 0
https://github.com/opensearch-project/opensearch-clients: 3
  jmazanec15: 0
  vamshin: 0
https://github.com/opensearch-project/opensearch-dashboards-functional-test: 4
https://github.com/opensearch-project/opensearch-dashboards-test-library: 2
  hyandell: 0
  dblock: 0
https://github.com/opensearch-project/opensearch-devops: 6
  gaiksaya: 0
  DandyDeveloper: 0
https://github.com/opensearch-project/opensearch-dsl-py: 2
  Yury-Fridlyand: 0
https://github.com/opensearch-project/opensearch-go: 3
  jmazanec15: 0
  vamshin: 0
https://github.com/opensearch-project/opensearch-hadoop: 
https://github.com/opensearch-project/opensearch-java: 6
  Bukhtawar: 0
  madhusudhankonda: 0
  saratvemulapalli: 0
https://github.com/opensearch-project/opensearch-js: 8
  boktorbb: 0
  kavilla: 0
  mihirsoni: 0
  seanneumann: 0
  dblock: 0
https://github.com/opensearch-project/opensearch-net: 7
  dblock: 0
  joshuali925: 0
https://github.com/opensearch-project/opensearch-net-abstractions: 7
  joshuali925: 0
  alexmeizer: 0
  guiangumpac: 0
  raymond-lum: 0
https://github.com/opensearch-project/opensearch-oci-object-storage: 
https://github.com/opensearch-project/opensearch-php: 4
  dblock: 0
https://github.com/opensearch-project/opensearch-plugin-template-java: 4
https://github.com/opensearch-project/opensearch-plugins: 2
https://github.com/opensearch-project/opensearch-py: 6
  dblock: 0
  Shephalimittal: 0
https://github.com/opensearch-project/opensearch-py-ml: 4
  ylwu-amzn: 0
  sean-zheng-amazon: 0
  mingshl: 0
https://github.com/opensearch-project/opensearch-rs: 4
https://github.com/opensearch-project/opensearch-ruby: 7
  robsears: 0
  vamshin: 0
https://github.com/opensearch-project/opensearch-sdk-java: 6
  CEHENKLE: 0
https://github.com/opensearch-project/opensearch-testcontainers: 2
  dblock: 0
https://github.com/opensearch-project/oui: 2
  seanneumann: 0
https://github.com/opensearch-project/performance-analyzer: 5
  kkhatua: 0
https://github.com/opensearch-project/performance-analyzer-rca: 5
  kkhatua: 0
https://github.com/opensearch-project/perftop: 5
  kkhatua: 0
  yujias0706: 0
https://github.com/opensearch-project/piped-processing-language: 6
  chloe-zh: 0
  nknize: 0
  CEHENKLE: 0
https://github.com/opensearch-project/project-meta: 1
https://github.com/opensearch-project/project-tools: 1
https://github.com/opensearch-project/project-website: 7
https://github.com/opensearch-project/project-website-search: 
https://github.com/opensearch-project/search-relevance: 1
https://github.com/opensearch-project/security: 5
https://github.com/opensearch-project/security-analytics: 3
https://github.com/opensearch-project/security-analytics-dashboards-plugin: 4
  lezzago: 0
  sbcd90: 0
  praveensameneni: 0
  getsaurabh02: 0
https://github.com/opensearch-project/security-dashboards-plugin: 6
https://github.com/opensearch-project/simple-schema: 
https://github.com/opensearch-project/spring-data-opensearch: 3
https://github.com/opensearch-project/sql: 8
  nknize: 0
  CEHENKLE: 0
https://github.com/opensearch-project/terraform-provider-opensearch: 4
  peterzhuamazon: 0
  prudhvigodithi: 0
  jackson-theisen: 0
  phillbaker: 0
https://github.com/opensearch-project/ux: 

In order to build trust in the security posture of our codebase, we should be more explicit about expectations of how quickly we will be fixing publicly disclosed vulnerabilities. I propose we add more clear language around frequency of our maintenance releases and our commitment to fixing CVSSv3 MEDIUM or higher severity vulnerabilities in them.

[from the OpenSSF Best Practices criteria page] (related to issue #93)

There MUST be no unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days.
The vulnerability must be patched and released by the project itself (patches may be developed elsewhere). A vulnerability becomes publicly known (for this purpose) once it has a CVE with publicly released non-paywalled information (reported, for example, in the National Vulnerability Database) or when the project has been informed and the information has been released to the public (possibly by the project). A vulnerability is considered medium or higher severity if its Common Vulnerability Scoring System (CVSS) base qualitative score is medium or higher. In CVSS versions 2.0 through 3.1, this is equivalent to a CVSS score of 4.0 or higher. Projects may use the CVSS score as published in a widely-used vulnerability database (such as the National Vulnerability Database) using the most-recent version of CVSS reported in that database. Projects may instead calculate the severity themselves using the latest version of CVSS at the time of the vulnerability disclosure, if the calculation inputs are publicly revealed once the vulnerability is publicly known. Note: this means that users might be left vulnerable to all attackers worldwide for up to 60 days. This criterion is often much easier to meet than what Google recommends in Rebooting responsible disclosure, because Google recommends that the 60-day period start when the project is notified even if the report is not public. Also note that this badge criterion, like other criteria, applies to the individual project. Some projects are part of larger umbrella organizations or larger projects, possibly in multiple layers, and many projects feed their results to other organizations and projects as part of a potentially-complex supply chain. An individual project often cannot control the rest, but an individual project can work to release a vulnerability patch in a timely way. Therefore, we focus solely on the individual project's response time. Once a patch is available from the individual project, others can determine how to deal with the patch (e.g., they can update to the newer version or they can apply just the patch as a cherry-picked solution).

What kind of business use case are you trying to solve? What are your requirements?

We need to define standards and guidelines around the tests for the contributions to improve the quality of the code.

What is the problem? What is preventing you from meeting the requirements?
Currently, we dont have any guidelines around tests hence code contributions and merges are very subjective at this point in time

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
Suggest adding below section to https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md

Test the Changes

Check whether tests already exist in the project to test the contribution. If so, make sure to execute it every time before merging the changes. If the test doesn’t exist, understand the type of testing requirement ( Integration? Unit? Acceptance? ) for the contribution and see if we can re-use the framework by setting up a new testing harness for the project.

If adequate testing for a contribution does not exist, it is our responsibility to make sure the tests are added either by requesting the contributor to add one or more tests OR by adding the tests ourselves. If we are merging a fix, the added tests should fail without the fix. If we are merging an enhancement, we should make sure that the tests thoroughly exercise the new functionality.

What are your assumptions or prerequisites?
Describe any assumptions you may be making that would limit the scope of this proposal.

What are remaining open questions?
List questions that may need to be answered before proceeding with an implementation.

We see some variations of users going crafty and checklists in PR templates.

opensearch-project/OpenSearch#1103

opensearch-project/OpenSearch#1091

opensearch-project/OpenSearch#1102

opensearch-project/opensearch-build#203

They also show up as a task checklist in the list of PRs, which is misleading.

Coming from opensearch-project/opensearch-plugins#92.

Each plugin has a short descriptive text blurb that says what it does: this appears in plugin-descriptor.properties (OpenSearch plugins) or package.json (OpenSearch Dashboards plugins), as well as on the project website's "source" page and in the "About" section of every repo. These were created one-by-one over the years as plugins were created, so looking at them all together now it would be hard for somebody new to the project to use these to understand what the plugins do.

We need to have post release checklist and processes to prepare for the next release.

Here is a scenario we hit 5 weeks after 1.0 release:
Based on our branching strategy we should have bumped OpenSearch main to 2.0.0 and all the plugins to consume respective branches. Until plugins needed to consume the latest changes in OpenSearch, the code was not bumped to use the right versioning. While each plugin was bumping up their versions, all its dependencies needed to as well causing a nightmare for a developer.
This defeats the purpose of having an ideology of fail fast but we are always consuming the stable versions until we really need the bleeding edge.
I believe this is a gap in process and should done right after the release for opensearch-project.

Ref:
opensearch-project/common-utils#48
opensearch-project/common-utils#58
opensearch-project/job-scheduler#44
opensearch-project/job-scheduler#49

Coming from #23, enable that for this repo.

Is your feature request related to a problem?

It would be nice to support encryption for incoming emails reporting security issues in the project (as pointed out here

What solution would you like?

Create public/private key pair. Figure out a way to distribute private key among security team members. Publish the public key and link to the security issue response policy added by #90

Coming from #11, every section needs a HowTo that needs to land here and apply to all projects in the org.

• Branching #13
• Releasing #13
• Labelling

What is the bug?

When repositories inherit .github they get MAINTAINERS.md from this repo. That has a lot of words. In most repos those MAINTAINERS.md are gutted to point to the .github MAINTAINERS.md, but not everywhere.

How can one reproduce the bug?

Compare https://github.com/opensearch-project/opensearch-py-ml/blob/main/MAINTAINERS.md (has everything) to https://github.com/opensearch-project/opensearch-plugin-template-java/blob/main/MAINTAINERS.md (has a reference to .github/MAINTAINERS.md).

What is the expected behavior?

1. All MAINTAINERS.md to be updated to be the minimal version.
   
2. Processes updated to make sure new MAINTAINERS.md are the minimal versions.

Do you have any additional context?

See #92 for ensuring all MAINTAINERS.md.

What kind of business use case are you trying to solve? What are your requirements?

Branch protection is used to avoid accidentally deleting branches, requiring multiple code reviewers, etc.

What is the problem? What is preventing you from meeting the requirements?

Branch protection varies repo to repo, write up what kind of branch protection we want and standardize/enforce.

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?

Write up what branch protection rules each repository should apply, at a minimum.

What are remaining open questions?

• Should we allow for feature/x branches?
• We have bots, such as Whitesource and https://github.com/dblock/create-a-github-issue that open PRs and need to work around branch protection, how do they do that?
• #26
• Developers can push a branch accidentally, then they can't delete it, how do they do that?
• Do we cleanup stale branches and how? (Does it require temporarily relaxing branch protection?)

Coming from #59, let's talk about emeritus maintainers.

• #59 (comment), "Does "Emeritus" afford anything aside from a mention in maintainers?"
• #59 (comment), "How does an emeritus maintainer get reinstanted?"

Looks like approvals aren't required in this repo to merge and I don't have admin access (I shouldn't). Bump minimum amount of approvals in this repo to 1 and protect main if it's not already.

Description
We should have a way to discuss different issues amongst maintainers and admins via either a mailing list or some other channel. Especially since there will be more and more maintainers from outside AWS - #59

What is the problem? What is preventing you from meeting the requirements?
Currently the discussions are all out in the open which is good for most cases. However, discussions regarding nominations and perhaps other types of discussions are the kind best had in a smaller forum.

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
Mailing lists, or in todays forum. There could be many other ways to do this. I would prefer not to have these done in Chime since the time zones would be really hard to work around.

What are your assumptions or prerequisites?
Most discussions should be out in the open, this is only for issues that are by design best had in the maintainer - admins forum.

Is your feature request related to a problem? Please describe.
OpenSearch has a support policy to indicating that the latest Minor Patch version for a Major version are supported and will see future releases. As of this writing 1.3.5 and 2.3.0 are supported and 3.0.0 is is under active development.

However, there are many branches in the codebase and the plugin codebases that are not supported - as in they will never be releases as part of an OpenSearch distribution. Branches like this include OpenSearch's 1.0, 1.1 releases and there are some plugins with opendistro-* branches.

Describe the solution you'd like
It should be easy for members and contributors of OpenSearch-project to know what branches are supported and if a branch is no longer supported.

Describe alternatives you've considered

• There could be a gadget on github repositories that shows the current in-progress version.
• Branches could be deleted after tags have been created.
• Branches could be cleaned of repository content with a pointer to "go here for latest supported versions"

Additional context
From opensearch-project/security#2009 (comment) there was a set of 5 pull requests to backport a changes so it could be consumed by a private fork. These changes depended on legacy build and test systems that are virtual non-functional. Rather than accepting merges for changes that 'might' work with failing CI, it seems more appropriate to guide contributors away from such courses of action

Problem: We say different things in different repositories for items, such as code of conduct, for no good reason.

https://github.com/opensearch-project/job-scheduler

https://github.com/opensearch-project/common-utils

etc.

Let's level up READMEs across the org to at least include standard things.

• Add logo and standard sections in README.md, e.g. inspired from common-utils/README.md, merge with your content.
• Use a standard CONTRIBUTING.md that links to one in this repo.
• Extract DEVELOPER_GUIDE.md, e.g. common-utils/DEVELOPER_GUIDE
• Ensure MAINTAINERS.md, e.g. common-utils/MAINTAINERS.md that links to general maintainer responsibilities.
• Ensure ADMINS.md, e.g. common-utils/ADMINS.md that links to general admin responsibilities.
• Add a standard SECURITY.md, e.g. common-utils/SECURITY.md.
• Add a standard RELEASING.md, e.g. common-utils/RELEASING.md.
• Cleanup TESTING.md if you have one.
• Cleanup any other .md content you have.

Feel free to take freedoms in these files that are specific to your projects where it makes sense. As a rule of thumb, make links for common things across the org into the .github repo, and add the custom things that are only relevant for your project in your own .md's. When in doubt, consider reducing the amount of mental overhead contributors have to have when making PRs across multiple repos in our org.

Example: opensearch-project/common-utils#32

During an evaluation of our repos last week, I noticed a number of repositories do not contain a MAINTAINERS.md file.

• opensearch-project/security-analytics#47
• opensearch-project/documentation-website#589 - issue from earlier this year
• opensearch-project/search-processor#15
• opensearch-project/dashboards-anywhere#83
• opensearch-project/opensearch-plugin-template-java#39
• opensearch-project/opensearch-plugins#170 - blank maintainer file
• opensearch-project/opensearch-oci-object-storage#17
• opensearch-project/opensearch-api-specification#51
• opensearch-project/opensearch-benchmark-workloads#53
• opensearch-project/dashboards-desktop#26
• opensearch-project/opensearch-catalog#4
• opensearch-project/project-website-search#38
• dashboards-notebooks: deprecated repo - no issue opened
• opensearch-project/opensearch-hadoop#25
• opensearch-project/simple-schema#17

Organizations can have READMEs now. This would help people navigate the large (and growing) OpenSearch org.

Example:
https://github.com/twitter

It comes from:
.github/profile/README.md

We need to write down what beta, release candidate and GA definitions are.

Update: the correct copyright is Copyright OpenSearch Contributors, without All Rights Reserved, without years for the OpenSearch part (leave years if any Elastic copyright is needed), and no Amazon-related stuff. See #24 for an example.

1. Do you have a Copyright OpenSearch Contributors in README?
2. Does copyright Copyright OpenSearch Contributors match between NOTICE.txt or NOTICE and README?
3. Did you remove the year from Copyright 2021 OpenSearch Contributors in README and NOTICE?
4. Do you have other copyrights properly represented in NOTICE, such as Elastic, if your code was forked?
5. Did you remove all Copyright Amazon and Affiliates?
6. Did you remove any other .md files that say anything about copyright?
7. Is your CONTRIBUTING.md up-to-date saying to use short versions?
8. Are you using short versions in your files? See #29

FAQ

What source header should be on new OpenSearch files?

Assuming an Apache-2.0 project, new OpenSearch files should state the following:

# Copyright OpenSearch Contributors
# SPDX-License-Identifier: Apache-2.0

What if the file is a modified Apache-2.0 file from another project, and what if it already has a source header?

If files have any existing headers keep them. Add this OpenSearch license header to the top of the file. Note that it has additional lines to indicate the file has been modified, and the copyright is identified at the end of the header:

# SPDX-License-Identifier: Apache-2.0
#
# The OpenSearch Contributors require contributions made to this file be licensed 
# under the Apache-2.0 license or a compatible open source license.
# 
# Modifications Copyright OpenSearch Contributors. See
# GitHub history for details.

What if the file is an OpenDistro file and it includes an OpenDistro for ElasticSearch project Amazon copyright header?

Amazon employees may remove the OpenDistro for ElasticSearch Amazon copyright header when bringing code into OpenSearch; Amazon are an OpenSearch Contributor. For example, the source header on this opendistro file.

    * /*
         * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
         *
         * Licensed under the Apache License, Version 2.0 (the "License").
         * You may not use this file except in compliance with the License.
         * A copy of the License is located at
         *
         *     http://www.apache.org/licenses/LICENSE-2.0
         *
         * or in the "license" file accompanying this file. This file is distributed
         * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
         * express or implied. See the License for the specific language governing
         * permissions and limitations under the License.
         */

After removing the OpenDistro for ElasticSearch header as seen above, use the OpenSearch header:

# Copyright OpenSearch Contributors
# SPDX-License-Identifier: Apache-2.0

What goes in the NOTICE file?

The NOTICE file should contain the name of the project, a top level copyright statement, and any additional required notices (such as copyright statements) that the source within the project requires be communicated to users. This would include any NOTICE files that apply to other Apache-2.0 licensed code within the project’s source.

Note that the NOTICE file only covers the content of the project. It should not contain notices related to dependencies that are provided separately from that project (for example, dependencies within a Gradle build file).

Via https://twitter.com/IntranetFocus/status/1414878242721476610:

On our site we say:

In this project and OpenSearch core we don't include "contributors":

Let's figure out what the right way of saying all these things is and make sure it's both explained and aligned.

• opensearch-project/project-website#199
• opensearch-project/alerting#136
• opensearch-project/alerting-dashboards-plugin#35
• opensearch-project/anomaly-detection#161
• opensearch-project/anomaly-detection-dashboards-plugin#67
• opensearch-project/asynchronous-search#28
• opensearch-project/common-utils#49
• opensearch-project/dashboards-notebooks#59
• opensearch-project/reporting#137
• opensearch-project/dashboards-visualizations#21
• opensearch-project/dashboards-visualizations#21
• opensearch-project/data-prepper#189
• opensearch-project/documentation-website#131
• opensearch-project/index-management#114
• opensearch-project/index-management-dashboards-plugin#38
• opensearch-project/job-scheduler#45
• opensearch-project/k-NN#75
• opensearch-project/logstash-output-opensearch#51
• opensearch-project/notifications#263
• opensearch-project/OpenSearch#1056
• opensearch-project/opensearch-build#152
• opensearch-project/opensearch-cli#27
• opensearch-project/piped-processing-language#5
• opensearch-project/project-website#252
• opensearch-project/security#1395
• opensearch-project/security-dashboards-plugin#808
• opensearch-project/sql#175
• opensearch-project/observability#84
• opensearch-project/opensearch-plugin-template-java#10
• opensearch-project/project-meta#5
• opensearch-project/OpenSearch-Dashboards#711
• opensearch-project/project-website-search#25

What kind of business use case are you trying to solve? What are your requirements?

Clarity on third-party software licenses (attributions) in OpenSearch projects.

In particular, the Data Prepper team would like to use automation to populate our THIRD-PARTY file and want to be sure we are meeting all the necessary requirements.

What is the problem? What is preventing you from meeting the requirements?

Different repositories have different mechanisms to indicate what third party software and licenses are included in the project. As a result, we are unsure what approach to mimic.

Some examples:

• OpenSearch has a sparse NOTICE.txt with more detailed licenses in sub-projects. e.g. https://github.com/opensearch-project/OpenSearch/tree/main/server/licenses
• OpenSearch Dashboards has what appears to be a full NOTICE.txt
• The security plugin has a THIRD-PARTY.txt without license text

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?

I don't have a concrete proposal. Having a set of guidelines would be helpful. I don't think we want a standard, because different projects use different tooling (e.g. Gradle vs. Yarn) which may result in different approaches for automated dependency extraction.

What are your assumptions or prerequisites?

Developers want to automate this process.

What are remaining open questions?

Hey;

As our first RFCs are starting to close, I've started thinking about where to stick our design documents for OpenSearch. There are a couple of different approaches:

1/ Design docs are attached to an issue within the repo where a change will be made.

• Pros: Local knowledge is kept within a repo
• Cons: With 30+ repos, it's hard to get an understanding of all the different designs that are being built or that make up OpenSearch

2/ Design docs are iterated on within a particular repo/issue, but when implementation begins, they're moved to a central repository (current thought: .github/designs).

• Pros: Better discoverability
• Cons: People have to remember to move them over (human process). If the design changes during implementation, they have to remember to update them (which they'd have to do whether it was in their repo or a central location)

I'm open to other suggestions, but I'm leaning towards #2. I think this would also make it easier to collect them on our website.

What do you think?

Is your feature request related to a problem?

The opensearch-sdk-java repo wants to enable automated issues for upgrading dependency versions.

See #185

What solution would you like?

While dependabot is an option, I believe Mend Renovate is a suprerior option. It is endorsed by OpenSSF alongside dependabot as an industry standard tool for dependency updates.

I have switched my external OS repo from dependabot to renovate and found it far superior for a few key reasons:

• it was always faster at alerting on dependency updates than dependabot
• it provides a header on the bump PR showing adoption percentage, test passing rate, age of the dependency, and other metrics that help evaluate if a dependency may break things
• it provides a "Dashboard" issue for a very quick look at pending bump PRs

Enabling this on opensearch-sdk-java requires an owner of opensearch-project enable it. Repos which choose to use it can also be enabled; formally requesting this for opensearch-sdk-java.

The maintainers of the main Opensearch project may wish to consider it as well, as a replacement for dependabot.

What alternatives have you considered?

Dependabot

Do you have any additional context?

Some quick links to my own repo demonstrating some of the benefits outlined above

• Bump PRs have a header with badges displaying the quality of the dependencies. See an example here. What the age of the version is, what percentage of Renovate users have upgraded, what percentage of them have passing tests. This is invaluable for evaluating major version bumps and adoption/quality.
• You can group/batch version bumps together. I have done this to batch my build plugins (to get a single PR instead of 3 for ones that release together, for example) to reduce the PR noise. Here's an example batch PR
• It creates a "Dashboard" issue that shows you any pending bumps or ones you've intentionally suppressed. See example here.
• It has the ability to auto-merge its own PRs, which I've enabled for minor and patch versions of build plugins. We probably don't want that feature here, but it's available

In the PR template it mentions 'New functionality has javadoc added.'

This would not apply to non-JVM projects (e.g. OpenSearch Dashboards (JS), OpenSearch-cli (golang), etc.)

Additionally it assumes Apache 2.0 and DCO, which is pretty safe but doesn't apply to project-website which is BSD 3 Clause and no DCO. For that project I've overridden the PR template since it's an exception.

Let's add some language to our SECURITY.md file around expectations for how the security issues mailbox will be staffed for the project.

Coming from this comment thread on the PR documenting the security issue response process:

how is the email address staffed? is it being monitored 24/7 or only during business hours in a specific location (e.g. US)?
is this internally a mailing list (i.e. you could eventually add non-AWS employees to the security maintainers) or is it a shared mailbox (i.e. only AWS employees can be added to it)?
[...]
maybe no specific commitment is needed, but it'd be good to know whether it can happen that all people with access to the mail address can be on vacation on the same time (think public holiday, etc.) or whether it'd be reasonable to expect a "we got your report and will be looking into it in the next few days and then get back to you" answer within 1-2 business days (thinking of the "we'll update you in 5 business days" emails from AWS security which they seem to be happy to send for several weeks in a row 🙄)

Add to MAINTAINERS

• What is the commit process from the maintainer POV? (e.g. puck-up any of the open PRs, is there an SLA?)
• What's specific about PRs in OpenSearch org (e.g. generally require 1 approval, merged by the contributor if they have contributor rights)?
• What do maintainers look for in PRs (e.g. tests)?
• What's the merge bar? (e.g. code is an improvement over what we have vs. perfect)
• What to look for in a PR? (e.g. ensure quality or security)
• How do we expect maintainers to behave? (e.g. be clear about what's a must have, should have or nice to have, and lead with empathy)
• Links to external resources about doing excellent code reviews.

Looking at https://lwn.net/Articles/857791/, GCC does not require contributors listed in MAINTAINERS to sign every commit. People probably do forget occasionally to do this, so maybe it's cleaner to automatically opt-in MAINTAINERS into DCO.

Developers with commit access may add their name to the DCO list in the MAINTAINERS file to certify the DCO for all future commits in lieu of individual Signed-off-by messages for each commit.

Let's up-level our repo descriptions!

How to write a good description:

• Use 1-2 full, but concise sentences
• Provide new information to the reader
• Use simple, friendly, non-corporate words
• Think like a Jeopardy clue:
  • Player: I’ll take Plugins for $400, Levar (https://www.goodhousekeeping.com/life/entertainment/a37144212/jeopardy-levar-burton-guest-host-reaction/)
  • Levar: An OpenSearch Dashboards plugin to perform real-time and historical anomaly detection on OpenSearch data.
  • Player: What is anomaly-detection-dashboards-plugin?
  • Levar: Correct!

Possible writing prompts:

• What problem does this repo solve for OpenSearch?
• How would you describe this repo to a new team member?
• How do you differentiate this repo from the rest of the repos in the GitHub org?

How to change the description on a repo

• In the upper right hand corner of the repo root, click on the gear next to “About”
• Edit the description
• Click “Save Changes”
• Pat yourself on the back for a job well done.

(related #37)