opensearch-project / opensearch-build

🧰 OpenSearch / OpenSearch-Dashboards Build Systems

• Releasing OpenSearch
  • Releases and Versions
  • Release labels
• Onboarding a New Plugin
• Building and Testing an OpenSearch Distribution
  • Building from Source
  • Assembling a Distribution
  • Building Patches
  • Min snapshots
  • CI/CD Environment
  • Build Numbers
  • Latest Distribution URL
  • Testing the Distribution
  • Checking Release Notes
  • Signing Artifacts
    • PGP
    • Windows
  • Signing RPM artifacts
• Making a Release
  • Releasing for Linux
  • Releasing for FreeBSD
  • Releasing for Windows
  • Releasing for macOS
• Utilities
  • Checking Out Source
  • Cross-Platform Builds
  • Sanity Checking the Bundle
  • Auto-Generating Manifests
• Deploying Infrastructure
• Contributing
• Getting Help
• Code of Conduct
• Security
• License
• Copyright

Releasing OpenSearch

Please refer to the release process document for detailed information on how to release the OpenSearch and OpenSearch Dashboards software.

Releases and Versions

The OpenSearch project releases as versioned distributions of OpenSearch, OpenSearch Dashboards, and the OpenSearch plugins. It follows semantic versioning. Software, such as Data Prepper, clients, and the Logstash output plugin, are versioned independently of the OpenSearch Project. They also may have independent releases from the main project distributions. The OpenSearch Project may also release software under alpha, beta, release candidate, and generally available labels. The definition of when to use these labels is derived from the Wikipedia page on Software release lifecycle. Below is the definition of when to use each label.

Release labels:

• Alpha - The code is released with instructions to build. Built distributions of the software may not be available. Some features many not be complete. Additional testing and development work is planned. Distributions will be postfixed with -alphaX where "X" is the number of the alpha version (e.g., "2.0-alpha1").
• Beta - Built distributions of the software are available. All features are completed. Additional testing and development work is planned. Distributions will be postfixed with -betaX where "X" is the number of the beta version (e.g., "2.0.0-beta1").
• Release Candidate - Built distributions of the software are available. All features are completed. Code is tested and minimal validation remains. At this stage the software is potentially stable and will release unless significant bugs emerge. Distributions will be postfixed with -rcX where "X" is the number of the release candidate version (e.g., "2.0.0-rc1").
• Generally Available - Built distributions of the software are available. All features are completed and documented. All testing is completed. Distributions for generally available versions are not postfixed with an additional label (e.g., "2.0.0").

Onboarding a New Plugin

Plugin owners can follow the Onboarding Process to onboard their plugins to the release process.

Building and Testing an OpenSearch Distribution

The distribution workflow builds a complete OpenSearch and OpenSearch Dashboards distribution from source. You can currently build 1.0, 1.1, 1.1-SNAPSHOT and 1.2 versions. This system performs a top-down build of all components required for a specific OpenSearch and OpenSearch Dashboards release, then assembles a distribution. The input to the system is a manifest that defines the order in which components should be built. All manifests for our current releases are here.

Building from Source

./build.sh manifests/1.3.0/opensearch-1.3.0.yml 

This builds OpenSearch 1.3.0 from source, placing the output into ./builds/opensearch.

See build workflow for more information.

Assembling a Distribution

./assemble.sh builds/opensearch/manifest.yml

The assembling step takes output from the build step, installs plugins, and assembles a full distribution into the dist folder.

See assemble workflow for more information.

Building Patches

A patch release contains output from previous versions mixed with new source code. Manifests can mix such references. See opensearch-1.3.1.yml for an example.

OpenSearch is often released with changes in opensearch-min, and no changes to plugins other than a version bump. This can be performed by a solo engineer following a cookbook. See also opensearch-build#1375 which aims to automate incrementing versions for the next development iteration.

Min Snapshots

Snapshots for OpenSearch core/min can be downloaded and used in CI's, local development, etc using below links:

Linux:

https://artifacts.opensearch.org/snapshots/core/opensearch/<version>-SNAPSHOT/opensearch-min-<version>-SNAPSHOT-linux-x64-latest.tar.gz

macOS:

https://artifacts.opensearch.org/snapshots/core/opensearch/<version>-SNAPSHOT/opensearch-min-<version>-SNAPSHOT-darwin-x64-latest.tar.gz

Windows:

https://artifacts.opensearch.org/snapshots/core/opensearch/<version>-SNAPSHOT/opensearch-min-<version>-SNAPSHOT-windows-x64-latest.zip

CI/CD Environment

We build, assemble, and test our artifacts on docker containers. We provide docker files in docker/ci folder, and images on staging docker hub repositories. All Jenkins pipelines can be found in jenkins. Jenkins itself is in the process of being made public and its CDK open-sourced.

See jenkins and docker for more information.

Build Numbers

The distribution URL and the build output manifest include a Jenkins auto-incremented build number. For example, the manifest from OpenSearch build 5905 contains the following.

build:
  name: OpenSearch
  version: 2.2.0
  platform: linux
  architecture: x64
  distribution: rpm
  id: '5905'

Latest Distribution URL

Use the latest keyword in the URL to obtain the latest build for a given version. For example https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.0/latest/linux/x64/rpm/dist/opensearch/manifest.yml redirects to build 5905 at the time of writing this.

The latest keyword is resolved to a specific build number by checking the index.json file. This file has contents such as this.

{"latest":"5905"}

The file is updated when a distribution build job is completed for the given product and version (or is created when such distribution job succeeds for the first time). Since one distribution build job consists of multiple stages for different combinations of distribution type, platform and architecture, the index.json is only modified once all stages succeed. With this said, the latest URL only works when the distribution build job succeeds at least once for the given product and version.

The resolution logic is implemented in the CloudFront URL rewriter. The TTL (time to live) is set to 5 mins which means that the latest URL may need up to 5 mins to get new contents after index.json is updated.

All the artifacts accessible through the regular distribution URL can be accessed by the latest URL. This includes both OpenSearch Core, OpenSearch Dashboards Core and their plugins.

For example, you can download the latest .tar.gz distribution build of OpenSearch 2.2.0 directly at https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.0/latest/linux/x64/tar/dist/opensearch/opensearch-2.2.0-linux-x64.tar.gz, without having to first download and parse the complete build manifest.

For plugin artifacts, you can also use the latest keyword to get the latest plugin artifacts with a known version. E.g. in order to get performance-analyzer x64 tarball artifacts for 2.1.0, you can obtain it with link https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.1.0/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-performance-analyzer-2.1.0.0.zip, which will redirect you to https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.1.0/5757/linux/x64/tar/builds/opensearch/plugins/opensearch-performance-analyzer-2.1.0.0.zip.

For bundled artifacts, here are some examples for LINUX and Windows:

• Linux Tar: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.4.0/latest/linux/x64/tar/dist/opensearch/opensearch-2.4.0-linux-x64.tar.gz
• Windows Zip: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.4.0/latest/windows/x64/zip/dist/opensearch/opensearch-2.4.0-windows-x64.zip

Testing the Distribution

Tests the OpenSearch distribution, including integration, backwards-compatibility and performance tests.

./test.sh <test-type> <test-manifest-path> <path>

See src/test_workflow for more information.

Checking Release Notes

Workflow to check if the release notes exist or not and shows the latest commit for OpenSearch and Dashboard distributions.

To run:

./release_notes.sh check manifests/2.2.0/opensearch-2.2.0.yml --date 2022-07-26

See src/release_notes_workflow for more information.

Signing Artifacts

For all types of signing within OpenSearch project we use opensearch-signer-client (in progress of being open-sourced) which is a wrapper around internal signing system and is only available for authenticated users. The input requires a path to the build manifest or directory containing all the artifacts or a single artifact.

Usage:

./sign.sh builds/opensearch/manifest.yml

The tool currently supports following platforms for signing.

PGP

Anything can be signed using PGP signing eg: tarball, any type of file, etc. A .sig file will be returned containing the signature. OpenSearch and OpenSearch dashboards distributions, components such as data prepper, etc. as well as maven artifacts are signed using PGP signing. See this page for how to verify signatures.

Windows

Windows signing can be used to sign windows executables such as .msi, .msp, .msm, .cab, .dll, .exe, .appx, .appxbundle, .msix, .msixbundle, .sys, .vxd, .ps1, .psm1, and any PE file that is supported by Signtool.exe. Various windows artifacts such as SQL OBDC, opensearch-cli, etc are signed using this method. Windows code signing uses EV (Extended Validated) code signing certificates.

Types of signing/Details	Digest	Cipher	Key Size
PGP	SHA1	AES-128	2048
Windows	SHA256	RSA
RPM	SHA512	RSA	4096

Signing RPM artifacts

RPM artifacts are signed via a legacy shell script which uses a macros template. See this commit for more information and this issue to add RPM artifact signing functionality to the above signing system. Currently we are only signing OpenSearch and OpenSearch dashboards RPM distributions using this method.

See src/sign_workflow for more information.

Making a Release

Releasing for Linux and Windows

The Linux / Windows release is managed by a team at Amazon following this release template (e.g. opensearch-build#2649).

Releasing for FreeBSD

The FreeBSD ports and packages for OpenSearch are managed by a community OpenSearch Team at FreeBSD. When a new release is rolled out, this team will update the port and commit it to the FreeBSD ports tree. Anybody is welcome to help the team by providing patches for upgrading the ports following the FreeBSD Porter's Handbook instructions.

Releasing for macOS

At this moment there's no official macOS distribution. However, this project does support building and assembling OpenSearch for macOS. See opensearch-build#37 and #38 for more details.

Utilities

Checking Out Source

The checkout workflow checks out source code for a given manifest for further examination.

./checkout.sh manifests/1.3.0/opensearch-1.3.0.yml

See src/checkout_workflow for more information.

Cross-Platform Builds

You can perform cross-platform builds. For example, build and assemble a Windows distribution on macOS.

export JAVA_HOME=$(/usr/libexec/java_home) # required by OpenSearch install-plugin during assemble
./build.sh manifests/1.3.0/opensearch-1.3.0.yml --snapshot --platform windows
./assemble.sh builds/opensearch/manifest.yml

This will produce dist/opensearch-1.3.0-SNAPSHOT-windows-x64.zip on Linux and macOS.

Sanity Checking the Bundle

This workflow runs sanity checks on every component present in the bundle, executed as part of the manifests workflow in this repository. It ensures that the component GitHub repositories are correct and versions in those components match the OpenSearch version.

The following example sanity-checks components in the OpenSearch 1.3.0 manifest.

./ci.sh manifests/1.3.0/opensearch-1.3.0.yml --snapshot

See src/ci_workflow for more information.

Auto-Generating Manifests

The manifests workflow reacts to version increments in OpenSearch and its components by extracting Gradle properties from project branches. When a new version is found, a new input manifest is added to manifests, and a pull request is opened (e.g. opensearch-build#491).

Show information about existing manifests.

./manifests.sh list

Check for updates and create any new manifests.

./manifests.sh update

See src/manifests_workflow for more information.

Deploying Infrastructure

This project uses jenkins as the build infrastructure for building, testing and releasing the artifacts. The infrastructure is deployed using CDK and code can be found in opensearch-ci repository.

Contributing

See developer guide and how to contribute to this project.

Getting Help

If you find a bug, or have a feature request, please don't hesitate to open an issue in this repository.

For more information, see project website and documentation. If you need help and are unsure where to open an issue, try forums.

Code of Conduct

This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ, or contact [email protected] with any additional questions or comments.

Security

If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our vulnerability reporting page. Please do not create a public GitHub issue.

License

This project is licensed under the Apache v2.0 License.

Copyright

Copyright OpenSearch Contributors. See NOTICE for details.

opensearch-build's Issues

[IntegTest] Onboard all plugins into automated integration tests for the distribution

Remaining items:

• opensearch-project/notifications#433
• opensearch-project/cross-cluster-replication#383
• opensearch-project/performance-analyzer#198
• opensearch-project/security#1761
• opensearch-project/reporting#348
• opensearch-project/notifications#258
• opensearch-project/observability#676
• opensearch-project/job-scheduler#146
• #1274
• opensearch-project/geospatial#115
• opensearch-project/dashboards-maps#26
• opensearch-project/security-analytics#64
• opensearch-project/security-analytics-dashboards-plugin#17
• opensearch-project/neural-search#24
• opensearch-project/dashboards-search-relevance#8

---

Run the Integration Test cases in the OpenSearch distribution building workflow for all of the plugins
Please make sure your plugin is a part of this manifest file inorder to run integration test via build workflow

⏳ Total Status:

---

🔎 OpenSearch Plugins:

Plugin Name	X64	ARM64	Infra CICD Integrated
AD Plugin	With security without security	With security without security
SQL Plugin	With security without security	With security without security
Alerting Plugin	With security without security	With security without security
PA/RCA Plugin opensearch-project/performance-analyzer#198	With security without security	With security without security
Security Plugin (this plugin relies on multiple different cluster setup, thus not bundled with Infra IntegTest pipeline #58 (comment))	With security	With security
IM Plugin	With security without security	With security without security
KNN Plugin	With security without security	With security without security
Reports Scheduler(with-security test not exist yet track opensearch-project/reporting#348)	With security without security	With security without security
Async-search	With security without security	With security without security
Notebooks(removed plugin)	- [ ] With security - [x] without security	- [ ] With security - [x] without security	- [x]
Job Scheduler(No need to test as other plugin use it already testing it)	- [ ] With security - [ ] without security	- [ ] With security - [ ] without security	- [ ]
CCR opensearch-project/cross-cluster-replication#383	With security without security	With security without security
Notifications (opensearch-project/notifications#258)	With security without security	With security without security
ML-commons	With security without security	With security without security
Observability (they are ok to have w/o security only test @vamsi-amazon for now, will add later opensearch-project/observability#676)	With security without security	With security without security
Geospatial opensearch-project/geospatial#115	With security without security	With security without security
Security Analytics	With security without security	With security without security
Neural Search	With security without security	With security without security

---

📊 OpenSearch Dashboards Plugins:

Plugin Name	X64	ARM64	Infra CICD Integrated
AD Dashboards Plugin	With security without security	With security without security
SQL (Workbench) Dashboards Plugin	With security without security	With security without security
Alerting Dashboards Plugin	With security without security	With security without security
IM Dashboards Plugin	With security without security	With security without security
Security Dashboards Plugin	With security	With security
Notebooks Dashboards Plugin (removed plugin combined with trace analytics since 1.2.0 later as observability)	- [x] With security - [x] without security	- [x] With security - [x] without security	- [x]
Reports Dashboards Plugin	With security without security	With security without security
Gantt Chart Dashboards Plugin	With security without security	With security without security
Observability Dashboards	With security without security	With security without security
Notifications Dashboards opensearch-project/notifications#433	With security without security	With security without security
Dashboards-maps opensearch-project/dashboards-maps#26	With security without security	With security without security
Security-Analytics-Dashboards-Plugin	With security without security	With security without security
Search-Relevance-Plugin	With security without security	With security without security

Document building end-to-end bundle

Is your feature request related to a problem? Please describe.
Users want to be able to reproduce building the end-to-end OpenSearch and OpenSearch dashboard bundle, and are asking how we did it for 1.0 (manually).

• Do #20 for all components
• Document access controls of how a central account can fetch components to make the bundle
• Open source scripts and jobs that generate the bundle assembly
• Sign bundle
• Include a manifest that has all the SHAs of commits of every component bundled
• Publish bundle to artifacts.opensearch.org

[Enhancement] Tweak folder structure to be more simple and clear

Describe the enhancement

The structure in opendistro-build is not well organized, and we end up with a lot of duplicated code and scripts stack in the repo folders. Some scripts even share 80% or more code which is very hard for maintenance and quality control.

See this example:
https://github.com/opendistro-for-elasticsearch/opendistro-build/blob/main/elasticsearch/linux_distributions/opendistro-tar-build.sh
https://github.com/opendistro-for-elasticsearch/opendistro-build/blob/main/kibana/linux_distributions/opendistro-kibana-build.sh

Expected behavior

The root folders should be separated by functionality, then by type, then by product, etc.

.
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── NOTICE
├── README.md
├── config
│   ├── opensearch
│   │   └── opensearch.yml
│   └── opensearch-dashboards
│       └── opensearch_dashboards.yml
├── release
│   ├── docker
│   ├── pkg
│   └── tar
├── scripts
│   └── setup_runners.sh
└── tools
    └── vulnerability-scan
        ├── wss-scan.config
        ├── wss-scan.sh
        └── wss-unified-agent.config

Screenshots

None

Additional context

None

provide no-jdk distribution

Is your feature request related to a problem? Please describe.
we want to be able to control the JDK being used (we have our own base docker images with a JDK configured the way we want (correct vendor, up-to-date security fixes, etc.) and thus don't need a JDK to be shipped with OpenSearch.

Describe the solution you'd like
please provide a no-jdk distribution (at least of the current "minimal" distribution) which - as the name implies - does not come with a pre-packaged JDK.

Describe alternatives you've considered
downloading the existing archive and manually removing the JDK would be a huge overhead and would also make the docker image larger (if we do this in the docker image; we could of course do it in a pre-step but that'd make things even more complicated) and it would be a non-standard setup which is harder for others to replicate.

Additional context
this has been discussed in the forum: No-jdk distribution of opensearch?

Pre-compile and host native plugins on artifacts.opensearch.org

Is your feature request related to a problem? Please describe.
Users are finding it hard to install the native plugins as they are not readily available to download when running sh opensearch-plugin install command

Describe the solution you'd like
All the internal plugins under /OpenSearch/tree/main/plugins are pre-compiled and hosted on https://artifacts.opensearch.org/releases/plugins/* directory

Describe alternatives you've considered
Manually building the plugins on local and making changes to point the install script to use the local version
opensearch-project/OpenSearch#951 (comment)

Reference:
opensearch-project/OpenSearch#951

[BUG] OpenSearch / Dashboards Docker Images is bigger than ODFE ones

Describe the bug
[BUG] OpenSearch / Dashboards Docker Images is bigger than ODFE ones

REPOSITORY                                                                             TAG                IMAGE ID       CREATED         SIZE
opensearchproject/opensearch                                                           1.0.0-rc1          4b324b9a7bca   10 days ago     1.48GB
opensearchproject/opensearch-dashboards                                                1.0.0-rc1          ca040d1558ad   10 days ago     1.58GB

amazon/opendistro-for-elasticsearch                                                    1.13.2             2a84eab453d6   2 months ago    1.25GB
amazon/opendistro-for-elasticsearch-kibana                                             1.13.2             5b24072710ec   2 months ago    1.17GB

Resolution
Seems like yum cache is not correctly removed in dockerfiles (rm -rf /var/cache/yum/*)
Will need to send a PR so it can be fixed in the next releases.

Expected behavior
They should be of similar sizes for both OpenSearch/Dashboards and ODFE images.

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: [Linux/Docker/AL2]
• Version [1.0.0-beta1, 1.0.0-rc1]

Additional context
Add any other context about the problem here.

[Feature] Add Docker Image Creation Scripts and Configs

Is your feature request related to a problem? Please describe.
We need to release the Docker Image Creation Scripts and Configs

Describe the solution you'd like
Related dockerfiles, configuration files, and scripts to automatically build docker images should be provided

Describe alternatives you've considered
A ReadMe if there is no scripts to describe the steps

Additional context
None

[Vote][Proposal] Release OpenSearch/Dashboards Multi-Arch Docker Images

What kind of business use case are you trying to solve? What are your requirements?

[Proposal] Release OpenSearch/Dashboards Multi-Arch Docker Images
Issue: #30

We would like the community to help us with the release.
Please use emotes 1️⃣ or 2️⃣ to record your vote.

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
We have created multi-arch docker image with both x64 and arm64, and have deployed to opensearchstaging repository. These images also consume the latest enhancements/fixes in #93 #95.

docker pull opensearchstaging/opensearch:1.0.0-multi-test
docker pull opensearchstaging/opensearch-dashboards:1.0.0-multi-test

We would like to release OpenSearch/Dashboards Multi-Arch Docker Images to production repository (opensearchproject).

There are two options right now:
(1️⃣) Release it now with the same 1.0.0 tag, this would overwrite the existing 1.0.0 tag.
(2️⃣) Release it during new upcoming 1.1.x release, this would delay the release but would not overwrite existing tags.

What are your assumptions or prerequisites?
Describe any assumptions you may be making that would limit the scope of this proposal.

What are remaining open questions?
List questions that may need to be answered before proceeding with an implementation.

Lower CI Java Version to Java 11

Java 11 is an LTS release, Java 14 is not. Elastic really shouldn't have made the minimum java version 14 prematurely. Lucene doesn't even require minimum 14. The community opened a PR to lower the minimum required java version back to 11 but it is stalled until CI can be lowered to match (otherwise CI will fail). The PR has stalled now for almost two months and the contributor has long given up. This is unfortunate and really quite unacceptable. Maybe we can get this taken care of to unblock that PR.

Update: the remaining work is as follows. OpenSearch has CI defined in a private freestyle project that runs gradle check on OpenSearch#main, branches and PRs. Extract that into Jenkinsfile that is committed to the OpenSearch repo, similar to https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/Jenkinsfile.

OpenSearch / OpenSearch Dashboards - macOS distribution (M1 ARM)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed macOS distribution (ARM64) artifact for OpenSearch / OpenSearch Dashboards

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there are macOS (ARM64) artifacts for OpenSearch / OpenSearch Dashboards

JAVA_HOME missing in dockerhub image

When using the official dockerhub image "opensearchproject/opensearch:1.0.0-rc1" the env var "JAVA_HOME" is not defined, breaking /tools/securityadmin.sh:
https://github.com/opensearch-project/security/blob/2ecb088a1ac58bd1ac4a0c1100492bc1566b9be7/tools/securityadmin.sh#L19

To fix it I manually set the env var JAVA_HOME=/usr/share/opensearch/jdk, but maybe this can be fixed in the image or in the plugin?

Optimize Docker image: reduce half of the size

Is your feature request related to a problem? Please describe.
The Docker image opensearchproject/opensearch is heavy compare with elasticsearch.
opensearch v1.0.0 is 915mb in Docker Hub, while elasticsearch v7.13.3 is 473mb.

Describe the solution you'd like
Optimize Dockerfile build stages. There's a waste of space introduce by a layer which cause the size is duplicated:

Layer 10 have chown command, whilst we can chown it in layer 6 (Dockerfile COPY command has --chown flag).

[Enhancement] Add a new deploy script for quick cluster creation

Is your feature request related to a problem? Please describe.
The original deployment script in ODFE is too complicated and takes a long time to run.
It also does not work outside of the existing GitHub Runners CI/CD workflows.
https://github.com/opendistro-for-elasticsearch/opendistro-build/blob/master/.github/scripts/setup_runners_service.sh

Describe the solution you'd like
We plan to recreate the script with a simpler and unified approach, so that the script can be run in any *nix environment.
The deployment should also be quick.
And it should include a cleanup process.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Create and use new vagrant boxes for packaging tests

Packaging tests use Vagrant boxes from elastic (https://app.vagrantup.com/elastic).

We may need to create new boxes and replace the existing ones in tests.

[BUG] Artifact distribution naming is not consistent with OpenSearch/plugins

Describe the bug
CPU architectures are named x86_64 and aarch64 in OpenSearch, where as rest of the plugins use x64 and arm64.
Also platform for MacOS has different naming conventions, OpenSearch/Dashboards use darwin and the plugins use macos.

• CPU architecture - Done PR opensearch-project/OpenSearch#612
• MacOS Platform

Release a standalone, minimal artifact

Is your feature request related to a problem? Please describe.

Release a standalone, minimal artifact, not just a bundle with plugins installed.

Naming

In https://discuss.opendistrocommunity.dev/t/naming-for-opensearch-1-0-distributions/5789 and https://discuss.opendistrocommunity.dev/t/convergence-of-opensearch-opendistro-plugins/5625/10 we discussed calling it -min.

Additional Context

opensearch-project/OpenSearch#632
opensearch-project/OpenSearch-Dashboards#319
https://github.com/opensearch-project/OpenSearch-Dashboards/issues/340

[BUG] Owner userid of files in opensearchproject/opensearch:1.0.0 causes error with docker + userns

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. docker pull opensearchproject/opensearch:1.0.0
2. When extracting layer f4eb0e854a62 the pull fails with failed to register layer: Error processing tar file(exit status 1): Container ID 12011476 cannot be mapped to a host ID

Expected behavior
The pulls succeeds, as it did for opensearchproject/opensearch:1.0.0-rc1

Host/Environment (please complete the following information):

• OS: Debian 10
• Version 1.0.0
• Docker 20.10.7, Swarm mode

Additional context
Looking at the linux-x64 tarballs of 1.0.0 and 1.0.0-rc1 from https://opensearch.org/artifacts/ the problem seems to come from the user ids stored in the tarball: The tarball opensearch-1.0.0-rc1-linux-x64.tar.gz of 1.0.0-rc1 which works has the files in the tarball owned by uid 995/gid 993

$ tar --numeric-owner -vtf ~/Downloads/opensearch-1.0.0-rc1-linux-x64.tar.gz | head
drwxr-xr-x 995/993           0 2021-06-03 22:49 opensearch-1.0.0-rc1/
drwxr-xr-x 995/993           0 2021-05-28 20:19 opensearch-1.0.0-rc1/lib/
-rw-r--r-- 995/993    13058405 2021-05-28 20:19 opensearch-1.0.0-rc1/lib/opensearch-1.0.0-rc1.jar
-rw-r--r-- 995/993        6546 2021-05-28 20:19 opensearch-1.0.0-rc1/lib/opensearch-plugin-classloader-1.0.0-rc1.jar

while the tarball opensearch-1.0.0-linux-x64.tar.gz of 1.0.0 which does not work has the files in the tarball owned by uid 12011476/gid 100 (note the uid 12011476 which matches the 'Container ID' in the docker error message)

$ tar --numeric-owner -vtf ~/Downloads/opensearch-1.0.0-linux-x64.tar.gz | head
drwxr-xr-x 12011476/100      0 2021-07-09 00:32 opensearch-1.0.0/
-rwxr-xr-x 12011476/100   3132 2021-07-09 00:32 opensearch-1.0.0/opensearch-tar-install.sh
drwxr-xr-x 12011476/100      0 2021-07-09 00:32 opensearch-1.0.0/bin/
-rwxr-xr-x 12011476/100   4890 2021-07-03 01:22 opensearch-1.0.0/bin/opensearch-env

which is outside of the 0 to 65536 uid range normally allowed in a user namespace (due to the default size of the dockremap subuid allocation).

I think the Dockerfile should probably normalize the uid/gid of the extracted files when unpacking the tarball here in line 53 so the uid/gid stored in the tarball does not leak into the container image. The chmod -R on line 81 is too late as at this point there is already a layer with the invalid uid after the copy on line 68.

Support installing Opensearch on macOS using homebrew

OpenSearch / OpenSearch Dashboards - DEB distribution (ARM64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed (ARM64)Deb artifact for OpenSearch / OpenSearch Dashboards

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there are no (ARM) Deb artifacts for OpenSearch / OpenSearch Dashboards

OpenSearch / OpenSearch Dashboards - Docker image (X64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide (X64)Docker images via DockerHub for OpenSearch / OpenSearch Dashboards (With plugins)

Describe alternatives you've considered
Generating the image using the source code

Additional context
Currently there are no Docker images for OpenSearch / OpenSearch Dashboards

Build OpenSearch 1.x branch nightly

Is your feature request related to a problem? Please describe.

We have a bunch of changes on OpenSearch 1.x branch towards 1.0 and plugins are starting to need that. Need an artifact to run integration tests.

Describe the solution you'd like

A .tar.gz build nightly off OpenSearch 1.x branch.

Additional context

Currently waiting on this, is opensearch-project/k-NN#22

[META] Automate backwards compatibility test support for OpenSearch project

Is your feature request related to a problem? Please describe.

We do a lot of manual testing. Need visible and repeatable automation for end-to-end upgrade paths described in https://opensearch.org/faq/#c3, including rolling and restart upgrades. Plugins should be able to contribute setup steps in an older version (data) and integration tests that run in mixed and upgraded cluster states.

Proposal

We would like to see CI/CD that shows that a specific bundle build of OpenSearch has been tested on a list of upgrade paths.
Here is our proposal for BWC/Upgrade testing for OpenSearch project.

All BWC tests include:

1. Rolling upgrade
2. Mixed Cluster search
3. Snapshot upgrade
4. Restart upgrade

All components within opensearch project will define bwctests:

1. Run BWC tests in each component.
2. Run BWC tests in each component in PR push.
3. Run BWC tests in release CI against individual component.
4. Run BWC tests in release CI against the bundle.

Tasks

P0

• Adding bwc test framework support for plugins: opensearch-project/OpenSearch#1002
• Adding BWC test framework automation and documentation: opensearch-project/opensearch-plugins#53
• Adding support to run BWC tests in plugin CI: #222

Release CI BWC test workflow: [Meta] #232

• Adding BWC test framework in release/nightly CI: #275
• Adding Jenkins job support: #492
• Add support for Dependency Installer: #535
• Add support to onboard to test manifest: #572
• Add support for TestRecorder/TestPublisher: #277
• Add support BWC test onboarding for developers: #223

P1

• BWC Tests against the bundle (Meta issue): #121
• Add support for Snapshot upgrades: opensearch-project/OpenSearch#1088
• Add support to run BWC tests in workflow via Docker #278
• Add support to run BWC tests in parallel: #243
• Add support to download previous version artifacts predictably for plugins: #184

Here is the design proposal for #121 :

1. All repositories define their bwc tests for A and B.
2. Nightly builds will run A for repo level testing via bwctest.sh script defined in each repository.
   Ref: https://github.com/opensearch-project/anomaly-detection/blob/main/integtest.sh
   Eg: ./integtest.sh bwcTest
3. We will rely on #122 and a job to the orchestrator to spin up various kinds of BWC test clusters.
4. Nightly builds will also run B for bundle level testing via bwctest.sh defined in each repository using the clusters setup by the orchestrator. The orchestrator job will make calls to the bwctest.sh script to perform various steps of testing during upgrade.
   Eg: For Rolling Upgrade, the orchestrator job will spin an old version (the versions should be defined via a manifest, TBD).
   a. Kick off ingestion via ./bwctest.sh RollingUpgradeIngestTest
   b. Upgrade a node to new version
   c. Kick off mixed cluster test via ./bwctest.sh MixedClusterSearchTest
   d. Upgrade all nodes to the latest version.
   e. Kick off upgraded cluster test via ./bwctest.sh UpgradedClusterTest
   f. Report test status and repeat for every repository.
5. Report bundle level test status.

OpenSearch / OpenSearch Dashboards - RPM distribution (ARM64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed (ARM64)RPM artifact for OpenSearch / OpenSearch Dashboards

• discuss approach to release RPM distribution (ARM64)
• document the approach in a github issue
• automate the RPM distribution builds

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there are no (ARM) RPM artifacts for OpenSearch / OpenSearch Dashboards

Dockerhub Description for Data Prepper wrong?

In the description you find this:

What is OpenSearch Data Prepper?

Data Prepper is a new component of OpenSearch that receives trace data from the OpenTelemetry collector, and aggregates, transforms, and normalizes it for analysis and visualization in Kibana

But should it not be:

What is OpenSearch Data Prepper?

Data Prepper is a new component of OpenSearch that receives trace data from the OpenTelemetry collector, and aggregates, transforms, and normalizes it for analysis and visualization in OpenSearch Dashboard

OpenSearch / OpenSearch Dashboards - TAR distribution (ARM64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed (ARM64)Tar artifact for OpenSearch / OpenSearch Dashboards

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there are no Tar artifacts for OpenSearch / OpenSearch Dashboards

OpenSearch Data-prepper docker image release 1.0.0 for OpenSearch

Is your feature request related to a problem? Please describe.
version 1.0 docker image release on 06/15

Describe the solution you'd like
https://github.com/opensearch-project/data-prepper/tree/1.0/release/docker

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

OpenSearch / OpenSearch Dashboards - macOS distribution (X64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed macOS (X64) artifact for OpenSearch / OpenSearch Dashboards

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there are macOS (X64) artifacts for OpenSearch / OpenSearch Dashboards

• opensearch-project/OpenSearch#1449
• opensearch-project/common-utils#87
• opensearch-project/alerting-dashboards-plugin#138
• opensearch-project/alerting#215
• opensearch-project/anomaly-detection-dashboards-plugin#105
• opensearch-project/anomaly-detection#275
• opensearch-project/asynchronous-search#52
• opensearch-project/dashboards-notebooks#92
• opensearch-project/reporting#192
• opensearch-project/dashboards-visualizations#34
• opensearch-project/index-management-dashboards-plugin#115
• opensearch-project/index-management#173
• opensearch-project/job-scheduler#76
• opensearch-project/k-NN#158
• opensearch-project/notifications#359
• opensearch-project/performance-analyzer#76
• opensearch-project/security-dashboards-plugin#846
• opensearch-project/security#1475
• opensearch-project/sql#250
• opensearch-project/observability#161
• #1589

[Infra] Setup lucene snapshot maven repository

A S3 bucket is needed to store snapshot builds of lucene for periodically updating development branches with evolving lucene features. This is supported in RepositoriesSetupPlugin.java and needs to be changed to an OpenSearch S3 bucket. Builds and publishing artifacts do not need to be automated. Committers and Maintainers should have write access to be able to manually push and update the codebase with updated snapshot builds.

OpenSearch / OpenSearch Dashboards - DEB distribution (X64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed (X64)Debian artifacts for OpenSearch / OpenSearch Dashboards

Describe alternatives you've considered
Generating the artifacts using the source code

Additional context
Currently there are no DEB artifacts for OpenSearch / OpenSearch Dashboards

• Refresh Button

---

• Design: #2460
• PRs:
  • #2526
  • #2950
  • #3060
  • #3090
  • opensearch-project/opensearch-build-libraries#109
  • #3095

Publish snapshots and release artifacts to maven

Currently plugins check out and build OpenSearch in their CI. They should be consuming dependent JARs published into a maven repository that hosts nightly / SNAPSHOT builds and release artifacts.

• Either sonatype or self-hosted Maven on top of S3 with credentials managed by infra
• Nightly or even after each check-in builds of OpenSearch, plugins, etc., for a list of well known branches
• All artifacts for published beta, RCs and releases
• README explains how to onboard any project to build and publish nightlies into Maven (ticket? issue?)
• README explains how to add a branch to be built and published
• Visibility into the builds - #102

[BUG] Install of analysis-icu plugin fails in beta1

Describe the bug
Trying to do install analysis-icu plugin from docker container fails with HTTP 403

To Reproduce
Steps to reproduce the behavior:

1. docker run -ti opensearchproject/opensearch:1.0.0-beta1 bash
2. bin/opensearch-plugin install analysis-icu
3. See error

-> Installing analysis-icu
-> Downloading analysis-icu from opensearch
-> Failed installing analysis-icu
-> Rolling back analysis-icu
-> Rolled back analysis-icu
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/analysis-icu/1.0.0/analysis-icu-1.0.0-beta1.zip
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1932)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1528)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224)
	at org.opensearch.plugins.InstallPluginCommand.downloadZip(InstallPluginCommand.java:438)
	at org.opensearch.plugins.InstallPluginCommand.downloadAndValidate(InstallPluginCommand.java:515)
	at org.opensearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:306)
	at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:260)
	at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:237)
	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:99)
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
	at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
	at org.opensearch.cli.Command.main(Command.java:103)
	at org.opensearch.plugins.PluginCli.main(PluginCli.java:60

Expected behavior
Plugin installed

Plugins
(none)

Screenshots
(None)

Host/Environment (please complete the following information):
Docker (macos)

Additional context
(none)

on-board opensearch-cli

Is your feature request related to a problem? Please describe.
on-board opensearch-cli
Describe the solution you'd like
We should able to upload our artifacts from github actions, run integration tests, package artifact and available from download page.

Describe alternatives you've considered
NA

Additional context
This is similar to odfe-cli.

Provide Amazon Linux 2 or non CentOS Docker Images

See original description here:
opendistro-for-elasticsearch/opendistro-build#696

Currently only CentOS 8 images are supported:
https://github.com/opensearch-project/OpenSearch/blob/0ba0e7cc26060f964fcbf6ee45bae53b3a9941d0/buildSrc/src/main/java/org/opensearch/gradle/DockerBase.java#L39

Would Alpine Images will be possible too?

Run end-to-end automated integration tests for bundle

Is your feature request related to a problem? Please describe.

We currently don't have/run integration tests on the OpenSearch bundle, including rolling and restart upgrades that include plugins.

Describe the solution you'd like

Extend the OpenSearch backwards compatibility infrastructure to install the bundle and perform checks such as, for example, whether security is enabled by default. Repeat with upgrades.

Release a standalone artifact, not just a bundle with plugins installed

Release a standalone -min artifact, not just a bundle with plugins installed.

Docker image on Kubernetes: complaines about ES_TMP dir

I'm trying to build a Terraform module to deploy OpenSearch to k8s. The OpenDistro images are running fine, if I spin up the OpenSearch images locally (single node) it's fine as well. But when I try to run it on k8s it's not starting:

Can you give me some pointers as to where this error originates? I tried setting the ES_TMPDIR to the data directory and creating an /tmp but no sigar.

OpenSearch / OpenSearch Dashboards Multi-Arch Docker image (X64 + ARM64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide (ARM64)Docker images via DockerHub for OpenSearch / OpenSearch Dashboards (With plugins)

Describe alternatives you've considered
Generating the image using the source code

Additional context
Currently there are no (ARM) Docker images for OpenSearch / OpenSearch Dashboards

on-board beats

On board following beats

1.     packetbeat-oss
2.     heartbeat-oss
3.     auditbeat-oss
4.     journalbeat-oss
5.     filebeat-oss
6.     metricbeat-oss

Turn on DCO checks for all opensearch plugins

Just like OpenSearch/Dashboards plugins need to have DCO checks that block merges unless passed.

Add dashboards-reports to integTest automation

Add integration with front-end dashboards-reports and back-end reports-scheduler for the dashboards-reports plugin.

---

Command to run integration tests for dashboards-reports (front-end plugin):

yarn test

Prerequisites:

• OpenSearch sample data (ecommerce, flights, logs) must be installed

---

Command to run integration tests for back-end reports-scheduler:

./gradlew integTest

Remote cluster:

./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200

OpenSearch / OpenSearch Dashboards - TAR distribution (X64)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed (X64)TAR artifact for OpenSearch / OpenSearch Dashboards

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there is no TAR artifact for both OpenSearch / OpenSearch Dashboards

Support FreeBSD

Is your feature request related to a problem? Please describe.

https://twitter.com/HackacadNet/status/1419267845704323080

Describe the solution you'd like

Upstream changes, provide a distribution.

Additional context

Not sure what it takes to do a FreeBSD distribution, but at the very least no code changes should be required.

Document and make GA release process repeatable and reproducible

Is your feature request related to a problem? Please describe.

Anyone should be able to run an OpenSearch release via a documented and reproducible process (minus access to signing keys).

Describe the solution you'd like

A document that anyone could follow to produce an OpenSearch alpha/beta/GA release.

• Complete #20 for each component
• Complete #73 for the bundle
• Document the process

Additional context

After a release someone asked where the 1.0.0 tags were. Our release process is not documented and expectations of when such things as tags are done are unclear.

[BUG] failed to register layer: Error processing tar file(exit status 1): Container ID 12011476 cannot be mapped to a host ID

Describe the bug
Pulling the opensearch 1.0.0 image fails with the following error

❯ docker pull opensearchproject/opensearch:1.0.0
1.0.0: Pulling from opensearchproject/opensearch
99a21848b482: Pull complete 
f4eb0e854a62: Extracting [==================================================>]  432.5MB/432.5MB
4f4fb700ef54: Download complete 
ffd3f715ed97: Download complete 
d19997574184: Download complete 
6f1e5d14588b: Download complete 
3b0e6bcbd287: Download complete 
failed to register layer: Error processing tar file(exit status 1): Container ID 12011476 cannot be mapped to a host ID

To Reproduce
Steps to reproduce the behavior:

1. Run docker pull opensearchproject/opensearch:1.0.0
2. See error

Expected behavior
docker pull should succeed

Plugins
N/A

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: Ubuntu
• Version: 21.04

Additional context
N/A

[Meta] OpenSearch / OpenSearch Dashboards - Windows distribution (X64)

Describe the solution you'd like
Provide signed (X64)Windows artifact for OpenSearch / OpenSearch Dashboards

Additional context
Currently there are no Windows artifacts for OpenSearch / OpenSearch Dashboards

Current State (2021/10/28)

You can build on Windows or build cross-platform for Windows on Linux or MacOS.

1. Install pre-requisites (everything needed in OpenSearch Dev Guide + maven for Security Plugin).
2. Build artifacts from source, ./build.sh manifests/1.1.0/opensearch-1.1.0.yml --platform windows
3. Assemble a bundle, ./assemble.sh buids/manifest.yml.
4. Produces opensearch-1.1.0-SNAPSHOT-windows-x64.zip.
5. Extract .zip, e.g. into C:\Program Files\OpenSearch
6. Run bash plugins/opensearch-security/tools/install_demo_configuration.sh and follow prompts to create demo certificate.
7. mkdir logs (opensearch-project/OpenSearch#1463)
8. Run bin\opensearch.bat, server will start.
9. curl https://localhost:9200 -u admin:admin --insecure

(Old Tasks)

Remaining

• Add Windows CI into OpenSearch and every component, passing tests
  • opensearch-project/OpenSearch#1448
  • opensearch-project/OpenSearch#1463
  • opensearch-project/security#1476
  • opensearch-project/common-utils#86
  • opensearch-project/opensearch-plugins#95
  • opensearch-project/k-NN#152
  • opensearch-project/sql#247
  • opensearch-project/common-utils#85
  • opensearch-project/performance-analyzer#74
  • #828
  • #826
    #827
• Create a Windows docker image to build on Windows
• Add Windows build to Jenkins
• #1430

Done

• opensearch-project/OpenSearch#1379
  opensearch-project/OpenSearch#1412
  #767
  #768
  #772
• #803
• #804
  #807
• #809

Design Document

• #2312

OpenSearch and OpenSearch Dashboards

• opensearch-project/OpenSearch#1448
• opensearch-project/OpenSearch-Dashboards#2451

Plugins Meta Issues

• opensearch-project/opensearch-plugins#95

Milestones

• #2306
• #2307
• #2308
• #2309
• #2310
• #2311

MSI Only Issue:

• #2929

securityadmin.sh can't find JAVA_HOME in docker container

Describe the bug
When setting customized certificates in a dockerized cluster, you need to call securityadmin.sh from ./plugins/opensearch-security/tools in a running container. The script securityadmin.sh is unable to find JAVA_HOME. After setting exporting JAVA_HOME like
export JAVA_HOME=/usr/share/opensearch/jdk
securityadmin.sh can be called without problems.
There should be a JAVA_HOME Variable set in the docker container

OSX Builds

It would be very nice to have a OSX build similar to what can be downloaded as a tar file directly with elasticsearch... The docker is nice but having something approachable where you can easily get to the configuration file and make changes would really help with tinkering and learning with opensearch... alternatively we'll just use elasticsearch directly to learn on but then as features drift it'll be less likely to look at opensearch again...

[Meta] OpenSearch / OpenSearch Dashboards - Support for RPM distribution

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Provide signed RPM artifact for OpenSearch / OpenSearch Dashboards

• discuss approach to release RPM distribution
• document the approach in a release documentation #956
• Publish the proposed based design of RPM with new requirements and execution plan
• #1117
  • #1443
  • Assemble process will have new param --distribution to specify which distribution to build
  • Uncouple distribution creation with min tarball/zip file extension
  • RPM will be built based on min tar + plugin zip installation + fpm bundle pkg
  • User should be able to use ./assemble builds/manifests.yml --distribution rpm to build rpm and get a rpm file in return for installation.
  • We design it to run on a specific Docker Image with ruby, rpm, fpm installed, not able to run directly on macOS directly. We will make changes to the docker files as well
  • Part of the pre/post scripts and sysvinit/systemd will be taken from ODFE.
  • Also need these scripts specifically for pkg from PA: https://github.com/opensearch-project/performance-analyzer/tree/main/packaging
• We will add the documentation to our design docs and propose to the community
  • #1452
• #1244

---

Tasks based on the updated requirements in #1452

[Milestone 1] OpenSearch build process design for RPM generation

• #1543
• #1544
• #1545

[Milestone 2] RPM Package Signing

• #1547

[Milestone 3] RPM Package Design

• #1548

[Milestone 4] RPM Package Publishing

• #1549
  • #2014
• #1550

[Milestone 5] RPM Package installation

• #1551
• #1552
• #1553

[Milestone 6] RPM Validation

• #1554
• #1555
• #1556
• #1557
• #1558
• #1951

Describe alternatives you've considered
Generating the artifact using the source code

Additional context
Currently there are no RPM artifacts for OpenSearch / OpenSearch Dashboards

[BUG] Docker OpenSearch-Dashboards Image does not work with Notebook Dashboards Plugins

Describe the bug
Docker OpenSearch-Dashboards images does not seem to work with Notebook Dashboards Plugin.
The default font is missing due to dependencies not installed there.

To Reproduce
Docker Pull from opensearchstaging/opensearch-dashboards:1.0.0-beta1
And run notebooks from the kibana gui.

Expected behavior
Notebooks show text with default fonts.

Plugins
Please list all plugins currently enabled.

Screenshots

[0508/000307.988915:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0508/000308.002757:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
Fontconfig error: Cannot load default config file
[0508/000308.050218:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0508/000308.067339:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0508/000308.078742:FATAL:platform_font_skia.cc(93)] Check failed: InitDefaultFont(). Could not find the default font
#0 0x55cac8bc7bf9 <unknown>
#1 0x55cac8b34783 <unknown>
#2 0x55cac8b47e23 <unknown>
#3 0x55cac952349e <unknown>
#4 0x55cac952488b <unknown>
#5 0x55cac951cb5e <unknown>
#6 0x55cac7d5a4d3 <unknown>
#7 0x55cac7e5fac6 <unknown>
#8 0x55cac7bc61ae <unknown>
#9 0x55cac7bc594d <unknown>
#10 0x55cac7bc5412 <unknown>
#11 0x55cac7b79d87 <unknown>
#12 0x55cac7b994b1 <unknown>
#13 0x55cac7b800b6 <unknown>
#14 0x55cac7b7fa38 <unknown>
#15 0x55caca10d237 <unknown>
#16 0x55cac8c5a9a9 <unknown>
#17 0x55cac8b2c964 <unknown>
#18 0x55cac7929f5f <unknown>
#19 0x55cac7091087 <unknown>
#20 0x55cac7aebc3e <unknown>
#21 0x55cac8b9e499 <unknown>
#22 0x55cac8b7537f <unknown>
#23 0x55cac8b852de <unknown>
#24 0x55cac8b84fd8 <unknown>
#25 0x55cac8bdc1d9 <unknown>
#26 0x55cac8b85bc9 <unknown>
#27 0x55cac8b62957 <unknown>
#28 0x55cac7a37273 <unknown>
#29 0x55cac7a38bf2 <unknown>
#30 0x55caca11630a <unknown>
#31 0x55cac8af4689 <unknown>
#32 0x55cac8af4386 <unknown>
#33 0x55caca00cced <unknown>
#34 0x55cac8aedad1 <unknown>
#35 0x55cac8b30c3d <unknown>
#36 0x55cac8b309bb <unknown>
#37 0x7f1108d8a0ba __libc_start_main
#38 0x55cac703702a _start
Task trace:
#0 0x55cac8b2c50b <unknown>
#1 0x55cac8b2c50b <unknown>
Received signal 6
#0 0x55cac8bc7bf9 <unknown>
#1 0x55cac8b34783 <unknown>
#2 0x55cac8bc7781 <unknown>
#3 0x7f110a44b7e0 <unknown>
#4 0x7f1108d9cc20 __GI_raise
#5 0x7f1108d9e0c8 __GI_abort
#6 0x55cac8bc65c5 <unknown>
#7 0x55cac8b48107 <unknown>
#8 0x55cac952349e <unknown>
#9 0x55cac952488b <unknown>
#10 0x55cac951cb5e <unknown>
#11 0x55cac7d5a4d3 <unknown>
#12 0x55cac7e5fac6 <unknown>
#13 0x55cac7bc61ae <unknown>
#14 0x55cac7bc594d <unknown>
#15 0x55cac7bc5412 <unknown>
#16 0x55cac7b79d87 <unknown>
#17 0x55cac7b994b1 <unknown>
#18 0x55cac7b800b6 <unknown>
#19 0x55cac7b7fa38 <unknown>
#20 0x55caca10d237 <unknown>
#21 0x55cac8c5a9a9 <unknown>
#22 0x55cac8b2c964 <unknown>
#23 0x55cac7929f5f <unknown>
#24 0x55cac7091087 <unknown>
#25 0x55cac7aebc3e <unknown>
#26 0x55cac8b9e499 <unknown>
#27 0x55cac8b7537f <unknown>
#28 0x55cac8b852de <unknown>
#29 0x55cac8b84fd8 <unknown>
#30 0x55cac8bdc1d9 <unknown>
#31 0x55cac8b85bc9 <unknown>
#32 0x55cac8b62957 <unknown>
#33 0x55cac7a37273 <unknown>
#34 0x55cac7a38bf2 <unknown>
#35 0x55caca11630a <unknown>
#36 0x55cac8af4689 <unknown>
#37 0x55cac8af4386 <unknown>
#38 0x55caca00cced <unknown>
#39 0x55cac8aedad1 <unknown>
#40 0x55cac8b30c3d <unknown>
#41 0x55cac8b309bb <unknown>
#42 0x7f1108d8a0ba __libc_start_main
#43 0x55cac703702a _start
  r8: 0000000000000000  r9: 00007fffff1330a0 r10: 0000000000000008 r11: 0000000000000246
 r12: 00007fffff134360 r13: 00007fffff133348 r14: 00007fffff134368 r15: 00007fffff134370
  di: 0000000000000002  si: 00007fffff1330a0  bp: 00007fffff1332f0  bx: 0000000000000006
  dx: 0000000000000000  ax: 0000000000000000  cx: 00007f1108d9cc20  sp: 00007fffff1330a0
  ip: 00007f1108d9cc20 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

Host/Environment (please complete the following information):

• OS: Linux AL2
• Version: 1.0.0-beta1

Additional context
Add any other context about the problem here.

[BUG] DCO check does not check that a valid e-mail is used

Looking at opensearch-project/OpenSearch@7c50669 the DCO check passed, but this wasn't a valid e-mail address.