Pre-compile and host native plugins on artifacts.opensearch.org about opensearch-build

[peterzhuamazon]

Push to candidate channel and did a test on repository-s3, which works:

$ ./opensearch-plugin install repository-s3
-> Installing repository-s3
-> Downloading repository-s3 from opensearch
[=================================================] 100%
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission opensearch.allow_insecure_settings read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed repository-s3 with folder name repository-s3

$ ll ../plugins/ | grep repository-s3
drwxr-xr-x 2 opensearch opensearch 4096 Oct  1 21:06 repository-s3

[AmiStrn]

It would be great to have a fix for this backported as far as possible. since some users would not be able to move to v1.0.0 without this fixed (therefore it would be a major breaking change between Elasticsearch and OpenSearch).
Additionally, those that are already using the workaround would benefit from this as they could avoid the pains of upgrading if they are not interested in v1.1.0 yet.

[peterzhuamazon]

We have fixed the issue in opensearch-project/OpenSearch#1233.
Native plugin for 1.1.0 will be available once 1.1.0 release.
Thanks

[peterzhuamazon]

@mch2

Run the command in both snapshots and releases version of min shows 2 different behaviors:

# Snapshots
$ ./opensearch-plugin install repository-s3
-> Installing repository-s3
-> Failed installing repository-s3
-> Rolling back repository-s3
-> Rolled back repository-s3
ERROR: attempted to install release build of official plugin on snapshot build of OpenSearch

# Releases
$ ./opensearch-plugin install repository-s3
-> Installing repository-s3
-> Downloading repository-s3 from opensearch
-> Failed installing repository-s3
-> Rolling back repository-s3
-> Rolled back repository-s3
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/repository-s3/1.0.0/repository-s3-1.0.0.zip
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1932)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1528)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224)
	at org.opensearch.plugins.InstallPluginCommand.downloadZip(InstallPluginCommand.java:438)
	at org.opensearch.plugins.InstallPluginCommand.downloadAndValidate(InstallPluginCommand.java:515)
	at org.opensearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:306)
	at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:260)
	at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:237)
	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:99)
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
	at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
	at org.opensearch.cli.Command.main(Command.java:103)
	at org.opensearch.plugins.PluginCli.main(PluginCli.java:60)

[peterzhuamazon]

Related issues during release of these artifacts:
opensearch-project/OpenSearch#996

[pkalliok-aiven]

We are also very much looking for mapper-size and analysis plugins. Is there any way to install them currently? Just build from source?

[tlfeng]

We are also very much looking for mapper-size and analysis plugins. Is there any way to install them currently? Just build from source?

Hi @pkalliok-aiven I think currently the the artifacts of the native plugins exist in artifacts.opensearch.org, but there is an issue when installing a plugin with its name (mentioned in issue opensearch-project/OpenSearch#996).
However, you could install by the URL, for example:
./opensearch-plugin install https://artifacts.opensearch.org/releases/plugins/mapper-size/1.0.0/mapper-size-1.0.0.zip

[peterzhuamazon]

native-plugins-1.1.0.log

[peterzhuamazon]

The build is completed for now.
We will use #686 to track the release of native plugins in 1.1.0.

[peterzhuamazon]

1.1.0 Native Plugins released for now.