engineerd / wasm-to-oci

Use OCI registries to distribute Wasm modules

WASM to OCI

The goal of this project is to propose an implementation of storing WebAssembly modules in OCI registries.

This project is built with the ORAS project, and currently works with:

• Distribution (open source, version 2.7+)
• Azure Container Registry
• Google Container Registry
• Google Artifact Registry
• Harbor Container Registry v2.0
• Bundle Bar
• GitHub Package Registry

Note that trying to push a WebAssembly module to Docker Hub is not supported at the time of writing this document, as Docker Hub does not accept unknown artifact types.

As more registries add support for OCI Artifacts, we will update the list of supported registries.

Installation

Pre-built binaries for macOS, Linux, and Windows are available in Releases. Download the pre-built binary for your operating system and add it to your path.

The following example shows adding the Linux binary to your path in Linux.

mv linux-amd64-wasm-to-oci wasm-to-oci
chmod +x wasm-to-oci
sudo cp wasm-to-oci /usr/local/bin

Usage

• login to your container registry using the docker CLI (or other tooling that your container registry provides. wasm-to-oci will use the credentials in ~/.docker/config.json)

• pushing to an OCI registry:

$ ls testdata
.rwxr-xr-x 4.1M radu canonicaljson.wasm
.rwxr-xr-x 1.6M radu  hello.wasm

$ wasm-to-oci push testdata/hello.wasm <oci-registry>.azurecr.io/wasm-to-oci:v1

Pushed: <oci-registry>.azurecr.io/wasm-to-oci:v1
Size: 1624962
Digest: sha256:9c82cbe576ee947c00435ac8053a800a1969f4757ae4a81f870f714674afc91a

• pulling from an OCI registry:

$ wasm-to-oci pull <oci-registry>.azurecr.io/wasm-to-oci:v1 --out test.wasm

Pulled: <oci-registry>.azurecr.io/wasm-to-oci:v1
Size: 1624962
Digest: sha256:4c7915b4c1f9b0c13f962998e4199ceb00db39a4a7fa4554f40ae0bed83d9510

$ wasmtime test.wasm
Hello from WebAssembly!

How does this work?

This leverages the OCI Artifacts proposal, whose goal is to enable the distribution of more cloud native artifacts using existing registry infrastructure, and uses it to store WebAssembly modules as single layer blobs in the registry.

This project defines a new set of unofficial media types used to identify a WebAssembly artifact - the artifacts project also describes the process for projects to apply for an official unique media type.

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.wasm.config.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.wasm.content.layer.v1+wasm",
      "digest": "sha256:4c7915b4c1f9b0c13f962998e4199ceb00db39a4a7fa4554f40ae0bed83d9510",
      "size": 1624962
    }
  ]
}

There is also experimental support for artifact signing with Notary v1 - see this article for more background on this topic.

wasm-to-oci's Issues

Add Bundle Bar as a supporting registry

The new bundle.bar artifact registry has now [support for WASM media types and wasm-to-oci.

This issue tracks adding it to the list of supporting registries.

Add GitHub Package Registry to list of supported services

GitHub Package Registry now allows Wasm artifacts to be pushed and pulled.
We should add it to the list.

Define and add media type for WITX files

WITX is a textual file format used to represent interfaces in WASI.

The CLI should be able to also push a WITX definition together with a module.

Pushing to GCR errors with `"docker-credential-gcloud": executable file not found in $PATH`

$ gcloud version
Google Cloud SDK 304.0.0
bq 2.0.58
core 2020.07.31
gsutil 4.52

wasm-to-oci version 0.1.1

Running wasm-to-oci push target/wasm32-wasi/debug/demo.wasm gcr.io/<registry_path> returns
`Error: error getting credentials - err: exec: "docker-credential-gcloud": executable file not found in $PATH, out: ```

docker pushing to gcr.io works fine.

Test and add AWS ECR to the list of supported registries

According to this blog post, AWS ECR should have support for storing OCI artifacts, meaning it should accept Wasm modules as well.
We should test that this is the case, then add it to the list.

If anyone with an AWS account wants to test it, that would be great!

Dockerhub now supports wasm modules - README inaccurate?

Not 100% clear to me if this is "complete" hence no activity for > 12 months, or depreciated/superseded?

README states

Note that trying to push a WebAssembly module to Docker Hub is not supported at the time of writing this document, as Docker Hub does not accept unknown artifact types.

Which is no longer accurate, or more accurately it does support wasm modules

Please allow different media types

If I am understanding this correctly, wasm-to-oci uses application/vnd.wasm.content.layer.v1+wasm media type. In the official artifact layer type, we can see this media type: https://github.com/opencontainers/artifacts/blob/main/artifact-authors.md#example-layer-types.

Is there any future plans to allow different media types?

New release

The last release seems to be a year old. It would be nice to have a release from the current main branch

ping @ThorstenHans @radu-matei

Also if you lack time to maintain the lib, I'll happy to step in and help

Try pushing to docker hub and fail

Is pushing to docker hub supported?

wasm-to-oci --log debug push target/wasm32-wasi/debug/hello-world-rust.wasm docker.io/jingweno/wasm:1234
Error: failed commit on ref "manifest-sha256:beb9a5b4269a91d4f6abb5d7c7c0e60bc0db2b90f38cac467aa9866791e73b1e": unexpected status: 403 Forbidden
Usage:
  wasm-to-oci push <module> <reference> [options] [flags]

Flags:
  -h, --help   help for push
      --sign   Signs the WebAssembly module and pushes the metadata to a trust server

Global Flags:
  -d, --dir string         Directory where the trust data is persisted to (default "/Users/jou/.wasm-to-oci")
      --insecure           allow connections to SSL registry without certs
      --log string         Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info")
      --server string      The trust server used (default "https://notary.docker.io")
  -t, --timeout string     Timeout for the trust server (default "5s")
      --tlscacert string   Trust certs signed only by this CA
      --use-http           use plain http and not https

Google Container Registry

I found this solution via DeisLabs' Krustlet

I'd like to use Google Container Registry (GCR) with wasm-to-oci but I'm struggling with the auth (pass-through to ORAS).

GCR supports OCI and, using ORAS, I'm able:

• authenticate using a (Google) access token (and demonstrate that GCR support OCI containers).
• push|pull images to a GCR repo.

REGISTRY="gcr.io"

gcloud auth print-access-token \
| oras login -u oauth2accesstoken --password-stdin https://${REGISTRY}

more ./artifact.txt 
Hello Freddie!

oras push gcr.io/${PROJECT}/hello-artifact:v1 ./artifact.txt
Uploading 0060e7e892de artifact.txt
Pushed gcr.io/${PROJECT}/hello-artifact:v1
Digest: sha256:c330be5cf8bbc34f87db380aaba72e774d7ee1064085b3e2ec2b071be292e2a2

rm artifact.txt 

oras pull gcr.io/${PROJECT}/hello-artifact:v1
Downloaded 0060e7e892de artifact.txt
Pulled gcr.io/${PROJECT}/hello-artifact:v1
Digest: sha256:c330be5cf8bbc34f87db380aaba72e774d7ee1064085b3e2ec2b071be292e2a2

cat ./artifact.txt 
Hello Freddie!

If I could understand how to reflect this authentication using wasm-to-oci, I think GCR could be added to your list of supported registries.

TODO: define media type for WebAssembly Text files

Currently, there is only support and mediaType defined for .wasm files.

There should also be support for .wat files.

wasmtime cannot run the wasm file in testdata

When I run hello.wasm using wasmtime, it reports the following error

Error: failed to run main module `hello.wasm`

Caused by:
    0: failed to instantiate "hello.wasm"
    1: command export '__rustc_debug_gdb_scripts_section__' is not a function

However, when I use the wasmtime official example ,which is the one compiled with rust. He will run successfully

I don't know if this problem is due to testdata, wasmtime version update, or my use

Cut a release and add cross platform binaries

Add a version command

It would be nice to have a version command to see what version of the wasm-to-oci CLI is being run. I can contribute this if its ok

Update Go to 1.13 and modules