canonical / kserve-rocks

ROCKs for Kserve

kserve-rocks

ROCKs for Kserve

kserve-rocks's Issues

Add CI for this repository

Context

We want to have CI for building testing and publishing ROCKS from this repository similar to this

What needs to get done

1. Implement CI for building, running tests and publishing ROCKS in this repository

Definition of Done

1. Functioning CI

update repo settings for branch protection, reviews, etc

Bug Description

This repo does not have a CODEOWNERS file or apply the typical branch protections and other settings. This should be fixed

To Reproduce

Environment

Relevant Log Output

-

Additional Context

No response

Create a rock for `kserve/agent:v0.11.2`

Context

Create a rock for KServe's agent based on its upstream Dockerfile.
Please make sure the latest version of kserve v0.11.2 is used.

What needs to get done

1. Create the rockcraft project that will be equivalent to the dockerfile
2. Add sanity tests to the rock directory
3. Add tox.ini file for enabling testing

Definition of Done

The rockcraft project is merged in main and the resulting container image is published in the charmedkubeflow container image registry.

Create a rock for `kserve/controller:v0.11.2`

Context

Create a rock for KServe's controller based on its upstream Dockerfile.
Please make sure the latest version of kserve v0.11.2 is used.

What needs to get done

1. Create the rockcraft project that will be equivalent to the dockerfile
2. Add sanity tests to the rock directory
3. Add tox.ini file for enabling testing

Definition of Done

The rockcraft project is merged in main and the resulting container image is published in the charmedkubeflow container image registry.

Add rock debugging information to `CONTRIBUTING.md`

Context

During rock integration here and here we encountered issues because the rocks we created were missing dependencies that required executing models to discover. We should add testing instructions, if not full scale integration tests, to capture this.

What needs to get done

1. add instructions like those in #28 to CONTRIBUTING.md
2. (stretch) automated integration tests using the above

Definition of Done

1. integration testing for rocks in this repo is documented

Update `sklearnserver` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

pmmlserver rock missing required dependency java

Bug Description

On trying to integrate the pmmlserver into the kserve controler for canonical/kserve-operators#208 it was found that the model server requires java be installed. We missed this I think because:

• the pmmlserver only tries to use java once it has a model. We have no integration tests in this rock repo, so likely we did not actually mount a model and try executing it during manual tests
• its easy to miss in the upstream dockerfile - they build FROM an image that includes java

To Reproduce

We can manually reproduce this by:

# Install the gcloud CLI tool: https://cloud.google.com/sdk/docs/install

# download the model locally
gsutil cp -r gs://kfserving-examples/models/pmml ./sample_model

# Build the rock and put it to docker
tox -e pack
tox -e export-to-docker

# mount the model into the container at runtime
docker run -v PATH_TO/sample_model:/mnt/models PMMLSERVER_IMAGE --model_name pmml-demo --model_dir=/mnt/models/pmml --http_port=8080

Environment

Relevant Log Output

-

Additional Context

No response

Update `xgbserver` ROCK to `0.13.0`

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

paddleserver rock missing `libgoml1`

Bug Description

#20 added the paddleserver rock, but during integration into the kserve operator we observed this:

2024-02-23T21:30:23.886Z [pebble] Service "paddleserver" stopped unexpectedly with code 1
2024-02-23T21:30:23.886Z [pebble] Service "paddleserver" on-failure action is "restart", waiting ~16s before restart (backoff 6)
2024-02-23T21:30:40.816Z [pebble] Service "paddleserver" starting: python -m paddleserver [ '--model_name= paddle-resnet50' --model_dir=/mnt/models --http_port=8080 ]
2024-02-23T21:30:41.713Z [paddleserver] Error: Can not import paddle core while this file exists: /usr/local/lib/python3.10/dist-packages/paddle/base/libpaddle.so
2024-02-23T21:30:41.723Z [paddleserver] Traceback (most recent call last):
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
2024-02-23T21:30:41.723Z [paddleserver]     mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/lib/python3.10/runpy.py", line 146, in _get_module_details
2024-02-23T21:30:41.723Z [paddleserver]     return _get_module_details(pkg_main_name, error)
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
2024-02-23T21:30:41.723Z [paddleserver]     __import__(pkg_name)
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/local/lib/python3.10/dist-packages/paddleserver/__init__.py", line 15, in <module>
2024-02-23T21:30:41.723Z [paddleserver]     from .model import PaddleModel  # noqa # pylint: disable=unused-import
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/local/lib/python3.10/dist-packages/paddleserver/model.py", line 18, in <module>
2024-02-23T21:30:41.723Z [paddleserver]     from paddle import inference
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/local/lib/python3.10/dist-packages/paddle/__init__.py", line 28, in <module>
2024-02-23T21:30:41.723Z [paddleserver]     from .base import core  # noqa: F401
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/local/lib/python3.10/dist-packages/paddle/base/__init__.py", line 36, in <module>
2024-02-23T21:30:41.723Z [paddleserver]     from . import core
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/local/lib/python3.10/dist-packages/paddle/base/core.py", line 380, in <module>
2024-02-23T21:30:41.723Z [paddleserver]     raise e
2024-02-23T21:30:41.723Z [paddleserver]   File "/usr/local/lib/python3.10/dist-packages/paddle/base/core.py", line 268, in <module>
2024-02-23T21:30:41.723Z [paddleserver]     from . import libpaddle
2024-02-23T21:30:41.723Z [paddleserver] ImportError: libgomp.so.1: cannot open shared object file: No such file or directory
2024-02-23T21:30:41.769Z [pebble] Service "paddleserver" stopped unexpectedly with code 1
2024-02-23T21:30:41.769Z [pebble] Service "paddleserver" on-failure action is "restart", waiting ~30s before restart (backoff 7)

This is caused by a missing dependency libgoml1, which the rock build uses during the build stage but does not stage for the final image. This mistake was missed because we tested the rock by running it without an actual model. More complete local testing could have caught it by doing what was listed in the reproduction section.

To Reproduce

With the image locally, do:

# download the model locally
gsutil cp -r gs://kfserving-examples/models/paddle/resnet ./sample_model/

# mount the model into the container at runtime
docker run -v PATH_TO/sample_model:/mnt/models PADDLESERVER_IMAGE --model_name paddle-resnet50 --model_dir=/mnt/models/resnet --http_port=8080

Environment

Relevant Log Output

-

Additional Context

No response

Use the entrypoint-service for pmml server

Context

To pass the arguments to workload container in ROCK we can now use the entrypoint-service option.

What needs to get done

• For the pmml-server rock use the entrypoint-service option.

Definition of Done

pmml-server ROCK uses the entrypoint-service option.

Update `kserve-controller` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

Create a ROCK for kserve/sklearnserver v0.11

Context

Create ROCK based on upstream Dockerfile. You can take the inspiration from pmml server ROCK in main branch. Don't forget to use ROCK's entrypoint service to correctly pass the arguments to workload container.

What needs to get done

1. Create ROCK
2. Manually push the ROCK to Dockerhub

Definition of Done

1. ROCK code is ready
2. ROCK is available at Dockerhub

Bump `pmmlserver` to 0.11.2

Context

Bump the KServe's pmmlserver rock to 0.11.2. This image is based on this Dockerfile.

What needs to get done

• Bump the version and reconcile any changes in the upstream dockerfile
• Test the rock (manually)

Definition of Done

1. rock is published

Create pmmlserver ROCK

Description

PMMLServer ROCK is part of a collectoin of KServe ROCKs. It needs to be created using ROCK building best practices.
It should be based on Kserver Docker container, eg. https://github.com/kserve/kserve/blob/master/python/pmml.Dockerfile
More information on PMMLServer can be found here
This is OpenJDK based container.

More details on Kserve can be found here
Charmed Kserver operators repository contains code that manages Kserve in Charmed Kubeflow deployments: https://github.com/canonical/kserve-operators

Bump the paddle rock to v0.11.2

Context

Bump the rock to the latest v0.11.x version

What needs to get done

1. bump the rock version and publish

Definition of Done

1. rock is published

Create a ROCK for xgbserver v0.11.2

Context

Similarly to #18 and #19 we should create a ROCK for the xgbserver of KServe for CKF 1.8

What needs to get done

1. Create the ROCK, which is based on poetry
2. Create tox files so that the ROCK is integrated with the CI
3. Ensure the ROCK gets built/published with the existing CI

Definition of Done

1. ROCK is being published to our docker.io/charmedkubeflow registry

Update `kserve-agent` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Update `paddleserver` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

Create a ROCK for kserve/paddleserver v0.11

Context

Create ROCK based on upstream Dockerfile. You can take the inspiration from pmml server ROCK in main branch. Don't forget to use ROCK's entrypoint service to correctly pass the arguments to workload container.

What needs to get done

1. Create ROCK
2. Manually push the ROCK to Dockerhub

Definition of Done

1. ROCK code is ready
2. ROCK is available at Dockerhub

fix storage-initializer rock's default args

Bug Description

canonical/rockcraft#476 clarifies how entrypoint-service's args should be specified to the command. This:

kserve-rocks/storage-initializer/rockcraft.yaml

Line 15 in af9dc6a

command: "/storage-initializer/scripts/initializer-entrypoint [ args ]"

should be updated to be:

 command: "/storage-initializer/scripts/initializer-entrypoint [ args ]" 

(where [ args ] is changed to [ ] (with a space between the brackets). The content between the brackets is interpreted as a default set of args that will be passed if none others are sent, eg these two are equivalent atm:

docker run this-image

docker run this-image args

To Reproduce

Environment

Relevant Log Output

-

Additional Context

Update `pmmlserver` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

Create a rock for `kserve/lgbserver:v0.11.2`

Context

Create a rock for KServe's lgbserver based on its upstream Dockerfile.
Please make sure the latest version of kserve v0.11.2 is used.

What needs to get done

1. Create the rockcraft project that will be equivalent to the dockerfile
2. Add sanity tests to the rock directory
3. Add tox.ini file for enabling testing

Definition of Done

The rockcraft project is merged in main and the resulting container image is published in the charmedkubeflow container image registry.

Update `lgbserver` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

kserve-rocks: Update `push` and `pull_request` CI

Context

canonical/bundle-kubeflow#797

What needs to get done

Update on_push.yaml and pull_request.yaml according to example.

Definition of Done

Workflows are updated

Create a rock for `kserve/storage-initializer:v0.11.2`

Context

Create a rock for KServe's storage-initializer based on its upstream Dockerfile.
Please make sure the latest version of kserve v0.11.2 is used.

What needs to get done

1. Create the rockcraft project that will be equivalent to the dockerfile
2. Add sanity tests to the rock directory
3. Add tox.ini file for enabling testing

Definition of Done

The rockcraft project is merged in main and the resulting container image is published in the charmedkubeflow container image registry.

pmmlserver v0.10.0 cant add arguments dynamically to command in service

Our newest track/v0.10.0 pmmlserver rock uses miniconda to install python dependencies and to run the workload because of that we had to run the rock service section like this

services:
  pmml-server:
    override: replace
    summary: "Pmml server service"
    startup: enabled
    command: bash -c 'export PATH=/opt/conda/bin/:${PATH} && eval $(/opt/conda/bin/conda shell.bash hook 2> /dev/null) && source /opt/conda/etc/profile.d/conda.sh && conda activate && python -m pmmlserver'
    environment:
      PYTHONPATH: /pmmlserver:/kserve

This command section makes it impossible to pass arguments to the command section. We need this because when kserve workloads are scheduled the container section looks like this:

containers:
  - args:
    - --model_name=pmml-demo
    - --model_dir=/mnt/models
    - --http_port=8080
    image: misohu/pmmlserver:v0.10.0_22.04_1
    imagePullPolicy: IfNotPresent

In the command section we need to run bash -c because we want to activate and setup the conda environment in the container. The the way how can be the env variables values resolved.

Create ROCK for torchserve-kfs

Kubeflow is using pytorch/torchserve-kfs image, as the KServe runtime image for serving PyTorch models
https://github.com/kserve/kserve/blob/f7de5e696e8d0e64e3ed2b2493ec64244291a5c9/install/v0.11.0/kserve-runtimes.yaml#L218

The source of this image lives in https://github.com/pytorch/serve/tree/master/kubernetes/kserve and it uses the pytorch/torchserve image as base. So in order to construct the ROCK we'll need to build the base torchserve image as well.

IIUC the base pytorch/torchserve image is defined and built in https://github.com/pytorch/serve/blob/v0.8.2/docker/Dockerfile

To summarize, our ROCK will need to

• live in a new repo, that aims to provide ROCKs for pytorch/serve repo
• Have a first layer that builds the pytorch/torchserve image
• Convert the dockerfile of https://github.com/pytorch/serve/tree/master/kubernetes/kserve on top

cc @i-chvets @SamirPS

Create ROCK for storage-initialiser for CKF 1.7

What needs to get done

1. Create the rockcraft.yaml file
2. Copy our CI automation for building the image on every PR and when a PR is merged

Why it needs to get done

In order for us to have our own built OCI Image for that component

Add integration tests to kserve server rocks

Context

The servers in this repo have simple integration tests described in their respective readme.md files - we should automate these and use them for CI/manual testing.

I tried looking into this a bit and what I got hung up on was how we needed to docker run the image, then curl at it. I wasn't sure how to manage this with tox. I looked into using tox-docker which does almost this, but it doesn't support passing args to the container. There were possible workarounds, but I didn't dig into it much

What needs to get done

1. automate the integration tests from readme.md for all server images

Definition of Done

1. server rocks have automated integration tests

Update `storage-initializer` ROCK

Context

Each charm in Kubeflow has a ROCK (or a set of ROCKs) it is using. The process of upgrading ROCKs usually means going to the component's upstream repository Dockerfile, comparing current ROCK version against the one in the repository and adding the missing bits.

What needs to get done

1. Clone the upstream's kubeflow repository.
2. Go to desired Dockerfile (first line in rockcraft.yaml should specify the full path).
3. Checkout to the branch or tag of the target version.
4. Checkout to the branch or tag of the version of the current ROCK.
5. Compare both files to spot the differences.

Definition of Done

1. New ROCK is ready.
2. Tests are passing.
3. ROCK is published under charmedkubeflow Dockerhub account.

Create a rock for `kserve/sklearnserver:v0.11.2`

Context

Create a rock for KServe's sklearnserver based on its upstream Dockerfile.

What needs to get done

1. Create the rockcraft project that will be equivalent to the dockerfile

Definition of Done

The rockcraft project is merged in main and the resulting container image is published in the charmedkubeflow container image registry.