Describe the bug
Issues with kubeflow-pipelines-traefik on current K3s
To Reproduce
Steps to reproduce the behavior:
on running K3s node ( v1.20.2+k3s1 ) ... where simple jupyter-minimal-traefik works
k3ai apply -g kubeflow-pipelines-traefik
output ... (see 3 Warnings w/ v1beta1 context, that conflicts with 1.20 version)
Plugin YAML group-name: kubeflow-pipelines-traefik
Plugin YAML content: [{github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=1.3.0 kustomize} {github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=1.3.0 kustomize}], name: kubeflow-pipelines
serviceaccount "application" deleted
serviceaccount "argo" deleted
serviceaccount "kubeflow-pipelines-cache" deleted
serviceaccount "kubeflow-pipelines-container-builder" deleted
serviceaccount "kubeflow-pipelines-metadata-writer" deleted
serviceaccount "kubeflow-pipelines-viewer" deleted
serviceaccount "ml-pipeline-persistenceagent" deleted
serviceaccount "ml-pipeline-scheduledworkflow" deleted
serviceaccount "ml-pipeline-ui" deleted
serviceaccount "ml-pipeline-viewer-crd-service-account" deleted
serviceaccount "ml-pipeline-visualizationserver" deleted
serviceaccount "ml-pipeline" deleted
serviceaccount "pipeline-runner" deleted
role.rbac.authorization.k8s.io "application-manager-role" deleted
role.rbac.authorization.k8s.io "argo-role" deleted
role.rbac.authorization.k8s.io "kubeflow-pipelines-cache-deployer-role" deleted
role.rbac.authorization.k8s.io "kubeflow-pipelines-cache-role" deleted
role.rbac.authorization.k8s.io "kubeflow-pipelines-metadata-writer-role" deleted
role.rbac.authorization.k8s.io "ml-pipeline-persistenceagent-role" deleted
role.rbac.authorization.k8s.io "ml-pipeline-scheduledworkflow-role" deleted
role.rbac.authorization.k8s.io "ml-pipeline-ui" deleted
role.rbac.authorization.k8s.io "ml-pipeline-viewer-controller-role" deleted
role.rbac.authorization.k8s.io "ml-pipeline" deleted
role.rbac.authorization.k8s.io "pipeline-runner" deleted
rolebinding.rbac.authorization.k8s.io "application-manager-rolebinding" deleted
rolebinding.rbac.authorization.k8s.io "argo-binding" deleted
rolebinding.rbac.authorization.k8s.io "kubeflow-pipelines-cache-binding" deleted
rolebinding.rbac.authorization.k8s.io "kubeflow-pipelines-cache-deployer-rolebinding" deleted
rolebinding.rbac.authorization.k8s.io "kubeflow-pipelines-metadata-writer-binding" deleted
rolebinding.rbac.authorization.k8s.io "ml-pipeline-persistenceagent-binding" deleted
rolebinding.rbac.authorization.k8s.io "ml-pipeline-scheduledworkflow-binding" deleted
rolebinding.rbac.authorization.k8s.io "ml-pipeline-ui" deleted
rolebinding.rbac.authorization.k8s.io "ml-pipeline-viewer-crd-binding" deleted
rolebinding.rbac.authorization.k8s.io "ml-pipeline" deleted
rolebinding.rbac.authorization.k8s.io "pipeline-runner-binding" deleted
configmap "metadata-grpc-configmap" deleted
configmap "ml-pipeline-ui-configmap" deleted
configmap "pipeline-install-config-m2k6bmc5m7" deleted
configmap "workflow-controller-configmap" deleted
secret "mlpipeline-minio-artifact" deleted
secret "mysql-secret-fd5gktm75t" deleted
service "cache-server" deleted
service "controller-manager-service" deleted
service "metadata-envoy-service" deleted
service "metadata-grpc-service" deleted
service "minio-service" deleted
service "ml-pipeline-ui" deleted
service "ml-pipeline-visualizationserver" deleted
service "ml-pipeline" deleted
service "mysql" deleted
deployment.apps "cache-deployer-deployment" deleted
deployment.apps "cache-server" deleted
deployment.apps "controller-manager" deleted
deployment.apps "metadata-envoy-deployment" deleted
deployment.apps "metadata-grpc-deployment" deleted
deployment.apps "metadata-writer" deleted
deployment.apps "minio" deleted
deployment.apps "ml-pipeline-persistenceagent" deleted
deployment.apps "ml-pipeline-scheduledworkflow" deleted
deployment.apps "ml-pipeline-ui" deleted
deployment.apps "ml-pipeline-viewer-crd" deleted
deployment.apps "ml-pipeline-visualizationserver" deleted
deployment.apps "ml-pipeline" deleted
deployment.apps "mysql" deleted
deployment.apps "workflow-controller" deleted
application.app.k8s.io "pipeline" deleted
Error from server (NotFound): error when deleting "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=1.3.0": persistentvolumeclaims "minio-pvc" not found
Error from server (NotFound): error when deleting "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=1.3.0": persistentvolumeclaims "mysql-pv-claim" not found
2021/02/22 21:24:08 Error during delete: exit status 1
warning: deleting cluster-scoped resources, not scoped to the provided namespace
namespace "kubeflow" deleted
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io "applications.app.k8s.io" deleted
customresourcedefinition.apiextensions.k8s.io "clusterworkflowtemplates.argoproj.io" deleted
customresourcedefinition.apiextensions.k8s.io "cronworkflows.argoproj.io" deleted
customresourcedefinition.apiextensions.k8s.io "scheduledworkflows.kubeflow.org" deleted
customresourcedefinition.apiextensions.k8s.io "viewers.kubeflow.org" deleted
customresourcedefinition.apiextensions.k8s.io "workflows.argoproj.io" deleted
customresourcedefinition.apiextensions.k8s.io "workflowtemplates.argoproj.io" deleted
serviceaccount "kubeflow-pipelines-cache-deployer-sa" deleted
clusterrole.rbac.authorization.k8s.io "kubeflow-pipelines-cache-deployer-clusterrole" deleted
clusterrolebinding.rbac.authorization.k8s.io "kubeflow-pipelines-cache-deployer-clusterrolebinding" deleted
Warning: networking.k8s.io/v1beta1 IngressClass is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 IngressClassList
ingressclass.networking.k8s.io "traefik-lb" deleted
Warning: networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Error from server (NotFound): error when deleting "https://raw.githubusercontent.com/kf5i/k3ai-plugins/main/common/traefik/kubeflow-pipeline-bind.yaml": ingresses.networking.k8s.io "pipeline-ingress" not found
2021/02/22 21:25:04 Error during delete: exit status 1
and then these containers (and sometimes more) never successfully start (and just perpetually CrashLoopBackOff)
metadata-grpc-deployment
ml-pipeline
ml-pipeline-persistenceagen
metadata-writer
Expected behavior
successful deployment
Screenshots
n/a
Additional context
Also, WRT to "k3ai --help" output, figured I could clone the "k3ai-plugins" repo and tweak the noted manifests that might contribute to warnings, but
- k3ai list --repo http://localhost/k3ai-plugins/contents/core (seems like a poor example syntax, since "contents" is not in place)
- k3ai list --repo http://localhost/k3ai-plugins/core (even with tweaked URI that seems to match repo artifacts)
Can't load cache:error fetching plugins content: cannot load plugins: invalid character '<' looking for beginning of value will use remote
cannot load plugins: invalid character '<' looking for beginning of value
- k3ai list --repo https://github.com/kf5i/k3ai-plugins/core (same is true for trying to use this repo directly from CLI)
Can't load cache:error fetching plugins content: cannot load plugins: invalid character 'N' looking for beginning of value will use remote
cannot load plugins: invalid character '<' looking for beginning of value