cryostatio / cryostat-helm Goto Github PK

Current Behavior

See also cryostatio/cryostat3#266

?f=b25nb2luZy5qZnI failed, error id: 77f79b02-233e-44f9-b36d-fa359db7d470-3: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])): Profile file contained no credentials for profile 'default': ProfileFile(profilesAndSectionsMap=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:117)
	at software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:45)
	at software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:128)
	at software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:54)
	at software.amazon.awssdk.awscore.internal.authcontext.AwsCredentialsAuthorizationStrategy.resolveCredentials(AwsCredentialsAuthorizationStrategy.java:100)
	at software.amazon.awssdk.awscore.internal.authcontext.AwsCredentialsAuthorizationStrategy.addCredentialsToExecutionAttributes(AwsCredentialsAuthorizationStrategy.java:77)
	at software.amazon.awssdk.services.s3.internal.signing.DefaultS3Presigner.invokeInterceptorsAndCreateExecutionContext(DefaultS3Presigner.java:366)
	at software.amazon.awssdk.services.s3.internal.signing.DefaultS3Presigner.presign(DefaultS3Presigner.java:308)
	at software.amazon.awssdk.services.s3.internal.signing.DefaultS3Presigner.presignGetObject(DefaultS3Presigner.java:230)
	at software.amazon.awssdk.services.s3.presigner.Producers_ProducerMethod_produceS3Presigner_439b267c3ef704c1a556181cdfd1b3e0d8559840_ClientProxy.presignGetObject(Unknown Source)
	at io.cryostat.recordings.Recordings.redirectPresignedDownload(Recordings.java:1009)
	at io.cryostat.recordings.Recordings$quarkusrestinvoker$redirectPresignedDownload_67828a00204dd3db027761f338e40a95b09257e0.invoke(Unknown Source)
	at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
	at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
	at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:145)
	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:576)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:840)

Expected Behavior

No response

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

It would be useful to set this by default to not require the user to specify it themselves.

I am using EKS with crossplane helm provider to deploy helm charts. Trying to deploy this chart I got:

create failed: failed to install release: chart requires kubeVersion: >=1.19.0 which is incompatible with Kubernetes v1.24.8-eks-ffeb93d

Removing line from Chart.yaml helped to run smoothly.

Current Behavior

Cryostat installed with the Helm Chart no longer sees any applications due to not setting CRYOSTAT_K8S_NAMESPACES.

Expected Behavior

Applications in Cryostat's namespace should be discovered.

Steps To Reproduce

1. Install Cryostat latest/2.3.0-SNAPSHOT using the Helm Chart
2. Check list of targets

Environment

No response

Anything else?

Describe the feature

We should have some CI jobs to lint/validate and run tests against helm chart releases.

Anything other information?

https://github.com/helm/chart-testing

We should include a values.schema.json file to validate chart values:
https://helm.sh/docs/topics/charts/#schema-files

This can be generated by readme-generator-for-helm.

Describe the feature

Pull requests from reviewers should automatically labelled safe-to-test.

Anything other information?

References: https://github.com/cryostatio/cryostat/blob/main/.mergify.yml

Once added, this can be referenced in the Chart.yaml.

For example, in the OpenShift Console, NOTES.txt is interpreted as Markdown. This can cause some variable names to be incorrect.

Similar to the operator, the Helm chart should offer configuration to deploy Cryostat without Grafana integration.

As done in the operator, we should change image versions to latest in our main branch.

Set images to released versions and release chart

Current Behavior

It seems like the Openshift OAuth proxy image does not specify a numeric non-root user. Thus, runAsNonRoot: true cannot be specified on pod.

$ podman image inspect quay.io/openshift/origin-oauth-proxy:latest --format '{{.Config.User}}'
0

Container failed to launch with status:

  - image: quay.io/openshift/origin-oauth-proxy:latest
    imageID: ""
    lastState: {}
    name: cryostat-authproxy
    ready: false
    restartCount: 0
    started: false
    state:
      waiting:
        message: 'container has runAsNonRoot and image will run as root (pod: "cryostat-579d45489-2nt7w_default(337845d4-41c2-4fbb-ab29-3882be6d771a)",
          container: cryostat-authproxy)'
        reason: CreateContainerConfigError

Expected Behavior

Openshift Oauth proxy container should launch successfully.

Steps To Reproduce

helm install cryostat --set authentication.openshift.enabled=true --set core.route.enabled=true ./charts/cryostat/

Environment

CRC version: 2.26.0+233df0
OpenShift version: 4.13.9

Anything else?

Should the chart default to set runAsUser for the proxy container?

The tests are failing because the cryostat-v2.4 branch is still using latest for various image tags. So this is actually deploying Cryostat 2.5.0-snapshot and not 2.4.0-snapshot.

Start with minimal functionality:

• TLS disabled
• emptyDir for storage
• No optional volume mounts

References:

• https://github.com/cryostatio/cryostat3/blob/997f1419e77a5776b7e92368069ca007b86cb287/smoketest/compose/auth_proxy.yml#L41
• https://github.com/cryostatio/cryostat3/blob/997f1419e77a5776b7e92368069ca007b86cb287/smoketest/compose/auth_proxy_htpasswd#L1
• https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/overview (search for htpasswd)

Hello!
Thanks for your project - it's awesome!

I'm trying to apply it in my K8S installation and it would be cool to have a few more options to configure:

1. Disable Grafana container entirely (in a case when we use something else or just don't need Grafana here)
2. Override the root web URL path (from "/" to "/cryostat", for instance)
3. Volumes for recordings

Thanks in advance!

The Release Drafter action creates draft releases, which will probably conflict with the releases that the Chart Releaser Action tries to make. I think it should actually be possible to substitute GitHub's built-in release notes generator for this action. We add a file to customize the format of the release notes, using labels as we do currently:
https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes#configuring-automatically-generated-release-notes

The Chart Releaser Action uses the GitHub Release API to generate release notes if they're not provided. Unfortunately this API doesn't provide a way to specify the previous tag to compare against. It does have a separate API to do this and return the release notes as JSON: https://docs.github.com/en/rest/releases/releases?apiVersion=2022-11-28#generate-release-notes-content-for-a-release

We can call this API inside the workflow which should already have the GitHub CLI installed:

# GitHub CLI api
# https://cli.github.com/manual/gh_api

gh api \
  --method POST \
  -H "Accept: application/vnd.github+json" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  /repos/OWNER/REPO/releases/generate-notes \
  -f tag_name='v1.0.0' \
 -f target_commitish='main' \
 -f previous_tag_name='v0.9.2' \
 -f configuration_file_path='.github/custom_release_config.yml' 

We can use jq to grab the body from the response and output it to a file. Then create a config.yaml with release-notes-file: /path/to/release-notes[1]. Then pass the config file to the Chart Releaser Action like this: https://github.com/helm/chart-releaser-action#example-using-custom-config

We might be able to use this action to quickly compute the previous semver tag. If not it should be easy to do ourselves.

[1] https://github.com/helm/chart-releaser/blob/fce24e7688b7dd9b6091ff2e84d1eaabce8de1ee/pkg/config/config.go#LL61C45-L61C63

Originally posted by @ebaron in #71 (comment)

We should have something here similar to our workflow for updating the web-client module. When a new commit is made to the gh-pages branch, CI should push an update to the submodule reference on https://github.com/cryostatio/cryostatio.github.io.

When you use the repository's GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN, with the exception of workflow_dispatch and repository_dispatch, will not create a new workflow run. This prevents you from accidentally creating recursive workflow runs. For example, if a workflow run pushes code using the repository's GITHUB_TOKEN, a new workflow will not run even when the repository contains a workflow configured to run when push events occur.

https://docs.github.com/en/actions/security-guides/automatic-token-authentication#using-the-github_token-in-a-workflow

Describe the feature

See cryostatio/cryostat3#355

Anything other information?

No response

The operator generates credentials for Grafana and places them in a secret, we should explore options for how to enable Grafana authentication here.

See: #4

Describe the feature

The chart should be updated to deploy the development builds of Cryostat 3.0.

• #113
• #114
• #115
• #116
• #117

Anything other information?

No response

Yeh exactly! https://docs.mergify.com/commands/restrictions/#command-restriction-format

Perhaps we want something along these lines instead:

commands_restrictions:
  backport:
    conditions:
      - sender=@team

Originally posted by @ebaron in #90 (comment)

Related: cryostatio/cryostat#1269

Simliar to cryostatio/cryostat#1268

          > https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container

Maybe it makes sense for us to apply a general Pod security context around everything, and then have optional container security contexts for each container within the Pod:

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container

Except for cases like this where we seem to know that a container will not run under the general Pod context, or at least on some common and supported k8s/OCP versions it won't, then we can provide a default for that particular container.

After saying all that, I see we do have separate container security contexts:

https://github.com/cryostatio/cryostat-helm?tab=readme-ov-file#jfr-data-source-container

But the new storage and db containers don't have their own. They are mistakenly reusing the core context.

Originally posted by @andrewazores in #133 (comment)

The credentials secret should use {{ .Release.Name }} instead of cryostat to avoid conflicting with other releases in the same namespace.

This should be set to ">=1.19.0".

Helm chart equivalent to cryostatio/cryostat-operator#399 (comment)

          Ah make sense to me! There is also another service case, related to service configurations.

With service type LoadBalancer:

helm install cryostat-with-svc ./charts/cryostat/ \
--set core.service.type=LoadBalancer \
--set grafana.service.type=LoadBalancer

Since its relying cloud providers, likely there are additional annotations required on the Service. For example, with AWS, I have been manually patching the Services after installing the chart:

service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-type: external

Do you think it would be good to allow setting those on chart values? Currently, we can only set Service's port and type.

Maybe, not this PR scope though.

Originally posted by @tthvo in #122 (comment)

After #114 (and #116), https://github.com/openshift/oauth-proxy/ can be deployed instead of oauth2-proxy. This should also respect the Basic auth configuration from #116 and its htpasswd file, if the user provides this configuration. The container should be dropped in in place of oauth2-proxy in all instances and should be configured to pass requests to the same containers in the same manners. The only effective difference to the user should be that the login screen looks different and the credentials are OpenShift account credentials rather than htpasswd Basic credentials.

Describe the feature

https://github.com/helm/chart-releaser-action/releases/tag/v1.6.0

Anything other information?

No response

It would be useful to support some form of authentication when accessing Cryostat.

See also: cryostatio/cryostat-operator#206

After cryostatio/cryostat#1083, Cryostat needs a CRYOSTAT_JMX_CREDENTIALS_DB_PASSWORD environment variable set for its JMX credentials database. At minimum, this can be generated using randAlphaNum.

Look into way we can publish releases to coincide with Cryostat project releases, and also snapshot releases of the latest development branch.

Relevant GH action:
https://helm.sh/docs/howto/chart_releaser_action/

See: cryostatio/cryostat-operator#347

See https://github.com/cryostatio/cryostat-operator/wiki/Security-and-Authz#scenario , #114

References:

• https://github.com/cryostatio/cryostat-operator/wiki/Security-and-Authz#scenario
• https://github.com/cryostatio/cryostat3/blob/997f1419e77a5776b7e92368069ca007b86cb287/smoketest.bash#L161
• https://github.com/cryostatio/cryostat3/blob/997f1419e77a5776b7e92368069ca007b86cb287/smoketest/compose/auth_proxy.yml#L32
• https://github.com/cryostatio/cryostat3/blob/997f1419e77a5776b7e92368069ca007b86cb287/smoketest/compose/auth_proxy_alpha_config.yaml#L1

This either depends on #116 , or else --skip-auth-route can be used initially so that the proxy can be deployed on its own acting as a passthrough gateway and requiring no authentication on any requests.

Like with the operator, it would be useful to allow persistence of recordings/templates/credentials with a Persistent Volume Claim.

We use readme-generator-for-helm to generate this section of the README by marking up our comments in values.yaml

That being said, I think helm chart (3.0) also has jmx auth enabled for cryostat. For it 2.x, CRYOSTAT_DISABLE_JMX_AUTH is set to true. I think the password are generated by default in the entrypoint. It's not reported in a k8s secret like on the operator side. Should it be generated by helm template or just disable auth like in 2.x?

Originally posted by @tthvo in cryostatio/cryostat-operator#815 (comment)

Scope: Cryostat 3.0

Question: Should JMX auth be disabled like 2.x? Otherwise, we should generate and same them in a k8s secret, similar to the operator.

Currently, the user has no way to find the credentials to authenticate over JMX for cryostat itself.

Describe the feature

Would be good to be able to edit CRYOSTAT_K8S_NAMESPACES to watch multiple namespaces.

Probabily also Role and RoleBinding needs fixes to support that

Anything other information?

No response

The current test is a boilerplate one, we should modify this test to check health endpoints where possible.