[thanos] allow setting objConfig as an object or string about helm-charts

[stevehipwell]

@onedr0p is there a reason you don't want to create your own secret for your use case? I can support using an object with toYaml in addition to the string but I wont support tpl on an object as this is bad practice, something like below.

{{- if .Values.objstoreConfig.create -}}
apiVersion: v1
kind: Secret
metadata:
  name: {{ include "thanos.objstoreConfigSecretName" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "thanos.labels" . | nindent 4 }}
data:
  {{ .Values.objstoreConfig.key }}: |-
  {{- with .Values.objstoreConfig.value }}
  {{- if kindIs "string" . }}
    {{- tpl . $ | b64enc | nindent 4 }}
  {{- else }}
    {{- toYaml . | b64enc | nindent 4 }}
  {{- end }}
  {{- end }}
{{- end -}}

[onedr0p]

Hi @stevehipwell you can see my deployment with the Bitnami chart here using Flux HelmRelease

https://github.com/onedr0p/home-ops/blob/main/kubernetes/apps/monitoring/thanos/app/helmrelease.yaml

Essentially I use Ceph's Bucket CR to create a bucket like this:

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: thanos-bucket-v1
  namespace: monitoring
spec:
  bucketName: thanos-v1
  storageClassName: ceph-bucket

This automatically generates a secret and configmap with the credentials and endpoint for the bucket. It would be nice to auto-populate the helm values using the Flux HelmRelease and valuesFrom:

  valuesFrom:
    - targetPath: objstoreConfig.config.bucket
      kind: ConfigMap
      name: thanos-bucket-v1
      valuesKey: BUCKET_NAME
    - targetPath: objstoreConfig.config.endpoint
      kind: ConfigMap
      name: thanos-bucket-v1
      valuesKey: BUCKET_HOST
    - targetPath: objstoreConfig.config.region
      kind: ConfigMap
      name: thanos-bucket-v1
      valuesKey: BUCKET_REGION
    - targetPath: objstoreConfig.config.access_key
      kind: Secret
      name: thanos-bucket-v1
      valuesKey: AWS_ACCESS_KEY_ID
    - targetPath: objstoreConfig.config.secret_key
      kind: Secret
      name: thanos-bucket-v1

I believe what you have there will work unless what I mentioned changes things.

[stevehipwell]

Is valuesFrom a HelmRelease construct then?

[onedr0p]

Correct https://fluxcd.io/flux/components/helm/helmreleases/#values-overrides

[stevehipwell]

In which case I'll add support in the next chart release (likely tomorrow).

[onedr0p]

Much appreciated, can't wait to try it out!