[Bug] ElasticSearch index not created when using existing SQL/Cassandra about helm-charts

[max-openline]

same issue is happening to me, have pods stuck on init state. I tried the workaround from @DanielCalvo but it not work on my case.

[DanielCalvo]

I stumbled upon this today while trying to configure temporal with an external PostgreSQL DB. I also found a workaround, but it isn't pretty.

We're deploying this through an ArgoCD application, and this is what it looks like:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
    name: temporal
    namespace: argocd
spec:
    syncPolicy:
        automated:
            selfHeal: true
            prune: true
    project: default
    destination:
        server: https://kubernetes.default.svc
        namespace: temporal
    source:
        path: charts/temporal
        repoURL: https://github.com/temporalio/helm-charts
        targetRevision: temporal-0.33.0
        helm:
            releaseName: temporal
            values: |-
                replicaCount: 1
                postgresql:
                  enabled: true
                prometheus:
                  enabled: true
                elasticsearch:
                  enabled: true
                grafana:
                  enabled: true
                cassandra:
                  enabled: false
                schema:
                  setup:
                    enabled: true
                  update:
                    enabled: true
                server:
                  config:
                    persistence:
                      default:
                        driver: sql
                        sql:
                          driver: postgres
                          host: temporal(...).eu-west-1.rds.amazonaws.com
                          port: 5432 #if you don't specify this, temporal defaults to port 3306 for postgresql, which is the default port for mysql!
                          user: postgresql
                          password: xxxx
                      visibility:
                        driver: sql
                        sql:
                          driver: postgres
                          host: temporal(...).eu-west-1.rds.amazonaws.com
                          port: 5432
                          user: postgresql
                          password: xxxx

All 4 temporal pods were stuck initializing. I only checked the worker pod which was was failing with:

waiting for elasticsearch index to become ready

This is due the es-index-setup job not being created.

I ended up cloning the repo, checking out the tag I was using above, putting the helm values on a file by themselves and templating the chart locally:

helm template temporal /home/daniel/repos/temporal-helm-charts/charts/temporal/ --values temporal-values.yml

And strangely enough this generated the yaml for the es-index-setup Job, which I then kubectl applied from my machine, which initialized temporal's elastic search instance, and now the pods are OK.

I ran out of time to troubleshoot why the helm chart has this strange behaviour, if it wasn't for this issue I would assume the problem was between my chair and keyboard, but now I'm not so sure.

Also it is worth bearing in mind that many temporal users will use a GitOps tool (likely ArgoCD or FluxCD) to deploy this helm chart, so it is also something worth validating.

Cheers

[emanuel8x8]

Do you happen to have any updates on this? Our team is also affected by this issue.