dex secret not found about homelab

[khuedoan]

Hi, dex-secrets is created by https://github.com/khuedoan/homelab/blob/master/platform/dex/templates/secret.yaml, could you please post the output of:

kubectl describe -n dex externalsecret dex-secrets

[donydonald1]

+ kubectl describe -n dex externalsecret dex-secrets
Name:         dex-secrets
Namespace:    dex
Labels:       argocd.argoproj.io/instance=dex
Annotations:  <none>
API Version:  external-secrets.io/v1beta1
Kind:         ExternalSecret
Metadata:
  Creation Timestamp:  2024-04-18T09:48:36Z
  Generation:          1
  Resource Version:    51172
  UID:                 94eb9cd1-310b-4a3d-8574-7ed4b326de5c
Spec:
  Data:
    Remote Ref:
      Conversion Strategy:  Default
      Decoding Strategy:    None
      Key:                  kanidm.dex
      Metadata Policy:      None
      Property:             client_id
    Secret Key:             KANIDM_CLIENT_ID
    Remote Ref:
      Conversion Strategy:  Default
      Decoding Strategy:    None
      Key:                  kanidm.dex
      Metadata Policy:      None
      Property:             client_secret
    Secret Key:             KANIDM_CLIENT_SECRET
    Remote Ref:
      Conversion Strategy:  Default
      Decoding Strategy:    None
      Key:                  dex.grafana
      Metadata Policy:      None
      Property:             client_secret
    Secret Key:             GRAFANA_SSO_CLIENT_SECRET
    Remote Ref:
      Conversion Strategy:  Default
      Decoding Strategy:    None
      Key:                  dex.gitea
      Metadata Policy:      None
      Property:             client_secret
    Secret Key:             GITEA_CLIENT_SECRET
  Refresh Interval:         1h
  Secret Store Ref:
    Kind:  ClusterSecretStore
    Name:  global-secrets
  Target:
    Creation Policy:  Owner
    Deletion Policy:  Retain
    Name:             dex-secrets
Status:
  Conditions:
    Last Transition Time:  2024-04-18T09:48:36Z
    Message:               could not get secret data from provider
    Reason:                SecretSyncedError
    Status:                False
    Type:                  Ready
Events:
  Type     Reason        Age                    From              Message
  ----     ------        ----                   ----              -------
  Warning  UpdateFailed  4m40s (x24 over 109m)  external-secrets  error retrieving secret at .data[0], key: kanidm.dex, err: secrets "kanidm.dex" not found 

this is also affecting other deployment as well and for some reasons none of the secrets generated works when trying to login to the deployments

woodpecker          pre-install-agent-secret-check-jsqrs                     0/1     Completed                    0                75m
woodpecker          woodpecker-agent-5b6945cc7b-8c49l                        0/1     CrashLoopBackOff             19 (2m41s ago)   75m
woodpecker          woodpecker-agent-5b6945cc7b-nrmmf                        0/1     CrashLoopBackOff             19 (2m52s ago)   75m

[kikokikok]

Same problem for me, I think the
kanidm.dex key is never creqted in the global-secrets ClusterSecretStore

[khuedoan]

kandim.dex should be created by default in the post install script, could you try running make post-install manually?

[kikokikok]

Well the postscript fails when calling the reset of users with the python k8s client. It doesn't return the expected json payload on the stdout as expected which causes an error on json deserialization.
When executing with a remote ssh into the container, I see the json paylod

bash-5.2# make postinstall
make: *** No rule to make target 'postinstall'.  Stop.
bash-5.2# make post-install
Traceback (most recent call last):
  File "/home/cklat/homelab/./scripts/hacks", line 256, in <module>
    main()
  File "/home/cklat/homelab/./scripts/hacks", line 247, in main
    kanidm_login(["admin", "idm_admin"])
  File "/home/cklat/homelab/./scripts/hacks", line 158, in kanidm_login
    password = reset_kanidm_account_password(account)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cklat/homelab/./scripts/hacks", line 152, in reset_kanidm_account_password
    return json.loads(resp)['password']
           ^^^^^^^^^^^^^^^^
  File "/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6/lib/python3.11/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 2 (char 1)

Manual bash inside the container:

kanidmd recover-account --output json admin
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: This is running as uid == 0 (root) which may be a security risk.
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: permissions on /data/server.toml may not be secure. Should be readonly to running uid. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: /data/server.toml has 'everyone' permission bits in the mode. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: /data/server.toml owned by the current uid, which may allow file permission changes. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: DB folder /data has 'everyone' permission bits in the mode. This could be a security risk ...
00000000-0000-0000-0000-000000000000 INFO     ｉ [info]: Running account recovery ...
{"password":"VU29tSLcAqjccXWez12dQKhKNuPNWcJDcQ34NXK1gGGFSGwN"}