jenkins-x-plugins / jx-secret Goto Github PK

As you may have notice, Kubernetes External Secrets has been deprecated and replace by https://github.com/external-secrets/external-secrets
#external-secrets/kubernetes-external-secrets#864

Do you know if it's possible to migrate it ?

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

jx secret replicate, should be smart enough to detect other (staging/production) namespaces and copy secrets to them.

manual workaround:

jx secret copy --selector secret.jenkins-x.io/replica-source=true -n jx -t toNamespace # or any other secret name

Version 0.0.144 help documentation shows jx-secret edit in examples:

ayla.khan@AKHAN-MBP:~/dev$ jx secret version
version: 0.0.144
ayla.khan@AKHAN-MBP:~/dev$ jx secret export --help
Exports the current populated values to a YAML file

Usage:
  jx-secret export [flags]

Examples:
  jx-secret edit
...

When inputting values with jx secret edit you can't input multiline values. I'm trying to edit jenkins-maven-settings.

If I have a secret within vault along the path:
secret/data/staging/secretFile
with the key File.txt

It is incorrectly parsed into yaml as:
secret/data/staging/secretFile/File/txt
The correct way is:
secret/data/staging/secretFile/File.txt

So the following import of the secrets does not work.

Running version: 3.1.158

As per docs at:
https://jenkins-x.io/v3/develop/reference/jx/secret/import/

But I get the error:
Error: unknown command "import" for "jx-secret"

jx secret populate fails for ASM with the message:

Error: failed to populate secrets: failed to save properties key: jx-admin-user properties: password, username on ExternalSecret jenkins-x-chartmuseum: error creating new secret for aws secret manager: : MissingRegion: could not find region configuration

The workaround is to set the region in the default section of the secret mapping file: https://github.com/jx3-gitops-repositories/jx3-eks-asm/blob/69a957feca79da5992fb81792b1849758d81f351/.jx/secret/mapping/secret-mappings.yaml#L4

Instead we want jx to populate the region from the requirements file.

Related to jenkins-x/jx#7941

right now we lazily populate a number of secrets such as the lighthouse oauth token + the tekton-git token from the user/token used to install the operator via jx admin operator.

However we don't replace the token if we re-run the jx admin operator.

We need some way to force those secret values to be wiped in the secret store; so that they get defaulted to the new jx admin operator values.

so in the default pipeline we can disable this step other than for a few kinds of secret storage (e.g. local and vault only).

Then for GSM / ASM / Azure we don't populate secrets inside the boot pipeline by default - making the IAM bindings simpler.

The user then runs jx secret populate up front before installing the operator

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

vbom.ml/[email protected]: unrecognized import path "vbom.ml/util" (https fetch: Get https://vbom.ml/util?go-get=1: EOF)

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Sometimes we could choose to store some secret into git because we use the vault-mutating-webhook.
Ex:

postgresqlPassword: vault:secret/data/postgres#POSTGRESQL_PASSWORD

It would be great to be able to annotate a secret with do not convert have it stored as is in the jx gitops repo.

When running jx secret vault shell the command exists directly, so you can't interact with the shell.

> jx secret vault shell
waiting for vault pod vault-0 in namespace jx-vault to be ready...
pod vault-0 in namespace jx-vault is ready
verifying we have vault installed
about to run: /Users/msv/.jx3/plugins/bin/vault-1.6.1 version
Vault v1.6.1 (6d2db3f033e02e70202bef9ec896360062b88b03)
verifying we can connect to vault...
about to run: /Users/msv/.jx3/plugins/bin/vault-1.6.1 kv list secret
Keys
----
accounts/
bucketrepo/
dockerrepo
jx/
lighthouse/
mysql
tekton/
vault is setup correctly!

managed to verify we can connect to vault
verifying we have vault installed
about to run: /Users/msv/.jx3/plugins/bin/vault-1.6.1 version
Vault v1.6.1 (6d2db3f033e02e70202bef9ec896360062b88b03)
using vault binary /Users/msv/.jx3/plugins/bin/vault-1.6.1
about to run: bash
> vault kv list secret
Get https://127.0.0.1:8200/v1/sys/internal/ui/mounts/secret: x509: certificate signed by unknown authority

The command that is run do start in an environment where vault works:

> jx secret vault shell -s vault --args kv --args list --args secret
waiting for vault pod vault-0 in namespace jx-vault to be ready...
pod vault-0 in namespace jx-vault is ready
verifying we have vault installed
about to run: /Users/msv/.jx3/plugins/bin/vault-1.6.1 version
Vault v1.6.1 (6d2db3f033e02e70202bef9ec896360062b88b03)
verifying we can connect to vault...
about to run: /Users/msv/.jx3/plugins/bin/vault-1.6.1 kv list secret
Keys
----
accounts/
bucketrepo/
dockerrepo
jx/
lighthouse/
mysql
tekton/
vault is setup correctly!

managed to verify we can connect to vault
verifying we have vault installed
about to run: /Users/msv/.jx3/plugins/bin/vault-1.6.1 version
Vault v1.6.1 (6d2db3f033e02e70202bef9ec896360062b88b03)
using vault binary /Users/msv/.jx3/plugins/bin/vault-1.6.1
about to run: vault kv list secret
Keys
----
accounts/
bucketrepo/
dockerrepo
jx/
lighthouse/
mysql
tekton/

I suspect stdin isn't connected correctly.

For external vault (with kubernetes auth enabled), it will be nice to support vaultMountPoint in the mapping file. Source: https://github.com/external-secrets/kubernetes-external-secrets#hashicorp-vault

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

dmitri.shuralyov.com/gpu/[email protected]: unrecognized import path "dmitri.shuralyov.com/gpu/mtl": https fetch: Get "https://dmitri.shuralyov.com/gpu/mtl?go-get=1": EOF

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

code.gitea.io/sdk/[email protected]: reading code.gitea.io/sdk/gitea/gitea/go.mod at revision gitea/v0.12.1: unknown revision gitea/v0.12.1

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

$ jx secret version
version: 0.0.221
$ jx version
version: 3.1.170

Full error:

error: failed to save properties key: test-document-db-postgresql properties: postgresql-postgres-password, postgresql-password on ExternalSecret test-document-db-postgresql: error setting azure key vault secret: error retrieving secret from key vault: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://XX-XXX.vault.azure.net/secrets/test-document-db-postgresql?api-version=7.1: StatusCode=0 -- Original Error: the MSI endpoint is not available. Failed HTTP request to MSI endpoint: Get http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01: context deadline exceeded

some charts create empty Secrets which are then populated later on with data by the application itself.

e.g. the tekton chart includes one such Secret
https://github.com/cdfoundation/tekton-helm-chart/blob/master/charts/tekton-pipeline/templates/webhook-certs-secret.yaml

so lets default to not converting Secrets with no data to ExternalSecrets by default.

Over time we can maybe add customization metadata to define if we include/exclude Secrets from the conversion

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

vbom.ml/[email protected]: unrecognized import path "vbom.ml/util" (parse https://vbom.ml/util?go-get=1: no go-import meta tags ())

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

so that folks can edit only secrets in a specific namespace ; or with a specific name or name filter or with a specific key.

e.g.

jx-secret edit --namespace foo --name lighthouse-hmac

or something like that

We may want to reuse the same filter arguments to jx-secret verify - so folks can just verify, say, lighthouse secrets or whatever

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

vbom.ml/[email protected]: unrecognized import path "vbom.ml/util" (https fetch: Get https://vbom.ml/util?go-get=1: EOF)

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

some helm charts create Secrets with data inside them (e.g. default configuration data or dynamically generated values).

it would be nice if jx secret populate could take the data generated from the helm chart's Secret resources and if the secret store has no data, pre-populate it so that kubernetes external secrets would be able to get the data from the secret store to populate the Secret

as its not updating all the properties at the same time, so kubernetes rejects the first property

right now we update the jx-basic-auth-htpasswd on each boot job. We should only update it if it actually changes

It seems like jx secret convert or populate in the jx3-kubernetes repo overrides all the secrets breaking existing running applications. This should be fixed asap IMO.
Workaround: For now I create external-secrets manually ask my charts to use existing secret

it would be nice if we supported a defaults template entry using to templating.

e.g. something like:

# .jx/secret/mapping/secret-mappings.yaml
apiVersion: secret.jenkins-x.io/v1alpha1
kind: SecretMapping
spec:
  defaults:
    backendType: vault
    mappingTemplate: 
      key:  "secret/data/{{ .Namespace }}/{{ .Chart }}/{{ .Secret }}/{{ . Entry }}"

so that folks can configure via go template expressions the default values like key/property based on template expressions

it might be nice to create a new command like jx-secret vault shell which:

• waits for vault to be setup via jx-secret vault wait
• port forwards the local laptop to vault via jx-secret vault portforward
• download the vault binary if its not already downloaded: https://www.vaultproject.io/downloads/
• sets up the various env vars so that the vault CLI binary can talk to vault
• opens a shell with the vault binary on the $PATH so that the user can type vault kv list secret and it work

When creating external secrets it would be nice if newly created secrets in vault service (GKE,Azure, Aws,...) would have namespace prefix in their name.

E.g. If you have 2 environments in one cluster (staging and beta), then you define secret, it should be mapped to different vault secrets so that you can rotate/change them independently and they are different across environmets .

Currently you have to do this by creating externalSecret resource manually and create links to two different vault secrets.

Inspiration is in this thread:
https://kubernetes.slack.com/archives/C9MBGQJRH/p1625211231317400

so that we can generate CAs and signed certs like helm

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

gomodules.xyz/jsonpatch/[email protected]: unrecognized import path "gomodules.xyz/jsonpatch/v2" (parse https://gomodules.xyz/jsonpatch/v2?go-get=1: no go-import meta tags ())

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.