jenkins-x-plugins / jx-pipeline Goto Github PK

We have installed JX 3 with GitLab on premise and we want thank all of you for your great work!

We observed one small issue with the jx pipeline start command which is not functioning as expected.

We are using the following command jx pipeline start --git-username .... --git-token ..
After that we are selecting a repo from our GitLab (from the interactive CLI).
In the next step we are asked for GitHub credentials and the command fails with:

ERROR: error creating a mirror clone: exit status 128 Cloning into bare repository '/var/tmp/gitcache490222991/our-organization/java-project-example'...
remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.
remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information.
fatal: Authentication failed for 'https://github.com/our-organization/java-project-example/'
error: failed to calculate in repo configuration: failed to load trigger config for repository our-organization/java-project-example for ref : failed to create repo client: error creating a mirror clone: exit status 128 Cloning into bare repository '/var/tmp/gitcache490222991/our-organization/java-project-example'...
remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.
remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information.
fatal: Authentication failed for 'https://github.com/our-organization/java-project-example/'

cc @ankitm123

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

code.gitea.io/sdk/[email protected]: reading code.gitea.io/sdk/gitea/gitea/go.mod at revision gitea/v0.13.0: unknown revision gitea/v0.13.0

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

The / in repo name is converted to -, which times out the import command when importing a nested gitlab repo.

error: failed to create Pull Request on the cluster git repository: failed to wait for the Pull Request https://gitlab.com/jx-gitlab-test/cluster/jx3-k3s-gitlab-vault/-/merge_requests/7 to merge: Timed out waiting for pull request https://gitlab.com/jx-gitlab-test/cluster/jx3-k3s-gitlab-vault/-/merge_requests/7 to merge. Waited 20m0s

Related to: jenkins-x/jx#7981

https://github.com/jenkins-x/jx-pipeline/releases/tag/v0.0.88

see helmboot

curl -L https://github.com/jenkins-x/jx-pipeline/releases/download/v0.0.88/helmboot-linux-amd64.tar.gz | tar xzv 
sudo mv helmboot /usr/local/bin

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

gonum.org/v1/[email protected]: unrecognized import path "gonum.org/v1/gonum": parse https://gonum.org/v1/gonum?go-get=1: no go-import meta tags ()

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

You can't start a pipeline for a pull request with jx pipeline start. If you try by for example give the branch with --branch the following necessary environment variables will not be set in in the build pod:

• PULL_NUMBER
• PULL_PULL_REF
• PULL_PULL_SHA

So you could for example get this error:

+ jx gitops pr variables
error: failed to : missing option: --pr

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

vbom.ml/[email protected]: unrecognized import path "vbom.ml/util" (parse https://vbom.ml/util?go-get=1: no go-import meta tags ())

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Hi, thanks for this plugin.

I wanted to notice that the artifacts for latest version did not build. Latest build is not installable automatically via jx.

https://github.com/jenkins-x-plugins/jx-pipeline/releases/tag/v0.1.0

related to jenkins-x-plugins/jx-build-controller#30
see this slack thread for the whole context: https://kubernetes.slack.com/archives/C9MBGQJRH/p1621233109180200

seems that even with #381 users are still seeing duplicated logs.

my guess is that we have a race condition between jx finishing to read the logs, and reloading the pod (to check its status), and the kube controller updating the pod status.
instead of "just" reloading the pod, we should be smarter, for example watching for pod changes with a small timeout (1 or 2 seconds)

I created a pipeline where a step could use an environment variable. After triggering the pipeline I looking at the pod and the supplied environment variable was not supplied.

Looking at the source code I don't see that the option field CustomEnvs is used.

jx pipeline start asks for credentials even though they are supplied with GIT_USERNAME / GIT_TOKEN or --git-username / --git-token. It seems like it is git that asks for it during git clone.

If a git credential store with the crediental is set up this doesn't happen. So it's typically in a use case like mine it happens: Executing jx pipeline start in a cronjob.

This bug didn't exist in version 0.0.143. I have a hunch that the regression appeared with jenkins-x/lighthouse#1358.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

vbom.ml/[email protected]: unrecognized import path "vbom.ml/util" (https fetch: Get https://vbom.ml/util?go-get=1: EOF)

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

I wanted to use my container images from private ACR repo so therefore I provided docker pull secret under ImagePullSecret in my both pr/release pipeline.yaml, just like below -:

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
spec:
  podTemplate:
    ImagePullSecrets:
    - name: tekton-container-registry-auth

preSubmit/postSubmit pipeline triggered via github webhook event
When the Lighthousejob is created for preSubmit/postSubmit pipeline triggered via github webhook event, it does contains the actual value of ImagePullSecrets provided in the pipeline(pr/release pipeline) definition, just like in above definition, and works fine, No issues.

preSubmit/postSubmit pipeline triggered via jx pipeline start cmd
but when the Lighthousejob created for a preSubmit/postSubmit pipeline(pr/release one), it's not having the original ImagePullSecrets value, instead the value is getting replaced with null(can be seen below ), may be its getting patched somewhere in between LHjob creation.
hence not able to pull containers image form private pvt registry and release pipelines are breaking.

podTemplate:
    ImagePullSecrets: null

if we are triggering pipeline via jx cli than it always has postsubmit Type no matter which --kind you pass as argument to jx pipeline start cmd, so any pipeline(pre/post) triggered using CLI is only having the issue.

JX beta verision -: version: 3.1.120 (with latest gitops upgrade already applied to cluster)

Slack thread for the same https://kubernetes.slack.com/archives/C9MBGQJRH/p1610040745132700
https://kubernetes.slack.com/archives/C9MBGQJRH/p1610220507242100 for more insights on the issue.

from slack https://kubernetes.slack.com/archives/C9MBGQJRH/p1626256285337700

...

okay so i replicated the weird thing with ~/.git-credentials

• create a ~/.git-credentials with
  https://GIT_USER:[email protected]
• run jx pipeline effective
• select presubmit/pr
• cat ~/.git-credentials
  you will now see a line https://:@github.com added.
• run jx gitops variables I get output of

jx gitops variables
WARNING: ignoring missing user name in git credentials file: /Users/admin/.git-credentials URL: https://:@github.com
about to run: git clone https://bitbucket.org/OWNER/CLUSTER_REPO.git /var/folders/x8/5mg84jyx4r70xjcqdrk39lzc0000gn/T/jx-git-637851936 in dir /var/folders/x8/5mg84jyx4r70xjcqdrk39lzc0000gn/T
Cloning into '/var/folders/x8/5mg84jyx4r70xjcqdrk39lzc0000gn/T/jx-git-637851936'...
about to run: git rev-parse --abbrev-ref HEAD in dir .
test/helm
WARNING: no $BUILD_ID found so cannot create the BUILD_NUMBER

right now jx pipeline start finds the first postsubmit and starts it:
https://github.com/jenkins-x/jx-pipeline/blob/master/pkg/cmd/start/start.go#L324-L330

we should use the --kind and the --context values to find the correct presubmit or postsubmit to start

If we try to import a repo with underscore in it's name - say jx_pipeline, then even if the configmap config in jx namespace gets updated, jx project import times out after the default 20 mins waiting.

The reason is that if the repo name is derived from o.Appname
But if o.AppName has any underscore, it gets removed by the function call here

While we can remove the naming.ToValidName function, it may break things downstream, so it's best to do a check in the wait function, by converting the dash (-) to underscore (_) in the check for trigger.
This should not break any other functionality.

see this slack thread for the whole context: https://kubernetes.slack.com/archives/C9MBGQJRH/p1621233109180200

the issue is with a pipelinerun with 5 (sequential) taskruns: if you try to get the logs while task 3 is running, the getRunningBuildLogs func will only see the first 3 tasks/pods, and so will never return the logs for the 2 other tasks/pods, because they don't exists yet at the beginning of the func when the internal state is constructed.

the fix is to reload the pipelinerun when all pods have been seen, if there are more tasks than pods.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

vbom.ml/[email protected]: unrecognized import path "vbom.ml/util" (https fetch: Get https://vbom.ml/util?go-get=1: EOF)

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

gomodules.xyz/jsonpatch/[email protected]: unrecognized import path "gomodules.xyz/jsonpatch/v2" (parse https://gomodules.xyz/jsonpatch/v2?go-get=1: no go-import meta tags ())

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

i have a pr job that lints all changes to my internal custom pipeline catalog. using jx pipeline lint --recursive ignores the 'tasks' folder

expected

jx pipeline lint --recursive should lint all pipelines in the tasks folder per the "-r, --recursive" help text

Recurisvely find all '.lighthouse' folders such as if linting a Pipeline Catalog

actual results

jx pipeline lint --r

.lighthouse/jenkins-x/lint-helm.yaml                                         OK
.lighthouse/jenkins-x/lint-pipelines.yaml                                    OK
.lighthouse/jenkins-x/pullrequest.yaml                                       OK
.lighthouse/jenkins-x/pullrequest.yaml                                       OK
.lighthouse/jenkins-x/pullrequest.yaml                                       OK
.lighthouse/jenkins-x/pullrequest.yaml                                       OK
.lighthouse/jenkins-x/pullrequest.yaml                                       OK
.lighthouse/jenkins-x/release.yaml                                           OK
.lighthouse/jenkins-x/triggers.yaml                                          OK
environment/.lighthouse/jenkins-x/pullrequest.yaml                           OK
environment/.lighthouse/jenkins-x/release.yaml                               OK
environment/.lighthouse/jenkins-x/triggers.yaml                              OK
packs/spring-boot-starters/.lighthouse/jenkins-x/candidate-image.yaml OK
packs/spring-boot-starters/.lighthouse/jenkins-x/pullrequest.yaml     OK
packs/spring-boot-starters/.lighthouse/jenkins-x/release.yaml         OK
packs/spring-boot-starters/.lighthouse/jenkins-x/triggers.yaml        OK

i currently have to resort to an ugly workaround that is dangerous to run locally since the --all flag errors out when there are other yaml files present in the target path

find . -type d -name 'charts' -exec rm -rf {} +
find . -type d -name 'preview' -exec rm -rf {} +

jx pipeline lint -a \
  --git-username="$GIT_USER" \
  --git-token="$GIT_TOKEN" \

results

.lighthouse/jenkins-x/lint-helm.yaml                                         OK
.lighthouse/jenkins-x/lint-pipelines.yaml                                    OK
.lighthouse/jenkins-x/pullrequest.yaml                                       OK
.lighthouse/jenkins-x/release.yaml                                           OK
environment/.lighthouse/jenkins-x/pullrequest.yaml                           OK
environment/.lighthouse/jenkins-x/release.yaml                               OK
packs/spring-boot-starters/.lighthouse/jenkins-x/candidate-image.yaml OK
packs/spring-boot-starters/.lighthouse/jenkins-x/pullrequest.yaml     OK
packs/spring-boot-starters/.lighthouse/jenkins-x/release.yaml         OK
tasks/spring-boot-starters/candidate-image.yaml                       OK
tasks/spring-boot-starters/pullrequest.yaml                           OK
tasks/spring-boot-starters/release.yaml                               OK

When using jx pipeline start It would be nice to preserve parameters set when starting the pipeline via lighthouse.

Use Case:
Starting new pipelines inside a pipeline without making comments on the PR

https://kubernetes.slack.com/archives/C9MBGQJRH/p1625150595279100

PipelineRuns might not have any executable tasks, typically due to when elements.

Such PipelineRuns are listed both when issuing jx pipeline logs -p and jx pipeline pods -p.

Also the status is stated as Running by jx pipeline pods, while jx pipeline grid list them as Pending. I'd say both are wrong and I think Succeeded would be most fitting. Or maybe a new status should be invented.