Random duplicate jobs

[ankitm123]

Randomly, duplicate jx jobs are created a few seconds apart (2-3 seconds).
This leads to wasting cloud resources.
Looking at the lighthouse webhook pod, only shows one lighthouse job was created, but build controller is creating 2 pipeline activities, which makes sense as there are 2 lighthouse jobs which creates 2 tekton pipelineruns.

I am not sure what is the root cause, but the second job does not have a GUID associated with it. It does have the created-by-lighthouse label set to true, so we know it was created by lighthouse (and not some weird github issue)
So, we can probably check if a GUID exists before creating a lighthousejob.

Long term: we want to understand the root cause of the duplicate jobs (a rogue goroutine may be)?

This is the label section of the duplicate job (missing guid):

labels:
    created-by-lighthouse: "true"
    lighthouse.jenkins-x.io/baseSHA: dff14889a6b9ef3023fa757f652e001918908a0d
    lighthouse.jenkins-x.io/branch: PR-404
    lighthouse.jenkins-x.io/buildNum: "1645026586795"
    lighthouse.jenkins-x.io/context: JenkinsX
    lighthouse.jenkins-x.io/job: pr
    lighthouse.jenkins-x.io/lastCommitSHA: sha
    lighthouse.jenkins-x.io/refs.org: org
    lighthouse.jenkins-x.io/refs.pull: "404"
    lighthouse.jenkins-x.io/refs.repo: repo
    lighthouse.jenkins-x.io/type: presubmit

Original job:

labels:
    created-by-lighthouse: "true"
    event-GUID: some-guid
    lighthouse.jenkins-x.io/baseSHA: dff14889a6b9ef3023fa757f652e001918908a0d
    lighthouse.jenkins-x.io/branch: PR-404
    lighthouse.jenkins-x.io/buildNum: "1645026586795"
    lighthouse.jenkins-x.io/context: JenkinsX
    lighthouse.jenkins-x.io/job: pr
    lighthouse.jenkins-x.io/lastCommitSHA: sha
    lighthouse.jenkins-x.io/refs.org: org
    lighthouse.jenkins-x.io/refs.pull: "404"
    lighthouse.jenkins-x.io/refs.repo: repo
    lighthouse.jenkins-x.io/type: presubmit

[ankitm123]

/assign

[ankitm123]

Actually, doing this will break manual trigger done using jx pipeline start which dont have an event guid associated with them.

[jobcespedes]

I have a similar issue: a manually merge pr in github web ui created two duplicate release jobs (and pipelines). I noticed two things:

1. event-GUID label of one of them is actually a commit sha 70d481af579866d881749c65298ed67b95483c28. The other one has d602e58a-0616-11ee-9ca0-9690a0c9bd4a and it does correspond to github webhook call header X-GitHub-Delivery of event type push
2. Github created two webhook calls almost at the same time: one event of type pull_request.closed and then one for push

@ankitm123 in case the info is of any help

[msvticket]

That the GUID isn't there point towards that the job is created by keeper, not webhook.