-
Notifications
You must be signed in to change notification settings - Fork 16
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: use tekton pipeline remote resolution
Signed-off-by: JordanGoasdoue <[email protected]>
- Loading branch information
1 parent
23283b6
commit 51ff3db
Showing
1 changed file
with
149 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,149 @@ | ||
--- | ||
title: Use Tekton Pipeline Remote Resolution | ||
linktitle: Use Tekton Pipeline Remote Resolution | ||
description: Create our own Tekton Resolver in order to allow Remote Resolution & uses:sourceURI | ||
weight: 100 | ||
--- | ||
|
||
## 1. Overview | ||
|
||
This document outlines how we can use Tekton Pipeline Remote Resolution removing some scope from Lighthouse | ||
|
||
## 1.1 Motivation | ||
|
||
Let's take a single PipelineRun I can use as example, I am running multiple tasks inside it, it can be: | ||
- in parallel | ||
- with runAfter | ||
- finally | ||
|
||
Now imagine I have multiple team repositories to maintain with all this logic inside (parallel, runAfter, finally tasks), | ||
Jx3 feature notation `image: uses:sourceURI` (that lets you inherit steps from a git repository without having to copy/paste the source code aross repositories) is amazing, but not enough here because it's only inside a task that the magic appears. | ||
|
||
The solution I believe is the following one: [Tekton Pipeline Remote Resolution](https://tekton.dev/vault/pipelines-main/resolution) | ||
- Remote Resolution is a Tekton feature that allows users to fetch tasks and pipelines from remote sources outside the cluster. Tekton provides a few built-in resolvers that can fetch from git repositories, OCI registries etc as well as a framework for writing custom resolvers. | ||
- We can create our own Jx3 Resolver: [How to write a Resolver](https://tekton.dev/vault/pipelines-main/how-to-write-a-resolver) | ||
- We would be able to put all the parallel, runAfter, finally tasks logic inside a remote Pipeline and just reference it inside our PipelineRuns (a lot easier maintenance !) | ||
- The resolution would happen after the PipelineRun is applied to Kubernetes and not before | ||
- We could implement this `image: uses:sourceURI` inside our jx3 tekton resolver | ||
- [this is the current tekton git resolver](https://github.com/tektoncd/pipeline/blob/main/pkg/resolution/resolver/git/resolver.go) | ||
|
||
- This way, we would be closer to Tekton new great solutions, and we would remove from the [WebHook/ChatOps Lighthouse](https://github.com/jenkins-x/lighthouse) the loads of pipelineRuns with `image: uses:sourceURI` logic that needs to happen before the PipelineRun is applied to Kubernetes | ||
|
||
This way, we can use Remote PipelineRef / TaskRef from git to have easier maintenance without copy paste, and still be able to use inside them `image: uses:sourceURI` | ||
|
||
## 1.2 Goal | ||
|
||
The goal is to create our own Jx3-Git Resolver to be able to use Remote Pipeline / Task from Git inside our PipelineRuns with the `image: uses:sourceURI` inheritance working. All this logic could then be removed from the [WebHook/ChatOps Lighthouse](https://github.com/jenkins-x/lighthouse) that could be more focused on Webhook / ChatOps things | ||
|
||
## 1.3 Requirements (I think) | ||
- Kubernetes version: 1.23.x | ||
- [Tekton Pipeline version: 0.41.0](https://github.com/tektoncd/pipeline/releases/tag/v0.41.0) | ||
- Kubernetes version: 1.22.x | ||
- Action required: If using Kubernetes 1.22, set PodSecurity flag to true to enforce a restricted pod security level in Tekton namespaces. See https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-graduated-or-deprecated-features for more information. | ||
|
||
|
||
## 2. Design proposal | ||
|
||
- Design our Jx3-Git Resolver following [Tekton Remote Pipeline Resolution Resolver Interface](https://github.com/tektoncd/pipeline/blob/main/pkg/resolution/resolver/framework/interface.go) | ||
- This would need to be able to Resolve remote Git Pipeline & Task | ||
- This would be nice to still having `image: uses:sourceURI` that allows steps inheritance, injection & cherry-picking | ||
- We would need to then remove this logic from Lighthouse | ||
- Lighthouse would just present a PipelineRun that is applied in Kubernetes cluster | ||
- All the PipelineRuns loads logic would be now inside the Jx3-Git Resolver | ||
|
||
## 3. Affected repositories | ||
|
||
As outlined above the following repositories need changes: | ||
|
||
- [ ] `cdfoundation/tekton-helm-chart` | ||
- [ ] `jenkins-x/jx3-versions` | ||
- [ ] `jenkins-x/lighthouse` | ||
|
||
## 4. Concerns | ||
|
||
## 4.1 Concerns about Jx3 Parameters injection | ||
|
||
I am talking about these Jx3 default Parameters: | ||
``` | ||
params: | ||
- name: BUILD_ID | ||
value: $(params.BUILD_ID) | ||
- name: JOB_NAME | ||
value: $(params.JOB_NAME) | ||
- name: JOB_SPEC | ||
value: $(params.JOB_SPEC) | ||
- name: JOB_TYPE | ||
value: $(params.JOB_TYPE) | ||
- name: PULL_BASE_REF | ||
value: $(params.PULL_BASE_REF) | ||
- name: PULL_BASE_SHA | ||
value: $(params.PULL_BASE_SHA) | ||
- name: PULL_NUMBER | ||
value: $(params.PULL_NUMBER) | ||
- name: PULL_PULL_REF | ||
value: $(params.PULL_PULL_REF) | ||
- name: PULL_PULL_SHA | ||
value: $(params.PULL_PULL_SHA) | ||
- name: PULL_REFS | ||
value: $(params.PULL_REFS) | ||
- name: REPO_NAME | ||
value: $(params.REPO_NAME) | ||
- name: REPO_OWNER | ||
value: $(params.REPO_OWNER) | ||
- name: REPO_URL | ||
value: $(params.REPO_URL) | ||
taskSpec: | ||
params: | ||
- description: the unique build number | ||
name: BUILD_ID | ||
type: string | ||
- description: the name of the job which is the trigger context name | ||
name: JOB_NAME | ||
type: string | ||
- description: the specification of the job | ||
name: JOB_SPEC | ||
type: string | ||
- description: '''the kind of job: postsubmit or presubmit''' | ||
name: JOB_TYPE | ||
type: string | ||
- description: the base git reference of the pull request | ||
name: PULL_BASE_REF | ||
type: string | ||
- description: the git sha of the base of the pull request | ||
name: PULL_BASE_SHA | ||
type: string | ||
- default: "" | ||
description: git pull request number | ||
name: PULL_NUMBER | ||
type: string | ||
- default: "" | ||
description: git pull request ref in the form 'refs/pull/$PULL_NUMBER/head' | ||
name: PULL_PULL_REF | ||
type: string | ||
- default: "" | ||
description: git revision to checkout (branch, tag, sha, ref…) | ||
name: PULL_PULL_SHA | ||
type: string | ||
- description: git pull reference strings of base and latest in the form 'master:$PULL_BASE_SHA,$PULL_NUMBER:$PULL_PULL_SHA:refs/pull/$PULL_NUMBER/head' | ||
name: PULL_REFS | ||
type: string | ||
- description: git repository name | ||
name: REPO_NAME | ||
type: string | ||
- description: git repository owner (user or organisation) | ||
name: REPO_OWNER | ||
type: string | ||
- description: git url to clone | ||
name: REPO_URL | ||
type: string | ||
``` | ||
|
||
Using Tekton Pipeline Remote Resolution instead of Lighthouse put the PipelineRuns Loads logic after Kubernetes applied the PipelineRun and not before. This means that when using remote Task / Pipeline, we don't want to hardcode inside them the default Jx3 Parameters because this would be too verbose. | ||
|
||
I believe then these Jx3 Parameters would need to be injected by this Jx3-Git Resolver, inside: | ||
- PipelineSpec | ||
- TaskSpec | ||
- Finally TasksSpec | ||
|
||
We could make this Injecton step optionnal only if a Lighthouse annotation is present, so even Tekton users | ||
could use the `image: uses:sourceURI` feature without directly using the Jx3 ecosystem. |