Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jobs-builder: add PR jobs for Fedora 35 #419

Merged
merged 1 commit into from
Feb 1, 2022
Merged

jobs-builder: add PR jobs for Fedora 35 #419

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions jobs-builder/jobs/include/ci_entrypoint.sh.inc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash
#
# Copyright (c) 2022 Red Hat, Inc.
# SPDX-License-Identifier: Apache-2.0
#
# Wrapper shell script to call kata-containers/tests/.ci/ci_entry_point.sh
# with expected environment variables exported.
#

set -e
set -x
Comment on lines +10 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Might be worth adding the usual header:

set -o errexit
set -o nounset
set -o pipefail

[ -n "$BASH_VERSION" ] && set -o errtrace
[ -n "${DEBUG:-}" ] && set -o xtrace

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script is just a thin wrapper to https://raw.githubusercontent.com/kata-containers/tests/main/.ci/ci_entry_point.sh , which in turn adds all those flags :)


curl -OL https://raw.githubusercontent.com/kata-containers/tests/main/.ci/ci_entry_point.sh
export DEBUG=true
export CI_JOB="{ci_job}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing $

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devimc thanks for reviewing it! The Jenkins Job Builder will interpret a value enclosed in curly brackets as a template variable. Example, suppose I set ci_jo: foobar somewhere on the yaml file, that will result on the line export CI_JOB="{ci_job}".

If you don't want it to interpret the value as a template variable then you can scape (use double curly brackets). That answers your next comment; ${{GOPATH}} becomes ${GOPATH}.

bash -x ci_entry_point.sh "$GIT_URL"
2 changes: 1 addition & 1 deletion jobs-builder/jobs/include/os2node.yaml.inc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#}
{%- if os == "centos8" -%}
centos8_azure
{%- elif os == "fedora35" -%}
{%- elif os in ["fedora35", "fedora-35"] -%}
fedora35_azure
{%- elif os == "ubuntu1804" -%}
ubuntu1804_azure || ubuntu1804-azure
Expand Down
180 changes: 180 additions & 0 deletions jobs-builder/jobs/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Copyright (c) 2022 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# This file contains the configurations to generate the pull request and
# baseline jobs.
#
---
###
# Define shareable YAML snippets.
###
- main_jobs_common_properties: &main_jobs_common_properties
name: 'common_job_properties'
project-type: freestyle
disabled: false
concurrent: {concurrent_toggle}
logrotate:
daysToKeep: 30
numToKeep: 30
# Convert the os variable to label name.
node: !include-jinja2: include/os2node.yaml.inc
wrappers:
- ansicolor:
colormap: "xterm"
- openstack:
single-use: True
- timestamps
- timeout:
timeout: 20
type: no-activity
- main_jobs_common_publishers: &main_jobs_common_publishers
name: 'default_publishers'
publishers:
- post-tasks:
- matches:
- log-text: .*
operator: OR
script: |
#!/bin/bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, why not add the standard shell header here too?

Random thought: I wonder if we need a .ci/kata_header.sh to make this easy to source and minimise duplication.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is yet another thin wrapper, this time to https://raw.githubusercontent.com/kata-containers/tests/main/.ci/teardown.sh . I just realized that unlike ci_entry_point.sh, the teardown.sh script doesn't set those shell properties. Maybe it is by design because we don't want it to fail early? (it collects logs; one or more commands might fail and it is ok?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding adding the .ci/kata_header.sh; then people like me will forget to source it :)


export GOPATH=$WORKSPACE/go
export GOROOT="/usr/local/go"
export PATH="${{GOPATH}}/bin:/usr/local/go/bin:/usr/sbin:/usr/local/bin:${{PATH}}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why double { here and only one { in the following line ?


cd $GOPATH/src/github.com/kata-containers/tests
.ci/teardown.sh "$WORKSPACE/artifacts"
- archive:
artifacts: "artifacts/*"
###
# Define jobs templates.
###
- job-template:
# Use to generate baseline jobs.
#
# Arguments:
# branch - the repository branch.
# os - the node Operating System in <name>-<version> format.
# arch - the node architecture (e.g x86_64, s390x, ppc64le, and so on).
# ci_job - the CI job type as defined in https://github.com/kata-containers/tests/blob/main/.ci/ci_job_flags.sh
# maintainers - a list of maintainer (use the github's mentions @somebody).
#
name: "kata-containers-{branch}-{os}-{arch}-{ci_job}-baseline"
<<: *main_jobs_common_properties
# Allow concurrent jobs by default. Specify `false` on the project definition otherwise.
concurrent_toggle: true
description:
!j2: |
<pre>
Baseline job.
/scheduled
status="Stable"
OS="{{ os }}"
arch="{{ arch }}"
CI_JOB="{{ ci_job }}"
repo="kata-containers"
type="baseline"
Maintainers:
{% for i in maintainers -%}
- {{ i }}
{% endfor %}
</pre>
scm:
- git:
url: https://github.com/kata-containers/kata-containers
branches:
- '*/{branch}'
wipe-workspace: false
triggers:
- timed: 'H 0 * * *'
builders:
- shell:
!include-raw: include/ci_entrypoint.sh.inc
<<: *main_jobs_common_publishers

- job-template:
# Use to generate pull request (PR) jobs.
#
# Arguments:
# repo - the repository name.
# os - the node Operating System in <name>-<version> format.
# arch - the node architecture (e.g x86_64, s390x, ppc64le, and so on).
# ci_job - the CI job type as defined in https://github.com/kata-containers/tests/blob/main/.ci/ci_job_flags.sh
#
name: "{repo}-{os}-{arch}-{ci_job}-PR"
<<: *main_jobs_common_properties
# Allow concurrent jobs by default. Specify `false` on the project definition otherwise.
concurrent_toggle: true
description:
!j2: |
<pre>
Pull Request (PR) job.
OS="{{ os }}"
arch="{{ arch }}"
CI_JOB="{{ ci_job }}"
repo="{{ repo }}"
type="PR"
</pre>
scm:
- git:
url: https://github.com/kata-containers/{repo}
branches:
- '${{sha1}}'
refspec: '+refs/pull/${{ghprbPullId}}/*:refs/remotes/origin/pr/${{ghprbPullId}}/*'
wipe-workspace: false
properties:
- github:
url: https://github.com/kata-containers/{repo}
triggers:
- github-pull-request:
auth-id: 'katacontainers'
github-hooks: true
# Trigger only on commenting phrase in the pull request.
only-trigger-phrase: true
# The expected phrase will be like "/(re)test-<OS Name>"
trigger-phrase:
!j2: '.*(\n|^|\s)/(re)?test(-{{ os.split("-")[0] }})?(\n|$|\s)+.*'
# Skip on commenting phrase.
skip-build-phrase: '.*\[skip\W+ci\].*'
cron: 'H/5 * * * *'
# List of organizations whose members are allowed to build.
org-list:
- kata-containers
# Members of allowed organizations will have admin rights.
allow-whitelist-orgs-as-admins: true
# Branches allowed to be tested.
white-list-target-branches:
- main
- stable-2.*
# Branches disallowed to be tested.
black-list-target-branches:
- master
- stable-1.*
builders:
- shell:
!include-raw: include/ci_entrypoint.sh.inc
<<: *main_jobs_common_publishers

###
# Define the projects
###
- project:
name: "Generate jobs for CRIO_K8S"
repo:
- kata-containers
- tests
branch:
- main
os:
- fedora-35
arch:
- x86_64
ci_job:
- CRIO_K8S
- CRIO_K8S_MINIMAL
maintainers:
- '@snir911'
- '@c3d'
Comment on lines +176 to +177
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooi, are you both in the same TZ? If so, maybe we could add a few more users: I'd say minimum of three, but even better would be to add a single GH team here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We agreed that baseline jobs should be owners (#403). @snir911 and @c3d are going to be the owners of those Fedora 35 jobs. So it is fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does having a user specified in maintainers here actually mean though? What if they are both unavailable? Is it simply for mail notification or similar?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understood your first question. The maintainers variable is used on the description of the job, so on the Jenkins UI it will be listed the names on the jobs' description. An example here: http://jenkins.katacontainers.io/view/Daily%20baseline/job/kata-containers-main-fedora-35-x86_64-CRIO_K8S-baseline/

The idea is to have their names on the job's description for the case a developer need to speak with the owners (e.g. developer's PR cannot be merged because a job is failing and she/he notices the baseline has failed too). The owners are also listed in a table on the view here. Having their names on the job's description just increase visibility. Do you see any problem with that?

jobs:
- 'kata-containers-{branch}-{os}-{arch}-{ci_job}-baseline'
- '{repo}-{os}-{arch}-{ci_job}-PR'