Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #844

Merged
merged 28 commits into from
Dec 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
da020e7
ci: Limit Github Actions token scope
Starkteetje Sep 23, 2022
081cde3
fix: fixes docker build and trivy scan
phbelitz Nov 3, 2022
e07b870
update: update cosign to 1.13.1
phbelitz Nov 3, 2022
debad39
ci: make coverage upload optional
phbelitz Nov 3, 2022
ad974f7
fix: cosign chaged error code
phbelitz Nov 3, 2022
08793d0
docs: Add pip update to fix commands for running tests
Starkteetje Nov 4, 2022
992a175
test: Add test to confirm image is logged on validation error
Starkteetje Nov 4, 2022
ba221c8
fix: Make logging of image consistent across validators
Starkteetje Nov 4, 2022
2b553e5
update: update mkdocs-material requirement from ~=8.5.3 to ~=8.5.8
dependabot[bot] Nov 4, 2022
73f884d
update: update setuptools requirement from ~=65.3.0 to ~=65.5.0
dependabot[bot] Nov 4, 2022
91d2436
update: update pytest-cov requirement from ~=3.0.0 to ~=4.0.0
dependabot[bot] Nov 4, 2022
10e3ddd
update: update pytest-mock requirement from ~=3.8.2 to ~=3.10.0
dependabot[bot] Nov 4, 2022
ff5ba31
update: update pytest-subprocess requirement from ~=1.4.1 to ~=1.4.2
dependabot[bot] Nov 4, 2022
6070883
ci: temporarily removed old k8s versions
phbelitz Nov 25, 2022
aceee43
ci: acquire sa token
phbelitz Nov 25, 2022
fd32a09
ci: Change deprecated set-output to GITHUB_OUTPUT variable
Starkteetje Nov 25, 2022
ad0ce79
ci: Enable tests for older k8s versions again
Starkteetje Nov 25, 2022
ba35976
fix: Don't clear image tag
richgerrard Oct 18, 2022
00b6d2a
fix: revert frozenlist changes
phbelitz Dec 2, 2022
2ed65f7
fix: allowlist CVE-2022-41717
phbelitz Dec 16, 2022
8b2feda
update: update pylint requirement from ~=2.15.3 to ~=2.15.9 (#842)
dependabot[bot] Dec 23, 2022
e34126c
update: update setuptools requirement from ~=65.5.0 to ~=65.6.3 (#830)
dependabot[bot] Dec 23, 2022
21c639f
update: update pytest-asyncio requirement from ~=0.19.0 to ~=0.20.3 (…
dependabot[bot] Dec 23, 2022
5409b83
update: update pytz requirement from ~=2022.2 to ~=2022.7 (#841)
dependabot[bot] Dec 23, 2022
df8e82b
update: update jsonschema requirement from ~=4.16.0 to ~=4.17.3 (#834)
dependabot[bot] Dec 23, 2022
9248bd1
feat: support tags and digest simultaneously
chgl Aug 22, 2022
420fb36
refactor: isort for package import
phbelitz Dec 23, 2022
7ef4625
feat: Allow configuration of Connaisseur's TLS certificate
Starkteetje Dec 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/actions/k3s-cluster/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ inputs:
outputs:
kubeconfig:
description: Path to kubeconfig file
value: ${{ steps.set-output.outputs.kubeconfig }}
value: ${{ steps.set-versions.outputs.kubeconfig }}
k3s-version:
description: "Installed k3s version, such as v1.20.0+k3s2"
value: "${{ steps.set-output.outputs.k3s-version }}"
value: "${{ steps.set-versions.outputs.k3s-version }}"
k8s-version:
description: "Installed k8s version, such as v1.20.0"
value: "${{ steps.set-output.outputs.k8s-version }}"
value: "${{ steps.set-versions.outputs.k8s-version }}"
helm-version:
description: "Installed helm version, such as v3.4.2"
value: "${{ steps.set-output.outputs.helm-version }}"
value: "${{ steps.set-versions.outputs.helm-version }}"

runs:
using: "composite"
Expand All @@ -49,12 +49,12 @@ runs:
shell: bash

- name: Set version output
id: set-output
id: set-versions
run: |
echo "::group::Set version output"
echo "::set-output name=kubeconfig::$HOME/.kube/config"
echo "::set-output name=k3s-version::$(k3s --version | sed 's/.*\(v[0-9][^ ]*\).*/\1/')"
echo "::set-output name=k8s-version::$(k3s --version | sed 's/.*\(v[0-9][^+]*\).*/\1/')"
echo "kubeconfig=$HOME/.kube/config" >> $GITHUB_OUTPUT
echo "k3s-version=$(k3s --version | grep 'k3s' | sed 's/.*\(v[0-9][^ ]*\).*/\1/')" >> $GITHUB_OUTPUT
echo "k8s-version=$(k3s --version | grep 'k3s' | sed 's/.*\(v[0-9][^+]*\).*/\1/')" >> $GITHUB_OUTPUT
echo "::endgroup::"
shell: bash

Expand Down
4 changes: 3 additions & 1 deletion .github/actions/safety/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ runs:
using: "composite"
steps:
- name: Install packages
run: pip3 install -r requirements_dev.txt
run: |
pip3 install --upgrade pip
pip3 install -r requirements_dev.txt
shell: sh
- name: Freeze packages
run: pip3 freeze > actual_package_versions.txt
Expand Down
67 changes: 56 additions & 11 deletions .github/workflows/cicd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ on:
- master
- develop

# Reduce default permission of GITHUB_TOKEN to nothing
# Repository can still be checked out during these jobs
permissions: {}

jobs:
build:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -68,7 +72,9 @@ jobs:
- uses: actions/checkout@v3
- name: Install packages
# Since we run inside an alpine based container, we cannot compile yarl and multidic
run: YARL_NO_EXTENSIONS=1 MULTIDICT_NO_EXTENSIONS=1 pip3 install -r requirements_dev.txt
run: |
pip3 install --upgrade pip
YARL_NO_EXTENSIONS=1 MULTIDICT_NO_EXTENSIONS=1 pip3 install -r requirements_dev.txt
- name: Lint
run: pylint --ignore-patterns=tests,coverage connaisseur

Expand All @@ -79,7 +85,10 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Install packages
run: pip3 install -r requirements_dev.txt && pip3 install .
run: |
apt update && apt install gcc build-essential -y
pip3 install --upgrade pip
pip3 install -r requirements_dev.txt && pip3 install .
- name: Install Git and curl
run: apt update && apt install -y git curl
- name: Test
Expand All @@ -88,7 +97,6 @@ jobs:
uses: codecov/[email protected]
with:
file: coverage.xml
fail_ci_if_error: true

bandit:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -212,6 +220,7 @@ jobs:
"deployment",
"pre-config",
"other-ns",
"configured-cert",
]
services:
alerting-endpoint:
Expand All @@ -229,7 +238,7 @@ jobs:
- uses: ./.github/actions/k8s-version-config
name: Setup k8s cluster
with:
k8s-version: v1.22
k8s-version: v1.25
- name: Set environment variables for alerting listener
run: |
CONTAINER=$(docker container ls --no-trunc --format "{{json . }}" | jq ' . | select(.Image|match("alerting-endpoint"))')
Expand Down Expand Up @@ -261,12 +270,7 @@ jobs:
strategy:
fail-fast: false
matrix:
k8s-version:
[
"v1.16",
"v1.17",
"v1.18",
"v1.19",
k8s-version: [
"v1.20",
"v1.21",
"v1.22",
Expand Down Expand Up @@ -302,6 +306,47 @@ jobs:
kubectl logs -n connaisseur -lapp.kubernetes.io/name=connaisseur --prefix=true
shell: bash

k8s-legacy-versions:
# k3s with older versions doesn't play with newer kernel, so we're running those on deprecated hosts, yay...
runs-on: ubuntu-18.04
needs: [build]
strategy:
fail-fast: false
matrix:
k8s-version: [
"v1.16",
"v1.17",
"v1.18",
"v1.19",
]
steps:
- uses: actions/checkout@v3
- name: Install yq
run: |
sudo snap install yq
- uses: actions/download-artifact@v3
with:
name: images
- uses: ./.github/actions/k8s-version-config
name: Setup k8s cluster
with:
k8s-version: ${{ matrix.k8s-version }}
- name: Configure Connaisseur
run: |
yq e '.deployment.imagePullPolicy = "Never"' -i helm/values.yaml
yq e '.policy +={"pattern": "docker.io/securesystemsengineering/connaisseur:v*"} | .policy[4].pattern style="double"' -i helm/values.yaml
yq e '.policy[4].validator = "allow"' -i helm/values.yaml
yq e '.deployment.replicasCount = "1"' -i helm/values.yaml
- name: Run pre-config and workload integration tests
run: |
bash tests/integration/integration-test.sh "pre-and-workload"
shell: bash
- name: Display k8s logs if integration test failed
if: failure()
run: |
kubectl logs -n connaisseur -lapp.kubernetes.io/name=connaisseur --prefix=true
shell: bash

upgrade-test:
runs-on: ubuntu-latest
needs: [build]
Expand All @@ -323,7 +368,7 @@ jobs:
- uses: ./.github/actions/k8s-version-config
name: Setup k8s cluster
with:
k8s-version: v1.22
k8s-version: v1.25
- uses: actions/checkout@v3
with:
ref: "master"
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ on:
schedule:
- cron: '40 6 * * 3'

# Reduce default permission of GITHUB_TOKEN to nothing
# Repository can still be checked out during these jobs
permissions: {}

jobs:
analyze:
name: Analyze
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/dockerhub-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
schedule:
- cron: '37 6 * * 3'

# No permissions needed
permissions: {}

jobs:
dockerhub-check:
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/docs-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
branches:
- develop

# Needs write on repo contents as it pushes to gh-pages branch
permissions:
contents: write

jobs:
deploy:
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
tags:
- 'v*.*.*'

# Needs write on repo contents as it pushes to gh-pages branch
permissions:
contents: write

jobs:
deploy:
runs-on: ubuntu-latest
Expand Down
64 changes: 55 additions & 9 deletions .github/workflows/nightly-scans.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ name: nightly-scans

on:
schedule:
- cron: '30 1 * * *'
- cron: "30 1 * * *"

# Reduce default permission of GITHUB_TOKEN to nothing
# Repository can still be checked out during these jobs
permissions: {}

jobs:
build:
Expand Down Expand Up @@ -49,8 +53,7 @@ jobs:
run: |
sudo snap install yq
- name: Get latest public image
run:
echo "LATEST-IMAGE=docker.io/$(yq e '.deployment.image' helm/values.yaml)" >> $GITHUB_ENV
run: echo "LATEST-IMAGE=docker.io/$(yq e '.deployment.image' helm/values.yaml)" >> $GITHUB_ENV
- name: Run Trivy
uses: aquasecurity/trivy-action@master
with:
Expand Down Expand Up @@ -78,8 +81,8 @@ jobs:
- uses: actions/upload-artifact@v3
if: failure()
with:
name: trivy-reports
path: trivy-reports
name: trivy-reports
path: trivy-reports

get-root:
runs-on: ubuntu-latest
Expand All @@ -99,7 +102,8 @@ jobs:
strategy:
fail-fast: false
matrix:
integration-test-arg: ["regular", "cosign", "deployment", "pre-config", "helm-repo"]
integration-test-arg:
["regular", "cosign", "deployment", "pre-config", "helm-repo"]
services:
alerting-endpoint:
image: securesystemsengineering/alerting-endpoint
Expand All @@ -115,7 +119,7 @@ jobs:
- uses: ./.github/actions/k3s-cluster
name: Setup K8s cluster
with:
k3s-channel: v1.22
k3s-channel: v1.25
- name: Set environment variables for alerting listener
run: |
CONTAINER=$(docker container ls --no-trunc --format "{{json . }}" | jq ' . | select(.Image|match("alerting-endpoint"))')
Expand Down Expand Up @@ -146,8 +150,50 @@ jobs:
strategy:
fail-fast: false
matrix:
k8s-version:
["v1.16", "v1.17", "v1.18", "v1.19", "v1.20", "v1.21", "v1.22", "v1.23", "v1.24", "v1.25"]
k8s-version: [
"v1.20",
"v1.21",
"v1.22",
"v1.23",
"v1.24",
"v1.25",
]
steps:
- uses: actions/checkout@v3
- name: Install yq
run: |
sudo snap install yq
- uses: ./.github/actions/k8s-version-config
name: Setup k8s cluster
with:
k8s-version: ${{ matrix.k8s-version }}
load-local-image: false
- name: Configure Connaisseur
run: |
yq e '.deployment.replicasCount = "1"' -i helm/values.yaml
- name: Run pre-config and workload integration tests
run: |
bash tests/integration/integration-test.sh "nightly-pre-and-workload"
shell: bash
- name: Display k8s logs if integration test failed
if: failure()
run: |
kubectl logs -n connaisseur -lapp.kubernetes.io/name=connaisseur --prefix=true
shell: bash


k8s-legacy-versions:
# k3s with older versions doesn't play with newer kernel, so we're running those on deprecated hosts, yay...
runs-on: ubuntu-18.04
strategy:
fail-fast: false
matrix:
k8s-version: [
"v1.16",
"v1.17",
"v1.18",
"v1.19",
]
steps:
- uses: actions/checkout@v3
- name: Install yq
Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
push:
tags: "v*"

# Reduce default permission of GITHUB_TOKEN to nothing
# Repository can still be checked out during these jobs
permissions: {}

jobs:
version-match:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -46,7 +50,7 @@ jobs:
- uses: ./.github/actions/k8s-version-config
name: Setup k8s cluster
with:
k8s-version: v1.22
k8s-version: v1.25
load-local-image: false
- name: Set environment variables for alerting listener
run: |
Expand Down Expand Up @@ -76,6 +80,8 @@ jobs:
publish_chart:
runs-on: ubuntu-latest
needs: [version-match, integration-test]
permissions:
contents: write
steps:
- name: Install Helm and Git
run: |
Expand Down
6 changes: 3 additions & 3 deletions .trivyignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
CVE-2022-23628
CVE-2022-28946
CVE-2022-28948
CVE-2022-32149
GHSA-69ch-w2m2-3vjp
CVE-2022-41717
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
NAMESPACE = connaisseur
IMAGE := $(shell yq e '.deployment.image' helm/values.yaml)
COSIGN_VERSION = 1.12.1
COSIGN_VERSION = 1.13.1

.PHONY: all docker install uninstall upgrade annihilate

Expand Down
3 changes: 1 addition & 2 deletions connaisseur/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,12 @@
from logging.config import dictConfig

from cheroot.server import HTTPServer
from cheroot.wsgi import Server
from cheroot.ssl.builtin import BuiltinSSLAdapter
from cheroot.wsgi import Server

from connaisseur.flask_application import APP
from connaisseur.logging_wrapper import ConnaisseurLoggingWrapper


if __name__ == "__main__":
LOG_LEVEL = os.environ.get("LOG_LEVEL", "INFO")

Expand Down
Loading