ci: added generators for build, test & helm sync #3
Conversation
- very WIP, such JS Ticket: None Changelog: None Signed-off-by: Manuel Zedel <[email protected]>
There was a problem hiding this comment.
There are some jobs missing (none for frontend, build and run backend acceptance and integration tests), as well as publishing so the trigger jobs would not work.
Another issue we will eventually have to deal with is how to deal with test coverage. By excluding jobs, it will also exclude test coverage unless we have a cleaver way of caching coverage. Or should we drop coverage collection?
| - docker buildx build | ||
| --cache-to type=registry,ref=${CI_REGISTRY_IMAGE}:ci_cache,mode=max | ||
| --cache-from type=registry,ref=${CI_REGISTRY_IMAGE}:ci_cache | ||
| --tag \${GITLAB_REGISTRY_TAG} | ||
| --tag ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME} | ||
| --file backend/services/${service}/Dockerfile | ||
| --build-arg GIT_COMMIT_TAG="${DOCKER_PUBLISH_COMMIT_TAG}" | ||
| --platform $MULTIPLATFORM_PLATFORMS | ||
| --provenance false | ||
| --push | ||
| \${EXTRA_DOCKER_ARGS} | ||
| . | ||
| `; |
There was a problem hiding this comment.
Could we instead use the make targets? That makes it easier to reproduce locally and make it more obvious to compare local vs CI builds.
make -C backend/services/${service} docker
| RULES_CHANGES_COMPARE_TO_REF, | ||
| service, | ||
| }) => ` | ||
| build:${service}:docker-multiplatform: |
There was a problem hiding this comment.
Building services in different jobs, although parallel is more wasteful as it cannot leverage the local go cache.
| "auditlogs", | ||
| "create-artifact-worker", | ||
| "deployments", | ||
| "deviceauth", | ||
| "deviceconfig", | ||
| "deviceconnect", | ||
| "devicemonitor", | ||
| "inventory", | ||
| "mender-gateway", | ||
| "tenantadm", | ||
| "useradm", | ||
| "workflows", |
There was a problem hiding this comment.
This list contains enterprise services and mender-gateway which is not part of this repository.
| - job: build:${service}:docker-multiplatform | ||
| artifacts: true | ||
| script: | ||
| - go test -trimpath -ldflags -s -w ./... |
There was a problem hiding this comment.
How do we deal with the test with the same toolchain version as we're building. Today we're not doing a very good job at that - versions managed inside gitlab tend to lag major versions (Go term - not semver) behind the Dockerfile.
| - go test -trimpath -ldflags -s -w ./... | |
| - go test -trimpath -ldflags='-s -w' ./... |
| tags: | ||
| - hetzner-amd-beefy | ||
| services: | ||
| - name: ${CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX}/docker:20.10.21-dind |
There was a problem hiding this comment.
This version of docker is over 3 years old. Could we make the docker version more transparent in the global variable section?
| - name: ${CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX}/docker:20.10.21-dind | |
| - name: ${CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX}/docker:${DOCKER_VERSION}-dind |
| rules: | ||
| ${getChangesRule({ RULES_CHANGES_COMPARE_TO_REF, service })} | ||
| - if: $CI_COMMIT_BRANCH =~ /^(staging)$/ |
There was a problem hiding this comment.
Maybe this is a topic for the QA meeting, but IMO we could drop the staging branch and deploy to staging from main.
| rules: | |
| ${getChangesRule({ RULES_CHANGES_COMPARE_TO_REF, service })} | |
| - if: $CI_COMMIT_BRANCH =~ /^(staging)$/ | |
| rules: | |
| ${getChangesRule({ "HEAD~1", service })} | |
| - if: $CI_COMMIT_BRANCH == main |
|
With regard to the Helm Chart sync, although I consider this a great rework, some key points have to be addressed before:
|
@alfrunes as RFC, resulting in: https://gist.github.com/mzedel/690a2fc1a3bde898ac17b10f2c912b22 - to be included via https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html#dynamic-child-pipelines and potentially extended with the per-service coverage publish jobs...
BUT! I just assume we would get the Dockerfile/ pkg issue to work somehow 🤞