Build multi OS for Docker #452

williamdes · 2024-07-14T09:36:45Z

Fixes #451
Fixes #450

codecov · 2024-07-14T09:42:27Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.60%. Comparing base (bf2c594) to head (ebd2ee0).
Report is 10 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #452      +/-   ##
==========================================
- Coverage   69.85%   69.60%   -0.26%     
==========================================
  Files          55       55              
  Lines        4249     4250       +1     
  Branches     3906     3909       +3     
==========================================
- Hits         2968     2958      -10     
- Misses       1281     1292      +11

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Dockerfile

SimonKagstrom · 2024-07-14T11:22:18Z

Man, you work so quickly! :-)

For now build tests do not pass on Alpine, I added a new workflow Can you help ? It looks like some cmake guru is needed:

/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../libssp_nonshared.a when searching for -lssp_nonshared
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: skipping incompatible /usr/lib/libssp_nonshared.a when searching for -lssp_nonshared

OK, so the fork-32 test fail to build (it appears, from the logs). The test was not really working anyway, so I removed it in master. Perhaps you could try to merge/rebase onto that?

By the way, did you forget this ?:

commit b31bdf569202087aa34681136ef0fee71e021cbc (HEAD -> multi-dist)
Author: William Desportes <[email protected]>
Date:   Sun Jul 14 12:19:47 2024 +0200

    chore: bump to v43

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 98608be..950cfd2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,7 +1,7 @@
 cmake_minimum_required (VERSION 3.12)
 project (kcov)
 
-set (PROJECT_VERSION_MAJOR 39)
+set (PROJECT_VERSION_MAJOR 43)

Yeah, I did! The git tag is what mostly determines the version. This is for some cmakery that another guy contributed, so I'm not completely sure why it's needed. But feel free to update it!

williamdes · 2024-07-14T11:56:10Z

Man, you work so quickly! :-)

Haha, 10 years of work ;p

OK, so the fork-32 test fail to build (it appears, from the logs). The test was not really working anyway, so I removed it in master. Perhaps you could try to merge/rebase onto that?

Done

Yeah, I did! The git tag is what mostly determines the version. This is for some cmakery that another guy contributed, so I'm not completely sure why it's needed. But feel free to update it!

Maybe we should remove the Cmakelist stuff and determine the version from that constant(s) ?

williamdes · 2024-07-14T12:13:47Z

It looks like I have the same result as when I tried to make tests work into the Docker image
See: https://github.com/williamdes/kcov/actions/runs/9927752250/job/27423092490#step:5:285

No tests seem to pass, not sure why

SimonKagstrom · 2024-07-14T13:53:52Z

It looks like I have the same result as when I tried to make tests work into the Docker image See: https://github.com/williamdes/kcov/actions/runs/9927752250/job/27423092490#step:5:285

No tests seem to pass, not sure why

Might be several reasons, I think. For one, it seems like maybe /sbin is not part of the PATH:

runTest (test_compiled.debuglink.runTest) ... sh: rm: not found
sh: cp: not found
objcopy: 'build-tests/main-tests-debug-file': No such file
sh: cp: not found
sh: cp: not found

but my guess is that this is caused by the alpine image not being run with --security-opt seccomp=unconfined, or alternatively that the builder runs qemu, in which case it's the same error as with the non-x86 linuxes.

Anyway, I'm OK with just building it under alpine, so if there's no easy fix, then I think we should just disable the tests there (like non-x86:es).

SimonKagstrom · 2024-07-14T13:58:44Z

Yeah, I did! The git tag is what mostly determines the version. This is for some cmakery that another guy contributed, so I'm not completely sure why it's needed. But feel free to update it!

Maybe we should remove the Cmakelist stuff and determine the version from that constant(s) ?

I can take a look at that. I guess the version should be settable from the git tag for starters at least.

williamdes · 2024-07-14T14:15:42Z

Thank you for the last commit, is there a way that I can call the function and get the output ?
Like I would have done with a Makefile
So I can use the output in the build docker shell script

SimonKagstrom · 2024-07-14T14:26:48Z

Thank you for the last commit, is there a way that I can call the function and get the output ? Like I would have done with a Makefile So I can use the output in the build docker shell script

Well, my commit actually didn't build on github (yes, I should have used a branch first...). I think it's due to missing tags, so I'll attempt to repair it before reverting it.

I'm not quite sure I understand what you want? You mean the version? If you have built kcov, you can get it via

ska@Simons-MacBook-Air kcov % src/kcov --version
kcov v42-149-g4cb9

williamdes · 2024-07-14T14:39:36Z

fatal: No names found, cannot describe anything.

Same as me today, see ad89c0f

You need to have fetch depth 0 or some other depth or maybe just try to enable fetch tags ?

I'm not quite sure I understand what you want? You mean the version? If you have built kcov, you can get it via

Indeed, but is there a way to get it before the build as I need to inject the version built into the metadata before launching docker

SimonKagstrom · 2024-07-14T14:42:31Z

The latest master should build again.

I guess it might be possible to replicate in ci.yml? It's really just boils down to git describe --abbrev=4 --tags HEAD (maybe with --git-dir).

williamdes · 2024-07-14T14:45:06Z

The latest master should build again.

Yay, can you try only switching on the fetch-tags with a fetch depth of 100 maybe ?
Just trying to lower the number of commits fetched for each CI run 😄

I guess it might be possible to replicate in ci.yml? It's really just boils down to git describe --abbrev=4 --tags HEAD (maybe with --git-dir).

Not sure what you mean here

With coreutils-env

williamdes · 2024-07-14T16:53:54Z

tests/tools/test_basic.py

@@ -26,7 +26,7 @@ def runTest(self):
 class lookup_binary_in_path(libkcov.TestCase):
    @unittest.expectedFailure
    def runTest(self):
-        os.environ["PATH"] += self.sources + "/tests/python"
+        os.environ["PATH"] += os.pathsep + self.sources + "/tests/python"


This was the patch needed to fix things up

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:./tests/python

without the separator nothing works correctly, and rm, mv and friends can not be found

Wow, good catch!

I wonder why it was like that? I guess maybe self.sources was an absolute path in some cases, and a relative one in another. Anyhow, good that it's fixed!

Best guess the last path in the list was not that important, but not on Alpine haha
Always add the separator 😄

Dockerfile

SimonKagstrom · 2024-07-14T17:10:32Z

.github/workflows/ci.yml

+      container:
+        image: alpine:3.20
+        # Needed to attach to a binary
+        options: --security-opt seccomp=unconfined


Maybe, as you suggested, add this to the instructions somewhere? I guess the docker info can be placed in a doc/docker.md file, to reduce the clutter in README.md.

SimonKagstrom · 2024-07-14T17:16:16Z

--security-opt seccomp=unconfined

That was it, it should be documented somewhere in this project What is the exact kernel capability kcov needs ? https://docs.docker.com/engine/security/seccomp/#significant-syscalls-blocked-by-the-default-profile

ptrace definately, and probably process_vm_readv/process_vm_writev and personality.

williamdes · 2024-07-14T17:50:58Z

It should be okay now, I updated the platform list for Debian. They do not have riscv64

SimonKagstrom · 2024-07-14T17:58:53Z

doc/docker.md

+kcov needs access the following system calls:
+- [`ptrace`](https://linux.die.net/man/2/ptrace)
+- [`process_vm_readv`](https://linux.die.net/man/2/process_vm_readv)/[`process_vm_writev`](https://linux.die.net/man/2/process_vm_writev)
+- [`personality`](https://linux.die.net/man/2/personality)


Well written with references!

SimonKagstrom · 2024-07-14T18:02:14Z

Merged, thanks a lot!

williamdes · 2024-07-14T18:02:33Z

Currently waiting on the Docker build to fail push (workflow dispatch is very cool !) after build success: https://github.com/williamdes/kcov/actions/runs/9929723485

williamdes · 2024-07-14T18:03:17Z

Merged, thanks a lot!

Thank you too for this very nice and active collaboration !
Let's monitor the build jobs 🤞🏻

SimonKagstrom · 2024-07-14T18:04:33Z

I'm sure it will build fie!

I saw the workflow_dispatch stuff, something to read up on in the future!

williamdes · 2024-07-14T18:07:03Z

Something I am a bit unsure about is why it looks like the docker build also triggers for PRs, maybe a side effect of workflow dispatch

williamdes mentioned this pull request Jul 14, 2024

docker: Build for debian by default #450

Closed

This comment was marked as resolved.

Sign in to view

williamdes had a problem deploying to Docker Hub July 14, 2024 10:27 — with GitHub Actions Failure

SimonKagstrom approved these changes Jul 14, 2024

View reviewed changes

Dockerfile Show resolved Hide resolved

williamdes had a problem deploying to Docker Hub July 14, 2024 11:33 — with GitHub Actions Failure

williamdes added 9 commits July 14, 2024 13:53

ci: build multi OS for Docker

f74616c

docs: update INSTALL.md

f6f8c11

ci: add metadata to the Docker image

f69e13c

ci: Allow to manually trigger docker builds

d49c5a6

ci: Add an Alpine build

985c9fb

ci: use push for Docker buildx and fix Alpine

847fecd

ci: add bash for Alpine

6bb7376

ci: add more libs to link for gcc (alpine)

9331711

ci: also fetch tags for git describe

ad89c0f

williamdes had a problem deploying to Docker Hub July 14, 2024 12:08 — with GitHub Actions Failure

ci: also allow not having tags (repo fork for example)

191540c

williamdes had a problem deploying to Docker Hub July 14, 2024 16:01 — with GitHub Actions Failure

williamdes added 9 commits July 14, 2024 18:27

ci: fix Alpine builds

5bf11c1

ci: add libbfd-dev for Docker

37af413

ci: add patchelf for tests

99343a3

ci: improve logic syntax

e26723e

ci: provide "python" in the path

c7c9753

ci: add binutils for objcopy

e4fd20d

ci: add a fallback and print the release version

178e11f

tests: lower timeout of attach_process_* tests

9b7cb54

ci: provide usr/bin/env to some scripts (alpine)

6611829

With coreutils-env

williamdes commented Jul 14, 2024

View reviewed changes

ci: fix PATH for tests

df77f82

williamdes marked this pull request as ready for review July 14, 2024 17:04

SimonKagstrom approved these changes Jul 14, 2024

View reviewed changes

williamdes had a problem deploying to Docker Hub July 14, 2024 17:12 — with GitHub Actions Failure

williamdes added 2 commits July 14, 2024 19:49

docs: Docker instructions

e7f41cc

ci: fixup platforms for Debian

c7a2700

SimonKagstrom reviewed Jul 14, 2024

View reviewed changes

SimonKagstrom merged commit 1e383e5 into SimonKagstrom:master Jul 14, 2024
9 checks passed

williamdes had a problem deploying to Docker Hub July 14, 2024 18:05 — with GitHub Actions Failure

williamdes deleted the multi-dist branch July 17, 2024 11:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build multi OS for Docker #452

Build multi OS for Docker #452

williamdes commented Jul 14, 2024 •

edited

Loading

codecov bot commented Jul 14, 2024 •

edited

Loading

This comment was marked as resolved.

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes Jul 14, 2024

SimonKagstrom Jul 14, 2024

williamdes Jul 14, 2024

SimonKagstrom Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

Build multi OS for Docker #452

Build multi OS for Docker #452

Conversation

williamdes commented Jul 14, 2024 • edited Loading

codecov bot commented Jul 14, 2024 • edited Loading

Codecov Report

This comment was marked as resolved.

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes Jul 14, 2024

Choose a reason for hiding this comment

SimonKagstrom Jul 14, 2024

Choose a reason for hiding this comment

williamdes Jul 14, 2024

Choose a reason for hiding this comment

SimonKagstrom Jul 14, 2024

Choose a reason for hiding this comment

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom Jul 14, 2024

Choose a reason for hiding this comment

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes commented Jul 14, 2024

SimonKagstrom commented Jul 14, 2024

williamdes commented Jul 14, 2024

williamdes commented Jul 14, 2024 •

edited

Loading

codecov bot commented Jul 14, 2024 •

edited

Loading