collect-only/report-only not working as expected with bash scripts

[9-para]

kcov --collect-only ./coverage ./test.sh working as expected tried with --debug looks good collecting & generating data.

While use the --report-only it's appending hash at the end of filename which does not exist in directory.

kcov --report-only ./coverage ./test.sh tried deleting coverage & regenerating along with --clean however each time it's creating same Hash.

[SimonKagstrom]

I think this might be an issue with bash coverage (and I guess Python will show the same problem). I've really only used collect/report with compiled code, so I don't think it's really an issue specific to MacOS.

[9-para]

You are right it's not issue specific to MacOS it's with bash coverage.

[SimonKagstrom]

Having investigated this further, there seems to be two issues: One is that the hash given to the filename in some instances uses the file content, and in some the file name, which explains the problem when running report-only.

However, I tried removing that part, and it still doesn't work, so there's more to it than that. Probably there are more instances where there's a mixup between filenames and contents, but I don't really know.

[williamdes]

Files: https://github.com/sudo-bot/action-shunit2/tree/main/docker/tests

 kcov v41-161-gce4b
Ran 1 test.
kcov: error: report-only selected, but the target directory /tmp/kcov//test.sh.242af937 does not exist

It looks like I ran into this one
Logs: https://github.com/sudo-bot/action-shunit2/actions/runs/9846037080/job/27182922365#step:4:6

Previous versions worked fine:

 kcov 38
Ran 1 test.
-rw-r--r--    1 root     root        3.7K Jul  8 19:47 /tmp/kcov/index.html

Logs: https://github.com/sudo-bot/action-shunit2/actions/runs/9845959366/job/27182662377#step:4:8

[SimonKagstrom]

I believe this was caused by the changes made for #343 , but I haven't really figured out what.

[SimonKagstrom]

PR-448 should get this back to how v38 worked. However, even that is not really working for bash scripts, and for me, I don't get coverage information with v38 either. I suspect it has really never worked with shell scripts.

The original motivation for this was to be able to use --collect-only on a "target" machine, which has the binaries, and --report-only on a machine where the source code resides. But that usecase is only relevant for compiled code.

[SimonKagstrom]

I'll leave this open, since it only fixes the kcov: error: report-only selected, but ... error, not the actual collect + report sequence for bash scripts.

[williamdes]

I suspect it has really never worked with shell scripts.

https://github.com/code-lts/sql-backup/blob/main/.github/workflows/tests.yml

Everything indicates I managed to make shunit2 and kcov work
3-4 years ago

[SimonKagstrom]

@williamdes yes, but was it with --collect-only + --report-only?

I see that my formulation was misleading: Bash scripts certainly work (and are verified in the testsuite), but I don't think collect + report has ever worked for them.

But then I wasn't able to build older than v37 on my Mac, so if there was a regression earlier than that, I might have missed it.

[williamdes]

@williamdes yes, but was it with --collect-only + --report-only?

report only, see https://github.com/sudo-bot/action-shunit2/blob/95e52443b93365b9ce847a7c630ccbfb2728f3c4/docker/tests/versions.sh#L9
Folder: https://github.com/sudo-bot/action-shunit2/tree/main/docker/tests

kcov: error: report-only selected, but the target directory /tmp/kcov//test.sh.ccf0d740f699f373 does not exist

Running commit 1e383e5

I notice here the weird double slash, should it not auto create the folder since I did not give it the name test.sh.ccf0d740f699f373 ?

[SimonKagstrom]

But was there a --collect-only for that directory first? Didn't show in the link at least.

The name is generated from the file content, but a symlink for the regular name is also used. The purpose of this is to allow rerunning with changed files, without trying to accumulate results from the last.

[williamdes]

But was there a --collect-only for that directory first? Didn't show in the link at least.

Nope, I guess this was a hidden bug as previous versions did not complain about it ?

That said the current option is documented (doc/kcov.1) as

   --report-only
         Only report HTML/Cobertura output, don't collect data.

And that is what I wanted, I only want the coverage file to send to codecov :)