Usage with BATS

[fuero]

I tried using this with BATS (and so has the BATS team), but it doesn't seem to work with current versions.

See details here: bats-core/bats-core#15

Perhaps you can shed some light on this?

[SimonKagstrom]

Could it be the same as in issues #213, #234 ? It sounds like you're running in ci systems, which might run builds as "root".

[fuero]

Thanks for the help. I've had success now inside of a docker container, obviously I messed up and ran kcov as root and missed that point. However, on my Arch workstation, it still doesn't work despite running as an unprivileged user.

[SimonKagstrom]

OK, are the symptoms the same in Arch and in the container?

[ko1nksm]

@fuero says here

For all environments: bats 1.1.0, kcov 36
Arch Linux: GNU bash, version 5.0.7(1)-release (x86_64-pc-linux-gnu)

According to this problem, BASH_XTRACEFD will be 2 (stderr) instead of default 782 on bash 5.0.x.

The bats (and successor bats-core) captures stdout and stderr at once (see sstephenson/bats#180).

The target script that run by bats output trace data to stderr, but trace data absorbed by bats. Therefore kcov cannot get trace data, I think.

This issue fixed in kcov v37.

[ivopieniak]

hi @SimonKagstrom , I have been encountering issues when trying to run kcov(v38) with bats(v1.3.0) with GNU bash, version 5.0.18(1)-release (x86_64-apple-darwin19.5.0). The command I am trying to run is:
#!/usr/bin/env bash kcov --bash-dont-parse-binary-dir --include-path=bin/monitor_pipeline.sh coverage lib/bats-core/bin/bats tests/monitor.pipeline.bats
The errors I get are:
kcov: error: Can't write helper
kcov: error: Can't start/attach to /Users/i.pieniak/working_dir/prototype_shellspec/bats-prototyping/lib/bats-core/bin/bats
kcov: error: Can't open directory /usr/local/bin/coverage/

In one configuration of the commands, I have managed to get the kcov to measure the coverage of the bats source files instead of the bash file mentioned in this issue - bats-core/bats-core#15
I have tried going through the code and willing to figure out what is happening, however this is beyond my knowledge at the moment. I'd really appreciate if you could get provide any hints.

[SimonKagstrom]

@ivopieniak that sounds more like a permission issue, are you able to write to the coverage/-directory? The "Can't write helper" means that a generated file cannot be placed in the output directory.

It looks like it's being run from /usr/local, maybe you could try to run it with the output being placed somewhere else, e.g.,

kcov --bash-dont-parse-binary-dir --include-path=bin/monitor_pipeline.sh /tmp/coverage lib/bats-core/bin/bats tests/monitor.pipeline.bats

so that output is placed on a location where (I think) you should be able to write.

[kinow]

Hi, just passing by searching for other issues, and found this old-ish one about bats + kcov. Our repository is not public at the moment, but we are happily using kcov + bats with no issues. Tested wit:

• Ubuntu LTS, and also in a container with debian:bullseye-slim that imports kcov/kcov:latest in a multi-stage build (kcov version reported is kcov v41-133-gf3edb.
• Bats master branch synced today, last commit from Apr 24, bats-core/bats-core@a751f3d
• Bats extensions (bats-support and bats-assert)

The command we use to run is something similar to this:

$ docker run --rm -it -v "${PWD}:/code" workflow:latest \
    	/usr/local/bin/kcov --include-path=./ ./coverage/ \
    	/usr/local/bin/bats --verbose-run  --recursive ./

On Ubuntu it uses my local user. In the container (in CICD) we use root user. No issues found so far. Thanks!

[SimonKagstrom]

Thanks for the report @kinow! I don't use bats myself (almost only using kcov for compiled code), so I have a hard time helping with these issues.

Good that the docker image is tested as well, although I note that the version reported there is slightly old.

Closing, since it should at least be possible to get working as per @Kinows report.