Tiobe Integration

[kadinsayani]

This PR adds code coverage for unit tests and snap system tests.

[tomponline]

Let me know when you need secrets.TICS_AUTH_TOKEN setup

[kadinsayani]

For the system tests using the MicroCloud snap, it looks like the only requirements to generate the coverage files are:

• GOCOVERDIR needs to be set in the snap confinement space - I'm creating a directory on the runner at /var/snap/microcloud/common/data/coverage; and
• The side loaded binary should be compiled with the -cover flag.

@tomponline, once I've got it working here, we can utilize a similar approach for LXD.

[kadinsayani]

There appears to be a problem with the MicroCloud CI tests because of a regression in upstream Ceph versioning. As a result, I haven’t tested this yet.

[kadinsayani]

Should be good to go, just waiting on https://github.com/kadinsayani/microcloud/actions/runs/11413516898/job/31763088034

[kadinsayani]

Good to go on this, here's an example successful run on my fork: https://github.com/kadinsayani/microcloud/actions/runs/11413516898/job/31763088034.

[kadinsayani]

We recently started using the go1.23.1 toolchain for MicroCloud. To use StaticCheck with Go 1.23, it needs to be built with Go 1.23. Hence why the Tics job throws errors like this one:

2024-10-19 04:00:23.592: [INFO 3012] Analyzing 30/58 /home/runner/work/microcloud/microcloud/cmd/microcloud/session.go
start analysis: 2024-10-19 04:00:23.592
  Initialization
  Initialization time: 0.001 sec
Error: [ERROR 5073] The following error(s) occurred while 'StaticCheck' was running: '/home/runner/go/pkg/mod/golang.org/[email protected]/src/maps/iter.go:51:20: cannot range over seq (variable of type iter.Seq2[K, V])'.

These errors are what caused the downtick in TQI scores across various Tiobe quality metrics. The analyzer runs are erroring out, so data isn't being sent to Tics for the coding standards runs.

For the security checks, I'm seeing the following error on every run:

[ERROR 5255] Command 'cov-build' failed with exit status 256, error: -fno-stack-protector ./cgo.go See log '/tmp/tics/11413516898_1_tics_tics-github-action/ticstmpdir/Coverity/cov-build-console.log'.

Tics uses Coverity for security scanning, here's something I found from their docs:

By default, cov-build returns the exit code from the native build.

Exit code 256 typically indicates that the system can't locate the binary to run the command, so I think this is also a dependency issue.

I propose that we bump the Go version in the Tics job to ensure we capture the right dependencies for Tics at the Install dependencies step. Testing now: https://github.com/kadinsayani/microcloud/actions/runs/11420168463.

[tomponline]

Yeah we should switch to go 1.23

[kadinsayani]

Good to go - all of the Tiobe runs are operational and the quality metrics are accurate.

https://github.com/kadinsayani/microcloud/actions/runs/11432414984/job/31830683250

[roosterfish]

Thanks for this! A few comments from my side, mostly for clarification.

[kadinsayani]

I've rebased and resolved a naming issue with the coverage file directories. Test run: https://github.com/kadinsayani/microcloud/actions/runs/11493847667/job/31990262151.

[kadinsayani]

I've rebased and resolved a naming issue with the coverage file directories. Test run: https://github.com/kadinsayani/microcloud/actions/runs/11493847667/job/31990262151.

Looks like there's still an issue with the naming. I'll try and fix this some time today. Just need a unique identifier to append so there aren't naming conflicts.

[kadinsayani]

Successful run: https://github.com/kadinsayani/microcloud/actions/runs/11516440151

[roosterfish]

Thanks!