workload.googleapis.com/apache.current_connections not available on all platforms

[martijnvans]

I'm working on enabling more test coverage for apache, and have found that current_connections is not available on centos-7, sles-12, or sles-15. I believe it's because that metric is specific to the event Multi-Processing Module (MPM), and isn't present on the other MPMs, which are used on older distros. Here's a snapshot of the apache server status page for various distros. Note that some of them are missing ConnsTotal:

centos 7:

        Total Accesses: 2
        Total kBytes: 1
        Uptime: 100
        ReqPerSec: .02
        BytesPerSec: 10.24
        BytesPerReq: 512
        BusyWorkers: 2
        IdleWorkers: 4
        Scoreboard: K_W___..........................................................................................................................................................................................................................................................

debian-11:

127.0.0.1
        ServerVersion: Apache/2.4.56 (Debian)
        ServerMPM: event
        Server Built: 2023-03-08T03:05:04
        CurrentTime: Wednesday, 22-Mar-2023 20:10:56 UTC
        RestartTime: Wednesday, 22-Mar-2023 20:09:16 UTC
        ParentServerConfigGeneration: 1
        ParentServerMPMGeneration: 0
        ServerUptimeSeconds: 100
        ServerUptime: 1 minute 40 seconds
        Load1: 0.21
        Load5: 0.16
        Load15: 0.07
        Total Accesses: 2
        Total kBytes: 1
        Total Duration: 0
        CPUUser: 0
        CPUSystem: 0
        CPUChildrenUser: 0
        CPUChildrenSystem: 0
        Uptime: 100
        ReqPerSec: .02
        BytesPerSec: 10.24
        BytesPerReq: 512
        DurationPerReq: 0
        BusyWorkers: 1
        IdleWorkers: 49
        Processes: 2
        Stopping: 0
        BusyWorkers: 1
        IdleWorkers: 49
        ConnsTotal: 0
        ConnsAsyncWriting: 0
        ConnsAsyncKeepAlive: 0
        ConnsAsyncClosing: 0
        Scoreboard: ______________________________________W___________....................................................................................................

rocky-linux-8:

        127.0.0.1
        ServerVersion: Apache/2.4.37 (rocky)
        ServerMPM: event
        Server Built: Feb 21 2023 18:10:58
        CurrentTime: Wednesday, 22-Mar-2023 20:17:13 UTC
        RestartTime: Wednesday, 22-Mar-2023 20:15:20 UTC
        ParentServerConfigGeneration: 1
        ParentServerMPMGeneration: 0
        ServerUptimeSeconds: 113
        ServerUptime: 1 minute 53 seconds
        Load1: 1.00
        Load5: 1.19
        Load15: 0.55
        Total Accesses: 2
        Total kBytes: 1
        Total Duration: 0
        CPUUser: .05
        CPUSystem: .09
        CPUChildrenUser: 0
        CPUChildrenSystem: 0
        CPULoad: .123894
        Uptime: 113
        ReqPerSec: .0176991
        BytesPerSec: 9.06195
        BytesPerReq: 512
        DurationPerReq: 0
        BusyWorkers: 1
        IdleWorkers: 74
        Processes: 3
        Stopping: 0
        BusyWorkers: 1
        IdleWorkers: 74
        ConnsTotal: 0
        ConnsAsyncWriting: 0
        ConnsAsyncKeepAlive: 0
        ConnsAsyncClosing: 0
        Scoreboard: __________________________W________________________________________________.....................................................................................................................................................................................................................................................................................................................................

sles-15:

        127.0.0.1
        ServerVersion: Apache/2.4.51 (Linux/SUSE) OpenSSL/1.1.1l
        ServerMPM: prefork
        Server Built: 2023-01-20 09:29:44.000000000 +0000
        CurrentTime: Wednesday, 22-Mar-2023 20:18:24 UTC
        RestartTime: Wednesday, 22-Mar-2023 20:16:45 UTC
        ParentServerConfigGeneration: 1
        ParentServerMPMGeneration: 0
        ServerUptimeSeconds: 98
        ServerUptime: 1 minute 38 seconds
        Load1: 0.23
        Load5: 0.28
        Load15: 0.13
        Total Accesses: 2
        Total kBytes: 0
        Total Duration: 3
        CPUUser: .03
        CPUSystem: .01
        CPUChildrenUser: 0
        CPUChildrenSystem: 0
        CPULoad: .0408163
        Uptime: 98
        ReqPerSec: .0204082
        BytesPerSec: 0
        BytesPerReq: 0
        DurationPerReq: 1.5
        BusyWorkers: 2
        IdleWorkers: 4
        Scoreboard: K___W_................................................................................................................................................

My question is, should we disable testing for the current_connections metric on all distros or introduce a mechanism to disable specific metrics on specific distros?

[jefferbrecht]

Would introducing the mechanism be a large effort? I'm not keen on removing existing coverage, especially for a really important metric like current_connections.

I'm not super familiar with apache; is the event MPM some kind of module that the user can install? In other words, is it still possible to get this metric on the currently-missing platforms if the user performs an extra step? If it's a common thing that people do then we might also just want to add the MPM thing to the test setup for those platforms.

[martijnvans]

It's pretty easy to add a mechanism to skip certain platforms for specific metrics. I'd just add a platforms_to_skip field to expected_metrics in metadata.yaml and look at that field in the go code at the right moment. Should be doable.

I looked into enabling event mode for all platforms, but I think it's a compile-time option, not an option that we can configure in our config files: https://httpd.apache.org/docs/2.4/mpm.html

[ridwanmsharif]

+1 for introducing a mechanism to disable specific metrics on specific distros. Its possible this is something we encounter with other applications too and would be nice to have a mechanism to test metrics against what we expect for that distro.