Skip to content

Commit

Permalink
docs: improving explanations on host level vs process level metriucs
Browse files Browse the repository at this point in the history
  • Loading branch information
bpetit committed Feb 12, 2024
1 parent abd1ef0 commit b3b027e
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Please check dev branch.

### Changed

- Global power metrics have changed and could give higher numbers than previously. Please have a look at the [documentation](https://hubblo-org.github.io/scaphandre-documentation/explanations/host_metrics.html).
- `scaph_self_mem_total_program_size`, `scaph_self_mem_resident_set_size` and `scaph_self_mem_shared_resident_size` are replaced by `scaph_self_memory_bytes` and `scaph_self_memory_virtual_bytes`, see https://github.com/hubblo-org/scaphandre/pull/274/files
- Refactored warp10 exporter, see https://github.com/hubblo-org/scaphandre/pull/291 and https://github.com/hubblo-org/scaphandre/issues/105
- Refactored exporters creation with clap4, see https://github.com/hubblo-org/scaphandre/pull/292, thanks @TheElectronWill
Expand Down
1 change: 1 addition & 0 deletions docs_src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

# Explanations

- [Explanations about host level power and energy metrics](explanations/host_metrics.md)
- [How scaphandre computes per process power consumption](explanations/how-scaph-computes-per-process-power-consumption.md)
- [Internal structure](explanations/internal-structure.md)
- [About containers](explanations/about-containers.md)
Expand Down
22 changes: 22 additions & 0 deletions docs_src/explanations/host_metrics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Explanations about host level power and energy metrics.

This is true starting **from Scaphandre >= 1.0.**

There are several [metrics](../references/metrics.md) available at the host level in Scaphandre:
- `scaph_host_power_microwatts` : always returned, computed from Record structs made from `scaph_host_energy_microjoules` metric
- `scaph_host_energy_microjoules` : always returned, either one value or a sum of values coming directly from RAPL counters (`energy_uj` files or direct read from an MSR)
- `scaph_host_rapl_psys_microjoules` : is available only when the PSYS [RAPL domain](explanations/rapl-domains.md) is available on the machine.

In addition to those metrics, you might want to build, on your time series database, the sum of process_ metrics to have a view of the weight of all processes on the host power. Using Prometheus, it would look like: `sum(scaph_process_power_consumption_microwatts{hostname="$hostname"}) / 1000000`, to get it in Watts.

Let's explain the relationship between those metrics, and what you could expect.

`host_power` metric will return :
1. If PSYS domain is available, a computed power coming from PSYS energy records
2. If not, a computed power which is the sum of per-socket power (PKG RAPL domain) + DRAM RAPL domain power

Briefly explained (see [RAPL domains](explanations/rapl-domains.md) for detailled explanations), PSYS covers most components on the machine ("all components connected to the SoC / motherboard" according to most documentations), so we return this wider ranged metric when available. If not we use a combination of PKG domain, that includes CPU and integrated GPU power, and DRAM domain, that includes memory power. The first options gives higher figures than the second, for now.

Suming the power of all processes, if the machine is mostly IDLE, you'll get a tiny percentage of the host machine, most likely. The difference between host power and the sum of processes power can be accounted as "power due to IDLE activity", in other words the power your machine demands for "doing nothing". The higher this difference on a long period of time (better seen as a graph), the higher chance that there is room for improvement in moving the workloads to another machine and shut the current machine down (and make it available for another project or to another organization to prevent from buying a new machine).

**Warning:** that being said, the way per-process power is computed is still biased and shall be improved in the following versions of Scaphandre. For now, the main key for allocation is CPU time. As host level power metrics include power usage of more and more components on the machine (work in progress) this allocation key will be more and more inaccurate. Future versions of this allocation model should include keys regarding the activity of other components than CPU. Enabling a better set of allocation keys for per-process power is part of the [roadmap](https://github.com/hubblo-org/scaphandre/projects/1).

0 comments on commit b3b027e

Please sign in to comment.