Do not emit execution id label by default in single binary

[eapolinario]

Tracking issue

Closes #3991

Why are the changes needed?

Single binary emits execution-id labels to a bunch of flytepropeller and flyteadmin metrics, but since the golang prometheus golang client is setup in such a way that no metric is ever dropped (as described in prometheus/client_golang#920 (comment)), this ends up causing an apparent memory leak due to how these high cardinality metrics are stored in-memory.

What changes were proposed in this pull request?

This PR removes the execution-id label from the single binary metrics and adds a comment to hint at the potential problems caused by that setting.

I confirmed that the metric is not enabled by default in any component, nor is set in any of the shipped helm charts, while being left as a last resort debugging tool.

One could argue that we should never have had that label in the first place and instead reach out to other debugging means, like open telemetry, etc, but that's a separate discussion.

How was this patch tested?

I had a local test where I'd have a workflow composed of a single cached task. Running that in a tight loop and confirming that the increase in memory of the single binary executable is now orders of magnitude slower than before the PR.

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.17%. Comparing base (5c147ef) to head (cc714d3).
Report is 155 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5704   +/-   ##
=======================================
  Coverage   36.17%   36.17%           
=======================================
  Files        1302     1302           
  Lines      109614   109614           
=======================================
+ Hits        39653    39658    +5     
+ Misses      65816    65810    -6     
- Partials     4145     4146    +1     

Flag	Coverage Δ
unittests-datacatalog	51.37% <ø> (ø)
unittests-flyteadmin	55.33% <ø> (+0.03%)	⬆️
unittests-flytecopilot	12.17% <ø> (ø)
unittests-flytectl	62.22% <ø> (+0.04%)	⬆️
unittests-flyteidl	7.12% <ø> (ø)
unittests-flyteplugins	53.34% <ø> (ø)
unittests-flytepropeller	41.70% <ø> (-0.02%)	⬇️
unittests-flytestdlib	55.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[davidmirror-ops]

Thank you! This is great as it simplifies a bit some of the queries in the Grafana user dashboard (example)