[Reporting] Add output size stats to telemetry metrics #112037
Conversation
tsullivan
added
v8.0.0
Team:AppServices
release_note:skip
|
Pinging @elastic/kibana-reporting-services (Team:Reporting Services) |
|
Pinging @elastic/kibana-app-services (Team:AppServices) |
| @@ -161,7 +175,7 @@ export async function getReportingUsage( | |||
| jobTypes: { | |||
| terms: { field: JOB_TYPES_FIELD, size: DEFAULT_TERMS_SIZE }, | |||
| aggs: { | |||
| appNames: { terms: { field: OBJECT_TYPES_FIELD, size: DEFAULT_TERMS_SIZE } }, // NOTE Discover/CSV export is missing the 'meta.objectType' field, so Discover/CSV results are missing for this agg | |||
There was a problem hiding this comment.
This comment is no longer valid as meta.objectType is added for all export types in the RequestHandler class.
| output_size: { | ||
| max: sizeMax?.value ?? null, | ||
| min: sizeMin?.value ?? null, | ||
| avg: sizeAvg?.value ?? null, | ||
| }, |
There was a problem hiding this comment.
I wonder whether a percentile based aggregation would not provide better insight into the questions we may have regarding output size (es docs).
Would it be more useful to know that the average output size is, e.g., 500kb or that 99% of outputs are below 700kb?
Percentile based aggregation can be more expensive to calcuate, which might disqualify it, but from what I can tell it looks pretty safe (also considering we are working with report documents so not sure there will be millions).
Something like:
output_size: {
25th: ...,
50th: ...,
75th: ...,
95th: ...,
99th: ..., // outliers
}Let me know what you think @tsullivan !
There was a problem hiding this comment.
++ I am in favor of this. At the time I implemented it with avg, I was hoping to use median instead, but there isn't an actual median aggregation. With percentiles, we get the median from the 50th percentile. This is good!
|
@elasticmachine merge upstream |
tsullivan
force-pushed
the
reporting/telemetry-add-reports-stats
branch
from
3895259 to
9d6b579
Compare
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
* [Reporting] Add output size stats to telemetry metrics * fix types * add output_size for each jobtype * add size metrics for each job type * use more mock data in unit tests * clean up test * update test snapshots * update telemetry mapping * SizeMetrics => SizePercentiles * DocCount interface * fix tests * Update get_export_stats.ts * update snapshots
…3057) * [Reporting] Add output size stats to telemetry metrics * fix types * add output_size for each jobtype * add size metrics for each job type * use more mock data in unit tests * clean up test * update test snapshots * update telemetry mapping * SizeMetrics => SizePercentiles * DocCount interface * fix tests * Update get_export_stats.ts * update snapshots
sophiec20
added
the
(Deprecated) Feature:Reporting
Summary
This PR adds size statistics on the reports in the cluster.