refactor: include metric output_batches into BaselineMetrics

[nmbr7]

Which issue does this PR close?

• Closes Include metric output_batches into BaselineMetrics #17027

Rationale for this change

output_batches should be a common metric in all operators, thus should ideally be added to BaselineMetrics

> explain analyze select * from generate_series(1, 1000000) as t1(v1) order by v1 desc;
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type         | plan                                                                                                                                                                                                                                 |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Plan with Metrics | SortExec: expr=[v1@0 DESC], preserve_partitioning=[false], metrics=[output_rows=1000000, elapsed_compute=535.320324ms, output_bytes=7.6 MB, output_batches=123, spill_count=0, spilled_bytes=0.0 B, spilled_rows=0, batches_split=0] |
|                   |   ProjectionExec: expr=[value@0 as v1], metrics=[output_rows=1000000, elapsed_compute=208.379µs, output_bytes=7.7 MB, output_batches=123]                                                                                            |
|                   |     LazyMemoryExec: partitions=1, batch_generators=[generate_series: start=1, end=1000000, batch_size=8192], metrics=[output_rows=1000000, elapsed_compute=15.924291ms, output_bytes=7.7 MB, output_batches=123]                     |
|                   |                                                                                                                                                                                                                                      |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.492 second

What changes are included in this PR?

• Added output_batches into BaselineMetrics with DEV MetricType
• Tracked through record_poll() API
• Changes are similar to feat: Add output_bytes to baseline metrics #18268
• Refactored assert_metrics macro to take multiple metrics strings for substring check
• Added output_bytes and output_batches tracking in TopK operator
• Added baseline metrics for RepartitionExec

Are these changes tested?

Added UT

Are there any user-facing changes?

Changes in the EXPLAIN ANALYZE output, output_batches will be added to metrics=[...]

[nmbr7]

Hey @2010YOUY01, please do review once you get time.

[2010YOUY01]

Thank you, this PR is well written.

This is good to go once the minor suggestions are addressed and tests pass.

[nmbr7]

@2010YOUY01, have done the requested changes, do check.

[nmbr7]

It seems, with the force_hash_collisions feature, the RepartitionExec is returning 1 output batch, will update the UT to take the feature flag into account.

https://github.com/apache/datafusion/actions/runs/19165362013/job/54785692452?pr=18491

[nmbr7]

@2010YOUY01, can you give it another go, updated the test and added baseline metrics in RepartitionExec

[nmbr7]

@2010YOUY01, it seems that #18262 will have proper baseline metrics for the RepartitionExec with the correct elapsed_compute calculation.

Should we remove the basic RepartitionExec BaselineMetrics I added here (from which we currently use the output_batches), or are we okay with having it, since once #18262 is merged, it should be fixed?

[2010YOUY01]

LGTM, thanks again!

Should we remove the basic RepartitionExec BaselineMetrics I added here (from which we currently use the output_batches), or are we okay with having it, since once #18262 is merged, it should be fixed?

It's okay to keep it, we just merged a big change in RepartitionExec, so #18262 has to be reworked, unfortunately...