Skip to content

Commit

Permalink
Beautify benchmark commit table
Browse files Browse the repository at this point in the history
  • Loading branch information
bentaculum committed Oct 12, 2023
1 parent 5e8f823 commit 110a668
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion .github/workflows/benchmark-pr.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,14 @@ def make_report(old_path, new_path, out_file):
new = load_stats(new_path)

# Merge on benchmark name
df = new[-1].merge(old[-1], on="Benchmark", suffixes=("_new", "_old"))
df = old[-1].merge(new[-1], on="Benchmark", suffixes=("_old", "_new"))

df["Percent Change"] = 100 * (df["mean_new"] - df["mean_old"]) / df["mean_old"]
df["Percent Change"] = df["Percent Change"].map("{:.2f}".format)

# Format runtimes
df["mean_old"] = df["mean_old"].map("{:.5f}".format)
df["mean_new"] = df["mean_new"].map("{:.5f}".format)

# Change column names to commit ids
df = df.rename(
Expand Down

1 comment on commit 110a668

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Mean (s) BASE 82fc051 Mean (s) HEAD 110a668 Percent Change
test_load_gt_data 1.69531 2.4573 44.95
test_load_pred_data 1.40477 2.01966 43.77
test_ctc_matched 5.67772 6.61579 16.52
test_ctc_metrics 157.785 205.064 29.96
test_ctc_div_metrics 0.4804 0.59343 23.53
test_iou_matched 16.2192 21.5324 32.76
test_iou_div_metrics 0.49306 0.5787 17.37

Please sign in to comment.