Tracing of setup and teardown actions in JUnit 4 #8030

daniel-mohedano · 2024-11-28T10:25:19Z

What Does This Do

Implements tracing of setup and teardown actions available in JUnit 4 (@Before, @After, @BeforeClass, @AfterClass, @BeforeParam, @AfterParam).

Motivation

To provide more complete information regarding test runs, the setup and teardown actions can be traced. This is already implemented for JUnit 5, which implies that it would also be nice to have for JUnit 4.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Squash your commits prior merging or merge using GitHub's Squash and merge
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-1224

pr-commenter · 2024-11-28T11:02:24Z

Benchmarks

Startup

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-11-29T11:41:13	2024-11-29T11:48:14
git_branch	master	daniel.mohedano/SDTEST-1224-setup-teardown-tracing-junit4
git_commit_date	1732876820	1732879953
git_commit_sha	`9f01834`	`948fe8a`
release_version	1.44.0-SNAPSHOT~9f01834ab7	1.44.0-SNAPSHOT~948fe8ac30
start_time	2024-11-29T11:40:59	2024-11-29T11:48:01

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1732881248	1732881248
ci_job_id	723438389	723438389
ci_pipeline_id	50141202	50141202
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.357 ms) : 1337, 1376
.   : milestone, 1357,
appsec (1.778 ms) : 1755, 1801
.   : milestone, 1778,
appsec_no_iast (1.767 ms) : 1742, 1792
.   : milestone, 1767,
iast (1.504 ms) : 1481, 1527
.   : milestone, 1504,
profiling (1.561 ms) : 1537, 1585
.   : milestone, 1561,
tracing (1.512 ms) : 1487, 1536
.   : milestone, 1512,
section candidate
no_agent (1.353 ms) : 1334, 1372
.   : milestone, 1353,
appsec (1.764 ms) : 1740, 1789
.   : milestone, 1764,
appsec_no_iast (1.77 ms) : 1745, 1794
.   : milestone, 1770,
iast (1.491 ms) : 1468, 1514
.   : milestone, 1491,
profiling (1.514 ms) : 1490, 1538
.   : milestone, 1514,
tracing (1.519 ms) : 1494, 1544
.   : milestone, 1519,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.357 ms [1.337 ms, 1.376 ms]	-
appsec	1.778 ms [1.755 ms, 1.801 ms]	421.323 µs (31.1%)
appsec_no_iast	1.767 ms [1.742 ms, 1.792 ms]	410.053 µs (30.2%)
iast	1.504 ms [1.481 ms, 1.527 ms]	147.444 µs (10.9%)
profiling	1.561 ms [1.537 ms, 1.585 ms]	204.183 µs (15.0%)
tracing	1.512 ms [1.487 ms, 1.536 ms]	154.958 µs (11.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.353 ms [1.334 ms, 1.372 ms]	-
appsec	1.764 ms [1.74 ms, 1.789 ms]	411.549 µs (30.4%)
appsec_no_iast	1.77 ms [1.745 ms, 1.794 ms]	416.781 µs (30.8%)
iast	1.491 ms [1.468 ms, 1.514 ms]	138.115 µs (10.2%)
profiling	1.514 ms [1.49 ms, 1.538 ms]	161.456 µs (11.9%)
tracing	1.519 ms [1.494 ms, 1.544 ms]	166.149 µs (12.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.368 µs) : 359, 398
.   : milestone, 378,
iast (492.647 µs) : 471, 514
.   : milestone, 493,
iast_FULL (655.985 µs) : 634, 678
.   : milestone, 656,
iast_GLOBAL (519.947 µs) : 499, 541
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (495.839 µs) : 475, 517
.   : milestone, 496,
iast_INACTIVE (451.754 µs) : 431, 472
.   : milestone, 452,
iast_TELEMETRY_OFF (489.516 µs) : 468, 511
.   : milestone, 490,
tracing (448.227 µs) : 428, 469
.   : milestone, 448,
section candidate
no_agent (380.47 µs) : 361, 400
.   : milestone, 380,
iast (494.585 µs) : 473, 516
.   : milestone, 495,
iast_FULL (660.218 µs) : 639, 682
.   : milestone, 660,
iast_GLOBAL (520.383 µs) : 499, 542
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (497.827 µs) : 476, 519
.   : milestone, 498,
iast_INACTIVE (453.714 µs) : 433, 474
.   : milestone, 454,
iast_TELEMETRY_OFF (479.762 µs) : 459, 501
.   : milestone, 480,
tracing (454.786 µs) : 434, 476
.   : milestone, 455,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	378.368 µs [358.682 µs, 398.053 µs]	-
iast	492.647 µs [471.36 µs, 513.934 µs]	114.279 µs (30.2%)
iast_FULL	655.985 µs [634.372 µs, 677.598 µs]	277.617 µs (73.4%)
iast_GLOBAL	519.947 µs [498.778 µs, 541.116 µs]	141.579 µs (37.4%)
iast_HARDCODED_SECRET_DISABLED	495.839 µs [474.587 µs, 517.09 µs]	117.471 µs (31.0%)
iast_INACTIVE	451.754 µs [431.219 µs, 472.29 µs]	73.387 µs (19.4%)
iast_TELEMETRY_OFF	489.516 µs [467.667 µs, 511.365 µs]	111.148 µs (29.4%)
tracing	448.227 µs [427.775 µs, 468.678 µs]	69.859 µs (18.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	380.47 µs [360.716 µs, 400.224 µs]	-
iast	494.585 µs [472.997 µs, 516.173 µs]	114.115 µs (30.0%)
iast_FULL	660.218 µs [638.762 µs, 681.674 µs]	279.748 µs (73.5%)
iast_GLOBAL	520.383 µs [498.97 µs, 541.795 µs]	139.913 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED	497.827 µs [476.233 µs, 519.422 µs]	117.358 µs (30.8%)
iast_INACTIVE	453.714 µs [432.983 µs, 474.444 µs]	73.244 µs (19.3%)
iast_TELEMETRY_OFF	479.762 µs [458.846 µs, 500.678 µs]	99.292 µs (26.1%)
tracing	454.786 µs [433.773 µs, 475.798 µs]	74.316 µs (19.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/SDTEST-1224-setup-teardown-tracing-junit4
git_commit_date	1732876820	1732879953
git_commit_sha	`9f01834`	`948fe8a`
release_version	1.44.0-SNAPSHOT~9f01834ab7	1.44.0-SNAPSHOT~948fe8ac30

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1732881900	1732881900
ci_job_id	723438390	723438390
ci_pipeline_id	50141202	50141202
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.398 s) : 15398000, 15398000
.   : milestone, 15398000,
appsec (15.0 s) : 15000000, 15000000
.   : milestone, 15000000,
iast (18.99 s) : 18990000, 18990000
.   : milestone, 18990000,
iast_GLOBAL (18.209 s) : 18209000, 18209000
.   : milestone, 18209000,
profiling (14.869 s) : 14869000, 14869000
.   : milestone, 14869000,
tracing (15.048 s) : 15048000, 15048000
.   : milestone, 15048000,
section candidate
no_agent (15.41 s) : 15410000, 15410000
.   : milestone, 15410000,
appsec (14.946 s) : 14946000, 14946000
.   : milestone, 14946000,
iast (18.36 s) : 18360000, 18360000
.   : milestone, 18360000,
iast_GLOBAL (17.692 s) : 17692000, 17692000
.   : milestone, 17692000,
profiling (15.258 s) : 15258000, 15258000
.   : milestone, 15258000,
tracing (14.933 s) : 14933000, 14933000
.   : milestone, 14933000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.398 s [15.398 s, 15.398 s]	-
appsec	15.0 s [15.0 s, 15.0 s]	-398.0 ms (-2.6%)
iast	18.99 s [18.99 s, 18.99 s]	3.592 s (23.3%)
iast_GLOBAL	18.209 s [18.209 s, 18.209 s]	2.811 s (18.3%)
profiling	14.869 s [14.869 s, 14.869 s]	-529.0 ms (-3.4%)
tracing	15.048 s [15.048 s, 15.048 s]	-350.0 ms (-2.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.41 s [15.41 s, 15.41 s]	-
appsec	14.946 s [14.946 s, 14.946 s]	-464.0 ms (-3.0%)
iast	18.36 s [18.36 s, 18.36 s]	2.95 s (19.1%)
iast_GLOBAL	17.692 s [17.692 s, 17.692 s]	2.282 s (14.8%)
profiling	15.258 s [15.258 s, 15.258 s]	-152.0 ms (-1.0%)
tracing	14.933 s [14.933 s, 14.933 s]	-477.0 ms (-3.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.339 ms) : 2297, 2380
.   : milestone, 2339,
iast (2.078 ms) : 2025, 2130
.   : milestone, 2078,
iast_GLOBAL (2.118 ms) : 2065, 2171
.   : milestone, 2118,
profiling (1.951 ms) : 1909, 1993
.   : milestone, 1951,
tracing (1.919 ms) : 1879, 1959
.   : milestone, 1919,
section candidate
no_agent (1.465 ms) : 1453, 1476
.   : milestone, 1465,
appsec (2.333 ms) : 2291, 2375
.   : milestone, 2333,
iast (2.076 ms) : 2023, 2128
.   : milestone, 2076,
iast_GLOBAL (2.118 ms) : 2065, 2170
.   : milestone, 2118,
profiling (1.954 ms) : 1911, 1998
.   : milestone, 1954,
tracing (1.917 ms) : 1878, 1957
.   : milestone, 1917,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.467 ms [1.455 ms, 1.478 ms]	-
appsec	2.339 ms [2.297 ms, 2.38 ms]	871.869 µs (59.4%)
iast	2.078 ms [2.025 ms, 2.13 ms]	610.9 µs (41.7%)
iast_GLOBAL	2.118 ms [2.065 ms, 2.171 ms]	651.145 µs (44.4%)
profiling	1.951 ms [1.909 ms, 1.993 ms]	484.71 µs (33.0%)
tracing	1.919 ms [1.879 ms, 1.959 ms]	452.412 µs (30.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.465 ms [1.453 ms, 1.476 ms]	-
appsec	2.333 ms [2.291 ms, 2.375 ms]	868.645 µs (59.3%)
iast	2.076 ms [2.023 ms, 2.128 ms]	611.137 µs (41.7%)
iast_GLOBAL	2.118 ms [2.065 ms, 2.17 ms]	653.324 µs (44.6%)
profiling	1.954 ms [1.911 ms, 1.998 ms]	489.733 µs (33.4%)
tracing	1.917 ms [1.878 ms, 1.957 ms]	452.758 µs (30.9%)

…-tracing-junit4

daniel-mohedano added 7 commits November 26, 2024 12:32

add tracing for junit4 setup/teardown actions

407e5c7

update instrumentation test fixtures

43ffd7e

add new instrumentation for junit4.13

a52a400

Revert test fixture changes

4b1a69e

generate test fixtures for junit 4.13 instrumentation

b4f036b

junit4.13 muzzle check

5497d99

remove inverse muzzle assert

e8ae392

daniel-mohedano added 3 commits November 28, 2024 13:44

additional tests for failing setup and teardown

0df9f72

Merge branch 'master' into daniel.mohedano/SDTEST-1224-setup-teardown…

b1886ed

…-tracing-junit4

add junit 4.13 instrumentation to instrumentation testing config

5004300

daniel-mohedano added the comp: ci visibility Continuous Integration Visibility label Nov 29, 2024

daniel-mohedano added 2 commits November 29, 2024 10:40

Merge branch 'master' into daniel.mohedano/SDTEST-1224-setup-teardown…

f35cbd4

…-tracing-junit4

add support for BeforeParam and AfterParam

948fe8a

daniel-mohedano requested a review from nikita-tkachenko-datadog November 29, 2024 17:16

daniel-mohedano marked this pull request as ready for review December 2, 2024 10:25

daniel-mohedano requested review from a team as code owners December 2, 2024 10:25

daniel-mohedano requested a review from nayeem-kamal December 2, 2024 10:25

nikita-tkachenko-datadog approved these changes Dec 2, 2024

View reviewed changes

daniel-mohedano merged commit cdee1c8 into master Dec 2, 2024
103 checks passed

daniel-mohedano deleted the daniel.mohedano/SDTEST-1224-setup-teardown-tracing-junit4 branch December 2, 2024 11:04

github-actions bot added this to the 1.44.0 milestone Dec 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracing of setup and teardown actions in JUnit 4 #8030

Tracing of setup and teardown actions in JUnit 4 #8030

daniel-mohedano commented Nov 28, 2024 •

edited

Loading

pr-commenter bot commented Nov 28, 2024 •

edited

Loading

Tracing of setup and teardown actions in JUnit 4 #8030

Tracing of setup and teardown actions in JUnit 4 #8030

Conversation

daniel-mohedano commented Nov 28, 2024 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Nov 28, 2024 • edited Loading

Benchmarks

Startup

Load

Parameters

Summary

Dacapo

Parameters

Summary

daniel-mohedano commented Nov 28, 2024 •

edited

Loading

pr-commenter bot commented Nov 28, 2024 •

edited

Loading