Improve IAST sampling defaults #7944

smola · 2024-11-13T09:25:07Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Squash your commits prior merging or merge using GitHub's Squash and merge
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2024-11-13T09:57:05Z

Benchmarks

Startup

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-11-13T09:34:58	2024-11-13T09:41:54
git_branch	master	smola/improve-iast-sampling-defaults
git_commit_date	1731489224	1731489879
git_commit_sha	`59388f1`	`8b8ca65`
release_version	1.43.0-SNAPSHOT~59388f19c2	1.43.0-SNAPSHOT~8b8ca65362
start_time	2024-11-13T09:34:44	2024-11-13T09:41:41

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1731491263	1731491263
ci_job_id	704941252	704941252
ci_pipeline_id	48880381	48880381
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~8b8ca65362, baseline=1.43.0-SNAPSHOT~59388f19c2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.746 ms) : 1723, 1769
.   : milestone, 1746,
appsec_no_iast (1.734 ms) : 1711, 1758
.   : milestone, 1734,
iast (1.487 ms) : 1465, 1510
.   : milestone, 1487,
profiling (1.521 ms) : 1497, 1546
.   : milestone, 1521,
tracing (1.476 ms) : 1452, 1500
.   : milestone, 1476,
section candidate
no_agent (1.349 ms) : 1330, 1368
.   : milestone, 1349,
appsec (1.711 ms) : 1687, 1734
.   : milestone, 1711,
appsec_no_iast (1.743 ms) : 1719, 1766
.   : milestone, 1743,
iast (1.475 ms) : 1452, 1497
.   : milestone, 1475,
profiling (1.479 ms) : 1457, 1502
.   : milestone, 1479,
tracing (1.483 ms) : 1459, 1507
.   : milestone, 1483,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.322 ms, 1.36 ms]	-
appsec	1.746 ms [1.723 ms, 1.769 ms]	404.91 µs (30.2%)
appsec_no_iast	1.734 ms [1.711 ms, 1.758 ms]	393.558 µs (29.4%)
iast	1.487 ms [1.465 ms, 1.51 ms]	146.326 µs (10.9%)
profiling	1.521 ms [1.497 ms, 1.546 ms]	180.532 µs (13.5%)
tracing	1.476 ms [1.452 ms, 1.5 ms]	135.542 µs (10.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.349 ms [1.33 ms, 1.368 ms]	-
appsec	1.711 ms [1.687 ms, 1.734 ms]	361.431 µs (26.8%)
appsec_no_iast	1.743 ms [1.719 ms, 1.766 ms]	393.386 µs (29.2%)
iast	1.475 ms [1.452 ms, 1.497 ms]	125.569 µs (9.3%)
profiling	1.479 ms [1.457 ms, 1.502 ms]	130.07 µs (9.6%)
tracing	1.483 ms [1.459 ms, 1.507 ms]	134.203 µs (9.9%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~8b8ca65362, baseline=1.43.0-SNAPSHOT~59388f19c2
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.598 µs) : 356, 395
.   : milestone, 376,
iast (489.769 µs) : 469, 511
.   : milestone, 490,
iast_FULL (644.27 µs) : 623, 666
.   : milestone, 644,
iast_GLOBAL (512.361 µs) : 491, 534
.   : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (484.366 µs) : 463, 505
.   : milestone, 484,
iast_INACTIVE (449.168 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (480.443 µs) : 459, 502
.   : milestone, 480,
tracing (445.11 µs) : 422, 468
.   : milestone, 445,
section candidate
no_agent (371.906 µs) : 352, 392
.   : milestone, 372,
iast (486.224 µs) : 464, 508
.   : milestone, 486,
iast_FULL (647.885 µs) : 626, 669
.   : milestone, 648,
iast_GLOBAL (501.181 µs) : 479, 523
.   : milestone, 501,
iast_HARDCODED_SECRET_DISABLED (486.497 µs) : 465, 508
.   : milestone, 486,
iast_INACTIVE (455.805 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (475.071 µs) : 454, 496
.   : milestone, 475,
tracing (442.488 µs) : 422, 463
.   : milestone, 442,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	375.598 µs [356.272 µs, 394.924 µs]	-
iast	489.769 µs [468.773 µs, 510.765 µs]	114.171 µs (30.4%)
iast_FULL	644.27 µs [622.984 µs, 665.556 µs]	268.672 µs (71.5%)
iast_GLOBAL	512.361 µs [490.739 µs, 533.984 µs]	136.763 µs (36.4%)
iast_HARDCODED_SECRET_DISABLED	484.366 µs [463.467 µs, 505.266 µs]	108.768 µs (29.0%)
iast_INACTIVE	449.168 µs [427.93 µs, 470.406 µs]	73.57 µs (19.6%)
iast_TELEMETRY_OFF	480.443 µs [458.909 µs, 501.977 µs]	104.844 µs (27.9%)
tracing	445.11 µs [422.437 µs, 467.784 µs]	69.512 µs (18.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	371.906 µs [352.207 µs, 391.606 µs]	-
iast	486.224 µs [464.5 µs, 507.948 µs]	114.317 µs (30.7%)
iast_FULL	647.885 µs [626.471 µs, 669.3 µs]	275.979 µs (74.2%)
iast_GLOBAL	501.181 µs [478.964 µs, 523.397 µs]	129.274 µs (34.8%)
iast_HARDCODED_SECRET_DISABLED	486.497 µs [464.955 µs, 508.039 µs]	114.59 µs (30.8%)
iast_INACTIVE	455.805 µs [434.697 µs, 476.913 µs]	83.898 µs (22.6%)
iast_TELEMETRY_OFF	475.071 µs [453.687 µs, 496.455 µs]	103.165 µs (27.7%)
tracing	442.488 µs [421.819 µs, 463.157 µs]	70.582 µs (19.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/improve-iast-sampling-defaults
git_commit_date	1731489224	1731489879
git_commit_sha	`59388f1`	`8b8ca65`
release_version	1.43.0-SNAPSHOT~59388f19c2	1.43.0-SNAPSHOT~8b8ca65362

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1731491789	1731491789
ci_job_id	704941253	704941253
ci_pipeline_id	48880381	48880381
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~8b8ca65362, baseline=1.43.0-SNAPSHOT~59388f19c2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.461 ms) : 1450, 1473
.   : milestone, 1461,
appsec (2.301 ms) : 2260, 2342
.   : milestone, 2301,
iast (2.062 ms) : 2010, 2113
.   : milestone, 2062,
iast_GLOBAL (2.109 ms) : 2057, 2162
.   : milestone, 2109,
profiling (1.931 ms) : 1889, 1972
.   : milestone, 1931,
tracing (1.911 ms) : 1871, 1951
.   : milestone, 1911,
section candidate
no_agent (1.462 ms) : 1451, 1474
.   : milestone, 1462,
appsec (2.312 ms) : 2271, 2352
.   : milestone, 2312,
iast (2.067 ms) : 2015, 2118
.   : milestone, 2067,
iast_GLOBAL (2.095 ms) : 2042, 2147
.   : milestone, 2095,
profiling (1.93 ms) : 1888, 1972
.   : milestone, 1930,
tracing (1.903 ms) : 1864, 1943
.   : milestone, 1903,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.461 ms [1.45 ms, 1.473 ms]	-
appsec	2.301 ms [2.26 ms, 2.342 ms]	839.749 µs (57.5%)
iast	2.062 ms [2.01 ms, 2.113 ms]	600.444 µs (41.1%)
iast_GLOBAL	2.109 ms [2.057 ms, 2.162 ms]	648.053 µs (44.4%)
profiling	1.931 ms [1.889 ms, 1.972 ms]	469.701 µs (32.1%)
tracing	1.911 ms [1.871 ms, 1.951 ms]	449.817 µs (30.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.462 ms [1.451 ms, 1.474 ms]	-
appsec	2.312 ms [2.271 ms, 2.352 ms]	849.356 µs (58.1%)
iast	2.067 ms [2.015 ms, 2.118 ms]	604.188 µs (41.3%)
iast_GLOBAL	2.095 ms [2.042 ms, 2.147 ms]	632.068 µs (43.2%)
profiling	1.93 ms [1.888 ms, 1.972 ms]	467.255 µs (32.0%)
tracing	1.903 ms [1.864 ms, 1.943 ms]	440.945 µs (30.2%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~8b8ca65362, baseline=1.43.0-SNAPSHOT~59388f19c2
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.472 s) : 15472000, 15472000
.   : milestone, 15472000,
appsec (15.252 s) : 15252000, 15252000
.   : milestone, 15252000,
iast (18.802 s) : 18802000, 18802000
.   : milestone, 18802000,
iast_GLOBAL (18.152 s) : 18152000, 18152000
.   : milestone, 18152000,
profiling (14.999 s) : 14999000, 14999000
.   : milestone, 14999000,
tracing (15.087 s) : 15087000, 15087000
.   : milestone, 15087000,
section candidate
no_agent (15.088 s) : 15088000, 15088000
.   : milestone, 15088000,
appsec (15.472 s) : 15472000, 15472000
.   : milestone, 15472000,
iast (18.826 s) : 18826000, 18826000
.   : milestone, 18826000,
iast_GLOBAL (17.902 s) : 17902000, 17902000
.   : milestone, 17902000,
profiling (15.423 s) : 15423000, 15423000
.   : milestone, 15423000,
tracing (15.107 s) : 15107000, 15107000
.   : milestone, 15107000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.472 s [15.472 s, 15.472 s]	-
appsec	15.252 s [15.252 s, 15.252 s]	-220.0 ms (-1.4%)
iast	18.802 s [18.802 s, 18.802 s]	3.33 s (21.5%)
iast_GLOBAL	18.152 s [18.152 s, 18.152 s]	2.68 s (17.3%)
profiling	14.999 s [14.999 s, 14.999 s]	-473.0 ms (-3.1%)
tracing	15.087 s [15.087 s, 15.087 s]	-385.0 ms (-2.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.088 s [15.088 s, 15.088 s]	-
appsec	15.472 s [15.472 s, 15.472 s]	384.0 ms (2.5%)
iast	18.826 s [18.826 s, 18.826 s]	3.738 s (24.8%)
iast_GLOBAL	17.902 s [17.902 s, 17.902 s]	2.814 s (18.7%)
profiling	15.423 s [15.423 s, 15.423 s]	335.0 ms (2.2%)
tracing	15.107 s [15.107 s, 15.107 s]	19.0 ms (0.1%)

Improve IAST sampling defaults

8b8ca65

smola added the comp: asm iast Application Security Management (IAST) label Nov 13, 2024

Merge branch 'master' into smola/improve-iast-sampling-defaults

0f4bf61

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve IAST sampling defaults #7944

Improve IAST sampling defaults #7944

smola commented Nov 13, 2024

pr-commenter bot commented Nov 13, 2024

Improve IAST sampling defaults #7944

Are you sure you want to change the base?

Improve IAST sampling defaults #7944

Conversation

smola commented Nov 13, 2024

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Nov 13, 2024

Benchmarks

Startup

Load

Parameters

Summary

Dacapo

Parameters

Summary