Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solve StressSuite flakiness #4

Merged
merged 1 commit into from
Feb 29, 2024
Merged

Solve StressSuite flakiness #4

merged 1 commit into from
Feb 29, 2024

Conversation

jchapuis
Copy link
Member

No description provided.

Copy link

what-the-diff bot commented Feb 29, 2024

PR Summary

  • Introduction of new Logger
    A new logging tool named endless.runtime.pekko.deploy has been added to the system configuration (a file called logback.xml). This tool will assist in tracking and recording events occurring during the execution of our application.

  • Update to Logging Level
    The amount of detail that our logging tool shares has been scaled back. In the logback.xml file, we shifted the logging level from DEBUG (which logs all minor details) to INFO (which logs general statuses and severe issues). This change will streamline our logs, making it easier to spot and diagnose significant problems.

  • Modification of Termination Timeout
    The terminationTimeout value in PekkoAccountsApp.scala, which is the time-out period in case a process doesn't end as expected, is reduced from 30 seconds to 5 seconds. This change will accelerate the detection of stuck processes and allow prompt notification or counteraction.

  • Adjustment of Node Waiting Time
    The atLeastOneNodeUpWaitingTime parameter in StressSuite.scala, which represents how long the system waits for at least one node to be up and running, is reduced from 5 minutes to 3 minutes. It allows for quicker response times for the testing suite and helps in faster identification of potential issues.

Copy link

Workflow Telemetry - CI / build

Workflow telemetry for commit 6b3b53b6da069389269618ecece5a7bdda63666f
You can access workflow job details here

Step Trace

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S
	Set up job : milestone, 1709243354000, 1709243356000
	Initialize containers : 1709243356000, 1709243375000
	Run actions/checkout@v2 : 1709243375000, 1709243375000
	Run coursier/[email protected] : 1709243375000, 1709243387000
	Run catchpoint/workflow-telemetry-action@v2 : 1709243387000, 1709243387000

Loading

CPU Metrics

chart_stacked_area_time_af786d02-bceb-4fb8-84a6-6c5580df618c

Memory Metrics

chart_stacked_area_time_4509b7e0-fd92-4ecc-8f18-73e81cd52a83

IO Metrics

Read Write
Network I/O chart_line_time_ee9e5bf3-5a39-40f7-9466-81241b48cc4e chart_line_time_5ed27e5c-00dd-4d5c-8e29-e1e9ca7344be
Disk I/O chart_line_time_529913c6-049d-4e1a-9ec8-1e5f054d6c3d chart_line_time_bde9d206-1445-49cb-b316-4946817dd8b4

Process Trace

Top 100 processes with highest duration

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S

Loading

Copy link

Workflow Telemetry - CI / build

Workflow telemetry for commit 5c4de5e5fdd9568f32bcd147d4c1e51395a04812
You can access workflow job details here

Step Trace

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S
	Set up job : milestone, 1709243704000, 1709243706000
	Initialize containers : 1709243706000, 1709243726000
	Run actions/checkout@v2 : 1709243726000, 1709243727000
	Run coursier/[email protected] : 1709243727000, 1709243738000
	Run catchpoint/workflow-telemetry-action@v2 : 1709243738000, 1709243739000

Loading

CPU Metrics

chart_stacked_area_time_9190e4d6-17d4-491e-bf7a-5fbdbe5846ee

Memory Metrics

chart_stacked_area_time_0f358c2f-6a71-4757-b9ac-bc6bd1cbddf3

IO Metrics

Read Write
Network I/O chart_line_time_fb726895-8a8a-43db-bb10-23fa6f48ca8a chart_line_time_bb796cda-064d-4b2f-82f0-273cbd8a226b
Disk I/O chart_line_time_fb0d0fd4-501a-436e-92da-2b4c27c5a1e8 chart_line_time_7d3e502e-9598-477d-a519-e5a516526691

Process Trace

Top 100 processes with highest duration

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S

Loading

Copy link

Workflow Telemetry - CI / build

Workflow telemetry for commit 2757d6925b8963e69c2a18a6b9c108b736b41a93
You can access workflow job details here

Step Trace

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S
	Set up job : milestone, 1709244280000, 1709244282000
	Initialize containers : 1709244282000, 1709244303000
	Run actions/checkout@v2 : 1709244303000, 1709244304000
	Run coursier/[email protected] : 1709244304000, 1709244315000
	Run catchpoint/workflow-telemetry-action@v2 : 1709244315000, 1709244315000

Loading

CPU Metrics

chart_stacked_area_time_fb93bdd9-0338-4e92-a4cf-7f494d75b5bf

Memory Metrics

chart_stacked_area_time_52b7aef7-13bb-424e-9635-b61bd93148c3

IO Metrics

Read Write
Network I/O chart_line_time_76ea0c0a-14f1-4f11-88b2-e6891f758664 chart_line_time_fa75df41-3d45-4473-b658-a8cb39482a7e
Disk I/O chart_line_time_2df9b53c-33b0-47ed-975e-fbcf2d00f99f chart_line_time_c10c792a-bb87-4873-ba02-b4472eb3ca54

Process Trace

Top 100 processes with highest duration

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S

Loading

Copy link

Workflow Telemetry - CI / build

Workflow telemetry for commit ad69ae0ae3f117336ddcc4279db95b36fcb47c97
You can access workflow job details here

Step Trace

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S
	Set up job : milestone, 1709244918000, 1709244919000
	Initialize containers : 1709244919000, 1709244940000
	Run actions/checkout@v2 : 1709244940000, 1709244942000
	Run coursier/[email protected] : 1709244942000, 1709244953000
	Run catchpoint/workflow-telemetry-action@v2 : 1709244953000, 1709244954000
	Run sbt scalafmtCheckAll compile coverage +test "repeat 3 example/multi-jvm-test" coverageReport coverageAggregate versionPolicyCheck documentation/makeSite : 1709244954000, 1709246146000

Loading

CPU Metrics

chart_stacked_area_time_9eb7f407-a52a-4ed4-8254-a08f5fa9f3a5

Memory Metrics

chart_stacked_area_time_80c3243e-463c-4a54-8a45-ea437ff1e3d6

IO Metrics

Read Write
Network I/O chart_line_time_448f8c90-7eaa-4e16-87e5-2183ed1f05c1 chart_line_time_ba492850-f889-498a-b7e0-d92596797e10
Disk I/O chart_line_time_7c419399-1f1f-478a-b033-b0aaeead8da2 chart_line_time_6fa3a8eb-3b06-4928-89a7-5fef60ada025

Process Trace

Top 100 processes with highest duration

gantt
	title build
	dateFormat x
	axisFormat %H:%M:%S

Loading

Copy link

codecov bot commented Feb 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.15%. Comparing base (740242d) to head (ad69ae0).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master       #4      +/-   ##
==========================================
+ Coverage   70.90%   71.15%   +0.24%     
==========================================
  Files          34       34              
  Lines         825      825              
  Branches       97       97              
==========================================
+ Hits          585      587       +2     
+ Misses        240      238       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jchapuis jchapuis merged commit 5d2623d into master Feb 29, 2024
3 checks passed
@jchapuis jchapuis deleted the fix-flakkiness branch February 29, 2024 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant