[eventd] Add incremental polling when waiting for capture service to start #18138
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@zbud-msft Can you please check if we can cherrypick this fix to 202305? If not can you pelase create a separate PR? |
qiluo-msft
reviewed
Feb 26, 2024
| @@ -547,8 +548,9 @@ capture_service::set_control(capture_control_t ctrl, event_serialized_lst_t *lst | |||
| case INIT_CAPTURE: | |||
| m_thr = thread(&capture_service::do_capture, this); | |||
| for(int i=0; !m_cap_run && (i < CAPTURE_SERVICE_POLLING_RETRIES); ++i) { | |||
| /* Wait max a second for thread to init */ | |||
| this_thread::sleep_for(chrono::milliseconds(CAPTURE_SERVICE_POLLING_DURATION)); | |||
| /* Poll to see if thread has been init, if so exit early. Add delay on every attempt */ | |||
There was a problem hiding this comment.
I added logic that eventd will not exit when capture service fails to initialize; warning logs are added, and main proxy/zmq service will continue to run.
|
hi @qiluo-msft @zbud-msft could you update the ADO? Is this a must to have for 202305? |
|
@zbud-msft This PR has conflict with 202305. Can you please check and if required create a separate PR for 202305? |
@dgsudharsan Currently depending on #18249 |
qiluo-msft
approved these changes
Mar 11, 2024
|
@yxieca Can you please help to cherry-pick this fix to 202311? |
mssonicbld
pushed a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Apr 3, 2024
…start (sonic-net#18138) ### Why I did it Addresses sonic-net#17350 ### How I did it Instead of a 1 second delay, we poll to check that the thread is available and after each poll increment the delay. There were situations where if there was less memory available, fixed polling would not be effective for starting zmq capture service. Add an incremental delay such that eventd can wait longer to start up capture service if system is too busy or overloaded, but still keep a max duration/retry limit so that we do not wait forever. #### How to verify it UT
|
Cherry-pick PR to 202311: #18552 |
mssonicbld
pushed a commit
that referenced
this pull request
Apr 4, 2024
…start (#18138) ### Why I did it Addresses #17350 ### How I did it Instead of a 1 second delay, we poll to check that the thread is available and after each poll increment the delay. There were situations where if there was less memory available, fixed polling would not be effective for starting zmq capture service. Add an incremental delay such that eventd can wait longer to start up capture service if system is too busy or overloaded, but still keep a max duration/retry limit so that we do not wait forever. #### How to verify it UT
mssonicbld
added
Included in 202311 Branch
and removed
Created PR to 202311 Branch
labels
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I did it
Addresses #17350
Work item tracking
How I did it
Instead of a 1 second delay, we poll to check that the thread is available and after each poll increment the delay. There were situations where if there was less memory available, fixed polling would not be effective for starting zmq capture service. Add an incremental delay such that eventd can wait longer to start up capture service if system is too busy or overloaded, but still keep a max duration/retry limit so that we do not wait forever.
How to verify it
UT
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)