-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiprocessing Manager resources (Queue) to be freed up during task stop #15212
Conversation
@adyeung @Junchao-Mellanox @vaibhavhd |
self.mpmgr.shutdown() | ||
|
||
# Wait for the process to exit | ||
self._task_process.join(self._stop_timeout_secs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should monitor_system_bus task and monitor_statedb_table task be join here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At the end of the system_service() function, there exists a call to monitor_system_bus.task_stop() and monitor_statedb_table.task_stop(), which are responsible for shutting down their respective tasks.
if self._task_process.is_alive(): | ||
os.kill(self._task_process.pid, signal.SIGKILL) | ||
|
||
if self._task_process.is_alive(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
log something here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added logs
Multiprocessing Manager resources (Queue) to be freed up during task stop
f72ab1c
to
b3ae16c
Compare
@Junchao-Mellanox Can you please check and review? Thanks |
@Junchao-Mellanox please check the latest comments here. We need to take this fix in sooner. @sg893052, please update the test description to indicate that the shutdown path is no longer hung/delayed. Please test that path if not already tested. |
ADO: 22090981 |
@sg893052 PR conflicts with 202211 branch |
@sg893052 PR conflicts with 202205 branch |
@sg893052 please help create separate PR for 202205/202211 branches. Thanks! |
Cherry-pick PR to 202305: #15595 |
Multiprocessing Manager resources (Queue) to be freed up during task stop
@sg893052 PR conflicts with 202205 branch |
Multiprocessing Manager resources (Queue) to be freed up during task stop
Multiprocessing Manager resources (Queue) to be freed up during task stop
Why I did it
Fix for issue #14964
Work item tracking
How I did it
Free up Multiprocessing Manager resource at task stop request
[self.mpmgr.shutdown() in task_stop]
How to verify it
time systemctl stop system-health.service
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)