Adding extra time counter to exit while loop so that controller manager doesn't freeze #458
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
krishnachaitanya7
changed the title
|
Hi @bmagyar! Can you please look into the PR? You commented on the previous issue I posted which I fixed in this PR. Thank you. Kindly let me know! |
|
@bmagyar can you let us know the status of this? This is a known bug in the controller manager. |
krishnachaitanya7
changed the title
|
@bmagyar -- still no updates on this. Have you been able to verify that this bug occurs on your end? |
bmagyar
reviewed
Jun 16, 2021
bmagyar
reviewed
Jun 16, 2021
|
Thanks for the fix, I'm releasing this now to melodic and noetic! |
|
Thank you so much @bmagyar! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses issue #443. When I switch the controllers sometimes the controller manager is stuck here at this unending while loop. The problem I am facing is similar to #265. The only way out of this loop is either when
In the first case, switch_params_.do_switch becomes False if switching is successful which is the ideal case
Exiting ROS Core means either simulation is shutdown or ROS core exited with non zero exit code
If the timeout is specified after the timeout the switching is stopped and the response code returned back is False. The timeout is calculated using the ROS time. When I ran my simulation using Gazebo the ROS time freezes hence the timeout is never possible. Due to this issue, the code is stuck at the aforementioned while loop and never comes out. This gives rise to Denial of Service. The simulation seems to be stuck. No further switching of controllers happen. Even the controller manager does not list the current controllers which are stopped or running because the controller manager is in denial of service and it's lock is not released. The only way out of this is to restart the entire simulation.
Hence I have added a time check in the while loop which uses system time contrary to ROS time and if something like this happens at least the code can safely exit rather than go into denial of service. The code edits suggested in this PR will give every opportunity for ROS time to function but it will also monitor the timeout from a system clock perspective and after timeout+1 seconds if the while loop didn't break, it will forcibly break out of the while loop. Finally when the while loop is exited by forcibly breaking out "False" is returned signaling the end-user that controller switching wasn't successful.