-
Notifications
You must be signed in to change notification settings - Fork 433
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi-Threaded Executor Starvation #2701
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* rephrase doc section of LoanedMessage Class. Signed-off-by: Tomoya Fujita <[email protected]> Co-authored-by: Chris Lalancette <[email protected]> Co-authored-by: Christophe Bedard <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: Chris Lalancette <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: HarunTeper <[email protected]>
* accept custom allocator for LoanedMessage. Signed-off-by: Tomoya Fujita <[email protected]> * move message allocator using directives to public. Signed-off-by: Tomoya Fujita <[email protected]> --------- Signed-off-by: Tomoya Fujita <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: Chris Lalancette <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: HarunTeper <[email protected]>
They are both legacy from times past, and are both no longer serving their intended purposes. Just remove them. Signed-off-by: Chris Lalancette <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: HarunTeper <[email protected]>
…onsistency Signed-off-by: HarunTeper <[email protected]>
…ait (ros2#2683) Signed-off-by: Janosch Machowinski <[email protected]> Signed-off-by: HarunTeper <[email protected]>
* Omnibus fixes for running tests with Connext. When running the tests with RTI Connext as the default RMW, some of the tests are failing. There are three different failures fixed here: 1. Setting the liveliness duration to a value smaller than a microsecond causes Connext to throw an error. Set it to a millisecond. 2. Using the SystemDefaultsQoS sets the QoS to KEEP_LAST 1. Connext is somewhat slow in this regard, so it can be the case that we are overwriting a previous service introspection event with the next one. Switch to the ServicesDefaultQoS in the test, which ensures we will not lose events. 3. Connext is slow to match publishers and subscriptions. Thus, when creating a subscription "on-the-fly", we should wait for the publisher to match it before expecting the subscription to actually receive data from it. With these fixes in place, the test_client_common, test_generic_service, test_service_introspection, and test_executors tests all pass for me with rmw_connextdds. Signed-off-by: Chris Lalancette <[email protected]> * Fixes for executors. Signed-off-by: Chris Lalancette <[email protected]> * One more fix for services. Signed-off-by: Chris Lalancette <[email protected]> * More fixes for service_introspection. Signed-off-by: Chris Lalancette <[email protected]> * More fixes for introspection. Signed-off-by: Chris Lalancette <[email protected]> --------- Signed-off-by: Chris Lalancette <[email protected]> Signed-off-by: HarunTeper <[email protected]>
* Make ament_cmake a buildtool dependency The `ament_cmake` package isn't needed at runtime and so should only be listed as a `buildtool_dependency`, as is done in most other packages. * Remove the ament_cmake dependency entirely Since there's already a buildtool dependency on ament_cmake_ros, having ament_cmake as well is redundant. Signed-off-by: Nathan Wiebe Neufeldt <[email protected]> Signed-off-by: HarunTeper <[email protected]>
For reasons I admit I do not understand, the deprecation warnings for StaticSingleThreadedExecutor on Windows happen when we construct a shared_ptr for it in the tests. If we construct a regular object, then it is fine. Luckily this test does not require a shared_ptr, so just make it a regular object here, which rixes the warning. While we are in here, make all of the tests camel case to be consistent. Signed-off-by: Chris Lalancette <[email protected]> Signed-off-by: HarunTeper <[email protected]>
It supports the events executor now, so re-enable the test. Signed-off-by: Chris Lalancette <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: Patrick Roncagliolo <[email protected]> Signed-off-by: HarunTeper <[email protected]>
Signed-off-by: HarunTeper <[email protected]>
…ait (ros2#2683) Signed-off-by: Janosch Machowinski <[email protected]>
For reasons I admit I do not understand, the deprecation warnings for StaticSingleThreadedExecutor on Windows happen when we construct a shared_ptr for it in the tests. If we construct a regular object, then it is fine. Luckily this test does not require a shared_ptr, so just make it a regular object here, which rixes the warning. While we are in here, make all of the tests camel case to be consistent. Signed-off-by: Chris Lalancette <[email protected]>
It supports the events executor now, so re-enable the test. Signed-off-by: Chris Lalancette <[email protected]>
Signed-off-by: HarunTeper <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request addressing the issues in #2360 #2645.
So far, I have added a test that detects starvation in the multi-threaded executor.
This test includes a mutually-exclusive callback group with two timers.
The executor should be alternating between these two tasks, never executing one task twice before the other.
To fix starvation, I have identified the following steps (which I was not able to completely implement yet):
These steps are based on the work I published here:
https://ieeexplore.ieee.org/document/9622336
https://daes.cs.tu-dortmund.de/storages/daes-cs/r/publications/teper2024emsoft_preprint.pdf
I have already tried to implement some of these steps, and I will also commit some of the changes to this fork this week. However, for step 4, I may require some help. I also noticed that my changes break some of the tests are currently part of rclcpp, as I move the functions that set callback group flags and trigger guard conditions.