Tuning Pelican for running xrootd under valgrind #1677
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bbockelm
force-pushed
the
valgrind_tuning
branch
from
e323de4 to
c60630c
Compare
jhiemstrawisc
requested changes
Nov 27, 2024
There was a problem hiding this comment.
I don't think we pull the actual defaults from docs/parameters.yaml yet, so the value we get from param.Xrootd_MaxStartupWait.GetDuration() is going to be 0 unless it's defined in the user's custom config. My guess is this is what's breaking the ticker.
Defaults can be defined in either config/resources/defaults.yaml or in our new config/config.go::SetServerDefaults
If xrootd is running under valgrind, the startup time may be much more than 10s. Provide a hidden tunable to give valgrind more time.
Under valgrind, its trivial to make the default cache timeouts hit frequently - make this adjustable so stress tests can succeed. Additionally, add a few helper RPMs for debugging under valgrind.
bbockelm
force-pushed
the
valgrind_tuning
branch
from
c60630c to
62a2854
Compare
turetske
approved these changes
Nov 27, 2024
jhiemstrawisc
approved these changes
Nov 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Valgrind is incredibly useful C++ debugging tool to check for leaks and other memory usage errors.
However, it also makes the application incredibly slow: it's often a 20x (or more) performance penalty. Xrootd caches hit the timeouts frequently.
This PR adds a few hidden tunables to allow us to increase timeouts enough for xrootd to run under valgrind. It also adds some standard C++ debugging tools into the development container.