Reduce memory churn on module load by reusing kernel BTF spec #487
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Before this change we always parsed the given BTFPath spec. In 99% of the cases, the path pointed to the default kernel BTF location. Due to the way the cilium/ebpf library handles passing BTF spec, this caused the kernels BTF spec to be effectively parsed twice, causing quite the memory churn. The docs mention that not passing `KernelTypes` for the program option will implicitly use the kernel BTF spec. While this is true, it also causes a lot of memory churn in the process, as each time the `btf.LoadKernelSpec` function gets called, the spec is copied. To work around this, kvisor detects if the given `BTFPath` is the default BTF location, parses the kernel BTF spec once and passes it to the ebpf lib via the `KernelTypes` option. Running benchmarks, this change allows to cut the number of allocations by ~1/3. Benchmark before optimization 5 224041936 ns/op 111770155 B/op 1213041 allocs/op Benchmark after optimization 6 173737552 ns/op 73678120 B/op 848361 allocs/op
|
Related fix in the cilium/ebpf library: cilium/ebpf#1690 |
anjmao
approved these changes
Feb 18, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before this change we always parsed the given BTFPath spec. In 99% of the cases, the path pointed to the default kernel BTF location. Due to the way the cilium/ebpf library handles passing BTF spec, this caused the kernels BTF spec to be effectively parsed twice, causing quite the memory churn.
The docs mention that not passing
KernelTypesfor the program option will implicitly use the kernel BTF spec. While this is true, it also causes a lot of memory churn in the process, as each time thebtf.LoadKernelSpecfunction gets called, the spec is copied.To work around this, kvisor detects if the given
BTFPathis the default BTF location, parses the kernel BTF spec once and passes it to the ebpf lib via theKernelTypesoption.Running benchmarks, this change allows to cut the number of allocations by ~1/3.
Benchmark before optimization
Benchmark after optimization