-
Notifications
You must be signed in to change notification settings - Fork 385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(gnolang): use slice not map for Attributes.data per usage performance #3437
Open
odeke-em
wants to merge
1
commit into
gnolang:master
Choose a base branch
from
odeke-em:gnovm-make-Attributes.data-use-slice-not-map
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
perf(gnolang): use slice not map for Attributes.data per usage performance #3437
odeke-em
wants to merge
1
commit into
gnolang:master
from
odeke-em:gnovm-make-Attributes.data-use-slice-not-map
+97
−6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…mance Noticed in profiling stdlibs/bytes that a ton of memory was being used in maps, and that's due to the conventional CS 101 that maps with O(1) lookups, insertions and deletions beat O(n) slices' performance, but when n is small, the memory bloat is not worth it and we can use slices as evidenced in profiles for which there was 30% perceptible reduction in RAM where * Before: ```shell Showing nodes accounting for 92.90MB, 83.87% of 110.76MB total Dropped 51 nodes (cum <= 0.55MB) Showing top 10 nodes out of 123 flat flat% sum% cum cum% 47.37MB 42.77% 42.77% 47.37MB 42.77% internal/runtime/maps.newarray 10.50MB 9.48% 52.25% 10.50MB 9.48% internal/runtime/maps.NewEmptyMap 8MB 7.22% 59.47% 8MB 7.22% github.com/gnolang/gno/gnovm/pkg/gnolang.(*StaticBlock).InitStaticBlock 7.51MB 6.78% 66.25% 13.03MB 11.76% github.com/gnolang/gno/gnovm/pkg/gnolang.Go2Gno 6.02MB 5.43% 71.68% 10.73MB 9.68% github.com/gnolang/gno/gnovm/pkg/gnolang.(*defaultStore).SetObject 4MB 3.61% 75.29% 4MB 3.61% github.com/gnolang/gno/gnovm/pkg/gnolang.NewBlock 3.47MB 3.13% 78.43% 3.47MB 3.13% github.com/gnolang/gno/gnovm/pkg/gnolang.(*Allocator).NewDataArray 2.52MB 2.27% 80.70% 3.52MB 3.18% github.com/gnolang/gno/gnovm/pkg/gnolang.toKeyValueExprs 2MB 1.81% 82.51% 2MB 1.81% runtime.allocm 1.51MB 1.36% 83.87% 1.51MB 1.36% runtime/pprof.(*profMap).lookup ``` ```shell Showing nodes accounting for 47.37MB, 42.77% of 110.76MB total ----------------------------------------------------------+------------- flat flat% sum% cum cum% calls calls% + context ----------------------------------------------------------+------------- 47.37MB 100% | internal/runtime/maps.newGroups 47.37MB 42.77% 42.77% 47.37MB 42.77% | internal/runtime/maps.newarray ----------------------------------------------------------+------------- 32.01MB 78.05% | github.com/gnolang/gno/gnovm/pkg/gnolang.preprocess1.func1 7MB 17.07% | github.com/gnolang/gno/gnovm/pkg/gnolang.evalConst (inline) 1.50MB 3.66% | github.com/gnolang/gno/gnovm/pkg/gnolang.constType (inline) 0.50MB 1.22% | github.com/gnolang/gno/gnovm/pkg/gnolang.tryPredefine.func1 0 0% 42.77% 41.01MB 37.03% | github.com/gnolang/gno/gnovm/pkg/gnolang.(*Attributes).SetAttribute 41.01MB 100% | runtime.mapassign_faststr ----------------------------------------------------------+------------- 4.50MB 100% | github.com/gnolang/gno/gnovm/pkg/test.(*TestOptions).runTestFiles 0 0% 42.77% 4.50MB 4.06% | github.com/gnolang/gno/gnovm/pkg/gnolang.(*Machine).RunFiles 4.50MB 100% | github.com/gnolang/gno/gnovm/pkg/gnolang.(*Machine).runFileDecls ``` and after: ```shell Showing nodes accounting for 61.99MB, 73.12% of 84.78MB total Showing top 10 nodes out of 196 flat flat% sum% cum cum% 19.50MB 23.00% 23.00% 19.50MB 23.00% github.com/gnolang/gno/gnovm/pkg/gnolang.(*Attributes).SetAttribute 12.52MB 14.76% 37.77% 18.02MB 21.26% github.com/gnolang/gno/gnovm/pkg/gnolang.Go2Gno 7.58MB 8.94% 46.70% 9.15MB 10.79% github.com/gnolang/gno/gnovm/pkg/gnolang.(*defaultStore).SetObject 5MB 5.90% 52.60% 5MB 5.90% github.com/gnolang/gno/gnovm/pkg/gnolang.(*StaticBlock).InitStaticBlock 3.47MB 4.09% 56.69% 3.47MB 4.09% github.com/gnolang/gno/gnovm/pkg/gnolang.(*Allocator).NewDataArray 3MB 3.54% 60.24% 3MB 3.54% github.com/gnolang/gno/gnovm/pkg/gnolang.NewBlock 3MB 3.54% 63.77% 3MB 3.54% github.com/gnolang/gno/gnovm/pkg/gnolang.Nx (inline) 2.77MB 3.26% 67.04% 2.77MB 3.26% bytes.growSlice 2.65MB 3.12% 70.16% 2.65MB 3.12% internal/runtime/maps.newarray 2.50MB 2.95% 73.12% 2.50MB 2.95% runtime.allocm ``` Fixes gnolang#3436
🛠 PR Checks SummaryAll Automated Checks passed. ✅ Manual Checks (for Reviewers):
Read More🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers. ✅ Automated Checks (for Contributors):🟢 Maintainers must be able to edit this pull request (more info) ☑️ Contributor Actions:
☑️ Reviewer Actions:
📚 Resources:Debug
|
Codecov ReportAttention: Patch coverage is
📢 Thoughts on this report? Let us know! |
jefft0
added
the
review/triage-pending
PRs opened by external contributors that are waiting for the 1st review
label
Jan 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
📦 🤖 gnovm
Issues or PRs gnovm related
review/triage-pending
PRs opened by external contributors that are waiting for the 1st review
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TL;DR: Despite the reduction in memory consumption, the preliminary direct benchmarks show increase in CPU time but in microseconds
Deep dive
Noticed in profiling stdlibs/bytes that a ton of memory was being used in maps, and that's due to the conventional CS 101 that maps with O(1) lookups, insertions and deletions beat O(n) slices' performance, but when n is small, the memory bloat is not worth it and we can use slices as evidenced in profiles for which there was 30% perceptible reduction in RAM where
and after:
Fixes #3436