Speedup arglist #13879
Open
Speedup arglist #13879
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bonzini
force-pushed
the
speedups
branch
4 times, most recently
from
c7bd527 to
f501fa9
Compare
eli-schwartz
requested changes
Nov 6, 2024
bonzini
force-pushed
the
speedups
branch
4 times, most recently
from
47d8542 to
8c0faca
Compare
bonzini
changed the title
bonzini
force-pushed
the
speedups
branch
2 times, most recently
from
234adf6 to
bc660cb
Compare
bonzini
force-pushed
the
speedups
branch
3 times, most recently
from
14150bd to
84b08ff
Compare
Fair enough! I've split the changes more finely, which should be easier to both review and apply individually. If you prefer to have multiple PRs let me know. |
bonzini
force-pushed
the
speedups
branch
2 times, most recently
from
60192f2 to
58a072d
Compare
|
Ping. :) |
|
LGTM at least. |
|
Cherry-picked 4 commits and merged to master: e542901...bb23db4 |
"Inline" CompilerArgs.__iter__() into CompilerArgs.__init__(), so that
replace list(Iterable) is replaced by the much faster list(List).
Before:
ncalls tottime cumtime
19268 0.163 3.586 arglist.py:97(__init__)
After:
ncalls tottime cumtime
18674 0.211 3.442 arglist.py:97(__init__)
Signed-off-by: Paolo Bonzini <[email protected]>
Unless an argument is marked as Dedup.OVERRIDDEN, pre_flush_set and
post_flush_set will always be empty and the loops in flush_pre_post()
will not be doing anything interesting:
for a in self.pre:
dedup = self._can_dedup(a)
if a not in pre_flush_set:
# This just makes new a copy of self.pre
new.append(a)
if dedup is Dedup.OVERRIDDEN:
# this never happens
pre_flush_set.add(a)
for a in reversed(self.post):
dedup = self._can_dedup(a)
if a not in post_flush_set:
# Here self.post is reversed twice
post_flush.appendleft(a)
if dedup is Dedup.OVERRIDDEN:
# this never happens
post_flush_set.add(a)
new.extend(post_flush)
In this case it's possible to avoid expensive calls and loops, instead
relying as much on Python builtins as possible. Track whether any options
have that flag and if not just concatenate pre, _container and post.
Before:
ncalls tottime cumtime
45127 0.251 4.530 arglist.py:142(__iter__)
81866 3.623 5.013 arglist.py:108(flush_pre_post)
76618 3.793 5.338 arglist.py:273(__iadd__)
After:
35647 0.156 0.627 arglist.py:160(__iter__)
78998 2.627 3.603 arglist.py:116(flush_pre_post)
73774 3.605 5.049 arglist.py:292(__iadd__)
The time in __iadd__ is reduced because it calls __iter__, which flushes
pre and post.
Signed-off-by: Paolo Bonzini <[email protected]>
self.post is only ever appended to on the right hand. However, it is then reversed twice in flush_pre_post(), by using "for a in reversed.post()" and appendleft() within the loop. It would be tempting to use appendleft() in __iadd__ to avoid the call to reversed(), but that is not a good idea because the loop of flush_pre_post() is part of a slow path. It's rather more important to use a fast extend-with-list-argument in the fast path where needs_override_check if False. For clarity, and to remove the temptation, make "post" a list instead of a deque. Signed-off-by: Paolo Bonzini <[email protected]>
|
Thanks, I reduced the rest to just three that provide the most bang for the buck (and are easier to review than it seems). |
12 tasks
bonzini
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This speeds up the target generation of QEMU, which is very slow.
2921 seconds (total time150134 seconds)