-
-
Notifications
You must be signed in to change notification settings - Fork 33.2k
Document how to add a bytecode specialization in Interpreter.md file #130831
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
faizanoor3001
wants to merge
10
commits into
python:main
Choose a base branch
from
faizanoor3001:add-bytecode-specialization-guide
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
3bf2baf
Updated the interpreters.md with the how to add a new bytecode specia…
faizanoor3001 2d2c7e4
Merge branch 'main' of https://github.com/python/cpython into add-byt…
faizanoor3001 9ec70df
Merge remote-tracking branch 'upstream/main' into add-bytecode-specia…
faizanoor3001 a023a52
Addressed the review comments
faizanoor3001 10a7144
Merge branch 'main' of https://github.com/python/cpython into add-byt…
faizanoor3001 cb03b20
Update section for Step1, reworded as per review comments for clarity
faizanoor3001 d9783ca
Added new lines between steps for better readability
faizanoor3001 852cbaf
Updated the text for the steps 3,4,5 as per comment
faizanoor3001 50ea367
Merge remote-tracking branch 'upstream/main' into add-bytecode-specia…
faizanoor3001 947a2d4
Apply suggestions from code review
hugovk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -506,6 +506,68 @@ After the last `DEOPT_IF` has passed, a hit should be recorded with | |
After an optimization has been deferred in the adaptive instruction, | ||
that should be recorded with `STAT_INC(BASE_INSTRUCTION, deferred)`. | ||
|
||
## How to add a new bytecode specialization | ||
|
||
Assuming you found an instruction that serves as a good specialization candidate. | ||
Let's use the example of [`CONTAINS_OP`](../Doc/library/dis.rst#contains_op): | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
1. Update below in [Python/bytecodes.c](../Python/bytecodes.c) | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
- Convert `CONTAINS_OP` to a micro-operation (uop) by renaming | ||
it to `_CONTAINS_OP` and changing the instruction definition | ||
from `inst` to `op`. | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
```c | ||
// Before | ||
inst(CONTAINS_OP, ...); | ||
|
||
// After | ||
op(_CONTAINS_OP, ...); | ||
``` | ||
|
||
- Add a uop that calls the specializing function: | ||
|
||
```c | ||
specializing op(_SPECIALIZE_CONTAINS_OP, (counter/1, left, right -- left, right)) { | ||
#if ENABLE_SPECIALIZATION | ||
if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { | ||
next_instr = this_instr; | ||
_Py_Specialize_ContainsOp(right, next_instr); | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
DISPATCH_SAME_OPARG(); | ||
} | ||
STAT_INC(CONTAINS_OP, deferred); | ||
DECREMENT_ADAPTIVE_COUNTER(this_instr[1].cache); | ||
#endif /* ENABLE_SPECIALIZATION */ | ||
} | ||
``` | ||
|
||
- Create a macro for the original bytecode name: | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
```c | ||
macro(CONTAINS_OP) = _SPECIALIZE_CONTAINS_OP + _CONTAINS_OP; | ||
``` | ||
|
||
2. Define the cache structure in [Include/internal/pycore_code.h](../Include/internal/pycore_code.h), | ||
at the very least, a 16-bit counter is needed. | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
```c | ||
typedef struct { | ||
uint16_t counter; | ||
} _PyContainsOpCache; | ||
``` | ||
|
||
3. Write the specializing function itself (`_Py_Specialize_ContainsOp`) in [Python/specialize.c ](../Python/specialize.c). | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Refer to other functions in that file for the pattern. | ||
|
||
4. Add a call to `add_stat_dict` in `_Py_GetSpecializationStats` which is in [Python/specialize.c ](../Python/specialize.c). | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
5. Add the cache layout in [Lib/opcode.py](../Lib/opcode.py) so that Python's | ||
`dis` module will know how to represent it properly. | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
6. Bump magic number in [Include/core/pycore_magic_number.h](../Include/internal/pycore_magic_number.h). | ||
hugovk marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
7. Run ``make regen-all`` on `*nix` or `build.bat --regen` on Windows. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seems like the only thing missing is adding an actual specialized variant. Maybe that's implied/obvious, but it wouldn't hurt to provide a dumb example of |
||
|
||
Additional resources | ||
-------------------- | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.