Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixup ast_transform docs #128

Merged
merged 2 commits into from
May 31, 2024
Merged

Fixup ast_transform docs #128

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a45d46d
Fixup ast_transform docs
esc May 29, 2024
524c482
fix syntax
esc May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/source/reference/datastructures.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,9 @@ SCFG classes

.. automodule:: numba_rvsdg.core.datastructures.scfg
:members:

AST Transforms
==============

.. automodule:: numba_rvsdg.core.datastructures.ast_transforms
:members:
116 changes: 58 additions & 58 deletions numba_rvsdg/core/datastructures/ast_transforms.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -438,23 +438,23 @@ def handle_for(self, node: ast.For) -> None:

Remember that the for-loop has a target variable that will be assigned,
an iterator to iterate over, a loop body and an else clause. The AST
node has the following signature:
node has the following signature::

ast.For(target, iter, body, orelse, type_comment)

Remember also that Python for-loops can have an else-branch, that is
executed upon regular loop conclusion.

def function(a: int) -> None
c = 0
for i in range(10):
c += i
if i == a:
i = 420 # set i arbitrarily
break # early exit, break from loop, bypass else-branch
else:
c += 1 # loop conclusion, i.e. we have not hit the break
return c, i
executed upon regular loop conclusion::

def function(a: int) -> None:
c = 0
for i in range(10):
c += i
if i == a:
i = 420 # set i arbitrarily
break # early exit, break from loop, bypass else
else:
c += 1 # loop conclusion, i.e. not hit break
return c, i

So, effectively, to decompose the for-loop, we need to setup the
iterator by calling 'iter(iter)' and assign it to a variable,
Expand All @@ -469,14 +469,14 @@ def function(a: int) -> None
However, it is possible to use the 'next()' method with a second
argument to avoid exception handling here. We do this so we don't need
to rely on being able to transform exceptions as part of this
transformer.
transformer::

i = next(iter, "__sentinel__")
if i != "__sentinel__":
...

Lastly, it is important to also remember that the target variable
escapes the scope of the for loop:
escapes the scope of the for loop::

>>> for i in range(1):
... print("hello loop")
Expand All @@ -490,73 +490,73 @@ def function(a: int) -> None
loop with some assignments and he target variable must escape the
scope.

Consider again the following function:

def function(a: int) -> None
c = 0
for i in range(10):
c += i
if i == a:
i = 420
break
else:
c += 1
return c, i
Consider again the following function::

def function(a: int) -> None:
c = 0
for i in range(10):
c += i
if i == a:
i = 420
break
else:
c += 1
return c, i

This will be decomposed as the following construct that can be encoded
using the available block and edge primitives of the CFG.

def function(a: int) -> None
c = 0
* __iterator_1__ = iter(range(10)) # setup iterator
* i = None # assign target, in this case i
while True: # loop until we break
* __iter_last_1__ = i # backup value of i
* i = next(__iterator_1__, '__sentinel__') # get next i
* if i != '__sentinel__': # regular iteration
c += i # add to accumulator
if i == a: # check for early exit
i = 420 # set i to some wild value
break # early exit break while True
else: # for-else clause
* i == __iter_last_1__ # restore value of i
c += 1 # execute code in for-else clause
break # regular exit break while True
return c, i
using the available block and edge primitives of the CFG::

def function(a: int) -> None:
c = 0
__iterator_1__ = iter(range(10)) ## setup iterator
i = None ## assign target, i
while True: # loop until we break
__iter_last_1__ = i ## backup value of i
i = next(__iterator_1__, '__sentinel__') # * get next i
if i != '__sentinel__': ## regular iteration
c += i # add to accumulator
if i == a: # check for early exit
i = 420 # set i to some wild value
break # early exit break while True
else: # for-else clause
i == __iter_last_1__ ## restore value of i
c += 1 # execute code in for-else
break # exit break while True
return c, i

The above is actually a full Python source reconstruction. In the
implementation below, it is only necessary to emit some of the special
assignments (marked above with a *-prefix above) into the blocks of the
assignments (marked above with a #-prefix above) into the blocks of the
CFG. All of the control-flow inside the function will be represented
by the directed edges of the CFG.

The first two assignments are for the pre-header:

* __iterator_1__ = iter(range(10)) # setup iterator
* i = None # assign target, in this case i
* ``__iterator_1__ = iter(range(10)) ## setup iterator``
* ``i = None ## assign target, i``

The next three is for the header, the predicate determines the end of
the loop.

* __iter_last_1__ = i # backup value of i
* i = next(__iterator_1__, '__sentinel__') # get next i
* if i != '__sentinel__': # regular iteration
* ``__iter_last_1__ = i ## backup value of i``
* ``i = next(__iterator_1__, '__sentinel__') # * get next i``
Copy link
Contributor

@kc611 kc611 May 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nitpick but the rendering of this particular line makes the RHS comment shift right quite a bit. Which overall makes it look very haphazard. Could all the RHS comments in this sections be shifted right by the same amount so as to match the indentation of this particular line

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indentation is such that it is easier to keep the lines in sync with the above as the indentation level is matched. For the output, they render without all the leading whitespace anyway as shown here. I would prefer thus to keep them as is.

Screenshot 2024-05-29 at 14 02 46

* ``if i != '__sentinel__': ## regular iteration``

And lastly, one assignment in the for-else clause

* i == __iter_last_1__ # restore value of i
* ``i == __iter_last_1__ ## restore value of i``

We modify the pre-header, the header and the else blocks with
appropriate Python statements in the following implementation. The
Python code is injected by generating Python source using f-strings and
then using the 'unparse()' function of the 'ast' module to then use the
'codegen' method of this transformer to emit the required 'ast.AST'
objects into the blocks of the CFG.
then using the ``unparse()`` function of the ``ast`` module to then use
the 'codegen' method of this transformer to emit the required
``ast.AST`` objects into the blocks of the CFG.

Lastly the important thing to observe is that we can not ignore the
else clause, since this must contain the reset of the variable i, which
will have been set to '__sentinel__'. This reset is required such that
the target variable 'i' will escape the scope of the for-loop.
will have been set to ``__sentinel__``. This reset is required such
that the target variable ``i`` will escape the scope of the for-loop.

"""
# Preallocate indices for header, body, else, and exiting blocks.
Expand Down