Codegen function from a tag string to support ReactPy's virtual DOM #9

jimbaker · 2022-05-21T01:07:51Z

We can avoid overhead of parsing a tag string on each usage by alternatively code generating a corresponding function, then memoizing it. I have implemented a gist to show how this can be done. This code implements an html tag for ReactPy:

Parses the tag string with HTMLParser from the stdlib, in a similar fashion as done in I am working in a branch #1.
Code generates a Python function. As part of this, it passes in the lambda-wrapped expressions to be interpolated as args. The signature of this function is compiled(vdom, /, *args), where vdom is the DOM constructing function. In this case, the vdom function in idom.core.vdom, and it is very similar to other tools like ViewDOM's VDOMNode constructor. I expect this will be a common pattern.
Calls the tag function as usual.
Memoizes the function for subsequent use. Note that lambda-wrapped expressions will change on each usage of the tag string, so we can use their underlying code object for the memoization key, which is constant.

For HTML without interpolation, this looks like the following. First, the example from the IDOM docs, followed by the equivalent with the new tag:

from reactpy import html
from reactpy_tag import html as html_tag

layout1 = html.div(
    html.h1("My Todo List"),
    html.ul(
        html.li("Build a cool new app"),
        html.li("Share it with the world!"),
    )
)

layout2 = html_tag"""
<div>
    <h1>My Todo List</h1>
    <ul>
        <li>Build a cool new app</li>
        <li>Share it with the world!</li>
    </ul>
</div>
    """

assert layout2 == layout1

Next I tried plugging this into a Hello, World example from ReactPy's intro:

from fastapi import FastAPI

from reactpy import component
from reactpy_tag import html
from reactpy.backend.fastapi import configure

a = 6
b = 7

@component
def HelloWorld():
    return html"""
    <h1>Ultimate answer: {a}*{b} is {a*b}</h1>
    """

app = FastAPI()
configure(app, HelloWorld)

Then assuming the above is in main.py, run with uvicorn main:app

This requires installing reactpy, fastapi, and uvicorn. So there's some setup required, but interesting to see how tag strings work with an interesting project.

The text was updated successfully, but these errors were encountered:

jimbaker · 2022-05-21T01:11:30Z

I should mention that I used FastAPI/Uvicorn to run the web app because I couldn't get the dev web server to work on my development build from the tag string branch in #1

jimbaker · 2022-05-21T02:22:40Z

As expected, implementing support for ViewDOM is straightforward. Here's the necessary diff - we just need to adapt for the minor calling convention difference between the vdom constructor in IDOM and VDOMNode constructor in ViewDOM:

$ diff vdomtag.py idomtag.py
10c10
< from viewdom import VDOMNode, render
---
> from idom.core.vdom import vdom, VdomDict
35c35
<         self.lines.append(f"{self.indent()}vdom('{tag}', {dict(attrs)!r}, [")
---
>         self.lines.append(f"{self.indent()}vdom('{tag}', {attrs!r}, ")
42c42
<         self.lines.append(f'{self.indent()}]){"," if self.tag_stack else ""}')
---
>         self.lines.append(f'{self.indent()}){"," if self.tag_stack else ""}')
100c100
<     return compiled(VDOMNode, *args)
---
>     return compiled(vdom, *args)

See https://gist.github.com/jimbaker/8ca34e920eb7245a0d1cc093fa8e91f0 for the full code.

jimbaker · 2022-05-21T02:36:58Z

So this almost works as expected:

from viewdom import render  # from https://github.com/pauleveritt/viewdom
from vdomtag import html  # from https://gist.github.com/jimbaker/8ca34e920eb7245a0d1cc093fa8e91f0

def Todo(prefix, label):
    return html'<li>{prefix}{label}</li>'

def TodoList(prefix, todos):
    return html'<ul>{[Todo(prefix, label) for label in todos]}</ul>'

b = html"""<html>
    <body attr=blah" yo={1}>
        {TodoList('High: ', ['Get milk', 'Change tires'])}
    </body>
</html>
"""

print(render(b))

resulting in

<html><body attr="blah&#34;" yo=""><ul><li>High: Get milk</li><li>High: Change tires</li></ul></body></html>

The problem is that the interpolation for the property yo= is not properly being put in the generated code. This should be a straightforward fix to (currently very naive) VdomCodeBuilder in the gist.

gvanrossum · 2022-05-21T03:19:47Z

Neat! I suspect trouble when the html parser buffers data -- I think if I call b.feed("x"); b.add_interpolation(i); b.feed("y") you might get the interpolations and constant text in the wrong order. But we can fix that in the parser if we desire. (You neatly traipsed around this in the demo by using {i} along with {j} as the data in the Gist, and by having no constant data at all in the demo above.)

jimbaker · 2022-05-21T03:54:47Z

Hah, these examples needs some proper development now. I too was surprised I could get away with what I did with HTMLParser 😁 I will put together a branch on this project so we can manage example tag libraries with requirements and tests - copying and pasting into gists is just too much.

One nice observation is that for DOM libraries like IDOM, it is possible to mix usage like html.h1('foo') with html"<h1>bar</h1>". (For ViewDOM, it needs to be using a different name.)

gvanrossum · 2022-05-21T04:59:27Z

No need for a branch. Just create a folder.

jimbaker · 2022-05-23T04:40:12Z

The problem with how we use html.parser and its interplay with interpolations should now be solved in https://github.com/jimbaker/tagstr/blob/main/examples/htmltaglib.py

Whether we should have a solution that depends on an internal part of html.parser, namely its rawdata attribute is another question. (Examples!) But in a nutshell, this attribute allows us to determine where in the parse we are - in a tag, and if so, what has been parsed up to that interpolation; or in the data portion of that tag. The internal goahead function should "handle data as far as reasonable", per its comment.

At some point, I will add proper tests.

gvanrossum · 2022-05-23T18:27:30Z

If this ends up in the language we should add a new API to htmllib to help us here. Until then we can depend on internals or clone htmllib and hack on it, whatever works. It's just a demo. If we can do it by depending on a few internals that's good enough.

rmorshea · 2022-06-01T05:51:55Z

The implementation from the Gist seems like something I wrote back when I was thinking of taking an approach similar to, but not quite the same as, pyxl - instead of parsing the HTML with HTMLParser I used htm, and instead of constructing a string you could compile() I built an AST you could exec directly.

jimbaker · 2022-06-01T06:17:57Z

The example code for this issue is now tracked in https://github.com/jimbaker/tagstr/blob/main/examples/htmltag.py

rmorshea · 2022-06-02T00:44:10Z

Another reference implementation I found using a PEG parser: https://github.com/michaeljones/packed

jimbaker · 2022-06-02T14:19:13Z

@rmorshea Given that we are targeting DSLs, it makes sense to have at least one example worked out for using a PEG parser like pyPEG or pyparsing.

gvanrossum · 2022-06-02T17:26:55Z

One trick that occurred to me when we're feeding the text into some kind of parser: Sometimes that parser may not be set up to handle interpolations (we already ran into this with the stdlib html parser). We could replace each interpolation with a special marker that includes the index of the interpolation (e.g. $$__1__$$) which is parsed as plain text, and then hunt for the markers in the resulting object tree and replace them with the corresponding interpolations.

pauleveritt · 2023-04-30T13:58:34Z

Jim's example includes the special marker. But there's still the open question about changing the parser internals. Do you think that's still on the table?

gvanrossum · 2023-04-30T22:11:44Z

I've honestly lost track what exactly this issue discusses, but if we wanted to add an API to the stdlib html parser to help it serve as a working example for this type of application I don't see why not. It would require someone (not me) to design that API and implement it, and some core dev (also not me) to approve it. I'm not sure if it would be required to spec this out in the PEP; my hunch is not (the PEP should focus on specifying the feature itself and providing a motivation so people understand how useful it might be). Honestly the stdlib html module seems a bit outdated, and there might be better 3rd party options available; we could see if one of those is interested in prototyping API support for tag strings.

pauleveritt · 2023-04-30T22:15:40Z

Let's close this ticket. I'm working on examples. I think I'll go in the direction of another example using a non-stdlib-htmlparser.

jimbaker mentioned this issue May 21, 2022

Example tags #8

Closed

11 tasks

rmorshea mentioned this issue May 23, 2022

Unify Component and VDOM Element Constructor Interface reactive-python/reactpy#755

Closed

rmorshea changed the title ~~Codegen function from a tag string to support IDOM virtual DOM~~ Codegen function from a tag string to support ReactPy's virtual DOM Apr 30, 2023

gvanrossum closed this as completed Apr 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Codegen function from a tag string to support ReactPy's virtual DOM #9

Codegen function from a tag string to support ReactPy's virtual DOM #9

jimbaker commented May 21, 2022 •

edited by rmorshea

Loading

jimbaker commented May 21, 2022

jimbaker commented May 21, 2022

jimbaker commented May 21, 2022

gvanrossum commented May 21, 2022

jimbaker commented May 21, 2022

gvanrossum commented May 21, 2022

jimbaker commented May 23, 2022

gvanrossum commented May 23, 2022

rmorshea commented Jun 1, 2022 •

edited

Loading

jimbaker commented Jun 1, 2022

rmorshea commented Jun 2, 2022

jimbaker commented Jun 2, 2022 •

edited

Loading

gvanrossum commented Jun 2, 2022

pauleveritt commented Apr 30, 2023

gvanrossum commented Apr 30, 2023

pauleveritt commented Apr 30, 2023

Codegen function from a tag string to support ReactPy's virtual DOM #9

Codegen function from a tag string to support ReactPy's virtual DOM #9

Comments

jimbaker commented May 21, 2022 • edited by rmorshea Loading

jimbaker commented May 21, 2022

jimbaker commented May 21, 2022

jimbaker commented May 21, 2022

gvanrossum commented May 21, 2022

jimbaker commented May 21, 2022

gvanrossum commented May 21, 2022

jimbaker commented May 23, 2022

gvanrossum commented May 23, 2022

rmorshea commented Jun 1, 2022 • edited Loading

jimbaker commented Jun 1, 2022

rmorshea commented Jun 2, 2022

jimbaker commented Jun 2, 2022 • edited Loading

gvanrossum commented Jun 2, 2022

pauleveritt commented Apr 30, 2023

gvanrossum commented Apr 30, 2023

pauleveritt commented Apr 30, 2023

jimbaker commented May 21, 2022 •

edited by rmorshea

Loading

rmorshea commented Jun 1, 2022 •

edited

Loading

jimbaker commented Jun 2, 2022 •

edited

Loading