Add utilities to convert from html to htpy #26

OleJoik · 2024-06-10T22:31:53Z

Convert HTML to htpy code

Maybe you already have a bunch of HTML, or templates that you would like to migrate to htpy.
We got you covered. The utility command html2htpy ships with htpy, and can be used to transform existing
html into Python code (htpy!).

$ html2htpy -h
usage: html2htpy [-h] [-f {auto,ruff,black,none}] [-i {yes,h,no}] [--no-shorthand] [input]

positional arguments:
  input                 input HTML from file or stdin

options:
  -h, --help            show this help message and exit
  -f {auto,ruff,black,none}, --format {auto,ruff,black,none}
                        Select one of the following formatting options: auto, ruff, black or none
  -i {yes,h,no}, --imports {yes,h,no}
                        Output mode for imports of found htpy elements
  --no-shorthand        Use explicit `id` and `class_` kwargs instead of the shorthand #id.class syntax

Lets say you have an existing HTML file:

<!-- index.html -->

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>htpy Recipes</title>
</head>
<body>
    <div id="header">
        <h1>Welcome to the cooking site</h1>
        <p>Your go-to place for delicious recipes!</p>
    </div>

    <div id="recipe-of-the-day" class="section">
        <h2>Recipe of the Day: <span class="highlight">Spaghetti Carbonara</span></h2>
        <p>This classic Italian dish is quick and easy to make.</p>
    </div>

    <div id="footer">
        <p>&copy; 2024 My Cooking Site. All rights reserved.</p>
    </div>
</body>
</html>

Now, if you run the command, it outputs the corresponding Python code (htpy).

$  html2htpy index.html

from htpy import body, div, h1, h2, head, html, meta, p, span, title

html(lang="en")[
    head[
        meta(charset="UTF-8"),
        meta(name="viewport", content="width=device-width, initial-scale=1.0"),
        title["htpy Recipes"],
    ],
    body[
        div("#header")[
            h1["Welcome to the cooking site"], p["Your go-to place for delicious recipes!"]
        ],
        div("#recipe-of-the-day.section")[
            h2["Recipe of the Day: ", span(".highlight")["Spaghetti Carbonara"]],
            p["This classic Italian dish is quick and easy to make."],
        ],
        div("#footer")[p["© 2024 My Cooking Site. All rights reserved."]],
    ],
]

Piping input/stdin stream

You can also pipe input to htpy, for example cat demo.html | html2htpy.

This can be combined with other workflows in the way that you find most suitable.
For example, you might pipe from your clipboard to htpy, and optionally direct the output to a file.

Linux

xclip -o -selection clipboard | html2htpy > output.py

Mac

pbpaste | html2htpy > output.py

Windows

powershell Get-Clipboard | html2htpy > output.py

Formatting the output

html2htpy can format the output Python code using black or ruff.
Select the preferred formatter with the -f/--format flag. Options are auto, ruff, black and none.

By default, the selection will be auto, formatting if it finds a formatter on path, preffering ruff if it's available.
If no formatters are available on path, the output will not be formatted.

Import options

You have a couple of options regarding imports with the -i/--imports flag.
Options are yes (default), h, no.

Module import of htpy: `--imports=h`

Some people prefer to import htpy as h instead of importing individual elements from htpy.
If this is you, you can use the --imports=h option to get corresponding output when using html2htpy.

$ html2htpy --imports=h example.html

import htpy as h

h.section("#main-section.hero.is-link")[
    h.p(".subtitle.is-3.is-spaced")["Welcome"]
]

Explicit id and class kwargs

If you prefer the explicit id="id", class_="class" kwargs syntax over the default htpy shorthand #id.class syntax, you can get it by passing the --no-shorthand flag.

<!-- example.html -->
<section id="main-section" class="hero is-link">
    <p class="subtitle is-3 is-spaced">Welcome</p>
</section>

Default shorthand yield `#id.class`

$ html2htpy example.html

from htpy import p, section

section("#main-section.hero.is-link")[
    p(".subtitle.is-3.is-spaced")["Welcome"]
]

No shorthand yields kwargs `id`, `class_`

$ html2htpy --no-shorthand example.html

from htpy import p, section

section(id="main-section", class_="hero is-link")[
    p(class_="subtitle is-3 is-spaced")["Welcome"]
]

Template interpolation to f-strings

html2htpy will try to convert template variables to pythonic f-strings:

template {{ variables }} -> f"template { variables }"

Note that other typical template syntax, such as loops {% for x in y %}, can not be transformed this way,
so you will often have to clean up a bit after html2htpy is done with its thing.

See the example below:

<!-- jinja.html -->
<body>
    <h1>{{ heading }}</h1>
    <p>Welcome to our cooking site, {{ user.name }}!</p>

    <h2>Recipe of the Day: {{ recipe.name }}</h2>
    <p>{{ recipe.description }}</p>

    <h3>Instructions:</h3>
    <ol>
        {% for step in recipe.steps %}
        <li>{{ step }}</li>
        {% endfor %}
    </ol>
</body>

$ html2htpy jinja.html

from htpy import body, h1, h2, h3, li, ol, p

body[
    h1[f"{ heading }"],
    p[f"Welcome to our cooking site, { user.name }!"],
    h2[f"Recipe of the Day: { recipe.name }"],
    p[f"{ recipe.description }"],
    h3["Instructions:"],
    ol[
        """        {% for step in recipe.steps %}        """,
        li[f"{ step }"],
        """        {% endfor %}    """,
    ],
]

OleJoik · 2024-06-10T22:59:39Z

Might be a better idea to have it parse files than to paste into the terminal, bit of jank but atleast it works.

pelme

This is GREAT! Thanks a lot for this nice command! Really looking forward to be using this.

htpy/cli/cli.py

tests/test_convert_html.py

htpy/utils/html_to_htpy.py

tests/test_convert_html.py

pelme

I had another look and left some comments. I think we should have a separate page in the docs with some examples too :)

htpy/__init__.py

tests/test_convert_html.py

OleJoik · 2024-06-11T20:26:13Z

Alright, I think I'm happy with the current state at this point. I've taken the liberty to resolve some of the comment threads (don't know if that is bad practice of me to do..?), maybe you'll do a final sweep and clarify if there is anything else you want changed?

Ill add a simple docs page

OleJoik · 2024-06-11T20:33:48Z

One thing I thought about might be an option to add imports for used elements in the output. Might be nice to save even a couple of more seconds.

That should be quick to add if you want it, but might also be another PR perhaps?

htpy/html2htpy.py

OleJoik · 2024-06-11T22:13:00Z

Also added some docs for review!

OleJoik · 2024-06-12T11:12:44Z

I updated the formatting to use subprocess as per your suggestion. Seems to work great. Also removed the extras dependency group, as I suppose the users mostly can install formatters themselves?

pelme

Thanks a lot for putting in the effort to finalize this. The quality of docs and tests are really nice. This feels like a very nice addition and it works really smooth!

docs/html2htpy.md

htpy/html2htpy.py

docs/html2htpy.md

htpy/html2htpy.py

tests/test_html2htpy.py

OleJoik · 2024-06-12T18:51:12Z

Alright, I think I've addressed the comments.

Thank you for all the thorough feedback and timely followup by the way.

OleJoik · 2024-06-12T19:59:02Z

There was this little thing I wanted to add:

Detect htpy imports

If you pass the -i/--imports flag, htpy elements detected will be included as
imports in the output. For example:

$ html2htpy --imports example.html

from htpy import p, section

section("#main-section.hero.is-link")[
    p(".subtitle.is-3.is-spaced")["Welcome"]
]

In the end, I couldn't help myself, so I added it now. Ill stop adding new things now, promise.

pelme · 2024-06-12T20:28:29Z

There was this little thing I wanted to add:

Detect htpy imports

If you pass the -i/--imports flag, htpy elements detected will be included as imports in the output. For example:

$ html2htpy --imports example.html
from htpy import p, section

section("#main-section.hero.is-link")[
    p(".subtitle.is-3.is-spaced")["Welcome"]
]
In the end, I couldn't help myself, so I added it now. Ill stop adding new things now, promise.

I like this a lot!! I also think that the imports should be added by default. Most people will probably pbpaste | html2htpy anyways and then it is easy to just not copy the import if it is not needed anyways.

Lately I have been using import htpy as h and find it nice. No need to fiddle with imports all the time.

What do you think about --imports=yes|h|no with yes being the default: from htpy import div, img, .... And --imports=hwould be import htpy as h / h.div[h.span["Hi"]]. --imports=no would be no imports at all.

We can add --imports=h later after this is merged but that would allow us to add it going forward.

OleJoik · 2024-06-12T22:03:19Z

I added it!

Import options

You have a couple of options regarding imports with the -i/--imports flag.
Options are yes (default), h, no.

Module import of htpy: `--imports=h`

Some people prefer to import htpy as h instead of importing individual elements from htpy.
If this is you, you can use the --imports=h option to get corresponding output when using html2htpy.

$ html2htpy --imports=h example.html

import htpy as h

h.section("#main-section.hero.is-link")[
    h.p(".subtitle.is-3.is-spaced")["Welcome"]
]

OleJoik · 2024-06-12T22:22:01Z

fyi, in addition to the unit tests, I did some manual testing with a bunch of large jinja templates that I had laying around, and it seems to behave well even for fairly ugly templates with lots of weird jinja shit.

OleJoik · 2024-06-12T23:01:03Z

I also did some proof-reading of the docs. Think I'm pretty much good from my side now.

pelme · 2024-06-13T05:43:58Z

Thanks A LOT for this and all the good discussions! I opened a PR with some small changes, please have a look #27 !

Add utilities to convert from html to htpy

22ef548

OleJoik mentioned this pull request Jun 10, 2024

HTML to HTPY converter for lazy people #25

Closed

pelme requested changes Jun 11, 2024

View reviewed changes

OleJoik added 5 commits June 11, 2024 17:42

Refactor cli and python interface into html2htpy

1c7335e

Minor refactor, use iterable instead of loop

698b4bf

Refactor tests (hardcoded expected, no formatting)

b1c5aeb

Undo unintentional formatting of __init__.py

3cf0827

String content escaping

98f283f

pelme requested changes Jun 11, 2024

View reviewed changes

htpy/__init__.py Outdated Show resolved Hide resolved

tests/test_convert_html.py Outdated Show resolved Hide resolved

tests/test_convert_html.py Outdated Show resolved Hide resolved

OleJoik added 2 commits June 11, 2024 21:53

simplify cli app, input from file or stdin

c4b71a0

Remove the import of html2htpy in init file

158215b

pelme reviewed Jun 11, 2024

View reviewed changes

htpy/html2htpy.py Outdated Show resolved Hide resolved

OleJoik added 3 commits June 11, 2024 23:07

Update import of html2htpy in test

bebe80e

Bugfix: accept jinja style templates w/period

1f93e2e

Docs for html2htpy

d1efa4b

OleJoik added 2 commits June 12, 2024 08:49

Removing select.select when reading stdin

731c97e

Update Formatting, -f auto/ruff/black/none from path

c60dd1e

pelme requested changes Jun 12, 2024

View reviewed changes

OleJoik added 7 commits June 12, 2024 18:28

Default shorthand syntax, --explicit -> kwargs id, class_

f91345f

Refactor, more descriptive function name

50c893f

fix failing test

b6bd44f

Avoid intermediate var in _get_formatter, immediate returns

8784204

html -> HTML, HTPY -> htpy, python -> Python

5d963a9

Get rid of __name__ == "__main__" in html2htpy

02af0f6

Adding black as dev dependency

32b5ca6

OleJoik added 6 commits June 12, 2024 19:38

Updates to docs after changes to --shorthand flag

d278063

Update tests for html2htpy

9bc3b20

Ruff lint --fix and ruff format

a5f799e

mypy lint

241e7ca

Remove 'Self', use 'Any'. Python 3.10 compatible

26b94fa

Ruff formatting

18355e6

OleJoik requested a review from pelme June 12, 2024 18:50

-i: Flag to include import htpy elements in output

7c00ffa

Remove "expected formatting" from test

b59ea12

OleJoik added 2 commits June 12, 2024 23:17

Bugfix: correct handling of void elements without / in endtag

be15515

Additional import options: notably --imports=h

0727a71

OleJoik added 4 commits June 13, 2024 00:38

--no-shorthand instead of --explicit options

5ac1e7a

Fix minor outdated info in docs

b0944d3

Fix typo in docs

2f2cabc

Another typo in docs

543b80d

Remove rogue print()

d79f42e

pelme merged commit b9d6335 into pelme:main Jun 13, 2024
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add utilities to convert from html to htpy #26

Add utilities to convert from html to htpy #26

OleJoik commented Jun 10, 2024 •

edited

Loading

OleJoik commented Jun 10, 2024

pelme left a comment

pelme left a comment

OleJoik commented Jun 11, 2024

OleJoik commented Jun 11, 2024

OleJoik commented Jun 11, 2024

OleJoik commented Jun 12, 2024

pelme left a comment

OleJoik commented Jun 12, 2024

OleJoik commented Jun 12, 2024 •

edited

Loading

pelme commented Jun 12, 2024

Detect htpy imports

OleJoik commented Jun 12, 2024

OleJoik commented Jun 12, 2024

OleJoik commented Jun 12, 2024

pelme commented Jun 13, 2024

Add utilities to convert from html to htpy #26

Add utilities to convert from html to htpy #26

Conversation

OleJoik commented Jun 10, 2024 • edited Loading

Convert HTML to htpy code

Piping input/stdin stream

Linux

Mac

Windows

Formatting the output

Import options

Module import of htpy: --imports=h

Explicit id and class kwargs

Default shorthand yield #id.class

No shorthand yields kwargs id, class_

Template interpolation to f-strings

OleJoik commented Jun 10, 2024

pelme left a comment

Choose a reason for hiding this comment

pelme left a comment

Choose a reason for hiding this comment

OleJoik commented Jun 11, 2024

OleJoik commented Jun 11, 2024

OleJoik commented Jun 11, 2024

OleJoik commented Jun 12, 2024

pelme left a comment

Choose a reason for hiding this comment

OleJoik commented Jun 12, 2024

OleJoik commented Jun 12, 2024 • edited Loading

Detect htpy imports

pelme commented Jun 12, 2024

Detect htpy imports

OleJoik commented Jun 12, 2024

Import options

Module import of htpy: --imports=h

OleJoik commented Jun 12, 2024

OleJoik commented Jun 12, 2024

pelme commented Jun 13, 2024

OleJoik commented Jun 10, 2024 •

edited

Loading

Module import of htpy: `--imports=h`

Default shorthand yield `#id.class`

No shorthand yields kwargs `id`, `class_`

OleJoik commented Jun 12, 2024 •

edited

Loading

Module import of htpy: `--imports=h`