WIP: add option to output xexpr of html #498

stevebyan · 2025-04-16T17:06:24Z

Output the body as an xexpr, and provide various metadata, including TOC links, in a struct rather than in html.

I've refactored html-render.rkt to optionally output the body of the document as an xexpr rather than as html, and provide document metadata including the TOC links in a struct rather than in html. My goal is to allow use of scribble's html in something like Greg Hendershott's Tadpole, and so I want to provide the document content in a form that can be applied to an xexpr template. I also want to provide the TOC links in a form that can be rendered by site templates using pure CSS to provide a responsive view of the navigation links.

I'm submitting this pull request to gather some feedback on my approach, before committing to doing more work to make this something actually mergeable into Scribble. To pull out the TOC as data rather than as html, I've duplicated a lot of code from the render-toc-view and render-onthispage-contents methods. A finished pull request would refactor render-toc-view and render-onthispage-contents to use the output of my list-of-toc-view and list-of-onthispage-contents. I'm also not sure at this point how I want to represent the TOC contents. Using structs would probably be better than the current hacked nested lists.

Is this a feature the Racket developers would consider including in Scribble? Except for the duplicated code, does my implementation approach look reasonable?

Output the body as an xexpr, and provide various metadata, including TOC links, in a struct rather than in html.

github-actions

Resyntax analyzed 4 files in this pull request and has added suggestions.

github-actions · 2025-04-18T14:39:49Z

scribble-lib/scribble/html-render.rkt

+                  (map (lambda (p) (if (or (part-whole-page? p ri) 
+                                           (and (part-style? p 'toc-hidden)
+                                                (all-toc-hidden? p)))
+                                       null
+                                       (flatten p prefixes #f)))
+                       (part-parts d)))))))


map-to-for: This map operation can be replaced with a for/list loop.

Suggested change

(map (lambda (p) (if (or (part-whole-page? p ri)

(and (part-style? p 'toc-hidden)

(all-toc-hidden? p)))

null

(flatten p prefixes #f)))

(part-parts d)))))))

(for/list ([p (in-list (part-parts d))])

(if (or (part-whole-page? p ri)

(and (part-style? p 'toc-hidden) (all-toc-hidden? p)))

null

(flatten p prefixes #f))))))))

Debugging details

Textual replacement

(line-replacement #:new-lines '#(" (for/list ([p (in-list (part-parts d))])" " (if (or (part-whole-page? p ri)" " (and (part-style? p 'toc-hidden) (all-toc-hidden? p)))" " null" " (flatten p prefixes #f))))))))") #:original-lines '#(" (map (lambda (p) (if (or (part-whole-page? p ri) " " (and (part-style? p 'toc-hidden)" " (all-toc-hidden? p)))" " null" " (flatten p prefixes #f)))" " (part-parts d)))))))") #:start-line 854)

Syntactic replacement

(syntax-replacement #:introduction-scope #<procedure:...and/syntax-local.rkt:148:2> #:new-syntax #<syntax:/home/runner/.local/share/racket/8.17.0.1/pkgs/resyntax/default-recommendations/for-loop-shortcuts.rkt:149:2 (for/list ((p (in-list (part-parts d)))) (if (or (part-whole-page? p ri) (and (part-style? p (quote toc-hidden)) (all-toc-hidden? p))) null (flatten p prefixes #f)))> #:original-syntax #<syntax:scribble-lib/scribble/html-render.rkt:854:18 (map (lambda (p) (if (or (part-whole-page? p ri) (and (part-style? p (quote toc-hidden)) (all-toc-hidden? p))) null (flatten p prefixes #f))) (part-parts d))> #:source (file-source #<path:/home/runner/work/scribble/scribble/scribble-lib/scribble/html-render.rkt>))

I’d prefer not to change any pre-existing logic in this commit, unless mflatt requests it.

github-actions · 2025-04-18T14:39:50Z

scribble-lib/scribble/html-render.rkt

+               (map (lambda (p)
+                      (let ([p (vector-ref p 0)]
+                            [prefixes (vector-ref p 1)]
+                            [from-d (vector-ref p 2)]
+                            [add-tag-prefixes
+                             (lambda (t prefixes)
+                               (if (null? prefixes)
+                                   t
+                                   (cons (car t) (append prefixes (cdr t)))))])
+                        (list
+                         (if (part? p)
+                             (format-number
+                              (collected-info-number
+                               (part-collected-info p ri))
+                              null)
+                             null)
+                         (if (toc-element? p)
+                             (render-content (toc-element-toc-content p)
+                                             from-d ri)
+                             (parameterize ([current-no-links #t]
+                                            [extra-breaking? #t])
+                               (list 
+                                (uri-unreserved-encode
+                                 (anchor-name
+                                  (add-tag-prefixes
+                                   (tag-key (if (part? p)
+                                                (car (part-tags/nonempty p))
+                                                (target-element-tag p))
+                                            ri)
+                                   prefixes)))
+                                (cond
+                                  [(part? p) "tocsubseclink"]
+                                  [any-parts? "tocsubnonseclink"]
+                                  [else "tocsublink"])
+
+                                (render-content
+                                 (if (part? p)
+                                     (strip-aux
+                                      (or (part-title-content p)
+                                          "???"))
+                                     (if (toc-target2-element? p)
+                                         (toc-target2-element-toc-content p)
+                                         (element-content p)))
+                                 from-d ri)))))))
+                    ps)))))


map-to-for: This map operation can be replaced with a for/list loop.

Suggested change

(map (lambda (p)

(let ([p (vector-ref p 0)]

[prefixes (vector-ref p 1)]

[from-d (vector-ref p 2)]

[add-tag-prefixes

(lambda (t prefixes)

(if (null? prefixes)

t

(cons (car t) (append prefixes (cdr t)))))])

(list

(if (part? p)

(format-number

(collected-info-number

(part-collected-info p ri))

null)

null)

(if (toc-element? p)

(render-content (toc-element-toc-content p)

from-d ri)

(parameterize ([current-no-links #t]

[extra-breaking? #t])

(list

(uri-unreserved-encode

(anchor-name

(add-tag-prefixes

(tag-key (if (part? p)

(car (part-tags/nonempty p))

(target-element-tag p))

ri)

prefixes)))

(cond

[(part? p) "tocsubseclink"]

[any-parts? "tocsubnonseclink"]

[else "tocsublink"])

(render-content

(if (part? p)

(strip-aux

(or (part-title-content p)

"???"))

(if (toc-target2-element? p)

(toc-target2-element-toc-content p)

(element-content p)))

from-d ri)))))))

ps)))))

(for/list ([p (in-list ps)])

(let ([p (vector-ref p 0)]

[prefixes (vector-ref p 1)]

[from-d (vector-ref p 2)]

[add-tag-prefixes (lambda (t prefixes)

(if (null? prefixes)

t

(cons (car t) (append prefixes (cdr t)))))])

(list (if (part? p)

(format-number (collected-info-number (part-collected-info p ri)) null)

null)

(if (toc-element? p)

(render-content (toc-element-toc-content p) from-d ri)

(parameterize ([current-no-links #t]

[extra-breaking? #t])

(list (uri-unreserved-encode

(anchor-name (add-tag-prefixes

(tag-key (if (part? p)

(car (part-tags/nonempty p))

(target-element-tag p))

ri)

prefixes)))

(cond

[(part? p) "tocsubseclink"]

[any-parts? "tocsubnonseclink"]

[else "tocsublink"])

(render-content (if (part? p)

(strip-aux (or (part-title-content p) "???"))

(if (toc-target2-element? p)

(toc-target2-element-toc-content p)

(element-content p)))

from-d

ri)))))))))))

Debugging details

Textual replacement

(line-replacement #:new-lines '#(" (for/list ([p (in-list ps)])" " (let ([p (vector-ref p 0)]" " [prefixes (vector-ref p 1)]" " [from-d (vector-ref p 2)]" " [add-tag-prefixes (lambda (t prefixes)" " (if (null? prefixes)" " t" " (cons (car t) (append prefixes (cdr t)))))])" " (list (if (part? p)" " (format-number (collected-info-number (part-collected-info p ri)) null)" " null)" " (if (toc-element? p)" " (render-content (toc-element-toc-content p) from-d ri)" " (parameterize ([current-no-links #t]" " [extra-breaking? #t])" " (list (uri-unreserved-encode" " (anchor-name (add-tag-prefixes" " (tag-key (if (part? p)" " (car (part-tags/nonempty p))" " (target-element-tag p))" " ri)" " prefixes)))" " (cond" " [(part? p) \"tocsubseclink\"]" " [any-parts? \"tocsubnonseclink\"]" " [else \"tocsublink\"])" " (render-content (if (part? p)" " (strip-aux (or (part-title-content p) \"???\"))" " (if (toc-target2-element? p)" " (toc-target2-element-toc-content p)" " (element-content p)))" " from-d" " ri)))))))))))") #:original-lines '#(" (map (lambda (p)" " (let ([p (vector-ref p 0)]" " [prefixes (vector-ref p 1)]" " [from-d (vector-ref p 2)]" " [add-tag-prefixes" " (lambda (t prefixes)" " (if (null? prefixes)" " t" " (cons (car t) (append prefixes (cdr t)))))])" " (list" " (if (part? p)" " (format-number" " (collected-info-number" " (part-collected-info p ri))" " null)" " null)" " (if (toc-element? p)" " (render-content (toc-element-toc-content p)" " from-d ri)" " (parameterize ([current-no-links #t]" " [extra-breaking? #t])" " (list " " (uri-unreserved-encode" " (anchor-name" " (add-tag-prefixes" " (tag-key (if (part? p)" " (car (part-tags/nonempty p))" " (target-element-tag p))" " ri)" " prefixes)))" " (cond" " [(part? p) \"tocsubseclink\"]" " [any-parts? \"tocsubnonseclink\"]" " [else \"tocsublink\"])" " " " (render-content" " (if (part? p)" " (strip-aux" " (or (part-title-content p)" " \"???\"))" " (if (toc-target2-element? p)" " (toc-target2-element-toc-content p)" " (element-content p)))" " from-d ri)))))))" " ps)))))") #:start-line 866)

Syntactic replacement

(syntax-replacement #:introduction-scope #<procedure:...and/syntax-local.rkt:148:2> #:new-syntax #<syntax:/home/runner/.local/share/racket/8.17.0.1/pkgs/resyntax/default-recommendations/for-loop-shortcuts.rkt:149:2 (for/list ((p (in-list ps))) (let ((p (vector-ref p 0)) (prefixes (vector-ref p 1)) (from-d (vector-ref p 2)) (add-tag-prefixes (lambda (t prefixes) (if (null? prefixes) t (cons (car t) (append prefixes (cdr t))))))) (list (if (part? p) (format-number (...> #:original-syntax #<syntax:scribble-lib/scribble/html-render.rkt:866:15 (map (lambda (p) (let ((p (vector-ref p 0)) (prefixes (vector-ref p 1)) (from-d (vector-ref p 2)) (add-tag-prefixes (lambda (t prefixes) (if (null? prefixes) t (cons (car t) (append prefixes (cdr t))))))) (list (if (part? p) (format-number (collected-in...> #:source (file-source #<path:/home/runner/work/scribble/scribble/scribble-lib/scribble/html-render.rkt>))

I’d prefer not to change any pre-existing logic in this commit, unless mflatt requests it.

scribble-lib/scribble/html-render.rkt

stevebyan · 2025-04-18T16:52:36Z

Regarding the build failures, while I prefer human-readable output files, if you don’t want to add fmt as a build dependency, I can just remove it from Scribble and add it to a build pass in my blog makefile.

samth · 2025-04-18T18:35:35Z

For human-readable sexp output, just use pretty-write from racket/pretty instead of using fmt.

stevebyan · 2025-04-18T20:35:43Z

For human-readable sexp output, just use pretty-write from racket/pretty instead of using fmt.

Thanks, I’ll make that change.

Also fix logic for 'no-toc+aux

mflatt · 2025-05-21T21:30:41Z

This seems generally like a fine idea to me. I expected --xexpr to be an output option at the same level as --html, though. Would providing the choice that way match the intent?

stevebyan · 2025-05-22T02:19:18Z

I have intentions of supporting htmls and html-tree output as well as html. It was a little easier in the code to just add a single additional option used in addition to one of the html options, but it’s probably better to add xexprs and xexpr-tree options and not allow an html option.

stevebyan · 2025-06-05T11:19:21Z

I’ve just learned that xexprs are not fully serializable. A number of the structures in the representation are transparent rather than prefab. Greg Hendershott apparently gets away with writing and reading xexprs to a file in his tadpole blog system because, unlike Scribble, his Markdown source doesn’t introduce any html comments, CDATA, or PCDATA. I’m working on a separate pull request to enable serializing xexprs using read and write.

Add option to output xexpr of html

8cb44e5

Output the body as an xexpr, and provide various metadata, including TOC links, in a struct rather than in html.

samth assigned mflatt Apr 18, 2025

github-actions bot reviewed Apr 18, 2025

View reviewed changes

stevebyan added 4 commits April 27, 2025 11:42

Replace fmt with pretty-write

aa6ab9e

remove extraneous whitespace

9db1a06

Resyntax if-let-to-cond

55ca2ae

Also fix logic for 'no-toc+aux

Remove unused require

0233376

stevebyan added 2 commits May 27, 2025 09:46

Revise xexpr options

6d222da

Support multi-file xexpr output

95e9109

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

WIP: add option to output xexpr of html #498

WIP: add option to output xexpr of html #498

Uh oh!

stevebyan commented Apr 16, 2025 •

edited

Loading

Uh oh!

github-actions bot left a comment

Uh oh!

github-actions bot Apr 18, 2025

Uh oh!

stevebyan Apr 28, 2025

Uh oh!

github-actions bot Apr 18, 2025

Uh oh!

stevebyan Apr 28, 2025

Uh oh!

Uh oh!

stevebyan commented Apr 18, 2025

Uh oh!

samth commented Apr 18, 2025

Uh oh!

stevebyan commented Apr 18, 2025

Uh oh!

mflatt commented May 21, 2025

Uh oh!

stevebyan commented May 22, 2025

Uh oh!

stevebyan commented Jun 5, 2025

Uh oh!

Uh oh!

Uh oh!

WIP: add option to output xexpr of html #498

Are you sure you want to change the base?

WIP: add option to output xexpr of html #498

Uh oh!

Conversation

stevebyan commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 18, 2025

Choose a reason for hiding this comment

Uh oh!

stevebyan Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 18, 2025

Choose a reason for hiding this comment

Uh oh!

stevebyan Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stevebyan commented Apr 18, 2025

Uh oh!

samth commented Apr 18, 2025

Uh oh!

stevebyan commented Apr 18, 2025

Uh oh!

mflatt commented May 21, 2025

Uh oh!

stevebyan commented May 22, 2025

Uh oh!

stevebyan commented Jun 5, 2025

Uh oh!

Uh oh!

stevebyan commented Apr 16, 2025 •

edited

Loading