9.10a1 General Purpose Blocks Feedback

[facelessuser]

Documentation: https://facelessuser.github.io/pymdown-extensions/extensions/blocks/

The purpose of this discussion is to allow feedback for the latest iteration of General Purpose Blocks 9.10.a1. It should be noted that any and all feedback is welcome, but there is a focus on feedback specifically related to syntax. In particular, we have two main points we are hoping to get feedback on, but again, any feedback on syntax is welcome.

Currently, we have two general options that are only available during the alpha (and maybe beta) stage. By the final release, we'd like to remove these options with a permanent solution.

1. Which is preferred, syntax with ::: or ///? Currently, the option colon_syntax allows selecting between the current default syntax of /// and the colon syntax (:::). Which is generally preferred by the community?

2. The new general purpose Blocks extension allows YAML configs in block headers to set per block options. YAML fences are currently optional and can be omitted, but using require_yaml_fences can make them mandatory.

   The thinking is that some people may find including the YAML fences tedious, so allowing them to be optional allows a user not use them, but this requires us to enforce a new line between block headers and block content, headers in this sense meaning the start of the block and the config.

   /// name
   option: value
   
   content requires a new line ^here^
   ///
   
   /// name
   
   even when there is no config, we still need a new line ^here^
   ///
   

   Requiring YAML fences means we can easily identify when there is a config, so no new line is required. This means a user can still add a new line if desired, but if not desired, they can omit it.

   /// name
   ---
   option: value
   ---
   content can use a new line ^here^ or omit it
   ///
   
   /// name
   no config, no new line needed
   ///
   

[waylan]

Very Cool! I read through the docs here. Is that the current up-to-date version? Of course, a rendered version would be better, but I understand why you wouldn't want to override the current stable version docs.

Regarding the two requested points of feedback...

1. I prefer ///. 👍 from me.
2. I'm indifferent on this one. I see advantages to both. I say you get to paint the shed as you built it. While you could leave the config option in place, I don't think it makes sense to have two valid (but incompatible) syntax forms out there. I understand that being able to play with both during the alpha/beta stage is helpful, but I agree that a choice should be made before final release with only one option. Maybe if I get some time later, I'll install and play with it.

[facelessuser]

Is that the current up-to-date version? Of course, a rendered version would be better, but I understand why you wouldn't want to override the current stable version docs.

I've updated the opening post with a documentation link. Yes, it is rendered. I just didn't think to post the link here.

I understand that being able to play with both during the alpha/beta stage is helpful, but I agree that a choice should be made before final release with only one option. Maybe if I get some time later, I'll install and play with it.

Yep, I do not want to support two different methods. I'm mainly looking for good arguments to sway me one way or the other. I do have my personal leanings, and worst case, I'll just go with that. This just gives me a chance to hopefully understand people's largest concerns before I commit to a final solution.

Assuming we don't have any amazing ideas that cause me to rethink the syntax, the next step will be to formalize the API and document it. During that period, I'll be more focused on making sure there is sufficient flexibility to give people what they need, or handle things I just didn't think of.

I think we are on our way to a reasonable generic block format.

[2bndy5]

Upon reading the tab docs, I noticed that tab 2 in the "new tabbed group" example links to the previous example's tab 2 (instead of switching to the tab).

[facelessuser]

You know, I think it is specifically because I'm running the example in a sand box. A page will generate each tab set with a unique ID, but since they are both rendered in a sandbox, they get the same starting ID for the tab set. The main docs are still using the Tabbed extension, but I'm trying to demonstrate the new tab functionality in these sandboxes, so conflicts. I think I'll just render the result using Tabbed outside of a sandbox as the output is identical. That'll fix the ID issues.

[facelessuser]

Once I switch over to using the new general purpose blocks for the entire documentation, this won't be an issue anymore.

[2bndy5]

Sorry to "be that guy". I'm just super analytical...

[facelessuser]

No worries, this was a good catch. I pushed some sane rendering now.

I hadn't realized that rendering tabs examples in sandboxes would cause this issue, but IDs need to be unique across the page, and if they aren't, well, you see what happens. I also have to run the general blocks in a sandbox for rendering as I don't have them loaded by default for the site yet, so they are done on demand in examples.

Docs should be fixed once GitHub refreshes. The examples get the point across now and work as expected. This is a non-issue once I formally use the new general-purpose blocks site-wide.

[2bndy5]

Yep, that fixed it! Thanks for being so incredibly responsive!

[squidfunk]

One quick thing I immediately noticed – the front matter separator --- induces a h2 element for when the extension is not enabled or supported, e.g. on GitHub Pages. This leads to the problem that the element is even listed in the table of contents.

Example

# Lorem ipsum

Lorem ipsum dolor sit amet.

/// details | Some summary
---
type: warning
---

Some content
///

Result

[waylan]

One of the benefits of prefixing every line of the config is that the entire config gets treated the same by other tools/parsers. An added benefit of using indentation as the prefix is that it is easier to work with in editors as they have built-in support for it (with the exception of web forms). So, if the concern is how other tools/parsers treat the blocks, then indentation is the best option IMO. However, if the desire is ease to create, then prefixes are not so good (including indentation when working in web forms). In that case, full line delimitators are better, whether those be blank lines or lines of repeating characters. Its a matter of priorities.

As YAML list items will get parsed as Markdown list items by other parsers, I don't see any reason to be concerned about the delimitators being parsed as HRs. In other words, if we use delimitators, then lets just stick with YAML's defaults (---). However, if that is undesirable, then per-line prefixes is the only reasonable way to avoid that, which means indentation is the way to go IMO.

[facelessuser]

I realize that we cannot fully prevent Markdown parsers and syntax highlighters from accidentally parsing or highlighting something in the config that we don't intend it to, but I do think the fence solution allows for far more egregious issues. I do realize now that switching to ... from --- only provides very minimal improvements, so if we stick with YAML fences, --- is really not that much worse.

The intent is not to completely be compatible with the existing Markdown. The big thing with general blocks is to:

1. Make creating special blocks easy and consistent so we don't have to invent various new syntaxes with all the related issues in doing that.
2. To prevent deeply nested indentation which can currently happen with admonitions, lists, and other constructs.
3. To minimize how painful it is for people to write using the special Markdown in their editors.
4. To a lesser extent, it would be nice to minimize the pain of looking at a non-standard Markdown in a GitHub repo, etc.

So, with all of that in mind, I think some sort of per-line delimiter makes sense. I am more than happy to adopt indentation as it will not propagate deeply nested constructs as it is only a syntax to continue the header of these blocks to define per-block options. It will be interesting to get opinions from the larger audience, but I think I'm okay with an indentation for config only. Worst case, if the majority of people really hate indentation, a special line character can be considered. This may be one of these things where whatever solution we settle on will make someone unhappy, but I'm to start with something and see where the conversation goes.

I think in alpha 2, I'll remove the YAML fence and the related option and try out indented configs and get feedback.

[ofek]

per-line delimiter

Example?

[facelessuser]

@ofek It would be something like what is shown below. You extend the header with some token:

/// name
/ key1: value
/ key2: value

content
///

[facelessuser]

When a place YAML fences in the bin, I may add a new alpha option to try out indentation vs line delimiter. That may help people get a feel for the two options so they can give feedback.

[vokimon]

I am biased towards how currently customblocks works, you know. So, take these proposals with a grain of salt.

Block marker

You ask for opinions. Well, I feel like ::: is nicer as /// is a common sequence in code comments. But no strong opinion here. Pandoc and R markdown use ::: for custom blocks. It is a similar use case but not exactly the same one, they just generate divs you can populate with attributes with the {} syntax. I use /// in the examples bellow just to use the current default.

Key parameters

Fenced or unfenced YAML? Neither. I do love YAML, but i feel that having any named/optional parameter specified as yaml metadata in every block type is too verbose. Even more verbose if we finally use fences for that. I feel like key parameters could be better solved inline, like:

/// blocktype key=val key2="val with spaces"
content
///

For long parameter lists, still we could give the option to split them in lines by using, for example, the line continuation escaping, which is a common idiom in many languages (not currently supported in customblocks but in the TODO list):

/// blocktype \
  key=val \
  key2="val with spaces"
content
///

I know that, by removing yaml parameters, we left an scenario uncovered: having structured data. But, i feel that they will be the 2% or less of the cases, and we are messing everybody with verbose syntax. If your block type really needs structured data, let block-type authors consider the yaml part of their content. We could even provide them a helper (say extract_metadata(content, fenced=true) ) for extension authors to use it if they need it. But, for simpler cases (the 98%), it could be resolved with a single line and not being that verbose.

Positional parameters

Regarding the positional parameters, for me the | is spurious. Whatever comes after the first word indicating the type, should be a parameter. To provide more than one parameter, I like bash like parameter passing is simple and a common ground: Split by words and quote multiword values.

Both, backslash line continuation and word parameter separation with quoting for spaces are common syntax patterns that many people are used to. Indeed yaml use them.

Flag parameters

Taking as reference what you can do with customblocks, I also miss flags (flag, noflag) which provide a natural talk like interface. An example:

/// figure myimage.jpg width=200px nolightbox thumb
This is the **caption**
///

• myimage.jpg is a positional parameter, the obvious parameter
• width is a regular key parameter
• nolightbox gives false to lightbox flag
• thumb gives true to thumb flag

Single line closing

Whenever you don't need any content the block gets kind of ugly.

/// figure myimage.jpg width=200px nolightbox thumb
///

An idea could be having single line closing if you do not need any content:

/// figure myimage.jpg width=200px nolightbox thumb ///

Of course, if content is to be indented neither option would be necessary.

Escaping closing tag

Also because the content is at the same indentation level, we could end up having some content containing ///.
Indeed whenever you have an example of the extension syntax.
I guess you solved it as the examples should be documented somehow but i couldn't find the examples.

Just some ideas.

[facelessuser]

Is that really the case? If it is, I was not aware. I will need to re-evaluate. It is a general requirement in a number of Markdown parsers I am aware of, so I assumed it would be in Python Markdown as well. As far as SuperFences is concerned, I think it is a requirement, so it is at least consistent with that plugin, which I maintain. See below in GitHub, reversing does not work.

```
````
this
````
```

EDIT: had to remove the visual rendering as it was pretty broken.

Regardless, I will document it for clarity.

[waylan]

As a reminder, Python-Markdown and PHP Markdown Extra were the very first implementations of Fenced Code Blocks. Michelf and and I came up with the syntax together (with input from others) and implemented it. We never discussed or implemented such a restriction. It would seem that the restriction was added later by other implementations. Personally, I was not aware that such a restriction was in place anywhere until you brought it to my attention. I still maintain that such a restriction should not exist.

[facelessuser]

While I do understand this now, I will have to consider the implications of relaxing this constraint in things I support. As far as things are right now, the outer fence of both Blocks and SuperFences hold to this restriction. I won't promise a relaxing of this, only that I'll look into it and consider this change as a possibility.

It is quite likely that this is a painless change, but I'll have to consider it carefully. I do imagine there is a likely possibility of breakage.

[facelessuser]

Okay, I was wrong, SuperFences does allow larger fences under smaller, the only requirement is that they are different. So, it would be reasonable to match that in Blocks.

My syntax highlighter hates it though, so I imagine this might be why I never use fences in this direction:

[facelessuser]

Next alpha will allow nesting of any length of fences as long as they differ from the current one. While the example below only shows escaping YAML config with /, there will be an alpha option to use indentation, I just haven't added it yet. There will also be some minor nesting fixes

[facelessuser]

Discussion is moving to the new Alpha 2 discussion thread: #1883. Please use the new thread for further discussions. I will be releasing the alpha 2 shortly.