Support simplified z-axis

[christianparpart]

An arbitrary Z-axis like Kitty would probably be overkill, especially because it puts a high burden on terminal authors that do not have a 3D API like OpenGL for rendering.

But reducing it to the minimal useful set should get the most out of it.

• render above text (basically like Sixel)
• replace text (EDIT: turns out this isn't easy for all TE devs? subject to removal)
• below text

For maximum compatibility with existing VT sequences, any screen buffer mutating VT sequence that affects text, will also affect image fragments. If a column is moved, its image fragment (regardless of z-axis) will be moved, too.

[christianparpart]

@dankamongmen, if I remember correctly (TMT), you where one amongst those raising interesting in having the ability to not just draw text above text, but also below. As I said, I generally dislike the idea of introducing an arbitrary z-axis due to its implied complexity on the TE side, I wanted to provide something similar, yet much simpler. Would you (in the context of notcurses and friends) be fine with above/below text for images, or would you want to need more than that?

[j4james]

In case you're looking for other ideas for this, my plan for mixed text/image layering in Windows Terminal was to add support for DEC's overstrike mode (?20), from the VK100. Architecturally there would still just be two layers - the text layer, and an image layer (with transparency). Regular text goes to the text layer, sixel images go to the image layer, and overstrike text is also written to the image layer.

That gives you the ability to layer text over images over text as many times as you want, because the sixel and overstrike operations write directly to the image layer, merging with whatever is already there. And when you want to clear a cell, you just write out regular text, which will erase the image layer above it (exactly as it already works with sixel).

The only limitation I can see with this approach is that you can't update layers independently of each other. But if an application really needed to do that, they could manage the layering themselves, and repaint the cell from the bottom up. And if you're concerned that would causing flickering, you can always implement double buffering using the paging APIs, and/or DECCRA.

Obviously you're building a brand new image protocol here, so I'm not suggesting you follow this exact approach. But my point is that a simple two-layer architecture of images over text should give apps everything they need, as long as you also have a mechanism for writing text to the image layer.

And any TEs that already supports sixel correctly should be able to support something like this without any changes to their architecture.