[BUG] ITERM2 pixel mode not working in VS Code

[wabiloo]

What am I doing?

Trying to display images in ITERM2 pixel mode in the integrated terminal in VS Code.
I first set terminal.integrated.enableImages to True in the VS Code Settings (in line with https://code.visualstudio.com/docs/terminal/advanced#_image-support)
I checked that imgcat worked in the integrated terminal

However instead of an image, the python code returns and prints a long base64 string

Expected behavior

Image is displayed in full glory within the integration terminal

To reproduce

Steps to reproduce the behavior:

• Open the script below in VS Code
• Execute it
• Observe what happens in the integration terminal

Minimum working example

If you are able, try to provide code for a minimum working example that produces the bug.

from chafa import *
from PIL import Image

# Init canvas config
config = CanvasConfig()


# Init database and get terminal info
term_db = TermDb()
term_info = term_db.detect()

# Get terminal capabilities
capabilities = term_info.detect_capabilities()

# Set config to appropriate modes
config.canvas_mode = capabilities.canvas_mode
config.pixel_mode = capabilities.pixel_mode

config.pixel_mode = PixelMode.CHAFA_PIXEL_MODE_ITERM2

# Calculate the appropriate geometry for the canvas
config.calc_canvas_geometry(32, 18, 10 / 20)

# Open image with PIL
image = Image.open(
    "./snake.jpg"
)

width = image.width
height = image.height
bands = len(image.getbands())

# Put image into correct format
pixels = image.tobytes()

# Init the canvas
canvas = Canvas(config)

# Draw to canvas
canvas.draw_all_pixels(PixelType.CHAFA_PIXEL_RGB8, pixels, width, height, width * bands)

print(bands)

# Write picture
print(canvas.print(fallback=False).decode())

System info

Please provide the following details:

• OS: MacOS (ARM)
• Chafa.py version: 1.1.2
• Python version: 3.11.7

[GuardKenzie]

Hi! Sorry for the super duper late reply to this. I've been busy with life stuff so I couldn't get around to investigating.

I haven't been able to get the iterm2 image protocol to work in VSCode myself (maybe it has something to do with jerch/xterm-addon-image#57, I'm honestly not sure) but sixels seem to work! You need to change your fallback parameter to True in print(canvas.print(fallback=False).decode()) and, of course, change the pixel mode to sixels and it should work!

---

Modified code

from chafa import *
from PIL import Image

# Init canvas config
config = CanvasConfig()


# Init database and get terminal info
term_db = TermDb()
term_info = term_db.detect()

# Get terminal capabilities
capabilities = term_info.detect_capabilities()

# Set config to appropriate modes
config.canvas_mode = capabilities.canvas_mode
config.pixel_mode = capabilities.pixel_mode

config.pixel_mode = PixelMode.CHAFA_PIXEL_MODE_SIXELS #!!!

# Calculate the appropriate geometry for the canvas
config.calc_canvas_geometry(32, 18, 10 / 20)

# Open image with PIL
image = Image.open(
  "./snake.jpg"
)

width = image.width
height = image.height
bands = len(image.getbands())

# Put image into correct format
pixels = image.tobytes()

# Init the canvas
canvas = Canvas(config)

# Draw to canvas
canvas.draw_all_pixels(PixelType.CHAFA_PIXEL_RGB8, pixels, width, height, width * bands)

print(bands)

# Write picture
print(canvas.print(fallback=True).decode()) #!!!

Output:

[AnonymouX47]

For what it's worth, see the last section of jerch/xterm-addon-image#57 (comment) .

To be specific, it seems the issue in this case is the payload size.

xterm-addon-image has a iipSizeLimit constructor parameter, which is the byte length limit for an IIP sequence.

Once a sequence exceeds this limit, it seems the addon stops consuming output. Hence, probably the reason you get a base64 string emitted.

On chafa's end, the issue is uncompressed TIFF is currently used for IIP sequences (see chafa-iterm2-canvas.c). This results in really large payloads. @hpjansson is aware and willing to change to PNG but will require an added dependency.

Anyways, even if the payload size were to be reduced by reducing the image size, the image probably still wouldn't show up as I don't think the addon supports decoding the TIFF format.

NOTE: This is all just my theory based on the facts available. I haven't put it to the test.

Thank you 😃.

[GuardKenzie]

Ah I see, that makes sense! Thanks for the info 🧡