wide characters

[asottile]

paste this in a bunch and things are broken :'(

🙃

[asottile]

hmm wow I seem to have accidentally found a way to implement wcwidth that doesn't involve wacky tables:

import curses
import unicodedata

CHARS = '\u2603\u200b\U0001f643x'


def wcwidth(c):
    try:
        win = wcwidth._window
    except AttributeError:
        win = wcwidth._window = curses.newwin(1, 10)

    win.addstr(0, 0, c)
    _, x = win.getyx()
    return x


def c_main(stdscr):
    stdscr.addstr(1, 0, 'C\tWIDTH\tNAME')
    stdscr.addstr(2, 0, '====================================')
    for i, c in enumerate(CHARS):
        stdscr.addstr(i + 3, 0, f'{c}\t{wcwidth(c)}\t{unicodedata.name(c)}')
    stdscr.get_wch()


if __name__ == '__main__':
    curses.wrapper(c_main)

renders as:

C       WIDTH   NAME
====================================
☃       1       SNOWMAN                                                         ​
        0       ZERO WIDTH SPACE
🙃      2       UPSIDE-DOWN FACE
x       1       LATIN SMALL LETTER X

[jquast]

I might suggest this problem (and solution) is deeper than you may think, I wrote a detailed article about the latest release of the python wcwidth library and ucs-detect CLI tool: https://jeffquast.com/post/terminal_wcwidth_solution/

[asottile]

makes sense, I took a look at how ucs-detect works and I think the curses-wcwidth I implemented above uses a similar approach (write character, detect position)? do you have some test characters / terminals I can validate with?

I still haven't finished the implementation of this in the editor (it's ~partially implemented but there's some busted edge cases around even/odd and scrolling)

[jquast]

The test characters I use are in source, where only 1 character is selected, https://github.com/jquast/ucs-detect/blob/master/ucs_detect/__init__.py#L24-L35

But you can get the full list of characters, by version, using this CLI utility: https://github.com/jquast/wcwidth/blob/master/bin/new-wide-by-version.py

I have gone ahead and uploaded the output of that here, https://gist.github.com/jquast/80297f0e07b8d8401f19d1e2ad7ff650

which I display in terminals using:

import json

def main():
    data = json.load(open('data.json', 'r'))
    for version, vals in data.items():
        print(f'|--| {version}')
        for val in vals:
            print(f'|{chr(val)}| {val}')
    print('|--|')


if __name__ == '__main__':
    main()

Best wishes !!

[asottile]

awesome -- thank you for this!