Unicode combining characters do not show up

[retupmoca]

It looks like unicode combining characters such as https://www.fileformat.info/info/unicode/char/0303/index.htm do not render correctly.

Modifying the example starter project as a test (see https://gist.github.com/retupmoca/68a4852a28b7cf6d2c39a90d7670212f for the full modified file), I added:

  // plain UTF-8 output of a random combining character:
  // https://www.fileformat.info/info/unicode/char/0303/index.htm
  std::cout << "done\xcc\x83:\n";

  auto summary = [&] {
    auto content = vbox({
        // converted UTF-8
        hbox({text(to_wstring(std::string("- done\xcc\x83:   "))), text(L"3") | bold}) | color(Color::Green),
        // UTF-16 / UCS-2
        hbox({text(L"- active\x0303: "), text(L"2") | bold}) | color(Color::RedLight),
        hbox({text(L"- queue:  "), text(L"9") | bold}) | color(Color::Red),
    });
    return window(text(L" Summary "), content);
  };

When I run the modified program, I see:

That is, the combining character renders correctly in my terminal if I directly output it, but displaying it via FTXUI does not appear to work.

[ArthurSonzogni]

Thanks for reporting this. I will take a look.

[ArthurSonzogni]

A ftxui::Pixel is:

struct Pixel {
  wchar_t character = U' ';
  bool blink = false;
  bool bold = false;
  bool dim = false;
  bool inverted = false;
  bool underlined = false;
  Color background_color = Color::Default;
  Color foreground_color = Color::Default;
};

So, a single wchar_t is not enough to store combining characters.
I am actually quite surprised this fallback so nicely. ;-)

Not fully sure how to overcome this issue.

[ArthurSonzogni]

Note that the character you are using can also be represented via a single character:

  auto document = text(L"- activẽ: ");
  auto screen = Screen::Create(Dimension::Full(), Dimension::Fit(document));
  Render(screen, document);
  screen.Print();

output

This doesn't solve this issue, but this might solve your particular problem?

[retupmoca]

Yeah, most of the common graphemes are available as a single codepoint - it's not currently a major issue for me, but I wanted to report it for completeness :)

[ArthurSonzogni]

I added a note into the README.

At first glance, this seems difficult to support. I will try to learn more about this unicode feature. If its specification is not too insane, there might be a way to support "multiple character per pixels" feature.

[magiblot]

In case it may help: magiblot/tvision#26

[ArthurSonzogni]

In case it may help: magiblot/tvision#26

This helps a lot! Sounds like you had the exact same problem and you solved it. Thank you!