Skip to content
This repository has been archived by the owner on Oct 1, 2023. It is now read-only.

Handle terminal encoding #26

Open
fredemmott opened this issue Feb 21, 2019 · 2 comments
Open

Handle terminal encoding #26

fredemmott opened this issue Feb 21, 2019 · 2 comments

Comments

@fredemmott
Copy link
Contributor

fredemmott commented Feb 21, 2019

If someone writes to stdout, we currently just send raw bytes. We should respect locale.

While most terminals are either C or some variant of UTF-8, a large variety of encodings are supported, e.g. on recent ubuntu:

fred@homeserver:~# LC_ALL=ja_JP.UTF-8 date
2019年  2月 21日 木曜日 08:52:14 PST
fred@homeserver:~# LC_ALL=ja_JP.EUC_JP date
2019ǯ  2�� 21�� ������ 08:52:22 PST

I suggest:

  • defining Hack source files/literals etc as UTF-8 (this needs some wider discussion)
  • keeping HSL IO interfaces as raw bytes
  • stop using HSL IO interfaces directly here - provide (read|write)(Text|Bytes) that handle locale if appropriate
@fredemmott
Copy link
Contributor Author

Another question: do any common platforms support UTF-16 terminals, e.g. if we end up supporting windows?

@fredemmott
Copy link
Contributor Author

Another question: do any common platforms support UTF-16 terminals, e.g. if we end up supporting windows?

Yes, though windows has recently gained UTF-8 support that we might be able to depend on in a few years, depending on what versions we target (if any).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant