Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: i18n-system-languages #569

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

carlosjeurissen
Copy link
Contributor

@carlosjeurissen carlosjeurissen commented Mar 18, 2024

This PR adds the proposal for both i18n.getPreferredSystemLanguages() and i18n.getSystemUILanguage() previously called i18n.getOSLanguage(), which all vendors are supportive on as discussed in #252

Copy link
Contributor

@rdcronin rdcronin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Carlos!

proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved
proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved
proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved
proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved
proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved
Comment on lines 38 to 50
```json
{
"name": "getOSLanguage",
"type": "function",
"nocompile": true,
"description": "Gets the UI language of the Operating System. This is different from $(ref:i18n.getUILanguage) which returns the UI language of the web browser.",
"parameters": [],
"returns": {
"type": "string",
"description": "An IETF BCP 47 language tag such as en-US or pt-BR."
}
}
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd lean towards describing these in typescript (for now), since it's a more portable format than Chromium's use of JSON schema. Especially things like "nocompile" are very Chromium-specific.

That said, if other browser vendors are fine with (or prefer) this, I don't feel strongly, since, well, this is obviously good enough for Chromium : )

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, TypeScript would make more sense.

"nocompile": true,
"description": "Gets the UI language of the Operating System. This is different from $(ref:i18n.getUILanguage) which returns the UI language of the web browser.",
"parameters": [],
"returns": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implies a synchronous return. Is that a requirement?

(I'm not sure if we'd have the OS language available in the renderer today, unlike the UI language)

Copy link
Contributor Author

@carlosjeurissen carlosjeurissen Mar 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Mozilla Firefox, the OS language is available in the renderer synchronously. In general the motivation to return synchronously is to match i18n.getUILanguage. Having it synchronously is also very welcome to load locale data as quickly as possible for bidi, lang attributes, locale specific files and potentially in the future, passing it to an API like i18n.getLanguageDictionary

**Document Metadata**

**Author:** carlosjeurissen

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This dropped the "sponsoring browser" item here.

That was added to ensure the proposals here don't become a graveyard of aspirational APIs and instead reflect ones that at least one browser is committed to implementing in the immediate future.

I'll need to do some quick investigation into the implementation. If it looks straightforward, I'm comfortable putting Google Chrome as the sponsoring browser.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added back the Sponsoring browser with a TBD.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm supportive of sponsoring for Safari and plan to implement it as soon as we have consensus.

proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved
@carlosjeurissen
Copy link
Contributor Author

@xeenon Thanks for reviewing and the willingness to implement this.

@rdcronin Thanks for reviewing and your comments. Most feedback has been implemented in the proposal. Thanks for offering to sponser the feature as browser.

@Rob--W Reading from your comment in the reading notes of Today's (2024-03-28) meeting:

This proposal doesn’t follow the new process. One of the aspects of the new process template is that there should be at least one browser committing to implementing the API.

It was under my impression there was general support for this feature across browsers and the next step would be a proposal. However I understand your concerns regarding ending up with many proposals without browsers moving forward to actually implement them. So I went ahead and created a patch for Firefox.
See: https://bugzilla.mozilla.org/show_bug.cgi?id=1888486

@carlosjeurissen carlosjeurissen added topic: localization i18n-tracker Group bringing to attention of Internationalization, or tracked by i18n but not needing response. labels Mar 28, 2024
@Rob--W
Copy link
Member

Rob--W commented Mar 28, 2024

@Rob--W Reading from your comment in the reading notes of Today's (2024-03-28) meeting:

This proposal doesn’t follow the new process. One of the aspects of the new process template is that there should be at least one browser committing to implementing the API.

It was under my impression there was general support for this feature across browsers and the next step would be a proposal.

Part of the proposal template includes an item about the "sponsoring" browser. As Devlin commented elsewhere in the PR, the purpose of that is to encourage proposals that reflect APIs that have a path towards realization rather than just being aspirational.

However I understand your concerns regarding ending up with many proposals without browsers moving forward to actually implement them. So I went ahead and created a patch for Firefox.
See: https://bugzilla.mozilla.org/show_bug.cgi?id=1888486

Thanks for the bug and patch!

proposals/i18n-getOSLanguage.md Outdated Show resolved Hide resolved

`i18n.getOSLanguage()` would synchronously return a [BCP47 language tag](https://www.w3.org/International/core/langtags/rfc3066bis.html) like `i18n.getUILanguage()` does right now.

It would follow the following signature for [i18n.json](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/extensions/common/api/i18n.json):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When referencing source code, try to use a permalink where possible. In this case there has been a request to use IDL instead of JSON, so this line will probably end up being dropped. If not, here is the permalink: https://chromium.googlesource.com/chromium/src/+/4299ce68496b32ba309e2f012e0db5b4b8cd478a/extensions/common/api/i18n.json

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know! Updated the proposal to reflect this!


**Summary**

Allow to get the language of your operating system as an [BCP47 language tag](https://www.w3.org/International/core/langtags/rfc3066bis.html).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the discussion starting from #252 (comment), should we rename the proposal to i18n.getOSLanguages and let it return an array?

And also, guarantee the array to be non-empty?

@hanguokai
Copy link
Member

hanguokai commented Mar 31, 2024

As the discussion deepens (#252), I think there are four concepts here:

  1. The Browser UI Language
  2. The Browser Accept Languages
  3. The OS UI Language
  4. The OS Accept Languages

So, ideally there should be four functions:

  1. i18n.getBrowserUILanguage()
  2. i18n.getBrowserAcceptLanguages()
  3. i18n.getOSUILanguage()
  4. i18n.getOSAcceptLanguages()

And deprecate the current i18n.getUILanguage() and i18n.getAcceptLanguages().

Or using another design:

// return a single string for UI language
i18n.getUILanguage(
    prefer: "browser" // or "os"
);

// return an array for accept languages
i18n.getAcceptLanguages(
    prefer: "browser" // or "os"
);

@carlosjeurissen carlosjeurissen changed the title Add getOSLanguage proposal Add i18n-system-languages proposal May 23, 2024
@carlosjeurissen
Copy link
Contributor Author

carlosjeurissen commented May 23, 2024

Updated the proposal which addresses the following:
i18n.getOSLanguage() has been renamed to i18n.getSystemUILanguage() to better indicate it is supposed to be the user interface of the operating system. Not the first preferred language specified in the operating system.

i18n.getPreferredSystemLanguages() has been added which returns the above mentioned preferred languages specified in the operating system. This aligns with the same API and name available in Electron apps.

For i18n.getPreferredSystemLanguages(), a patch for Firefox has been made, see: https://bugzilla.mozilla.org/show_bug.cgi?id=1888486


#### Use Cases

Having the original system language(s) is useful for language-related extensions and for extensions who want to provide translations more true to the system language. For example, to adapt to a specific sub-language, like Belgium-Dutch (nl-BE). While i18n.getUILanguage would return 'nl'.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example here is not very clear. There is a tendency to think that the primary language subtag (nl in the examples here) is "the language", but there are language variations that depend on additional subtags, such as script or region. For example, zh-Hans (Simplified Chinese) vs. zh-Hant (Traditional Chinese). Thus, the UI language is not guaranteed to be a single subtag.

It's also more useful/helpful to talk about the locale than it is to talk about the language, even if, for historical reasons, we use the word "Language" in the API names.

Note that locale means more than just language adaptation. Frequently, locale-based variations in number, date, time, measurement, etc. formatting constitute a key part of adaptation.

Note too that the use cases here are incomplete. The PR lists a number of different reasons, but you only have one in the document itself.

Thus:

Suggested change
Having the original system language(s) is useful for language-related extensions and for extensions who want to provide translations more true to the system language. For example, to adapt to a specific sub-language, like Belgium-Dutch (nl-BE). While i18n.getUILanguage would return 'nl'.
Extension authors might want to obtain the locale used by the user agent's host environment in order to format values (such as numbers, dates, and so forth) or do other locale-affected operations (such as list sorting) according to expectations of the user. For example, formatting the date `2024-06-15` varies between `en-US` (6/15/2024) and `en-GB` (15/6/2024), whereas `i18n.getUILanguage` might return only `en` (English).
Extension authors might want to obtain the locale used by the user agent's host environment to better match the runtime environment's localization. For example, the operating system might be running in `ff-NG` (Fulani as used in Nigeria), but the user agent might not be localized into this language and return `en` (English) for `i18n.getUILanguage`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphillips Thanks so much for the suggestions! Applied the suggested change.

Also added a better example of the purest form use-case which describes i18n.getUILanguage returning a language tag with additional subtags which might not be sufficient for the user.

Lastly rewritten sentences to use locale instead of language where it makes sense based on your suggestion.


### Schema

`i18n.getPreferredSystemLanguages()` would asynchronously return a list of [BCP47 language tag](https://www.rfc-editor.org/bcp/bcp47.html) like `i18n.getAcceptLanguages()` does right now.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably you want to say that it returns an ordered list (BCP47 calls this a "language priority list").

Note that most operating systems don't provide a feature like this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphillips thanks for noting and clarifying! Updated the sentence to reflect these suggestions. It is indeed meant to be an ordered list. As far as I know both mac and windows support this. But I could be wrong?

@carlosjeurissen carlosjeurissen changed the title Add i18n-system-languages proposal Proposal: i18n-system-languages Jun 19, 2024
@xeenon
Copy link
Collaborator

xeenon commented Jun 29, 2024

@carlosjeurissen There seems to have been a rebase issue, where this PR picked up a bunch of other changes from the repo outside of your proposal.

@carlosjeurissen
Copy link
Contributor Author

@xeenon Thanks for noting! Has been addressed.

@carlosjeurissen
Copy link
Contributor Author

Based on some of the feedback during TPAC, updated this PR and updated the Firefox patch.
See:
https://phabricator.services.mozilla.com/D206064
https://bugzilla.mozilla.org/show_bug.cgi?id=1888486

@@ -67,11 +70,14 @@ It would follow the following signature for [i18n.json](https://chromium.googles
"name": "getSystemUILanguage",
"type": "function",
"nocompile": true,
"description": "Gets the current UI language of the Operating System. This is different from $(ref:i18n.getUILanguage) which returns the UI language of the web browser.",
"description": "Gets the current UI locale of the Operating System. This is different from $(ref:i18n.getUILanguage) which returns the UI locale of the web browser. This locale could include additional information like preferred datetime format and number system if supported by the OS.",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the "additional information" part a standard? I know Apple platforms can include it, but that isn't something we have had in web extensions before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xeenon yes, this is specified by unicode. This was discussed during the shared session with the WECG/I18NWG meeting at TPAC. However, giving a further look at this I see how this should not be part of the getSystemUILanguage. Updated the PR to reflect this. Still, providing this information can be useful, but it should be implemented using some other method / proposal.

xeenon pushed a commit to xeenon/WebKit that referenced this pull request Oct 3, 2024
https://bugs.webkit.org/show_bug.cgi?id=280586

Reviewed by NOBODY (OOPS!).

WECG issue: w3c/webextensions#252
WECG proposal: w3c/webextensions#569

Also changed the i18n APIs to return `"und"` instead of `undefined` when there is no language code.

* Source/WebKit/Platform/spi/Cocoa/FoundationSPI.h:
* Source/WebKit/Shared/Extensions/WebExtensionUtilities.mm:
(WebKit::toWebAPI): Return "und" where then is no language code.
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPILocalizationCocoa.mm:
(WebKit::WebExtensionAPILocalization::getPreferredSystemLanguages):
(WebKit::WebExtensionAPILocalization::getSystemUILanguage):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPILocalization.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPILocalization.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPILocalization.mm:
(TestWebKitAPI::localeStringInWebExtensionFormat): Return "und".
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18n)):
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18nWithFallback)):
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18nWithoutMessages)):
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18nWithoutDefaultLocale)):
webkit-commit-queue pushed a commit to xeenon/WebKit that referenced this pull request Oct 3, 2024
https://bugs.webkit.org/show_bug.cgi?id=280586

Reviewed by Brian Weinstein.

WECG issue: w3c/webextensions#252
WECG proposal: w3c/webextensions#569

Also changed the i18n APIs to return `"und"` instead of `undefined` when there is no language code.

* Source/WebKit/Platform/spi/Cocoa/FoundationSPI.h:
* Source/WebKit/Shared/Extensions/WebExtensionUtilities.mm:
(WebKit::toWebAPI): Return "und" where then is no language code.
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPILocalizationCocoa.mm:
(WebKit::WebExtensionAPILocalization::getPreferredSystemLanguages):
(WebKit::WebExtensionAPILocalization::getSystemUILanguage):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPILocalization.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPILocalization.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPILocalization.mm:
(TestWebKitAPI::localeStringInWebExtensionFormat): Return "und".
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18n)):
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18nWithFallback)):
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18nWithoutMessages)):
(TestWebKitAPI::TEST(WKWebExtensionAPILocalization, i18nWithoutDefaultLocale)):

Canonical link: https://commits.webkit.org/284637@main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
i18n-tracker Group bringing to attention of Internationalization, or tracked by i18n but not needing response. topic: localization
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants