Feature/extend romanization rules #398
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Link to specific rules where applicable - Add :specifiesLanguageForm to define a pattern
olovy
approved these changes
Nov 9, 2022
along with script definitions and notes on some possible improvements.
along with clarifications and notes on possible improvements.
by using data from unicode.org (provided that we are OK with their license and document us using it).
|
I've added some more data to be used, and some comments on what we may still need (e.g. using the |
olovy
reviewed
Nov 18, 2022
olovy
reviewed
Nov 18, 2022
Co-authored-by: Olov Ylinenpää <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds precision to the transformed language forms, to solve the conflation of
iso-1995as the specific "ISO:9 from 1995, for Cyrillic to Latin script".It is unclear if it is manageable and informative (or complicating) to do like this in general. At least in part, assessing this comes from how we use this data in the romanization feature.
If the rules are just informative support data, this is reasonably a good addition. But if we want to use these definitions in runtime algorithms (specifically to build BCP47 tags from
codecomponents), by this design, the algorithm would have to "climb up" viabroaderuntil if finds a code (though probably, as here, just one step). Might be OK, but it is more complex. For maintaining predefined codes I believe it is OK. But if we are to generate new:TransformedLanguageForm:s when the romanization API is called, we need to assess the complexity when using these definitions.