Hangul Jamo sequences for Old Hangul syllables (without precomposed glyphs) are not combined into a syllable in the vertical mode #79
Comments
|
I'll also check Nanum YetHangul (Old Hangul) fonts. |
|
Nanum YetHangul fonts do not have this issue (even though it has an issue with u+302e/302f placement in the vertical layout). Vertical metrics for u+1147 might have an issue. |
|
What other combining LV sequences are present in the example text other than <1147 1167>? |
|
Assigning to kenlunde |
|
@kenlunde there are a lot of LV combinations and they are all fine except for <U+1147 U+1167>. https://namu.wiki/w/%ED%9B%88%EB%AF%BC%EC%A0%95%EC%9D%8C has the full text (Use 'Find in page' to search for '원문') that can be copy'n'pasted. And, here's a file for you. I tested U+1147 followed by either U+1167 and U+1163 with or without a trailing consonant. All of them have the same issue. Horizontal layout is fine, but in the vertical layout, they're not combined into a syllable. I have yet to test them outside Chrome (i.e. directly with Harfbuzz), but as I wrote above, Nanum YetHangul fonts work fine. |
|
u1147.txt above rendered with Chrome - horizontal layout is fine, but vertical layout is broken.
|
|
Metrics- or feature-wise, there is nothing special or out of the ordinary for the glyphs for U+1147 in Noto Sans CJK, and given that horizontal rendering is okay, I suspect that there is a bug in the rendering engine (Harfbuzz). |
|
@kenlunde Perhaps, you're right. I'm trying to sort things out. It's not just U+1147 but all other Old Hangul syllables are broken when I tested Noto Sans CJK with 'hb-shape' or 'hb-util' with Even in Chrome, somehow in my screenshot in the bug report has only U+1147-led-sequences broken. However, when I enumerate all the possible syllables starting with a given leading consonant and lay them out vertically in Chrome and Firefox, all of them are broken regardless of which L leads syllables. The way they're broken in Chrome/Firefox is different from the way they're broken in standalone HB test utils ( hb-shape/hb-view). Strange. That can be due to a version difference in harfbuzz. I'll try the same version as used by Chrome/Firefox. JFYI: @behdad After a bit more tinkering, I'll file a bug against HB with a sample test case. |
jungshik
changed the title
|
A mystery solved in the sense that I figured out what's going on. In the screenshot I included in my bug report, all the Old Hangul syllables (except for the one) have pre-composed glyphs (they belong to 500 Old Hangul syllable set). That is, hmj.txt is made entirely of those 500 old Hangul syllables + CJK ideograph except for just one ( {U+1147, U+1167} ). So, there's no issue in the vertical mode for all of them except for one. They're just single glyphs and advancement is not a concern. Old Hangul syllables like {U+1147, U+1167; rank 1176}, {U+1100, U+119E, U+11EB; rank 830) outside the 500 pre-composed glyph set have to be put together with multiple glyphs via OT and vertical advancement/positioning becomes an issue. {U+1147, U+1167} was ok in Naver's Nanum BarunGothic YetHangul or Nanum MyeongJo YetHangul because they have a precomposed glyph for the sequence. The same is true of <U+1100, U+119E, U+11EB>. In the screenshot below, the 5th syllable from the bottom is <U+1100, U+119E, U+11EB>. On the left (Nanum Myeongjo YetHangul), it's correct (it's a precomposed glyph) while on the right, it's not composed into a syllable (Noto Sans CJK).
As for Harfbuzz's standalone test program vs Chrome/Firefox, Behdad pointed out to me that it's just a hb-view issue (not shaping issue) . By updating to the tip of the tree, it's gone. hb-view and Chrome/Firefox behave exactly the same way. |
|
In the following screenshots, a few Old Hangul syllables are shown correctly in the vertical mode while the vast majority of them are not shaped correctly. Those few belong to the 500-syllable set with precomposed glyphs.
|
|
Filed harfbuzz/harfbuzz#414 |
|
@kenlunde See the harfbuzz bug for a sample string. Does it work correctly in InDesign ? |
|
InDesign doesn't support combining jamo. The only way for this to work in InDesign using the current version is to add the combining jamo lookups to the 'calt' GSUB feature. Version 1.000 did this, but we removed the lookups in Version 1.001. Even if I were were to restore the combining jamo lookups in the 'calt' GSUB feature, I doubt that we'd see the desired result for vertical writing. |
|
Thanks, @kenlunde It looks like it's a bug in vmtx table. Below is the output of hb-shape with With Note that the glyph for U+1167 (gid64509) has ax=0 for horizontal layout while the same glyph has ay=-1000 instead of '0' in the vertical layout. vmtx table has the following: hmtx table has the following: I think 'height' for cid64509 (and other vowels and trailing consonants) in vmtx should be 0. |
The above alone didn't fix the problem, though. The vertical origin (to be calculated from CFF charstring) seems to be off. |
|
Three things seem to be necessary, two of which are easy, which are to add the following 'vmtx' table overrides for all combining 'vjmo' and 'tjmo' glyphs (a regular expression is used to represent the Unicode-based working glyph names): The first override places the Y origin at the bottom of the em-box, whose effect is to shift the glyph upward by 1,000 units. The second override sets the vertical advance to be zero. However, the glyph also needs to be shifted 920 units to the right, which is not possible via the 'vmtx' table. Here is a special version of NotoSansCJKkr-Regular.otf.zip that adds these overrides to the combining 'vjmo' and 'tjmo' glyphs (738 glyphs in total), and you can see that their positions and advances are now correct, but that they still need an additional 920-unit shift to the right. |
|
Thank you, @kenlunde for a test font. The vertical position is still off with that (I'm aware that the horizontal position is not supposed to be fixed in the test font) but in a different way than 1.004. This is the test string: This is the output from hb-shape with 1.004: This is from your test font: hb-view output with 1.004:
hb-view output with the test version:
|
|
Hmm... for the vertical direction (y-coordinate), hb-shape output (ay, dy) seems correct with the test version, but somehow hb-view output (png) is all messed up even along y-axis. |
|
It seems to be a bug in hb-view. With Chrome, I got this expected result (along the y-axis, glyphs are correctly positioned).
|
|
That is what is expected, given the 'vmtx' changes that I specified in the test font. The open question is now how to effect the X-axis changes without adding 738 seemingly duplicate glyphs to the font. What would be useful is a 'vert' GPOS (not GSUB) feature. |
Yup ! I didn't expect the horizontal position to be correct per your earlier comment. For the X-axis change, I agree with you that a 'vert' GPOS would do the trick. |
|
Give this version of NotoSansCJKkr-Regular.otf.zip a whirl. It includes a 'vert' GPOS feature that shifts the 'vjmo' and 'tjmo' glyphs 920 units to the right. I get the expected results in the TextEdit app on the latest macOS (I removed the inter-character spaces):
|
|
Thanks, @kenlunde Apparently, Chrome's "sense" of the sign is the opposite of what you expect (vert GPOS is applied, but V and T are shifted to the left and instead of to the right)
And, Firefox is also different. V and T are shifted even farther to the right.
Let me try hb-shape and hb-view (even though hb-view's vertical positions are off). |
|
This may be a macOS bug. I initially specified 920 as the shifting value, because that matches my understanding of this metrics value, but saw the opposite results. I then tried -920, which gave me the correct output in TextEdit. |
|
Please try this NotoSansCJKkr-Regular.otf.zip, which uses 920 instead of -920. |
|
Thanks, @kenlunde With the latest version, Chrome's rendering is like that of Firefox with the previous version and Firefox's is like that of Chrome with the previous version. :-) Chrome: There might be a unit issue in Chrome/Firefox. The amount of shift seems too much. |
|
Firefox with the latest version:
|
|
Please keep me posted. At least now you have a viable font with which to test. |
|
Working on this issue further, I built two test fonts this morning that use two different approaches, both of which should work in environments that support combining jamo and the 'vert' GPOS (not GSUB) feature. I am trying to get a sense of which approach is the better or preferred one. The first font, NotoSansCJKjp-Regular.otf, uses 'vmtx' table settings to override the vertical advance (set to zero) and vertical origin (shifted 1000 units up) of the V and T glyphs for combining jamo, and the 'vert' GPOS feature to shift them 460 units to the right. In the 'vert' GPOS feature, only the XPlacement value record is set (to 460). The second font, NotoSansCJKkr-Regular.otf, uses only the 'vert' GPOS feature to accomplish all three things for the V and T glyphs for combining jamo, by setting the XPlacement value record to 460 units (shifting the glyph to the right 460 units), the YPlacement value record to 1000 units (shifting the glyph 1000 units up), and the YAdvance value record to -1000 units (making it zero-width in vertical writing). The first font means that combining jamo in vertical writing will be partially broken in environments that do not support the 'vert' GPOS feature. The glyphs will be placed correctly along the Y-axis and will have a zero vertical advance, but will appear half their width to the left. The second font means that combining jamo will be completely broken in environments that do not support the 'vert' GPOS feature, which makes the lack of support a bit more obvious. In terms of table sizes, the second font is about 3K smaller, which is due to the size of the 'VORG' table: 3,892 bytes versus 928 bytes. The 'GPOS' table sizes vary by a mere eight bytes: 43,630 bytes versus 43,638 bytes. For good measure, and to provide additional testing fodder, the glyphs for U+20DD COMBINING ENCLOSING CIRCLE, U+3099 COMBINING KATAKANA-HIRAGANA VOICED SOUND MARK, and U+309A COMBINING KATAKANA-HIRAGANA SEMI-VOICED SOUND MARK have been modified in both fonts to be zero-width and placed to the left of the origin (0,0), and are also given the same treatment via the 'vmtx' table and 'vert' GPOS feature for vertical writing, though the XPlacement value record in the 'vert' GPOS feature is set to 500 units, not 460 units. It would be great if @behdad could double-check my work on these fonts, especially in terms of the 'vert' GPOS feature value record settings. Someone should also test these fonts in the latest HarfBuzz, which appears to be the only environment that supports this functionality. |
|
I think the suggestion in adobe-fonts/source-han-sans#34 (comment) makes a lot of sense, and I wonder if this is right approach. This is not limited to Hangul, but for any number of characters forming a grapheme cluster, for example “ẍ” should have the mark staying on top of the x when rendered upright in vertical direction, but this does not seem to be the case right now. Should this be handled by the layout engine? (HarfBuzz? Its clients?) i.e. treating grapheme clusters as one unit during vertical layout as they IMO should? |
@khaledhosny At least for now, from the practical pov, I'm afraid we have to live with fonts doing the job (as Ken did to Noto Sans CJK). |
|
@jungshik: Have you confirmed that both font solutions ('vmtx' table overrides for Y advance and Y origin plus 'vert' GPOS feature for XPlacement versus 'vert' GPOS feature for XPlacement, YPlacement, and YAdvance) work in the latest HarfBuzz? |
|
@kenlunde , let me try the second approach (vert-GPOS-only solution). The first apparoch (vert-GPOS + vmtx changes) was confirmed to work as reported earlier in this bug. |
|
@jungshik: Yes, I am most interested in whether the 'vert' GPOS feature for XPlacement, YPlacement, and YAdvance solution works in the latest HarfBuzz, as the other solution has been confirmed to work. |
Why not? It has worked forever. |
|
Thank you, @behdad. |
|
@behdad: To clarify, I wasn't doubting that HarfBuzz supports this use case, but rather I am seeking confirmation that I set the values correctly, in particular those for YPlacement and YAdvance, which I translated from the 'vmtx' table overrides that did the same in the first solution. That is why I asked @jungshik to confirm this. |
|
Chrome ToT (a few day old with HarfBuzz 1.4.2) works fine with a GPOS-only font as shown below:
Chrome 56.0.2924.87 (with HarfBuzz 1.3.?) does not with a GPOS-only font. It works fine with GPOS + vmtx changes. I haven't tried HarfBuzz 1.3.? standlone (hb-view, hb-shape) with a GPOS-only font. Blink (Chrome) might have to a culprit, but I don't know without trying HB 1.3.x standalone.
|
|
It is great to see the GPOS-only font working correctly. |
|
Quick question for @behdad: The same 'vert' GPOS feature settings (XPlacement, YPlacement, and YAdvance) for U+20DD, U+3099, and U+309A as described earlier in this issue can be applied to U+302A through U+302D, right? I ask, because that means that the vertical glyphs for those four characters can be removed, and the 'vert' GPOS feature can be used to correctly place the (non-spacing) horizontal glyphs for vertical writing. |
Yes. Not sure about YAdvance though, as we zero those for mark glyphs in many shapers. But you shouldn't need YAdvance for those, right? |
|
@behdad: The General_Category value for U+302A through U+302D is Mn (Nonspacing_Mark: a nonspacing combining mark (zero advance width)), which is the same as U+3099 and U+309A, so I would imagine that YAdvance should be set to -1000, which will mean a zero-unit vertical advance. |
Humm? You mean YAdvance in GPOS be adjusted by -1000 to zero-out the existing advance? Just checked and our Hangul shaper doesn't zero mark advances. So whatever you set in vmtx and then adjust in GPOS will be effective. |
|
Yes, in lieu of making the Y-axis adjustments in the 'vmtx' table and the X-axis adjustment in the vert' GPOS feature, both X- and Y-axis adjustments would be made in the 'vert' GPOS feature, mainly to consolidate the settings in one place. If I were to override the Y-axis origin and advance via the 'vmtx' table, the only setting that the 'vert' GPOS feature would need is XPlacement. |
|
Works for me. Probably not for other implementations.
…On Sat, Feb 25, 2017 at 12:09 PM, Dr. Ken Lunde ***@***.***> wrote:
Yes, in lieu of making the Y-axis adjustments in the 'vmtx' table and the
X-axis adjustment in the vert' GPOS feature, both X- and Y-axis adjustments
would be made in the 'vert' GPOS feature, mainly to consolidate the
settings in one place.
If I were to override the Y-axis origin and advance via the 'vmtx' table,
the only setting that the 'vert' GPOS feature would need is XPlacement.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#79 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAIuK85vUxaMHf3kqLwjCCrsDG7mzM_8ks5rgIpigaJpZM4LzSbG>
.
--
behdad
http://behdad.org/
|
|
Still reproduced in LibreOffice 6.0.1. |
@KrasnayaPloshchad what about 6.2.2? :)
@kenlunde is this how v2.x is done? :) (This issue can soon be closed, but leaving open to track answers to those 2 questions :) |
|
@davelab6 This is somewhat of a hornets' nest that really doesn't really need an open issue to track it, as it affects sequences other than those for combining jamo, and is high on my priority list. The background is that I attempted to modify the description of the 'vert' feature to allow for GPOS implementations, but Microsoft pushed back, claiming that their implementations (GDI and DirectWrite) already do this. As a result, Microsoft is now tasked to document this, in terms of affected characters, expected metrics, and the metrics transformations that are performed in vertical writing mode. This is the only way that other implementations that currently support 'vert' as a GPOS feature, such as HarfBuzz, CoreText, and WebKit, can provide the same results as GDI/DirectWrite. In other words, this is a waiting game. Microsoft provides a spec, and then other implementations change to accommodate it. This is at least a couple years out. |
Got it. Filed OpenType/opentype-layout#12 to track. |
{U+1147, U+1167} (whether followed by U+302E or not} is not combined into a syllable in the vertical writing mode.
{{{
ᅇᅧ〮
ᅇᅧ
}}}
Vertical: the last (leftmost) two lines have the sequences in question:
Horizontal: the last (bottommost) two lines have the sequences in question:
Will check other syllables starting with U+1147. This is checked with Chrome 55.0.2883.87. I'll also check with HarfBuzz 1.4.2.
@kenlunde
The text was updated successfully, but these errors were encountered: