Changing font format to LVGL embedded format and adding a python tool to create fonts

[raphatex]

• Adding diacritics in translations.h
• TODO : Creating a python tool for font generation
• TODO : Updating Fonts

(Actually I innitially wanted to generate fonts with the .exe and then create the .py for later use but If think I'm gonna start with the .py)

[raphatex]

This is actually "Creating : converter.py" sorry I messed the commit message

[raphatex]

So I have created a very crude python font converter
Todo:

• improve quality (~the same as the exe if possible)
• make it more user friendly
• Make a GUI

[ducalex]

I'm impressed that you've already made a python tool, this is nice!

Now that retro-go will ship with a generator, I think we should add the .ttf (or whatever) fonts to the repo to preserve as reference if someone wants to regenerate the .c fonts with more glyphs or sizes.

[raphatex]

Actually I cheated a bit, I used ChatGPT to help me with the .ttf font extraction ;)
But understanding the custom encoding for the output took me a few hours 😅
I documented all my research directly at the beginning of the python converter for future dev.

Right now the tool does not make results as good as the .exe I still have to do some adjustments.

I agree to add the .tff in the repo !

[raphatex]

I created a UI with a font renderer ! To make it easier to change font settings

[ducalex]

Oh wow, having a preview is a game changer! I remember having to flash retro-go 50+ times back when I was trying to find the perfect parameters for the .exe generator...

[raphatex]

Yeah absolutely I've been through this struggle too with Open Sans but now it's gonna be way faster ^^
I'm currently trying to add a way to directly "edit" the c font like Paint software directly on the canvas that would be the cherry on the cake for the tool !

[raphatex]

I was looking on the web for inspirations from other tools and I came across LVGL font converter
https://lvgl.io/tools/fontconverter
This tool uses a special type to store the "header" for each character and precomputes the index of the first byte of each character.
I feel like this approach is better than Retro-Go's one in terms of performance. Only issue is that it uses a bit more memory to store the indexes (probably around 16bit(2 Bytes) more per character)
In the end we should loose around 161*2 = 322 Bytes / 2.5Kbit per font or 1.6KBytes / 12.8Kbit for 5 fonts.

So I was wondering if it was worth the try since we are making our own tool ?

/*-----------------
 *    BITMAPS
 *----------------*/

/*Store the image of the glyphs*/
static LV_ATTRIBUTE_LARGE_CONST const uint8_t glyph_bitmap[] = {
    /* U+0020 " " */
    0x0,

    /* U+0021 "!" */
    0xff, 0xcf,

    /* U+0022 "\"" */
    0xff, 0xf0,

    /* U+0023 "#" */
    0x16, 0x14, 0x7f, 0x24, 0x24, 0xfe, 0x28, 0x68,

    /* U+0024 "$" */
    0x10, 0xf9, 0x43, 0x87, 0xc3, 0xc5, 0xbe, 0x10,

    /* U+0025 "%" */
    0x71, 0x1b, 0x23, 0x68, 0x3a, 0x0, 0x5c, 0x16,
    0xc4, 0xd8, 0x8e,

    /* U+0026 "&" */
    0x3c, 0x30, 0x1c, 0x6, 0xf, 0xb6, 0x7b, 0x18,
    0xfe,

    /* U+0027 "'" */
    0xfc,

    /* U+0028 "(" */
    0x6f, 0x6d, 0xb3, 0x60,

    /* U+0029 ")" */
    0xd9, 0xb6, 0xde, 0xc0,
};

/*---------------------
 *  GLYPH DESCRIPTION
 *--------------------*/

static const lv_font_fmt_txt_glyph_dsc_t glyph_dsc[] = {
    {.bitmap_index = 0, .adv_w = 0, .box_w = 0, .box_h = 0, .ofs_x = 0, .ofs_y = 0} /* id = 0 reserved */,
    {.bitmap_index = 0, .adv_w = 61, .box_w = 1, .box_h = 1, .ofs_x = 0, .ofs_y = 0},
    {.bitmap_index = 1, .adv_w = 80, .box_w = 2, .box_h = 8, .ofs_x = 2, .ofs_y = 0},
    {.bitmap_index = 3, .adv_w = 92, .box_w = 4, .box_h = 3, .ofs_x = 1, .ofs_y = 5},
    {.bitmap_index = 5, .adv_w = 147, .box_w = 8, .box_h = 8, .ofs_x = 1, .ofs_y = 0},
    {.bitmap_index = 13, .adv_w = 122, .box_w = 7, .box_h = 9, .ofs_x = 0, .ofs_y = -1},
    {.bitmap_index = 21, .adv_w = 176, .box_w = 11, .box_h = 8, .ofs_x = 1, .ofs_y = 0},
    {.bitmap_index = 32, .adv_w = 153, .box_w = 9, .box_h = 8, .ofs_x = 1, .ofs_y = 0},
    {.bitmap_index = 41, .adv_w = 54, .box_w = 2, .box_h = 3, .ofs_x = 1, .ofs_y = 5},
    {.bitmap_index = 42, .adv_w = 80, .box_w = 3, .box_h = 9, .ofs_x = 1, .ofs_y = -1},
    {.bitmap_index = 46, .adv_w = 80, .box_w = 3, .box_h = 9, .ofs_x = 1, .ofs_y = -1},
    {.bitmap_index = 50, .adv_w = 92, .box_w = 5, .box_h = 5, .ofs_x = 0, .ofs_y = 3},
    {.bitmap_index = 54, .adv_w = 147, .box_w = 7, .box_h = 6, .ofs_x = 1, .ofs_y = 0},
    {.bitmap_index = 60, .adv_w = 67, .box_w = 3, .box_h = 3, .ofs_x = 0, .ofs_y = -2},
    {.bitmap_index = 62, .adv_w = 73, .box_w = 3, .box_h = 1, .ofs_x = 1, .ofs_y = 3},
    {.bitmap_index = 63, .adv_w = 67, .box_w = 2, .box_h = 2, .ofs_x = 1, .ofs_y = 0}
};

[ducalex]

I feel like this approach is better than Retro-Go's one in terms of performance. Only issue is that it uses a bit more memory to store the indexes (probably around 16bit(2 Bytes) more per character) In the end we should loose around 161*2 = 322 Bytes / 2.5Kbit per font or 1.6KBytes / 12.8Kbit for 5 fonts.

So I was wondering if it was worth the try since we are making our own tool ?

Yes, if you feel there is a better way then you can rewrite our font format and renderer to something completely different. And if you convert the basic font to your new format then we could eventually ditch the two other rendering modes as well.

It would be nice to have a single format for all fonts at last, that is fully documented and with a generator.

The size of the font in flash doesn't bother me, as long as your new renderer doesn't use more stack space and isn't slower than the current one, I will accept it happily :) If anything a 16bit index seems like it would be a win, we could add more like languages like cyrillic glyphs or even add some custom icons to replace things like the wifi/battery icons.

If you do decide to write a new renderer, I'd request you update your PR description to match this new goal (for clarity).

[raphatex]

Sorry, I didn't had enough time in the past days but I will try to work on the new font system during this week !

[ducalex]

I'm going to tag a new retro-go release before merging this PR. I want to be sure that this PR is merged in a known stable base because every time I touch the rendering code it introduces subtle bugs and this time won't be any different I'm sure...

What's the status on your side? Do you feel this PR is ready for review from my side?

[raphatex]

I'm going to tag a new retro-go release before merging this PR. I want to be sure that this PR is merged in a known stable base because every time I touch the rendering code it introduces subtle bugs and this time won't be any different I'm sure...

What's the status on your side? Do you feel this PR is ready for review from my side?

On my side :

• the renderer code is done (I might do some adjustment on this line: yOffset = glyph_dsc[c].ofs_y-2)
• the C font editor python program is 99% working but in the future i'd like to improve it to get more personnalization options (+ icons creation)
• the font converter is working and produces decent results but not 100% pixel perfect. I have to find the right settings for the advance_width but it's just a matter of time.

Lastly I have to write some documentation on the format

[raphatex]

I updated the font editor so we can now select a single character on the preview and edit it, here is a screenshot:

I now have to add more personalizations options...