Add localization support/strings

[romaninsh]

So need to establish a standard way how to add translations and create initial file(s) for atk4/core project strings.

As a test - verify that local messages are properly translated if the language is set.

[romaninsh]

also would act as an example for other repositories and add-ons...

[pkly]

Since the translations are reliant on App and AppScopeTrait, wouldn't it make more sense to put the translations into atk4/ui?

My suggestion is to add a new directory to atk4/ui called i18n or something along those lines, and put the translations there in a single file. Preferably json or something like that, not pure php.

Please let me know when you've figured out something with it, because I'd like to translate everything that's being used in my project into Polish as soon as possible. (data/ui/core/dsql).
It'd be preferable for you to keep an english -> english file as a translation base as well.

As for plugins and code from outside of atk that wishes to use it for translation - maybe add methods to register new files/directories to the translator.

[abbadon1334]

looking at atk4/core there are only exception messages.

@romaninsh i think the @pkly options is much usable.

we can :

1. Open a branch for : wip-translation on atk4/ui
2. Add TranslatableTrait to View?
3. Find strings to be translated and start adding App->_($message) for testing direct plural piped translation (*)
4. Add ui/i18n/en/{files}.json
5. App => `$locale_path = DIR . '/i18n';
6. Test with TranslatorSymfonyTrait
7. Test with TranslatorI18NextTrait ( @pkly https://github.com/abbadon1334/atk4-i18next-php) (**)
8. Figure out how to get Caption plural forms for \atk4\data\Model

(*) piped = "single|plural";
(**) need to be implemented, few days and i will add it

@romaninsh what you think about this roadmap?

[pkly]

Unless there's a huge amount of text (and there really isn't) there isn't much reason to split the files in translation too much probably.
Since this translation sysmem relies on a domain of the translation, why not make it so it's like
/ui/i18n/{language}/{domain}.json, that way you can safely input new domains and update the current one.
Maybe have a fallback for when there's no domain, but I don't remember if that's an option or not.

And it's not like atk4/core, /data and /dsql have not to be translated, they all depend on core to begin with, so just add the TranslateableTrait to the classes in those places and just swap the strings so they can be translated, then let the ui App handle the translation, since there's a fallback in the TranslateableTrait there's no real need to worry there.

@abbadon1334 in the 8th point, plural forms you mean the caption for models? You could just pass an array $caption_plurals or something like that, just assume it's already been translated?
Or use that weird symphony string?

18NextTrait seems like a nice addition down the line, but it should be like the Symphony one and not really be required.

[abbadon1334]

@abbadon1334 in the 8th point, plural forms you mean the caption for models? You could just pass an array $caption_plurals or something like that, just assume it's already been translated?

Yes, but must be defined the correct way to define translations, i think one way can be, if is defined a Translation use caption from translation e.g. :

at simple level we can make the library working in the same way, for a more advanced level of translations we need keys.

class User extends Model {
$caption = 'User'; // no plural form at all 
}

or

class User extends Model {
$caption = 'User|Users'; // '{singular}|{plural}'
}

But i think $caption must be used as a fallback plural form.

• I think first must be checked if $translator->_('model.user',$parameters) return a translation.
• • if not fallback to direct string translation with defined $translator->_($user->caption,$parameters);

Probably once is called, never call again, even if Translator has already loaded translations and this call is only an array lookup key.

What you think about this?
i try to remain in simple cases, because if we go further we enter in complex cases and as we said we don't want to have a complete translation system by default in atk4

[abbadon1334]

Unless there's a huge amount of text (and there really isn't) there isn't much reason to split the files in translation too much probably.
Since this translation sysmem relies on a domain of the translation, why not make it so it's like
/ui/i18n/{language}/{domain}.json, that way you can safely input new domains and update the current one.
Maybe have a fallback for when there's no domain, but I don't remember if that's an option or not.

i think ui.json can be enough for ui

[romaninsh]

@pkly - each repository should hold it's own translations. We can't have UI hold all the dependency translations and what about add-ons?

[romaninsh]

@abbadon1334 remember I suggested that we make "translator" a development dependency for the core? This way we can test the messages and also make use of JSON lib.

But we also need to use namespace -> path resolution.

[romaninsh]

Either translations in each library OR create atk4/translations as a separate repository and store all strings there.

HOWEVER - making PRs on multiple repositories is always a pain, so I prefer this:

1. have it in same repository
2. i18n is not a very good name, perhaps translation would be ok.
3. is composer able to help us with routing those?
4. while test-suite is executed, we can have a "Mock Translator" which would throw exception if any un-translated message is processed, thus ensuring translation coverage.

[pkly]

@abbadon1334 the caption solution is already mediocre when it comes to translations, since you cannot run functions when defining member variables in a class in PHP, which is what you need when translating obviously.
I've already ran into this problem and solved it by initializing the caption in the init function (or the constructor should work too I suppose)

Since the caption only really applies to the name of the model, why bother with holding plural forms in it? I like the idea of adding a new variable a bit better, but I'll just use whatever's required.

@romaninsh
While there's some merit to adding translations to each library separately you'll end up with a mess. Since actually translating text is something that's not done by the default TranslationTrait, you'll end up needing something to actually figure out what languages are available, what strings and where to load them from.

If you'd like to keep translations separate from the UI, to make them accessible without actually using atk4/ui then making a new repository for them seems like the best choice. It could also be kept neat in terms of languages and translations, since they could just be added with pull requests, and it could release updates way more often, since the data in it would be mostly static.
Like you said, it would require a bit more work, since you'd have to input the "normal" string into your code and then make a pull request.

I'd suggest simply adding a i18n or translations folder to any repo that needs them and simply putting the language files there then. I'm not super keen on doing this, but if you wanna do it this way I'm not gonna argue. Both ways have issues.

As for resolution of namespaces and paths - why not simply make a class that fetches those, and allow it to "get" more directories (absolute paths only for sanity's sake), which will in turn contain additional translations? That way you could have addons contain their translations, and with a single line of code "register" themselves into the translator resolution.

It could be a bit problematic though, as you're not using singletons. There's no real way to refer to the resolution class easily by everything at the same time, unless it'd be contained in the AppScopeTrait, or something similar.

I don't agree that i18n is a bad name. It's widely used.

I can't really give you any input towards composer-related things, as I don't use composer.

[romaninsh]

Makes sense.

How about we integrate https://lokalise.co/how-it-works ?

So we need provide strings file (https://docs.lokalise.co/en/collections/652248-supported-file-formats) and the platform can be used for translators to fill in the gaps and generate & download translations!

Basically - lokalize can pull the original strings, let people translate on-line and then automatically PR back translations into original repo. It also works with multiple repositories, so shouldn't be an issue to connect those too.

[pkly]

Isn't that just overkill? I'd suggest focusing on actually getting translations running and then having volunteers translating text as needed.

What you could do is write a script that'd fetch all the strings that are used in the ->_() functions, and just parse that. It could probably be done with a regex and go over every repo that uses those functions.

That way you could automatically create the english base files and make it easier for other people (like me) to translate it into their languages.

[PhilippGrashoff]

Volunteering for German, just tell me what to do :)

[DarkSide666]

https://poedit.net/wordpress for Wordpress _() methods, but maybe it could also work for ->_()

[pkly]

Maybe leave automatic translation for now and simply focus on getting it out?

[romaninsh]

Apologies I haven't had much time on this, so I didn't integrate the platform yet. I'll try to set up integration this week.

[pkly]

Kinda sucks that basically nothing happened with this still.
I'm unsure why you can't just push updates to atk4 and allow the code to run using translations, even if limited in actual languages.

[romaninsh]

you're right, let me follow-up...

[pkly]

I saw that @abbadon1334 is working on i18next implementation (I don't really like his version much because of how atk4 centric it is, so I'll actually try working on my own port), but from what I understand the dependancy on the translation trait for atk4 could easily be done already a month ago. (without depending on a specific library to translate)

[acicovic]

Hi guys,

If I understand correctly, this must be done so we can proceed further?

What's the status of this?

Thanks!

[mvorisek]

related with atk4/ui#715, atk4/ui#1482 and atk4/ui#1833