How to manage installers dedicated for each language?

[auanasgheps]

Description of the new feature/enhancement

How should we manage those apps that have a dedicated installer for each language?
Popular packages are Firefox and WinRAR.

I tried using "Add Locale" within YAMLCreate.ps1, but only allows the app info to be localized, not an installer.

I tried adding manually another installer to the Mozilla.Firefox.installer.yaml, but the validator marked the additional package as duplicated.

Proposed technical implementation details (optional)

When using Add Locale feature, add the option for a dedicated installer.
The main PackageName.installer.yaml will still have a default English package installed by default, but if the user locale or setting is set to a specific language, install the package from the corresponding language YAML.

I hope it makes sense.

[OfficialEsco]

As the one who reworked the YAMLCreate and created the "Add Locale" function i can say i did not think about this, the "Add Locale" function is basically only to write the information in another language..

Firefox 82.0.1 is little example of how to do en-US and es-MX and here is a installer manifest with multiple languages but no locale manifest.
The problem with Firefox is that they use a different locale code than Windows so its a bit hard to know which url goes to which locale... (The manifest with multiple manifest is correct, but i removed quite a few)

If someone knows Powershell they can easily modify the Reworked YamlCreate, however the WinGet team is working on winget create which hopefully includes what my script does, just better and more functions 😄

Feel free to ask any questions around manifest creation :)

[denelon]

This is a particularly challenging area. We do support multiple installer nodes in a way you can specify the locale for the installer with BCP47 format. You would specify the "InstallerLocale" in the "Installer" node. Note: this does require the multi-file manifest structure.

[jedieaston]

I just did some messing around with the Firefox 79 manifest (the one with all of the possible languages) to add arm64 and x86 versions (as Mozilla puts out different installers for those), and it looks like there's a limit in the schema of 128 installers per manifest (with every language and architecture permutation, there's around 231(!) installers for Firefox).

PS C:\Users\Easton Pillay\projects\winget-pkgs\manifests\m\Mozilla\Firefox\79.0> winget validate .
Manifest validation failed.
Manifest Error: Schema validation failed.
Error context: <root>[Installers] Description: Array should contain no more than 128 elements.
Error context: <root> Description: Failed to validate against schema associated with property name 'Installers'.
 File: Mozilla.Firefox.installer.yaml

PS C:\Users\Easton Pillay\projects\winget-pkgs\manifests\m\Mozilla\Firefox\79.0>   

@denelon is there any chance this limit can be increased in the current version of the schema? This is an odd circumstance, but probably one we'll run into for other apps as well.

I've attached the file below so someone can check to make sure I didn't duplicate some by mistake (I triple checked this time though, so....).

Mozilla.Firefox.installer.yaml.txt

[OfficialEsco]

@jedieaston did you use the locales i kept or did you add some your self?
I am 100% sure i deleted a few because i could not find a BCP47 format for it...
According to Chocolatey's Firefox LanguageChecksum.csv there is 97 Languages which then brings us up to 291 installers 😮

[denelon]

512 installers? 🤦‍♂️

[auanasgheps]

Why not manage them individually in each file? Wouldn't be easier?

[jedieaston]

@OfficialEsco I used the locales that were already in the manifest (that you added). It's possible that Chocolatey isn't checking to make sure there is a matching BCP47 locale for it, I'd have to look at that CSV file again to make sure.

But yes @denelon, it looks like 512 should be the new limit (at least). Can the bot verify that many installers for a single manifest now without getting stuck?

[jedieaston]

Thinking about it for Firefox, if we wanted to support user installation too (which is not how it's set up now but we could), we'd have to add another installer listing for that too, so 290*2 is 580. Maybe we need 1024?

I can hear @wingetbot sobbing.

[auanasgheps]

Thanks for clarifying, but is there a better way of approaching this?
Having all installers is a single file might be challenging. Why not adding each of them to their language yaml?

[denelon]

Firefox is currently an outlier compared with the bulk of the other packages. Most packages handle localization internally with a single installer and resource files rather than unique builds per locale.

[denelon]

I'm beginning to think it might be better to see if it would be worth looking at changing how Firefox is deployed. If they could upgrade to MSIX, or even MSI, and do localization in resource files this scenario would essentially "go away". I'm concerned about how long it would take to validate literally hundreds of installers, and then try to fix the "one" or "two" that for some reason fail validation. It seems quite challenging.

[auanasgheps]

WinRAR does the same, but it's a very small package. Don't know other apps, but surely there are more.

Don't know if Firefox will ever make this change: I've used it since a decade and there's always been dedicated installers as far as I remember. Either way: who's going to have a conversation with them? :)

Having a dedicated installer in each language file would ease the trouble when validating an update, if an error is found only one installer needs to be verified again.

[jedieaston]

if an error is found only one installer needs to be verified again.

Hypothetically the same thing could be done via a Git diff, the bot could just see what entries in the installer array changed and test those. And then when updating to a new version of Firefox we don't have to go through 90 files and update them, we can just update 1. The locale specific info shouldn't be changing much once it is translated a single time, version numbers are generally language independent.

Firefox has been this way since 3.x, I concur someone should reach out (presumably there's some kind of carrier pigeon Microsoft can send out when they'd like to talk to a Windows ISV) but if that won't work, perhaps the parallel jobs stuff in Azure Pipelines is worth a look?

(There'd have to be a bunch of restructuring to the way the pipeline works probably, depending on what exactly the installer verification step is doing. But if it's just "spin up Windows VM -> run winget with argument -> check output" that is rather parallelizable since you can just spin up a Windows VM for every installer entry. Unless you're trying to conserve licensing costs.. ;D)

[auanasgheps]

Thanks for clarifying.
Thunderbird also follows the same distribution logic.