Rework template translation finding

[rowanseymour]

So that it can work with locales that we get from WhatsApp like en-US using a smart matcher and so we don't have to map those on the RP side and then map them back in courier.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b00666e) 87.66% compared to head (323786c) 87.64%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1197      +/-   ##
==========================================
- Coverage   87.66%   87.64%   -0.02%     
==========================================
  Files         262      262              
  Lines       10961    10952       -9     
==========================================
- Hits         9609     9599      -10     
- Misses        929      930       +1     
  Partials      423      423              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rowanseymour]

this case is a little weird because it's just defaulting to the first translation... but it does seem better to generally return something here than creating a message without a template that probably going to fail.

[norkans7]

the language from WA are like en_GB, es_MX not exactly en-GB and es-MX, so we substitute the underscore with the hyphen somewhere?

[rowanseymour]

@norkans7 yeah I think we can say that locale is always <lang>-<COUNTRY> - and for flows and contacts language is always 3 chars, but for external stuff it might sometimes be only 2-chars.

[rowanseymour]

Been thinking about this a bit more I think we when syncing we should store both the original language code, and our current language (3 chars) + country fields on the template translation object. That way the engine can continue to work with xxx or xxx-YY format locales, but in mailroom, when queuing the message to courier, we can lookup the original language code on the translation object, and queue that.