Add ITranslationProvider.LoadTranslationsAsync() #15886

hishamco · 2024-04-26T10:11:33Z

No breaking changes

coderabbitai · 2024-04-26T10:11:45Z

📝 Walkthrough

Walkthrough

The updates across OrchardCore's localization modules primarily focus on transitioning from synchronous to asynchronous methods. This shift enhances the system's efficiency by supporting non-blocking operations, especially beneficial when loading translations from various data stores. Key interfaces and classes within the localization framework have been refactored to include async methods, with corresponding deprecations of their synchronous counterparts.

Changes

File Path	Summary of Changes
`src/OrchardCore/OrchardCore.Localization.Abstractions/IPluralStringLocalizer.cs`	Modified `IPluralStringLocalizer` interface to include `GetTranslation` method directly within the interface definition, removing nested interfaces.
`src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectHtmlLocalizer.cs` `src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectStringLocalizer.cs`	Updated `PortableObjectHtmlLocalizer` to use asynchronous method `GetTranslationAsync` and `GetAwaiter().GetResult()`. Converted synchronous methods to asynchronous in `PortableObjectStringLocalizer.cs` for improved performance and responsiveness.
`src/OrchardCore/OrchardCore.Localization.Abstractions/ITranslationProvider.cs` `src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs` `src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs` `src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoFilesTranslationsProvider.cs`	Refactored interfaces and classes across these files to introduce asynchronous methods for loading translations, deprecating synchronous methods and enhancing performance through async operations.

Assessment against linked issues

Objective	Addressed	Explanation
Async support for loading translations (#11356)	✅
Non-blocking operations for data stores (#11356)	✅

The changes effectively address the primary objectives outlined in the linked issue #11356 by incorporating asynchronous methods across the localization interfaces and their implementations. This refactoring ensures that the system can handle translation loading operations asynchronously, thereby avoiding blocking calls and enhancing performance when interacting with various data storage solutions.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 4

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectStringLocalizer.cs

coderabbitai

Actionable comments posted: 4

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectStringLocalizer.cs

coderabbitai

Actionable comments posted: 1

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectHtmlLocalizer.cs

Piedone

Since the ultimate consumers of these services, PortableObjectHtmlLocalizer and PortableObjectStringLocalizer retrieve the localization values synchronously, I doubt we get a performance benefit but there is a chance of deadlocks.

Please provide end-to-end benchmarks for these services so we can see if we gain something.

src/OrchardCore/OrchardCore.Localization.Abstractions/ITranslationProvider.cs

Piedone · 2024-04-26T14:06:35Z

src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs

+    /// <inheritdoc />
+    public async Task<CultureDictionary> GetDictionaryAsync(CultureInfo culture)
+    {
+        var cachedDictionary = _cache.GetOrCreate(CacheKeyPrefix + culture.Name, k => new Lazy<Task<CultureDictionary>>(async () =>


This now caches not the dictionary but a Task that produces the dictionary. While this looks working at a first glance, this is not what should really happen. Rather, the dictionary itself needs to be cached. I don't think the Lazy makes sense here anyway.

Piedone · 2024-04-26T14:07:21Z

src/OrchardCore/OrchardCore.Localization.Core/OrchardCore.Localization.Core.csproj

+  <ItemGroup>
+    <PackageReference Include="System.Linq.Async" />
+  </ItemGroup>


This doesn't seem necessary.

It's necessary to convert from IAsyncEnumerable to Enumberable

It still builds if I remove it.

Really!! I got 4 errors, how the IAsyncEnumerable extension methods be there without referencing a proper package?

I dunno :).

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs

Co-authored-by: Zoltán Lehóczky <[email protected]>

coderabbitai

Actionable comments posted: 0

Out of diff range and nitpick comments (1)

src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs (1)

39-62: Consider caching the dictionary directly instead of the task to avoid potential issues with multiple concurrent requests.

This reverts commit 93a802e.

Piedone · 2024-05-02T00:02:33Z

src/OrchardCore/OrchardCore.Localization.Abstractions/ILocalizationManager.cs

If the other changes in the PR prove to be suitable, then ILocalizationFileLocationProvider can be made async too.

Why, it just retrieve a set of locations

sebastienros · 2024-05-02T00:15:05Z

src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs

-        /// <inheritdoc />
-        public CultureDictionary GetDictionary(CultureInfo culture)
-        {
-            var cachedDictionary = _cache.GetOrCreate(CacheKeyPrefix + culture.Name, k => new Lazy<CultureDictionary>(() =>


The Lazyness here is important as it prevents r-entrancy. Without it multiple threads (concurrent requests?) can get into the body and run. The MemoryCache is not blocking on the factory body. With the Lazy, all concurrent requests can potentially create a Lazy instance, but only one will be added to the cache and executed.

sebastienros · 2024-05-02T00:16:56Z

src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs

+    /// <inheritdoc />
+    public async Task<CultureDictionary> GetDictionaryAsync(CultureInfo culture)
+    {
+        var cachedDictionary = await _cache.GetOrCreateAsync(CacheKeyPrefix + culture.Name, async (e) =>


Sicne we don't want entries to be revoked here (we don't, right? even on memory pressure), we can use a keyed Dictionary and remove the prefix on the key (because there can't be conflicts anymore). I did it recently, I am sure you will recall.

Can we use ConccurentDictionary?

Yes, a ConcurrentDictionary actually since a culture could be read while another one is being added to it. But still a keyed service so we don't have to use IMemoryCache. We should only use it when we need expiration rules which we don't want here.

Please also add a comment in the code to remember why we need a ConcurrentDictionary (I will forget myself)

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs

sebastienros · 2024-05-02T00:19:57Z

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs

 {
+    private static readonly Dictionary<char, char> _escapeTranslations = new()


Perfect candidate for FrozenDictionary.
Or just remove this dictionary and use a switch the compiler is clever enough to use a dictionary of ifs based on the number of items for best performance, so we don't have to think about it too much.

I will create another PR for that because it might need extra testing and benchmarking

github-actions · 2024-05-02T14:30:02Z

This pull request has merge conflicts. Please resolve those before requesting a review.

…lizedString>

github-actions · 2024-08-07T21:36:31Z

This pull request has merge conflicts. Please resolve those before requesting a review.

MikeAlhayek · 2024-08-15T17:43:01Z

@hishamco for instructions on how to resolve the merge conflicts due to #16572 please follow the step listed in this comment.

# Conflicts: # src/OrchardCore/OrchardCore.Localization.Abstractions/ILocalizationManager.cs # src/OrchardCore/OrchardCore.Localization.Abstractions/IPluralStringLocalizer.cs # src/OrchardCore/OrchardCore.Localization.Abstractions/ITranslationProvider.cs # src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs # src/OrchardCore/OrchardCore.Localization.Core/OrchardCore.Localization.Core.csproj # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoFilesTranslationsProvider.cs # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectHtmlLocalizer.cs # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectStringLocalizer.cs # test/OrchardCore.Tests/Localization/LocalizationManagerTests.cs # test/OrchardCore.Tests/Localization/PoParserTests.cs # test/OrchardCore.Tests/Localization/PortableObjectStringLocalizerTests.cs

sebastienros · 2024-09-05T17:55:34Z

src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs

-                {
-                    translationProvider.LoadTranslations(culture.Name, dictionary);
-                }
+            var dictionary = new CultureDictionary(culture.Name, rule ?? _defaultPluralRule);


The LoadTranslationsAsync should be move to a private AwaitedLoadtranslationAsync method with the async/await keywords such that we don't need it in the GetDictionaryAsync.

sebastienros · 2024-09-05T18:00:09Z

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectHtmlLocalizer.cs

@@ -41,7 +41,7 @@ public override LocalizedHtmlString this[string name]
                if (_localizer is IPluralStringLocalizer pluralLocalizer && arguments.Length == 1 && arguments[0] is PluralizationArgument)
                {
                    // Get an unformatted string and all non plural arguments (1st one is the plural count).
-                    var (translation, argumentsWithCount) = pluralLocalizer.GetTranslation(name, arguments);
+                    var (translation, argumentsWithCount) = pluralLocalizer.GetTranslationAsync(name, arguments).GetAwaiter().GetResult();


All this PR to come back to this pitfall, for every single invokation of the localizer. It was better before, even if it was not reading the file asynchronously once)

Maybe because we didn't do it the right way from the beginning :)

github-actions · 2024-11-05T00:57:34Z

It seems that this pull request didn't really move for quite a while. Is this something you'd like to revisit any time soon or should we close? Please comment if you'd like to pick it up.

# Conflicts: # src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoFilesTranslationsProvider.cs # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs # src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectStringLocalizer.cs

Add ITranslationProvider.LoadTranslationsAsync()

11d28ed

hishamco requested a review from sebastienros April 26, 2024 10:13

coderabbitai bot reviewed Apr 26, 2024

View reviewed changes

IPluralStringLocalizer.GetTranslationAsync()

62a0e0b

coderabbitai bot reviewed Apr 26, 2024

View reviewed changes

Fix the build

f8246fd

coderabbitai bot reviewed Apr 26, 2024

View reviewed changes

src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PortableObjectHtmlLocalizer.cs Outdated Show resolved Hide resolved

Piedone requested changes Apr 26, 2024

View reviewed changes

hishamco and others added 2 commits April 26, 2024 20:33

Apply suggestions from code review

0041fbf

Co-authored-by: Zoltán Lehóczky <[email protected]>

Address feedback

ee4a668

coderabbitai bot reviewed Apr 26, 2024

View reviewed changes

Use IAsyncEnumerable

93a802e

This comment was marked as off-topic.

Sign in to view

Revert "Use IAsyncEnumerable"

3035a04

This reverts commit 93a802e.

This was referenced May 1, 2024

Monthly community metrics report for 2024-04-01..2024-04-30 #15929

Closed

Monthly community metrics report for 2024-04-01..2024-04-30 Emmanjr/OrchardCore#2

Open

Piedone reviewed May 2, 2024

View reviewed changes

Piedone mentioned this pull request May 2, 2024

Remove unnecessary async code #15941

Merged

sebastienros reviewed May 2, 2024

View reviewed changes

hishamco mentioned this pull request May 2, 2024

PoParser PERF #15943

Merged

github-actions bot added the merge conflict label May 2, 2024

Merge branch 'main' into hishamco/translation-provider

239bc7f

github-actions bot removed the merge conflict label May 2, 2024

hishamco added 3 commits May 3, 2024 05:55

Merge branch 'main' into hishamco/translation-provider

5cd5a85

Fix merge conflict

8fdfc09

GetAllStringsFromCultureHierarchyAsync() return IAsyncEnumerable<Loca…

47050eb

…lizedString>

github-actions bot added the merge conflict label Aug 7, 2024

hishamco added 2 commits August 15, 2024 22:40

Merge branch 'main' into hishamco/translation-provider

431cb66

sebastienros reviewed Sep 5, 2024

View reviewed changes

github-actions bot added the stale label Nov 5, 2024

hishamco removed the stale label Nov 5, 2024

hishamco added 2 commits December 13, 2024 22:48

Update

89330e1

hishamco force-pushed the hishamco/translation-provider branch from ef0a4b2 to 89330e1 Compare December 14, 2024 00:05

github-actions bot removed the merge conflict label Dec 14, 2024

Merge branch 'main' into hishamco/translation-provider

bebc0af

		{
		private static readonly Dictionary<char, char> _escapeTranslations = new()

Add ITranslationProvider.LoadTranslationsAsync() #15886

Are you sure you want to change the base?

Add ITranslationProvider.LoadTranslationsAsync() #15886

Conversation

hishamco commented Apr 26, 2024

coderabbitai bot commented Apr 26, 2024 • edited Loading

Walkthrough

Changes

Assessment against linked issues

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

Documentation and Community

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

Piedone left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented May 2, 2024

github-actions bot commented Aug 7, 2024

MikeAlhayek commented Aug 15, 2024

Choose a reason for hiding this comment

sebastienros Sep 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 5, 2024

coderabbitai bot commented Apr 26, 2024 •

edited

Loading

sebastienros Sep 5, 2024 •

edited

Loading