Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make package loading lazy #126

Open
da-tanabe opened this issue Sep 22, 2020 · 0 comments
Open

Make package loading lazy #126

da-tanabe opened this issue Sep 22, 2020 · 0 comments

Comments

@da-tanabe
Copy link
Contributor

The performance improvements from #125 have uncovered an unexpected problem—race conditions in package loading as compared to transaction reads. For a single dazl application talking to a ledger, this is not a problem because dazl internally sequences package loading before transaction stream reading. However, when other applications are also talking to the ledger (and also loading their own models and creating contracts), dazl of course can't impose order of package loading/transaction reading in the same way, and will typically fail to read contracts that come in "around the same time" as a package loading. The previous behavior slowed dazl down enough that hitting this race condition was comparatively rarer (but it still did exist).

Separately, dazl application startup time could still be faster; sequential package loading at startup means that all DARs of a ledger are eagerly loaded, even if they are never used. This means a dazl application talking to a ledger of even moderate DAR counts (5–10) can take a few minutes to launch, regardless of how many packages are required to understand the ACS of a ledger.

The fix to both of these problems is to make package loading be lazier—upon realizing that a package ID is required, "block" and load the corresponding package from the Ledger API; only once the package is fully processed and parsed, continue. Separately, as an optimization, we may elect to continue reading packages in the background that are not yet needed but might be needed in the future for reads (or for the write side on unqualified template names), but this should not be in conflict with allowing reads of packages to occur when encountering previously unknown package IDs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant