Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialization performance is poor for large products #44

Open
henryem opened this issue Jun 15, 2019 · 0 comments
Open

Deserialization performance is poor for large products #44

henryem opened this issue Jun 15, 2019 · 0 comments

Comments

@henryem
Copy link
Contributor

henryem commented Jun 15, 2019

Currently, as noted in the project page, we use an O(N*M) algorithm for deserializing products with N fields repeated a total of M times. In profiling the deserialization of a complex deeply-nested object, I found that we end up spending a huge amount of time in the resulting calls to CodedInputStream.skipField.

One way to eliminate this is to (1) create a map from field index to parser, (2) loop through fields, passing each to its appropriate parser, and (3) finally build the results of each parser. I have prepared a PR that does this, which I'll be submitting soon (once my previous PR, which is a dependency, is merged). My PR also reduces memory pressure for deserializing nested objects by eliminating allocation of extra byte buffers in all cases except coproduct parsing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant