-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Avoid intermediate values when serializing and deserializing proto3 JSON #670
base: master
Are you sure you want to change the base?
Conversation
This uses jsontool library to avoid allocating intermediate values when generating proto3 JSON strings. Note that the code is not tested for correctness yet, and well-known types are not supported. AOT results: Before: ToProto3JsonString(RunTime): 40474.54 us. ToProto3JsonString(RunTime): 40340.84 us. ToProto3JsonString(RunTime): 40403.9 us. After: ToProto3JsonString(RunTime): 26978.906666666666 us. ToProto3JsonString(RunTime): 26891.893333333333 us. ToProto3JsonString(RunTime): 26945.04 us. Diff: -33% JS results: Before: ToProto3JsonString(RunTime): 71551.72413793103 us. ToProto3JsonString(RunTime): 72103.44827586207 us. ToProto3JsonString(RunTime): 72035.71428571429 us. After: ToProto3JsonString(RunTime): 45422.22222222222 us. ToProto3JsonString(RunTime): 45266.666666666664 us. ToProto3JsonString(RunTime): 46813.95348837209 us. Diff: -36%
@@ -223,6 +223,13 @@ abstract class GeneratedMessage { | |||
{TypeRegistry typeRegistry = const TypeRegistry.empty()}) => | |||
_writeToProto3Json(_fieldSet, typeRegistry); | |||
|
|||
String toProto3JsonString( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider passing an "indent" value
AOT before: FromProto3JsonString(RunTime): 33019.11475409836 us. FromProto3JsonString(RunTime): 32874.45901639344 us. FromProto3JsonString(RunTime): 33174.40983606558 us. AOT after: FromProto3JsonString(RunTime): 22567.494382022473 us. FromProto3JsonString(RunTime): 22654.606741573032 us. FromProto3JsonString(RunTime): 22849.06818181818 us. Diff: -31% JS before: FromProto3JsonString(RunTime): 34568.96551724138 us. FromProto3JsonString(RunTime): 35368.42105263158 us. FromProto3JsonString(RunTime): 35892.857142857145 us. JS after: FromProto3JsonString(RunTime): 85875 us. FromProto3JsonString(RunTime): 83916.66666666667 us. FromProto3JsonString(RunTime): 82880 us. Diff: +130%
@@ -223,6 +223,13 @@ abstract class GeneratedMessage { | |||
{TypeRegistry typeRegistry = const TypeRegistry.empty()}) => | |||
_writeToProto3Json(_fieldSet, typeRegistry); | |||
|
|||
String toProto3JsonString( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this really be called writeToProto3Json
to maintain consistency?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additions in this PR will be inconsistent unless we want to do breaking changes.
Currently in master branch (and also in the latest release) we have these generators:
- writeToBuffer
- writeToCodedBufferWriter
- writeToJson
- writeToJsonMap
- toProto3Json
So the proto3 method is already inconsistent.
Without changing it, we can name the new methods toProto3...
, or writeToProto3...
. Either way it will be inconsistent.
Should we rename toProto3Json
? @sigurdm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Argh - I see - I introduced that inconsistency - not sure what is the better path forwards.
I guess if we want to be consistent we do:
@Deprecated('Use `writeToProto3JsonMap`');
toProto3Json() => writeToProto3JsonMap();
writeToProto3JsonMap();
writeToProto3Json();
If we prefer to avoid churn over consistency we keep it as toProto3JsonString()
....
I guess it's up to your judgement.
Looking at the benchmark results again, we have 3 regressions:
If we keep the current JSON object parser in This means extra ~300 lines of code to maintain, per JS format.. When testing, we can avoid duplicating test code by running the tests on JS as well. Since tests are ordinary Dart programs we can compile them to JS with |
I'm splitting this into two PRs, for generating JSON and parsing JSON. #683 updates JSON generators. |
This uses jsontool library to avoid allocating intermediate values when
generating and parsing proto3 JSON strings.
AOT results
JS results
Fixes #662
cc @rakudrama @lrhn