Wire Serialization Performance

[jinxidoru]

I recently found Wire and was initially very excited about it as an alternative to the standard protobuf compiler for Java. I like the style of java that is output. But I was disappointed when I ran a few performance tests against it compared to the standard compiler. For serialization, I saw a 10x speed drop with the Wire implementation. I saw serialization for a particular protobuf message go from an average of 80ms to 800ms+. Note that my particular use-case involves some very large protobuf messages. This particular one was 23MB in size after serialization. But that level of performance degradation is not really reasonable for me.

I'm wondering if there's something wrong in the way I'm using the library. I tried to find documentation on optimization or performance concerns, but could not find anything. I thought that perhaps the first execution would be slow until the system had an opportunity to cache the serializer, but subsequent executions were roughly the same speed, maybe ever so slightly faster. So yeah... I'm just trying to find a way to improve serialization speed.

[JakeWharton]

Without access to the benchmarks it's impossible to know what you saw before we even begin to find the root cause as to the performance discrepancy which would have to happen before we made any recommendations let alone changes.

[jinxidoru]

Ok, I'll explain what I've done and the results I've seen. Then I'll post a link to a repo that contains the code. Thank you so much for looking at this. I have created a very simple protobuf file which has a main message (Foo) which contains a repeated object full of 50,000 Bar objects. The Bar class contains a small collection of strings and int32s. I have constructed this both with wire and protobuf and compared the amount of time it takes to convert to a 16MB byte array. I get the following results on my computer. Note that wire is roughly 2.5x slower than protobuf.

wire........ 189ms  (16269542B)
protobuf.... 79ms  (16269542B)

I then started to play around with the contents of Bar. If I remove all but the first string, wire is actually faster (28ms vs 45ms). The minute I add in one of the integer fields, the wire serialization jumps up significantly (108ms vs 46ms). Note that protobuf didn't increase by any significant margin, but wire jumped by 80ms with a simple integral field. Note that the value of the file is fairly small, topping out just over a million. With each additional integer added back in, I saw a sizable increase in time, though each delta steadily decreased.

Alternatively, I tried adding the second string field back in. That led to an increase from 28ms to 53ms. That seems reasonable since we are basically doubling the effort (one string to two strings). But note that it is a much bigger increase than we saw when adding the int32. It seems very odd that an integer field would have a bigger impact on performance than a string. The opposite case would make far more sense to me.

It appears to me that there must be an issue in the handling of integers. Let me know if there's any more information you need from me.

[jinxidoru]

https://github.com/jinxidoru/wire-vs-protobuf

[jinxidoru]

./gradlew run

[oldergod]

We have a benchmark module in Wire as well https://github.com/square/wire/tree/master/wire-benchmarks

I'll try (next year) to port your sample to it, unless you feel like doing so in the meantime.

[jinxidoru]

Sorry for all the messages, but I've been poking at this a bit more and wanted to relay to you the weirdness that I'm seeing. First of all, I missed a test case in my previous message. I was thinking that it was the addition of int32 fields that messed up the performance. No, it actually appears to be the interplay between int32 and string. So a message with just int32 fields or just string is fast. The minute you have both, performance drops through the floor.

Secondly, I've been trying to dig a bit deeper, hoping to find the actual cause but I'm starting to encounter very strange stuff that might exceed my knowledge of the inner workings of Java. The code below takes about 110ms to run on my machine. What's odd is that, if I comment out literally any of the individual write operations, the performance speeds up to around 20-30ms. The absolutely defies explanation for me. What's more, this only occurs with the ReverseProtoWriter. If I switch to ProtoWriter, then the speed immediately drops to 25ms and commenting out a line has an incremental benefit. This is very strange, but maybe this will ring a bell of you.

In any case, in the process of this investigation, I found that ProtoWriter performs nearly as fast as protobuf. It's averaging about 20-30% slower, but that's an acceptable trade-off when considered against the other gains that wire provides.