SchemaLoader.load very slow at runtime

[martypitt]

Hi

I'm trying to parse some generated (in-memory) protobuf schemas to allow reading/writing of protobuf messages.

I've used the RepoBuilder as an example of how to acheive this.

Specifically, I'm using a SchemaLoader with a okio.Fakefilesystem.

When I call schemaLoader.load() I see a very long hang. Interestingly, this doesn't happen in unit tests, only at runtime.

After some digging, it seems to be locking inside CoreLoader. Here's a quick stack of where the issue occurs:

• SchemaLoader.loadSchema() ->
• linker.link(...) ->
• Linker: sourceFiles += getFileLinker("google/protobuf/descriptor.proto") ->
• CoreLoader.load("google/protobuf/descriptor.proto") ->
• ResourceFileSystem.roots lazy init -> toClasspathRoots()
• at this point, we seem to be loading every jar on the classpath and opening it as a zip file. This appears to be where we spend a lot of time.
  • I wonder if this is why this only happens at runtime, given the classpath is much larger

Am I doing something wrong here?
Can I avoid having iterate all the jars?

[martypitt]

After trial and error, I've realised that wire-runtime wasn't on my classpath. I've added it, and the issue seems to have mysteriously stopped - but I can't explain why.

If someone could validate that the above approach is valid / reasonable way of trying to achieve my goal, I guess I can write this one up to 🤷‍♂️

[swankjesse]

Yeah that sounds good. We do something similar in Cash App to implement grpc-reflect.

I suspect it needs wire-runtime for the built-in .proto files.

One warning: our API stability for wire-schema has been much less stable than runtime things. We’ll try to keep this use-case in mind!

[martypitt]

Great, thanks for the response!

Warnng noted, I'll bear that in mind.