Any GRPC client code example running on browser using j2cl and Bazel?

[seyoung-song-walmart-com]

Hi,
Is there any example GRPC client code using j2cl and Bazel?

[sgammon]

@seyoung-song-walmart-com Having investigated this before as a user (I'm not a Googler), you will probably Have a Bad Time, unless things have changed recently. I would humbly recommend Buf and their Connect RPC protocol

[niloc132]

Can you comment on getting Buf to work with J2CL?

[niloc132]

I've spent a lot of time over the last few years on this, and the state of affairs is not great. I'm looking to improve things, but I don't have a mandate/funding to aggressively pursue this at this time.

My frame of reference here is using Apache Arrow Flight, which is protobuf-over-gRPC, with flatbuffer payloads in some of the protobuf fields, and then some columnar data in another protobuf field to represent the data itself. It was a business requirement to use these tools, so we tried to do as little proxying on top of this as possible (e.g. Buf's "just make everything JSON" or "just make the protobuf binary payload to be base64'd text instead").

Streaming was a requirement (and is purported to be one of the great features that gRPC has), and other platforms were already capable of using gRPC... and gRPC-web's documentation describes it as being a browser-compatible implementation of gRPC.

The Good:

• gRPC isn't actually compatible with browsers at all... but there are simple proxies to transform to grpc-web, and gRPC-web itself is fairly simple, it isn't too hard to make your own client that runs in the browser. There are also JS and TS clients available
• https://github.com/google/j2cl-protobuf is an existing protobuf implementation unofficially provided by Google

That's it I'm afraid, the rest of the news is bad or how to slightly overcome the bad.

The Bad:

• The official gRPC-web client can only handle unary (i.e. one request, one response) calls fully. Server-stream (one request, many responses) is supported if the request/response are text - no binary data allowed. No client-stream, no bidirectional stream.
• The j2cl-protobuf implementation above also only supports text (this could have changed, I haven't looked in the last many months).
• The jsinterop-generator tool cannot produce usable jsinterop bindings from the official grpc-web client or its generated closure-compatible output. See @typedef on a property fails to build jsinterop-generator#45, Giving a class both a static and instance members with a matching union type causes invalid java to be generated jsinterop-generator#46, If a static property and an instance property are defined on the same type, the instance property is omitted jsinterop-generator#47.
• There are alternative implementations of grpc-web out there, https://github.com/improbable-eng/grpc-web/ is the best we found - it does support server-streaming binary payloads, but omitting client-streaming is a limitation of browser APIs. Also, that implementation is unmaintained.
• gRPC itself requires http/2 - technically grpc-web does not require this, but for streaming use cases it practically does demand h2. In turn, browsers don't support h2 without SSL, so either all local development is using self signed certs, or you have annoying timeouts and limits to how many streams you can have open, or you need an alternate transport for grpc. We started with the alternative implementation above which provides a websocket, but websockets are expensive to initiate, and we hit the limit of how many we could create, so made our own multiplexed implementation.

What worked for us:

• We cooked up a mess of bash+sed on the alternative implementation of grpc-web (and the official protobuf-js, which does support binary) that can emit nearly-valid Java+JsInterop, but a few changes have to be made by hand due to the jsinterop-generator bugs. The process is to read typescript into tsickle, emit externs, read those externs into jsinterop-base, emit Java+JsInterop.
• Unary and server streaming works great over h2 and grpc-web in this way. Client and bidirectional stream calls over h2 are replaced by a pair of calls, one for the server stream, and another call (with extra metadata to point to the server stream and to sequence the client calls) to make when the client wants to send a payload.
• All of this works nicely over websockets when h2 is not available... but browsers don't have an API to indicate if h2 is in use, so some assumptions must be made here.
• We also didn't want to run separate processes for the grpc-to-grpc-web proxy, so we first helped to ship the grpc-servlet implementation, then created a servlet filter to transparently in-process translate from grpc to grpc-web, and a websocket handler to do the same (all Apache licensed, though most of the rest of the server is merely "source available").

Finally, note that we're still using GWT 2, due to to a number of factors, but the specific code mentioned here should all be compatible with J2CL as well.

All in all, this gets us all four types of gRPC calls, with binary payloads, and no extra proxies required, working with local development. This has been deployed in production for quite a while - we're hitting bugs in Chrome (kills h2 calls if unrelated network changes happen) and Envoy (ending streams will sometimes throw away DATA and HEADER frames in envoy rather than forward them), but no complaints with keeping our client logic working over gRPC-web with Java.

[seyoung-song-walmart-com]

Thank you very much for detail information!