Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove serialization workarounds for ie 6/7 and rhino (#9578) #9876

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

codemasterover9000
Copy link
Contributor

Removed the workarounds for IE6/7 issues and rhino. IE 6/7 are not relevant anymore today. Not sure if this is the rhino issue referenced in the code: https://bugzilla.mozilla.org/show_bug.cgi?id=179068. But it seems to have been fixed a long time ago.

Fixes #9578

Copy link
Contributor

@niloc132 niloc132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on, legacy code can be a slog to remove... I will note for posterity that at least up to IE9 (and I think 10?) there is some memory leak that occurs in this code as well, but I can't remember the specifics - but since even IE11 is effectively dead, we should go ahead and ignore this.


It looks like this still leaves the eval() call in the client code, if the server ends up sending version < 8 (and the default is 7), see

if (readVersion(encoded) < SERIALIZATION_STREAM_JSON_VERSION) {
// Versions prior to 8 uses almost JSON with Javascript is special cases; e.g., using ].concat([,
// non-stringified NaN/Infinity or ''+'' concatenated strings.
results = eval(encoded);
and
public static final int SERIALIZATION_STREAM_VERSION = 7;

I can see an argument for removing this in phases, so that old clients will still work - but if that's the plan, I'd want to consider bumping the SERIALIZATION_STREAM_VERSION to 8, and warning on old versions.

Also: the rhino notes are about the legacy dev mode implementation of gwt-rpc (to avoid needing to emulate all the string operations in JS). It is technically possible that this will break GWTTestCase for htmlunit "hosted mode" tests (hosted mode == legacy dev mode)

* The payload is almost a JSON literal except for some nuances, like unicode and array concats.
* We have a specialized devmode version to decode this payload, because the webmode version
* requires multiple round-trips between devmode and the JSVM for every single element in the
* payload. This can require several seconds to decode a single RPC payload. The RpcDecoder
* operates by doing a limited JS parse on the payload within the devmode VM using Rhino,
* avoiding the cross-process RPCs.
.

To deal with that and still kill this old code, I'd consider removing the linked file above and replacing it with the supersource version (the second link in this reply, except with the @GwtScriptOnly removed, and javadoc updated to indicate that it is no longer supersourced).

What do you think?

writePayload(stream);
writeStringTable(stream);
writeHeader(stream);
StringBuffer buffer = new StringBuffer(capacityGuess);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as we're updating this, consider replacing StringBuffer with StringBuilder for better single-threaded performance? From https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/StringBuffer.html

As of release JDK 5, this class has been supplemented with an equivalent class designed for use by a single thread, StringBuilder. The StringBuilder class should generally be used in preference to this one, as it supports all of the same operations but it is faster, as it performs no synchronization.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated to use StringBuilder instead of StringBuffer and changed the default serialization format to JSON.

I'm not sure how best to implement the rest of your suggestions though. But they seem logical.

: Integer.MAX_VALUE;

while (i < length && charVector.getSize() < maxSegmentVectorSize) {
while (i < length) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed. The same loop definition is in the line above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the extra loop

@lgemeinhardt
Copy link
Contributor

Any chance to get this PR in the next release (because this enabled a CSP w/o "unsafe-eval")?

@niloc132
Copy link
Contributor

Anyone who wants to pick this up and continue it, removing other related dead code, would be welcome to do so. I don't presently have a plan to take this on, so we're looking for a contributor or sponsor to focus on it. See notes in my review above for other steps that should be taken to correct this more than superficially.

@codemasterover9000
Copy link
Contributor Author

Would it be possible to just merge this now and make a separate issue for the dead code removal? The current code fails with any site that uses CSP under the specific conditions of the issue as unsafe-* will probably not be included (as it should) when a CSP is active.

@niloc132
Copy link
Contributor

@codemasterover9000 to confirm, the work you're doing here mostly checks two boxes so far:

  • Update the default wire format version from the server from 7 to 8
  • Remove some code that was used to write version 7

Unless I'm mistaken, you could just configure your own project to use version 8, and avoid the codepaths that you are removing in this PR? That would solve this except for too-big payloads (strings longer than MAX_STRING_NODE_LENGTH, - a much smaller PR could update that with no risk of being half finished, and just make that constant able to be configured by another system property, so you could set the limit as high as you wanted (Integer.MAX_VALUE for example).

I'm guessing you haven't run all tests on this PR, but looking briefly at some dev mode tests (which use rhino in legacy dev mode) it appears that some tests may fail from this change. Notably, here's the test that validates that Rhino still has the bug (and so this PR will break it):

/*
* Note: this test verifies a issue with the Rhino parser that limits the size of a single string
* node to 64KB. If this test starts failing, then the Rhino parser may have been fixed to support
* larger strings and the string concat workaround could be removed.
*/
public void testRead_stringOver64KB() {
ClientSerializationStreamReader reader = new ClientSerializationStreamReader(null);
int stringLength = 0xFFFF;
StringBuilder builder = new StringBuilder(stringLength);
for (int i = 0; i < stringLength; i++) {
builder.append('y');
}
// Push the string size over 64KB.
builder.append('z');
try {
reader.prepareToRead("["
+ "1," // String table index
+ "[\"" + builder.toString() + "\"],"
+ "0," // flags
+ AbstractSerializationStream.SERIALIZATION_STREAM_VERSION // version
+ "]");
fail("Expected SerializationException");

Running the *RPC*Suite classes via ant on your branch (which is just old enough that it can easily run in Java 8 for easier legacy dev mode testing) - cd into user/ and run

ant test.dev.htmlunit  '-Dgwt.junit.testcase.includes=**/*RPC*Suite.class'

I show one failure: ValueTypesTest.testString_over64KBWithUnicode()

Testcase: testString_over64KBWithUnicode took 1.59 sec
	Caused an ERROR
The response could not be deserialized
com.google.gwt.user.client.rpc.IncompatibleRemoteServiceException: The response could not be deserialized
	at com.google.gwt.user.client.rpc.impl.RequestCallbackAdapter.onResponseReceived(RequestCallbackAdapter.java:221)
	at com.google.gwt.http.client.Request.fireOnResponseReceived(Request.java:227)
	at com.google.gwt.http.client.RequestBuilder$1.onReadyStateChange(RequestBuilder.java:412)
	at com.google.gwt.dev.shell.MethodAdaptor.invoke(MethodAdaptor.java:103)
	at com.google.gwt.dev.shell.MethodDispatch.invoke(MethodDispatch.java:72)
	at com.google.gwt.dev.shell.OophmSessionHandler.invoke(OophmSessionHandler.java:172)
	at com.google.gwt.dev.shell.BrowserChannelServer.reactToMessagesWhileWaitingForReturn(BrowserChannelServer.java:341)
	at com.google.gwt.dev.shell.BrowserChannelServer.invokeJavascript(BrowserChannelServer.java:222)
	at com.google.gwt.dev.shell.ModuleSpaceOOPHM.doInvoke(ModuleSpaceOOPHM.java:121)
	at com.google.gwt.dev.shell.ModuleSpace.invokeNative(ModuleSpace.java:573)
	at com.google.gwt.dev.shell.ModuleSpace.invokeNativeObject(ModuleSpace.java:293)
	at com.google.gwt.dev.shell.JavaScriptHost.invokeNativeObject(JavaScriptHost.java:91)
	at com.google.gwt.core.client.impl.Impl.apply(Impl.java)
	at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:351)
	at com.google.gwt.dev.shell.MethodAdaptor.invoke(MethodAdaptor.java:103)
	at com.google.gwt.dev.shell.MethodDispatch.invoke(MethodDispatch.java:72)
	at com.google.gwt.dev.shell.OophmSessionHandler.invoke(OophmSessionHandler.java:172)
	at com.google.gwt.dev.shell.BrowserChannelServer.reactToMessages(BrowserChannelServer.java:296)
	at com.google.gwt.dev.shell.BrowserChannelServer.processConnection(BrowserChannelServer.java:551)
	at com.google.gwt.dev.shell.BrowserChannelServer.run(BrowserChannelServer.java:368)
	at java.lang.Thread.run(Thread.java:750)
Caused by: com.google.gwt.user.client.rpc.SerializationException: Failed to parse RPC payload
	at com.google.gwt.user.client.rpc.impl.ClientSerializationStreamReader.prepareToRead(ClientSerializationStreamReader.java:322)
	at com.google.gwt.user.client.rpc.impl.RemoteServiceProxy.createStreamReader(RemoteServiceProxy.java:259)
	at com.google.gwt.user.client.rpc.impl.RequestCallbackAdapter.onResponseReceived(RequestCallbackAdapter.java:214)
Caused by: java.lang.RuntimeException: FAILED ASSERTION
	at com.google.gwt.dev.js.rhino.Context.codeBug(Context.java:769)
	at com.google.gwt.dev.js.rhino.Parser.sourceAddString(Parser.java:1427)
	at com.google.gwt.dev.js.rhino.Parser.primaryExpr(Parser.java:1345)
	at com.google.gwt.dev.js.rhino.Parser.memberExpr(Parser.java:1163)
	at com.google.gwt.dev.js.rhino.Parser.unaryExpr(Parser.java:1079)
	at com.google.gwt.dev.js.rhino.Parser.mulExpr(Parser.java:1031)
	at com.google.gwt.dev.js.rhino.Parser.addExpr(Parser.java:1015)
	at com.google.gwt.dev.js.rhino.Parser.shiftExpr(Parser.java:1003)
	at com.google.gwt.dev.js.rhino.Parser.relExpr(Parser.java:987)
	at com.google.gwt.dev.js.rhino.Parser.eqExpr(Parser.java:976)
	at com.google.gwt.dev.js.rhino.Parser.bitAndExpr(Parser.java:966)
	at com.google.gwt.dev.js.rhino.Parser.bitXorExpr(Parser.java:956)
	at com.google.gwt.dev.js.rhino.Parser.bitOrExpr(Parser.java:946)
	at com.google.gwt.dev.js.rhino.Parser.andExpr(Parser.java:935)
	at com.google.gwt.dev.js.rhino.Parser.orExpr(Parser.java:924)
	at com.google.gwt.dev.js.rhino.Parser.condExpr(Parser.java:908)
	at com.google.gwt.dev.js.rhino.Parser.assignExpr(Parser.java:890)
	at com.google.gwt.dev.js.rhino.Parser.primaryExpr(Parser.java:1253)
	at com.google.gwt.dev.js.rhino.Parser.memberExpr(Parser.java:1163)
	at com.google.gwt.dev.js.rhino.Parser.unaryExpr(Parser.java:1079)
	at com.google.gwt.dev.js.rhino.Parser.mulExpr(Parser.java:1031)
	at com.google.gwt.dev.js.rhino.Parser.addExpr(Parser.java:1015)
	at com.google.gwt.dev.js.rhino.Parser.shiftExpr(Parser.java:1003)
	at com.google.gwt.dev.js.rhino.Parser.relExpr(Parser.java:987)
	at com.google.gwt.dev.js.rhino.Parser.eqExpr(Parser.java:976)
	at com.google.gwt.dev.js.rhino.Parser.bitAndExpr(Parser.java:966)
	at com.google.gwt.dev.js.rhino.Parser.bitXorExpr(Parser.java:956)
	at com.google.gwt.dev.js.rhino.Parser.bitOrExpr(Parser.java:946)
	at com.google.gwt.dev.js.rhino.Parser.andExpr(Parser.java:935)
	at com.google.gwt.dev.js.rhino.Parser.orExpr(Parser.java:924)
	at com.google.gwt.dev.js.rhino.Parser.condExpr(Parser.java:908)
	at com.google.gwt.dev.js.rhino.Parser.assignExpr(Parser.java:890)
	at com.google.gwt.dev.js.rhino.Parser.primaryExpr(Parser.java:1253)
	at com.google.gwt.dev.js.rhino.Parser.memberExpr(Parser.java:1163)
	at com.google.gwt.dev.js.rhino.Parser.unaryExpr(Parser.java:1079)
	at com.google.gwt.dev.js.rhino.Parser.mulExpr(Parser.java:1031)
	at com.google.gwt.dev.js.rhino.Parser.addExpr(Parser.java:1015)
	at com.google.gwt.dev.js.rhino.Parser.shiftExpr(Parser.java:1003)
	at com.google.gwt.dev.js.rhino.Parser.relExpr(Parser.java:987)
	at com.google.gwt.dev.js.rhino.Parser.eqExpr(Parser.java:976)
	at com.google.gwt.dev.js.rhino.Parser.bitAndExpr(Parser.java:966)
	at com.google.gwt.dev.js.rhino.Parser.bitXorExpr(Parser.java:956)
	at com.google.gwt.dev.js.rhino.Parser.bitOrExpr(Parser.java:946)
	at com.google.gwt.dev.js.rhino.Parser.andExpr(Parser.java:935)
	at com.google.gwt.dev.js.rhino.Parser.orExpr(Parser.java:924)
	at com.google.gwt.dev.js.rhino.Parser.condExpr(Parser.java:908)
	at com.google.gwt.dev.js.rhino.Parser.assignExpr(Parser.java:890)
	at com.google.gwt.dev.js.rhino.Parser.expr(Parser.java:880)
	at com.google.gwt.dev.js.rhino.Parser.statementHelper(Parser.java:775)
	at com.google.gwt.dev.js.rhino.Parser.statement(Parser.java:360)
	at com.google.gwt.dev.js.rhino.Parser.parse(Parser.java:123)
	at com.google.gwt.dev.js.JsParser.parseImpl(JsParser.java:133)
	at com.google.gwt.dev.js.JsParser.parse(JsParser.java:88)
	at com.google.gwt.user.client.rpc.impl.ClientSerializationStreamReader.prepareToRead(ClientSerializationStreamReader.java:313)

That does not fail before your changes. I have not yet run the full set of tests, was waiting for the branch to be more completed to try to proceed.

What do you think of trying to limit the scope of the patch as I proposed above, so that it can be disabled on a per-project basis, until we manage these remaining issues and complete the related work?

@codemasterover9000
Copy link
Contributor Author

Yes we use version 8. I agree that making the threshold a system property as you suggested would be a better option for a less impactful change.

I made a new pull request here #9961

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RPC format relies on eval for large payloads and long strings
5 participants