Copy HttpContent into array to avoid buffer being released too early

[sbrunk]

We've run into issues when decoding large JSON bodies using the netty cats backend. For some requests, decoding failed, due to parts of the input JSON missing.

The issue doesn't occur every time and is not deterministic, but we were able to track it down to this PR: #3489.

We believe that the issue is caused by the buffer sometimes being released to early, as the fs2 Chunk doesn't copy it and httpContent.release() sometimes trigger a deallocation by netty before the buffer is read into an array later. Our proposed fix is to explicitly copy the buffer into an array and only then call release().

We've added a new performance test for large JSON bodies that reliably shows the error. You should be able to compare the difference yourself by running:

perfTests/runMain sttp.tapir.perf.apis.ServerRunner netty.cats.Tapir
perfTests/Gatling/testOnly sttp.tapir.perf.PostLongJsonSimulation

Before:

After:

@pektinasen

[sbrunk]

Here's also a standalone Scala CLI script as Gist to demonstrate the issue.

[adamw]

Thanks for the investigation and the fix! I turned this into a regular test, since we're talking about correctness, not only performance here. I suspected that the ZIO integration might have a similar problem, but it turns out their Chunk.fromByteBuffer already copies data into an array. We'll also see how the other backends fare with the new test ;)

[sbrunk]

Thanks for getting it to the finish line!