fix 14540

[dylan-conway]

What does this PR do?

fixes #14540

Fixes crashes caused by invalid pointers copied from old lexer state.

How did you verify your code works?

Added a test that relies on std.ArrayList capacity growth

[robobun]

❌ @dylan-conway, your commit 37baaef has 12 failures in #7150:

test/js/bun/http/bun-serve-static.test.ts - timeout on 🐧 3.20 x64

test/js/node/test/parallel/worker-nested-uncaught.test.js - segmentation fault on 🐧 3.20 x64

test/js/node/test/parallel/worker-nested-uncaught.test.js - segmentation fault on 🐧 3.20 x64-baseline

test/integration/next-pages/test/dev-server.test.ts - 1 failing on 🐧 3.20 x64

test/integration/next-pages/test/dev-server.test.ts - 1 failing on 🐧 3.20 x64-baseline

test/cli/hot/hot.test.ts - timeout on 🐧 22.04 x64

test/cli/hot/hot.test.ts - timeout on 🐧 3.20 x64

test/cli/hot/hot.test.ts - timeout on 🐧 3.20 x64-baseline

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 3.20 aarch64

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 11 x64-baseline

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 22.04 x64-baseline

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 12 x64

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 22.04 x64

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 3.20 x64

test/js/node/test/parallel/fs-watch-recursive-linux-parallel-remove.test.js - 1 failing on 🐧 3.20 x64-baseline

test/v8/v8.test.ts - 22 failing on 🐧 3.20 aarch64

test/v8/v8.test.ts - 22 failing on 🐧 3.20 x64-baseline

test/v8/v8.test.ts - 22 failing on 🐧 3.20 x64

test/js/node/test/parallel/timers-ordering.test.js - 1 failing on 🐧 22.04 x64

test/js/node/child_process/child_process.test.ts - 1 failing on 🐧 3.20 aarch64

test/js/node/child_process/child_process.test.ts - 1 failing on 🐧 3.20 x64

test/js/node/child_process/child_process.test.ts - 1 failing on 🐧 3.20 x64-baseline

test/js/bun/http/serve.test.ts - SIGTRAP on 🐧 3.20 aarch64

test/js/bun/http/serve.test.ts - SIGILL on 🐧 3.20 x64

test/js/bun/http/serve.test.ts - SIGILL on 🐧 3.20 x64-baseline

test/js/web/fetch/fetch-leak.test.ts - 1 failing on 🪟 2019 x64

test/integration/next-pages/test/next-build.test.ts - 1 failing on 🍎 13 x64

test/integration/next-pages/test/next-build.test.ts - 1 failing on 🐧 22.04 x64-baseline

test/js/bun/shell/leak.test.ts - 1 failing on 🪟 2019 x64

[Jarred-Sumner]

The length is not the main problem - the main problem is on array resize, the previous pointer is now invalid memory. This wouldn't fix it, as it reuses the memory.

I think we would instead need to lazily clone it

[dylan-conway]

it copies the possibly new pointer from the new lexer, and sets the length from the original lexer. This way we can ensure the pointers are valid, new values aren't used, and memory isn't wasted.

doing this for temp_buffer_u16 is unnecessary though, it's length is set to 0 after each use so we can ignore it. Will update

[Jarred-Sumner]

the pointer can become invalid memory though when its resized later. even if the ptr and length match up.