Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
JavaScript: make bytes literals use
Uint8Array
(not number[]
)
Until now, byte array literals translated for JavaScript were of type `number[]`, as TypeScript would call this type (in pure JS terminology, it's an `Array` object with numbers). For example, a value instance defined as `instances/v/value: '[65, 126]'` would be translated as `this._m_v = [65, 126];` in the generated JS code. However, byte arrays created in any other way would use the [`Uint8Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array) type, which is the standard type for byte arrays in JavaScript. Namely any byte array parsed from stream would be a `Uint8Array` object and also a *non-literal* byte array like `[123, 246 + 0].as<bytes>` would be generated as `new Uint8Array([123, 246 + 0])`. I believe that using a different type for byte array literals than for byte arrays created in any other way was never intentional. It seems we simply forgot to override `doByteArrayLiteral` in `JavaScriptTranslator` (but `doByteArrayNonLiteral` is overridden correctly, which is weird), so we ended up using the default `BaseTranslator.doByteArrayLiteral` implementation, which produces the `[65, 126]` syntax. JavaScript is dynamically typed, so you can treat `number[]` the same as `Uint8Array` in most cases (e.g. indexing will work the same), but this discrepancy now causes problems when porting our JavaScript runtime library to TypeScript (see kaitai-io/kaitai_struct_javascript_runtime#25), because you would have to annotate a lot of parameter types as `number[] | Uint8Array` (which doesn't really make sense). Moreover, I discovered that generating byte array literals as `number[]` creates a bug in this case: ```ksy meta: id: bytes_to_s instances: literal: value: '[65, 126].to_s("UTF-8")' ``` Before this commit, `[65, 126].to_s("UTF-8")` would be translated as `KaitaiStream.bytesToStr([65, 126], "UTF-8")` for JavaScript. But our `bytesToStr` implementation won't actually accept `number[]` for non-ASCII encodings. See [`KaitaiStream.js:650-657`](https://github.com/kaitai-io/kaitai_struct_javascript_runtime/blob/2acb0d8b09fde9c95fc77ee3019f6d6b1c73f040/KaitaiStream.js#L650-L657): ```js KaitaiStream.bytesToStr = function(arr, encoding) { if (encoding == null || encoding.toLowerCase() === "ascii") { return KaitaiStream.createStringFromArray(arr); } else { if (typeof TextDecoder === 'function') { // we're in the browser that supports TextDecoder return (new TextDecoder(encoding)).decode(arr); } else { ``` If `TextDecoder` is available (which is the case in a browser or since Node.js 11), the error when trying to access the `literal` instance above will look like this in the browser (Google Chrome 129): ``` TypeError: Failed to execute 'decode' on 'TextDecoder': parameter 1 is not of type 'ArrayBuffer'. at KaitaiStream.bytesToStr (https://ide.kaitai.io/devel/lib/_npm/kaitai-struct/KaitaiStream.js:656:42) at BytesToS.get (eval at initCode (https://ide.kaitai.io/devel/js/v1/kaitaiWorker.js:156:9), <anonymous>:27:38) at fetchInstance (https://ide.kaitai.io/devel/js/v1/kaitaiWorker.js:123:20) ... ``` ... and like this in Node.js 12: ``` TypeError [ERR_INVALID_ARG_TYPE]: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of Array at TextDecoder.decode (internal/encoding.js:412:15) at Function.KaitaiStream.bytesToStr (/runtime/javascript/KaitaiStream.js:656:42) at BytesToS.get (compiled/javascript/BytesToS.js:26:38) at /tests/spec/javascript/test_bytes_to_s.js:8:24 ... ``` So using `Uint8Array` for all byte arrays will not only be more consistent, but will also fix this bug.
- Loading branch information