Large numbers in object tree

[cugu]

Larger values are displayed incorrectly in the object tree pane. The converter pane shows them correctly.

The screenshot shows a selection of the bytes 0xbbbbbbbbaaaaaaaa as 0xbbbbbbbbaaaaa800 in the object tree.

[GreyCat]

I believe that's limitation of JavaScript, as it effectively does not support 64-bit integers :(

[cugu]

Yes, that should the problem in this case, but it's solved for the converter pane.

[GreyCat]

Generally, web IDE just runs JavaScript generated by KS compiler, so there's not much that web IDE can do about it — we need to solve it on code generation level.

One of the options is to rely on 3rd party big integer library: for example, web IDE uses Peter Olson's BigInteger.js. However, there is no universal standard as well...

[koczkatamas]

I would split this problem into two parts:

1. parsing fields only
2. using these big integer fields in expressions

Possible solutions for parsing only

We could make the JS runtime extensible by either accepting an optional interface which can handle parsing BigInteger from an Uint8Array (Kaitai would do the other endianness / bit magic), or either by wrapping the current KaitaiStream and overwrite the read[SU]8[lb]e, readBits methods.

I like the first method a bit better: although it is less optimal performance-wise (especially if we have to copy bits into a custom array in readBits before calling the BigInteger parser), it would make much easier for our users to put Kaitai into their already existing solution and I think these are rare operations generally.

In contrast for the second method we don't have to modify the runtime, but have to reimplement some of its features (like readBits) for our specific BigInteger library which probably won't work with an other implementations.

Solution for expressions

As far as I know there is no operator overloading, it is not even planned for newer JS versions. So if we want to use operations on big integers, we have to generate code like a.And(0x80).Equals(0x80). But this would be probably much slower for smaller integers, so we should not to do for numbers smaller than eg. 6 bytes.

I don't know @GreyCat how hard this modification would be compiler-wise?

[koczkatamas]

@cugu I saw you'd like to use Kaitai in Go. You might want to join the conversion in this issue: kaitai-io/kaitai_struct#146

[generalmimon]

Duplicate of kaitai-io/kaitai_struct#183