[Fix]: integer overflow in JumpTable.SubStr

[nan01ab]

Description

Fix integer overflow in JumpTable.SubStr

Fixes #3495

Type of change

• Optimization (the change is only an optimization)
• Style (the change is only a code style for better maintenance or standard purpose)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[cschuchardt88]

@shargon why isn't there a vm limit in this

[roman-khimov]

I'd also add some tests for INT64, like:

                byte(opcode.PUSHINT64), 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F,
                byte(opcode.PUSH2),

It'll fail (in NeoGo it's at instruction 22 (SUBSTR): not an int32), but just to make sure.

[shargon]

Rebase needed

[shargon]

I think we don't need a HF, previously could be a DoS but not difference in the execution. Isn't it? @roman-khimov

[roman-khimov]

That's the question of "can we arrange a set of parameters that would fail with the new code, but succeed with the old one". This requires some probing. I'm not exactly sure of Rent details and how this can be related to machine memory available (I can malloc() more memory than I have physically), also how Slice() reacts to negative count.

I'd include it into Echidna for safety reasons, but if we can prove it can't be exploited to change execution result then OK, it can go without a HF.