jsparser: 3in should be an error

[masnagam]

The specification said in 12.9.3 Numeric Literals:

The SourceCharacter immediately following a NumericLiteral must not be an IdentifierStart or DecimalDigit.

This cannot be handled with the following modified production rules using a lookahead term:

NumericLiteral ::
  MaybeNumericLiteral [lookahead ∉ IdentifierStart]

MaybeNumericLiteral ::
  DecimalLiteral
  DecimalBigIntegerLiteral
  NonDecimalIntegerLiteral[+Sep]
  NonDecimalIntegerLiteral[+Sep] BigIntLiteralSuffix
  LegacyOctalIntegerLiteral

Initially, we tried solving this issue in this way, but an infinite loop occurs when computing the number of lookahead characters needed. Because:

CodePoint ::
  HexDigits[~Sep] [> but only if MV of |HexDigits| ≤ 0x10FFFF]

IdentifierStart is defined with CodePoint and CodePoint can take any number of characters.

Probably, this is the reason why the specification does not use the production rules like above.

[masnagam]

Another issue is regarding Unicode ID_Start property.

Currently, dfagen cannot process the Unicode ID_Start property. We replace it used in the lexical grammar with ASCII ID_Start by transpile.js. As a result, the geneerated automaton cannot work with Non-ASCII ID_Start (and ID_Continue). So, we need additional error handling in the lexer implementation. But this error handling does not work properly for lookahead transitions.