Skip to content

Commit

Permalink
lexer: allow _ $ ¥ for money
Browse files Browse the repository at this point in the history
  • Loading branch information
Artoria2e5 committed Apr 23, 2021
1 parent f9a22b7 commit 309bc8c
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 10 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ $ make repl
- arithmetic expressions
- built-in functions
- first-class and higher-order functions • closures
- Unicode identifiers (UAX #31, XID). No Emojis yet.
- Unicode identifiers (UAX #31, XID) plus monetary extensions (`[¥$_]`). No Emojis yet.
- Full double-quoted string syntax from Rust-lang.
- Ridiculous naming for the Aba-aba. Comparison with Monkey:

Expand Down
14 changes: 6 additions & 8 deletions src/lexer/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,27 +48,25 @@ pub fn is_whitespace(c: char) -> bool {
}

/// True if `c` is valid as a first character of an identifier.
/// See [Rust language reference](https://doc.rust-lang.org/reference/identifiers.html) for
/// a formal definition of valid identifier name.
/// Compared to Rust, we additionally allow $ and ¥.
fn is_id_start(c: char) -> bool {
// This is XID_Start OR '_' (which formally is not a XID_Start).
// We also add fast-path for ascii idents
('a'..='z').contains(&c)
|| ('A'..='Z').contains(&c)
|| c == '_'
|| c == '$'
|| c == '¥'
|| (c > '\x7f' && unicode_xid::UnicodeXID::is_xid_start(c))
}

/// True if `c` is valid as a non-first character of an identifier.
/// See [Rust language reference](https://doc.rust-lang.org/reference/identifiers.html) for
/// a formal definition of valid identifier name.
/// Compared to Rust, we additionally allow $ and ¥.
fn is_id_continue(c: char) -> bool {
// This is exactly XID_Continue.
// We also add fast-path for ascii idents
('a'..='z').contains(&c)
|| ('A'..='Z').contains(&c)
|| ('0'..='9').contains(&c)
|| c == '_'
|| c == '$'
|| c == '¥'
|| (c > '\x7f' && unicode_xid::UnicodeXID::is_xid_continue(c))
}

Expand Down
2 changes: 1 addition & 1 deletion web/src/js/editor/monkey-mode.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ CodeMirror.defineSimpleMode('monkey', {
{ regex: /[-+\/*=<>!]||||/, token: 'operator' },
{ regex: /[\{\[\(]/, indent: true },
{ regex: /[\}\]\)]/, dedent: true },
{ regex: /\p{XID_Start}\p{XID_Continue}*|[a-z$][\w$]*/u, token: 'variable' },
{ regex: /[\p{XID_Start}$¥_][\p{XID_Continue}$¥]*/u, token: 'variable' },
],
comment: [],
meta: {},
Expand Down

0 comments on commit 309bc8c

Please sign in to comment.