Regex Tokenizer

Regex Tokenizer is a simple library for tokenizing regex expressions.

Its main use is to:

• Break regex into proper parts (creating expression tree - so called token)
• Validate string against token- as opposed to normal regexp validation, it makes possible for string to be matched up to regexp parts

Whole module is in general rewritten jquery.inputmask plugin from RobinHerbots InputMask for standalone use.

Library depends only on Jquery and is written as AMD module.

Methods

RegexTokenizer()

Constructor. Builds special tokenizer regexp. Regexp based on slevithan/regex-colorizer

analyzeRegex(regex)

Analyzes given regex and breaks it into token. Returns special token object.

validateRegexToken(token, fromGroup, tokenValidator)

Validates given string using given token. Arguments are as follows:

• token - token object generated by analyzeRegex method
• fromGroup - information about whether tokenValidator comes from group or normal regexp
• tokenValidator - Special token validator object or string. It is usually string. special object could be used when user wants to match string against particular part of the token.

Tests

Just run npm install && karma start from within app directory

Examples (taken from tests)

Simple Regexp

var regex = /^te?s[a-zA-Z]$/; // Build simple regexp
var tokens = tokenizer.analyzeRegex(regex); // Build token object

tokenizer.validateRegexToken(tokens, false, 'reeesm'); // Returns false
tokenizer.validateRegexToken(tokens, false, 'te'); // Returns true
tokenizer.validateRegexToken(tokens, false, 'ts'); // Returns true
tokenizer.validateRegexToken(tokens, false, 'test'); // Returns true

Regexp with groups

var regex = /^te(s[a-zA-Z]*){1,2}\d{1,2}$/;
var tokens = tokenizer.analyzeRegex(regex);

tokenizer.validateRegexToken(tokens, false, 'tertst'); // Returns false
tokenizer.validateRegexToken(tokens, false, 'te12'); // Returns false
tokenizer.validateRegexToken(tokens, false, 'testst1'); // Returns true
tokenizer.validateRegexToken(tokens, false, 'tesr43');  // Returns true

Regexp with nested groups

var regex = /^te(st(a.){1,2}(b)?){2,}$/;
var tokens = tokenizer.analyzeRegex(regex);

tokenizer.validateRegexToken(tokens, false, 'rest'); // Returns false
tokenizer.validateRegexToken(tokens, false, 'testabf'); // Returns false
tokenizer.validateRegexToken(tokens, false, 'testacagb');  // Returns true
tokenizer.validateRegexToken(tokens, false, 'testacavstanb');  // Returns true